MemOS
Self-evolving memory OS for LLM & AI Agents: ultra-persistent memory, hybrid-retrieval, and cross-task skill reuse, with 35.24% token savings and DeepSeek Harness support.
- 12k
- TypeScript
- Apache-2.0
- 3 h ago
RouteAlign
Repo MemTensor/RouteAlign, read from GitHub 1 min ago. MemTensor has not joined Take My Repo.
MemTensor/RouteAlign is a public repo on GitHub by MemTensor. RouteAlign. It declares the Apache-2.0 license on GitHub. It is written mainly in Python, has 125 stars and last changed on 2026-10-08. Nobody has reviewed it on Take My Repo yet, which says nothing about its quality. Read from GitHub on 2026-11-20.
In the maintainer's words. Shown from GitHub; Take My Repo does not keep a copy.
Selecting a large language model service requires estimating how measured model performance transfers to the demands of a new task. Aggregate scores average over heterogeneous task demands, while semantic similarity alone can overlook differences in the capabilities needed for successful completion. RouteAlign makes task requirements the common basis for constructing model profiles and selecting services. A compact profiler predicts capability weights and a difficulty level for each request. Historical requirements determine how observed outcomes contribute to difficulty-conditioned model profiles and to capability-conditioned local estimates; incoming requirements determine how those measurements inform the current decision. A shared quality scorer integrates evidence across these scopes, service metadata determines eligibility, and a generation-cost preference controls selection among eligible services.
A requirement-aware formulation of LLM service selection. Task competence, service eligibility, and generation-cost preference are separated explicitly. Ten capability coordinates and four difficulty levels describe what successful completion requires; service metadata such as context capacity, supported modalities, and tool access determines admissibility; the coefficient Ξ» expresses the application preference. A capability weight influences the quality comparison among eligible services, while an explicit execution requirement decides eligibility itself, so high estimated quality cannot compensate for a known violation of an execution condition.
Requirement-profile alignment as a measurement and selection mechanism. The same requirement coordinates organize evidence on both sides of the comparison. Historical capability weights distribute observed outcomes across capability-by-difficulty profile cells, and reweight neighboring outcomes within each retrieved neighborhood; incoming weights read the global profile at the predicted difficulty and recompose per-capability local estimates. A shared ridge scorer integrates both scopes of evidence with conditional priors and relative candidate comparisons through ten candidate-agnostic features, and selection maximizes estimated quality minus Ξ» times estimated generation cost. Model identity indexes the evidence without defining a learned output class.
An empirical assessment from selection value to evidence maintenance. Four research questions cover selection quality and generation expenditure, the contribution of each evidence group, execution overhead with and without requirement prediction, and scorer reuse under controlled catalog changes. Across nine catalog configurations, coefficients fitted on the initial seven candidates remain effective through five additions, one version replacement, one removal, and one repricing, provided candidate evidence is refreshed.
Evaluation uses the recorded requests, responses, task scores, and generation costs of LLMRouterBench: ten datasets (AIME, ArenaHard, GPQA, HLE, LiveCodeBench, LiveMathBench, MMLU-Pro, SimpleQA, SWE-bench, ΟΒ²-bench) with 13 candidate LLM services, partitioned by dataset and prompt at seed 42 with a 70% training fraction, giving 8,710 training and 3,738 test requests.
| Policy | Sample-Avg (%) | Dataset-Avg (%) | Cost ($) |
|---|---|---|---|
| RouteAlign | 63.07 | 68.64 | 110.42 |
| RouteLLM (strongest competing policy) | 61.69 | 66.42 | 109.27 |
| RouterDC | 58.56 | 65.24 | 122.62 |
| AvengersPro | 58.91 | 63.63 | 61.35 |
| GPT-5 (highest Dataset-Avg among candidates) | 58.37 | 65.11 | 124.82 |
| Gemini 2.5 Pro (highest Sample-Avg among candidates) | 59.18 | 59.55 | 233.97 |
Python 3.10 or later.
pip install -r requirements.txt
PyTorch and Transformers are only needed to regenerate query embeddings from a local profiler checkpoint; the reproduction bundle ships with the caches already computed.
bash data/router_21K_Qwen3-0.6B_learned_fusion/run.sh
The bundle is self-contained and fully offline: requirement profiles, both model-profile channels, and query embeddings are read from cache, and ridge fitting plus scoring take seconds. Expected output, also shipped under output/ for direct comparison:
Dataset-Avg accuracy: 68.64%
Sample-Avg accuracy: 63.07%
Total cost: $110.42
The stages below regenerate every artifact in the bundle. Steps that call LLM endpoints resume from checkpoints and read credentials from environment variables referenced in the configs.
# 1. Convert LLMRouterBench records and split train/test (requires the # LLMRouterBench package; the prepared splits already ship in the bundle) python -m adapter.data_prep --config config/adapter_config.yaml # 2. Annotate question metadata and capability weights with a strong model python -m adapter.build_question_bank --config config/adapter_config.yaml # 3. Grade candidate responses and aggregate model profiles python -m adapter.run_offline --config config/adapter_config.yaml # 4. Pure profile-based selection evaluation on the test split python -m adapter.evaluate --config config/adapter_config.yaml
The shared-scorer path used for the reported numbers continues with:
# 5. Requirement profiles for the training split python -m adapter.profile_train_tasks --config config/adapter_config.yaml \ --train data/seed42_split0.7/train.jsonl \ --output data/seed42_split0.7/train_task_profiles.jsonl # 6. Outcome profiles aggregated from training records python -m adapter.build_profile_from_records \ --train data/seed42_split0.7/train.jsonl \ --task-profiles data/seed42_split0.7/train_task_profiles.jsonl \ --model-pool config/model_pool.yaml \ --output data/seed42_split0.7/model_profiles_data_driven.json # 7. Scorer fitting and evaluation (see the bundle run.sh for all arguments) python -m adapter.evaluate_learned_fusion --config config/demo_router_config.yaml \ --knn-k 150 --knn-beta 5.0 --knn-cap --ridge-alpha 1.0 --cost-lambda 0.4
The registry in config/trained_models.yaml covers six local checkpoints (SFT and GRPO variants at 0.6B, 1.7B, and 4B). Edit model_path entries to point at your checkpoints, then:
bash scripts/serve_trained_models.sh start # one sglang service per GPU python -m adapter.run_trained_eval # profile-based selection per checkpoint python -m adapter.run_trained_fusion # full shared-scorer pipeline per checkpoint
Both drivers support --models for subsets, --max-samples-per-dataset for smoke tests, and --summary-only for aggregation.
from src import DemoRouterConfig, RoutingEngine engine = RoutingEngine(DemoRouterConfig.load("config/demo_router_config.yaml")) decision = engine.route("Compare the revenue trends across these three reports") result = engine.answer("Compare the revenue trends across these three reports")
The engine requires a reachable requirement-profiler endpoint (a local checkpoint served by sglang, or any OpenAI-compatible API configured under task_profile_model) and prepared model profiles. route returns only the decision; answer additionally calls the selected candidate, which requires live endpoints in config/model_pool.yaml.
βββ src/ # RouteAlign framework
β βββ capabilities.py # Capability dimensions, difficulty levels, tiers
β βββ schemas.py # Request, grading, profile, and decision structures
β βββ prompts.py # Grading and requirement-profiler prompts
β βββ llm_client.py # OpenAI-compatible client with exponential backoff
β βββ question_bank.py # Plug-in question-bank loading
β βββ model_pool.py # Candidate service attributes
β βββ grading.py # Capability grading of candidate responses
β βββ profiling.py # Outcome-profile aggregation, confidence, tiers
β βββ task_profiler.py # Online requirement prediction
β βββ request_features.py # Request metadata detection
β βββ router.py # Eligibility filter and compatibility scoring
β βββ offline.py # Offline measurement pipeline with resume
β βββ engine.py # Online selection engine
β βββ config.py # Configuration loading
βββ adapter/ # LLMRouterBench adapter and evaluation pipeline
β βββ data_prep.py # Benchmark conversion, split, stratified sampling
β βββ build_question_bank.py # Metadata and capability-weight annotation
β βββ gen_model_pool.py # Candidate-registry template with price estimation
β βββ run_offline.py # Offline grading and profiling driver
β βββ evaluate.py # Test-set selection evaluation
β βββ profile_train_tasks.py # Requirement profiles for training requests
β βββ build_profile_from_records.py # Outcome profiles from training records
β βββ compute_router_embeddings.py # Query embedding caches from the profiler
β βββ knn_priors.py # Neighborhood statistics and local estimates
β βββ evaluate_with_cached_tasks.py # Selection from cached requirement profiles
β βββ evaluate_strategy_map.py # Conditional priors and difficulty strata
β βββ evaluate_learned_fusion.py # Shared scorer fitting and evaluation
β βββ run_trained_eval.py # Batch evaluation of fine-tuned profilers
β βββ run_trained_fusion.py # Full shared-scorer pipeline per checkpoint
βββ config/ # Framework, adapter, catalog, and per-profiler configs
βββ scripts/serve_trained_models.sh # sglang serving for local profiler checkpoints
βββ data/router_21K_Qwen3-0.6B_learned_fusion/ # Reproduction bundle with expected outputs
api_key or base_url written in all capitals is resolved from the environment at runtime, so set GRADER_API_KEY, METADATA_API_KEY, and MODEL_ENDPOINT_API_KEY before running stages that call external endpoints.config/model_pool.yaml carries service attributes for the thirteen evaluated candidates. Prices are least-squares estimates from benchmark usage records and latency fields are placeholders; verify both against your provider before live selection.src/capabilities.py and shared by the grader, the profiler, and the question-bank annotator, which keeps the three annotation procedures consistent by construction.MemTensor/RouteAlign is a public repo on GitHub by MemTensor. RouteAlign.
It declares the Apache-2.0 license on GitHub.
Its maker has not tagged a system, so it is unknown here. Its README says what it needs.
Open "Take it" on its page: Take My Repo writes an install plan for your machine and hands it to your AI agent (Claude Code, Codex or Cursor), which asks you before each step. Take My Repo has not tested it on any machine.
Its last change on GitHub was on 2026-10-08. Take My Repo does not judge from that whether it is maintained.
Nobody has reviewed it on Take My Repo yet, which says nothing about its quality.
Self-evolving memory OS for LLM & AI Agents: ultra-persistent memory, hybrid-retrieval, and cross-task skill reuse, with 35.24% token savings and DeepSeek Harness support.

π A personal AI agent & local memory hub for all AI agents, gives every AI one shared, fully controlled memory and persistent context β all AI remember the same you. Now supports Claude Code, Codex, OpenClaw and Hermes Agent etc.

Official MemOS Cloud plugin for OpenClaw. Enables long-term memory for agents by recalling context before execution and saving conversations after each run.

SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution

Paper: βMEMRL: SELF-EVOLVING AGENTS VIA RUNTIME REINFORCEMENT LEARNING ON EPISODIC MEMORYβ Open-Source Code
This page is also available as structured data, with the same sources and dates.
/api/v1/resources/gh/MemTensor/RouteAlignTake My Repo has not tested this on your machine. Requirements come from the project's own documentation.