llm4free
LLM4Free — All-in-one Python toolkit for web search, AI interaction (40+ free providers), digital utilities, and more. Formerly WebScout.
- 382
- Python
- Apache-2.0
- 3 wk ago
Codebase indexing and semantic search engine
Repo OEvortex/vortexa, read from GitHub 6 h ago. OEvortex has not joined Take My Repo.
OEvortex/vortexa is a public repo on GitHub by OEvortex. Codebase indexing and semantic search engine. It declares the Apache-2.0 license on GitHub. It is written mainly in Python, has 3 stars and last changed on 2026-11-03. Nobody has reviewed it on Take My Repo yet, which says nothing about its quality. Read from GitHub on 2026-11-22.
In the maintainer's words. Shown from GitHub; Take My Repo does not keep a copy.
Codebase indexing and semantic search engine
Dense + sparse hybrid retrieval · AST-aware chunking · LMDB persistence · MCP server
vortexa is a standalone codebase indexing and semantic search engine designed for AI agents and developers. It builds a persistent, hybrid search index over source code using:
The result: natural language code search that understands intent, not just keywords.
results = indexer.search("authentication middleware that validates JWT tokens", top_k=5) # → Finds the right files even if they use "auth", "verify", "token" instead of "authentication"
vortexa can run as a standalone Python library, be embedded into any agent, or serve as an MCP server for LLM tools.
| Semantic search | Find code by describing what it does in natural language — no exact-string matching needed. |
| Hybrid retrieval | Combines dense embeddings (semantic meaning) with BM25 (keyword precision) using adaptive alpha weighting. |
| AST-aware chunking | Splits source code at function/class/block boundaries using tree-sitter when available, falls back to line-based splitting. |
| Incremental indexing | Content-hash memoization means only changed files are re-indexed. Full re-index avoids redundant embedding computations. |
| Persistent storage | LMDB-backed vector store survives restarts. Embedding cache avoids recomputing identical content. |
| Live watch mode | Background thread polls for file changes and auto-re-indexes with configurable debounce. The watcher uses the same extension set as the index, so it never loops on files it does not index. |
| Realtime progress | A live progress bar reports file scanning, chunking, model loading, embedding, keyword indexing, and saving — on stderr, so JSON and MCP output stay clean. |
| MCP server | Expose search, resolve, explain, status, and reindex tools for MCP-compatible agents (Claude Code, Cursor, etc.) |
| Zero mandatory heavy deps | Core requires only numpy, lmdb, and pathspec. Model2Vec and tree-sitter are optional extras. |
# Core (BM25 + line-based chunking) pip install vortexa # Full (Model2Vec embeddings + tree-sitter AST chunking) pip install "vortexa[full]" # With MCP server support pip install "vortexa[mcp]"
from vortexa.core.indexer import CodebaseIndexer indexer = CodebaseIndexer(root=".") stats = indexer.index() print(f"Indexed {stats.indexed_files} files, {stats.total_chunks} chunks") print(f"Languages detected: {stats.languages}")
results = indexer.search("CSV parser implementation", top_k=5) for r in results: print(f"{r.chunk.file_path}:{r.chunk.start_line} score={r.score:.3f}") print(f" {r.chunk.content[:150].strip()}") print()
Output:
src/parsers/csv_parser.py:42 score=0.892
def parse_csv(filepath: str, delimiter: str = ",") -> list[dict]:
"""Parse a CSV file into a list of dictionaries."""
with open(filepath, "r") as f:
tests/test_csv_parser.py:15 score=0.756
def test_parse_csv_with_header():
result = parse_csv("test.csv")
assert len(result) == 3
from vortexa.core.indexer import CodebaseIndexer from vortexa.core.types import ChunkConfig # Default chunking (aim for 50-line chunks, 5-line overlap) indexer = CodebaseIndexer(root="/path/to/project") stats = indexer.index() # → IndexStats(indexed_files=127, total_chunks=843, languages={"python": 45, "typescript": 32, ...}) # Custom chunk configuration indexer = CodebaseIndexer( root=".", chunk_config=ChunkConfig(chunk_size=100, chunk_overlap=10), ) stats = indexer.index(force=False, include_text_files=True) # Force full re-index stats = indexer.index(force=True)
Track indexing progress in your own UI with the progress callback:
from vortexa.core.progress import IndexProgress def on_progress(event: IndexProgress) -> None: print(f"{event.phase}: {event.current}/{event.total} {event.message}") stats = indexer.index(progress=on_progress)
# Hybrid search (auto-weighted semantic + BM25) results = indexer.search("error handling", top_k=10) # Pure semantic search results = indexer.search("database connection pool", top_k=5, alpha=1.0) # Pure BM25 keyword search results = indexer.search("parse csv", top_k=5, alpha=0.0) # Symbol lookup (find definitions by name) results = indexer.find_symbol("ConnectionPool", top_k=5) # Related chunks (find chunks similar to a given chunk index) results = indexer.find_related(chunk_idx=3, top_k=5)
Each result is a SearchResult with:
| Field | Type | Description |
|---|---|---|
chunk.file_path |
str |
Relative file path |
chunk.start_line |
int |
Start line number |
chunk.end_line |
int |
End line number |
chunk.content |
str |
Code snippet (up to 500 chars) |
chunk.language |
str |
Detected programming language |
chunk.lineage |
Lineage |
Source path + byte offsets |
chunk.chunk_hash |
str |
Content hash for memoization |
score |
float |
Relevance score (0–1) |
source |
str |
"semantic", "bm25", or "hybrid" |
from vortexa.interfaces.watcher import IndexWatcher watcher = IndexWatcher(indexer, poll_interval=3.0) watcher.start() # Background thread, polls every 3s, debounces 2s # ... files change on disk, auto-re-index happens ... watcher.stop()
# Index statistics stats = indexer.stats() # → {indexed_files: 127, total_chunks: 843, languages: {...}, memo_hits: 42, memo_misses: 15} # Reset indexer.clear() # Delete the persistent index
Build or update the index without running a search. A realtime progress bar is drawn on stderr; stdout stays machine-readable JSON:
# Build or incrementally update the index (default: current directory) vortexa index # Index a specific repository and include Markdown/JSON/YAML vortexa index /path/to/project --include-text # Rebuild from scratch vortexa index --force # Check whether the index is ready (does not build anything) vortexa status # Human-readable status instead of JSON vortexa status --plain
Example progress output:
[vortexa] Indexing /path/to/project
[vortexa] Scanning files: 128/210 (60%) ETA 1s — src/auth/middleware.py
[vortexa] Embedding chunks: 512/940 (54%) ETA 4s
[vortexa] ✓ 210 files · 940 chunks · 64 reused · 876 embedded (6.42s)
vortexa index prints JSON statistics by default:
{
"root": "/path/to/project",
"model": "VTXAI/vtx-embed-7M-lf2",
"indexed_files": 210,
"total_chunks": 940,
"languages": {"python": 142, "typescript": 68},
"changed_files": 12,
"removed_files": 1,
"memo_hits": 64,
"embedded_chunks": 876,
"up_to_date": false,
"index_time_ms": 6418.2,
"errors": []
}Progress is suppressed with --quiet (both progress and summaries) or
--no-progress (bar only). Both flags are also available on search,
resolve, and explain.
The installed vortexa command can also search a codebase directly:
# Search the current working directory vortexa -q "authentication middleware that validates JWT tokens" # Search a specific codebase root vortexa -q "CSV parser implementation" /path/to/project # Pass Kilo-style environment details; `Working directory` is used as the root vortexa -q "error handling" "Working directory: /path/to/project Workspace root folder: /"
Useful flags:
| Flag | Description |
|---|---|
-q, --query |
Search query. Quote multi-word queries. |
--root |
Codebase root to index and search. Overrides environment_details. |
--top-k |
Maximum number of results to return. Default: 10. |
--alpha |
Semantic weight from 0.0 to 1.0; defaults to adaptive weighting. |
--min-score |
Drop results below this score. Zero-score noise is always removed. |
--include-text |
Include text files such as .md, .json, and .yaml in the index. |
--force |
Force a full re-index before searching. |
--no-index |
Search the existing index only. |
--quiet |
Suppress progress and index status messages on stderr. |
--no-progress |
Hide the realtime indexing progress bar. |
--plain |
Print human-readable results instead of JSON. |
--model |
Embedding model ID or alias (mini, nano; both use LF2 by default). Use mini-full or nano-full for LF4. |
-f, --fast |
Explicitly select LF2 for a full-model alias. The mini and nano aliases already use LF2. |
By default CLI output is JSON:
[
{
"file": "src/auth/middleware.py",
"lines": "12-48",
"score": 0.892,
"source": "hybrid",
"content": "def validate_jwt(token: str) -> User: ..."
}
]The vortexa command still starts the MCP server when no query is provided. You can also start the server explicitly:
vortexa serve
# or
vortexa-servevortexa supports configurable embedding models. The default is mini
(VTXAI/vtx-embed-7M-lf2), using the native LF2 2-bit checkpoint. Use
mini-full or nano-full to select the original LF4 checkpoints.
| Alias | Model ID | Description |
|---|---|---|
mini |
VTXAI/vtx-embed-7M-lf2 |
Default native 2-bit 7M model. 256-dimensional embeddings. |
nano |
VTXAI/vtx-embed-1M-lf2 |
Default native 2-bit 1M model. 64-dimensional embeddings. |
mini-full |
VTXAI/vtx-embed-7M |
Original LF4 checkpoint for maximum quality. |
nano-full |
VTXAI/vtx-embed-1M |
Original LF4 lightweight checkpoint. |
# Use the nano model (smaller, faster) vortexa -q "authentication" --model nano /path/to/project # Use mini explicitly (same as default) vortexa -q "authentication" --model mini /path/to/project # Use a custom HuggingFace model ID vortexa -q "authentication" --model VTXAI/vtx-embed-1M /path/to/project
from vortexa.core.indexer import CodebaseIndexer # Use nano model (LF2 by default) indexer = CodebaseIndexer(root="/path/to/project", model_id="nano") # Use mini model (LF2 by default) indexer = CodebaseIndexer(root="/path/to/project", model_id="mini") # Opt into the original LF4 checkpoint indexer = CodebaseIndexer(root="/path/to/project", model_id="mini-full")
The embedding.py module also provides SentenceTransformerEmbedder
for transformer-based models (requires the sentence-transformers package).
vortexa can be used as a standalone embedding inference engine for VTXAI models, similar to how sentence-transformers works but purpose-built for Vortex-Embed models.
from vortexa.core.inference import embed # Encode a single string (default: mini model) vec = embed("India is a diverse country") # Encode multiple strings with the nano model vecs = embed(["Indian cricket team is strong", "Chennai is a major city"], model="nano") # Use any HuggingFace model ID vecs = embed(["Indian agriculture output"], model="VTXAI/vtx-embed-7M") # Shape: (1, D) for single text, (N, D) for batch print(vec.shape)
# Encode text with the mini model (default) vortexa embed "India is a diverse country" # Encode with the nano model vortexa embed "Indian cricket team is strong" --model nano # Encode multiple strings vortexa embed "India has 28 states" "Chennai is in Tamil Nadu" --model nano # Truncate embeddings with Matryoshka vortexa embed "India has 28 states" --model nano --dim 64 # Read one text string per line from a file vortexa embed --file queries.txt --model mini # Use a custom model ID vortexa embed "Indian monsoon patterns" --model VTXAI/vtx-embed-7M
# Auto-indexes current directory, serves on stdio python -m vortexa.interfaces.mcp_server # Or via the installed entry point vortexa serve
On startup it indexes the current working directory and reports realtime progress on stderr:
[vortexa] Indexing /projects/my-app
[vortexa] Scanning files: 127/127 (100%) — src/main.py
[vortexa] Embedding chunks: 843/843 (100%) ETA 0ms
[vortexa] ✓ 127 files · 843 chunks · 0 reused · 843 embedded (2.41s)
[vortexa] Ready: 127 files, 843 chunks in 2410ms
[vortexa] Auto-reindex watcher started (polling every 3s)
The server exposes five tools:
| Tool | Description | Arguments |
|---|---|---|
search |
Semantic + BM25 hybrid code search | query (str), top_k (int, default 10), min_score (float, default 0.0) |
resolve |
Feature-level search with graph context, tests, imports, callers, and callees | query (str), top_k (int, default 5) |
explain |
Explain a file, file:line location, or symbol |
location (str) |
status |
Check whether the index is ready, still indexing, empty, or failed | none |
reindex |
Incrementally update or force-rebuild the index with MCP progress notifications | force (bool), include_text (bool or null) |
Use search for discovery, resolve when you need feature context, and
explain when the file or symbol is already known. Call status before relying
on results while startup indexing may still be running, and use reindex only
after large external changes. The server indexes the current directory and
auto-reindexes it while running.
You can also start the server for a specific root without changing directory:
vortexa serve --root /path/to/project --include-text vortexa serve --root /path/to/project --no-watch
Add to your MCP configuration file (~/.cursor/mcp.json or Claude Code's mcp_servers config):
{
"mcpServers": {
"vortexa": {
"command": "python",
"args": ["-m", "vortexa.interfaces.mcp_server"],
"cwd": "/path/to/your/project"
}
}
}The agent will now have access to semantic code search — it can find functions, classes, and patterns by describing them in natural language. This is significantly more effective than grep or rg for exploratory queries.
Vortexa ships with an Agent Skills-compatible skill at
skills/vortexa/SKILL.md. It teaches compatible
agents when to prefer Vortexa, how to choose between search, resolve, and
explain, and how to interpret returned file paths, line ranges, scores, and
graph context.
The skill follows the standard SKILL.md structure: YAML frontmatter with a
lowercase hyphenated name and trigger-oriented description, followed by
workflow instructions. Copy or link the skills/vortexa directory into the
skills directory used by your agent environment.
--- name: vortexa description: Use Vortexa to index and semantically search local codebases... ---
Use the installed MCP server when available; otherwise the same workflow is
available through the vortexa search, vortexa resolve, and
vortexa explain commands.
vortexa/
├── core/
│ ├── indexer.py # CodebaseIndexer — main orchestrator
│ ├── chunking.py # AST-aware (tree-sitter) + line-based chunking
│ ├── embedding.py # Embedding model wrappers (Model2Vec, SentenceTransformers, LF4)
│ ├── language.py # Language detection & file extension mapping
│ ├── progress.py # UI-agnostic indexing progress events
│ └── types.py # Shared types (Chunk, ChunkConfig, IndexStats, SearchResult, ...)
├── storage/
│ ├── vector_store.py # LMDB-backed persistent vector store
│ ├── bm25.py # BM25 keyword index with persistent storage
│ └── walker.py # File system walker with .gitignore support
├── search/
│ ├── search.py # Hybrid search orchestrator (dense + sparse)
│ ├── ranking.py # Result ranking & symbol query detection
│ └── tokens.py # Identifier tokenization (camelCase, snake_case)
└── interfaces/
├── cli.py # Command-line index/search/status entrypoint
├── mcp_server.py # MCP server (stdio transport)
├── progress.py # Realtime terminal/MCP progress bar renderer
└── watcher.py # Live file poller with debounced auto-reindex
sequenceDiagram
participant User as User Code
participant Indexer as CodebaseIndexer
participant Walker as File Walker
participant Chunker as Chunking Engine
participant Embedder as Embedding Model
participant Store as LMDB Vector Store
participant BM25 as BM25 Index
participant Search as Search Engine
User->>Indexer: index()
Indexer->>Walker: walk_files(root, extensions)
Walker-->>Indexer: file_paths
loop Each file
Indexer->>Chunker: chunk_source(source, language)
Chunker-->>Indexer: list[Chunk]
Indexer->>Embedder: embed(chunks)
Embedder-->>Indexer: vectors
Indexer->>Store: store(vectors, chunks)
Indexer->>BM25: index(chunks)
end
Indexer-->>User: IndexStats
User->>Search: search(query)
Search->>Store: query(vector)
Search->>BM25: query(tokens)
Search->>Search: hybrid_fusion(results)
Search-->>User: list[SearchResult]
graph LR
A[Source Files] --> B[File Walker<br/>.gitignore aware]
B --> C[Language Detector]
C --> D{AST Available?}
D -->|Yes| E[Tree-sitter Parser<br/>Function/class boundaries]
D -->|No| F[Line-based Splitter<br/>Configurable size/overlap]
E --> G[Chunk Set]
F --> G
G --> H[Embedding Model<br/>VortexEmbedderV4 (LF4) / Model2Vec / SentenceTransformer]
G --> I[BM25 Tokenizer]
H --> J[(LMDB Vector Store)]
I --> K[(BM25 Index)]
J --> L[Content Hash Memo]
K --> L
L --> M[Skip unchanged files]
graph TD
subgraph "Public API"
Indexer["core.indexer<br/>CodebaseIndexer"]
Search["search.search<br/>search_hybrid()"]
end
subgraph "Core"
Chunking["core.chunking<br/>chunk_source()"]
Embedding["core.embedding<br/>Embedder"]
Language["core.language<br/>detect_language()"]
Types["core.types<br/>Chunk, ChunkConfig, ..."]
end
subgraph "Storage"
VectorStore["storage.vector_store<br/>LMDB Vector Store"]
BM25["storage.bm25<br/>BM25 Index"]
Walker["storage.walker<br/>walk_files()"]
end
subgraph "Interfaces"
CLI["interfaces.cli<br/>Command-line search"]
MCP["interfaces.mcp_server<br/>FastMCP server"]
Watcher["interfaces.watcher<br/>IndexWatcher"]
end
Indexer --> Chunking
Indexer --> Embedding
Indexer --> Language
Indexer --> Types
Indexer --> VectorStore
Indexer --> BM25
Indexer --> Walker
Indexer --> Search
Search --> Embedding
Search --> VectorStore
Search --> BM25
Search --> Types
CLI --> Indexer
MCP --> Indexer
MCP --> Watcher
Watcher --> Walker
| Package | Required | Used For |
|---|---|---|
numpy |
Yes | Vector operations, embedding inference |
lmdb |
Yes | Persistent vector and chunk metadata storage |
bm25s |
Yes | Fast BM25 keyword index and persistence |
pathspec |
Yes | .gitignore pattern matching in file walker |
model2vec |
Optional | Alternative static embeddings |
huggingface-hub |
Yes (default model) | Loading LF2 defaults (VTXAI/vtx-embed-7M-lf2, VTXAI/vtx-embed-1M-lf2) |
tokenizers |
Yes (default model) | HF tokenizer for embedding model |
safetensors |
Yes (default model) | Safe tensor loading for 4-bit weights |
sentence-transformers |
Optional | Transformer-based dense embeddings |
model2vec |
Optional | Alternative static embeddings |
tree-sitter-language-pack |
Optional | AST-aware code chunking |
fastmcp |
Optional | MCP server for LLM tool integration |
Install optional groups:
pip install "vortexa[full]" # model2vec + sentence-transformers + tree-sitter pip install "vortexa[full, mcp]" # everything including MCP server
Copyright 2025 VortexAI
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
OEvortex/vortexa is a public repo on GitHub by OEvortex. Codebase indexing and semantic search engine.
It declares the Apache-2.0 license on GitHub.
Its maker has not tagged a system, so it is unknown here. Its README says what it needs.
Open "Take it" on its page: Take My Repo writes an install plan for your machine and hands it to your AI agent (Claude Code, Codex or Cursor), which asks you before each step. Take My Repo has not tested it on any machine.
Its last change on GitHub was on 2026-11-03. GitHub counts 0 open issues and pull requests. Take My Repo does not judge from that whether it is maintained.
Nobody has reviewed it on Take My Repo yet, which says nothing about its quality.
LLM4Free — All-in-one Python toolkit for web search, AI interaction (40+ free providers), digital utilities, and more. Formerly WebScout.

Supercharge your GitHub Copilot with multiple AI providers
A powerful Model Context Protocol (MCP) server for web search and URL content extraction using DuckDuckGo.
unofficial API designed to interact with Google's Gemini
Minimalist coding agent harness with a Textual TUI and headless CLI. <1k-token system prompt, 18+ LLM providers, AGENTS.md + skills context, session tree, prompt/auto permissions.