toon
🎒 Token-Oriented Object Notation (TOON) – compact, human-readable serialization of JSON data for LLM prompts. TypeScript SDK, CLI, benchmarks.
- 26k
- TypeScript
- MIT
- 9 d ago

Toonify: Compact data format reducing LLM token usage by 30-60%
Repo ScrapeGraphAI/toonify, read from GitHub 5 min ago. ScrapeGraphAI has not joined Take My Repo.
ScrapeGraphAI/toonify is a public repo on GitHub by ScrapeGraphAI. Toonify: Compact data format reducing LLM token usage by 30-60%. It declares no license on GitHub. Public does not mean open source: without a license, nobody is granted the right to reuse it. It is written mainly in Python, has 367 stars and last changed on 2026-02-06. Nobody has reviewed it on Take My Repo yet, which says nothing about its quality. Read from GitHub on 2026-11-20.
In the maintainer's words. Shown from GitHub; Take My Repo does not keep a copy.
A compact, human-readable serialization format designed for passing structured data to Large Language Models with significantly reduced token usage.
TOON achieves CSV-like compactness while adding explicit structure, making it ideal for:
pip install toonify
For development:
pip install toonify[dev]
With Pydantic support:
pip install toonify[pydantic]
from toon import encode, decode # Encode Python dict to TOON data = { 'products': [ {'sku': 'LAP-001', 'name': 'Gaming Laptop', 'price': 1299.99}, {'sku': 'MOU-042', 'name': 'Wireless Mouse', 'price': 29.99} ] } toon_string = encode(data) print(toon_string) # Output: # products[2]{sku,name,price}: # LAP-001,Gaming Laptop,1299.99 # MOU-042,Wireless Mouse,29.99 # Decode TOON back to Python result = decode(toon_string) assert result == data
# Encode JSON to TOON toon input.json -o output.toon # Decode TOON to JSON toon input.toon -o output.json # Use with pipes cat data.json | toon -e > data.toon # Show token statistics toon data.json --stats
TOON supports direct conversion from Pydantic models:
from pydantic import BaseModel from toon import encode_pydantic, decode_to_pydantic # Define Pydantic models class User(BaseModel): id: int name: str email: str # Encode Pydantic models to TOON users = [ User(id=1, name='Alice', email='alice@example.com'), User(id=2, name='Bob', email='bob@example.com') ] toon = encode_pydantic(users) print(toon) # Output: # [2]{id,name,email}: # 1,Alice,alice@example.com # 2,Bob,bob@example.com # Decode TOON back to Pydantic models decoded_users = decode_to_pydantic(toon, User) assert all(isinstance(u, User) for u in decoded_users)
Features:
See examples/pydantic_usage.py for more examples.
TOON provides a powerful feature to generate response structure templates that can be included in LLM prompts. This tells the model exactly what format to return data in, without needing to provide examples with actual data.
from toon import generate_structure # Define the expected response structure schema = { "name": "name of the person", "age": "age of the person", "occupation": "job description of the person" } # Generate the structure template structure = generate_structure(schema) print(structure) # Output: # name: <name of the person> # age: <age of the person> # occupation: <job description of the person> # Use in your LLM prompt prompt = f"""Extract person information from the text and return it in this format: {structure} Text: [your text here...]"""
For arrays and complex structures:
schema = { "products": [{ "name": "product name", "price": "price in USD", "rating": "rating from 1-5" }] } structure = generate_structure(schema) print(structure) # Output: # products[N]{name,price,rating}: # <product name>,<price in USD>,<rating from 1-5> # ...
With Pydantic models:
from pydantic import BaseModel, Field from toon import generate_structure_from_pydantic class Product(BaseModel): name: str = Field(description="product name") price: float = Field(description="price in USD") in_stock: bool = Field(description="availability status") # Generate structure from model structure = generate_structure_from_pydantic(Product) # Use in LLM prompts without providing examples
Benefits:
See examples/structure_template_usage.py for comprehensive examples.
# Simple key-value pairs
title: Machine Learning Basics
chapters: 12
published: true
Primitive arrays (inline):
temperatures: [72.5,68.3,75.1,70.8,73.2]
categories: [electronics,computers,accessories]
Tabular arrays (uniform objects with header):
inventory[3]{sku,product,stock}:
KB-789,Mechanical Keyboard,45
MS-456,RGB Mouse Pad,128
HD-234,USB Headset,67
List arrays (non-uniform or nested):
tasks[2]:
Complete documentation
Review pull requests
server:
hostname: api-prod-01
config:
port: 8080
region: us-east
Strings are quoted only when necessary:
,, :, ", newlines)true, false, null)simple: ProductName
quoted: "Product, Description"
escaped: "Size: 15\" display"
multiline: "First feature\nSecond feature"
encode(data, options=None)Convert Python object to TOON string.
Parameters:
data: Python dict or listoptions: Optional dict with:
delimiter: 'comma' (default), 'tab', or 'pipe'indent: Number of spaces per level (default: 2)key_folding: 'off' (default) or 'safe'flatten_depth: Max depth for key folding (default: None)Example:
toon = encode(data, { 'delimiter': 'tab', 'indent': 4, 'key_folding': 'safe' })
decode(toon_string, options=None)Convert TOON string to Python object.
Parameters:
toon_string: TOON formatted stringoptions: Optional dict with:
strict: Validate structure strictly (default: True)expand_paths: 'off' (default) or 'safe'default_delimiter: Default delimiter (default: ',')Example:
data = decode(toon_string, { 'expand_paths': 'safe', 'strict': False })
encode_pydantic(model, options=None, exclude_unset=False, exclude_none=False, exclude_defaults=False, by_alias=False)Convert Pydantic model(s) to TOON string.
Parameters:
model: Pydantic model instance or list of model instancesoptions: Same as encode() functionexclude_unset: If True, exclude fields that were not explicitly setexclude_none: If True, exclude fields with None valuesexclude_defaults: If True, exclude fields with default valuesby_alias: If True, use field aliases instead of field namesExample:
from pydantic import BaseModel from toon import encode_pydantic class User(BaseModel): id: int name: str email: str | None = None user = User(id=1, name='Alice') toon = encode_pydantic(user, exclude_none=True)
decode_to_pydantic(toon_string, model_class, options=None)Decode TOON string to Pydantic model(s).
Parameters:
toon_string: TOON formatted stringmodel_class: Pydantic model class to instantiateoptions: Same as decode() functionReturns:
Example:
from pydantic import BaseModel from toon import decode_to_pydantic class User(BaseModel): id: int name: str toon = "id: 1\nname: Alice" user = decode_to_pydantic(toon, User)
generate_structure(schema, options=None)Generate a TOON structure template from a schema definition for use in LLM prompts.
Parameters:
schema: Schema definition as dict or list
{"field_name": "description"}{"field": {"nested": "description"}}{"field": [{"item_field": "description"}]}options: Optional dict with:
delimiter: 'comma' (default), 'tab', or 'pipe'indent: Number of spaces per level (default: 2)Returns:
Example:
from toon import generate_structure schema = { "name": "name of the person", "age": "age of the person", "occupation": "job description" } structure = generate_structure(schema) print(structure) # Output: # name: <name of the person> # age: <age of the person> # occupation: <job description> # Use in LLM prompt: prompt = f"Extract person info in this format:\n{structure}"
generate_structure_from_pydantic(model_class, options=None, include_descriptions=True)Generate a TOON structure template from a Pydantic model for use in LLM prompts.
Parameters:
model_class: Pydantic model class (BaseModel subclass)options: Same as generate_structure()include_descriptions: If True, use field descriptions from modelReturns:
Example:
from pydantic import BaseModel, Field from toon import generate_structure_from_pydantic class User(BaseModel): id: int = Field(description="user identifier") name: str = Field(description="full name") email: str = Field(description="email address") structure = generate_structure_from_pydantic(User) print(structure) # Output: # id: <user identifier> # name: <full name> # email: <email address>
usage: toon [-h] [-o OUTPUT] [-e] [-d] [--delimiter {comma,tab,pipe}]
[--indent INDENT] [--stats] [--no-strict]
[--key-folding {off,safe}] [--flatten-depth DEPTH]
[--expand-paths {off,safe}]
[input]
TOON (Token-Oriented Object Notation) - Convert between JSON and TOON formats
positional arguments:
input Input file path (or "-" for stdin)
optional arguments:
-h, --help show this help message and exit
-o, --output OUTPUT Output file path (default: stdout)
-e, --encode Force encode mode (JSON to TOON)
-d, --decode Force decode mode (TOON to JSON)
--delimiter {comma,tab,pipe}
Array delimiter (default: comma)
--indent INDENT Indentation size (default: 2)
--stats Show token statistics
--no-strict Disable strict validation (decode only)
--key-folding {off,safe}
Key folding mode (encode only)
--flatten-depth DEPTH Maximum key folding depth (encode only)
--expand-paths {off,safe}
Path expansion mode (decode only)
Collapse single-key chains into dotted paths:
data = { 'api': { 'response': { 'product': { 'title': 'Wireless Keyboard' } } } } # With key_folding='safe' toon = encode(data, {'key_folding': 'safe'}) # Output: api.response.product.title: Wireless Keyboard
Expand dotted keys into nested objects:
toon = 'store.location.zipcode: 10001' # With expand_paths='safe' data = decode(toon, {'expand_paths': 'safe'}) # Result: {'store': {'location': {'zipcode': 10001}}}
Choose the delimiter that best fits your data:
# Tab delimiter (better for spreadsheet-like data) toon = encode(data, {'delimiter': 'tab'}) # Pipe delimiter (when data contains commas) toon = encode(data, {'delimiter': 'pipe'})
JSON (247 bytes):
{
"products": [
{"id": 101, "name": "Laptop Pro", "price": 1299},
{"id": 102, "name": "Magic Mouse", "price": 79},
{"id": 103, "name": "USB-C Cable", "price": 19}
]
}TOON (98 bytes, 60% reduction):
products[3]{id,name,price}:
101,Laptop Pro,1299
102,Magic Mouse,79
103,USB-C Cable,19
Use TOON when:
Use JSON when:
git clone https://github.com/ScrapeGraphAI/toonify.git
cd toonify
pip install -e .[dev]pytest pytest --cov=toon --cov-report=term-missing
python examples/basic_usage.py python examples/advanced_features.py
Benchmarked across 50 diverse, real-world datasets:
💰 Cost Impact: At GPT-4 pricing, TOON saves $2,147 per million API requests and $5,408 per billion tokens.
📊 View Full Benchmark Results →
Contributions are welcome! We appreciate bug fixes, feature additions, documentation improvements, and more.
Quick Start:
git checkout -b feature/amazing-feature)pytest)git commit -m 'Add amazing feature')git push origin feature/amazing-feature)For detailed guidelines, please see our Contributing Guide.
MIT License - see LICENSE file for details.
Python implementation inspired by the TypeScript TOON library at toon-format/toon.
Made with love by the ScrapeGraph team
ScrapeGraphAI/toonify is a public repo on GitHub by ScrapeGraphAI. Toonify: Compact data format reducing LLM token usage by 30-60%.
It declares no license on GitHub. Public does not mean open source: without a license, nobody is granted the right to reuse it.
Its maker has not tagged a system, so it is unknown here. Its README says what it needs.
Open "Take it" on its page: Take My Repo writes an install plan for your machine and hands it to your AI agent (Claude Code, Codex or Cursor), which asks you before each step. Take My Repo has not tested it on any machine.
Its last change on GitHub was on 2026-02-06. Take My Repo does not judge from that whether it is maintained.
Nobody has reviewed it on Take My Repo yet, which says nothing about its quality.
🎒 Token-Oriented Object Notation (TOON) – compact, human-readable serialization of JSON data for LLM prompts. TypeScript SDK, CLI, benchmarks.

💫 Industrial-strength Natural Language Processing (NLP) in Python


Open Lakehouse Format for Multimodal AI. Convert from Parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..


🤖 AI-powered web scraping editor with visual workflow builder. Build, test & deploy web scrapers using natural language. Powered by ScrapeGraphAI & LangGraph.


Streamlit demo of Scrapegraph-ai for GPT4-hackaton

Official Python SDK for the ScrapeGraph AI API. Smart scraping, search, crawling, markdownify, agentic browser automation, scheduled jobs, and structured data extraction

CLI for AI-powered web scraping, structured data extraction, web search, multi-page crawling, and scheduled page-change monitoring powered by the ScrapeGraph AI API. Turn any URL into markdown, HTML, screenshots, links, images, summaries, or brand assets; pull structured JSON with a prompt + schema; watch pages on a cron with webhook alert
This page is also available as structured data, with the same sources and dates.
Take My Repo has not tested this on your machine. Requirements come from the project's own documentation.