jsonlt

The Python reference implementation of JSONLT (JSON Lines Table), a data format for storing keyed records in append-only files. JSONLT builds on JSON Lines and optimizes for version control. Modifications append new lines rather than rewriting existing content, producing clean and meaningful diffs.

Note

This package is under active development. The API may change before the 1.0 release.

Resources

• Specification: formal definition of the JSONLT format
• Documentation: guides, tutorials, and API reference
• Conformance tests: language-agnostic test suite

Installation

pip install jsonlt-python

# Or

uv add jsonlt-python

Requires Python 3.10 or later.

Quick start

from jsonlt import Table

# Create a table with initial records
table = Table.from_records(
    "users.jsonlt",
    [
        {"id": "alice", "role": "admin", "email": "alice@example.com"},
        {"id": "bob", "role": "user", "email": "bob@example.com"},
    ],
    key="id",
)

# Read records
user = table.get("alice")  # Returns the record or None
exists = table.has("bob")  # Returns True

# Update a record
table.put({"id": "alice", "role": "admin", "email": "alice@newdomain.com"})

# Delete records (appends a tombstone)
table.delete("bob")

# Later, load the existing table
table = Table.from_file("users.jsonlt")

# Iterate over all records
for record in table.all():
    print(record)

The underlying file after these operations:

{"$jsonlt":{"key":"id","version":1}}
{"id":"alice","email":"alice@example.com","role":"admin"}
{"id":"bob","email":"bob@example.com","role":"user"}
{"id":"alice","email":"alice@newdomain.com","role":"admin"}
{"id":"bob","$deleted":true}

When to use JSONLT

JSONLT works well for configuration, metadata, and small-to-medium datasets where you want human-readable files that play nicely with Git. It's a good fit when you need keyed record storage but don't want the overhead of a database, and when you want to see exactly what changed in a pull request. The append-only design supports safe concurrent access from multiple processes.

JSONLT is not a database. For large datasets, high write throughput, or complex queries, consider SQLite or a proper database.

Compound keys

JSONLT supports multi-field compound keys for composite identifiers:

orders = Table.from_records(
    "orders.jsonlt",
    [
        {"customer_id": "alice", "order_id": 1, "total": 99.99},
        {"customer_id": "alice", "order_id": 2, "total": 149.99},
    ],
    key=("customer_id", "order_id"),
)

order = orders.get(("alice", 1))

Transactions

Transactions provide snapshot isolation and atomic writes with conflict detection:

from jsonlt import Table, ConflictError

table = Table("counters.jsonlt", key="name")

with table.transaction() as tx:
    counter = tx.get("visits")
    new_count = (counter["count"] + 1) if counter else 1
    tx.put({"name": "visits", "count": new_count})
# Commits automatically; rolls back on exception

# Handle concurrent modification conflicts
try:
    with table.transaction() as tx:
        tx.put({"name": "counter", "value": 42})
except ConflictError as e:
    print(f"Conflict on key: {e.key}")

Dictionary-like access

Tables can be used like dictionaries:

# Get a record (raises KeyError if not found)
user = table["alice"]

# Set a record (key in record must match)
table["alice"] = {"id": "alice", "role": "admin"}

# Delete a record
del table["bob"]

# Check membership
if "alice" in table:
    print("Found alice")

# Iterate over keys
for key in table:
    print(key)

# Get record count
print(len(table))

Methods like pop(), setdefault(), and update() also work. The keys(), values(), and items() methods return sorted lists rather than views to maintain JSONLT's deterministic key ordering.

Equality

Tables support value-based equality comparison:

table1 = Table("users.jsonlt", key="id")
table2 = Table("users.jsonlt", key="id")

# Equal if same path, key specifier, and records
if table1 == table2:
    print("Tables have identical content")

Two tables are equal when they have the same resolved path, key specifier, and record state. Transactions are equal when they reference the same parent table instance and have identical snapshot state.

Tables and transactions are mutable and therefore not hashable (cannot be used as dictionary keys or in sets).

Finding records

# Find all records matching a predicate
expensive = table.find(lambda r: r.get("price", 0) > 100)

# Find with limit
top_3 = table.find(lambda r: r.get("in_stock"), limit=3)

# Find the first match
first = table.find_one(lambda r: r.get("category") == "electronics")

Maintenance

# Compact the file (removes tombstones and superseded records)
table.compact()

# Clear all records
table.clear()

# Force reload from disk
table.reload()

API summary

Table

Method	Description
Table(path, key)	Open or create a table
Table.from_records(path, records, key)	Create table with records
Table.from_file(path)	Load existing table
get(key)	Get a record by key, or None
has(key)	Check if a key exists
put(record)	Insert or update a record
delete(key)	Delete a record
all()	Iterate all records
keys()	Iterate all keys
items()	Iterate (key, record) pairs
count()	Number of records
find(predicate, limit=None)	Find matching records
find_one(predicate)	Find first match
transaction()	Start a transaction
compact()	Remove historical entries
clear()	Remove all records
reload()	Reload from disk

Tables support table[key], table[key] = record, del table[key], len(table), key in table, and for key in table.

Transaction

Method	Description
get(key)	Get from snapshot
has(key)	Check in snapshot
put(record)	Buffer a write
delete(key)	Buffer a deletion
commit()	Write to disk
abort()	Discard changes

Transactions support the same dictionary-like access as tables.

Exceptions

All exceptions inherit from JSONLTError:

Exception	Description
ParseError	Invalid file format
InvalidKeyError	Invalid or missing key
FileError	I/O error
LockError	Cannot get lock
LimitError	Size limit exceeded
TransactionError	Invalid transaction state
ConflictError	Write-write conflict

Conformance

This library implements the JSONLT 1.0 Specification. It provides both Lenient and Strict Parser conformance profiles, and generates Strict-conformant output by default.

As the reference implementation, jsonlt-python passes the JSONLT conformance test suite. Implementers of other JSONLT libraries can use these tests to verify compatibility.

Acknowledgements

The JSONLT format draws from related work including BEADS, which uses JSONL for git-backed structured storage.

AI disclosure

The development of this library involved AI language models, specifically Claude (Anthropic). AI tools contributed to drafting code, tests, and documentation. Human authors made all design decisions and final implementations, and they reviewed, edited, and validated AI-generated content. The authors take full responsibility for the correctness of this software.

License

MIT License. See LICENSE for details.