LDF: a compact table format for language models

Field notes

LDF organizes a document as tables with typed headers and rows; stable nested objects can become dotted columns, while arrays of objects use child tables joined through the parent’s first column. Its examples distinguish a missing key from a present null and describe a sparse-key rule for values present in fewer than half of rows. The published benchmark uses o200k_base on the first 1,000 records in its datasets and includes results as well as reported misses.