Skip to content

Persistence and Compatibility

include/tess/persistence/archive.h provides a dependency-free, versioned binary envelope for authoritative world fields. It is a cold-path save/load primitive, not a live replication stream or a derived-product cache format.

Archive Schema

Applications define a PersistenceSchema with a stable 64-bit application schema ID, schema version, and ordered PersistedField declarations. Each field declaration assigns a stable 64-bit field ID and field version to a tag already present in the world's FieldSchema. IDs are deliberately explicit: C++ type names, RTTI values, and addresses are not stable persistence keys.

The initial format supports bool, integral, scoped-enum, float, and double columns whose scalar width is 1, 2, 4, or 8 bytes. Values use canonical little-endian encoding; floating-point values preserve their IEEE bit pattern. In C++20 a scoped enum is guaranteed to have a fixed underlying type. The archive conservatively rejects all unscoped enums at compile time because the language does not provide a trait that distinguishes their fixed and non-fixed forms. Applications can persist an unscoped enum through an explicit scalar field or a later codec extension.

The archive accepts every scoped-enum value representable by its underlying type, including values that do not name a declared enumerator. This keeps conversion from hostile bytes defined without assuming that tess knows the application's enum domain. Applications must validate those values after load, or migrate them while moving to a new field/schema version.

Envelope and Compatibility

save_world_archive writes format version, shape and chunk extents, lattice identity/version, key-layout version, application schema identity/version, tess library version, residency kind, field descriptors, and canonical chunk-key-ordered records. Each record contains the stable chunk lifecycle state, active flags, entity count, and selected authoritative field columns. WorldArchiveResidency distinguishes dense and sparse envelopes. A CRC-32 covers the complete canonical archive except the checksum's own four-byte slot, so accidental damage to compatibility metadata is detected before that metadata drives a compatibility decision.

Format v1 freezes an eight-byte magic followed by its 113-byte fixed header: the format version begins at byte 8, the declared body size at byte 12, the four-byte checksum slot at byte 20, and the descriptor/body region at byte 121. The CRC input is the concatenation of bytes [0, 20) and [24, end); the stored checksum bytes are neither zero-filled nor included. The full golden-byte test pins this prefix, scalar byte order, and checksum framing so a layout change must use a new format version rather than silently changing v1.

inspect_world_archive validates the magic, lengths, checksum, dimensions, unique field IDs and field-descriptor encodings, dense archives' complete logical chunk count, canonical unique chunk keys, and complete body before a world is involved. Inspection does not decode field scalar payloads. load_world_archive classifies shape, lattice, key layout, residency, schema, field, and sparse-capacity compatibility before mutation, then decodes every scalar in a complete preflight pass before preparing sparse residency or writing a field. Scalar corruption therefore leaves the target unchanged. Its WorldArchiveResult and WorldArchiveInfo retain the source metadata; WorldArchiveStatus distinguishes damage from compatibility decisions.

Status precedence is intentional. Once the complete magic and format-version word are available, an unknown version is UnsupportedFormat without applying v1's envelope equations; a future version may extend or reinterpret that framing. For v1, envelope and integrity failures win before typed compatibility, including a checksum failure in otherwise incompatible metadata. For a structurally valid archive, typed load checks shape, lattice, key layout, residency, schema identity, schema version, field descriptors, residency capacity/dense chunk completeness, and scalar encodings in that order. This gives callers one stable primary diagnosis without weakening the no-mutation preflight guarantee.

A differing application schema version returns WorldArchiveStatus::MigrationRequired; it is never silently reinterpreted. The intended migration is explicit: load with the old PersistenceSchema and old world representation, transform authoritative values in application code, then save with the new schema. A changed field descriptor under the same schema version is FieldMismatch, which identifies an incorrectly versioned schema rather than guessing a conversion.

Loaded State

Successful loads restore selected fields, active flags, stable chunk state, and entity counts. Dirty history, content/topology version counters, residency generations, region graphs, paths, products, and caches are intentionally not serialized. Dense loads preserve each target chunk's existing version counters and advance both monotonically, so derived products warmed before an in-place load cannot become valid again through a reset-to-zero alias. Sparse loads evict and rematerialize their archived resident set: page metadata counters restart, while each page receives a new world-monotonic residency generation. Sparse graphs and caches use those generations or the residency fingerprint to reject pre-load derived state. Every loaded chunk receives a full-chunk topology invalidation; the caller supplies the dirty mask. Derived state must be rebuilt against the loaded authoritative fields.

Dense archives contain every shaped chunk. Sparse archives contain the current resident working set in canonical key order and reject a target whose fixed capacity cannot hold that set before evicting anything. The current SparseResidentWorld has no non-resident backing store—eviction discards the page—so the archive cannot invent previously evicted contents. Applications that own durable chunk streaming must save authoritative chunks before eviction and coordinate those records with their external store.

Integrity and Trust Boundary

CRC-32 detects accidental truncation or corruption; it is not authentication and does not make a hostile archive trustworthy. File size limits, provenance, signatures, encryption, atomic file replacement, and user/account policy stay with the application and its I/O layer. tess consumes and produces byte spans and does not open paths, perform network I/O, or manage credentials.