Data Traversal & Versioning
The -3 Mapping Standard for managing multi-gigabyte historical updates and maintaining in-memory traversal routes without archiving raw data.
The -3 Mapping Standard
States like Mississippi frequently release massive voter roll updates (up to 2.5GB). Across a few years, this data velocity will rapidly chew up memory bounds on consumer devices. Marigold is an optimized inference engine—we are not a historical data analysis tool or a data lake.
To support anomaly tracking across time while remaining lightweight, Marigold strictly enforces a -3 Voter Roll Mapping default. This means Marigold will keep the optimized data traversal maps (the pointers and metadata matrices) in memory for the 3 most recently uploaded state files.
Optimizing for Workflows, Not Storage
The mapping files are extremely cheap to store compared to the raw JSON blobs. While we could theoretically keep mappings in memory forever, the DAIO vision mandates a query-driven, optimized UX.
By maintaining only the recent -3 mapping states, we ensure that local memory footprint remains minimal. If a partner needs deep historical analysis, they must calculate the delta (∆) locally and utilize our Bi-Directional Delta API to visualize the aberrations. We provide the map; we are not the source of record. That responsibility lies with the user's hard drive or the external vendor's database.