Filebase | Spt

| Index Type | Use Case | |------------|----------| | (B-tree) | namespace = 'processed' | | Range (B-tree) | size > 1e6 or created_at >= '2025-01-01' | | Full-text (inverted) | content_type ~ 'application/pdf' AND text ~ 'invoice' | | Spatial (R-tree) | gps within radius(40.7128, -74.0060, 10km) | | Bloom filter | Existence checks on rare tags (e.g., checksum_present=true ) |

Query example (GraphQL-like):

files( namespace: "archive" predicate: "mime_type = application/x-parquet" sort: "size DESC" limit: 100 ) file_id version content_hash metadata retention_days, owner spt filebase

1. Abstract The SPT Filebase is a specialized, high-performance data storage and retrieval architecture designed for managing structured, semi-structured, and unstructured file assets within an SPT (Single Point of Truth or Stream-Process-Transform) ecosystem. Unlike traditional file systems or blob stores, the SPT Filebase implements a dual-hash content addressing scheme, transactional versioning, and a predicate-based query engine over file metadata. This document explores its internal design, indexing mechanics, concurrency model, and fault-tolerance guarantees. 2. Core Architecture An SPT Filebase instance is composed of four logical layers: | Index Type | Use Case | |------------|----------|

with filebase.transaction(isolation="SERIALIZABLE") as txn: f = txn.get("spt://abc123/raw/sensor.bin") f.metadata["processed_at"] = now() f.content = new_data txn.commit() # creates new version atomically The index is built on a reverse index from metadata predicates to file versions . Supported index types: Supported index types: | Layer | Function |

| Layer | Function | |-------|----------| | | Handles file writes, chunking, deduplication, and initial hash computation. | | Index Layer | Maintains a persistent LSM (Log-Structured Merge) tree over metadata tags, file IDs, and temporal markers. | | Storage Layer | A content-addressable block store (CABS) with optional erasure coding and tiered compression (Zstandard, LZ4, or LZMA). | | Query Layer | Exposes a SQL-like or GraphQL interface for predicate-based file discovery and retrieval. | 3. File Identity & Addressing Every file in the SPT Filebase is identified by a triple: