MOS Data Model

Firestore as the only primary datastore for a concurrent, retry-heavy pipeline, designed so that partial failures still converge on one consistent state.

Overview

Marketing OS runs on Firestore: 69 flat top-level collections, no subcollections, plain-string foreign keys, and 122 hand-declared composite indexes. The goal: let concurrent, retryable, partly failing pipeline work converge on a single consistent state without a relational database.

Core design decisions

• Flat collections with explicit foreign keys.
• Three document-ID strategies, including deterministic IDs so writes are idempotent.
• Embed bounded children, reference unbounded ones.
• Content-addressed cache documents, so repeated pipeline steps are free.
• Six lifecycle axes derived at read time by pure functions, never stored, so they can't drift.
• One canonical source for every computed number, such as credits and access.
• ISO strings for business dates, Timestamps for system events.
• Every denormalized field has a stated reason.
• Deny-by-default security rules; indexes and rules managed as code.

Integrity

• Zod validation on every write.
• Transactions only at contention points: leases and claims.
• Hash-based alert deduplication.
• A read-only audit that reports cross-collection contradictions.
• Schema changes are additive and optional; removed collections are documented in code.

Success criteria

• No collection scans on the request path.
• Safe re-runs by construction.
• One author for every stored number.
• Client access is read-only.
• Schema changes deploy with no migration window.

Stack

• DATABASE: Firestore (Native mode), Firebase Admin SDK
• AUTH: Firebase Auth custom claims
• VALIDATION: TypeScript 5 (strict), Zod
• AWS: S3 (binaries), SQS (work queues), CloudWatch
• DEV: Firebase Emulator Suite, rules + indexes in git