Technology · Document store
MongoDB
A document database storing flexible BSON records. When to embed vs reference, indexes, replica sets with elections, sharding mechanics, and write concerns.
11 min read
MongoDB is a distributed document database that stores records as semi-structured BSON (binary JSON). Instead of spreading an entity across dozens of normalized tables joined by foreign keys, MongoDB groups related data into nested documents that can be read and updated in a single disk operation.
When to reach for it
- Polymorphic or evolving schemas: Catalogs with varied product attributes, content management systems, event metadata, or user-customized form fields.
- Hierarchical data: Entities naturally read together, such as an order with line items and shipping addresses, or an inbox thread with replies.
- Rapid iterative development: Adding fields without running blocking schema migrations across terabytes of rows.
- Horizontal sharding with rich querying: Sharding across servers while preserving secondary indexes and aggregations, unlike DynamoDB's strict key-access model.
For rigid relational financial ledgers requiring multi-table ACID transactions and heavy analytics, PostgreSQL or a dedicated warehouse is usually a cleaner fit.
The document model: embed vs reference
The central design decision in MongoDB is whether child data belongs inside the parent document (embedding) or in a separate collection (referencing).
Embedding (Denormalization)
- Single read retrieves whole entity
- Atomic updates to parent and children without distributed transactions
- Document size grows toward the 16 MB limit
- Unbounded arrays hurt memory and disk performance
Pick it when: 1-to-few relationships where child data is always queried with the parent (line items on an order, addresses for a user).
Referencing (Normalization)
- Documents remain small and bounded
- Independent updates without rewriting the parent
- Requires application-side joins or $lookup aggregation pipelines
- No native atomic writes across collections without multi-document transactions
Pick it when: 1-to-many or 1-to-unbounded relationships (logs for a device, comments on a viral post, followers of a user).
Indexing strategies
MongoDB uses B-tree indexes. Without an index, every query performs a collection scan checking every document on disk.
- Compound indexes: Order fields by the ESR rule (Equality first, Sort second, Range last). Indexing
{ status: 1, created_at: -1, total: 1 }satisfies equality on status, returns pre-sorted dates, and filters totals without in-memory sorting. - Multikey indexes: Indexing an array field creates an index entry for every element in the array, enabling fast lookups for tags or category lists.
- Partial and TTL indexes: Partial indexes index only documents matching a filter expression, saving RAM. TTL indexes automatically delete documents after a timestamp expires, handy for temporary sessions and rate-limit counters.
Replica sets and high availability
A MongoDB replica set is an active-passive cluster with a single Primary and multiple Secondaries (typically a 3-node P-S-S quorum).
- Writes go exclusively to the Primary, which records mutations sequentially in its local replication log (oplog).
- Secondaries asynchronously stream and apply the oplog, keeping their local data files in sync.
- If the primary misses heartbeats for 10 seconds, secondaries initiate a Raft-style election to choose a new primary with the freshest oplog position.
- If an old primary had un-replicated writes before crashing, upon recovery it fetches the new primary's oplog, writes its un-replicated mutations to a
rollback/directory, and steps down to a secondary.
Read and write concerns
MongoDB lets you tune consistency per operation rather than per cluster:
w: 1 (Acknowledged)
- Lowest write latency
- Primary memory ack
- Data lost if primary crashes before replicating to secondaries
Pick it when: Metrics, analytics pings, non-critical telemetry.
w: majority (Durable)
- Survives primary failure without rollback
- Committed to majority quorum
- Latency bound by network roundtrip to nearest secondary
Pick it when: Financial records, user credentials, orders.
readConcern: majority
- Never reads dirty data that could be rolled back during an election
- Slightly higher read latency
Pick it when: Critical read-after-write flows.
Add j: true (journaled) to guarantee the operation was flushed to the on-disk journal before acknowledging, protecting against simultaneous power loss on all nodes.
Sharding architecture
When a collection outgrows one machine's storage or write throughput, MongoDB shards data horizontally across replica sets.
- mongos routers: Stateless routing proxies that applications connect to. They inspect queries and route them directly to the target shard.
- Config servers: A dedicated replica set holding the cluster metadata, shard key ranges, and chunk locations.
- Shard key selection: Determines how data distributes across shards.
- Hashed shard key: Hashes the field value to distribute writes evenly across shards, avoiding hotspots on monotonic IDs (like ObjectId or timestamps). Range queries must scatter-gather across all shards.
- Ranged shard key: Keeps contiguous ranges together. Efficient for targeted range queries, but sequential keys (like auto-increment IDs or timestamps) route 100% of writes to a single hot shard.
- Chunks and the balancer: Data is split into logical chunks (default 64 MB). When chunks become uneven across shards, a background balancer migrates chunks between shards without taking the cluster offline.
Numbers to quote
- Max BSON document sizestore larger payloads in GridFS or S3
- 16 MB
- Oplog sizemust hold enough hours of buffer for backup/restore
- typically 5–10% of disk
- Replica set election timeclients buffer writes during failover
- typically 2–5 seconds
- Point lookup latencywith indexed queries in wiredTiger cache
- 1–5 ms
- Max members per replica set
- 50 nodes (7 voting members)
Where it fits vs Postgres and DynamoDB
MongoDB
- Rich secondary indexes, aggregation pipeline, text search
- Horizontal auto-sharding
- Nested documents feel native in code
- Operational overhead of mongos and config servers
- Memory-hungry WiredTiger cache
Pick it when: Document-centric domains with varied schemas and high write volume.
PostgreSQL (with JSONB)
- Full ACID relational power
- Battle-tested SQL optimizer
- GIN indexes on JSON fields
- Manual sharding (Citus or foreign data wrappers)
- More rigid schema migrations
Pick it when: Relational data with occasional semi-structured attributes.
DynamoDB
- Zero maintenance serverless scaling
- Predictable single-digit ms latency
- Native AWS IAM integration
- Strict 400 KB item limit
- Rigid access-pattern design
- Expensive ad-hoc scans
Pick it when: Key-value and partition-key lookups on AWS with zero ops.
Where it bites
- Memory saturation: MongoDB's WiredTiger storage engine allocates 50% of available RAM (minus 1 GB) as cache. If your active working set (data + indexes) exceeds RAM, performance falls off a cliff as reads hit disk.
- Scatter-gather queries: If a query does not include the shard key,
mongosmust broadcast the request to every shard in the cluster and merge results, creating high latency tails. - Document growth relocations: Modifying embedded documents that expand existing documents causes fragmentation and storage bloat.
Try it
Multi-tenant SaaS Platform HLD
Design a document-backed multi-tenant store with customizable schemas.