MongoDB / Learn

MongoDB is a document-oriented NoSQL database. It stores data as BSON (Binary JSON) documents in collections. Unlike relational databases, it does not require a fixed schema, making it flexible for evolving data structures.

Indexes store a small portion of the collection's data in a sorted structure. This allows MongoDB to find matching documents without scanning every document in the collection (COLLSCAN). Without indexes, queries on large collections are very slow.

The aggregation pipeline passes documents through stages like $match (filter), $group (aggregate), $project (reshape), $sort, $limit, and $lookup (join). Each stage transforms the data for the next, enabling complex analytics queries.

A replica set is a group of mongod instances that replicate data between each other. One node is primary (accepts writes); others are secondaries (replicate data). If the primary fails, an election promotes a secondary, providing automatic failover.

$lookup performs a left outer join between the current collection and a foreign collection, matching documents by a local field to a foreign field. The joined documents are added as an array field to each input document.

Sharding horizontally partitions data across multiple servers (shards) using a shard key. Each shard holds a subset of the data. A mongos router directs queries to the appropriate shard(s), enabling MongoDB to scale beyond the capacity of a single machine.

$setOnInsert only applies its update when the operation results in an insert (upsert). If the document already exists and is being updated, $setOnInsert has no effect. It is commonly used with upsert: true to initialise fields only on creation.

MongoDB automatically generates an ObjectId for _id if not provided. An ObjectId is 12 bytes: 4-byte Unix timestamp, 5-byte random value (machine+process), and 3-byte incrementing counter. This makes it globally unique and roughly sortable by creation time.

explain('executionStats') returns the winning query plan and detailed stats like nReturned, totalDocsExamined, and totalKeysExamined. A high totalDocsExamined relative to nReturned means the query is doing a collection scan and likely needs an index.

A sparse index only includes documents that contain the indexed field. Documents missing the field are excluded from the index entirely. This is useful for optional fields where most documents don't have the field, keeping the index small.

insertOne() inserts exactly one document and returns an InsertOneResult with acknowledged: true and the insertedId. The older insert() method is deprecated. insertMany() accepts an array and is used for multiple documents.

When multiple fields are specified in a find() filter object, MongoDB applies implicit AND logic: all conditions must be true. Here, only documents where status is 'shipped' AND amount is greater than 100 are returned.

$exists: true returns documents that contain the specified field, even if the field's value is null. $exists: false returns documents where the field is absent. Note that a field with value null IS present; $exists: true would match it.

$in matches documents where the field value equals any of the values in the given array. It is equivalent to an OR condition across those values. For array fields, $in matches if any element in the field array equals any value in the $in array.

$set updates or creates the specified fields without modifying any other fields in the document. If a field doesn't exist, $set creates it. To replace an entire document you would use replaceOne() or omit update operators entirely.

$unset removes the specified field from the document. The field key and its value are both deleted. You can pass any string as the value (conventionally an empty string or 1) since only the key matters to $unset.

$inc atomically increments a numeric field by the given value. A negative value decrements the field. This is safe for concurrent updates because MongoDB applies the increment atomically at the document level.

$push appends one element to an existing array. If the field does not exist, $push creates it as a single-element array. To prevent duplicates, use $addToSet instead. $push can also be used with $each to append multiple values at once.

$pull removes all elements from an array that match a specified value or condition. $pop removes the first element ($pop: -1) or the last element ($pop: 1) by position. They serve different use cases for array manipulation.

$addToSet treats the array as a set and only adds the element if it is not already present. $push always appends, even if the value already exists. $addToSet is ideal for maintaining a list of unique tags, roles, or identifiers.

With upsert: true, if no document matches the filter, MongoDB inserts a new document combining the filter fields and the update fields. If a document matches, it is updated normally. The result object includes an upsertedId when an insert occurs.

$elemMatch ensures that a single array element satisfies all the specified criteria simultaneously. Without $elemMatch, each condition could be satisfied by different elements. It is essential when matching multi-field subdocuments inside arrays.

$match filters documents using the same query syntax as find(). Placing $match early in the pipeline reduces the number of documents passed to subsequent stages, improving performance and allowing MongoDB to use indexes on the filter fields.

$group requires an _id expression that specifies the grouping key. Setting _id to null groups all documents into one. Accumulator expressions like $sum, $avg, $push, and $first compute values across all documents in each group.

$unwind outputs one document for each element of the specified array field. The array field is replaced with the individual element value. This is useful before $group or $lookup when you need to operate on individual array elements.

A compound index is created by passing an object with multiple field-direction pairs to createIndex(). MongoDB traverses the index using the leftmost prefix rule: queries must use fields from left to right for the index to be effective.

A compound index on {a, b, c} supports queries on {a}, {a, b}, and {a, b, c}. Queries on {b} or {c} alone cannot use the index because they skip the leftmost prefix field. Index field order matters significantly for which queries benefit.

A TTL (Time To Live) index is a special single-field index on a Date field. MongoDB's background thread removes documents when the date field value plus the expireAfterSeconds setting is in the past. It is commonly used for sessions, caches, and logs.

A covered query is one where all queried fields and all projected fields are part of the same index. MongoDB can return results from the index alone without fetching actual documents, making it extremely fast. The explain() output shows IXSCAN with no FETCH stage.

COLLSCAN (collection scan) means MongoDB examined every document in the collection to find matches, which is slow at scale. IXSCAN (index scan) means MongoDB used an index to efficiently locate matching documents. IXSCAN is almost always preferable for large collections.

w: 'majority' tells MongoDB to wait until the write is acknowledged by a majority of voting members in the replica set. This provides strong durability guarantees because the write survives even if the primary fails and a new primary is elected.

j: true means MongoDB will not acknowledge the write until it has been written to the on-disk journal. This protects against data loss from a process crash, because the journal survives restarts. Combined with w: 'majority', it provides the strongest durability.

The 'nearest' read preference selects the member with the lowest network round-trip time, regardless of whether it is primary or secondary. It is useful for globally distributed applications where latency matters more than reading the absolute latest data.

Read concern 'majority' returns data that has been acknowledged by a majority of voting replica set members. This prevents reading data that could be rolled back if the primary fails. It provides a consistent view of data that is guaranteed to be durable.

Read concern 'linearizable' guarantees that a read reflects all prior writes that were majority-committed before the read began. It provides the strongest guarantee but has higher latency. It can only be used with primary read preference.

Multi-document transactions require a ClientSession. You start a session, call startTransaction(), pass the session object to every operation in the transaction, then commit or abort. All operations in the session are atomic across documents and collections.

Multi-document ACID transactions require a replica set (introduced in MongoDB 4.0). Standalone instances do not support them. Sharded cluster transactions were added in MongoDB 4.2, building on the replica set transaction foundation.

GridFS splits large files into chunks (default 255 KB each) stored in a 'chunks' collection, with metadata in a 'files' collection. It works around the 16 MB BSON document limit and supports streaming of large files like videos and images.

Capped collections have a fixed maximum size in bytes (and optionally a document count limit). When the collection is full, new inserts overwrite the oldest documents in insertion order. They support tailable cursors, making them useful for log-like workloads.

A tailable cursor works only on capped collections. After exhausting all current documents, it remains open and returns new documents as they are inserted. This is similar to 'tail -f' in Unix and is useful for real-time log or event stream consumption.

Change streams use the oplog to notify applications of inserts, updates, deletes, and schema changes in real time. They require a replica set or sharded cluster. Applications call collection.watch() to open a change stream and iterate over change events.

The oplog (operations log) is a special capped collection (local.oplog.rs) on each replica set member. It records every write operation applied to the primary in an idempotent form. Secondaries tail the primary's oplog to replicate all writes.

An arbiter holds no data and cannot become a primary. Its only role is to vote in elections, providing an odd number of votes to break ties. Arbiters are used to achieve election quorum with fewer hardware resources than a full data-bearing secondary.

The shard key determines how MongoDB distributes documents across shards. Each document's shard key value maps to a chunk range, and each chunk lives on one shard. Choosing a high-cardinality, evenly distributed shard key is critical for balanced clusters.

Ranged sharding assigns contiguous value ranges of the shard key to each shard, which can cause hotspots with monotonically increasing keys. Hashed sharding applies a hash function first, distributing documents more evenly but losing range query locality.

When a query does not include the shard key, mongos must broadcast it to all shards (scatter) and merge the results (gather). This is far less efficient than a targeted query that includes the shard key and routes to exactly one shard.

$project controls which fields appear in the output documents. Setting a field to 1 includes it; setting it to 0 excludes it. You can also add computed fields using expressions. _id is included by default and must be explicitly set to 0 to exclude it.

_id is included by default in every query result. To exclude it, explicitly set _id: 0 in the projection. This is the only case where you can mix inclusion (1) and exclusion (0) in the same projection document.

$mul multiplies the current value of a field by the given number. If the field does not exist, $mul sets it to 0 (for integer/decimal types). It is atomic and useful for applying percentage increases or unit conversions.

$min updates a field to the new value only if that value is lower than the existing field value. If the field does not exist, $min sets it to the specified value. Its counterpart $max updates only when the new value is greater than the current value.

deleteOne() removes exactly one document matching the filter (the first one found by the query plan). deleteMany() removes all documents that match the filter. Both return a result object with a deletedCount property indicating how many documents were removed.

replaceOne() replaces all fields of the matched document with the replacement document (keeping the same _id). updateOne() uses update operators ($set, $inc, etc.) to modify specific fields. Using updateOne() without operators would throw an error.

$rename renames a field by removing the old field and creating a new field with the new name and the same value. It is equivalent to performing $set on the new name and $unset on the old name atomically in a single operation.

$currentDate sets a field to the server's current date as a BSON Date type (by default) or a Timestamp. It is useful for automatically stamping updatedAt fields without relying on the client to provide the correct time.

$regex matches string fields against a regular expression. The $options modifier (e.g., 'i' for case-insensitive, 'm' for multiline) controls matching behaviour. For performance, use an anchored prefix regex (^pattern) which can utilise a string index.

$all matches documents where the array field contains every value in the provided list, regardless of order or additional elements. A document with tags ['sale', 'electronics', 'new'] would match. $all is equivalent to multiple $elemMatch conditions for simple scalar values.

$size matches documents where the array field contains exactly the specified number of elements. It does not support ranges (you cannot use $gt with $size); for range-based array length queries you need to store the count as a separate field and index it.

Embedding places related data inside the same document (denormalised), which is great for data accessed together and has a one-to-few relationship. Referencing stores a reference ID and requires a separate query or $lookup, suitable for large or frequently updated related data.

Referencing is better when the related data grows without bound, is updated independently of the parent, exceeds the 16 MB document limit when embedded, or needs to be shared across many parent documents. Embedding works best for data always accessed together.

BSON documents in MongoDB are limited to 16 MB. This limit prevents individual documents from consuming excessive RAM and network bandwidth. For files or data exceeding this limit, use GridFS which splits content into 255 KB chunks stored in a separate collection.

BSON (Binary JSON) is a binary-encoded format used internally by MongoDB. It extends JSON with additional types like Date, ObjectId, Binary, Decimal128, and Int64. BSON is more efficient to parse and supports richer data types than plain JSON.

Decimal128 is a 128-bit IEEE 754-2008 decimal floating-point type with 34 significant decimal digits and a range of 10^-6143 to 10^6144. It avoids binary floating-point rounding errors, making it the correct choice for currency amounts and precise scientific data.

MongoDB drivers treat undefined values the same as missing fields and omit them when serialising to BSON. The BSON Undefined type exists for legacy reasons but is deprecated. To store an explicit 'no value', use null, which maps to the BSON Null type.

$addFields is like $project but preserves all existing fields. It adds new computed fields or overwrites existing ones. It is useful when you want to enrich documents with derived values without having to explicitly include every existing field in a $project stage.

$replaceRoot makes a specified subdocument the new root of the output document. The original root is discarded. It is commonly used after $unwind or $lookup to lift embedded documents to the top level for further processing.

$facet processes multiple independent aggregation pipelines within the same stage, each receiving the same input documents. The results of all sub-pipelines are returned in a single document. It is ideal for generating multiple aggregations (counts, ranges, categories) in one query.

$bucket groups documents into user-defined ranges based on a field value. You specify boundaries like [0, 10, 50, 100] and each document is placed in the range its value falls into. $bucketAuto automatically determines equal-count boundaries.

$out writes all output documents to a named collection, replacing the collection entirely if it exists. Unlike $merge, $out is all-or-nothing: if the pipeline fails, the existing collection is left unchanged. $out must be the last stage in the pipeline.

$merge (introduced in MongoDB 4.2) writes pipeline results to an existing collection with fine-grained control: you can insert new documents, update existing ones, replace them, or run custom merge pipelines. $out replaces the entire target collection atomically.

A text index tokenises string fields, removes stop words, and stems terms. You query using $text: { $search: 'terms' }. A collection can have only one text index but it can cover multiple fields. Use $meta: 'textScore' to sort by relevance.

MongoDB automatically creates a multikey index when the indexed field contains an array. Each array element gets its own index entry. This allows efficient queries on array contents but means a compound index cannot have more than one array field.

A partial index includes only documents that match a filter expression (e.g., { status: { $eq: 'active' } }). It is smaller and faster than a full index because it excludes irrelevant documents. Queries must include the filter condition to use the index.

MongoDB supports 2dsphere indexes for GeoJSON data and 2d indexes for legacy coordinate pairs. They enable operators like $near, $geoWithin, and $geoIntersects for proximity searches, polygon containment, and geographic shape queries.

mongodump outputs BSON binary files that preserve all BSON types and are best for full backups. mongoexport produces JSON or CSV files, which are human-readable and portable but may lose type fidelity (e.g., Dates become strings). Use mongodump for backups and mongorestore to restore.

The standard MongoDB URI format is mongodb://[user:password@]host[:port][/database][?options]. The mongodb+srv:// scheme is a DNS SRV variant that discovers replica set members automatically. Options like replicaSet, authSource, tls, and readPreference are passed as query parameters.

bulkWrite() accepts an array of insertOne, updateOne, updateMany, deleteOne, deleteMany, and replaceOne operations. With ordered: true (default), it stops on the first error. With ordered: false, it continues all operations and returns a combined error list.

A Mongoose Schema defines field names, types, default values, validators, and middleware hooks. A Model is created from a Schema using mongoose.model('Name', schema) and provides a class with static methods (find, create, updateOne) and instance methods for interacting with the MongoDB collection.

Mongoose virtuals are schema properties that are not persisted to MongoDB. They are computed on the fly from other fields (e.g., a fullName getter that concatenates firstName and lastName). Virtuals can have both get and set functions and can be populated like references.

Mongoose populate() performs a separate query to fetch referenced documents and replaces the ObjectId field with the full document. It is similar to $lookup but works at the ODM level. populate() can be nested and configured to select only specific fields.

Mongoose middleware (hooks) are functions executed at specific points in a document's lifecycle. Pre hooks run before the operation (e.g., hashing a password before save). Post hooks run after the operation completes. They are attached with schema.pre('save', fn) and schema.post('find', fn).

$lookup is a left outer join. When no documents in the 'from' collection match the join condition, the 'as' field is set to an empty array []. The source document is still included in the output. You can filter out non-matches with a subsequent $match on the array being non-empty.

$count outputs a single document with one field (named by the string argument) whose value is the total count of documents at that stage. It is equivalent to $group with $sum:1 but more concise. It is always placed after filtering stages.

$cond is a ternary operator: if the 'if' expression is true, it returns the 'then' value; otherwise it returns the 'else' value. It can be used in $project, $addFields, $group, and other expression contexts to create conditional computed fields.

$first returns the value of a field from the first document processed in each group. Without a preceding $sort, the 'first' document is indeterminate. With $sort: { date: 1 } before $group, $first gives the earliest value per group, which is a common pattern.

Atlas Search is a full-text search engine built on Apache Lucene, available on MongoDB Atlas. It provides relevance scoring, fuzzy matching, autocomplete, synonyms, facets, and complex query operators via the $search aggregation stage. MongoDB's built-in text index is simpler and less feature-rich.

$type matches documents where the given field contains a value of the specified BSON type. You can use the type name (e.g., 'string', 'int', 'date') or the numeric BSON type code. It is useful for finding documents with mixed types stored in the same field.

findOneAndUpdate() returns the original document (before the update) by default. To return the modified document, set the returnDocument option to 'after' (driver API) or returnNewDocument: true (shell). This method atomically finds, updates, and returns a document in one operation.

$ne matches documents where the field value does not equal the specified value. It also matches documents where the field does not exist. This means $ne: null matches documents where the field has a non-null value. For strict not-equal-and-must-exist, combine with $exists: true.

$nin (not in) matches documents where the field value does not match any value in the provided array. Like $ne, it also matches documents where the field is absent. It is the complement of $in and is useful for blacklist-style exclusion queries.

$skip discards the first N documents from the pipeline and forwards the remaining documents to the next stage. Combined with $sort and $limit, it implements cursor-based pagination. For large datasets, using cursor-based pagination with _id comparisons is more efficient than $skip.

With ordered: true (default), insertMany() processes documents sequentially and stops at the first error. Documents inserted before the failure remain committed; documents after are not attempted. With ordered: false, all documents are attempted and all errors are collected.

$dateToString converts a BSON Date value to a formatted string. The format uses strftime-style specifiers like %Y (year), %m (month), %d (day), %H (hour), %M (minute), %S (second). It is commonly used in $project or $group to format dates for output or grouping.

A unique index rejects any insert or update that would result in a duplicate value for the indexed field. Multiple documents with null values are allowed unless combined with a sparse index. Compound unique indexes enforce uniqueness across the combination of indexed fields.

$concat takes an array of string expressions and joins them into one string with no separator. To add separators, include them as literal string elements in the array. If any input is null or missing, the result is null unless you handle it with $ifNull.

In a replica set election, a node must receive votes from a majority of voting members. Nodes with higher priority values are preferred as candidates. Among equal-priority nodes, the one with the most recent oplog entry (closest to the former primary's state) wins the election.

The default read preference is 'primary', meaning all reads go to the primary member. This ensures you always read the most recent data. Using 'secondary' can improve throughput but may return stale data if the secondary lags behind the primary.

$avg calculates the arithmetic mean of all numeric values of the specified expression across documents in the same group. Non-numeric values and missing fields are ignored. It is one of the standard accumulators available in $group, $project, and $addFields.

When used as a $group accumulator, $addToSet collects field values from all documents in each group into an array, including each unique value only once. The output array order is not guaranteed. Use $push instead if you want all values including duplicates.

mongos is the routing layer in a sharded cluster. Applications connect to mongos instead of individual shards. mongos reads chunk distribution metadata from the config servers and routes each operation to the correct shard(s). Multiple mongos instances can run for load balancing and high availability.

For a query to be covered: every field in the query filter must be in the index, every field in the projection must be in the index, and _id must be excluded from the projection (unless _id is in the index). When all conditions are met, MongoDB reads only index data, with no document fetches, resulting in maximum performance.