Unlocking Data Potential: Synergizing EventStoreDB and MongoDB for Optimal Data Management

Introduction

There are many new types of databases available today, and EventStoreDB as a state transition database is one of them. In contrast, MongoDB has been around for a while and is a popular document database.

What is the relationship between EventStoreDB and MongoDB? What are their respective strengths? And do they work well together?

Keep reading to find out more!

What is EventStoreDB?

EventStoreDB is a database built for event sourcing. It is a state transition database that stores historical events of facts that occurred in an application as an event log. Unlike traditional databases, which usually only store the current state and overwrite past data, EventStoreDB retains the past data.

A state transition database consists of two major components:

1. Events are state transitions that represent an important business decision in your application (e.g. OrderPlaced, OrderPaid, OrderDelivered). Because they capture business actions taken, they are more granular and fundamental than state changes that are based on database operations. (e.g. OrderInserted, OrderUpdated, OrderDeleted)
2. The Log is an append-only log that captures all events that occur in your application. Each appended event is assigned a sequential number. This is a global sequence number in the log which is incremented whenever a new event is appended. This allows the log to record the exact order in which the events arrive at the database.

Strengths of EventStoreDB

Where EventStoreDB Shines

EventStoreDB shines as the source of truth of core business data, particularly in complex systems that frequently change and have myriads of business constraints. 

What is MongoDB?

MongoDB is a document database that focuses on flexibility and scalability in data management. Unlike relational databases, MongoDB stores data in schema-less and JSON-like documents, allowing for more intuitive representation and querying of complex data structures.

A document database like MongoDB comprises two major components:

1. Documents: These are the core units of data in MongoDB. Each document is a set of key-value pairs, similar to JSON objects. Documents can contain nested documents and arrays, allowing for the representation of complex hierarchical data structures. This contrasts with the flat, table-like structure of relational databases. Typical operations on documents include Insert, Update, Delete, and Find, which align with the Create, Read, Update, and Delete (CRUD) operations in database management.
2. Collections: MongoDB stores documents in collections, which are comparable to tables in relational databases. However, unlike tables, collections do not enforce a strict schema. This means that documents within the same collection can have different fields. Collections also support various indexing options to optimize query performance.

MongoDB also supports advanced features like aggregation for data analysis, replication for data availability and redundancy, and sharding for horizontal scalability.

Strengths of MongoDB

Where MongoDB Shines

MongoDB shines as a widely distributed storage for data with a simple model that doesn’t have complex transactional requirements but requires high scalability.

How EventStoreDB and MongoDB Complement Each Other

EventStoreDB and MongoDB work extremely well together in practice, where their benefits complement each other.

EventStoreDB is typically used as the source of truth or the system of record. It acts as an authoritative source where data is stored and serves as the only location for updates. Because:

• It records data at a more granular level (via events) and retains it over time, allowing it to store data at a higher definition than any other type of database.
• Its auditability facilitates the distribution of data to downstream systems in an eventually consistent manner.

MongoDB, on the other hand, is perfect as the cache or what we call the read model. A downstream read-only view or projection of the source of truth. Because:

• It has superb capability to horizontally scale reads, making it suitable for distributing data to a large audience.
• Its data structure is flexible, developer-friendly, and offers a wide range of querying functions, making it an ideal cache that is optimized for quick retrieval.

For example, EventStoreDB is suitable for storing credit card transactions, while MongoDB would be perfect for storing account summary JSON tailored to a particular user web page.

How to integrate EventStoreDB & MongoDB

At a high level, an application change is saved as an event in EventStoreDB. The event is then published to a custom projector service, which subscribes to new events and updates the MongoDB replica set.

Here is the process description in detail:

1. The requestor (a user) sends an API request to update the application (e.g. withdraw from account)
2. The API receives the request and makes sure the change satisfies all business constraints (e.g. balance below credit limit)
3. The API generates an event and appends it to EventStoreDB’s event log (e.g. withdrew from account event)
4. EventStoreDB publishes the event using the persistent subscription feature
5. The projector, responsible to update MongoDB, receives the event from the persistent subscription
6. The projector, retrieves the document(s) required, updates it, and sends it back to MongoDB (e.g. account summary document updated)
7. MongoDB’s primary node updates the document on itself
8. MongoDB’s secondary node replicates the update from the primary node
9. Many end users queries results from the MongoDB replica set

Closing

EventStoreDB and MongoDB work well together when EventStoreDB serves as the source of truth and MongoDB is used for projections or as a read model.

This maximizes the fidelity of the data stored in the source of truth, whilst enabling the distribution of that data to optimized forms to a large number of audiences.