Articles

Convince your CTO to use Event Sourcing

Diego Martin  |  26 May 2020

As software engineers, we have probably encountered situations where we needed to convince somebody to go with a particular technology or approach over some other. On some occasions, especially among developers, the sole idea of suggesting a more modern option would be dangerously sufficient to win over the team because, well.., we tend to like shiny things way too much! Luckily, the closer to management we involve in the discussions, the more solid our arguments and the more business-oriented our reasoning will be.

This article isn’t about the dangers of not choosing the right tool for the job independently of how shiny this tool is, nor is it about persuading or selling techniques. Instead, I summarize some of the advantages of Event Sourcing over what I refer to as traditional systems and how most businesses could benefit from it when applied to their software, which is something any CTO would surely want to know should this conversation arise.

But first, let’s begin by reviewing the basics.

What is Event Sourcing?

Event Sourcing is a simple yet powerful concept that allows the state of an application to be represented as a sequence of events instead of as a snapshot of the current state. In other words, the current state can be inferred at any time from a list of the ordered facts that represent each one of the changes that have affected the system. Check out our beginner's guide here to learn more.

The idea is far from new, and we can find examples of it in many environments, a bank account ledger one of the most self-descriptive ones. Our current account balance could be represented as a number and a currency (e.g.: 10 EUR) or as the result of the sum of all the transactions that have happened in that account from the very beginning (e.g.: 0 EUR + 20 EUR + (-15 EUR) + 5 EUR).

Example of a bank account ledger

The result for our specific needs would be in both cases the same, our account balance, but the second approach has some advantages as you may already suspect.

Audit trail for free

Not only are we achieving the same result by applying each one of the events that happened, but we are also documenting every single step that had an effect on the state with what this entails in terms of useful information for developers and the business. With Event Sourcing we are able to know how the system achieved a certain state in addition to what state that is.

The events are stored in an event store or event log in an ordered manner as they happened and logically grouped by streams associated to what generated them. In order to better explore and correlate these events, they could be read again to extract its information and project it in whichever shape fits the needs.

Audit trail of events

Most of the traditional systems end up including some kind of tracing feature to help developers troubleshoot issues when they arise because it’s crucial to know what the system was doing exactly before the problem was revealed in order to reproduce the same scenario and find a solution. This is something that Event Sourcing gives us for free as every change is recorded, becoming the only source of truth.

Imagine a traditional system with an undesirable situation where an application crashes, but we are unable to reproduce what happened just before because a developer forgot to add some traces. Now imagine an even trickier scenario where a traditional system has traces everywhere, giving a false sense of security, but for whatever reason, these logs have incorrect data that doesn’t match what happened. Would we trust the logs? Would we trust the current state? What if somebody edited the state without going through the software’s business logic validation?

Those situations are not so uncommon and are painful to deal with, so having our source of truth as an out of the box trustworthy event log composed of immutable and append-only facts that cannot be changed once they have happened can make our lives better.

Easier testing

Automated testing is a must nowadays. It provides security to developers when refactoring, maintaining, or adding more functionality in the code, it contributes to making a more reliable software overall, and it serves as reliable and up to date documentation about the current system’s features.

In order to test functionalities, we go through a given-when-then approach where we prepare the scenario, execute the functionality and make assertions on the result to ensure it does what it’s meant to, and it doesn’t do what it’s not supposed to.

When using Event Sourcing, these scenarios become easier to prepare and more readable because the system’s preconditions can be built out of a set of events in a self-descriptive way. Remember, every possible mutation within the system is represented with a fact.

Testing workflow

Furthermore, replaying all the events up until a specific point in time would present us with the same system’s state as it was at that moment, allowing us to essentially travel in time.

Zero data loss

Being able to better troubleshoot and test is certainly useful for technical people and provides current value to the delivery teams, but Event Sourcing is also capable of generating future value to business.

Screen grab of South Park episode

There is a funny South Park episode where some gnomes show up at night to collect underwear. When asked about their plans, they reveal an interesting business model in which the first step is to accumulate the items, and the third and last step is to obtain the profit. The hilarious part, source of multiple memes, is that they don’t describe the second step, it is unknown how they turn the collection into profit.

Business decisions are generally better when they’re based on data and, because of that, we value data as one of the most important assets a company has even if, like in the case of the gnomes, we don’t yet know how we could turn this data into business value. And that’s fine as long as we can analyze this data when required.

Who would dare to say that some transaction or information is irrelevant and dispensable? We may not think of a current use for certain data but that doesn’t mean this data won’t be valuable later on. Event Sourcing can be of great worth to businesses due to its zero data loss nature.

Some businesses admit that information is one of the most important things to be collected and secured but have implemented solutions where mutability can mean a loss of data. With traditional CRUD systems, every time there is a write operation to update or delete a record there is a loss of data unless the previous state before the mutation can be restored. But, as mentioned earlier, if a system’s state cannot be rebuilt out of its logs, there wouldn’t be any guarantee on what really happened.

Reports as if they existed before

Although closely related to the previous zero data loss subject, the reporting potential of event-sourced systems deserves its own section. This is, in fact, one of the most appreciated things about Event Sourcing that makes management excited.

Let’s see it with a realistic example.

Imagine you are a software engineer working for an online clothing store where customers can browse articles, choose sizes and colors, add accessories, etc. When users are happy with the contents of their shopping basket, they can proceed to checkout and enter their payment details. Quite a familiar scenario, isn’t it?

There is a meeting to discuss better ways of analyzing customers’ behavior in order to improve cross-selling and offers, a member of the marketing department suggests that a customer that decides to remove some item from the basket just before checking out maybe doing so due to having second thoughts, but it’s still a potential customer for these articles. Maybe this buyer has other items and thinks it’s already spending too much, or perhaps it had already bought the same t-shirt in the past in a different color and was tempted to buy it again with a slightly different design.

In any case, the meeting attendees think that the idea has a lot of potential and decide to start tracking at their website this situation and record details for every item that is removed from a shopping basket less than a minute before checking out. They are going to create a report that is fed with these details once this item removed tracking data feature is released into production and determine which users should they be targeting for some tempting offers regarding the items they could have bought.

After a few sprints, the day has arrived when the new shopping basket that captures this behavior is finally in production, and the managers cannot wait to see data displayed on the report. But the report shows no information just yet because there hasn’t been any checkout since the new release. They have to wait a little longer to see the first data.

Wouldn’t you like to go back in time to that meeting and tell your colleagues that thanks to Event Sourcing you could create this report as a new read model, replay all the events from the beginning of time to create the new projection and, by the time the replay is finished, view all the data in the report without waiting for new purchases as if the report had always been there from day one?

Generating reports from an Event Store

Furthermore, wouldn’t you like to have used Event Sourcing from the beginning and have all this potential available? Well, then it’s time to convince your CTO now so that your software can also travel in time!


Photo of Diego Martin

Diego Martin Diego is a software developer based in Málaga, Spain. He is passionate about Domain Driven Design and distributed architectures. After spending 3 years in Ireland and 7 years in New Zealand, he returned to Spain where he continues delivering software mostly with .NET and Javascript technologies and where he has founded Sunny Attic Software. He is also a father, a husband, an eager traveller and a not so good guitar player.

diego.martin@sunnyatticsoftware.com