Vispera learns more from photographs with EventStoreDB

Background

Vispera was founded in 2014 and is focused on helping large grocers execute excellent displays in their stores and franchises. They use image processing and machine learning to make better processes for their clients to monitor stock levels, staff productivity and analyze business performance. Their clients include multinational names such as Unilever, Coca-Cola, Carlsberg, Danone, and many more.

The Challenge

In the early part of the 2010s, grocery suppliers had a cumbersome process for analyzing displays in retailers and stock levels. It was a manual process that required staff to be sent to stores directly to count by hand and assess the stock themselves, then return to their office to enter the data. The founders of Vispera saw this process and knew they could do better. In 2014, Vispera was founded to create a way to analyze stock levels for large grocery retailers. 

Inspired by academia, they set out to modernize the stock analysis method. Vispera solutions can track various retail products' distribution, availability, and shelf space from a single photo for CPG companies and grocery retailers. 

Kaan Can Fidan is the Chief Architect at Vispera and has been there since the company had only 10 employees. The earliest versions of their software were built with C++, and described as “quite low level and hard to maintain”. Kaan Can Fidan discovered that the first proof of concept would not serve the company forever. “We realized we would have to rewrite the system, it would not scale well”. To make a product that would scale with their company and the demands for their product, they would need a more robust technology. “I was in charge of that rewrite, and I had to choose some standard technologies, not invent them myself”.

Implementation

“We started off where everyone starts off, with a relational database in the back with… an HTTP server… on the front” This worked for Vispera for 2 years whilst their business demands were small. “Once the business requirements started accumulating, it was obvious that we would have to change that design”.

Putting his academic background to good use, Kaan Can Fidan began researching solutions that could fulfill the brief and be capable of scaling as the company needed. The solution would have to be able to receive the pictures from a smart device, then pass them through a pipeline that would extract and process the necessary information using technologies that include Object Detection, Recognition, and Optical Character Recognition (OCR), as well as metadata from the image: time, date, location, etc. When building this system, Kaan and the team realized that they would need the same data stored in different databases for different purposes. When he was researching this issue, Kaan came across Event-Driven Architecture, and was drawn to it based on articles by Martin Fowler and videos by Greg Young. Kaan wanted an architecture that would provide an iron-clad paper trail, with audit and transactional data together. Event-Driven Architecture met all of the requirements.

“The major difference between Kafka and Event Store is the ‘one stream per aggregate’. If you can go back and look at a singular instance or event, [EventStoreDB] much more powerful”.
Kaan Can Fidan, Chief Architect, Vispera

They explored several options for Event Driven Architecture, including Apache Kafka. The team found that Kafka was not going to be able to do everything they wanted: “The major difference between Kafka and Event Store is the ‘one stream per aggregate’. If you can go back and look at a singular instance or event, [EventStoreDB] much more powerful”. Apache Kafka would also require the system to take regular snapshots, which would add extra maintenance burden. Vispera also required concurrency guarantees; so that it would be possible to know all areas of the system are up to date. Optimistic concurrency was important to the Vispera team. “Achieving [this] with Kafka was more challenging than achieving it with Event Store”. 

Read this blog to learn about the difference between EventStoreDB and Kafka.

Kaan researched Event Sourcing and Event-Driven Architecture, learning from online talks by Greg Young and Martin Fowler. This gave him the confidence to create a proof of concept for the system Vispera required. “When we saw that the proof of concept was working really well, we just went on with it; we never did Production on Kafka, we just switched to using Event Store. …Everything we wanted to do was harder with Kafka... It might be possible, but it’s easier with Event Store”. 

Kaan accepts that there may be a use case for Kafka in the future, but he hopes Event Store will scale with Vispera to meet all their needs. 

Away with the Clouds

It was vital to Vispera to have their system available at all times, so they needed it to be hosted on a Cloud platform. At the time they were moving their system to Production, Event Store Cloud was not available on Azure yet. “We wanted to use Event Store Cloud right away when we were moving to Production, we didn’t want to deploy it ourselves. We didn’t want to manage it or take backups as we didn’t have many people to look after those”. They hosted their system on another cloud service, but this required extra staff for monitoring. “We said we would deploy Event Store on Azure ourselves until Event Store Cloud was ready”. When Event Store Cloud came along, Vispera were delighted to move over as it gave them peace of mind that someone else at Event Store would manage the availability of the product. Meaning they could just focus on making their software better.

Event Driven Horizon

“When we add new functionality to the system we can just… run it from the beginning of time and generate new metrics on past events… we can pretend that the new service was running from the beginning of time!”
Kaan Can Fidan, Chief Architect, Vispera

Although Kaan had to spend some time explaining the benefits of Event Driven Architecture and Event Sourcing to the rest of the company, he was confident that in the event of a catastrophe with their data, they would be able to restore the system without the need for backups.

“Without deleting or updating any existing data we can just copy events from one Event Store to another one and filter out any corrupted data…We [could] save our data without having a backup…We [wouldn’t even have to take our Production [service] down!”

The Future

Ultimately, Event-Driven Architecture and Event Store Cloud have helped Vispera create a scalable, safe system that can grow with the company in the future. Even without external backups, their system is safe and stable. “By switching to event-driven architecture, you can split your domain into bounded contexts, making work assignments much easier.” 

Event-Driven Architecture powered by Event Store has also allowed Kaan and his team to gain new insights from old data: “When we add new functionality to the system we can just… run it from the beginning of time and generate new metrics on past events… we can pretend that the new service was running from the beginning of time!”

“Event Driven Architecture… has more value than people realize”.

Kaan would suggest to anyone thinking of using EventStoreDB to use Event Store Cloud. “It gave us ease of mind… “if you don’t have on-site Site Reliability Engineers or developers, just deploy Event Store Cloud!”