Oh Dear! monitors your entire website, not just the homepage. You'll get a notification as soon as your website is down, a monthly uptime report, a warning a few days before your SSL certificate expires and much more! Start your free 10 day trial now!

Event store optimizations in laravel-event-sourcing

Original – by Freek Van der Herten – 5 minute read

About a year ago, we released laravel-event-projector. It focused on adding projectors, an important concept in event sourcing, to Laravel.

After the release of the package, we continually kept improving it. We added aggregates, a way to test those, a brand new section in the our documentation that explains event sourcing from scratch, and DX improvements all across the board.

We now feel confident that the package is a good starting point for getting started with event sourcing in Laravel. That's why we're renaming the package to laravel-event-sourcing.

The package also includes some memory optimizations, which were added by my colleague Rias. In this blog post, I'd like to walk you through the changes.

Using your own event store

In previous versions laravel-event-projector, the storage of events was tied to Eloquent. We had a StoredEvent model that handled the (de-)serialization of an event.

Before an aggregate can be used, it needs to be reconstituted from previous events. If you have a very active aggregate, that needs to be rebuilt using a great many events, the time serializing and deserializing StoredEvent models will add up.

That's why we added an interface that allows you to use any storage you want. This would allow you to create an event store in which manually build queries (for example with DB:: statements, without using Eloquent).

namespace Spatie\EventSourcing;

use Illuminate\Support\LazyCollection;

interface StoredEventRepository
{
    public function retrieveAll(string $uuid = null): LazyCollection;

    public function retrieveAllStartingFrom(int $startingFrom, string $uuid = null): LazyCollection;

    public function persist(ShouldBeStored $event, string $uuid = null): StoredEvent;

    public function persistMany(array $events, string $uuid = null): LazyCollection;

    public function update(StoredEvent $storedEvent): StoredEvent;
}

If you need an example, you can take a look at how we implemented EloquentStoredEventRepository, which is the repository that will be used by default.

After creating your implementation, don't forget to specify the class name of your repository in the stored_event_repository key of the config file.

Memory optimizations

You might have noticed the usage of LazyCollection in the StoredEventRepository. This is a feature introduced in Laravel 6. Under the hood, it leverages PHP's generators to work with large collections, while still keeping memory use low.

Let's take a look at the implementation of retrieveAll in EloquentStoredEventRepository

public function retrieveAll(string $uuid = null): LazyCollection
{
    /** @var \Illuminate\Database\Query\Builder $query */
    $query = $this->storedEventModel::query();

    if ($uuid) {
        $query->uuid($uuid);
    }

    return $query->orderBy('id')->cursor()->map(function (EloquentStoredEvent $storedEvent) {
        return $storedEvent->toStoredEvent();
    });
}

Notice the usage of cursor() here. This will significantly reduce memory usage because Laravel will, at any given moment, only hold one model in memory. Under the hood, this uses MySQL cursors. As I understand it, instead of having the results in memory of the laravel app, MySQL will hold a result set and send the results one by one to the Laravel app.

Rias performed some benchmarks using his Macbook Pro (processor: 2.3 Ghz Intel Core i5). These are the numbers for calling retrieveAll with 100 000 events to be retrieved.

laravel-event-projector with Collection:

  • peak memory usage: 267MB of memory
  • execution time: 7.0077209472656 seconds

laravel-event-sourcing with LazyCollection:

  • peak memory usage: 18MB of memory
  • execution time: 7.478010892868 seconds

Obviously, peak memory usage is much lower/better using LazyCollection at the expense of a slightly worse execution time.

Upgrading from laravel-event-projector the package

If you were on v3 of the laravel-event-projector, upgrading to laravel-event-sourcing is easy. You have to perform these steps:

  1. Change "laravel-event-projector":"^3.0" to "laravel-event-sourcing":"^v1.0" and run composer update
  2. The namespace has changed, so you need to replace Spatie\EventProjector by Spatie\EventSourcing in your entire project.

If you're still on v1 or v2 of laravel-event-projector then you need to upgrade to v3 first like described in our upgrade guide.

In closing

I'm proud of how our package grew from a simple way to handle projectors to the easiest way to get started with event sourcing in Laravel. I did not do this by myself. My colleagues Seb & Rias did a lot of polishing, I got some good advice from my buddy Dries. Frank De Jonge's EventSauce package inspired some of the features.

laravel-event-sourcing isn't the first package that was built by Spatie. Take a look at this big list of Laravel and PHP stuff we've opensourced.

Stay up to date with all things Laravel, PHP, and JavaScript.

Follow me on Twitter. I regularly tweet out programming tips, and what I myself have learned in ongoing projects.

Every two weeks I send out a newsletter containing lots of interesting stuff for the modern PHP developer.

Expect quick tips & tricks, interesting tutorials, opinions and packages. Because I work with Laravel every day there is an emphasis on that framework.

Rest assured that I will only use your email address to send you the newsletter and will not use it for any other purposes.

Comments

You can comment on this post by replying to this tweet.
silviu liked on 24th September 2019
Rias Van der Veken retweeted on 24th September 2019
Spatie retweeted on 23rd September 2019
David Cottila retweeted on 23rd September 2019
Morten Poul Jensen liked on 23rd September 2019
Mozammil liked on 23rd September 2019
Андреј Ранковски liked on 23rd September 2019
Axel Pardemann retweeted on 23rd September 2019
Axel Pardemann liked on 23rd September 2019
Robin Malfait liked on 23rd September 2019
Matthew Poulter liked on 23rd September 2019
James liked on 23rd September 2019
Joshua Weaver liked on 23rd September 2019
oluwajubelo loves VueJS 🚨 retweeted on 23rd September 2019
bernard kssy retweeted on 23rd September 2019
Bojan Angjelkoski liked on 23rd September 2019
Neo liked on 23rd September 2019
Cristian N liked on 23rd September 2019
Bojan Angjelkoski retweeted on 23rd September 2019
Bojan Angjelkoski replied on 23rd September 2019
Awesome work. Thank you!
Jimmy Lipham liked on 23rd September 2019
Kimleang liked on 23rd September 2019
Dries Vints retweeted on 23rd September 2019
Dries Vints liked on 23rd September 2019
PHP Synopsis retweeted on 23rd September 2019