How we created over 200 PHP and Laravel packages

Original – May 27th 2020 by Freek Van der Herten – 15 minute read

I work at and co-own a company named Spatie, which specializes in creating large Laravel applications for our clients. Our team is rather small: we consist of only 10 people. At first glance we are a software development company like there are so many others. But there’s one thing that sets our company apart: we have an open source first policy. We try to create and contribute to open source as much as possible.

Currently we have around 200 open source repositories on GitHub. Our packages have been downloaded nearly 75 million times. They are being downloaded 4,5 million times each month. Probably we will hit 100 million downloads by the end of this year.

We've also just released our premium video course, titled Laravel Package Training. In this course, we use our experience to teach you how to build reliable and maintainable packages.

This all did not happen overnight. Read on to learn the story behind our open source efforts.

The first package

When I got started to work for Spatie, somewhere around 2007, I dared to write my own framework. The projects I was working on at that time were very small. Submitting a simple form was the most advanced thing these sites could do.

After a few years our projects got a bit more advanced. I realised that writing and maintaining my own framework wasn’t going to cut it anymore. I heard about this fancy thing called Zend Framework 1 and started using that.

Let’s skip ahead a few more years. In 2012 and 2013 I felt that the PHP ecosystem was pretty stale and I considered switching to the Ruby on Rails framework. But then I bumped into Laravel 4.0. I was immediately impressed by the expressive syntax and the big focus on developer happiness. I used it for a couple of projects and the more I learned about it, the more I loved using it.

At that time a guy called Jeffrey Way started his next project Laracasts: a video tutorial site dedicated to Laravel. Realising that his videos could speed up my learning process immensely I immediately bought a lifetime subscription.

On the 4th of March in 2014 Jeffrey published a video titled “Continuous Integration With Travis“. My mind was blown. Travis and the integration with GitHub looked so cool that I wanted to use it. A couple of months before he made the Travis video Jeffrey made a miniseries on package development. Those videos sparked the thought in my mind that I could create a package of my own.

In a project at the time we needed to automatically create screenshots of a website. With the things I learned in the Laracasts-videos I started working on my first package Browsershot. In essence it was just a simple wrapper around PhantomJS (the current version uses Headless Chrome and Puppeteer).

It was pretty exciting working in the open. I was thrilled every time the download counter got up and was incredibly happy that other people started writing about it.

More packages!

In the Zend Framework days, I used a custom-built application template, called Blender, to kickstart all client projects. It was a sort of mini-CMS. Besides the traditional CMS-functions Blender could do a lot of cool things: pull in data from Analytics to show graphs about the usage of the site, handle uploaded files, subscribe users to a MailChimp list, …

To power new projects I needed a Laravel powered version of Blender and started creating it from scratch. I soon realised that some of the things I was porting to Laravel could be useful for other developers as well and decided to export these functionalities to external packages.

In quick succession packages like laravel-newsletter, laravel-medialibrary and laravel-analytics were made.

I started to enjoy package development more and more. The positive feedback that I got early on from package users was very encouraging. I always considered coding purely as work, but because it was so much fun creating code that others could use, I started coding in my free time as well.

Besides working on packages I also polished the aforementioned application template called Blender and open sourced that too.

Currently every new package that we create gets born inside a client project. In almost every project we find some functionality that can be extracted to its own package.

The benefits of creating packages

There are many benefits of creating packages that make time working on open source code well spent.

A lot can be learned while creating a package. Each package needs to be carefully crafted. It provides a clear and understandable syntax. Like Laravel, I want that all our packages put a great emphasis on developer happiness. They should be easy to use. They should have great documentation. They should have clear tests. Thinking about how other people will use your code will make you a better developer.

The issues reported and the PRs submitted by the users of our packages provided another chance of learning. People can point to mistakes that are in our packages and can propose interesting new features that we didn’t think about.

Sometimes, as a package gains some traction, you get quality code for free. Our laravel-fractal package is a developer friendly wrapper around The League’s Fractal package. I coded up the basic functionality I needed myself and tagged that 1.0.0. In the next few weeks I almost daily got a pull request adding another great feature to the package. And now it supports almost everything League’s Fractal can do. I think 90% of the code of that package was written by the community.

Of course we’re also dogfooding ourselves. Our own packages get used on most projects. If we discover a bug in a package used in a project we can very quickly fix that and distribute that fix through the power of Composer to our other projects.

There are also commercial benefits. The past years we’ve landed some very cool projects because of our open source work. In most cases these new clients were already using our stuff. Even though attracting clients isn’t the primary goal of our open source efforts, it sure is a nice side effect. Getting known in the world allowed us to move from smallish multi-week web projects to building massive multi-year Laravel apps. (Don't listen to people that say Laravel can't scale, they are wrong 🙂)

The positive feedback from package users also gave me enough confidence to start a blog and start speaking publicly. The past years I’ve spoken at several local users groups and a lot of conferences world wide.

I had the pleasure of giving talks in Washington, Amsterdam, Tokyo, New Delhi, New York, Istanbul and of course my own country, Belgium. At all those occasions I got to meet a lot of awesome fellow developers. That all probably wouldn’t have happened if I stayed in my little bubble. Working on open source helped getting to connect to the community. That sounds a bit corny, but that doesn’t make it less true.

With the current ongoing pandemic, public speaking has moved to online talks. And while that is fun too, I can't wait to start travelling again when it's safe to do so, and meet old friends and make new ones.

Making time to work on open source

People often ask if it isn’t very time-consuming to create packages. The truth is that it indeed takes a lot of time. Creating the code for the package itself, writing tests, writing documentation and getting the word out all takes a fair amount of time.

And that’s just the start. When a first stable release gets tagged the work is not over. Maintaining a package, responding to issues and reviewing PR’s takes a lot of time (and dedication) too. At the time of this writing we’ve responded to over 9 000 issues and reviewed and merged a lot of PRs.

We do long-term planning at Spatie, but we also have a weekly short-term planning meeting. When scheduling out the coming week we only plan four days. So we have one work day we can be a bit flexible with. Do not imagine that day as like a fixed day, that time is mostly spread out in that week.

In that “fifth” day everybody in our company is allowed to work on open source stuff or their own projects (which, if possible, will be open sourced as well). For our employees this has a nice benefit that they can keep things interesting by switching between working on client projects and open source stuff.

Personally, I do enjoy working on packages in my free time as well. So a few times a week I spend some time in the evening to continue improving the released packages and create new stuff.

Top 10 of most popular packages

When taking the amount of downloads into consideration these are our 10 most popular packages:

Some of these packages, like image-optimizer, probably get a big download boost because there are required by other packages.

All of the packages above have been downloaded over a million times.

Noteworthy packages

laravel-query-builder can build Eloquent queries from API requests. We coded this up in several separate projects before we decided to package it up. This package now gets used in almost every project at Spatie. My colleague Alex is the primary author and is doing an excellent job maintaining the package. Laravel-query-builder also got featured in Matt Stauffer's talk on exploring the JSON:API at Laracon Online 2020.

laravel-event-sourcing is probably the best entry point for event sourcing in the Laravel ecosystem. It started out small with only support for projections, but has meanwhile grown to a fully featured package, with aggregates, testing helpers, and a lot of DX niceties. This package also has the most extensive docs as I had to explain what event sourcing is for the ground up. Together with my colleague Brent, I'm working on a big project where this package will be the backbone of the entire app. Fun times!

phpunit-snapshot-assertions is a very nice testing package created by my colleague Seb. Snapshot testing is a way to test without writing a specific assertion yourself. To learn more about it, watch this video I made on the package on Laracasts.

laravel-view-models is a package made by my colleague Brent. Have you ever made a controller where you had to do a lot of work to prepare variables to be passed to a view? You can move that kind of work to a so-called view model. In essence, view models are simple classes that take some data, and transform it into something usable for the view. You can see this package in action, in the talk I did at Laracon US 2019

blade-x offers an easy way to create custom html tags in Blade. Meanwhile we've abandoned the package, because the functionality is now available natively in Laravel. I have very fond memories on how we created this package. During lunch, this idea was launched (and honestly I forgot who specifically), but everybody was very enthusiastic about it. Our project managers were not in the office for the day, so we decided to stop doing client work for the rest of the day and all worked together in the meeting room to make this package a reality. No clients were harmed in the process.

How can you show appreciation for our efforts

Like mentioned above, we spend a lot of time creating and maintaining open source software. The easiest way to support us financially is by buying or subscribing to one of our paid products. We tried to put as much love into these as in our open source work, and we hope it shows.

Currently we offer these three paid products:

Laravel Package Training: a video course where we learn you to build packages as good as the ones we release ourselves. The method to build packages shown there is the result of many years of experience with package building.
Mailcoach: a self-hosted email marketing platform that integrates with services like Amazon SES, Mailgun, Postmark or Sendgrid to send out bulk mailings affordably. I'm using this myself to send out my own newsletter.
Flare: an exception tracker specifically built for Laravel apps (we made this together with Beyondcode).

You can also opt to become our sponsor on GitHub. People who sponsor Spatie get access to the video section on our company website. From Laravel best practices to things that keep the team busy, these video series will give you a great insight into how we work and how you can improve your web development skills. Sponsors also get early access to packages we are creating.

A final way to support us is by sending us a postcard. All our packages have a special license called Postcardware. If any of our stuff makes it into your production environment, we’d highly appreciate you sending us a postcard. This is our address:

Spatie
Kruikstraat 22, Box 12 2018 Antwerp
Belgium

Every week the postman delivers some postcards to us. For my team and I it’s nice to know that our stuff is being used all over the globe. We’ve gotten postcards from all continents. We even got one from Vanuatu which, for us, is literally on the other side of the globe. You can take a look at our postcard collection as well. They all are published on our website.

The future

We don’t have a fixed roadmap for new packages. It depends a bit on the client projects we’re handling. If there is something there that can be solved in a generic way, we’ll create a package.

There’s been a time when I thought that, because we’d already solved a lot of common problems, the package output of our team was going to diminish a bit. But it’s quite the opposite. Almost every member of our team has some good package ideas.

We currently have about four new packages in the pipeline, which we hope to share some more info on soon.

In closing

Creating and working on open source packages continues to be a truly fun experience. On a technical level I’ve learned lots of things that I wouldn’t have learned when working solely on client projects. Like you’ve probably read between the lines, it also brings me great joy that my co-workers enjoy and appreciate the time given to work on our open source body of work.

I hope you’ve enjoyed the story behind our open source efforts. The big conclusion is that there are only winners in this story. If you are in a position in your company to advocate or to actively work on open source, I highly recommend you do so.

If you haven’t used our packages before, take a look at the list on our company website. Probably we’ve made some things that could be useful to you. If you like our efforts, consider supporting us.

Freek.dev