Scout APM is PHP application performance monitoring designed for developers. With tracing logic that ties issues back to the line of code causing them, you can pinpoint n+1 queries, memory leaks, and other abnormalities in real time so you can knock them out and get back to building a great product. Start your free 14-day trial today and get the performance insight you need in less than 4 minutes.

Selling digital products using Laravel part 7: Importing package documentation from GitHub

Original – by Freek Van der Herten – 5 minute read

On our site you'll find the documentation of our bigger bigger packages, like laravel-medialibrary, laravel-backup, laravel-event-sourcing, ... Let's take a look at how that works.

screenshot

The basis of this documentation is a collection of markdown files that is included in the various packages. Here's the docs folder of laravel-medialibrary where those markdowns reside.

The markdown files of all those GitHub repositories are imported using the ImportDocsFromRepositoriesCommand, which is executed every 5 minutes.

Let's take a look at the handle method of that command.

public function handle()
{
    $this->info('Importing docs...');

    $loop = Factory::create();

    $updatedRepositoriesValueStore = UpdatedRepositoriesValueStore::make();

    $updatedRepositoryNames = $updatedRepositoriesValueStore->getNames();

    $this
       ->convertRepositoriesToProcesses($updatedRepositoryNames, $loop)
      ->pipe(fn (Collection $processes) => $this->wrapInPromise($processes));

    $loop->run();

    $updatedRepositoriesValueStore->flush();

    $this->info('All done!');
}

There are several interesting things at play here. The ImportDocsFromRepositoriesCommand command does not import every repository with docs that we have, but only repositories that were recently updated. The names of those updated repositories are in the $updatedRepositoryNames array.

The contents of $updatedRepositoriesValueStore is what we call a value store. A value store is nothing more than a class that wraps a JSON file on disk with some nice write/read methods. If you want to use a value store in your project, take a look at the spatie/valuestore package.

On each of the repositories on GitHub with docs, we configured a webhook to https://spatie.be/webhooks/github. The webhook is executed whenever a file in the repo is changed. In the HandleGitHubWebhookController the incoming webhook is handled. Here's the __invoke` method of that controller. This method will verify that request is a valid request from GitHub. It will determine the repo name where the request originates from, and it will save that repo name in the value store.

public function __invoke(Request $request)
{
    $this->ensureValidRequest($request);

    $payload = json_decode($request->getContent(), true);

    $updatedRepositoryName = $payload['repository']['full_name'] ?? null;

    if ($updatedRepositoryName === null) {
        return;
    }

    UpdatedRepositoriesValueStore::make()->store($updatedRepositoryName);
}

Now that you know how we determine for which repos we should import docs, let's go back to the ImportDocsFromRepositoriesCommand and focus on these lines:

$this
   ->convertRepositoriesToProcesses($updatedRepositoryNames, $loop)
   ->pipe(fn (Collection $processes) => $this->wrapInPromise($processes));

$loop->run();

For each of the updated repositories, we are going to create a process that will copy the docs markdown files of the repo to the storage directory of our Laravel app.

The cool thing to note here is that all those processes will be registered at a loop. When the loop is started, this happens here, all process will execute simultaneously. This will significantly improve the time needed to fetch all docs.

All the processes are wrapped in a promise by the all function. This allows us to execute code when all processes are finished in the then method.

protected function wrapInPromise(Collection $processes): void
{
    all($processes->toArray())
        ->then(function () {
            $this->info('Fetched docs from all repositories.');

            $this->info('Caching Sheets.');

            $pages = app(Sheets::class)->collection('docs')->all()->sortBy('weight');

            cache()->store('docs')->forever('docs', $pages);

            $this->info('Done caching Sheets.');
        })
        ->always(function () {
            File::deleteDirectory(storage_path('docs-temp/'));
        });
}

When the callable in then is executed, all repositories will have been fetched to the storage_path('docs') directory. Our homegrown spatie/sheets package can parse a directory with markdown files to HTML. It will also wrap each fetched document in an Eloquent-like class so you can easily get the title, content, and things defined in the Markdown front matter. To know more about this powerhouse package, read its readme on GitHub.

The sheets package is configured in such a way that all files on the docs disk (which has its root set to storage_path('docs')) are being handled. All docs are sorted by weight (which is defined in the front matter). Sorting the collection of docs is an expensive operation, so we cache the result.

The DocsController is responsible for displaying the docs on our site. It will use those docs we cached previously.

This series is continued in part 8: Mailing updates and news using Mailcoach.

Stay up to date with all things Laravel, PHP, and JavaScript.

Follow me on Twitter. I regularly tweet out programming tips, and what I myself have learned in ongoing projects.

Every two weeks I send out a newsletter containing lots of interesting stuff for the modern PHP developer.

Expect quick tips & tricks, interesting tutorials, opinions and packages. Because I work with Laravel every day there is an emphasis on that framework.

Rest assured that I will only use your email address to send you the newsletter and will not use it for any other purposes.

Comments

Webmentions

Web Developer MF liked on 24th October 2020
Songhua Hu liked on 22nd October 2020
Tauseef shah liked on 22nd October 2020
KGG liked on 22nd October 2020
Steve Bauman liked on 22nd October 2020
Florian Wartner liked on 22nd October 2020
Spatie retweeted on 22nd October 2020
Florian Voutzinos ⚡ liked on 22nd October 2020
José Cage liked on 22nd October 2020
Tom Witkowski liked on 22nd October 2020
Robin Dirksen liked on 13th October 2020
Richard Radermacher liked on 13th October 2020