Many are using heroku, and I guess with a similar config to our initial one:

web: vendor/bin/heroku-php-nginx -C nginx_app.conf web/
worker: bin/console queue:worker

This seems right but something was missing

• custom ini was not applied
• the worker could crash and would not respawn at all (this was bad!)

to get our ini to work we passed it directly to the php command, next was to have a simple bash loop to always have a worker running.

worker: while true; do php --php-ini web/.user.ini bin/console queue:worker; sleep 1; done

There must be a better way

It’s almost painful to look at that worker configuration… There must be a way to respawn. Sadly I did not find any Bundles / finished solutions.

Note: I did not add the php –php-ini web/.user.ini  to all examples (but it should be there!)

The Idea

A command should behave kind of like a ThreadPool and restart it’s children once they quit. And in the end I want to call it like this:

worker: bin/console thread:pool --threads=4 queue:worker

The Solution

to make our process handling easier use symfony/process. Next just create a new Console command to handle several processes and keep track of them.

composer require symfony/process

And a very basic Pool:

$threads = 4;
$command = 'bin/console something';

while (true) {
  for ($i = 0; $i < $threads; $i ++) {
    if ($pool[$i] instanceof \Symfony\Component\Process\Process 
        && ($pool[$i]->isRunning() || !$pool[$i]->isStarted())
    ) {
      continue;
    }

    // here we start the process
    $pool[$i] = new \Symfony\Component\Process\Process($command);
    $pool[$i]->start();
  }

  usleep(500000); // this is important, you want to sleep some time
}

This makes it possible to have a single command trigger the amount of workers we need.

You can see the complete command here: gist.github.com/wodka/23475d36cf13e956b8db7578bf6251ed

Logging

this is something I missed initially. Heroku collects logs that are written to stdout – therefore we have to get the output of the processes written to our main process.

Thankfully the Process::start takes a callable with 2 arguments, $type and $buffer.