Laravel queues that don't fall over: Horizon + Supervisor on a cheap VPS

Every few weeks a junior in my team (or a freelancer in my DMs) shows me a production server where the queue worker is running inside screen. It works until the server reboots, or the worker eats a bad job and dies at 3am, and then the client calls me asking why no invoices went out since Tuesday. Uff.

So here is the setup we actually use. Nothing fancy. A 2 vCPU / 4 GB VPS for about 12 dollars a month runs Redis, Horizon and three small client apps without breaking a sweat.

1. Horizon needs Redis

Horizon only works with the Redis queue driver. Install it, publish the config, point the queue at Redis:

composer require laravel/horizon
php artisan horizon:install

# .env
QUEUE_CONNECTION=redis
REDIS_CLIENT=phpredis

One thing people skip: if Redis is on the same box, check maxmemory-policy. For a queue you want noeviction. With allkeys-lru Redis will happily delete your pending jobs when memory gets tight and nobody gets an error. Silent data loss, my favourite.

2. The Horizon config we start every project with

// config/horizon.php
'environments' => [
    'production' => [
        'supervisor-1' => [
            'connection' => 'redis',
            'queue' => ['high', 'default', 'emails'],
            'balance' => 'auto',
            'autoScalingStrategy' => 'time',
            'minProcesses' => 1,
            'maxProcesses' => 8,
            'balanceMaxShift' => 1,
            'balanceCooldown' => 3,
            'tries' => 3,
            'timeout' => 90,
            'memory' => 128,
        ],
    ],
],

Queue order matters: high is drained before default. We put password resets and OTP emails in high, PDF reports and exports in default. Nobody should wait for a login code because finance is exporting 40,000 rows.

3. The setting that made our jobs run twice

This one cost us an angry client and about 300 duplicate emails. In config/queue.php the Redis connection has retry_after, default 90 seconds. When a worker picks a job, Laravel reserves it for that many seconds. If the job is still running after retry_after, Laravel assumes the worker died and puts the job back on the queue. Another worker picks it up. Now it runs twice, and there is no exception anywhere, so you will not find it in the logs.

// config/queue.php
'redis' => [
    'driver' => 'redis',
    'connection' => 'default',
    'queue' => env('REDIS_QUEUE', 'default'),
    'retry_after' => 180,
    'block_for' => null,
],

The rule from the docs, which I now repeat in every code review: the worker timeout must always be a few seconds shorter than retry_after. Our Horizon timeout is 90, retry_after is 180. If a job genuinely needs 5 minutes, it goes on its own queue with its own supervisor and bigger numbers, or better, it gets split into smaller jobs.

4. Supervisor keeps Horizon alive

; /etc/supervisor/conf.d/horizon.conf
[program:horizon]
process_name=%(program_name)s
command=php /var/www/app/current/artisan horizon
autostart=true
autorestart=true
user=deploy
redirect_stderr=true
stdout_logfile=/var/www/app/shared/storage/logs/horizon.log
stopwaitsecs=3600

sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start horizon

stopwaitsecs should be longer than your longest job, otherwise Supervisor kills a worker in the middle of a job during a restart.

5. Deploys: terminate, don't forget

Workers are long running processes. They load your code once and keep it in memory. If you deploy and do not restart them, they will keep running yesterday's code, and you will spend an evening wondering why your fix "is not working". Last line of every deploy script:

php artisan horizon:terminate

Horizon finishes the current jobs, exits, Supervisor starts it again with the new code. Also schedule the snapshot so the metrics graphs in the dashboard actually have data (Laravel 11, in routes/console.php):

Schedule::command('horizon:snapshot')->everyFiveMinutes();

6. Lock the dashboard

In production, /horizon is protected by the viewHorizon gate in HorizonServiceProvider. Put real emails or a role check there. Do NOT "fix" the 403 by making the gate return true. I have seen job payloads with customer phone numbers sitting on an open Horizon page. Yaar.

What it costs

For reference, one of our client boxes: 2 vCPU, 4 GB, Redis + Horizon + 3 apps, about 40k jobs a day, load average under 0.6. Failed jobs we check every morning with php artisan queue:failed before chai. That is the whole system. Boring, cheap, and it has not paged me since January.

If you are still on queue:work inside screen, do one thing this weekend and move it to Supervisor. Thank me later 🐘