life dev log

Introduction

In Rails 8, both Active Job and Action Cable use the async adapter by default in the development environment. The async adapter is a simple, single-process, memory-based system that behaves differently from production.

When async processing doesn’t work as expected — for example, the wait option in retry_on being ignored — you need to switch to Solid Queue, the same adapter used in production. However, Solid Queue runs jobs in a separate process, which causes Turbo Broadcasts to stop reaching the browser. This happens because Action Cable’s async adapter can only deliver messages within the same process. To fix this, you also need to switch Action Cable’s adapter to Solid Cable.

This article covers the background of this problem and the setup steps to resolve it.

About the Author

I have been studying programming at fjord bootcamp for 2 years and 1 month. I am currently building an AI chat app as my final individual project.

Development Environment

Rails 8
Docker (PostgreSQL)

How I Ran Into This Problem

I set the wait option on Active Job’s retry_on method, but it did not wait for the specified interval.
I switched Active Job’s adapter from async to Solid Queue. The retry_on timing worked correctly, but Turbo Broadcasts stopped reaching the browser.
I switched Action Cable’s adapter from async to Solid Cable. Turbo Broadcasts started working again.

Background Knowledge

Async Adapter (Active Job)

The default Active Job adapter in Rails 8’s development environment. It runs jobs immediately in an in-memory thread pool. It has the following limitations:

retry_on with wait (delayed execution) does not work correctly
Unfinished jobs are lost when the process stops
Only works within a single process

In production, Solid Queue is used by default.

Reference: Active Job Basics - Rails Guide

Async Adapter (Action Cable)

The default Action Cable adapter in Rails 8’s development environment. It delivers broadcast messages to subscribers within the same process via memory.

Cannot receive broadcasts from other processes
Turbo Broadcasts from Solid Queue jobs do not reach the browser

Note: Active Job’s async and Action Cable’s async are separate implementations that happen to share the same name and the same limitations (single-process, memory-based).

Reference: Action Cable Overview - Rails Guide

Solid Queue

The production queue backend for Active Job. It uses a database instead of memory to manage job scheduling, execution, and retries. Delayed execution and retry strategies work correctly.

By default in Rails 8, Solid Queue runs as a Puma plugin (plugin :solid_queue). This avoids the need to start a separate process, but internally it still runs as a forked, separate process. When job load increases, it can be split into an independent process using bin/jobs.

Reference: Solid Queue - GitHub

Solid Cable

A subscription adapter for Action Cable. It uses a database instead of memory to pass WebSocket messages between processes. Because the database acts as an intermediary, broadcasts from Solid Queue jobs can reach the browser.

Reference: Solid Cable - GitHub

Why Both Changes Are Needed

[Before] async + async
Puma (HTTP + ActionCable + ActiveJob) ← All in one process, broadcasts work

[Solid Queue only]
Puma (HTTP + ActionCable) ←→ SolidQueue (ActiveJob) ← Separate process, broadcasts don't arrive

[Both] Solid Queue + Solid Cable
Puma (HTTP + ActionCable) ←→ DB ←→ SolidQueue (ActiveJob) ← Broadcasts arrive via DB

Setup Steps

1. [Solid Queue] Configure queue_adapter

# config/environments/development.rb

# Switch from async to solid_queue
config.active_job.queue_adapter = :solid_queue

# Connect to the queue database (matches the "queue" key in database.yml)
config.solid_queue.connects_to = { database: { writing: :queue } }

2. [Solid Queue] Configure Puma Plugin

# config/puma.rb

# Start Solid Queue from Puma in development
# In production, this is controlled by the SOLID_QUEUE_IN_PUMA env var (set in Kamal's deploy.yml)
plugin :solid_queue if ENV["SOLID_QUEUE_IN_PUMA"] || Rails.env.development?

3. [Solid Cable] Configure cable.yml

# config/cable.yml

development:
  adapter: solid_cable           # Switch from async to solid_cable
  connects_to:
    database:
      writing: cable             # Matches the "cable" key in database.yml
  polling_interval: 0.1.seconds  # How often to poll the DB for new messages
  message_retention: 1.day       # How long to keep messages in the DB

4. [Solid Queue / Solid Cable] Add Databases to database.yml

# config/database.yml

default: &default
  adapter: postgresql
  encoding: unicode
  username: monalin
  password: password
  host: db  # In Docker, specify the DB service name (required for queue/cable DB connections)
  max_connections: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

development:
  primary:
    <<: *default
    database: monalin_development          # Existing application DB
  queue:
    <<: *default
    database: monalin_development_queue    # Solid Queue DB
    migrations_paths: db/queue_migrate
  cable:
    <<: *default
    database: monalin_development_cable    # Solid Cable DB
    migrations_paths: db/cable_migrate

Note on host: db: The queue and cable databases read directly from database.yml, so the host must be specified in the default section.

5. Create Tables

# Stop the server (to disconnect from the DB)
docker compose down

# Create the queue and cable databases and their tables
docker compose run --rm web bin/rails db:prepare

# Restart the server
docker compose up

Note: db:prepare creates the database and loads the schema if it doesn’t exist, or runs pending migrations if it does. Make sure db/queue_schema.rb and db/cable_schema.rb exist in your project before running this command. These files are generated automatically when you run rails new in Rails 8.

References

Photo by Possessed Photography on Unsplash

Sign in to your account

Setting Up Solid Queue + Solid Cable + Turbo Broadcast in Development