Sidekiq: Background Job Processing in Ruby

Sidekiq is a Ruby library for processing background jobs, helping you move time-intensive tasks out of your web requests. It excels at handling work that would otherwise slow down your application, including image processing, external API calls, sending emails, and generating reports.

Unlike other job processing libraries, Sidekiq uses threads instead of processes, allowing it to handle thousands of jobs concurrently with minimal memory overhead. It relies on Redis for job storage and coordination, providing durability that survives application restarts. Companies like GitHub, Shopify, and Stripe trust Sidekiq to process millions of jobs daily in production environments.

This tutorial walks you through setting up Sidekiq, configuring workers, and implementing reliable background processing in a Ruby application.

Let's dive in!

Prerequisites

You'll need Ruby 3.0 or later installed on your system. Check the official Ruby installation guide if you haven't set it up yet. Sidekiq depends on Redis for storing jobs, so you'll need Redis running locally. The Redis installation documentation covers the setup process for different operating systems.

After installing these dependencies, verify Redis is operational:

redis-cli ping

A successful connection returns:

PONG

If you see Could not connect to Redis, start the Redis server:

redis-server

Step 1 — Setting up the project

With Redis confirmed working, create a new directory for your Sidekiq project:

mkdir sidekiq-tutorial && cd sidekiq-tutorial

Initialize a new Ruby project with Bundler:

bundle init

This creates a Gemfile where you'll specify your dependencies. Add Sidekiq and the Redis client to the Gemfile:

source 'https://rubygems.org'

gem 'sidekiq', '~> 8.0'
gem 'redis', '~> 5.4'

Install the dependencies:

bundle install

Sidekiq is now available in your project. In the following steps, you'll create workers and configure job processing.

Step 2 — Creating your first background worker

Background workers in Sidekiq are Ruby classes that include the Sidekiq::Job module and define a perform method containing your job logic.

Create a file for your first worker and add the following code:

require 'sidekiq'

Sidekiq.configure_client do |config|
  config.redis = { url: 'redis://localhost:6379/0' }
end

class WelcomeWorker
  include Sidekiq::Job

  def perform(user_email)
    puts "Sending welcome email to #{user_email}"
    # In production, this would contain your email delivery logic
  end
end

WelcomeWorker.perform_async('user@example.com')

The Sidekiq.configure_client block establishes the Redis connection that Sidekiq uses for storing jobs. The WelcomeWorker class includes Sidekiq::Job, which provides the methods needed for job processing.

The perform method contains your actual job logic. In this example, it accepts an email address and logs a message. Real applications would include code for sending emails, calling APIs, or processing data.

The perform_async method enqueues a job for background processing. Sidekiq serializes the arguments and stores the job in Redis, where a worker process will pick it up.

Run the file to enqueue a job:

ruby welcome_worker.rb

INFO  2025-11-04T06:37:26.188Z pid=49776 tid=12lc: Sidekiq 8.0.9 connecting to Redis with options {size: 10, pool_name: "internal", url: "redis://localhost:6379/0"}

This adds a job to Redis, but nothing processes it yet. You need to start a Sidekiq worker process that pulls jobs from Redis and executes them.

Create a configuration file for the Sidekiq worker:

:concurrency: 5
:queues:
  - default

This configuration tells Sidekiq to process up to 5 jobs concurrently and to watch the default queue for work.

Start the Sidekiq worker process:

bundle exec sidekiq -r ./welcome_worker.rb -C sidekiq.yml

You'll see output showing Sidekiq starting up and processing your job:

...
INFO  2025-11-04T06:38:55.740Z pid=50133 tid=121h: Starting processing, hit Ctrl-C to stop
INFO  2025-11-04T06:38:55.749Z pid=50133 tid=120d jid=6ddf3b94163108a110accf71 class=WelcomeWorker: start
Sending welcome email to user@example.com
INFO  2025-11-04T06:38:55.749Z pid=50133 tid=11xx jid=729fb0b83fb76794dddbd0c7 class=WelcomeWorker: start
Sending welcome email to user@example.com
INFO  2025-11-04T06:38:55.750Z pid=50133 tid=120d jid=6ddf3b94163108a110accf71 class=WelcomeWorker elapsed=0.0: done
INFO  2025-11-04T06:38:55.750Z pid=50133 tid=11xx jid=729fb0b83fb76794dddbd0c7 class=WelcomeWorker elapsed=0.0: done

The worker processed the job from Redis and executed your perform method. Jobs are now running in the background, separate from your main application.

Step 3 — Working with multiple queues and priorities

Applications often need different types of jobs with varying urgency levels. Sidekiq lets you organize jobs into named queues and configure which queues workers process first.

Create a new worker for data export tasks:

require 'sidekiq'

Sidekiq.configure_client do |config|
  config.redis = { url: 'redis://localhost:6379/0' }
end

class ExportWorker
  include Sidekiq::Job
  sidekiq_options queue: 'low_priority'

  def perform(report_name, format)
    puts "Generating #{report_name} report in #{format} format"
    # Report generation logic would go here
  end
end

ExportWorker.perform_async('sales_summary', 'pdf')

The sidekiq_options method assigns this worker to the low_priority queue instead of default. This separation allows you to control how workers allocate their resources across different job types.

Now create a notification_worker.rb high-priority worker for urgent notifications:

require 'sidekiq'

Sidekiq.configure_client do |config|
  config.redis = { url: 'redis://localhost:6379/0' }
end

class NotificationWorker
  include Sidekiq::Job
  sidekiq_options queue: 'critical'

  def perform(message)
    puts "Sending critical notification: #{message}"
    # Notification delivery logic
  end
end

NotificationWorker.perform_async('System maintenance in 5 minutes')

Update your sidekiq.yml to process queues in priority order:

:concurrency: 5
:queues:
  - [critical, 8]
  - [default, 4]
  - [low_priority, 1]

The numbers represent weights that determine how often Sidekiq checks each queue. With these weights, Sidekiq checks the critical queue 8 times, default 4 times, and low_priority once per polling cycle. Critical jobs get processed faster, while low-priority work happens when resources are available.

Create a workers.rb file that requires all your workers:

require_relative 'welcome_worker'
require_relative 'export_worker'
require_relative 'notification_worker'

Start Sidekiq with the updated configuration:

bundle exec sidekiq -r ./workers.rb -C sidekiq.yml

Enqueue jobs from different queues in a separate terminal and observe how Sidekiq prioritizes the critical queue. First, enqueue the jobs:

ruby -r ./workers.rb -e "NotificationWorker.perform_async('Urgent alert'); ExportWorker.perform_async('monthly_report', 'csv'); WelcomeWorker.perform_async('new@user.com')"

Return to the first terminal, you'll see output similar to this:

INFO  2025-11-04T06:42:34.170Z pid=50739: Starting processing, hit Ctrl-C to stop
INFO  2025-11-04T06:42:34.179Z pid=50739 tid=133n class=NotificationWorker: start
Sending critical notification: System maintenance in 5 minutes
INFO  2025-11-04T06:42:34.179Z pid=50739 tid=133n class=NotificationWorker elapsed=0.001: done
INFO  2025-11-04T06:42:34.177Z pid=50739 tid=130r class=ExportWorker: start
Generating sales_summary report in pdf format
...
Sending welcome email to user@example.com
INFO  2025-11-04T06:42:34.181Z pid=50739 tid=133v class=WelcomeWorker elapsed=0.001: done
INFO  2025-11-04T06:43:05.070Z pid=50739 tid=1337 class=NotificationWorker: start
Sending critical notification: System maintenance in 5 minutes
...
Sending critical notification: Urgent alert
INFO  2025-11-04T06:43:05.070Z pid=50739 tid=133v class=NotificationWorker elapsed=0.0: done
INFO  2025-11-04T06:43:05.070Z pid=50739 tid=133f class=ExportWorker: start
Generating monthly_report report in csv format
INFO...
Sending welcome email to new@user.com
INFO  2025-11-04T06:43:05.070Z pid=50739 tid=130r class=WelcomeWorker elapsed=0.0: done

Sidekiq processes jobs concurrently using multiple threads (notice the different tid values), so all three jobs start nearly simultaneously.

The highlighted sections show critical notifications being processed—you can see multiple NotificationWorker jobs executing as Sidekiq's weighted queue configuration checks the critical queue more frequently. With the weights set to [critical, 8], [default, 4], and [low_priority, 1], critical jobs receive 8 times more attention than low-priority tasks, ensuring urgent notifications are handled promptly even under heavy load.

The error shows that when you start Sidekiq with backup_worker.rb, it tries to process the previously enqueued ReminderWorker job but can't find the class definition. You need to require the reminder_worker.rb file or clear old jobs from Redis.

Step 4 — Scheduling jobs for future execution

Beyond immediate background processing, Sidekiq handles delayed execution and recurring jobs. The perform_in and perform_at methods schedule jobs for future execution.

Create a reminder worker:

require 'sidekiq'

Sidekiq.configure_client do |config|
  config.redis = { url: 'redis://localhost:6379/0' }
end

class ReminderWorker
  include Sidekiq::Job

  def perform(user_id, message)
    puts "Reminder for user #{user_id}: #{message}"
  end
end

# Execute 30 seconds from now
ReminderWorker.perform_in(30, 123, 'Meeting starts soon')

# Execute at a specific time
execution_time = Time.now + 3600  # One hour from now
ReminderWorker.perform_at(execution_time, 456, 'Review quarterly report')

Run the file to schedule these jobs:

ruby reminder_worker.rb

INFO  2025-11-04T07:15:22.345Z pid=51234 tid=12ab: Sidekiq 8.0.9 connecting to Redis with options {size: 10, pool_name: "internal", url: "redis://localhost:6379/0"}

The jobs are stored in Redis with their scheduled execution times. Start Sidekiq to process them:

bundle exec sidekiq -r ./reminder_worker.rb -C sidekiq.yml

Wait for 30 seconds and you'll see the first reminder execute:

INFO  2025-11-04T06:53:56.045Z pid=53549 tid=165p jid=8d6daa9db8ea7f61be182a38 class=ReminderWorker: start
Reminder for user 123: Meeting starts soon
INFO  2025-11-04T06:53:56.047Z pid=53549 tid=165p jid=8d6daa9db8ea7f61be182a38 class=ReminderWorker elapsed=0.001: done
INFO  2025-11-04T06:54:10.246Z pid=53549 tid=165p jid=6ef9d0229dfcd5e296cb2e13 class=ReminderWorker: start
Reminder for user 123: Meeting starts soon
INFO  2025-11-04T06:54:10.247Z pid=53549 tid=165p jid=6ef9d0229dfcd5e296cb2e13 class=ReminderWorker elapsed=0.001: done

For recurring jobs, you'll need the sidekiq-cron extension. Add it to your Gemfile:

source 'https://rubygems.org'

gem 'sidekiq', '~> 8.0'
gem 'redis', '~> 5.4'
gem 'sidekiq-cron', '~> 2.3'

Install the new dependency:

bundle install

Create a worker for daily backups:

require 'sidekiq'
require 'sidekiq-cron'

Sidekiq.configure_client do |config|
  config.redis = { url: 'redis://localhost:6379/0' }
end

Sidekiq.configure_server do |config|
  config.redis = { url: 'redis://localhost:6379/0' }
end

class BackupWorker
  include Sidekiq::Job

  def perform
    puts "Starting database backup at #{Time.now}"
    # Backup logic goes here
  end
end

# Schedule to run daily at 2 AM
Sidekiq::Cron::Job.create(
  name: 'Daily Backup',
  cron: '0 2 * * *',
  class: 'BackupWorker'
)

The cron expression 0 2 * * * follows the standard cron format: minute, hour, day of month, month, day of week. This particular expression means "at 2:00 AM every day."

Here are more examples of cron expressions you might use:

# Every 15 minutes
cron: '*/15 * * * *'

# Every Monday at 9 AM
cron: '0 9 * * 1'

# First day of every month at midnight
cron: '0 0 1 * *'

# Every 6 hours
cron: '0 */6 * * *'

Start Sidekiq to activate the scheduled job:

bundle exec sidekiq -r ./backup_worker.rb -C sidekiq.yml

You'll see Sidekiq confirm the cron job registration:

INFO  2025-11-04T06:51:10.919Z pid=53119 tid=14fj: Cron Jobs - added job with name Daily Backup in the namespace default
...
INFO  2025-11-04T06:51:10.939Z pid=53119 tid=14fj: Starting processing, hit Ctrl-C to stop

The backup job now runs automatically according to its schedule, with no additional intervention required.

Step 5 — Monitoring jobs with the Web UI

Sidekiq includes a web dashboard that provides visibility into your job processing. The dashboard shows active jobs, retry queues, scheduled jobs, and historical metrics.

Add the necessary gems to your Gemfile for the web UI:

source 'https://rubygems.org'

gem 'sidekiq', '~> 8.0'
gem 'redis', '~> 5.4'
gem 'sidekiq-cron', '~> 1.12'
gem 'sinatra', '~> 4.2'
gem 'rackup', '~> 2.2'
gem 'puma', '~> 7.1'
gem 'rack-session', '~> 2.0'

Install the dependencies:

bundle install

Generate a session secret key for CSRF protection:

ruby -e "require 'securerandom'; File.write('.session.key', SecureRandom.hex(32))"

Create a configuration file for the web UI:

require 'sidekiq/web'
require 'rack/session'

# Configure session middleware for CSRF protection
use Rack::Session::Cookie, secret: File.read('.session.key'), same_site: true, max_age: 86400

# Load your workers
require_relative 'welcome_worker'
require_relative 'export_worker'
require_relative 'notification_worker'

run Sidekiq::Web

Start the web server:

bundle exec rackup config.ru -p 9292

INFO  2025-11-04T07:34:37.310Z pid=57441 tid=191d: Sidekiq 8.0.9 connecting to Redis with options {size: 10, pool_name: "internal", url: "redis://localhost:6379/0"}
Puma starting in single mode...
* Puma version: 7.1.0 ("Neon Witch")
* Ruby version: ruby 3.4.6 (2025-09-16 revision dbd83256b1) +PRISM [arm64-darwin24]
*  Min threads: 0
*  Max threads: 5
*  Environment: development
*          PID: 57441
* Listening on http://127.0.0.1:9292
* Listening on http://[::1]:9292
Use Ctrl-C to stop

Open your browser and navigate to http://localhost:9292. The dashboard displays an overview of your Sidekiq system:

The Overview page shows processed and failed job counts, queue sizes, and retry statistics. The Busy tab displays currently executing jobs with their arguments and execution time. The Queues page lists all queues with their job counts and lets you manage individual queues.

The Retries section shows jobs that failed and are waiting for their next retry attempt. You can manually retry or delete these jobs. The Scheduled tab displays jobs waiting for their execution time, including one-time delayed jobs.

The Dead page contains jobs that exhausted all their retries. These jobs remain in Sidekiq for 180 days before permanent deletion, giving you time to investigate failures and manually retry if needed.

Final thoughts

Sidekiq is a powerful Ruby library that makes background job processing simple, fast, and reliable. By using threads and Redis, it handles thousands of concurrent jobs efficiently while keeping memory usage low.

With features like multiple queues, job scheduling, and monitoring through its web UI, Sidekiq helps developers offload slow or heavy tasks, improve application performance, and maintain smooth, scalable workflows in production.

Got an article suggestion? Let us know

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.