What you need to know to start integrating Mike Perham's Sidekiq threaded work queue with your Rails app.

1. Sidekiq Basics
Lyon.rb
12 feb 2014
Kurt Sussman
@neophiliac

2. What is Sidekiq?
●
Async job manager
●
Released by Mike Perham in January 2012
●
Developed for theclymb.com
●
Drop-in replacement for Resque (almost)
●
Uses Celluloid gem for concurrency

3. Async challenges
●
Adds complexity (redis, sidekiq daemon)
●
Workers must be thread-safe and idempotent
●
Timing-related bugs are difficult to reproduce

4. Thread-safety
●
●
●
●
Uses all shared variables in an atomic way,
unless each is allocated to a specific instance
of the worker
Does not call non-thread-safe methods
Does not use APIs or database in a non-atomic
way
The G{I,V}L does NOT make Ruby thread-safe

5. Idempotency
●
●
An idempotent operation is one that has no
additional effect if it is called more than once
with the same input parameters
Sidekiq will retry jobs if there is an error

6. Why Sidekiq?
●
●
Sidekiq is faster, uses fewer resources, and is
well-supported
TheClymb claims 100% uptime for sidekiq*
Name
Database
delayed_job
SQL
Resque
redis
Sidekiq
redis
Threads
1
1
many
* but they restart sidekiq daemons every day

7. Submit a job
Mailer.welcome(@user.email)
becomes
MailWelcomeWorker.perform_async(@user.id)

8. Schedule a job
●
#perform_in(interval, *args)
●
Worker.perform_in(5.days, @user.id)
●
#perform_at(timestamp, *args)
●
Worker.perform_at(5.days.from_now, @user.id)

9. Testing
●
require 'sidekiq/testing'
●
Sidekiq::Testing.fake! (the default)
●
Worker.drain # force processing of queued
jobs
●
Sidekiq::Testing.inline!
●
Workers are POROs

10. Testbed
●
github.com/neophiliac/sidekiq-example
●
Disclaimer: not a real app!
●
git diff master sidekiq
●
Uses foreman to start redis and sidekiq
●
Includes (some) rspec tests

11. Further reading...
●
https://github.com/mperham/sidekiq/wiki/_pages
●
http://www.mikeperham.com/2014/01/04/the-clymb-2013/
●
https://github.com/neophiliac/sidekiq-example
●
●
http://www.sitepoint.com/comparing-background-processinglibraries-sidekiq/
https://github.com/JustinLove/autoscaler