Job Queue - Ruby (on Rails)
Integrating Ruby on Rails with HireFire Job Queue is quick and easy when using the
Add it to your Gemfile:
This library injects middleware into your stack, exposing an HTTP GET endpoint that HireFire will periodically request in order to get the latest information on your job queue(s).
In addition to that, it also ships with a bunch of convenient macros for common worker libraries in the Ruby eco-system, including:
- Delayed Job
- Queue Classic
The library is open source, you can view the source here.
For this example we'll use the popular Sidekiq library.
Assuming that you have the following worker entry in your Procfile:
worker: bundle exec sidekiq
You'll need to setup a corresponding worker resource for HireFire. Create an initializer at
app_root/config/initializers/hirefire_resource.rb and add the following code:
HireFire::Resource.configure do |config| config.dyno(:worker) do HireFire::Macro::Sidekiq.queue end end
The Sidekiq macro provided by
hirefire-resource simply counts all the jobs in all of your queues and returns an integer representing that number of jobs. When HireFire requests the HTTP GET endpoint at the following URL:
This library will transform your resource configuration to a JSON response in the following format:
HireFire will then read this information in order to determine the current queue size of your worker dynos. Using this, in combination with your auto-scaling configuration in the HireFire user interface, HireFire will determine how and when to scale your worker dynos.
HIREFIRE_TOKEN mentioned in the above code snippet can be found in the HireFire UI when creating/updating a dyno manager. It's just an environment variable that you'll need to add to your Heroku application later on.
What if you have multiple queues? This is quite common, and with HireFire you can deal with each queue separately by scaling multiple Procfile entries individually.
For example, let's say your Procfile not only contains a worker entry, but also an urgent_worker entry:
worker: bundle exec sidekiq -q default urgent_worker: bundle exec sidekiq -q urgent
All that you have to do to make this work is configure the following resources in your initializer:
HireFire::Resource.configure do |config| config.dyno(:worker) do HireFire::Macro::Sidekiq.queue(:default) end config.dyno(:urgent_worker) do HireFire::Macro::Sidekiq.queue(:urgent) end end
With this in place the library will now transform your resources in a JSON response with the following format:
Confirm that it works
With all this in place, start the local development server and access the following url:
You should now see a JSON response containing your queue sizes. If you do, congratulations, you're done! Deploy it to Heroku.
Now that you've integrated
hirefire-resource into your application and deployed it to Heroku, log in to HireFire and create two managers named
urgent_worker and configure them based on your auto-scaling requirements.
Don't forget to add the previously mentioned
HIREFIRE_TOKEN environment variable to your Heroku application.