Creating self starting worker machines for fun and profit – Avi Tzurel

I wrote a blog post in my other personal blog about how to create a machine that will manage it’s own lifecycle.

I work a lot with background workers, and I constantly need a better way to manage my Resque worker machines, so I wrote a post about how I did it.

I booted up over 50 servers with about 600 Resque workers using this method, and all with a click of a button in the UI.

Worth your time to read it if you are using Resque/Sidekiq and you need to add workers on rush-times of your site or something.

Creating self starting worker machines for fun and profit – Avi Tzurel.

Where are my engines?

Right now, I am working on an engine for the Gogobot application.

As part of my work on the engine, I am working with the Mongoid gem and the Fabrication gem, both great gems b.t.w.

The problem with Fabrication is that it wasn’t really adapted to working with Engines and it was relying a lot on Rails.root for where to locate the fabricator files.

Since in an engine, the application is located in spec/dummy I needed a better way locate the fabricators.

ok… so far the story of how I got to even writing this post.

To solve my problem I needed to grab all of the engines that are included inside the Rails application.

Turns out, there’s a real easy way to do it.

1
Rails.application.railties.engines
view raw find_engines.rb hosted with ❤ by GitHub

You can look at my solution including this line in the pull request I have open for Fabrication here: https://github.com/paulelliott/fabrication/pull/114

Working with Resque? perform_later is at 1.0, you should use it

Last night, I released the 1.0 version of perform_later.

Now or later?

The gem is being used in production for over 6 months now, in a big-scale website (Gogobot), so I thought it’s time to put in some extra needed work, some major refactoring and to release 1.0.

What’s new?

  • 1.0 is basically the same in the core functionality, I refactored big bits of the codebase, it’s much cleaner and more readable then before (still need to work quite a bit on code documentation)
  • Added the later method support for ActiveRecord models.
  • Added the loner support, now you can mark methods to be loners, meaning only a single instance of this method can exist in the queue at all times, this is super useful when a user does many actions that will result in say a cache clear for the user, there’s no reason to do a clear more then once. (and there are many more cases like this)

So, right off the README, here’s simple usage for this gem

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
class User < ActiveRecord::Base
def long_running_method
# Your code here
end
later :long_running_method
 
def long_running_method_2
# Your code here
end
later :long_running_method_2, queue: :some_queue_name
 
def lonely_long_running_method
# Your code here
end
later :lonely_long_running_method, :loner => true, queue: :some_queue_name
end
view raw user.rb hosted with ❤ by GitHub

The best thing about perform_later (not new in this version) is that you can pass in any param, and it will get translated into JSON with no problem.

You can pass in AR objects, hashes, arrays, anything you need, there’s no need to rewrite your methods.

I would love your comments, feel free to contribute issues, pull requests, documentation or anything else.

undefined method `relation’ for nil:NilClass on Heroku

I host almost 100% of my open source projects on Heroku, it’s amazing how easy it is to setup and how “out-of-your-way” they are.

Today, while working on one of my open source project I encountered this error:

1 2 3 4 5 6
plan.update_attributes!({:contract_id_with_trial => 2133290})
NoMethodError: undefined method `relation' for nil:NilClass
/app/.bundle/gems/ruby/1.9.1/gems/arel-2.0.9/lib/arel/crud.rb:12:in `update'
/app/.bundle/gems/ruby/1.9.1/gems/activerecord-3.0.3/lib/active_record/persistence.rb:255:in `update'
/app/.bundle/gems/ruby/1.9.1/gems/activerecord-3.0.3/lib/active_record/locking/optimistic.rb:77:in `update'
/app/.bundle/gems/ruby/1.9.1/gems/activerecord-3.0.3/lib/active_record/attribute_methods/dirty.rb:68:in `update'
view raw heroku_error hosted with ❤ by GitHub

It was right after I deployed and migrated the database.

After about 5 minutes of head scratching I came across an answer on stackoverflow saying you have to restart your app after a migration for it to pick up on schema changes.

Had a DAH! moment there :-)

The simplest Enum you will ever find for your ActiveRecord models

I have been using a really useful snippet for a while now.

While dropping it today into a project I realized just how powerful it is:

  1. it’s a drop-in and will work on any project
  2. it’s database agnostic
  3. it’s dead simple
  4. it’s not sensitive to enum changes
  5. and more and more

Just as an into, an enum is a way to have string represented as integers in your database.

Why?

Because integers are much faster to index and query (at least on mysql).

MySQL has an enum solution but ActiveRecord can’t really use it without some nasty hacking and it’s really messy when you want to add another param.

Enough Said, here’s the snippet

1 2 3 4 5 6 7 8 9
STATUS = { pending: 0, active: 1, inactive: 2, deleted: 3 }
 
def status
STATUS.key(read_attribute(:status))
end
 
def status=(s)
write_attribute(:status, STATUS[s])
end
view raw account.rb hosted with ❤ by GitHub

This gives you the ability to work with the Account model like so:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
>> account = Account.first
Account Load (0.2ms) SELECT `accounts`.* FROM `accounts` LIMIT 1
=> #<Account id: 4, name: "KensoDev", subdomain: "kensodev", user_id: 6, created_at: "2012-05-08 16:06:31", updated_at: "2012-05-08 16:29:11", logo_file_name: nil, logo_content_type: nil, logo_file_size: nil, logo_updated_at: nil, token: "5022N7VSAD", plan_id: 1, project_count: 1, contact_count: 1, quota_used: 0, notification_interval: nil, billing_token: "", status: 1>
>> account.status
=> :active
>> account.status = :pending
=> :pending
>> account.save
SQL (0.1ms) BEGIN
Account Load (0.3ms) SELECT `accounts`.`id` FROM `accounts` WHERE (`accounts`.`subdomain` = BINARY 'kensodev') AND (`accounts`.id <> 4) LIMIT 1
User Load (0.2ms) SELECT `users`.* FROM `users` WHERE (`users`.`id` = 6) LIMIT 1
Plan Load (0.2ms) SELECT `plans`.* FROM `plans` WHERE (`plans`.`id` = 1) LIMIT 1
User Load (0.3ms) SELECT `users`.`id` FROM `users` WHERE (LOWER(`users`.`email`) = LOWER('avi@kensodev.com')) AND (`users`.id <> 6) LIMIT 1
[paperclip] Saving attachments.
AREL (0.2ms) UPDATE `accounts` SET `status` = 0, `updated_at` = '2012-05-08 16:29:40' WHERE (`accounts`.`id` = 4)
[paperclip] Saving attachments.
SQL (1.5ms) COMMIT
=> true
>>

As you can see, it’s being persisted into the database as an integer, but you work with symbols/strings which is much nicer and cleaner.