I won’t cover a complete Ruby and Rails/Ramaze/Sinatra installation. This post assumes you have that much down pat, so we’ll pick up the ball with installing FreeTDS, unixODBC, Ruby ODBC

If you’re using Ubuntu 8.10, you can get away with installing everything from packages with the following:

apt-get install  unixodbc freetds-common tdsodbc \
             libodbc-ruby1.8 \
             libdbi-ruby1.8 libdbd-odbc-ruby \
             autoconf make

NOTE: I wrote this article and right before publishing, attempted to replicate these results on both mac OS X and CentOS 5. Both failed to work with the technique I thought I had all nice and worked out. I spent several hours over many months trying to find out exactly what’s different between the packages, but never got very far with my investigations. For a long time, it looked to me like both the ODBC packages and the FreeTDS packages need to be tweaked to properly handle [SERVER]\[DB_INSTANCE] construct that MSDE uses vs. just [SERVER] that the full-fledged SQL Server uses. This, after all, is the convention of the Windows ODBC driver and DSN definitions. However, there’s a really simple solution that I hit upon on Actual Technologies’ website that led me to a true universal solution and that’s to learn the port the INSTANCE runs on and set up the DSN specifically to that port!

But beware, the above doesn’t get you all of the freetds tools, most importantly the tsql command which can help you with debugging issues before getting into the ODBC game (that is, direct TDS connections to your SQL Server). So, to cover the most distros possible, I’ll document the “from source” approach for *nix systems and macports for Macs so you’ll find two sets of installation instructions for every step.

Preparing your Environment

I prefer to pull down all source packages into my local user directory under a “~/src” folder. The instructions that follow assume you’re in this folder. If you’re not, then please make the appropriate substitutions as you follow along. Additionally, I configure and compile most things as a regular user then use sudo to install system-wide as a “super user.”

Install unixODBC

unixODBC provides the ODBC connectivity for the TDS driver.

wget http://www.unixodbc.org/unixODBC-2.2.14.tar.gz
tar zxfv unixODBC-2.2.14.tar.gz
cd unixODBC-2.2.14
./configure  --enable-gui=no
make
sudo make install

On Macs

sudo port install unixODBC

On CentOS

sudo yum install unixODBC unixODBC-devel

Install FreeTDS

FreeTDS provides the open source implementation of the protocol implemented by both Sybase and Microsoft SQL Server.

wget ftp://ftp.ibiblio.org/pub/Linux/ALPHA/freetds/stable/freetds-stable.tgz
tar zvxf freetds-stable.tgz
cd freetds-0.82/
sudo ./configure  --with-unixodbc=/usr/local
sudo make
sudo make install

On Macs

sudo port install freetds

On CentOS

sudo yum install freetds

Install ruby-odbc

The ruby-odbc package provides the ruby bindings for ODBC connectivity.

wget http://www.ch-werner.de/rubyodbc/ruby-odbc-0.9995.tar.gz
tar zvxf ruby-odbc-0.9995.tar.gz
cd ruby-odbc-0.9995/
ruby extconf.rb
make
sudo make install

On Macs

sudo port install rb-odbc

Install ruby DBI

wget http://rubyforge.org/frs/download.php/655/ruby-dbi-all-0.0.23.tar.gz
tar zvxf ruby-dbi-all-0.0.23.tar.gz
cd ruby-dbi-all
ruby setup.rb config --with=dbi,dbd_odbc
ruby setup.rb setup
sudo ruby setup.rb install

Configure FreeTDS and ODBC

If you’re connecting to either SQL Server 2000 or SQL Server 2005 Standard, Enterprise, etc., then the typical approach is to add an entry for the server to /etc/freetds/freetds.conf and then reference that entry in the /etc/odbc.ini. Sometimes, these files get installed in /usr/local/etc. If so, then I will usually symlink the files into /etc for convenience, so that’s what’s referenced below.

My server is called “apvdbs01″ (the WINS name) with an IP Address of 10.0.2.2 and the database I’m connecting to is called “test_development” while “username” and “passwd” should be substituted with an valid user account and credentials for successfully connection to the database server.

Add a server entry for your server (which I typically give same name as Windows Server) to /etc/freetds.conf, so that host points to the SQL Server address:

[apvdbs01]
host = 10.0.2.2
port = 1433
tds version = 7.0

TDS version 7.0 is for SQL Server 2000. If you have SQL Server 2005, then tds version is 8.0.

tsql –S apvdbs01 –U username –P passwd

Once you can connect directly with the TDS drivers, move on to adding the ODBC driver entries.

Add the FreeTDS ODBC driver by editing “/etc/odbcinst.ini” (make sure “/usr/local/lib/libtdsodbc.so” exists first):

[FreeTDS]
Description     = TDS driver (Sybase/MS SQL)
Driver          = /usr/lib/odbc/libtdsodbc.so
Setup           = /usr/lib/odbc/libtdsS.so

Add a new DSN to “/etc/odbc.ini”:

[test_development]
Driver          = FreeTDS
Description     = ODBC connection via FreeTDS
Trace           = No
ServerName      = apvdbs01
Database        = test_development

So, “FreeTDS” is the driver that we gave for the lib TDS ODBC connector in the odbcinst.ini file (this allows you to install and maintain multiple versions of the libtdsodbc.so file). We reference this driver in the odbc.ini file with the “Driver” entry and we reference the actual server data in the freetds.conf file via the “ServerName” entry.

Now that you have ODBC configured for FreeTDS, test it with:

	isql test_development username passwd

But that doesn’t work for MSDE!

The above will get you connected to a standard or enterprise SQL Server 2000/2005 installation, but does not work for an Embedded SQL Server 2000/2005 installation. After trying many permutations, I finally came up with a configuration that actually works. One of the key things to know about MSDE is that the MSDE server instance is in the form of “servername\instance” rather than straight up “servername” as with standard/enterprise installations. This is known as a “named instance.” The trick to connecting? Find out the port the named instance is listening on and connect to that port rather than the 1433 port!

You can determine the port used by the instance by using the Server Network Utility in the Microsoft SQL Server program group on the Windows server where your database resides. Select the instance name from the drop down list and select TCP/IP from the list of enabled protocols. This will show you the port number for that instance.

[test_msde2000]
Driver         = FreeTDS
Description    = ODBC connection via FreeTDS
Server         = 10.0.2.2
Database       = my_test_db
Port           = 1348
TDS_Version    = 7.0

I could then connect to the MSDE instance like so:

isql test_msde2000 username passwd

Test ODBC connectivity from Ruby

If both tsql and isql commands work, then you’re well on your way to connecting to SQL Server via Ruby. To test Ruby with the above ODBC configurations:

irb
require "dbi"
dbh = DBI.connect("dbi:ODBC:test_development", "username", "passwd")

Connecting with Sequel

One of my favorite tools for connecting to SQL DBMS’ is Sequel, which is maintained by Jeremy Evans. To connect using Sequel, install the gem:

sudo gem install sequel

And then give this a shot:

irb
require 'rubygems'
require 'sequel'

DB = Sequel.odbc('test_development', :db_type => 'mssql', :user => 'username', :password => 'passwd')

References

http://www.freetds.org/userguide/confirminstall.htm

http://www.unixodbc.org/odbcinst.html

http://www.actualtech.com

Preamble (a.k.a. Ramblings)

I can hear you exclaiming, “Two hours!? Are you nuts? You really don’t want to do that on a web application,” and you would be correct; I did not. But alas, my users wanted it and they waved money in front of my face to get it done. What’s a poor sap like me to do but to comply?

So, what is this crazy task that needs to run so urgently it couldn’t wait until a midnight cronjob task to kick it off? Well, the task is basically exporting data from the live production database to a reporting database and the users sometimes have to generate those reports as soon as humanly possible against the then current production data. The problem is, the export process takes anywhere from 15 minutes to almost two hours, depending on how much data has changed since the last midnight run.

Such a long-running process is clearly outside the normal bounds of a website’s usual request/respond lifecycle and, if you’re like me and fronting your web app with Phusion’s Passenger, you’re quite at risk of Passenger’s process recycling action killing the task in mid-stream and losing your work in progress.

I didn’t quite know what I was going after when I started solving this problem and the main thing I could think to google for was “background jobs ruby,” which led to a rash of articles about backgrounDRB and many many rubyists complaining about its many shortcomings and creating their own spin on what, at core is known as a message queue. Many of the articles I read just did not cover the whole scope of the problem I faced and usually had simple solutions that lived well within the boundaries of their Rails (or Ramaze) framework’s inner workings.

Since I had a process that was capable of consuming resources upwards of two hours, I knew I had to completely escape the bounds of the framework and do the heavy lifting without fear that the parent process’ lifespan would risk the job getting terminated mid-stream. I also didn’t need a Rails plugin because, well, I’m building a Ramaze application and those fancy plugins just aren’t much of a help to me there.

One of the things I’m loving about Ramaze is that its forcing me to learn the Ruby language and its also forcing me to work harder to understand the components I’m pulling together in a solution. With Rails, I just pulled a plugin out of thin air, plugged it in and moved along. Which is great for rapid development, but more often than not, I have found myself later in a bind after the site’s live finding out that wonderful plugin is running way too slow and not knowing a darn thing about how to fix the problem without tearing up the whole plugin with a complete rewrite (in many cases).

With Ramaze, I feel much closer to the language, and I’m also finding that there’s just a lot of fluff I flat out don’t need that the Rails framework brings along, or rather, encourages developers to toss into the soup. On the other hand, Ramaze’s philosophical approach of squeezing every ounce of functionality out of every line of code has really pushed me towards being a better Ruby programmer and yet the same old time crunch every nearly developer faces actually forces me to get the job done with less code and with utilizing only the critical components actually necessary to get the job done. Which amounts to a one-two knockout punch for me because the solutions I deploy are consistently rock solid and I fully understand what I just put out there end-to-end.

Getting Started

The full project source can be found on github.com and you will also find my database enhancements to ProgressBar on github.com as well.

This project utilizes the following:

• Ruby Language
• Ramaze
• beanstalkd
• JQuery
• A modified version of ProgressBar (included in the demo source)
• These Gems
  • daemons
  • json
  • beanstalk-client

The Players

With the ramblings out of the way, its time to get to the heart of the matter at hand. There are three players in this game:

1. The trigger – This is the process that starts the whole chain reaction. In this case, the web app where the user can click a link to start the export. In my case, that web app was built with Ramaze.
2. The message queue – This is the middleman in the game. He sits around and accepts the message from trigger and passes the buck to the worker. We call him, “beanstalkd.”
3. The worker – This is the process doing all the work. Amazingly, Ruby was up for the task and the daemons gem is the ticket to making this work.

So lets cover these in more detail…

The Worker (sort of…)

The worker has got to always be around to do the job whenever that job comes in, so this suggested to me that I needed a daemonized process to watch the message queue for incoming jobs. Many solutions that I saw called for simple cronjobs running every minute or 5 minutes or whatever. But I couldn’t fathom why I’d want to resort to such hackery. How are you going to scale that? What happens if two cronjob instances fire and grab the same job request? What about if the job crashes or the server dies in mid-job? How is it reliably restarted? How do I ensure long-running jobs don’t pile up and consume all memory and CPU resources (e.g. every minute start processing the next big job and next thing you know, ten of ‘em are running when you only have the capacity to run one or two)? Simply put: cronjob workers is brute force mentality and rife with potential pitfalls.

Point being, I rapidly came to understand that I needed a message queue to serialize the jobs until the workers could take the jobs and run them and guarantee the jobs run before flushing them away. What was surprising, was the number of message queue implementations out there in the wild! Luckily, beanstalkd existed and had a very elegant implementation and API well-suited for Ruby. I easily installed from source to a Ubuntu server and also installed via macports on my Macbook Pro laptop AND installed the ruby client AND tested the small demos in irb all in about 30 minutes. Sadly I can’t say the same for RabbitMQ, Starling/Workling, ApacheMQ, and others I tried in vain to get up and going with. So, beanstalkd won out by default. Go to the beanstalkd website and follow their instructions. Its that simple.

How are we gonna keep up with the jobs?

Although the message queue has a way of peeking into it to learn about number of jobs and what’s running and the status of said jobs, the background processes I was going to be doing was going to run long enough that I wanted to pass along exactly what stage in the process the job was crunching and this meant building periodic update logging into the actual background job itself. The problem then is, how do we pass the information cleanly between worker and web app?

After some thinking on the matter, I chose to pass along progress updates via the database since all the headaches of I/O contention, persistence and deadlocking had already been solved there. I wasn’t quite sure how I was going to format my status updates and post the info to the database until I thought about the great little ProgressBar gem that I was already using for the task when I kicked it off at console. After a few minutes of looking into the internals of ProgressBar, I knew I could easily extend it to report its status to the database rather than emitting to the console. An hour later, I had my DbProgressBar and another hour later I had my unit tests written and passing.

Oh, yeah, the Worker…

I digressed from talking about the worker to covering the message queue and passing messages around, so lets get back to that worker script. As I pointed out earlier, I needed a worker script to do the heavy lifting for the job at hand and I wanted that puppy daemonized. One thing was perfectly clear in my mind: I didn’t want cronjobs nor Threads that depended on the parent processes staying alive (remember how Passenger aggressively scrubs ruby processes from memory!) nor any other means that depended on the parent process within the Ramaze framework kicking off the process directly.

The “daemons” rubygem fits the bill perfectly! I was really surprised by this little gem of a gem. Daemons makes it extremely easy to wrap ordinary Ruby scripts in the usual start|stop|restart|status world you find inside the typcial /etc/init.d/ Linux folder. What blew my mind was that I didn’t have to look at the usual long-winded case statement for all those daemonizing commands. Just focus on writing the main task for the worker, wrap that task in an infinite loop, install it rc.d folders as appropriate and you’re pretty much done. The following shows an example worker script:

require 'rubygems'
require 'daemons'
require 'beanstalk-client'
require 'sequel'

ROOT_DIR = File.dirname(File.expand_path(__FILE__))
$LOAD_PATH.unshift(File.join(ROOT_DIR, 'lib'))

require 'dbprogressbar'
require 'sequel_output'

Daemons.run_proc('process_astroid_jobs', :log_output => true) do
  DB = Sequel.sqlite(File.join(ROOT_DIR, 'demo.db'))
  SequelOutput.prepare_database(DB)
  beanstalk = Beanstalk::Pool.new(["0.0.0.0:11300"])

  loop do
    job = beanstalk.reserve
    jh = job.ybody
    puts "processing #{jh[:job_id]}"

    total = 15
    pbar = DbProgressBar.new("job ##{jh[:job_id]}", total,
      SequelOutput.new(jh[:job_id], DB))

    total.times do
      pbar.inc
      sleep(1)
    end

    pbar.finish
    puts "finished #{jh[:job_id]}"
    job.delete
  end
end

There are a few key things to note about the above script. Firstly, wrapping it in the Daemons.run_proc tells the daemons gem we want to run this like we’d run a typical server. SequelOutput is the extension I wrote for ProgressBar to get it to outout to the database and at the start of the script, we ensure the database is ready (namely that the “progress_bars” table is created) before we go into our infinite loop to process jobs.

Next up, the loop itself. Its not a runaway infinite loop that’ll end up consuming all resources because we are calling the beanstalk queue with “reserve” which is a blocking call, thus we sit and wait until a job comes through. As soon as we get a new job request, we create the DbProgressBar and get to work processing….twiddling our thumbs for 15 seconds in this case! When the job’s completed, we remove it form the message queue and begin again.

The Trigger

Finally, with the back-end done, we could focus on enabling the user to trigger the job on the front-end. Of course, that was as simple as firing up a connection to the beanstalk server and pushing a job onto the queue in a controller’s action. But how do we watch the job and report to the user what’s going on without overwhelming our server? The answer came along with a “Comet with Ramaze” example by Pistos. What a lovely idea: Let the server intentionally hold onto the client’s request until it had something to report back! I knew it’d never hold onto the request for more than 2 or 3 seconds, so this was a perfectly workable model for keeping traffic levels sane and timely.

This was also my first time playing around with JSON in AJAX calls and so I got to spend some time flexing my JQuery muscles a bit and also discovered the Ruby json gem that makes it dirt simple to turn a hash into a JSON response. This is just a small excerpt from the controller/main.rb check_job_progress action:

# report status as finished when the job is completed
if progress[:job_finish]
  return {:progress_text => "finished.", :progress_status => "finished"}.to_json

# otherwise report the percent done as the job status 
else
  return {:progress_text => progress[:progress_text].gsub(' ', ' '),
      :progress_status => progress[:progress_percent]}.to_json
end

Notice the call to the to_json to convert that hash to JSON response and how clean the Ruby code is even though we’re passing a rich data object back to our browser client. I substituted blanks with the   character to keep the progress bar from collapsing all those empty blanks between the two vertical bars. I decided to pass on a progress status so that I could check when things were done and thus stop polling the server for job updates. This is just a demo, so try not to snicker too much at my use of plain English to convey critical job statuses. At least the intent is clear!

The JQuery script is also correspondingly simple.

function check_progress() {
	job_progress_element = $('#job-progress');
	$.getJSON('/check_job_progress', {job_id: job_progress_element.attr('value')},
	  function(data){
	    job_progress_element.html(data.progress_text);
      	if (data.progress_status != "finished") check_progress();
	});
}

$(document).ready(function() {
	if($('#job-progress').length) check_progress();
});

As you can see, it simply asks the server for an update and when it receives one, updates itself and asks again! It will check the job status after each update to see if the job has completed or not, thus stopping its persistent nagging once we’ve reached the Kingdom of Far, Far Away.

References

Here are a few other references, not mentioned above, that were invaluable in helping me pull this solution together.

Ruby Beanstalkd Distributed Worker Basics
How to Create a Daemon Process Using Ruby and the Daemons RubyGem
Err the Blog – About this blog Beanstalk Messaging Queue”>

Its not that I don’t think these tools are worthy contenders. Most of them, I have used fairly extensively myself over the years. The suggested contenders are solid and mature projects, but they feel decidedly “old hat” to me much as apache, postfix, vim, bind, and the linux kernel probably feels to the system administrator. In other words, they’re ubiquitous, so why signal them out with some sort of top 100 reward?

These days, I have adopted Ruby as my language of choice, so I have moved away from Python and PHP. While this has largely meant I develop Ruby on Rails applications. However, as my Ruby skills grow, this also means going far and wide in many other ruby-based tools (such as rake, sake, and rack for system-level or middleware tools). I am now using ruby for quick and dirty systems administration tasks (such as the short script I wrote to find all currently assigned IP addresses on our office network), automating deployments (capistrano, puppet, chef), marshaling data from one database system to another, and even with extensive image-processing (resizing, conversion of formats, compressing, etc.).

So what, in my view, warrants a top 100 award recognition? I like to see new and exciting projects on these lists. If a project’s on the list, I like to know that they are extremely active and rapidly gaining mind-share and penetration into the community. I want to know a little bit about how they put a new spin on an old problem and the impact they have been able to make, not only in the project’s direct vertical niche, but across the board. A few years ago, Ruby on Rails arrived on the scene as a game-changer, not only for putting Ruby on the map as a viable language, but as a bona fide development framework for delivering web-based services. Ruby on Rails turned the Java world on its head and forced Java developers to drastically improve their productivity tools then in common practice; they evangelized convention over configuration and brought many fantastic ideals such as Agile development and RESTful APIs to the forefront of many, many developers’ minds.

To my way of thinking, though, Ruby on Rails is now old news. Newcomers will stumble upon Ruby and Rails through word of mouth or simply googling for a solution to some problem they’re trying to solve. That’s the problem I see with top-100 lists paying tribute year after year to the same old mature projects. Whether its Ruby on Rails or Word Press or Joomla or Django, all of these are easily discovered and well-written about by anybody with decent querying skills. So what’s out there today that’s refreshing and hard to find? Yet, is making a huge impact on the community? Here are a few that I think are worthy contenders (and yes, they are a little heavy on the Ruby side)…

Rack

Rack: a Ruby Webserver Interface. What I find phenomenal about Rack is its adoption by so many frameworks that are delivering web-based services. Micro-frameworks like Ramaze and Camping and Sinatra were early adopters of Rack and such adoption led to far smaller code-bases in each as well as significant performance boosts and, as if that wasn’t enough, enabled much easier transition from Ruby 1.8 to 1.9.

Inkscape and ImageMagick

Speaking of image processing, I am not sure if many have encountered ImageMagick and the Ruby bindings offered via RMagick, but definitely worth a look-see. As of this year, I do almost all of my creative graphics work in Inkscape and The Gimp and use ImageMagick to automate some pretty impressive transformations of those images outside the GUI. Before Inkscape became mature enough to use on daily basis (which, by my reckoning is only in the last 18 months or so), I was heavily reliant on Adobe’s Illustrator and companion Photoshop apps. I never would’ve imagined two years ago that I could completely do away with these packages, let alone begin to automate much of the repetitive tasks that I put those resulting images through (resizing, cropping, web-optimizing, etc.) with a collection of well-written OSS tools.

Ramaze and Sequel

One last pair of tools I think is worthy of consideration, especially for web-based ruby developers is Sequel and Ramaze. Sequel is an object-relational-mapping library that makes working with SQL databases a real pleasure for Ruby developers. The maintainers seem to have taken the best of the best and rolled those ideas into one coherent and extremely fast library for accessing just about any database out there. I have fooled around with DataMapper and have worked extensively with ActiveRecord and I must say Sequel really is rung or two above these libraries and I look forward to continuing to adopt Sequel.

I ran into Sequel by way of the Ramaze project, which is a micro-framework for building web-based applications. Unlike Ruby on Rails, which tends to generate tons of scaffold/skeletal code for you to begin hacking away on to build your projects, Ramaze takes the opposite tack in that it provides little to no scaffolding or auto-generated code. Instead, the mindset is squeezing the maximum amount of functionality out of every line of code you do write. For a Ruby on Rails developer coming to Ramaze, the feeling is very much like the first time I took off on a bicycle without the training wheels attached. You feel wildly uncontrolled and like you may fall at any moment, but a little encouragement from the warm friendly community gets you going straight and true and the amount of code you’re NOT writing, reading, reviewing, nor specking tests for really begins to jump out at you!

The Problem

I am working on a content management system (CMS) where I want the pages to have a hierarchical structure that turns into a menu with sub-menus. The content manager needs to also be able to order these pages so that the menu structure renders in the desired order.

The Solution

Two plugins jumped to mind almost immediately: acts_as_tree and acts_as_list. The tree plugin will manage the hierarchy, hinging off the parent_id field of the model whilist the list plugin uses the position column to manage the order. What’s unique here is that I have never used both on one model, but doing so was surprisingly easy:

   1  class Page < ActiveRecord::Base
   2    acts_as_tree
   3    acts_as_list
   4  end

Constructing a Hierarchical Menu

Before going too far, if you haven’t seen the Suckerfish menus, yet, please do check out the article as it will help you quickly see how I approached the menu rendering. Secondly, to install the two plugins, its a simple pair of command line calls as follows:

   1  script/plugin install acts_as_tree
   2  script/plugin install acts_as_list

With the CSS handling all the styling, all I needed was to render the nested unordered lists. I began by grabbing all of the pages at the top level (where parent_id is null) with this bit of code:

   1  class Page < ActiveRecord::Base
   2    acts_as_tree :order => :position
   3    acts_as_list
   4 
   5    def self.top_level_pages
   6      find(:all, :conditions => ["parent_id IS NULL"], :order => :position)
   7    end
   8  end

If you noticed the “:order => :position” clause and thought, “but acts_as_list handles that for you,” then you have spotted the first “gotcha” I encountered with using both tree and list on a model. The tree plugin loses the position ordering that the list plugin mixes in and adding these order clauses in preserves the order of the records. With the query to get the top-level menus in place, I set the @pages variable by calling Page.top_level_pages in the controller and then rendered with this call in my view:

   1  <div id="site-navigation">
   2  	<ul>
   3  		<%= render(:partial => "/layouts/site_navigation", :collection => @pages) %>
   4  	</ul>
   5  </div>

To make it all nice and nested, I simply recursively called the same site_navigation partial on the children like so:

   1  <li><%= link_to(site_navigation.name, url_for(site_navigation))  %></li>
   2  <ul><%= render(:partial => "/layouts/site_navigation", :collection => site_navigation.children) %></ul>

To give the user ability to move the menu items around, I added “move_up” and “move_down” actions to the Page Controller in which I called the “move_higher” and “move_lower” methods that are mixed in by the acts_as_list. I realized rather quickly that the position index was getting out of sync and added a reindex method to the Page model to clean up the data along with a scope declaration to the model. Along the way, I also realized that if I moved a page from one parent node to another parent node, I potentially opened a gap in the position index in the collection of Pages and this again breaks the position index sequencing. The acts_as_list mix-in expects the position index to always go [0, 1, 2, 3, ...N]. Whenever this is not the case, the move_higher and move_lower methods stop working and the user interface no longer responds correctly. So this is why we care so much about the position index sequence. So, to handle scoping of the position index sequence correctly and to handle Pages being moved to another parent, we arrive at this final version of the Page model:

   1  class Page < ActiveRecord::Base
   2    acts_as_tree :order => :position
   3    acts_as_list :scope => :parent_id
   4 
   5    before_save :keep_position_sane
   6 
   7    def self.top_level_pages
   8      find(:all, :conditions => ["parent_id IS NULL"], :order => :position)
   9    end
  10 
  11    def self.reindex_top_level_pages(recurse = true, departing_child = nil)
  12      reindex_pages(self.top_level_pages, recurse, departing_child)
  13    end
  14 
  15    def reindex_children(recurse = true, departing_child = nil)
  16      Page.reindex_pages(children, recurse, departing_child)
  17    end
  18 
  19    private
  20 
  21    # takes a given array of pages and recursively (or not) reindexes
  22    # if departing_child is supplied, it is removed from the array so 
  23    # that former siblings are reindexed as though it was already 
  24    # removed from the collection.
  25    def self.reindex_pages(pages, recurse, departing_child)
  26      pages.select{|r| r != departing_child}.each_with_index do |page, index|
  27        page.reindex_children(true) if recurse
  28        page.update_attributes(:position => index + 1)
  29      end
  30      true
  31    end
  32 
  33    # When the parent id of a node changes, the acts_as_list gets lost, so 
  34    # we need to reindex the affected nodes to keep things sane
  35    def keep_position_sane
  36      return unless self.parent_id_changed?
  37 
  38      # reindex the group this page is being removed from
  39      if self.parent_id_was.nil? then
  40        Page.reindex_top_level_pages(false, self)
  41      else
  42        Page.find(self.parent_id_was).reindex_children(false, self)
  43      end
  44 
  45      # make this page the last sibling of the new parent group of pages
  46      last_page = (self.parent_id.nil? ? Page.top_level_pages.last : Page.find(self.parent_id).children.last)
  47      self.position = (last_page.nil? ? 1 : last_page.position + 1)
  48      true
  49    end
  50  end

Check out the “keep_position_sane” callback. You’ll see a nifty application of the new Dirty records feature of ActiveRecord (which I believe was released with Rails 2.2) and was covered by Ryan Diagle in his post,
What’s New in Edge Rails: Dirty Objects. In order to detect that the parent node was indeed changing and which parent’s collection the Page belonged to, I checked the “parent_id_was” value. In this case, I had to handle top level pages being moved into another Page’s collection as well as A sub-page being promoted to top-level.

A Handy Rake Task

If you’re wondering why the recursive parameter and otherwise unnecessarily complex procedures, its because I also added a rake task to recursively reindex all pages in the menu hierarchy:

   1    namespace :pages do
   2      desc "re-index page positions"
   3      task :reindex => :environment do
   4        Page.reindex_top_level_pages
   5      end
   6    end

Conclusion

Rails has really come a long ways since its early days, but its still not without its little “gotchas” and sometimes its tough to uncover silent failures (like with the position order clause not being rendered from the acts_as_list plugin) when mix-ins from different plug-ins are utilized. Hopefully, this article shows you a few new tricks and how to utilize somewhat competing plugins safely within one model. The new Dirty attributes feature of ActiveRecord definitely made the chore of implementing this functionality much easier and sane than it would’ve been back in the Rails 1.x days.

Introducing Ramaze

One I found is Ramaze. I have to say that Ramaze is a very minimalist approach to a framework that is actually rather enlightening. For one coming from a Rails’ world, it can feel a bit sparse at first. But lately, I’m attributing my lack of comfort to being put on my toes to do some actual real Ruby coding. Its both a little scary and liberating at the same time. I have found that I am enjoying Ruby a lot more as a result of my exposure to Ramaze and my thought process is actually radically changing, almost like a Visual Basic programmer going to Borland Delphi might feel.

So I decided to take a hard look at performance metrics between Rails and Ramaze’s. At the same time, I wanted to see how MRI 1.8.7 stacked up against Ruby Enterprise Ed. 1.8.6, so I’m satisfying a dual curiosity herein. Being quite happy with Phusion Passenger, I decided to set things up with Apache 2.2.9 + Passenger + Rails/Ramaze. Since I’m quite comfortable with ERB and ActiveRecord, I tweaked Ramaze to be a bit more like the familiar Rails world I am familiar with.

Un-DRYing the Results

There’s no point in generating a lot of data and reporting on it if others cannot repeat yourself (CRY!) in replicating the results, so with that in mind, I attempt to refrain from further bad turn of words by busying myself in pushing everything I utilized to github.com and documenting the basic bootstrapping of Ubuntu Intrepid. For these tests in particular, I used two scripts, bootstrap-passenger-std.sh and bootstrap-passenter-ent.sh to set up four distinct configurations under the Ruby1.8 interpreter. Ruby 1.9.1. was a bit difficult to set up and there is a bootstrap-passenger-191.sh script that I assembled as I installed things, but I haven’t run it to ensure it works end-to-end.

I utilized this simple call to Apache Bench to run through the four test scenarios:
All data collected was based on 10,000 requests at 100 concurrent users after an initial “warmup” of the passenger threads with start of 100 requests at 10 concurrent users. The following command shows the basic set of parameters issued to Apache Bench:

ab -n 10000 -c 100 http://demo.u64rails01.local/posts

What was tested?

In order to compare, as close as possible, apples to apples, I opted to use Active Record and ERubis in Ramaze to set up the same MVC classes in both the Rails and Ramaze project. ActiveRecord is probably the heaviest weighted ORM that can be utilized with Ramaze. I tried briefly to get Erubis working with Rails so that the two frameworks’ projects would be more closely aligned than the out-of-box ERB rendering Rails ships with would have afforded. I did attempt Erubis with Rails, however, I was either doing something very wrong or Erubis under Rails takes a sizable performance hit and I stuck with out-of-box ERB for these tests given its better performance. As such, Rails uses out of box ERB rendering in these tests.

The application itself is ridiculously simple, offering only one model and performing only a read against the database, but hopefully, an effective start towards building this benchmark framework towards more substantive benchmark tests to come. For now, lets take a look at how Rails and Ramaze stack up to each other and what affect the underlying interpreter can have on performance. Standard packaged Main Ruby Interpreter (MRI) 1.8.7 consisted of simple package install provided by the Ubuntu Intrepid repositories whle Ruby Enterprise Edition (REE) 1.8.6, which hails from the same folks bringing you Phusion Passenger, is compiled from source.

As a bonus, I also compiled and clocked metrics for Ramaze on 1.9.1. This was a tricky environment to get set up and working. And my initial results made me think I did something horribly wrong with compiling and configuring Ruby 1.9.1. However, I fully replicated Antonio Cangiano’s benchmarks in my test environment (see The Great Ruby Shootout (December 2008)) and MRI 1.9.1. did indeed crush the other Ruby interpreters in Fibonacci number crunching (amongst others). The theory tossed around in the Ramaze IRC channel was that 1.9.1. change in implementation of the String class (and it being unicode native now) slows things down a good bit with string processing.

The Results

The following table tabulates the results gathered from Apache Bench as well as passenger-memory-stat. I ran a short run to warm things up (100 requests, 10 concurrently). And then ran the big bench at 10,000 requests, 100 concurrently, and then ran again and recorded the results. At around 5,000 requests, I ran passenger-memory-stat to collect the data during mid-processing and then again at the end. These tests were run on a Ubuntu 64-bit Intrepid 8.10 virtual server that was allocated 1 CPU core and 512MB of RAM. Swap space was never triggered (0k utilization), and CPU utilization was roughly 85% within the Virtual Machine during the test runs. The host machine, which is four core CPU, showed one core busy at near 100% with the other three cores pretty much idle.

Throughput

Everybody likes to talk about throughput, so lets start there. As you can see, MRI and Rails was the worst performer while REE and Ramaze clocked in with the highest throughput in terms of requests per second. Be forewarned that throughput doesn’t necessarily equate to user response times. This metric simply tells you the load at which your system is capable of delivering content at.

Time per Request

Time per request helps you see about how long an average user request takes for a given load. As you can see here, with times in the milliseconds, we’re talking about numbers that will easily be swallowed in the noise of network latency. Even so, its not hard to be impressed by what the REE folks managed to pull out of their hat with regards to performance on the main ruby interpreter.

Memory Consumption

This final graph shows the sizeable dent REE makes in memory consumption for running a Rails application. Ramaze is already pretty efficient and lean and REE made practically no dent in memory consumption here. However, REE’s use of the faster memory allocation routines does have a good effect on boosting Ramaze’s performance (as seen in above graphs), so there’s still lots to be gained by running Ramaze with REE.

Conclusion

For me, I have two very, very viable solutions before me. Rails on Enterprise Ruby or Ramaze on Ruby 1.8.7. The final choice comes down to how much I can bet on my own Ruby prowess. Ramaze is a very lean framework that gets you going, but doesn’t get you a nine inch pillowed mattress topping for your bed like Rails does. Because Ramaze is so lean, I realize I will have to really take my Ruby skills to the next level and truly understand the ruby code that other developers produce. Unlike Rails, where there’s gobs and gobs of plugins and blogs on the topic that make pretty much any need simple to satisfy and get working, I have to know my Ruby to get a solution implemented on Ramaze. Not a bad thing, but certainly a more challenging and potentially rewarding path to take.

Aside from my personal concerns about making the jump, looking at these results, it is good to see that I have options and how easy it is to get a good deployment up and working compared to 3 or 4 years ago when I was banging my head against the wall for days on end to keep lighttpd + fcgi processes up and running around the clock. In these tests, it was very, very satisfying that not one single crash or dropped request was recorded in all the benchmarking I performed. Mongrel came along and made a big impact, but you still needed something like monit to watch Mongrel and restart it on ocvasion. Now Passenger has come along and made yet another big impact in the ease of deploying Rails, Ramaze, or any of the other ruby-based frameworks. We’ve come a long ways, indeed. I, for one, certainly look forward to what the Ruby sphere will bring us over the next couple of years.

Another drawback I have with Google Syntax highlighter is that the page renders without highlighting and then all the code listings change to syntax highlighted text a few moments later.

Introducing Ultraviolet, a syntax highlighter supporting over 50 syntaxes and any TextMate theme.  Ultraviolet is based on Oniguruma, a regular expression library that comes standard with Ruby 1.9. Ultraviolet requires a little more effort for me to cut and paste, but I am definitely much happier with the end results and Wordpress no longer mungs my posts. Only real complaint I have is that I can’t read the posted code inline in HTML format (but I can read it in Visual mode!).

Ruby Script

   1  require 'rubygems'
   2  require 'ramaze'
   3  require 'maruku'
   4  require 'sequel'
   5 
   6  $LOAD_PATH.unshift(__DIR__)
   7 
   8  require 'env'
   9 
  10  require 'vendor/flickr'
  11  require 'vendor/haml_maruku_filter'
  12  require 'vendor/image_science_cropped_resize'
  13 
  14  require 'model/init'
  15  require 'controller/init'
  16 
  17  require 'db/init' if SOCIAR.mode == :dev
  18 
  19  r = SOCIAR.ramaze
  20 
  21  =begin
  22  if r.gzip
  23    require 'ramaze/contrib/gzip_filter'
  24    gzip = Ramaze::Filter::Gzip
  25    gzip.trait :threshold => 1
  26    Ramaze::Dispatcher::Action::FILTER << gzip
  27  =end
  28 
  29  Ramaze.start :adapter => r.adapter,
  30               :host => r.host,
  31               :port => r.port,
  32               :boring => r.boring

Shell Script

   1  #!/bin/sh
   2 
   3  ### BEGIN INIT INFO
   4  # Provides:          lm-sensors
   5  # Required-Start:    $local_fs
   6  # Required-Stop:
   7  # Default-Start:     S
   8  # Default-Stop:
   9  # Short-Description: lm-sensors
  10  # Description:       hardware health monitoring
  11  ### END INIT INFO
  12 
  13  . /lib/lsb/init-functions
  14 
  15  [ -f /etc/default/rcS ] && . /etc/default/rcS
  16  PATH=/bin:/usr/bin:/sbin:/usr/sbin
  17  PROGRAM=/usr/bin/sensors
  18 
  19  test -x $PROGRAM || exit 0
  20 
  21  case "$1" in
  22    start)
  23  	log_begin_msg "Setting sensors limits"
  24  	if [ "$VERBOSE" = "no" ]; then
  25  		/usr/bin/sensors -s 1> /dev/null 2> /dev/null
  26  		/usr/bin/sensors 1> /dev/null 2> /dev/null
  27  	else
  28  		/usr/bin/sensors -s
  29  		/usr/bin/sensors > /dev/null
  30  	fi
  31  	log_end_msg $?
  32  	;;
  33    stop)
  34  	;;
  35    force-reload|restart)
  36  	$0 start
  37  	;;
  38    *)
  39  	log_success_msg "Usage: /etc/init.d/sensors {start|stop|restart|force-reload}"
  40  	exit 1
  41  esac
  42 
  43  exit 0

Apache Config

   1  <VirtualHost *:80>
   2  	ServerName www.stinger.local
   3  	ServerAdmin webmaster@localhost
   4 
   5  	DocumentRoot /var/www/
   6  	<Directory />
   7  		Options FollowSymLinks
   8  		AllowOverride None
   9  	</Directory>
  10  	<Directory /var/www/>
  11  		Options Indexes FollowSymLinks MultiViews
  12  		AllowOverride None
  13  		Order allow,deny
  14  		allow from all
  15  	</Directory>
  16 
  17  	ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
  18  	<Directory "/usr/lib/cgi-bin">
  19  		AllowOverride None
  20  		Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
  21  		Order allow,deny
  22  		Allow from all
  23  	</Directory>
  24 
  25  	ErrorLog /var/log/apache2/error.log
  26 
  27  	# Possible values include: debug, info, notice, warn, error, crit,
  28  	# alert, emerg.
  29  	LogLevel warn
  30 
  31  	CustomLog /var/log/apache2/access.log combined
  32 
  33      Alias /doc/ "/usr/share/doc/"
  34      <Directory "/usr/share/doc/">
  35          Options Indexes MultiViews FollowSymLinks
  36          AllowOverride None
  37          Order deny,allow
  38          Deny from all
  39          Allow from 127.0.0.0/255.0.0.0 ::1/128
  40      </Directory>
  41 
  42  </VirtualHost>

Environment

This post will guide you through setting up the following environment on Ubuntu Intrepid (8.10):

• Apache 2.2.9
• Passenger 2.0.6
• ActiveRecord 2.2.2
• Ramaze 2009.01
• Erubis 2.6.2
• MySql 5.0.67 (assumed installed)

To get started:

sudo apt-get install apache2 passenger
sudo passenger-install-apache2-module

Install the Rack, Ramaze, Activerecord, and Erubis gems

sudo gem install rack, ramaze, activerecord, erubis

Establishing Project

Now create your first ramaze project with:

ramaze --create demo

Then change to the demo directory. If you see a start.ru file, rename it to config.ru or Passenger won’t find it. Create a file called passenger.rb with the following:

require 'rubygems'
require 'ramaze'

# Add directory start.rb is in to the load path, so you can run the app from
# any other working path
$LOAD_PATH.unshift(__DIR__)

# Initialize controllers and models
require 'controller/init'
require 'model/init'

Ramaze::Global.sourcereload = false
Ramaze::Global.sessions = true
Ramaze::Log.ignored_tags = [:debug, :info]

Important: Do not include “Ramaze.start …” in this file or this will surely mess things up for Passenger.

Now edit the config.ru file (formerly start.ru):

/usr/bin/env rackup
# start.ru for ramaze apps
# use thin>=0.6.3
# thin start -r start.ru
#
# rackup is a useful tool for running Rack applications, which uses the
# Rack::Builder DSL to configure middleware and build up applications easily.
#
# rackup automatically figures out the environment it is run in, and runs your
# application as FastCGI, CGI, or standalone with Mongrel or WEBrick�..all from
# the same configuration.

require 'app'
Ramaze.trait[:essentials].delete Ramaze::Adapter
Ramaze.start
run Ramaze::Adapter::Base

Add the following virtual host file to /etc/apache2/sites-enabled/001-ramaze-demo

<VirtualHost *:80>
    ServerName ramaze.demo.local
    RackBaseURI /
    RailsAutoDetect off
    RackAutoDetect on
    DocumentRoot /var/www/ramaze/demo/public
</VirtualHost>

And edit your /etc/hosts file to include “ramaze.demo.local” so it resolves to localhost.

Database

Next, create a database called ramaze_demo (or your choice) and add a posts table and
some data via SQL or migrate scripts (this being a minimal demonstration and all that).

create table posts (
    id        int(11) not null auto_increment
  , title     varchar(255) not null
  , body      varchar(1024) not null
  , primary key (id)
);

insert posts values (1, "First Post", "Thank hrnt for getting passenger talking to ramaze.");
insert posts values (2, "Another Post", "Ask Pistos what SelfMarks is, because he is bored with diigo");
insert posts values (3, "IRC, you say?", "Visit #ramaze on Freenode");

Model

Edit the model.init.rb file with the following:

require 'activerecord'
require 'cleanup_connection_patch'

  ActiveRecord::Base.establish_connection(
    :adapter  => "mysql",
    :database => "ramaze_demo",
    :host     => "localhost",
    :username => "root",
    :password => "secret"
  )

class Post < ActiveRecord::Base
end

Important: cleanup_connection_patch comes from github.com and makes the ActiveRecord behave correctly in a threaded environment. Please see Monkey patching ActiveRecord to automatically release connections « coderrr for more information.

Controller

With the model squared away, time to turn our attention to the controller. Edit controller/init.rb:

class Controller < Ramaze::Controller
  layout '/page'
  helper  html
  engine :Erubis
end

class PostsController < Controller
  def index
    @posts = Post.find(:all)
  end
end

View

Create a folder under view for posts and add the following Erubis template view/posts.rhtml:

<h1>Listing Posts</h1>

<ul>
  <% for post in @posts %>
      <li><%= A(post.title, :href => "/post/#{post.id}", :title => post.body) %></li>
  <% end %>
</ul>

Note: that while Rails uses filename.html.erb, the Erubis engine expects *.rhtml extension in Ramaze.

Running

Test out the above with webrick, mongrel, or thin with:

ruby start.rb

If all goes well, you should see the three postings above listed.

Launch Apache2, if you haven't already and point your localhost's browser at http://ramaze.demo.local/posts

You should now have a listing of posts rendered through Apache via Passenger.

Conclusion

I am finding much to like about Ramaze, especially after developing with Rails for a few years now and watching it grow at an alarming rate in complexity. Ramaze is a refreshing return to basics. I am finding that Ramaze just may be able to give me most of what I need while giving me a sizeable performance boost in the process. Here's a rails bench on the equivalent of the above:

Concurrency Level:      50
Time taken for tests:   4.277 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      10218492 bytes
HTML transferred:       9595000 bytes
Requests per second:    233.83 [#/sec] (mean)
Time per request:       213.828 [ms] (mean)
Time per request:       4.277 [ms] (mean, across all concurrent requests)
Transfer rate:          2333.42 [Kbytes/sec] received

And this is Ramaze:

Concurrency Level:      50
Time taken for tests:   2.248 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      9815000 bytes
HTML transferred:       9555000 bytes
Requests per second:    444.91 [#/sec] (mean)
Time per request:       112.383 [ms] (mean)
Time per request:       2.248 [ms] (mean, across all concurrent requests)
Transfer rate:          4264.42 [Kbytes/sec] received

Elemental provides enumerated collection of elements that allow you to associate ruby symbols to arbitrary “display” values, thus allowing your code to think symbolically and unambiguously while giving you the means to easily display what end-users need to see. Additionally, symbols are associated with ordinal values, allowing easy storage/retrieval to persistent stores (databases, files, marshalling, etc) by saving the  Element#value as appropriate (String by default, Fixnum if you “persist_ordinally”).

The primary aim of Elemental is to collect and abstract literals away from your code logic. There’s an old programmer’s wisdom that you should not encode your logic using literal values, especially those the end-user is exposed to.

Complete details provided in README.txt

http://github.com/mwlang/elemental/tree/master

http://rubyforge.org/projects/elemental/

From Gem:

sudo gem install elemental

From Source:

gem install bones, rake, test-unit
git clone git://github.com/mwlang/elemental.git
cd elemental
rake gem:package
cd pkg
sudo gem install elemental

Example of Using:
Here’s an example of how Elemental might be used with a Rails application. In your ~/config/initializers/elementals.rb file:

require 'elemental'

class PublicStatus
  extend Elemental

  member :unpublished,      :display => "Not Published", :default => true
  member :editor_approval,  :display => "Editorial Approval Needed"
  member :published,        :display => "Published"
  member :archived,         :display => "Archived"
end

class CommentType
  extend Elemental
  persist_ordinally

  member :all,        :display => "Anyone can post"
  member :moderated,  :display => "Moderated", :default => true
  member :closed,     :display => "Closed to new posts"
end

Create a “posts” table with this migration:

class CreatePosts < ActiveRecord::Migration
  def self.up
    create_table :posts do |t|
      t.string :title
      t.text :body
      t.string :public_status
      t.integer :comment_type

      t.timestamps
    end
  end

  def self.down
    drop_table :posts
  end
end

In your ~/app/views/posts/edit.html.erb:

<h1>Editing post</h1>

<% form_for(@post) do |f| %>
  <%= f.error_messages %>

  <%= render :partial => 'form' %>

  <p>
    <%= f.submit "Update" %>
  </p>
<% end %>

<%= link_to 'Show', @post %> |
<%= link_to 'Back', posts_path %>

Similarly for ~/app/views/posts/new.html.erb

<h1>New post</h1>

<% form_for(@post) do |f| %>
  <%= f.error_messages %>

	<%= render :partial => 'form' %>

  <p>
    <%= f.submit "Create" %>
  </p>
<% end %>

<%= link_to 'Back', posts_path %>

An form partial that utilizes the Elemental classes above to populate a pair of drop down lists:

<p>
	<label>Title</label><br />
	<%= text_field :post, :title, :value => @post.title %>
</p>
<p>
	<label>Body</label><br />
	<%= text_area :post, :body, :value => @post.body %>
</p>

<p><label>Public Status</label><br />
  <%= select "post", "public_status",
    PublicStatus.sort.map{|a| [a.display, a.value]},
      { :include_blank => false, :selected => PublicStatus.defaults.first.value }
  %></select>
</p>

<p><label>Comment Type</label><br />
  <%= select "post", "comment_type",
    CommentType.sort.map{|a| [a.display, a.value]},
      { :include_blank => false, :selected => CommentType.defaults.first.value }
  %></select>
</p>

An example ~/app/views/posts/index.html.erb file:

<h1>Listing posts</h1>

<table border=1 cellpadding=5px>
  <tr>
    <th>Post</th>
    <th>Public status</th>
    <th>Comment type</th>
    <th colspan=3>Actions</th>
  </tr>

<% for post in @posts %>
  <tr>
    <td><%= link_to post.title, post, :title => post.body %></td>
    <td><%=h PublicStatus[post.public_status].display %></td>
    <td><%=h CommentType[post.comment_type].display %></td>
    <td><%= link_to 'Show', post %></td>
    <td><%= link_to 'Edit', edit_post_path(post) %></td>
    <td><%= link_to 'Destroy', post, :confirm => 'Are you sure?', :method => :delete %></td>
  </tr>
<% end %>
</table>

<br />

<%= link_to 'New post', new_post_path %>

My System

2.6.18-92.1.22.el5 #1 SMP Tue Dec 16 11:57:43 EST 2008 x86_64 x86_64 x86_64 GNU/Linux

The Problem

If you simply execute:

sudo gem install mysql

You’ll get the following errors on a 64-bit system:

$ sudo gem install mysql
Building native extensions.  This could take a while...
ERROR:  Error installing mysql:
        ERROR: Failed to build gem native extension.

/usr/local/bin/ruby extconf.rb install mysql
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lm... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lz... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lsocket... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lnsl... yes
checking for mysql_query() in -lmysqlclient... no
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers.  Check the mkmf.log file for more
details.  You may need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/usr/local/bin/ruby
        --with-mysql-config
        --without-mysql-config
        --with-mysql-dir
        --without-mysql-dir
        --with-mysql-include
        --without-mysql-include=${mysql-dir}/include
        --with-mysql-lib
        --without-mysql-lib=${mysql-dir}/lib
        --with-mysqlclientlib
        --without-mysqlclientlib
        --with-mlib
        --without-mlib
        --with-mysqlclientlib
        --without-mysqlclientlib
        --with-zlib
        --without-zlib
        --with-mysqlclientlib
        --without-mysqlclientlib
        --with-socketlib
        --without-socketlib
        --with-mysqlclientlib
        --without-mysqlclientlib
        --with-nsllib
        --without-nsllib
        --with-mysqlclientlib
        --without-mysqlclientlib

Gem files will remain installed in /usr/local/lib/ruby/gems/1.8/gems/mysql-2.7 for inspection.
Results logged to /usr/local/lib/ruby/gems/1.8/gems/mysql-2.7/gem_make.out

After digging into the gem options and logs a bit and looking at where mysql actually got installed, I discovered that it was hitting all the locations the 32-bit version would normally install to, but not the 64-bit version, which ends up in /usr/lib64/mysql.

Installing the Ruby gems

To properly install the Ruby gems for mysql, you’ll have to explicitly tell the gem where the mysql config is and it will then handle all the rest.

gem install mysql -- --with-mysql-config=/usr/lib64/mysql/mysql_config

With that, you should see the following:

Building native extensions.  This could take a while...
Successfully installed mysql-2.7
1 gem installed

Check it out with:

irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'mysql'
=> true
irb(main):003:0> quit

Life is good.

The following Browser Edit form is what we’re going for. That is, having a list of Operating Systems to check off while editing a Browser object:

class CreateBrowsers < ActiveRecord::Migration
  def self.up
    create_table :browsers do |t|
      t.string :name
      t.string :version
      t.string :short_name
      t.timestamps
    end
  end

  def self.down
    drop_table :browsers
  end
end

class CreateOses < ActiveRecord::Migration
  def self.up
    create_table :oses do |t|
      t.string :vendor
      t.string :name
      t.string :version
      t.string :short_name
      t.timestamps
    end
  end

  def self.down
    drop_table :oses
  end
end

The Browser and Os table are very straightforward migration scripts. However, pay attention to the next script as we do a couple of interesting things. A) we tell the migration not to create the default ID column and B) we use the t.references declaration in defining our columns.

class BrowsersOses < ActiveRecord::Migration
  def self.up
      create_table :browsers_oses, :id => false do |t|
        t.references :browser
        t.references :os
        t.timestamps
      end
    end

    def self.down
      drop_table :browsers_oses
  end
end

Note that I used the singular version of the model names. I originally used the plural form and got browsers_id and oses_id instead of the expected browser_id and os_id column names.

Which brings me to another issue I encountered. “Os” didn’t pluralize properly, so I needed to add a custom inflector to the project’s config/initializers/inflections.rb file:

ActiveSupport::Inflector.inflections do |inflect|
  inflect.irregular 'os', 'oses'
end

For simplicity and clarity, I chose the irregular form since I didn’t figure on needing to match regular expressions against more complex model names.

Since there are many articles dedicated to the model side of things where relationships are concerned, I will skimp on discussing model implementations. Please see many-to-many-dance-off by Josh Susser or the has_and_belongs_to_many page on the Ruby on Rails wiki site for a couple of classics on the topic.

The models are straightforward, as shown below. Note that the browsers_oses table does not get its own model. Rails simply expects the table to be a concatenation of the two models pluralized and the model names in alphabetical order.

class Os < ActiveRecord::Base
  has_and_belongs_to_many :browsers
end

class Browser < ActiveRecord::Base
  has_and_belongs_to_many :oses
end

I’m not a big fan of the default views the rails scaffolding produces, but I do tend to run script/generate scaffold “model_name” to get the ball rolling quickly, anyway. So fire up a couple of views with:

script/generate scaffold browser
script/generate scaffold os

Note: I removed all previously generated files and regenerated the Os model and views after adding the irregular plural inflection rule described above.

Once I get my models and views, the first thing I do is create a form partial for each:

app/views/oses/_form.html.erb

<fieldset>
	<legend>OS</legend>
	<p>
		<label for="vendor">Vendor</label>
		<%= text_field :os, :vendor, :value => @os.vendor %>
	</p>

	<p>
		<label for="name">Name</label>
		<%= text_field :os, :name, :value => @os.name %>
	</p>

	<p>
		<label for="version">Version</label>
		<%= text_field :os, :version, :value => @os.version %>
	</p>

	<p>
		<label for="short_name">Short Name</label>
		<%= text_field :os, :short_name, :value => @os.short_name %>
	</p>
</fieldset>

There’s nothing fancy about the Os form as there’s no references to select drop-downs or the likes. With the above wrapped in a fieldset with legend of Os. I then tweak app/views/oses/edit.html.erb like so:

<h1>Editing OS</h1>

<% form_for(@os) do |f| %>
  	<%= f.error_messages %>
	<%= render :partial => 'form' %>

	<fieldset>
    	<%= f.submit "Update" %>
		<%= link_to 'Show', @os %> |
		<%= link_to 'Back', oses_path %>
  	</fieldset>
<% end %>

And similarly for app/views/oses/new.html.erb:

<h1>New OS</h1>

<% form_for(@os) do |f| %>
  	<%= f.error_messages %>
	<%= render :partial => 'form' %>

  	<fieldset>
    	<%= f.submit "Create" %>
		<%= link_to 'Back', oses_path %>
  	</fieldset>
<% end %>

app/views/oses/show.html.erb is nearly a cut-n-paste of the _form.html.erb file with the input tags stripped out:

<fieldset>
	<legend>OS</legend>
	<p>
		<label for="vendor">Vendor</label>
		<%= @os.vendor %>
	</p>

	<p>
		<label for="name">Name</label>
		<%= @os.name %>
	</p>

	<p>
		<label for="version">Version</label>
		<%= @os.version %>
	</p>

	<p>
		<label for="short_name">Short Name</label>
		<%= @os.short_name %>
	</p>
</fieldset>

<fieldset>
	<%= link_to 'Edit', edit_os_path(@os) %> |
	<%= link_to 'Back', oses_path %>
</fieldset>

It would be nice if the tag fields were action aware and intelligently generated inputs or text rendering according to whether new, edit, or show was the action, wouldn’t it??

The OsesController model is unchanged from the generated scaffolding code, so I won’t list its code here. Lets hop over to Browsers model and view as this is the model I do the most interesting things with. I will be rendering the Browser inputs so that one of the input is a checkbox list of all available OSes. To do this, I am effectively performing a “find all” on the Os model, so I DRY my code by adding the following method to the BrowsersController:

def get_all_oses
  @oses = Os.find(:all, :order => 'vendor, version')
end

Declaring small, single purpose methods is a good practice to be in the habit of. As it happened, I decided at some point to order the checklist by vendor and version. Instead of having to change code in four or five methods, I only had to update the one get_all_oses method. A tad bit of extra work up front pays off!

Similarly, compound names get their own method in the model class so I have ability to easily change without hunting down references everywhere in the view. So the Os model presented above becomes:

class Os < ActiveRecord::Base
  has_and_belongs_to_many :browsers

  def display_name
    "#{vendor} #{name} #{version}"
  end
end

And then I reference like so in the Browser view’s form partial:

<fieldset>
	<legend>Browser</legend>
	<p>
		<label for="name">Name</label>
		<%= text_field :browser, :name, :value => @browser.name %>
	</p>

	<p>
		<label for="version">Version</label>
		<%= text_field :browser, :version, :value => @browser.version %>
	</p>

	<p>
		<label for="oses">Operating Systems</label>
		<% @oses.each do |os| %>
			<%= check_box_tag(
                            "browser[os_list][#{os.id}]", 
                            "1", 
                            @browser.oses.detect{|d| d == os}) %>
			<%= "#{os.display_name}"%><br />
		<% end %>
	</p>

	<p>
		<label for="short_name">Short Name</label>
		<%= text_field :browser, :short_name, :value => @browser.short_name %>
	</p>
</fieldset>

A few important things to note here about the Operating Systems checklist:

• the @oses variable is a list of all Oses as per the get_all_oses method.
• I gave the checkbox name “browser[os_list][#{os.id}]” which causes the params to collect all checkboxes into an “os_list” hash that is contained in the browser keyword on the params hash. Keep this in mind because we’re going to leverage this in a moment.
• I don’t do anything with the values of the checkbox, so the value assigned (“1″ in this case) is of little concern.
• The @browsers.oses.detect{|d| d == os} line activates the checkbox if the browser already has the Os associated with it. (quite important for edit actions).

So the Operating Systems checklist is simply an iteration over all OSes, emitting a checkbox input control for each and setting the checked flag accordingly. So how do we handle this in the controller? We don’t! That’s right. Not in the controller, but in the model. This is “The Rails Way” as described in the Skinny Controller, Fat Model approaches championed by Jamis Buck and The Rails Way crew. My approach is to add a attribute accessor called “os_list” to the Browser model and a callback to the after_save event. So the Browser model now becomes:

class Browser < ActiveRecord::Base
  has_and_belongs_to_many :oses

  attr_accessor :os_list
  after_save :update_oses

  private 

  def update_oses
    oses.delete_all
    selected_oses = os_list.nil? ? [] : os_list.keys.collect{|id| Os.find_by_id(id)}
    selected_oses.each {|os| self.oses << os}
  end
end

So there’s a bit of magic! The update_attributes and save methods will assign the os_list hash from the params to our accessor “os_list” property automatically and thus provide us convenient access to the hash data in our callback handler. This is Rails at its best. Paul Barry gets the credit for being the first to put it together (at least for me).

What this method does is first clear out all the currently associated Oses. Then, if there are no selected Oses (in which case, items will be []), nothing happens with the loop, otherwise, each item is pushed onto the browser.oses list. When I first started writing Ruby on Rails apps, I was a stickler for not hitting the database unnecessarily and I would write lots of conditionals to avoid doing so.

However, the number of bugs I found in such logic eventually led me down the path of aggressively reducing lines of code in situations where it simply does not matter. Here, we’re talking about, at most, about 10 operating systems defined in the app I’m writing and I am very infrequently going to be adding new browsers to the list (about as often as a major version comes out as a rule of thumb). Is it really worth the extra code and the extra specs (you are writing test cases, aren’t you?) to gain so little? Rails is smart. It’ll issue a delete statement only if there are actually Oses in the list and it’ll only issue one DELETE statement, too! Check this session history out (for PostgreSQL):

  SQL (0.000153)   BEGIN
  SQL (0.000337)   DELETE FROM "browsers_oses" WHERE browser_id = 14 AND os_id IN (8,9)
  SQL (0.009701)   COMMIT
  SQL (0.000131)   BEGIN
  SQL (0.000194)   INSERT INTO "browsers_oses" ("browser_id", "updated_at", "os_id", "created_at") VALUES (14, '2008-10-24 19:33:43.107494', 8, '2008-10-24 19:33:43.107494')
  SQL (0.000649)   COMMIT
  SQL (0.000094)   BEGIN
  SQL (0.000191)   INSERT INTO "browsers_oses" ("browser_id", "updated_at", "os_id", "created_at") VALUES (14, '2008-10-24 19:34:01.088552', 9, '2008-10-24 19:34:01.088552')
  SQL (0.000570)   COMMIT
  SQL (0.000086)   BEGIN
  SQL (0.000199)   INSERT INTO "browsers_oses" ("browser_id", "updated_at", "os_id", "created_at") VALUES (14, '2008-10-23 20:24:59.809242', 2, '2008-10-23 20:24:59.809242')
  SQL (0.000657)   COMMIT
  SQL (0.000076)   BEGIN
  SQL (0.000110)   COMMIT
Redirected to http://localhost:3000/browsers/14
Completed in 0.08188 (12 reqs/sec) | DB: 0.01585 (19%) | 302 Found [http://localhost/browsers/14]

Now, if you’ve got a situation where 100’s of users are performing a similar action and the small doses truly add up to something significant in your daily logs, then definitely optimize. Just be cognizant of your use-case and profile your application for performance before getting fancy with the conditional updates, otherwise you’re in the trap of premature optimization and not getting something else done you might have wish you were!

Ok, having digressed a bit, lets get back on course. So what does the Browser Controller now look like? Well, the only thing I have to add is a call to “get_all_oses” just in case of a failure to save and the redirect back to the same page occurs. So this is what we have:

def get_all_oses
  @oses = Os.find(:all, :order => 'vendor, version')
end  

# POST /browsers
# POST /browsers.xml
def create
  @browser = Browser.new(params[:browser])
  get_all_oses

  respond_to do |format|
    if @browser.save
      flash[:notice] = 'Browser was successfully created.'
      format.html { redirect_to(@browser) }
      format.xml  { render :xml => @browser, :status => :created, :location => @browser }
    else
      format.html { render :action => "new" }
      format.xml  { render :xml => @browser.errors, :status => :unprocessable_entity }
    end
  end
end

# PUT /browsers/1
# PUT /browsers/1.xml
def update
  params[:oses] ||= {}
  @browser = Browser.find(params[:id])
  get_all_oses

  respond_to do |format|
    if @browser.update_attributes(params[:browser])
      flash[:notice] = 'Browser was successfully updated.'
      format.html { redirect_to(@browser) }
      format.xml  { head :ok }
    else
      format.html { render :action => "edit" }
      format.xml  { render :xml => @browser.errors, :status => :unprocessable_entity }
    end
  end
end

One last trick for the eagle eyes: The form image I posted positions the labels in bold above the inputs and with a trailing colon, but nary a <BR> element (or colons) to be seen in the views. I accomplished this with simple CSS styling by switching the label element to a block element (rather than inline as is the default) and then providing the colon as additional content:

label {
	display: block;
	font-weight: bold;
}
label:after {
	content: ':';
}

So there you have it. Leveraging the Rails framework appropriately, you can easily provide a checklist generated from one model on another’s edit form and have those records maintained in a many-to-many relationship without bloating your views or controllers.