What was tested?

There are two things I have grown to love since switching from Rails to Ramaze and that’s Sequel as the object relational mapper (ORM) and Erubis as the templating engine. So these are exactly the two components you will find in both of these tests. Fortunately, both frameworks make it exceedingly easy to utilize these components whereas I had to jump through a few loops to successfully test ActiveRecord in Ramaze against Rails then-only choice as an ORM.

The main goal in this benchmarking effort is to compare apples to apples and I tried to control for that at every level in the stack from deployment method and worker threads to as close as possible code and computations in rendering each page. Although the applications remain ridiculously simple, they do test a variety of things most developers would do in a typical app in terms of configuring a sane rendering engine and layout that pulls content from an action’s template. Some miscellaneous computations were done for each page such as globally set titles and searching for “posts” to render.

All project sources and the data collected and reported herein is available in the 2010 folder on github.com. I am not entirely sure what the Padrino folks did to control for variables in their benchmarking tests, but here’s what I did:

1. Same hardware: Macbook Pro 2.33 GHz Intel Core 2 Duo with 4GB RAM, OS X 10.5.8.
2. Deployed via Phusion Passenger 2.2.11 on Apache 2.2.14 with six workers running both in live/production mode (using each framework’s default for these environments).
3. Same Ruby MRI: ruby 1.8.7 (2009-06-12 patchlevel 174) [i686-darwin9] for both.
4. Sequel as the ORM, Erubis as the rendering engine.
5. Nearly identical page and layout templates (varying only by the name “Ramaze” and “Padrino” in the title tags).
6. A representive set of identical computations in each project (current time, globally set title, helpers that generate links, etc.
7. Identical sqlite3 data file with identical model declaration and querying through Sequel.

I utilized this simple call to Apache Bench to run through the two test scenarios:
All data collected was based on 10,000 requests at 100 concurrent users after an initial “warmup” of the passenger threads with start of 100 requests at 10 concurrent users. The following command shows the basic set of parameters issued to Apache Bench:

ab -n 10000 -c 100 http://padrino.macdoze.local/

The Results

The following table tabulates the results gathered from Apache Bench as well as passenger-memory-stat. I ran a short run to warm things up (100 requests, 10 concurrently). And then ran the big bench at 10,000 requests, 100 concurrently, and then ran again and recorded the results. At around 5,000 requests, I ran passenger-memory-stat to collect the data during mid-processing. Swap space was never triggered (0k utilization) during these tests, and CPU utilization was roughly 75% during the test runs.

Throughput

Everybody likes to talk about throughput, so lets start there. As you can see, Padrino processed the requests 1.34 times faster than Ramaze. In other words, Padrino is pushing bytes at about 25% faster than Ramaze is. Be forewarned that throughput doesn’t necessarily equate to user response times. This metric simply tells you the load at which your system is capable of delivering content at.

Time per Request

Time per request helps you see about how long an average user request takes for a given load. As you can see here, with times in the milliseconds, we’re talking about numbers that will easily be swallowed in the noise of network latency. Even so, we are looking at a similar performance ratio as with the throughputs above.

Memory Consumption

This final graph shows a surprising difference in memory consumption between the two micro-frameworks. Here, Ramaze trumps Padrino’s memory footprint, utilizing only 63% of the memory that Padrino requires. This means that you can run more workers per server using Ramaze than you can with Padrino before you consume all the RAM on the server.

Conclusion

Padrino has certainly set the new standard for raw throughput performance in Ruby-based web frameworks, but don’t be deceived into thinking its blowing its competition away by leaps and bounds. If you’re one of those shops that is fortunate to have an infinite supply of hardware to throw at the problem, then Padrino makes a good choice for maximizing throughput per server. I know I will certainly strongly consider utilizing Padrino in a couple of up-coming projects.

I have found that one area Padrino is shining where Ramaze is not quite there yet, is an excellent website with just the right information to get started developing new projects. I was really impressed with Sinatra’s approach to named routes with the RESTful inspired get/post/put/delete DSL that is further enhanced by Padrino’s helpers. The recent Padrino enhancements to their router helpers for “mounting” more than one application is a bit more intuitive to me, especially with the ample documentation on just how to configure for all the most common routing scenarios than Ramaze’s. I still have not quite implemented mulitiple mini-apps under Ramaze despite multiple attempts.

Ramaze has been around long enough that it has a sizable body of legacy/obsolete documentation floating around the ‘net that you need to wade through before finding the right documentation that’s relevant to the post Ramaze/Innate split/re-architecture. The documentation’s certainly there for Ramaze, but if you don’t know exactly where to look, or your Google-fu is suffering during your time of need, the only place to find it is by resorting to reading the source or hoping someone on the #Ramaze IRC readily finds it on your behalf (which they’re pretty darn good at doing).

Padrino also differs itself from Ramaze and following the Rails camp a little more by introducing an opinionated project structure and set of helpers that is well-documented and easy to find. Ramaze, on the other hand, strives hard not to push opinions onto the developers on how they choose to structure their projects. Because of Padrino’s choices, they’re also able to provide a nice set of well-rounded generators and rake tasks to give you a much nicer “out of box experience” especially suitable for the Rubyist that is still learning his way around the Ruby language. I would love to see the Ramaze community come together to generate an equally strong set of documentation and best-practices rolled into a few good generators to help the aspiring developer get off the ground with Ramaze a lot faster from the get-go.

Both Ramaze and Padrino have strong technical merits, and as you can see from the benchmarks above, both are quite performant platforms in their own right with Padrino having the edge in speed and Ramaze having the edge in memory consumption. For developers like me churning out many different small websites for various customers and running all of those websites off one physical box, Ramaze continues fill my needs in that corner with its smaller memory consumption footprint. 1.3 microseconds for a collection of customers registering 25 to 100 page views a day each isn’t enough to of a difference to concern myself with a wholesale switch in platforms. However, I definitely see a bright future for Padrino in my toolbox as I endeavor to build collections of reusable mini-apps that can be mounted in many easily configurable combinations for each deployment.

I have been doing some data loading from one database into another using Pentaho’s Kettle and that got me curious about JRuby since Pentaho’s toolset is built upon Java. Quite simply, my work with Kettle got me wondering, “why not JRuby?” I’ve been hearing some great things about JRuby and how it greatly leverages the Java stack to bring you all manner of improvements over Ruby MRI, notably scalability on any one of the many Java deployment container servers and native-system threading capabilities.

I didn’t pull down the latest and greatest JRuby libraries to my mac. I simply got what I could get via macports:

sudo port install apache-ant
sudo port install jruby

I don’t know if the apache-ant port was actually needed or not, but I’d read that JRuby depended on ant to compile its components, so, fast as I could type the two port commands, and watch the terminal session actually install both, I was done. The above got me jruby 1.4.0 (ruby 1.8.7 patchlevel 174) (2010-04-22 6586) (Java HotSpot(TM) Client VM 1.5.0_22) [i386-java] and I was ready to play.

The JDBC Drivers

What I was really curious to find out is whether JRuby and sequel was all that hard to get set up and working with Sql Server 2000 MSDE and MySQL. Both proved fairly simple to get working, although I had to patch Sequel 3.10.0 to get MySQL working with JDBC.

Microsoft SQL Server JDBC Driver

I was impressed with the Microsoft SQL Server JDBC Driver 3.0. It worked as advertised and was pretty straightforward to install, although I had to Google a bit to find out *how* since wading through the help docs didn’t help me. I found out that simply putting the jar file in JRuby’s lib folder was all that was needed. If you use port to install JRuby as I did above, then that folder on your mac is:

/opt/local/share/java/jruby/lib

The latest GA version of the SQL Server JDBC 3.0 driver at the time of this writing was version 3.0.1301.101.

MySQL JDBC Driver

MySQL’s JDBC driver was equally easy to download and copy the jar to the above referenced JRuby lib folder. The latest GA driver version at the time of this writing was 5.1.12. One small change I had to make to Sequel 3.10.0 was to add the Statement.RETURN_GENERATED_KEYS option to the executeUpdate call in Sequel’s adaptors/jdbc.rb. The following thread on sequel-talk covers this.

The Script

To find out how JRuby and Ruby MRI stacked up, I wrote a script, available on github.com, that generates 20,000 rows of fake data that might be typical of what you’d find on an invoice/order table in an e-commerce site. I then clone this data to another table. Both of these actions are fairly heavy I/O operations and the JRuby page on benchmarking warns this will be a bit slower than MRI. The test results tend to bear this out. The real-world scenario where I am doing this sort of cloning is copy and consolidate data from various databases into a data-warehouse setting from production transactional systems.

The Results

Running the script in Ruby MRI and in JRuby, I found JRuby to be generally slower than Ruby MRI but still roughly on par when subjected to the usual single threaded approach to writing Ruby code that most developers use. However, JRuby definitely showed some marked improvements when you took the time to craft a threaded approach to the problem while Ruby MRI took a performance hit with a threaded approach. Unfortunately, boosting the thread count from 4 to 8 did almost nothing, so there’s some practical level to how many threads can be utilized in a scenario like this. I did notice during the test runs that my CPU and disk and network I/O never quite maxed out although the JRuby in server mode seemed to to push things the hardest with CPU hovering at about 146% (200% max given core 2 duo Intel).

None of these numbers inspire me to use Ruby or JRuby for any heavy lifting any time soon. Pentaho’s Kettle may sway me to use a Java solution as it can scale a bit further with appropriate resources than my current solution of Delphi compiled executable running on Windows XP. The Delphi-based solution can churn through about 2 million rows of data in 15 minutes. These solutions in Ruby, by comparison would take roughly 8 to 14 hours to churn through the same 2 million rows of data.

>> jruby -S test.rb
      user     system      total        real
populate sample_data table: creating the sample_data table...
populating sample_data with 20000 rows
sample data:   100% |oooooooooooooooooooooooooooooooooooooooooo| Time: 00:01:50
111.085000   0.000000 111.085000 (111.085000)

clone sample_data to cloned_data: creating the cloned_data table...
cloning 20000 records for sample_data
sample_data:   100% |oooooooooooooooooooooooooooooooooooooooooo| Time: 00:01:03
 63.268000   0.000000  63.268000 ( 63.267000)

threaded clone: cloning 20000 records for sample_data
sample_data:   100% |oooooooooooooooooooooooooooooooooooooooooo| Time: 00:00:37
 38.000000   0.000000  38.000000 ( 38.000000)

>> jruby -S test.rb --server
      user     system      total        real
populate sample_data table: creating the sample_data table...
populating sample_data with 20000 rows
sample data:   100% |oooooooooooooooooooooooooooooooooooooooooo| Time: 00:01:26
 86.989000   0.000000  86.989000 ( 86.989000)

clone sample_data to cloned_data: creating the cloned_data table...
cloning 20000 records for sample_data
sample_data:   100% |oooooooooooooooooooooooooooooooooooooooooo| Time: 00:00:55
 55.652000   0.000000  55.652000 ( 55.652000)

threaded clone: cloning 20000 records for sample_data
sample_data:   100% |oooooooooooooooooooooooooooooooooooooooooo| Time: 00:00:30
 31.070000   0.000000  31.070000 ( 31.070000)

>> ruby test.rb
      user     system      total        real
populate sample_data table: creating the sample_data table...
populating sample_data with 20000 rows
sample data:   100% |oooooooooooooooooooooooooooooooooooooooo| Time: 00:01:45
 85.370000   4.460000  89.830000 (105.049892)

clone sample_data to cloned_data: creating the cloned_data table...
cloning 20000 records for sample_data
sample_data:   100% |oooooooooooooooooooooooooooooooooooooooo| Time: 00:00:47
 31.700000   3.500000  35.200000 ( 47.930685)

threaded clone: cloning 20000 records for sample_data
sample_data:   100% |oooooooooooooooooooooooooooooooooooooooo| Time: 00:00:56
 39.450000   3.760000  43.210000 ( 57.226705)

I won’t cover a complete Ruby and Rails/Ramaze/Sinatra installation. This post assumes you have that much down pat, so we’ll pick up the ball with installing FreeTDS, unixODBC, Ruby ODBC

If you’re using Ubuntu 8.10, you can get away with installing everything from packages with the following:

apt-get install  unixodbc freetds-common tdsodbc \
             libodbc-ruby1.8 \
             libdbi-ruby1.8 libdbd-odbc-ruby \
             autoconf make

NOTE: I wrote this article and right before publishing, attempted to replicate these results on both mac OS X and CentOS 5. Both failed to work with the technique I thought I had all nice and worked out. I spent several hours over many months trying to find out exactly what’s different between the packages, but never got very far with my investigations. For a long time, it looked to me like both the ODBC packages and the FreeTDS packages need to be tweaked to properly handle [SERVER]\[DB_INSTANCE] construct that MSDE uses vs. just [SERVER] that the full-fledged SQL Server uses. This, after all, is the convention of the Windows ODBC driver and DSN definitions. However, there’s a really simple solution that I hit upon on Actual Technologies’ website that led me to a true universal solution and that’s to learn the port the INSTANCE runs on and set up the DSN specifically to that port!

But beware, the above doesn’t get you all of the freetds tools, most importantly the tsql command which can help you with debugging issues before getting into the ODBC game (that is, direct TDS connections to your SQL Server). So, to cover the most distros possible, I’ll document the “from source” approach for *nix systems and macports for Macs so you’ll find two sets of installation instructions for every step.

Preparing your Environment

I prefer to pull down all source packages into my local user directory under a “~/src” folder. The instructions that follow assume you’re in this folder. If you’re not, then please make the appropriate substitutions as you follow along. Additionally, I configure and compile most things as a regular user then use sudo to install system-wide as a “super user.”

Install unixODBC

unixODBC provides the ODBC connectivity for the TDS driver.

wget http://www.unixodbc.org/unixODBC-2.2.14.tar.gz
tar zxfv unixODBC-2.2.14.tar.gz
cd unixODBC-2.2.14
./configure  --enable-gui=no
make
sudo make install

On Macs

sudo port install unixODBC

On CentOS

sudo yum install unixODBC unixODBC-devel

Install FreeTDS

FreeTDS provides the open source implementation of the protocol implemented by both Sybase and Microsoft SQL Server.

wget ftp://ftp.ibiblio.org/pub/Linux/ALPHA/freetds/stable/freetds-stable.tgz
tar zvxf freetds-stable.tgz
cd freetds-0.82/
sudo ./configure  --with-unixodbc=/usr/local
sudo make
sudo make install

On Macs

sudo port install freetds

On CentOS

sudo yum install freetds

Install ruby-odbc

The ruby-odbc package provides the ruby bindings for ODBC connectivity.

wget http://www.ch-werner.de/rubyodbc/ruby-odbc-0.9995.tar.gz
tar zvxf ruby-odbc-0.9995.tar.gz
cd ruby-odbc-0.9995/
ruby extconf.rb
make
sudo make install

On Macs

sudo port install rb-odbc

Install ruby DBI

wget http://rubyforge.org/frs/download.php/655/ruby-dbi-all-0.0.23.tar.gz
tar zvxf ruby-dbi-all-0.0.23.tar.gz
cd ruby-dbi-all
ruby setup.rb config --with=dbi,dbd_odbc
ruby setup.rb setup
sudo ruby setup.rb install

Configure FreeTDS and ODBC

If you’re connecting to either SQL Server 2000 or SQL Server 2005 Standard, Enterprise, etc., then the typical approach is to add an entry for the server to /etc/freetds/freetds.conf and then reference that entry in the /etc/odbc.ini. Sometimes, these files get installed in /usr/local/etc. If so, then I will usually symlink the files into /etc for convenience, so that’s what’s referenced below.

My server is called “apvdbs01″ (the WINS name) with an IP Address of 10.0.2.2 and the database I’m connecting to is called “test_development” while “username” and “passwd” should be substituted with an valid user account and credentials for successfully connection to the database server.

Add a server entry for your server (which I typically give same name as Windows Server) to /etc/freetds.conf, so that host points to the SQL Server address:

[apvdbs01]
host = 10.0.2.2
port = 1433
tds version = 7.0

TDS version 7.0 is for SQL Server 2000. If you have SQL Server 2005, then tds version is 8.0.

tsql –S apvdbs01 –U username –P passwd

Once you can connect directly with the TDS drivers, move on to adding the ODBC driver entries.

Add the FreeTDS ODBC driver by editing “/etc/odbcinst.ini” (make sure “/usr/local/lib/libtdsodbc.so” exists first):

[FreeTDS]
Description     = TDS driver (Sybase/MS SQL)
Driver          = /usr/lib/odbc/libtdsodbc.so
Setup           = /usr/lib/odbc/libtdsS.so

Add a new DSN to “/etc/odbc.ini”:

[test_development]
Driver          = FreeTDS
Description     = ODBC connection via FreeTDS
Trace           = No
ServerName      = apvdbs01
Database        = test_development

So, “FreeTDS” is the driver that we gave for the lib TDS ODBC connector in the odbcinst.ini file (this allows you to install and maintain multiple versions of the libtdsodbc.so file). We reference this driver in the odbc.ini file with the “Driver” entry and we reference the actual server data in the freetds.conf file via the “ServerName” entry.

Now that you have ODBC configured for FreeTDS, test it with:

	isql test_development username passwd

But that doesn’t work for MSDE!

The above will get you connected to a standard or enterprise SQL Server 2000/2005 installation, but does not work for an Embedded SQL Server 2000/2005 installation. After trying many permutations, I finally came up with a configuration that actually works. One of the key things to know about MSDE is that the MSDE server instance is in the form of “servername\instance” rather than straight up “servername” as with standard/enterprise installations. This is known as a “named instance.” The trick to connecting? Find out the port the named instance is listening on and connect to that port rather than the 1433 port!

You can determine the port used by the instance by using the Server Network Utility in the Microsoft SQL Server program group on the Windows server where your database resides. Select the instance name from the drop down list and select TCP/IP from the list of enabled protocols. This will show you the port number for that instance.

[test_msde2000]
Driver         = FreeTDS
Description    = ODBC connection via FreeTDS
Server         = 10.0.2.2
Database       = my_test_db
Port           = 1348
TDS_Version    = 7.0

I could then connect to the MSDE instance like so:

isql test_msde2000 username passwd

Test ODBC connectivity from Ruby

If both tsql and isql commands work, then you’re well on your way to connecting to SQL Server via Ruby. To test Ruby with the above ODBC configurations:

irb
require "dbi"
dbh = DBI.connect("dbi:ODBC:test_development", "username", "passwd")

Connecting with Sequel

One of my favorite tools for connecting to SQL DBMS’ is Sequel, which is maintained by Jeremy Evans. To connect using Sequel, install the gem:

sudo gem install sequel

And then give this a shot:

irb
require 'rubygems'
require 'sequel'

DB = Sequel.odbc('test_development', :db_type => 'mssql', :user => 'username', :password => 'passwd')

References

http://www.freetds.org/userguide/confirminstall.htm

http://www.unixodbc.org/odbcinst.html

http://www.actualtech.com

Its not that I don’t think these tools are worthy contenders. Most of them, I have used fairly extensively myself over the years. The suggested contenders are solid and mature projects, but they feel decidedly “old hat” to me much as apache, postfix, vim, bind, and the linux kernel probably feels to the system administrator. In other words, they’re ubiquitous, so why signal them out with some sort of top 100 reward?

These days, I have adopted Ruby as my language of choice, so I have moved away from Python and PHP. While this has largely meant I develop Ruby on Rails applications. However, as my Ruby skills grow, this also means going far and wide in many other ruby-based tools (such as rake, sake, and rack for system-level or middleware tools). I am now using ruby for quick and dirty systems administration tasks (such as the short script I wrote to find all currently assigned IP addresses on our office network), automating deployments (capistrano, puppet, chef), marshaling data from one database system to another, and even with extensive image-processing (resizing, conversion of formats, compressing, etc.).

So what, in my view, warrants a top 100 award recognition? I like to see new and exciting projects on these lists. If a project’s on the list, I like to know that they are extremely active and rapidly gaining mind-share and penetration into the community. I want to know a little bit about how they put a new spin on an old problem and the impact they have been able to make, not only in the project’s direct vertical niche, but across the board. A few years ago, Ruby on Rails arrived on the scene as a game-changer, not only for putting Ruby on the map as a viable language, but as a bona fide development framework for delivering web-based services. Ruby on Rails turned the Java world on its head and forced Java developers to drastically improve their productivity tools then in common practice; they evangelized convention over configuration and brought many fantastic ideals such as Agile development and RESTful APIs to the forefront of many, many developers’ minds.

To my way of thinking, though, Ruby on Rails is now old news. Newcomers will stumble upon Ruby and Rails through word of mouth or simply googling for a solution to some problem they’re trying to solve. That’s the problem I see with top-100 lists paying tribute year after year to the same old mature projects. Whether its Ruby on Rails or Word Press or Joomla or Django, all of these are easily discovered and well-written about by anybody with decent querying skills. So what’s out there today that’s refreshing and hard to find? Yet, is making a huge impact on the community? Here are a few that I think are worthy contenders (and yes, they are a little heavy on the Ruby side)…

Rack

Rack: a Ruby Webserver Interface. What I find phenomenal about Rack is its adoption by so many frameworks that are delivering web-based services. Micro-frameworks like Ramaze and Camping and Sinatra were early adopters of Rack and such adoption led to far smaller code-bases in each as well as significant performance boosts and, as if that wasn’t enough, enabled much easier transition from Ruby 1.8 to 1.9.

Inkscape and ImageMagick

Speaking of image processing, I am not sure if many have encountered ImageMagick and the Ruby bindings offered via RMagick, but definitely worth a look-see. As of this year, I do almost all of my creative graphics work in Inkscape and The Gimp and use ImageMagick to automate some pretty impressive transformations of those images outside the GUI. Before Inkscape became mature enough to use on daily basis (which, by my reckoning is only in the last 18 months or so), I was heavily reliant on Adobe’s Illustrator and companion Photoshop apps. I never would’ve imagined two years ago that I could completely do away with these packages, let alone begin to automate much of the repetitive tasks that I put those resulting images through (resizing, cropping, web-optimizing, etc.) with a collection of well-written OSS tools.

Ramaze and Sequel

One last pair of tools I think is worthy of consideration, especially for web-based ruby developers is Sequel and Ramaze. Sequel is an object-relational-mapping library that makes working with SQL databases a real pleasure for Ruby developers. The maintainers seem to have taken the best of the best and rolled those ideas into one coherent and extremely fast library for accessing just about any database out there. I have fooled around with DataMapper and have worked extensively with ActiveRecord and I must say Sequel really is rung or two above these libraries and I look forward to continuing to adopt Sequel.

I ran into Sequel by way of the Ramaze project, which is a micro-framework for building web-based applications. Unlike Ruby on Rails, which tends to generate tons of scaffold/skeletal code for you to begin hacking away on to build your projects, Ramaze takes the opposite tack in that it provides little to no scaffolding or auto-generated code. Instead, the mindset is squeezing the maximum amount of functionality out of every line of code you do write. For a Ruby on Rails developer coming to Ramaze, the feeling is very much like the first time I took off on a bicycle without the training wheels attached. You feel wildly uncontrolled and like you may fall at any moment, but a little encouragement from the warm friendly community gets you going straight and true and the amount of code you’re NOT writing, reading, reviewing, nor specking tests for really begins to jump out at you!

Introducing Ramaze

One I found is Ramaze. I have to say that Ramaze is a very minimalist approach to a framework that is actually rather enlightening. For one coming from a Rails’ world, it can feel a bit sparse at first. But lately, I’m attributing my lack of comfort to being put on my toes to do some actual real Ruby coding. Its both a little scary and liberating at the same time. I have found that I am enjoying Ruby a lot more as a result of my exposure to Ramaze and my thought process is actually radically changing, almost like a Visual Basic programmer going to Borland Delphi might feel.

So I decided to take a hard look at performance metrics between Rails and Ramaze’s. At the same time, I wanted to see how MRI 1.8.7 stacked up against Ruby Enterprise Ed. 1.8.6, so I’m satisfying a dual curiosity herein. Being quite happy with Phusion Passenger, I decided to set things up with Apache 2.2.9 + Passenger + Rails/Ramaze. Since I’m quite comfortable with ERB and ActiveRecord, I tweaked Ramaze to be a bit more like the familiar Rails world I am familiar with.

Un-DRYing the Results

There’s no point in generating a lot of data and reporting on it if others cannot repeat yourself (CRY!) in replicating the results, so with that in mind, I attempt to refrain from further bad turn of words by busying myself in pushing everything I utilized to github.com and documenting the basic bootstrapping of Ubuntu Intrepid. For these tests in particular, I used two scripts, bootstrap-passenger-std.sh and bootstrap-passenter-ent.sh to set up four distinct configurations under the Ruby1.8 interpreter. Ruby 1.9.1. was a bit difficult to set up and there is a bootstrap-passenger-191.sh script that I assembled as I installed things, but I haven’t run it to ensure it works end-to-end.

I utilized this simple call to Apache Bench to run through the four test scenarios:
All data collected was based on 10,000 requests at 100 concurrent users after an initial “warmup” of the passenger threads with start of 100 requests at 10 concurrent users. The following command shows the basic set of parameters issued to Apache Bench:

ab -n 10000 -c 100 http://demo.u64rails01.local/posts

What was tested?

In order to compare, as close as possible, apples to apples, I opted to use Active Record and ERubis in Ramaze to set up the same MVC classes in both the Rails and Ramaze project. ActiveRecord is probably the heaviest weighted ORM that can be utilized with Ramaze. I tried briefly to get Erubis working with Rails so that the two frameworks’ projects would be more closely aligned than the out-of-box ERB rendering Rails ships with would have afforded. I did attempt Erubis with Rails, however, I was either doing something very wrong or Erubis under Rails takes a sizable performance hit and I stuck with out-of-box ERB for these tests given its better performance. As such, Rails uses out of box ERB rendering in these tests.

The application itself is ridiculously simple, offering only one model and performing only a read against the database, but hopefully, an effective start towards building this benchmark framework towards more substantive benchmark tests to come. For now, lets take a look at how Rails and Ramaze stack up to each other and what affect the underlying interpreter can have on performance. Standard packaged Main Ruby Interpreter (MRI) 1.8.7 consisted of simple package install provided by the Ubuntu Intrepid repositories whle Ruby Enterprise Edition (REE) 1.8.6, which hails from the same folks bringing you Phusion Passenger, is compiled from source.

As a bonus, I also compiled and clocked metrics for Ramaze on 1.9.1. This was a tricky environment to get set up and working. And my initial results made me think I did something horribly wrong with compiling and configuring Ruby 1.9.1. However, I fully replicated Antonio Cangiano’s benchmarks in my test environment (see The Great Ruby Shootout (December 2008)) and MRI 1.9.1. did indeed crush the other Ruby interpreters in Fibonacci number crunching (amongst others). The theory tossed around in the Ramaze IRC channel was that 1.9.1. change in implementation of the String class (and it being unicode native now) slows things down a good bit with string processing.

The Results

The following table tabulates the results gathered from Apache Bench as well as passenger-memory-stat. I ran a short run to warm things up (100 requests, 10 concurrently). And then ran the big bench at 10,000 requests, 100 concurrently, and then ran again and recorded the results. At around 5,000 requests, I ran passenger-memory-stat to collect the data during mid-processing and then again at the end. These tests were run on a Ubuntu 64-bit Intrepid 8.10 virtual server that was allocated 1 CPU core and 512MB of RAM. Swap space was never triggered (0k utilization), and CPU utilization was roughly 85% within the Virtual Machine during the test runs. The host machine, which is four core CPU, showed one core busy at near 100% with the other three cores pretty much idle.

Throughput

Everybody likes to talk about throughput, so lets start there. As you can see, MRI and Rails was the worst performer while REE and Ramaze clocked in with the highest throughput in terms of requests per second. Be forewarned that throughput doesn’t necessarily equate to user response times. This metric simply tells you the load at which your system is capable of delivering content at.

Time per Request

Time per request helps you see about how long an average user request takes for a given load. As you can see here, with times in the milliseconds, we’re talking about numbers that will easily be swallowed in the noise of network latency. Even so, its not hard to be impressed by what the REE folks managed to pull out of their hat with regards to performance on the main ruby interpreter.

Memory Consumption

This final graph shows the sizeable dent REE makes in memory consumption for running a Rails application. Ramaze is already pretty efficient and lean and REE made practically no dent in memory consumption here. However, REE’s use of the faster memory allocation routines does have a good effect on boosting Ramaze’s performance (as seen in above graphs), so there’s still lots to be gained by running Ramaze with REE.

Conclusion

For me, I have two very, very viable solutions before me. Rails on Enterprise Ruby or Ramaze on Ruby 1.8.7. The final choice comes down to how much I can bet on my own Ruby prowess. Ramaze is a very lean framework that gets you going, but doesn’t get you a nine inch pillowed mattress topping for your bed like Rails does. Because Ramaze is so lean, I realize I will have to really take my Ruby skills to the next level and truly understand the ruby code that other developers produce. Unlike Rails, where there’s gobs and gobs of plugins and blogs on the topic that make pretty much any need simple to satisfy and get working, I have to know my Ruby to get a solution implemented on Ramaze. Not a bad thing, but certainly a more challenging and potentially rewarding path to take.

Aside from my personal concerns about making the jump, looking at these results, it is good to see that I have options and how easy it is to get a good deployment up and working compared to 3 or 4 years ago when I was banging my head against the wall for days on end to keep lighttpd + fcgi processes up and running around the clock. In these tests, it was very, very satisfying that not one single crash or dropped request was recorded in all the benchmarking I performed. Mongrel came along and made a big impact, but you still needed something like monit to watch Mongrel and restart it on ocvasion. Now Passenger has come along and made yet another big impact in the ease of deploying Rails, Ramaze, or any of the other ruby-based frameworks. We’ve come a long ways, indeed. I, for one, certainly look forward to what the Ruby sphere will bring us over the next couple of years.

Another drawback I have with Google Syntax highlighter is that the page renders without highlighting and then all the code listings change to syntax highlighted text a few moments later.

Introducing Ultraviolet, a syntax highlighter supporting over 50 syntaxes and any TextMate theme.  Ultraviolet is based on Oniguruma, a regular expression library that comes standard with Ruby 1.9. Ultraviolet requires a little more effort for me to cut and paste, but I am definitely much happier with the end results and Wordpress no longer mungs my posts. Only real complaint I have is that I can’t read the posted code inline in HTML format (but I can read it in Visual mode!).

Ruby Script

   1  require 'rubygems'
   2  require 'ramaze'
   3  require 'maruku'
   4  require 'sequel'
   5 
   6  $LOAD_PATH.unshift(__DIR__)
   7 
   8  require 'env'
   9 
  10  require 'vendor/flickr'
  11  require 'vendor/haml_maruku_filter'
  12  require 'vendor/image_science_cropped_resize'
  13 
  14  require 'model/init'
  15  require 'controller/init'
  16 
  17  require 'db/init' if SOCIAR.mode == :dev
  18 
  19  r = SOCIAR.ramaze
  20 
  21  =begin
  22  if r.gzip
  23    require 'ramaze/contrib/gzip_filter'
  24    gzip = Ramaze::Filter::Gzip
  25    gzip.trait :threshold => 1
  26    Ramaze::Dispatcher::Action::FILTER << gzip
  27  =end
  28 
  29  Ramaze.start :adapter => r.adapter,
  30               :host => r.host,
  31               :port => r.port,
  32               :boring => r.boring

Shell Script

   1  #!/bin/sh
   2 
   3  ### BEGIN INIT INFO
   4  # Provides:          lm-sensors
   5  # Required-Start:    $local_fs
   6  # Required-Stop:
   7  # Default-Start:     S
   8  # Default-Stop:
   9  # Short-Description: lm-sensors
  10  # Description:       hardware health monitoring
  11  ### END INIT INFO
  12 
  13  . /lib/lsb/init-functions
  14 
  15  [ -f /etc/default/rcS ] && . /etc/default/rcS
  16  PATH=/bin:/usr/bin:/sbin:/usr/sbin
  17  PROGRAM=/usr/bin/sensors
  18 
  19  test -x $PROGRAM || exit 0
  20 
  21  case "$1" in
  22    start)
  23  	log_begin_msg "Setting sensors limits"
  24  	if [ "$VERBOSE" = "no" ]; then
  25  		/usr/bin/sensors -s 1> /dev/null 2> /dev/null
  26  		/usr/bin/sensors 1> /dev/null 2> /dev/null
  27  	else
  28  		/usr/bin/sensors -s
  29  		/usr/bin/sensors > /dev/null
  30  	fi
  31  	log_end_msg $?
  32  	;;
  33    stop)
  34  	;;
  35    force-reload|restart)
  36  	$0 start
  37  	;;
  38    *)
  39  	log_success_msg "Usage: /etc/init.d/sensors {start|stop|restart|force-reload}"
  40  	exit 1
  41  esac
  42 
  43  exit 0

Apache Config

   1  <VirtualHost *:80>
   2  	ServerName www.stinger.local
   3  	ServerAdmin webmaster@localhost
   4 
   5  	DocumentRoot /var/www/
   6  	<Directory />
   7  		Options FollowSymLinks
   8  		AllowOverride None
   9  	</Directory>
  10  	<Directory /var/www/>
  11  		Options Indexes FollowSymLinks MultiViews
  12  		AllowOverride None
  13  		Order allow,deny
  14  		allow from all
  15  	</Directory>
  16 
  17  	ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
  18  	<Directory "/usr/lib/cgi-bin">
  19  		AllowOverride None
  20  		Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
  21  		Order allow,deny
  22  		Allow from all
  23  	</Directory>
  24 
  25  	ErrorLog /var/log/apache2/error.log
  26 
  27  	# Possible values include: debug, info, notice, warn, error, crit,
  28  	# alert, emerg.
  29  	LogLevel warn
  30 
  31  	CustomLog /var/log/apache2/access.log combined
  32 
  33      Alias /doc/ "/usr/share/doc/"
  34      <Directory "/usr/share/doc/">
  35          Options Indexes MultiViews FollowSymLinks
  36          AllowOverride None
  37          Order deny,allow
  38          Deny from all
  39          Allow from 127.0.0.0/255.0.0.0 ::1/128
  40      </Directory>
  41 
  42  </VirtualHost>

Environment

This post will guide you through setting up the following environment on Ubuntu Intrepid (8.10):

• Apache 2.2.9
• Passenger 2.0.6
• ActiveRecord 2.2.2
• Ramaze 2009.01
• Erubis 2.6.2
• MySql 5.0.67 (assumed installed)

To get started:

sudo apt-get install apache2 passenger
sudo passenger-install-apache2-module

Install the Rack, Ramaze, Activerecord, and Erubis gems

sudo gem install rack, ramaze, activerecord, erubis

Establishing Project

Now create your first ramaze project with:

ramaze --create demo

Then change to the demo directory. If you see a start.ru file, rename it to config.ru or Passenger won’t find it. Create a file called passenger.rb with the following:

require 'rubygems'
require 'ramaze'

# Add directory start.rb is in to the load path, so you can run the app from
# any other working path
$LOAD_PATH.unshift(__DIR__)

# Initialize controllers and models
require 'controller/init'
require 'model/init'

Ramaze::Global.sourcereload = false
Ramaze::Global.sessions = true
Ramaze::Log.ignored_tags = [:debug, :info]

Important: Do not include “Ramaze.start …” in this file or this will surely mess things up for Passenger.

Now edit the config.ru file (formerly start.ru):

/usr/bin/env rackup
# start.ru for ramaze apps
# use thin>=0.6.3
# thin start -r start.ru
#
# rackup is a useful tool for running Rack applications, which uses the
# Rack::Builder DSL to configure middleware and build up applications easily.
#
# rackup automatically figures out the environment it is run in, and runs your
# application as FastCGI, CGI, or standalone with Mongrel or WEBrick�..all from
# the same configuration.

require 'app'
Ramaze.trait[:essentials].delete Ramaze::Adapter
Ramaze.start
run Ramaze::Adapter::Base

Add the following virtual host file to /etc/apache2/sites-enabled/001-ramaze-demo

<VirtualHost *:80>
    ServerName ramaze.demo.local
    RackBaseURI /
    RailsAutoDetect off
    RackAutoDetect on
    DocumentRoot /var/www/ramaze/demo/public
</VirtualHost>

And edit your /etc/hosts file to include “ramaze.demo.local” so it resolves to localhost.

Database

Next, create a database called ramaze_demo (or your choice) and add a posts table and
some data via SQL or migrate scripts (this being a minimal demonstration and all that).

create table posts (
    id        int(11) not null auto_increment
  , title     varchar(255) not null
  , body      varchar(1024) not null
  , primary key (id)
);

insert posts values (1, "First Post", "Thank hrnt for getting passenger talking to ramaze.");
insert posts values (2, "Another Post", "Ask Pistos what SelfMarks is, because he is bored with diigo");
insert posts values (3, "IRC, you say?", "Visit #ramaze on Freenode");

Model

Edit the model.init.rb file with the following:

require 'activerecord'
require 'cleanup_connection_patch'

  ActiveRecord::Base.establish_connection(
    :adapter  => "mysql",
    :database => "ramaze_demo",
    :host     => "localhost",
    :username => "root",
    :password => "secret"
  )

class Post < ActiveRecord::Base
end

Important: cleanup_connection_patch comes from github.com and makes the ActiveRecord behave correctly in a threaded environment. Please see Monkey patching ActiveRecord to automatically release connections « coderrr for more information.

Controller

With the model squared away, time to turn our attention to the controller. Edit controller/init.rb:

class Controller < Ramaze::Controller
  layout '/page'
  helper  html
  engine :Erubis
end

class PostsController < Controller
  def index
    @posts = Post.find(:all)
  end
end

View

Create a folder under view for posts and add the following Erubis template view/posts.rhtml:

<h1>Listing Posts</h1>

<ul>
  <% for post in @posts %>
      <li><%= A(post.title, :href => "/post/#{post.id}", :title => post.body) %></li>
  <% end %>
</ul>

Note: that while Rails uses filename.html.erb, the Erubis engine expects *.rhtml extension in Ramaze.

Running

Test out the above with webrick, mongrel, or thin with:

ruby start.rb

If all goes well, you should see the three postings above listed.

Launch Apache2, if you haven't already and point your localhost's browser at http://ramaze.demo.local/posts

You should now have a listing of posts rendered through Apache via Passenger.

Conclusion

I am finding much to like about Ramaze, especially after developing with Rails for a few years now and watching it grow at an alarming rate in complexity. Ramaze is a refreshing return to basics. I am finding that Ramaze just may be able to give me most of what I need while giving me a sizeable performance boost in the process. Here's a rails bench on the equivalent of the above:

Concurrency Level:      50
Time taken for tests:   4.277 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      10218492 bytes
HTML transferred:       9595000 bytes
Requests per second:    233.83 [#/sec] (mean)
Time per request:       213.828 [ms] (mean)
Time per request:       4.277 [ms] (mean, across all concurrent requests)
Transfer rate:          2333.42 [Kbytes/sec] received

And this is Ramaze:

Concurrency Level:      50
Time taken for tests:   2.248 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      9815000 bytes
HTML transferred:       9555000 bytes
Requests per second:    444.91 [#/sec] (mean)
Time per request:       112.383 [ms] (mean)
Time per request:       2.248 [ms] (mean, across all concurrent requests)
Transfer rate:          4264.42 [Kbytes/sec] received

I am using Mac OS X Leopard 10.5.4 at the time of this writing. These are the steps I took to fix that most hideous of all colors, dark blue on black background:

1. Set your terminal theme colors to your liking
2. Set up ls to colorize directory listings by default. There are two ways to do this: Either set the CLICOLOR Environment variable or add an alias for ls to include the -G flag. I chose the CLICOLOR flag:

   export CLICOLOR=1
   

3. Next up, install the ANSI color SIMBL plugin found here: http://niw.at: works of Yoshimasa Niwa. Make sure Terminal.app is closed while you install the plugin then relaunch Terminal to find under its main menu, “Color Preferences…”
4. Finally, edit that dark blue away!

If you’re interested in tailoring the output of the colors even further, you can also export the LSCOLORS variable:

export LSCOLORS="exfxcxdxbxegedabagacad"

And then change according to the manual entry for ls (“man ls”):

One final trick to get my bash shell exactly like I like it was to set the prompt. This can likewise be done by editing ~/.profile:

PS1="[\u@\h \W] "

I combed the documentation for days trying to figure out exactly how to set up a server to hand out time to all my other servers. Normally, my Google-fu can pierce just about any need, but “time” “server” “local” and so on are all too common terms in both client-and-server and client-only configurations and by far and away, the bulk of the documentation guides you through the latter. The documentation that guides you through the former is dense with references to peers, stratum, broadcasting, multi-casting, and so on.

So, here’s how to set up an NTP service as both client and local time server on CentOS 4:

yum install ntp

Then edit the /etc/ntp.conf file and set up the public ntp server pools and uncomment the broadcast line and change IP mask to match your network’s:

server 0.rhel.pool.ntp.org
server 1.rhel.pool.ntp.org
server 2.rhel.pool.ntp.org
broadcast 192.168.1.255

Modify your /etc/sysconfig/iptables to allow other servers to connect:

-A RH-Firewall-1-INPUT -p udp -m udp --dport 123 -j ACCEPT

Finally, go to the “client server” and install ntp on it just as you did with the ntp server, except, this time, you edit your /etc/ntp.conf file to point to your new local time server. In my case, I set up two servers to give out times and I added ntp.1 and ntp.2 to my local DNS servers, so my config file for the “client servers” uses:

server ntp.1.example.com
server ntp.2.example.com

Its really that simple! Its probably not the most secure way to do the job as there’s plenty of information overload in the ntp documentation on configuring securely. However, with all of my servers sitting in a trusted VLAN and the DNS entries are only served to these servers and NTP port is blocked between this VLAN and others, I reckon it would be fairly hard to hijack the local ntp services. In other words, the simple approach is good enough, for now.

Today, we have Pradipta’s Rolodex!
digg link Reddit.

To all developers out there unfortunate enough to get caught in this web, thanks, for you have made me laugh out loud harder than I have in a good while!

Cheerios!