Registration for the Ruby on OS X Conference is now open. Tickets are selling much faster than expected, so don’t wait too long if you’re thinking about attending.

The conference is about building Mac apps with Ruby and Cocoa. We’ve tried to organise a conference that’s of interest to Ruby developer who want to get started with Mac desktop application development, as well as experienced RubyCocoa and MacRuby developers, and Cocoa/Objective-C developers interested in using Ruby.

We’ve gathered a great collection of speakers from the USA, Japan, and Europe, all of which are active in the RubyCocoa and MacRuby communities. This will be a small and intimate conference with sessions in the morning and afternoon, followed by an informal hackfest in the evening.

This morning I was deploying an application originally built on Rails 1.1.5 which we’ve recently ported to Rails 2.3. It’s an introductory online statistics course aimed at people who study psychology and social sciences. Most of the pages serve static content straight from the database, so we were never really concerned about performance.

After I finished deploying the application I was tailing the production logs and I noticed that the application took a lot less time completing page requests than before. Just for fun I decided to run some benchmarks on my development box¹.

Please don’t interpret these numbers as Rails 2.3 is three times faster, because that’s not what these numbers mean. However, when you port you application to Rails 2.3 your server load will very likely go down.

¹ Mac Pro, 2 x 2.8 Ghz, 4 GB 800 Mhz DDR2. Apache with Passenger 2.1.0 & MySQL 5.1. Applications were running in production. Benchmarks were generated with Apache Bench with concurrency 5 and 2000 requests.

Back in the days when Rails didn’t care about the encoding of the database connection you sometimes ended up with UTF-8 encoded strings from the browser travelling over a ISO-8859-1 encoded connection with MySQL to a UTF-8 database. This isn’t a big problem in itself as long and you don’t slice strings on the database level. When you decide to migration your application to a newer version of Rails you might run into trouble.

You know you have this problem when the characters from the first column of the following table look like the characters in the second column.

Fortunately it’s pretty easy to fix, you just need to convince mysqldump that it’s operating on a Latin-1 database and load the dump back into the database. Make sure you have a backup before you try this though!

$ mysqldump --skip-set-charset --default-character-set=latin1 databasename > fixed.sql
$ mysql databasename < fixed.sql

Now make sure that your Rails app talks UTF-8 to the database by setting the encoding in database.yml:

production:
  adapter: mysql
  database: databasename
  encoding: utf8

Today is a nice day.

Because Rails 2.3 release candidate 1 was just released which, amongst others, contains my patch to support nested model attributes in your model and view.

• The Rails 2.3 Release notes.
• The excellent introduction by Ryan Daigle.
• The original introduction blog post.

Enjoy!

X-Mas came early this year ;-)

Checkout the changelog:

• Rack support. Thanks to Kematzy for the original suggestion/work.
• Fixed “issue” people had with the default non-vhost based directory not working anymore. Thanks to “spike” for debugging this.
• Made the vhost config file extension configurable as well. Patch by Felipe Mathies.
• Backported RubyCocoa + Ruby 1.8.7 fix. Patch by Grant Hollingworth.
• Removed the “Allow mod rewrite rules” checkbox, we found it’s not worth the UI real estate.

Most users probably want to download the “stable” 1.2 release.

If you understand why stable has been quoted, you can track development and contribute on: github.com/alloy/passengerpane

Please report any bugs you may find at: fingertips.lighthouseapp.com/projects/13022

It has been possible to specify gem dependencies for your application for a while. But sometimes when moving to another machine we would complain about there not being a development dependency option for config.gem as there is for gems. But we hadn’t actually given it any thought…

Today I had some weird test failures, which were related to not having test-spec 0.9 installed on my macbook. Most tests did pass because I did have an older version installed. So after a few minutes of thinking about it, I finally figured out the stupid simple solution. Simply define the dependencies in config/environments/test.rb!

# Development gems:
config.gem 'test-spec', :version => '0.9', :lib => 'test/spec'
config.gem 'mocha', :version => '0.9.3'

This one is probably obvious to some, but as far as I know it has never been actively promoted. And it’s just too easy to think of ;-)

Unichars is a simple wrapper around Glib2 Unicode functions. You can use it to speed up certain methods on Unicode string. Currently supported are: upcase, downcase, reverse, and size. The cool thing about it is that it works seamlessly with ActiveSupport::Multibyte and it works great without ActiveSupport::Multibyte.

I guess Unichars is not a very exiting name like God, Vlad the Deployer, or Gerard Joling but I guess I’m not that kind of a guy.

You can install Unichars with Rubygems:

$ gem install unichars

Or you can fetch it from Github:

$ git clone git://github.com/Manfred/unichars.git
$ cd unichars
$ rake gem:install

With Rails

The examples in the README tell you how to use Unichars with Rails 2.1 or newer. I’ll just re-iterate how it’s done.

First you make sure you load the library, the easiest way to do this is with config.gem in environment.rb:

config.gem 'unichars'

Or when you dislike gems, you can just require it:

require 'unichars'

When you’re not using config.gem, you have to make sure ActiveSupport is loaded before Unichars, otherwise the Rails integration won’t magically work.

After that you have to tell ActiveSupport::Multibyte to use the Unichars class as proxy class. You can do that in an initializer or at the end of your environment.rb. I would recommend doing it in config/initializers/unichars.rb.

ActiveSupport::Multibyte.proxy_class = Unichars

Now all of Rails will automatically use the Unichars character proxy, you can also use it yourself:

'青山'.mb_chars.reverse #=> '山青'

Without Rails, but with ActiveSupport

require 'activesupport'
require 'unichars'
ActiveSupport::Multibyte.proxy_class = Unichars
$KCODE = 'u'

Other than that it’s similar to Rails:

'Sluß'.mb_chars.upcase #=> 'SLUSS'

A good time to talk about LC_CTYPE real quick. Note that Glib2 picks that up from your environment, so your results may vary depending on what it’s set too.

Without training wheels

require 'unichars'

If you don’t use ActiveSupport, you can still use Unichars because it comes with a light version of the Chars proxy. You will have to wire it yourself though:

class String
  def mb_chars
    Unichars.new(self)
  end
end

'Copy-®'.mb_chars.size #=> 6

Without anything

Finally, you can just use the Glib2 wrapper and roll your own solution:

require 'glib'
Glib.utf8_upcase('Comme des Garçons').upcase #=> 'COMME DES GARÇONS'

Questions?

If you have any questions or issues, please use the Github Wiki Wiki as much as possible. If you want to discuss anything you can find me on Freenode in #rails-contrib. Have fun with Unichars!

Just a quick warning: when you use ActiveRecord::Base.connection.execute you get a Mysql::Result instance, this is a very thin wrapper around the actual result returned from libmysqlclient. This means you have to free the memory of the result table, failing to do so will result in erratic query times because somewhere a garbage collector will do it for you.

class Person < ActiveRecord::Base
  # Returns the last name of someone given the ID
  def self.last_name(id)
    result = connection.execute("SELECT last_name FROM people" +
      " WHERE id = %d" % id)
    last_name = result.fetch_row.first
    result.free
    last_name
  end
end

When you want to select a number of entire rows, you can use the already safe select_rows.

The problem you run into when trying to write unit tests for methods using random data is that you can’t predict what random data it’s going to use. The idea behind a unit test is that you control the input and test whether the output is as you expected, so that’s a problem.

Let’s assume you have a token generator like this:

module Token
  DEFAULT_LENGTH = 8
  CHARACTERS = ('a'..'f').to_a + ('0'..'9').to_a
  
  def self.generate(options={})
    srand
    options[:length] = DEFAULT_LENGTH if options[:length].nil?
    (1..options[:length]).map { CHARACTERS.rand }.join
  end
end

Now you can write tests to see if it generates tokens of the correct length, doesn’t generate homogeneous tokens and doesn’t generate the same token a number of times in a row. Unfortunately these tests could succeed a number of times and suddenly fail for no apparent reason. Usually this happens when you’re on vacation and one of your collegues is running the test on some weird architecture leaving them baffled.

The solution is to make Array#rand predictable by stubbing out the normal implementation:

class Array
  def round_robin_rand_list=(list)
    @round_robin_rand_list = list
  end
  
  def rand
    current = @round_robin_rand_list.shift
    @round_robin_rand_list.push(current)
    current
  end
end

Now you can write a test that checks the output, because you control what the random method returns:

class TokenTest < Test::Unit::TestCase
  def test_token_generate
    Token::CHARACTERS.round_robin_rand_list = [0, 1, 14]
    assert_equal 'ab8ab8ab', Token.generate
    Token::CHARACTERS.round_robin_rand_list = [7, 12, 15, 3]
    assert_equal '169d169d', Token.generate
  end
end

Obviously you need to clean up after yourself in tests so you can’t just redefine methods. It would be really cool if we could do something like this:

class TokenTest < Test::Unit::TestCase
  def test_token_generate
    Token::CHARACTERS.define_round_robin_method(:rand, ['a', 'b', '8'])
    assert_equal 'ab8ab8ab', Token.generate
    Token::CHARACTERS.undefine_round_robin_method(:rand)
  end
end

We’ve implemented this as round_robin_method.rb, complete with its own testsuite. You could go even further and hook it into TestUnit to automatically undefine the round robin methods after a test has run. We’ll leave that as an exercise for another day.

Yesterday Michael Koziarski merged the updated version of ActiveSupport::Multibyte into Rails. The initial reason for the update was Ruby 1.9 compatibility but it turned into a complete overhaul. Not just the code, but also the documentation was revised.

For most people the only noticeable change is the move from String#chars to String#mb_chars. People relying heavily on ActiveSupport::Multibyte probably want to read on.

String#chars renamed to String#mb_chars

One of the initial reasons to use a proxy to access characters back in 2006 was to make Rails future proof in case Ruby got some kind of Unicode support on String. Unfortunately Matz decided to use String#chars for one of these features so we had to change the method name. People running on Ruby <= 1.8.6 will get a nice deprecation warning.

String#mb_chars now returns a proxy on Ruby 1.8 and returns self on Ruby 1.9.

Note that the Ruby 1.9 String class does not implement methods like String#normalize. We’re still trying to figure out how to approach this limitation. For now, you might want to do:

class String
  def normalize(normalization_form=ActiveSupport::Multibyte.default_normalization_form)
    ActiveSupport::Multibyte::Chars.new(self).normalize(normalization_form)
  end
end

No more automatic tidying of bytes

Multibyte no longer attempts to convert broken encoding in strings to a valid UTF-8. The String#tidy_bytes method still exists if you need this functionality.

Duck-typing aid

Strings are notoriously hard to duck-type because they include Enumerable, which makes them hard to differentiate from Arrays. Rails already had some duck-typing help in place for Date, Time and DateTime. We decided to implement the same thing on String and Chars.

'Bambi and Thumper'.acts_like?(:string) #=> true
'Bambi and Thumper'.mb_chars.acts_like?(:string) #=> true

So if you catch yourself using str.is_a?(String) please consider using acts_like?.

Different way of registering backends

Instead of registering a handler on the Chars class, you now set the proxy_class on ActiveSupport::Multibyte.

ActiveSupport::Multibyte.proxy_class = UTF32Chars

Note that this removes a level of indirection, which speeds up the entire Multibyte implementation quite a bit.

If you’ve implemented your own handler, please look at the implementation of ActiveSupport::Multibyte::Chars on how to convert it to work with the new implementation. In most cases this should be a trivial exercise. Don’t hesitate to contact me if you need help.

Overrideable default normalization form

The default normalization form can now be set on ActiveSupport::Multibyte instead of updating a constant.

ActiveSupport::Multibyte.default_normalization_form = :kd

See ActiveSupport::Multibyte::NORMALIZATIONS_FORMS for valid normalization forms.

To get the latest version, please see the Passenger preference pane page.

Yes yes, it’s update time!

This version comes with important fixes and some requested improvements. I’ll let the changelog speak for itself:

• Honor custom environments that a user might have set.
• Fixed problem with restarting Apache. After saving an application Apache should now automatically be restarted. Thanks to Ciarán Walsh.
• Added support for ServerAlias and add those entries to the hosts db.
• Reload the applications from disk when the preference pane is brought back to the front. Any changes made to the vhosts from elsewhere will be reflected in the UI.
• Moved all hardcoded paths into a config module. Added a config for Apache 2 as installed by MacPorts. Thanks to Ciarán Walsh.
• The host table list was editable. Thanks to Ciarán Walsh.
• Fixed bugs in parsing custom user defined data in vhosts.
• Create a tmp dir before touching restart.txt if none exists.
• Replace underscores with hyphens in hostnames. Thanks to Bryan Liles.

Most users probably want to download the “stable” 1.1 release.

If you understand why stable has been quoted, you can track development and contribute on: github.com/alloy/passengerpane

Please report any bugs you may find at: fingertips.lighthouseapp.com/projects/13022

Eloy and I will be in Berlin Monday through Thursday. We’re not attending RailsConf, but we are meeting up with a small detachment of European Caboosers to hack on projects and have a good time. We’ve rented a big apartment near the conference which we’re using as our HQ.

If you want to meet up or want to discuss anything related to projects we’re working on let us know and we’ll hook up.

Lately we’ve been aliasing dangerous methods in our tests to make sure we don’t accidentally break something. It’s not very nice when tests delete your favorite episode of Sesame Street or add a MySQL user, so we had a lot of snippets like this laying around:

module Kernel
  mattr_accessor :allow_system
  self.allow_system = false

  alias original_system system

  def self.system(*args)
    if allow_system
      original_system(*args)
    else
      raise RuntimeError, "You're trying to do a system call, which is probably not a very good idea in a test."
    end
  end
end

We refactored it and now we specify the same thing like this:

ActiveResource::Connection.add_allow_switch :request
Kernel.add_allow_switch :system, :default => true

The code is currently included in on_test_spec, our plugin for writing Rails specs on test/spec.

To get the latest version, please see the Passenger preference pane page.

We blogged how using Passenger made development easier, but being the Mac User Interface Junkies we are, we’d like to take it one step further.

This was a great opportunity to play with preference panes in RubyCocoa. Thanks to Jason Foreman, and his repository of templates, I didn’t have to spend a lot of time finding out how to initialize a prefPane bundle.

It turns out that creating basic features with a mediocre interface (the default stuff you get from Interface Builder) can be done fairly quickly. However, making it look and feel like a preference pane from the “OEM fruit company” is a different story. Who would have thought?! ;)

Thijs doing interface design with the latest available technologies. Can you spot the advanced z-index technique he used? Hint: Starts with a P.

Anyways, back to the introduction; after roughly 2 weeks we give you an OS X System Preferences pane that will configure Apache and set up a local hostname for running any Rails application using Phusion Passenger. Getting your Rails app up and running is now a matter of seconds.

Most users probably want to download the “stable” 1.0 release.

If you understand why stable has been quoted, you can track development and contribute on: github.com/alloy/passengerpane

Please report any bugs you may find at: fingertips.lighthouseapp.com/projects/13022

Note that Passenger preference pane requires OS X 10.5.2 (or at least 10.5.0 and install RubyCocoa 0.13.2 yourself) and Passenger 2.0.1.

In the wake of RubyEnRails 2008 comes a special Amsterdam.rb meeting with presentations at the TTY office. Confirmed presentations are:

• A quick overview of the Sellaband test framework
• A demo of the Ruby implementation of Quicktest: Rushtest
• Bye bye Ferret, long live ultrasphinx / sphinx (search engine plugin)
• Quick introduction of OSC and the Monome – Sam Aaron
• Short and Sweet; writing maintainable Ruby code – Manfred Stienstra

Hope to see you there!

In this episode, Manfred answers some questions from our viewers about last week’s episode where Eloy defined a method called ‘Object’.

In this episode, Eloy shows how to set up a class with default attributes in a single line of code.

In Smalltalk code and data are always kept together. In Ruby this isn’t the case. In this episode, Manfred looks at a poor man’s version of keeping your data with your code.

Lately a few things have been bugging me about Rails development using script/server. First, I can’t test through SSL and for applications who switch between SSL and non-SSL you really want your development environment too look as much like production as possible. And secondly, I have to manually manage my app server with script/server. This is a bit of a pain because some of our applications use ActiveResource to communicate so I need to start several app processes.

Proxying trough Apache solved the SSL problem, but now I had to remember on which ports I had to start my app server.

The solution turned out to be Passenger. First we install the passenger gem and compile mod_passenger.so.

$ gem install passenger
$ passenger-install-apache2-module

After that we turn on Apache at System Preferences ➔ Sharing ➔ Web Sharing and edit the webserver configuration. I added everything to /etc/apache2/users/manfred.conf but Apache doesn’t really care where you put it, just remember to load mod_passenger.so before using Passenger specific configuration options.

Set up the Passenger configuration as explained at the end of the install script.

LoadModule passenger_module /Library/Ruby/Gems/1.8/gems/passenger-1.0.1/ext/apache2/mod_passenger.so
RailsSpawnServer /Library/Ruby/Gems/1.8/gems/passenger-1.0.1/bin/passenger-spawn-server
RailsRuby /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby

Make sure our apps run in development mode.

RailsEnv development

Allow Apache serve files from our development directories.

<Directory "/Users/manfred/Code">
    Order allow,deny
    Allow from all
</Directory>

Finally, configure virtual hosts for our various projects.

NameVirtualHost *:80

<VirtualHost *:80>
DocumentRoot "/Users/manfred/Code/project1/public"
ServerName project1.local
</VirtualHost>

<VirtualHost *:80>
DocumentRoot "/Users/manfred/Code/project2/public"
ServerName project2.local
</VirtualHost>

Update: Adam Meehan has a short writup on how to connect ruby-debug to a Rails instance running in Passenger.

It’s time for another ‘Morning coffee’ meeting. You’re invited to come chat with your fellow web developers over a nice cup of coffee.

Please leave a comment if you’re coming, any questions you might have are welcome too.

We’ve been working with Eloy ever since he impressed us with his RubyCocoa demo at the 2007 RubyEnRails conference, and we are very happy that he recently decided to join the Fingertips team. He is not only a great developer but also a very nice guy. It is a joy to have him on board.

Eloy has been a Ruby developer for over four years. He is an active member of the RubyCocoa community where he started the Rucola project, a project that allows you to write RubyCocoa apps without XCode. He lives in Amsterdam on a boat together with his wife Dionne and their two cats.

Today Ninh Bui and Hongli Lai from Phusion visited the Fingertips office to introduce us to their Passenger Apache module for Rails. We’re very impressed with their work; installation and deployment is fast and easy and everything just works out of the box.

These guys did a really important job by completely removing the deployment hassle that has always been associated with Rails. They’ve made Rails deployment boring.

We’ll be testing Passenger on a server that’s running 15 small to medium Rails apps. You can expect an in-depth article about our experiences when Passenger will be released ‘any day now, honestly’.

If you’re interested in using Passenger in a bigger environment (for example, if you’re a shared hosting provider who wants to offer Rails support), you should get in touch with these guys to talk about the professional services they provide. Not only because they’re smart, but also because they’re a lot of fun to chat with. We really enjoyed their company.

Ruby has dedicated keywords like if and else to define conditional logic. Other languages, like IO, use methods for conditional execution. In this episode Manfred shows how you can use a class in Ruby to do something similar.

The release of Rails 2.0 made me realize once more what a great project Rails is.

We’ve seen a lot of changes since 1.0; the community working with and on Rails has grown enormously and Rails is now running some really big sites. Even though this has undoubtedly put pressure on the core team, the feel of the project hasn’t changed much.

In this release most new features were driven by people outside of the core team. This is mostly due to the willingness of the core team to accept patches and interact with the community. That takes dedication.

Most developers on the core team run their applications directly on trunk which means that gem releases are no longer a personal need. This, combined with the fact that people are always looking to get their patch in at the last minute, makes it very hard to get a new major release out the door. I therefore applaud David Heinemeier Hansson for putting his foot down and tagging 2.0 (only to find a small bug and having to tag 2.0.1 a day later). That takes courage.

I’m really happy that the core team has been able to keep Rails fun, because in the end, that’s what drives my productivity as a developer.

It’s time for another coffee morning. Like always you’re invited to come chat about Rails, Django, Seaside and web development in general over a nice cup of coffee.

Please leave a comment if you’re coming, any questions you might have are welcome too.

In this episode Sam wonders whether Ruby and Rails are ready for the Enterprise.

Please join us in welcoming Sam Aaron to our team.

Sam has been active in the Ruby and Rails communities for some years; he founded the Newcastle Ruby and Rails user group and was a speaker at RailsConf Europe this year. He enjoys writing and regularly publishes Ruby articles for InfoQ. Sam was also the technical reviewer for ‘Beginning Google Maps Applications with Rails and Ajax’ from Apress and a contributor to ‘The Rails Way’ published in the Addison-Wesley Professional Ruby Series .

In addition to being a fond advocate of both Ruby and Rails, Sam is interested in the aesthetics of programming languages, language oriented programming, and domain specific languages, which were the general subjects of his Ph.D. thesis.

Sam loves cycling (which is great here in Amsterdam) and also enjoys getting out into the countryside where he likes to walk, scramble and camp.

This article is left here for historical purposes, please read the comments carefully after you’ve read the article.

Even though Ryan Daigle already covered it in his usual timely fashion, I’d like to share some real life examples of how we use the relatively new rescue_from functionality.

Because I usually prefer methods that raise exceptions instead of returning a boolean, like, say, save! instead of just save, I thought it would be nice if I could deal with some common exceptions on a higher level. This lead me to write a Rails patch based on the exception_handler plugin, which I previously used in some projects.

class PostsController < ApplicationController
  def create
    @post = Post.create!(params[:post])
  rescue ActiveRecord::RecordInvalid
    render :action => :new
  end

  def update
    @post = Post.find(params[:id])
    @post.update_attributes!(params[:post])
  rescue ActiveRecord::RecordInvalid
    render :action => :edit
  end
end

class ApplicationController < ActionController::Base
  rescue_from ActiveRecord::RecordInvalid do |exception|
    render :action => (exception.record.new_record? ? :new : :edit)
  end
end

class PostsController < ApplicationController
  def create
    @post = Post.create!(params[:post])
  end
  
  def update
    @post = Post.find(params[:id])
    @post.update_attributes!(params[:post])
  end
end

And it just works.

In a day or three I will be in Pisa for Rails to Italy where I am giving a talk on Ruby patterns in the Rails internals. Shout if you happen to be in the area and want to meet up, you know how it works.

A lot of people are on Rails Edge nowadays. Besides all the nifty new features they also encounter bugs. There are two ways to work around these bugs:

1. Lock your Rails external to an older revision or freeze it to vendor
2. Find a patch on Trac or write one of your own and apply it

If you decide to apply patches, you’re going to need some help. Our solution is to put all the patches that need to be applied in vendor/patches and put the following Rake task in lib/tasks/patches.rake:

task :patch => 'patches:apply'

namespace :patches do
  desc "Apply all patches from vendor/patches to root"
  task :apply do
    Dir.chdir(RAILS_ROOT) do
      Dir["vendor/patches/*"].each do |patch|
        system "patch -p0 < \"#{patch}\""
      end
    end
  end

  desc "Revert all patches applied in vendor/plugins and vendor/rails" +
       "through SVN"
  task :revert do
    system "svn revert --recursive vendor/plugins vendor/rails"
  end
end

Assuming you’re using Capistrano 2, you can add the following to your deployment recipe to automatically run the patches for each deployment.

after "deploy:update_code", "deploy:patch"

namespace :deploy do
  task :patch, :roles => :app do
    run "rake patches:apply"
  end
end

Now all you have to do is make sure that all the patches apply cleanly and remove them when they’re accepted into Rails.

Today we bring you the next episode, Function Composition: Redux.

In this episode Manfred shows how you can override the === method on your own classes to do advanced matching.

With the summer vacations behind us it’s time for another coffee morning. Like always you’re invited to come chat about Rails, RailsConf Europe experiences, Django, Seaside and web development in general over a nice cup of coffee.

Last time quite a few people asked me about Seaside, but unfortunately I don’t know that much about it. Anyone with Seaside experience is extra super invited.

Please leave a comment if you’re coming, any questions you might have are welcome too.

If you also happen to be attending this year’s RailsConf in Berlin, come say hello when you see Manfred or me running around.

Also, good news! While his keynote wasn’t too interesting, DHH did announce that the core team is preparing a Rails 2.0 preview release which they hope to get out by the end of the conference.

Martin Sadler from Working With Rails interviewed me for the second time about the Rails Hackfest. In the interview I talk about Hackfest experiences, the Rails community in Amsterdam and my typical working days.

“One of the things I’ve learnt during the Hackfest is that competition isn’t always the best driver for quality.”

Interested? Go and read the entire interview.

In this episode Norbert shows how memoization is implemented and how you can use it to speed up slow methods.

For those of you on Edge Rails, since changeset [7189] appears to have broken the current acts_as_paranoid we’ve been getting some test errors. There’s a quick patch I wrote available in this Pastie, but because scope_out is now recommended by Rick Olson himself this should only be seen as a migration path. Hopefully this will save you some trouble.

Here is the third episode in which Manfred shows how & maps to the to_proc method and what you can do with it.

Recently we contracted a local company to help out with a Rails project. Their primary job was to implement payments. During an audit or their work I found the following helper:

module CoursesHelper
  include ActiveMerchant::Billing 
  def issuers
    @gateway = IdealGateway.new(
         :merchant            => IDEAL_MERCHANT_ACCOUNT,
         :sub_id              => IDEAL_SUB_ID,
         :password            => IDEAL_PRIVATE_KEY_PASS,
         :private_cert        => IDEAL_PRIVATE_CERT,
         :language            => IDEAL_LANGUAGE,
         :private_key         => IDEAL_PRIVATE_KEY,
         :authentication_type => IDEAL_AUTHENTICATION_TYPE
    )
    
    list = Array.new()
    begin
      response = @gateway.issuers
      if response.success?
        list=Array.new()
        list.push({"issuerName"=>"Kies uw bank...", "issuerID"=>0})
        list=list + (response.params["list"])
      end
    rescue
      list = [{:issuerName=>"Kies uw bank...", :issuerID=>""},
      {:issuerName=>"ofline Simulator", :issuerID=>"00"}]
    end
    return list.map {|i| [ i["issuerName"], i["issuerID"] ] }

  end
  
end

Helpers are for keeping simple code out of your view. They should never include complicated logic. A helper should be so simple that you’d almost dare to deploy without testing.

Also, because it’s a good idea to keep coupling between classes to a minimum, the payment gateway should only be called from classes directly related to payment processing.

Finally, you don’t want such a large number of constants in your code. There are better ways to store the gateway configuration.

The contracter was asked to solve these issues. The gateway initialization was refactored to only appear once and the gateway object was assigned to a constant named IDEAL_GATEWAY. The code to find all issuers was moved to the Payment model.

class Payment < ActiveRecord::Base
  def self.ideal_issuers
    response = IDEAL_GATEWAY.issuers
    response.params["list"]
  end
end

module CoursesHelper
  
  def issuers
    list=Array.new()
    # 'Kies uw bank' means 'Choose your bank'
    list.push({"issuerName"=>"Kies uw bank...", "issuerID"=>0}) 
    list=list + Payment.ideal_issuers
    return list.map {|i| [i["issuerName"], i["issuerID"]] }
  end
  
end

Unfortunately, the issuers helper still looks more like Python than Ruby. I would have written it like this:

module CoursesHelper
  def issuers
    [['Kies uw bank...', 0]] + Payment.ideal_issuers.map \
      { |i|[i['issuerName'], i['issuerID']] }
  end  
end

I don’t think you should include instructions on how to use a drop-down as one of the options inside it. Furthermore, a default value makes no sense in this case. People always have to choose a bank, otherwise they can’t continue with the payment process.

After some refactoring I was left with the code below. I’ve also moved it to the helpers module for the payments controller, because that’s where the form is rendered.

module PaymentsHelper
  def issuers
    Payment.ideal_issuers.map { |i| [i['issuerName'], i['issuerID']] }
  end
end

After our presentation at RubyEnRails 2007 we decided to share some of our code snippets with the world. Here is the second episode in which Manfred shows how you can make your objects sortable by defining the boat operator.

Let’s meet up one more time before the holidays. The goal remains unchanged: a good chat about our experiences with Rails, Django, Seaside and other next generation web frameworks over a strong cup of coffee.

Please leave a comment to tell us you’ll be there or if you have any questions.

After our presentation at RubyEnRails 2007 we decided to share some of our code snippets from the presentation with the world. Here is the first episode in which Norbert shows function composition in Ruby.

Yesterday the three of us attended RubyEnRails 2007, a Dutch one day conference about Ruby and Rails held in our hometown Amsterdam. We had a great time meeting up with all kinds of Rails developers, almost Rails developers and entrepreneurs.

Nic Williams kicked off the conference being his Australian self and sporting his caboose shirt. In his talk he layed out his views on the future of Rails.

Norbert and I did a live hacking session with Ruby, we showed a few ways to clean up your Ruby code and a few ways to make it almost unreadable. I think we lost some people along the way, but I hope everyone enjoyed the talk.

One of the things that struck me at the conference was that the talks were really diverse, with lots of real world examples and live coding. As Geoffrey Grosenbach mentioned in his closing talk, this really gave the conference it’s own identity. I hope to see more of this in the future. Next stop, RailsConf Europe.

Geoffrey Grosenbach interviews Robert Gaal

On the right Justin Halsall

Eloy Duran and Nic geeking out after a talk

If you’re thinking of building a cool snazzy Rich Internet Application front-end in Flex for your RESTful Rails application then please stop dreaming.

There’s no way to extract the headers from an HTTP response in ActionScript 3 so you can’t get the id of a newly created resource from the ‘Location’ header and you can’t tell the difference between a ‘500 Internal Server Error’, a ‘404 Not Found’ or a ‘422 Validation Error’.

There’s also no way to get the response body for anything not in the 2xx range.

Oh, and you can only do a GET or a POST, no PUT or DELETE, at least not without a proxy.

If you can prove me wrong, please do.

In a few weeks Manfred and me are giving a talk on Rubyisms at the Dutch RubyEnRails conference. From the site:

Norbert and Manfred take you on a journey along the coast of Ruby, where they try to uncover gems between the driftwood and seaweed while they steer clear of the word ‘metaprogramming’ with great dexterity.

If you’re in or around Amsterdam on June 7th, do come by. We’re looking forward to it!

We had a little Code to Test Ratio showdown at the office today, probably because there was a woman present. It got me wondering what the Code to Test Ratio for other people is. So everybody, please post the CTTR of your current project in the comments.

Note: in Rails you can find your CTTR with rake stats.

We’re looking for a full-time Ruby on Rails developer.

Send us a link to a publicly available site or application, show us your code, or tell us about open source projects you’ve contributed to. We’d like to see what you’ve done with Rails.

If you’re interested, please send an email to thijs@fngtps.com

When you’re testing uploads with attachment_fu, your files end up in RAILS_ROOT/public by default. This is not very handy because they might override your carefully uploaded bunny pictures in development. You can easily solve this by overriding the full_filename method on your attachment model. Let’s assume you have something like this.

class Asset < ActiveRecord::Base
  belongs_to :post  
  has_attachment :content_type => :image, :storage => :file_system
end

Then you can add the following to the file your tests are defined in:

class Asset
  def full_filename(thumbnail = nil)
    file_system_path = (thumbnail ? thumbnail_class : self).attachment_options[:path_prefix].to_s
    File.join(Dir::tmpdir(), file_system_path, *partitioned_path(thumbnail_name_for(thumbnail)))
  end
end

This will make your files get written to /tmp/public. If you have multiple tests that have to override the attachment class, it’s probably best to put it in a separate file.

I was trying to make a glossary of words grouped by their first letter, but I wanted words starting with the letter é grouped with words starting with the letter e. No small feat you might imagine. Wrong.

dict = words.inject({}) do |dict, word|
  letter = word.chars.decompose[0..0].downcase.to_s
  dict[letter] ||= []
  dict[letter] << word; dict
end

The reason this works is that letters like é have a decomposed form in Unicode, this form consists of a latin letter and a accent modifier. I’m not sure what happens if you run Arabic through this code, but we’ll cross that bridge when we get there.

If you’ve installed the Rubyforge gem, which is a dependency of just about every gem in the wild through Hoe, then you might have see the OpenSSL::SSL::SSLError when trying to connect to a server over SSL. This is caused by http_access2 because it refuses to connect with SSL without verifying the certificates.

The problem is a little bit tricky because SOAP4r abstracts you away from http_access2 and that makes it hard to set the proper configuration on the HTTP client. Fortunately SOAP4r has a way to configure it’s subsystem.

Let’s assume that you’ve defined a SOAP::RPC::Driver somewhere.

class WebrApi < ::SOAP::RPC::Driver
  DefaultEndpointUrl = "https://webr.com/api"
  [snip]
end

There are two ways of making the error go away, the first is to tell http_access2 to stop verifying certificates and just get on with the connection.

proxy = WebrApi.new
proxy.options['protocol.http.ssl_config.verify_mode'] = OpenSSL::SSL::VERIFY_NONE

This is not recommended because one of the ideas behind SSL is that you can always verify that you’re talking to the correct server. If you want to keep this feature you will have to tell http_access2 which certificates to use.

proxy = WebrApi.new
client.options['protocol.http.ssl_config.verify_mode'] = OpenSSL::SSL::VERIFY_PEER
client.options['protocol.http.ssl_config.ca_file'] = '/etc/ssl/certs/certification_authority.crt'
client.options['protocol.http.ssl_config.client_cert'] = '/etc/ssl/certs/client.cert'
client.options['protocol.http.ssl_config.client_key'] = '/etc/ssl/certs/client.key'

You can find more advanced configuration examples in the SOAP4r source in sample/soap/ssl.

Changeset 6185, or so we assume, broke our fix for Basic Authentication in Safari when using RESTful Rails routes.

Luckily it wasn’t too hard to fix the fix and we’ve decided to make a plugin out of it so we can easily keep the fix in sync with Rails for all our projects. You can find safari_basic_auth_fix in our Subversion repository. The current fix makes sure that semicolons are encoded for every outgoing URL and it decodes the semicolons for all the incoming URLs.

Update: Changeset 6485 removed semicolons as a action separator, so this shouldn’t be a problem anymore once you upgrade to Rails trunk.

As you may have noticed OpenStruct#id always returns the object id of the OpenStruct instance, even when you set id.

>> o = OpenStruct.new :id => 2
=> #<OpenStruct id=2>
>> o.id
(irb):4: warning: Object#id will be deprecated; use Object#object_id
=> 9850940

Fortunately there is a simple way around this.

OpenStruct.__send__(:define_method, :id) { @table[:id] }

Now OpenStruct behaves like we want.

>> o.id
=> 2

Note that hash and object_id still work fine. You probably want to keep in mind that we’ve redefined the default OpenStruct behaviour and it might cause problems elsewhere.

Again a great meeting. I was amazed to see how much Rails is being used these days.

A little less than a year ago at the first meeting, only a handful of the developers visiting were using Rails professionally. This time I not only wasn’t able to find anyone who was not yet getting paid to write Rails code, but I also talked with folks from three different startups using Rails; soocial, SugarStats, and Wakoopa.

The last few weeks we noticed the dreaded question marks on our sites running against MySQL 5.0. We thought we did everything to make sure our servers, databases, tables, clients and connections understood UTF-8, but somehow connections to the database were reset back to Latin1 after some time.

Instead of trying to fix the problem in Rails/Ruby/libmysql I decided to squash the problem in the MySQL server configuration. By default we were seeing this:

mysql> SHOW VARIABLES LIKE 'character\_set\_%';
+--------------------------+--------+
| Variable_name            | Value  |
+--------------------------+--------+
| character_set_client     | latin1 | 
| character_set_connection | latin1 | 
| character_set_database   | latin1 | 
| character_set_filesystem | binary | 
| character_set_results    | latin1 | 
| character_set_server     | latin1 | 
| character_set_system     | utf8   | 
+--------------------------+--------+

So I set the following in /etc/mysql/my.cnf:

[mysqld]
character-set-server = utf8

[client]
default-character-set = utf8

Which forces all the encoding to go to UTF-8 by default:

mysql> SHOW VARIABLES LIKE 'character\_set\_%';
+--------------------------+--------+
| Variable_name            | Value  |
+--------------------------+--------+
| character_set_client     | utf8   | 
| character_set_connection | utf8   | 
| character_set_database   | utf8   | 
| character_set_filesystem | binary | 
| character_set_results    | utf8   | 
| character_set_server     | utf8   | 
| character_set_system     | utf8   | 
+--------------------------+--------+

We’re very happy to welcome Norbert Crombach to the Fingertips team. He started this morning.

Norbert has been working with Ruby on Rails since the very first public release when a good friend – who has since moved to Io – convinced him Ruby really was better than Perl.

Norbert always struck us as a talented and extremely bright developer who seemed to be doing very well on his own. He did however get fed up with the business side of running his own company, which made him decide to join us so he could spend more time doing actual development work.

For the next few months Norbert will be working remotely from Eindhoven while he’s trying to find a place to live here in Amsterdam.

Time for another one. The goal remains unchanged: a good chat about our experiences with Rails, Django, Seaside and other next generation web frameworks over a strong cup of coffee.

Photo by Rodney Ramdas

Please leave a comment to tell us you’ll be there or if you have any questions.

I’m sure we’ve all been there; trying to save a list of associated objects together with the instance they belong to. Fortunately with the right tools this can be really simple.

Let’s assume we have a question in a questionnaire and need to specify an arbitrary number of options to this question. We could construct the following database schema.

create_table :questions do |t|
  t.column :stem,        :text
end

create_table :options do |t|
  t.column :question_id, :integer
  t.column :label,       :text
  t.column :feedback,    :text
  t.column :correct,     :boolean, :default => false
end

There are basically three ways I know about to assign multiple options to a question from one page. The first option is to create a Question instance in the database before rendering the form so we can use remote_form to send an Ajax request and directly associate the Option instance. An advantage of this solution is that all data is directly saved to the database, which makes losing any data less likely. Another advantage is that the functionality is spread nicely over the responsible controllers. First you perform a create on the QuestionsController, after that you perform multiple creates on the OptionsController. The disadvantage is that you have to create a Question with default values or by circumventing the initial validation. This could generate some ‘blank’ Question instances in the database which you will have to garbage collect.

The second option is a slight variation on the first option. Instead of creating a Question to link the Options to you keep all the generated Option id’s in the session and link them to the Question after validation. This solution also benefits from nice separation of concern in the controllers, but on the downside it also leaves you with unlinked Option objects you will have to garbage collect.

The third option is to keep all the information about the Questions and Options in the DOM tree of your from and post it all at once. This used to be hard, requiring a lot of custom JavaScript and browser tricks. Fortunately WHATWG is here to help; Web Forms 2.0 defines a repetition model for repeating form controls. Browser support for these interaction models is years away (except for Opera 9, which has an experimental implementation), that’s why there’s a JavaScript library to accelerate the adaptation.

It’s really easy to use the library in your Rails application. Download the repetitionmodel library, extract the zipfile and put the javascript files in public/javascripts. When you’re done with that we can get back to coding our application. Include the JavaScript file somewhere in your views.

<%= javascript_include_tag 'repetition-model-p' %>

We just mentioned that we want to send the entire form at once, we can use validates_associated to make sure the Question is never saved when the options aren’t valid. This allows you to easily validate all the information from the form at once. The models to go with the database look something like this.

class Question < ActiveRecord::Base
  has_many :options, :dependent => :delete
  validates_presence_of :stem
  validates_associated :options
end

class Option < ActiveRecord::Base
  belongs_to :question
  validates_presence_of :label
end

Build the new and edit forms for the QuestionController like you would normally do. A the bottom of the form we will add the HTML to edit the options.

<h2>Options</h2>
<ol id="options">
  <%= render :partial => 'options/option', :collection => @question.options %>
  <%= render :partial => 'options/option', :object => Option.new,
    :locals => {:option_counter => '[option]'} %>
</ol>
<p><button type="add" template="option">New option</button></p>

The first render is for all the existing options, the second render is for what the repetition model calls a template, the template should always be the last in the list of controls.

In app/views/options/_option.rhtml we put the following. This partial doubles as a template for new controls and as a partial for existing options in the database.

<% if option_counter == '[option]' -%>
<li id="option" repeat="template" repeat-start="0">
<% else -%>
<li repeat="<%= option_counter %>">
<% end -%>
  <div>
    <div><label>Label</label></div>
    <%= text_area_tag "options[#{option_counter}][label]", option.label,
      :rows => 1, :cols => 40 %>
  </div>
  <div>
    <div><label>Feedback</label></div>
    <%= text_area_tag "options[#{option_counter}][feedback]", option.feedback,
      :rows => 1, :cols => 40 %>
  </div>
  <div>
    <label><%= radio_button_tag "options[correct]", option_counter, option.correct?,
      :id => "options_correct_#{option_counter}" %> Correct</label>
  </div>
  <div>
    <button type="remove">Delete option</button>
  </div>
</li>

In the HTML you see some extra attributes used by the JavaScript to determine what to do with them. When the partial is rendered with an new Option, the template is flagged by setting the repeat attribute to template. The id of this element is used as a handle for our controls, in our case this is ‘option’. The repeat-start attribute tells the javascript how many empty controls to generate initially from the template, we don’t want any so we’ve set it to 0. Note that we explicitly set the option_counter to ‘[option]’, this is the variable notation for the repetition model. When a new control is instanciated from the template this variable is replaced by the the index of the new control. The first control gets index 0, the second gets index 1 and so forth.

When the partial is rendered with a collection, the magic variable option_counter is set to the index of the collection every time the partial is rendered. We use this index to set the repeat attribute of the list item, this signals to the JavaScript that this is an already instantiated control. The JavaScript will start counting from the largest index when it instantiates a new control.

Finally we want the user to add and remove option controls in our page, this is done with the ‘New option’ and ‘Delete option’ buttons. Their type attribute signals what we want the JavaScript to do with the repetition blocks. In case of the ‘New option’ button the template attribute tells the JavaScript which template to instantiate.

In addition to managing the controls and template in the DOM tree, the JavaScript also takes care of disabling buttons on appropriate times and setting the CSS display property of the template to none. They really thought of everything.

The advantage of this solution is that you can use all the standard Rails tricks to keep your database clean. The biggest disadvantage is that the create method in your QuestionsController becomes a lot more complex.

Let’s hope native browser implementations follow quickly.

Today the Rails Core Team released Ruby on Rails 1.2. The long awaited new version is of course full of features and fixes. The thing we’re most excited about is the inclusion of ActiveSupport::Multibyte. But it doesn’t stop with multibyte support, Rails now ships with UTF-8 as a default in all parts of the framework, something we couldn’t have dreamed of a year ago.

We’re looking for a full-time Ruby on Rails developer. If you’re interested, please send email to thijs@fngtps.com

A few weeks ago Amazon launched UnSpun, a web application to collectively manage lists of all sorts.

During signup I was presented with the following.

I know Internet Explorer fixes a lot of broken encoding by guessing the true encoding for just about everything, maybe that’s why they never noticed during development?

I’ve had this problem myself on a few occasions. Because geographical information is commonly extracted from text files and loaded into a database you always have to be really careful to transcode any data extracted from text files to the same encoding as the database. In the case of ISO-8859-1/15, which is commonly used in west-european countries, there is a really simple oneliner to transcode to utf-8.

source.unpack('C*').pack('U*')

From the Mac OS X Leopard Technology Overview:

Leopard Server features a built-in installation of the powerful and productive Ruby on Rails web application framework. Ruby on Rails is a full stack framework optimized for sustainable productivity. Leopard Server will ship with Mongrel for simplified development and deployment of web-based applications.

That’s great. I only hope this bug will be fixed too. It would be somewhat ironic if you can’t use HTTP Basic Authentication in Safari with the new RESTful Ruby on Rails urls.

If you have access to the latest pre-release version of Mac OS X Leopard, please visit http://onautopilot.com/test;webkit and let us know if you get asked for a username and password or if it’s still broken.

Update: Tim found that you can make this work by url-escaping the semicolon. Add the following to your ApplicationController in app/controllers/application.rb:

# make HTTP Authentication work on Safari for RESTful Rails
def url_for(options = {}, *parameters_for_method_reference)
  result = super(options, parameters_for_method_reference)
  if request.env['HTTP_USER_AGENT'].to_s.include? 'AppleWebKit' 
    result.is_a?(String) ? result.gsub(';', '%3B') : result
  else
    result
  end
end

This is from a nice new app we’ve been working on for the last month or so. Any idea what it is and who we’re building this for?

Update: First right answer gets two great books. Can’t tell you what they are about without giving away the answer, I’m afraid…

Last week I posted a short screencast to show some features of ActiveSupport::Multibyte. After typing through the entire screencast twice I decided to automate the process. Screenager, the automated screencast typer, was born.

Download screencast (QuickTime, 544KB)

You can download Screenager from my personal Subversion repository, you will also need a recent version of ActiveSupport.

svn export https://dwerg.net/svn/screenager/trunk screenager
cd screenager
svn export --force http://svn.rubyonrails.org/rails/trunk/activesupport/lib
./screenager --speed 2 http://www.fngtps.com/files/2/2006/10/activesupport.rb

The version currently in SVN evaluates the Ruby code with eval using a clean binding every time you start a new screenplay. I really wanted to use the Freaky Freaky Sandbox for this, but it’s in heavy development and didn’t run at all when I was coding this. Sandbox and multiline Ruby statements are planned for future versions.

It was a lot of fun chatting with other developers at our second ‘morning coffee’ meeting today. Here’s a short video to give you a feel of the event:

Download video (Quicktime, 1.7MB)

As of revision #5223 ActiveSupport::Multibyte is part of Rails. Now everyone can enjoy multibyte safeness in their applications. Needless to say we are really happy. To show some of the features of Multibyte’s String#chars proxy I’ve put together a short screencast.

If you have any questions about ActiveSupport::Multibyte, please consult the API documentation and the Trac Wiki first. Enjoy.

Download screencast (QuickTime, 1.9MB)

My new favorite idiom to use in Ruby is the proxy pattern. But I’m not using it to reduce the memory footprint or to manage some complex resource. I’m using it to keep the original class clean of my torrent of methods and to make my API easy to use.

One of the biggest problems with adding methods to a core class is the chance of collision with other libraries or code, in order to keep this chance low you don’t want to flood classes with new methods. Imagine we want to perform some textual operations on a String, for example ‘Textilize’ it.

require 'redcloth'
RedCloth.new('h1. Proxies!').to_html

But that doesn’t look very nice, especially if we have to duplicate it. Duplication will get us in trouble when we want to change the parameters we send to RedCloth or when we want to switch to another textile processor.

class Formatters
  def self.textilize(str)
    RedCloth.new(str).to_html
  end
end

Formatters.textilize 'h1. Proxies!'

Even though we packaged the code nicely in a method, I’m still not satisfied. This will require us to type that long classname before every call to textilize. And what if we want to perform a number of operations on the same string? Brackets will have to come into play and Lisp our code.

module Formatters
  def textilized
    RedCloth.new(self).to_html
  end
  
  def unnewlined
    self.gsub!(/\r\n/, "\n")
  end
end

String.send :include, Formatters

"h1. Proxies!\r\n".unnewlined.textilized #=> "<h1>Proxies!</h1>"

Very nice! But there are still some problems with this solution. The formatting methods might accidentally override other methods on String, especially if we define even more of these formatting methods. Even though the methods work on the string, they don’t really have much to do with the String class.

This is where the proxy class comes into the picture, it will only take one method on the String class and work as a portal to all our formatting code. In the process the proxy class is even going to make our solution pluggable.

require 'rubygems'
require 'redcloth'

module Formatting
  class Formatter
    
    def initialize(str)
      @str = str
    end
    
    def to_str
      @str
    end
    alias_method :to_s, :to_str
  end
  
  module StringExtension
    def format
      Formatting::Formatter.new self
    end
  end
  
  def register(mod)
    Formatting::Formatter.send :include, mod
  end
  module_function :register
end

module Formatting::Textilize
  def textilized
    @str = RedCloth.new(@str).to_html
    self
  end
end
Formatting.register Formatting::Textilize

module Formatting::UnNewline
  def unnewlined
    @str.gsub!(/\r\n/, "\n")
    self
  end
end
Formatting.register Formatting::UnNewline

String.send :include, Formatting::StringExtension

See how the well placed self enables us to to chain methods?

"h1. Proxies!\r\n".format.unnewlined #=> #<Formatting::Formatter:0x1087c4c @str="h1. Proxies!\n">
"h1. Proxies!\r\n".format.unnewlined.textilized.to_s #=> "<h1>Proxies!</h1>"

There we go, a nice proxy implementation with all the benefits of the original solution. Unfortunately there are some drawbacks, instead of a String instance the methods return a Formatter instance. The to_str keeps us safe in most cases, like concatenation, but not in all cases. Further cloaking measures are left as an exercise for the reader (hint: method_missing, the boat and Comparable).

If you want to see a really cool proxy implementation, check out ActiveSupport::Multibyte.

Since we really enjoyed the first ‘morning coffee’ meeting, it is time for another one. The goal remains unchanged: a good chat about our experiences over a strong cup of coffee.

We have decided, however, to broaden the scope beyong Rails developers. Anyone working with next-generation web application frameworks, like Django or even Seaside, is invited too.

Please leave a comment to tell us you’ll be there or if you have any questions.

Today I’ve been merging the Rails multibyte support from Julik’s unicode hacks plugin into the current edge source. After a few testruns my Mac Mini started complaining about the normalization conformance tests…

Finished in 102.769516 seconds.

460 tests, 353652 assertions, 0 failures, 0 errors

I guess I’ll have to fix the Rakefile so I don’t overheat everyone’s computer.

URoR stands for ‘Unicode Ruby on Rails’ which is a series on using Unicode with Rails. In this second article I’ll show you how to enable the (somewhat limited) UTF-8 support in Ruby and Rails. (first article)

Let’s break a string

Suppose you’re using the truncate helper like this:

<p><%= truncate 'Iñtërnâtiônàlizætiøn', 12 %></p>

The result is something like:

Iñtërn…

Because the helper truncates the string to 12 bytes, it slices the codepoint for the ‘â’ in halve. The result is an invalid sequence which can’t be rendered.

Fix this by adding the following to the top of config/environment.rb:

# Add basic utf-8 encoding support 
$KCODE = 'UTF8'

And the result will be:

Iñtërnâti…

The string is now truncated to 12 codepoints instead of 12 bytes.

What’s happening here?

Setting KCODE to 'UTF8' tell Ruby that your source code is encoded as UTF-8. Some libraries like CGI and some parts of Rails look at KCODE to find out if they need to process strings in a UTF-8 friendly way.

You can now also require the jcode library you get some basic UTF-8 encoding support in Ruby. More about this in a future article.

Not all is good

Although it’s great that truncate has been fixed to work with UTF-8 you should be aware that this is not the case for all helpers:

<p><%= excerpt 'Iñtërnâtiônàlizætiøn', 'nâtiôn', 2 %></p>

This currently always breaks no matter if KCODE is set or not:

…rnâtiônàl…

You can again get the code from our subversion repository.

Looks like the caboose folks are ready for Rails Documentation Project proposals. If you want to work on the Rails documentation then please tell a little bit about yourself, why you’re qualified, and what you’d like to work on.

Tim Bray in the (for now) final Ruby Ape Diaries entry (emphasis added):

It’s easy to make people angry about this subject, and some of the angry people have a point; certain aspects of Unicode are, on the surface at least, objectively racist; for example, why does UTF-8 encoding of characters become progressively less efficient as you move from the languages of the Western hemisphere to those of the East?

Having said all that, it is my opinion that Unicode works pretty well, and in terms of making the Internet useful to the many peoples of Earth, is part of the solution, not part of the problem. And for that reason, I think that any language that doesn’t do a real good job at Unicode isn’t a very good citizen. And I think Ruby has a major problem in this area. Solutions are promised; we’ll see. And hey, in a few weeks I’m going to get up a stage in a room in Denver full of Rubyists and talk about this stuff; we’ll see whether they let me out of town alive.

Someone please, please record his talk; I’m really looking forward to what he has to say on the subject.

URoR stands for ‘Unicode Ruby on Rails’ which is going to be a series on using Unicode with Rails. In this first article I’ll show you how to set the Content-Type header so that the browser knows what you’re sending. (second article)

Set it in an after filter

On the web, the One And Only Sensible Encoding for Unicode is UTF-8, so that’s what we’re going to use. First, make sure your editor is set to save all files encoded as UTF-8. Then create a new Rails application and generate a controller called ‘static’ with an ‘index’ action so that we have something to test with.

$ rails uror
$ cd uror/
$ ./script/generate controller static index

Now add the following to app/views/static/index.rhtml (just copy it from this page and paste it into your editor):

<p>Iñtërnâtiônàlizætiøn</p>

Run the Rails application with ./script/server and go to /static/index where you should get something garbled that looks like this:

Iñtërnâtiônà lizætiøn

The problem is that you haven’t told the browser that you’re using UTF-8. Fix this by changing app/controllers/application.rb to:

class ApplicationController < ActionController::Base
  after_filter :set_encoding
  
  protected
  
  def set_encoding
    headers['Content-Type'] ||= 'text/html'
    if headers['Content-Type'].starts_with?('text/') and !headers['Content-Type'].include?('charset=')
      headers['Content-Type'] += '; charset=utf-8'
    end
  end
end

The set_encoding after filter does two things:

1. It sets the Content-Type header to text/html, but only if no Content-Type header has yet been set. This is exactly what Rails would have done anyway, but we’re doing it here so that…
2. It adds charset=utf-8 to every Content-Type header for a text type when no charset has yet been set.

If you now reload the page the problem is fixed because the browser is no longer receiving a:

Content-Type: text/html

header, but:

Content-Type: text/html; charset=utf-8

Also set it in your Lighttpd or Apache configuration

It’s a good idea to set the UTF-8 encoding in your web server configuration too. For Apache add the following in public/.htaccess or your main configuration:

AddDefaultCharset utf-8

For Lighttpd, change mimetype.assign in config/lighttpd.conf to:

mimetype.assign = (  
  ".css"        =>  "text/css; charset=utf-8",
  ".gif"        =>  "image/gif",
  ".htm"        =>  "text/html; charset=utf-8",
  ".html"       =>  "text/html; charset=utf-8",
  ".jpeg"       =>  "image/jpeg",
  ".jpg"        =>  "image/jpeg",
  ".js"         =>  "text/javascript; charset=utf-8",
  ".png"        =>  "image/png",
  ".swf"        =>  "application/x-shockwave-flash",
  ".txt"        =>  "text/plain; charset=utf-8"
)

Now all static stuff like 404.html and cached pages are also sent with the correct encoding in the Content-type header.

Even add it to the head

If you want make it easy for people to save your pages to disk and open them with the correct encoding later on, you might want to add the following inside the head element of your html pages:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

Always do this last as it may mask any trouble you might be having with the http headers.

Update: You can now get all URoR code from our subversion repository.

Update 2:: The upcoming 1.2 release of Rails will add utf-8 as the default charset for all renders, so you’ll no longer need the after filter.

Although I fully agree with David that Joel Spolsky’s Language Wars is one of the purest forms of FUD against Ruby and Rails ever, I do think Joel got this one right:

I for one am scared of Ruby because (1) it displays a stunning antipathy towards Unicode […]

Sad as it may be, this fear is mostly justified.

In march I wrote a post about our fcgi monitoring and management tool Fcgimon. Today I fixed some rough edges and decided to call this version 1.0.

The biggest change is the way Fcgimon gathers and presents it’s data from `ps`. Earlier versions would just find all the dispatchers and spawners in the process list and show them. This means that not running application would just not show up in the list. Fcgimon now shows the status of all the configured projects, which makes it easier to detect broken spawners. Fcgimon will still show all fcgi processes and spawners outside of the configured projects, but under the normal list and with a small warning.

Minor changes include memory usage information and in general nicer output.

If you like what you see you can download Fcgimon from our Subversion repository.

https://fngtps.com/svn/server-scripts/trunk/fcgimon/

Never try to find the authenticated user in a Rails app running on MySQL with something like:

@authenticated = Person.find_by_id session[:authenticated_id]

Only use find_by_id if session[:authenticated_id] is not nil:

unless session[:authenticated_id].nil?
  @authenticated = Person.find_by_id session[:authenticated_id]
end

When you do a find_by nil the following query is executed:

SELECT * FROM people WHERE (people.`id` IS NULL) LIMIT 1;

Most of the time this returns an empty set, but if the previous query was an insert into the people table, MySQL will return the row for the last insert. This is because NULL somehow returns the generated id:

mysql> SELECT id FROM people WHERE id IS NULL;
Empty set (0.00 sec)

mysql> INSERT people SET username='bob';
Query OK, 1 row affected (0.08 sec)

mysql> SELECT id FROM people WHERE id IS NULL;
+----+
| id |
+----+
| 11 |
+----+
1 row in set (0.00 sec)

mysql> SELECT id FROM people WHERE id IS NULL;
Empty set (0.00 sec)

It would take some time and determination to exploit this, but it’s certainly not impossible.

Update: This was fixed in Rails just after the release of 1.2.1.

I always follow the Ruby on Rails development timeline to see if my patches get applied and if there are any nice new features. Turns out that apart from the usual stuff, there’s the funnies on page three. Just like in the newspaper.

Changeset 4661 Changeset 4678 Changeset 4700

Last week our Paypal integration for Dr Stat suddently started acting up. During posts to the notification URL it threw OpenSSL::SSL::SSLError (unknown protocol) errors. I frantically began searching the web for problems with OpenSSL and Ruby, but I couldn’t find anything.

The solution turned out to be really simple. The Paypal gem had been updated and now supports SSL connections. Apparently it raises exceptions when you don’t use SSL now. So I changed Paypal::Notification.ipn_url from the HTTP to the HTTPS url and everything started working again.

You can now subscribe to our podcast with talks from the Rails ‘show and tell’ meeting:

Subscribe with your default feed reader
Subscribe with iTunes

Right now we only have ‘Unicode in Rails’ available, but you can expect more talks in the coming days. It’s also possible to download individual talks.

A big ‘thank you’ to Andy Lo A Foe, Rien Swagerman from Resource Studio, Simon de Haan from Eight Media, Flurin Egger from DigitPaint and Julian ‘Julik’ Tarkhanov for their talks.

We also like to thanks Greenpeace International for letting us use the basement and especially the Application Development Team for helping out with the gear.

More photos can be found in our Rails ‘show and tell’ meeting flickr set.

The Unicode roadmap thread on the Ruby Lang mailing list is now almost 100 messages long.

Yukihiro Matsumoto (matz):

I am too optimized for Ruby string operations using Regexp.

Tim Bray:

Julian ‘Julik’ Tarkhanov:

I think this thread is going to end the same as the one in 2002 did.

Read the whole damn thing if you want to know more about the gritty details. In case it makes your head hurt, go read On the Goodness of Unicode, Characters vs. Bytes and The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) first.

Here’s the list of talks for the Rails ‘show and tell’ meeting.

Introduction from Greenpeace

Martin Lloyd from the Greenpeace International Application Development Team will give a short overview of the work they’ve done with Rails and Django.

Audio (AAC) | Audio (MP3)

Web Services

Andy Lo A Foe will show us how to connect your Rails project to legacy Java and PHP applications using XML-RPC, SOAP and REST-style web services.

Audio with slides (AAC) | Audio (MP3) | Slides (PDF)

Content Management with Rails

Rien Swagerman from Resource Studio will talk about his experiences with building a CMS on Rails that’s tailored to the needs of his design studio.

Audio with slides (AAC) | Audio (MP3) | Slides (PDF)

Why Django?

Simon de Haan from Eight Media will talk about why they choose Django and not Rails as their default web development framework. An excellent opportunity to learn what the other hot new web framework is all about.

Audio with slides (AAC) | Audio (MP3) | Slides (PDF)

Mailing newsletters

Flurin Egger from DigitPaint will talk about the development of EntopicMail, a Rails application for mailing out newsletters that’s currently in use at organisations like Philips, NEN and the dutch Ministry of Health, Welfare and Sport.

Audio with slides (AAC) | Audio (MP3) | Slides (PDF)

Unicode in Rails

Julian ‘Julik’ Tarkhanov will show us why the lack of Unicode support in Ruby is a problem when developing Rails applications. He’ll also talk about how he developed a very nice way to properly handle Unicode data in Rails.

Audio with slides (AAC) | Audio (MP3) | Slides (PDF)

Update: Added audio and slides.

The current Ruby version has no Unicode String class like in Python or Java. This makes it hard for Rails to support multibyte encodings.

The following code snippet from the truncate helper is a good example:

if $KCODE == "NONE"
  text.length > length ? text[0...l] + truncate_string : text
else
  chars = text.split(//)
  chars.length > length ? chars[0...l].join + truncate_string : text
end

This was added to make the helper work with multibyte characters, but it is far from beautiful.

A few days ago, Julian proposed to add a proxy to the string class for accessing characters instead of bytes. I think this is an excellent and very nice solution.

You access the proxy with the char method on a string object. You can for example get the number of characters with:

text.chars.length

The char method is aliased as u, so you can also write:

text.u.length

Which to me looks even nicer.

Using the proxy, you could replace the six lines of code from the truncate helper with:

text.chars.length > length ? text.chars[0...l] + truncate_string : text

That’s a whole lot more obvious. And don’t be fooled, this is just as fast as the longer version because the proxy only uses the multibyte safe methods when $KODE is set.

Apart from making the Rails code easier to understand and maintain, the proxy can also save application developers a lot of work.

The proxy and the patches to the Rails code are currently in development as a plugin you can get from Subversion. Even though the plugin is called ‘Unicode hacks’ for historical reasons, it’s actually a very clean solution by now. There’s also a proposed patch to the Rails source.

Please try this one out and give your feedback.

The goal for this new meeting is to have a few developers talk about the stuff they do with Rails. It’s going to be a fast-moving and informal meeting; talking time will be limited to 10 minutes and there will be lots of opportunity for asking questions. We’ll have both dutch and english language speakers.

What: 6 short presentations by Rails developers

When: Thursday, June 22nd, 2006, 14:30-16:30

Where: The basement of the Greenpeace International headquarters at the Ottho Heldringstraat 5 in Amsterdam

Registration required: Leave a comment with your email address to register for this meeting (click ‘also leave contact information’).

Last week we’ve started working on ‘Rhubarb on Rails’, a new version of a Greenpeace application for running letter writing actions. It’s going to replace a PHP based tool that was a quick replacement for an old Zope application that decided to roll over and die somewhere in the fall of 2005.

The nice thing about working for the Greenpeace development team is that they thoroughly get the Agile development methodology. Even better is that they do a lot of their work ‘in public’ as you can see on their Work in Progress weblog.

Rhubarb will be released under an Open Source license, so it can be used and improved on by other big NGO’s or small local organisations. You follow our work and download the source from our development site.

Update: The project has been renamed to ‘Write-A-Letter’.

I just found out that in Spain they’re drinking beer instead of coffee at Rails meetings.

Martin Fowler in ’Evaluating Ruby’ (emphasis added):

Too many things are hard to judge that way - hence we spend so much of our time on client projects being slowed down by technology that sounded good on a golf course. My preference is to make this judgment based on experience - find people who have a track record for delivering in the mainstream environments and who have tried using Ruby.

We recently received a two page REQUEST FOR PROPOSAL for a ‘Social Network Site’:

[…] has developed a concept that requires the development of a niche social network solution, incorporating some of the most innovative features of Web 2.0: RoR, Social Networks with extensive search features, Web Services, Blogs, Tagging, Photo Management, RSS, Internal Messaging, etc. Users are able to sign up, create a profile with pictures and personal data, set preferences and demographic information. This site differentiates itself from the other social networks by providing innovative features in a specific niche.

[…] would like the site to contain the following features.

• User Registration
• User Profile Setup
• User addition to a Network
• Photo management
• Ability to post personal blogs
• Ability to comment on a members profile
• Messaging capabilities
• RSS integration
• Ability to search the social network with varying search parameters
• External partner data feed and integration through SOAP/XML API’s
• Extensive niche specific features (further details in Phase 2).

Based on this information, we were asked to ‘provide a detailed estimate with breakdown by task (description of task, duration, total number of hours, etc.). ’ and to ‘a timeline for completion of the project (start date, completion date, milestones)’. Within 5 days.

Maybe I’m being arrogant here, but I don’t see how someone can realistically expect a developer to write a fixed time, fixed budget proposal based on this game of buzzword bingo. Can you?

The Greenpeace International Application Development Team is currently learning Ruby on Rails by writing an application during a three-day retreat.

Yesterday we dropped by for a few hours together with Andy to help the developers get started. Because of networking issues we didn’t get as far as we’d hoped, but after a few hours everybody was at least up and running.

This morning we ran the developers through a 90-minute crash course explaining the basics of Rails. It still amazes me how PHP and Java developers suddenly get this somewhat amazed but very happy look on their face when they realize how much nicer using Rails is going to be compared to the stuff they’ve been doing before.

You can follow the efforts of the development team on their weblog.

When we proposed a Rails meeting two weeks ago, we thought we’d be having coffee with 4, maybe 5 fellow developers from Amsterdam. We never expected more than 30 people to show up, certainly not people all the way from Eindhoven, Venlo or Groningen!

It was a lot of fun chatting about Rails with all of these people. Please leave a comment if you’d enjoyed the meeting too.

We’re currently planning another fun event for Rails developers. We’ll post more information about it as soon as we get some things confirmed.

Update: You can find some photos here and here. For more comments on the meeting, see this, this and this post (all dutch).

There’s a lot of interest in the Rails meeting we proposed last week.

We’ll answer some of the questions that keep coming up.

Wouldn’t it be kinda busy?

Yes, probably, but we don’t think it will be too crowded.

I have to leave very early because I’m coming from Venlo/Groningen/someplace else far away. Can’t you start later?

Please remember that this is nothing more than an informal meeting at a regular coffee bar. It’s not a conference or seminar, there are no speakers or workshops.

It would be great if you could make it, but please keep in mind that there will be nothing more to it than buying a cup of coffe and having a good chat about Rails.

Will it be easy to park my car there?

No, parking in Amsterdam is never easy. See bereikbaar amsterdam for more information on where and how to park your car.

How do I get there from the central station?

Take tram 4, 9, 16, 24 or 25 and get out at the Muntplein. Cross the bridge to hotel de l’Europe. You’re now on the nieuwe Doelenstraat where you’ll find the Coffee Company after 100 meters on the right side.

In the last few months we found that there are more and more Dutch people working with Ruby on Rails. So far we’ve kept in touch by email and instant messaging, but now we’d like to meet up in person. Our goal is to have a good chat about our experiences with Rails, and of course a strong cup of coffee is a precondition for such a thing.

When: Thursday, March 30th, 2006, 9:30 AM

Where: The Coffee Company on the corner of the Nieuwe Doelenstraat and the Kloveniersburgwal in Amsterdam

This is an informal meeting intended for people working with Rails, developers and designers who want to get started, and everyone wondering if Rails might be a good fit for their project.

Please leave a comment to tell us you’ll be there or if you have any questions.

Update: We’ve answered some of the questions that keep coming up in this post.

When we started using Switchtower Capistrano to deploy our projects, we had some trouble with Lighttpd herding the fcgi processes. Because the standard Capistrano tasks expect the processes to be managed externally anyway, we decided to stop using Lighttpd for this.

Most of the existing tools for managing fcgi processes are designed to do complex stuff like load balancing across different servers. What we needed was a simple tool for managing multiple Rails applications on a single server. So we wrote Fcgimon.

Fcgimon manages fcgi processes using the spinner and spawner scripts that come with Ruby On Rails.

You specify all Rails applications and the number of fcgi processes you want to have running in a configuration file. Then use fcgimon to start or stop all fcgi processes for any single application, or start and stop all applications at once.

Fcgimon also generates snippets that can be included in the main Lighttpd configuration file. This makes it much easier to keep the number of running fcgi processes and the ports listed in the Lighttpd configuration file in sync.

You can download the latest version of Fcgimon from our Subversion repository. Please give it a try and tell us what you think.

I really love Basecamp, but there’s this little issue with attached files that keeps biting me. The problem is that the url of the file changes after you’ve saved a new message.

So, I’m working on a design, and I’d like to show it to my client. After I upload the file, I copy the url by right clicking its link. Then I paste the url with formatting codes that include the image in the message body:

After I post the message, the image is missing:

So I click ‘Edit’ to find out what I did wrong. As you can see, the url for the attached file has changed. I copy and paste the new url in the message body:

And now it works:

What probably happens is that uploaded files are saved to a temporary location when you’re working on a new message. You have to post the message before a message object is created. Only after a message object has been created, is there a message id where attachment objects can be linked to. The problem is that you simply can’t link the uploaded files to a message that doesn’t yet exist.

Please note that this is a completely sensible solution for this kind of application; I’m probably one of the very few people who likes to include images inside the message body.

When you write an application for managing the content of a website this is much more of an issue. The people who publish articles on their website are very likely to want to include images.

For simple systems, we often take the easy route. You first have to save the article as a draft:

Only then can you attach and include images:

For more complex systems, we’ve solved this problem by saving a record for the article to the database right after you click ‘Add article’ so that we always have an article id to link the attachments to.

These articles have a status attribute that is set to ‘new’. The status is changed to ‘draft’ or ‘published’ after you save the article. If you don’t save a new article, it gets garbage collected within a few days.

This is what the relevant parts of the article model look like:

class Article < ActiveRecord::Base
  has_many :assets, :order => :position, :dependent => true

  def self.collect_garbage
    resources = Resource.find :all, :conditions => "status = 'new' AND
      created_at < DATE_SUB(NOW(), INTERVAL 7 DAY)"
    resources.each {|resource| resource.destroy}
  end
end

Article.collect_garbage is called in the list action of the article controller, but you could also use a cronjob of find some other way to call it regularly.

This is what the relevant parts of the article controller look like:

  def new
    @article = Article.new
    @article.save
    redirect_to :action => 'edit', :id => @article.id
  end

  def edit
    @article = Article.find(params[:id])
    if request.post?
      @article.attributes = params[:article]
      if params[:upload]
        # handle asset upload
        asset = @article.assets.build
        if asset.handle_upload(params['asset'])
          asset.save
          flash['highlight'] = "asset#{asset.id}"
        end
      else 
        # handle article submit
        if @article.save
          flash['highlight'] = "article#{@article.id}"   
          redirect_to :action => 'list'
          return false
        end
      end
    end
  end

You can view an example of how this works (Quicktime).

A few months back we decided to release the inject plugin, however we didn’t post it to the Rails wiki because we were still reorganizing our domain structure and didn’t have anonymous Subversion access to our repositories.

For now we have two plugins in the trunk, but more might follow in the future.

Inject

The inject plugin allows you to do cross-domain AJAX-like http calls. You can find (a little bit) more information on the Rails wiki entry for Inject.

Url for domain

Url for domain adds :subdomain support to the url_for method. Actually it adds this to a the url rewriter code, so you can use this options on every helper/method with url_for-like syntax. Again, more information can be found on the Rails wiki entry for Url for domain

Have fun with the plugins and lets us know if you have any problems. We have a little project coming up which uses both these plugins, so keep an eye out for that.

In the past I usually tested the feeds a Rails application generated by writing a functional test that checked the HTTP status code and matched certain strings in the feed using a regular expression. If that checked out I hand-tested the feed using the online feedvalidator. Needless to say, this became very cumbersome after a few times, especially for one of our projects that generates a whole list of different feeds depending on the state of the account. Time to add validation to the functional tests.

Unfortunately the feedvalidator is (not yet) written in Ruby, so we have to do system calls to a python script to validate. I installed the feedvalidator in the script directory, which looked like the correct place to put this. A protected method on the testcase makes sure we can easily test feeds.

def validate_feed(content)
  validate = File.dirname(__FILE__) + '/../../script/feedvalidator/validate.py'
  path = Pathname.new(File.dirname(__FILE__) + '/../tmp')
  Tempfile.open('feed', path.cleanpath) do |tmpfile|
    tmpfile.write(content)
    tmpfile.flush
    result = `#{validate} #{tmpfile.path} A`
    unless result =~ /No errors or warnings/
      raise "Feed did not validate: #{result}"
    end
  end
end

So now the testcases looks like this:

def test_atom
  get :atom
  validate_feed @response.body
  assert_equal 'application/atom+xml', @response.headers['Content-Type']
end

Note that this validates the feeds on level ‘A’ which only tests for MUST directives. For completeness you should probably still test either with the online validator or temporarily set the validation to ‘AA’ or even ‘AAA’.

Rumor has it that if you validate your feeds at ‘AAA’ level Mark Pilgrim will come to your house and crown you Prins of the Feeds. Lucky lucky!

If you’re still unsure if you should worry about utf-8 encoded strings, look no further than the Projectionist RSS feed:

See those &#82’s in the headlines list and the big R in the title? That’s where a poor right double quotation mark was split in halve.

Chad Fowler explains why opening classes in Ruby is a good idea. He also mentions how plugins allow the Rails core team to push out ‘unstable’ features to the public.

The Rails team is already using plugins like this as a way to get new functionality into circulation as a kind of proving ground before things make it into Rails core.

I really like this model because it allows the Rails core team to keep the svn trunk clean from any unstable code. The code can be tested in the wild through plugins, when the feature is used enough and appears to be stable, it can be included. The downside is that plugins which open Rails classes have to be re-written for every release.

By default, Ruby has no encoding or character set support in its String class. You’ll notice this when you try to slice a string on a multibyte character:

"Café"[2..3] #=> "f\303"

You get "f\303" instead of "fé" because Ruby slices the string on the byte and not on the character boundary.

Encodings and Character Sets

To represent a written language in a computer you need a character set and an encoding. The character set is a mapping from an integer to a glyph (an image representing a character). The encoding is the way you represent those integers in a sequence.

For instance the ASCII character set consists of only 128 characters and can thus be represented in just 7 bits. Encoding ASCII is trivial; each character fits in one byte and a string is a sequence of bytes.

Languages that use more that 255 glyphs can’t fit each character into one byte, so they need another way to encode their strings, for instance by using two bytes to encode one character. History has produced a long list of different character sets and encodings, certainly not all interchangeable.

Unicode is an effort to unite all known character sets, so we have to deal with only one. Unicode also makes it possible to use multiple languages using widely different character in one document. UTF-8 and UTF-16 are the most widely used encodings for Unicode strings.

Fixing String in Ruby

The String class assumes that every byte in the string represents a single character. This means that String#length and String#slice may not return the result you expect if you’re handling multibyte characters.

Some of this behaviour can be fixed with the jcode library. The jcode library updates a number of methods on the String class: chop!, chop, delete!, delete, squeeze!, squeeze, succ!, succ, tr!, tr, tr_s!, and tr_s. It also adds jlength and jcount. The encoding assumed in a string is globally defined in the global variable KCODE.

$KCODE = 'UTF8'
require 'jcode'

"Café".jlength #=> 4

The drawback of this solution is that it doesn’t include much used methods like String#slice.

Another solution is the Unicode library by Yoshida Masato. This library contains some functions to handle UTF-8 encoded strings.

Fixing String in Rails

Following the howto on using Unicode strings in Rails can get a bit baroque to say the least.

It’s easy to break things; when a helper uses the slice method on a multibyte strings, it might chop a character in half as we saw above. Even worse, validations like validates_length_of can fail or succeed when you don’t expect them to.

A nice way to fix these problems would be to have multibyte support in the String class in such a way that it works exactly like the current String class. It looks like this is planned for a future version of Ruby.

Julian Tarkhanov proposed to fix the string class by overriding the existing one with Unicode aware methods where necessary. He has hacked up a version using the Unicode library and packaged it as a plugin. The plugin also puts your database server in UTF-8 mode and updates the Content-Type header sent by Rails.

The UTF-8 hack

Testing the hack

Overriding default classes can be dangerous. Overriding widely used classes like the String class can be a recipe for disaster. Fortunately Rails has good test coverage so we can get an idea of how well the hack works.

I did an svn export of the Rails stable branch (essentially 1.0 with a few patches). Then I put string-overrides.rb from the plugin in the root of the export and I copied the Unicode library from my RubyGems directory to the root of the export, so I wouldn’t have to use require_gem. After that I added the following lines to all the libraries in Rails (actionpack, activerecord, etc):

$KCODE = 'UTF8'

$:.unshift(File.dirname(__FILE__) + '/../../')
require 'unicode'
require 'string_overrides'

The changes to the String class introduced 3 errors and 1 failure. The failure is because of a missing \n in the result, which seems to be harmless. I’m not sure if the three new errors are fatal, but I suppose they can be fixed. You can download the complete test results.

Speed

Adding UTF-8 support to the String class is likely to result in a speed penalty. Most web applications push a lot of text around, so using this hack could potentially lower the performance of an application substantially.

Testing is knowing, so I tested the plugin with two Rails applications. The first is a small application that displays a page with some messages and a form. The second application is a production application, on which I tested the account page. Ab2, the apache benchmark program, was used to test the number of requests per second.

I have to warn in advance that this doesn’t give a very accurate impression of the actual performance, but it does give an impression of the speed penalty I got by using the Unicode plugin.

You can view the complete ab2 output.

The performance impact is certainly noticeable. Although I’m not certain what part of the implementation is causing this performance penalty, the introduced levels of indirection in the string class probably don’t help. A native C implementation could probably be a lot faster.

Unresolved issues

By creating this plugin we haven’t resolved all our problems. One of the biggest problems is that we can only process UTF-8 encoded strings. Although all the input from forms should be in UTF-8 as specified in our HTML documents and headers, information from other sources, like the filesystem, could still be in a different encoding. In this setup we have to make sure we don’t send data in a different encoding than we promise in our headers. Sure, there are solutions like iconv to re-encode this data, but life would be a lot simpler if we didn’t have to think about this.

Summary

Although a native Ruby character set and encoding aware String class would be the ultimate solution, the Unicode hack plugin for Rails provides you with the tools to use UTF-8 in your Rails application. This support comes with a noticeable performance penalty.

Further reading

• The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
• Characters vs. Bytes
• UTF-8 and Unicode FAQ
• Short presentation about Unicode

Stills from the Snakes and Rubies video

David explains how you should market your project at the Snakes and Rubies event:

Well, making a stir. Like taking a big target and then picking on it. I really recommend Java, it works great. There are so many Java programmers out there and you just have to poke at them a little bit and then they go like bananas and link to you like mad. And if you poke them in the eyes they’ll go even better bananas.

That works pretty well for like breaking throught the ‘early awareness wall’ where basically what you need is just get it out there. Then you probably want to switch horses at some point in the game when you want those Java programmers to come over. That’s a good idea and I hope that we’ve done that at least somewhat when we can resist the temptations.

If you’re in any way interested in Rails or Django or both, you’ll probably enjoy the video of the event.

One of the more convenient features in Ruby on Rails is page caching. Simply add caches_page :show to the top of a controller class, and all pages rendered by the show action are written to disk automatically. On subsequent request, these pages will be served straight from disk without invoking Rails at all.

This works because of rewrite rules that basically tell the webserver to append .html to the request path. If the webserver can find a file using the resulting path, the webserver will send it. If not, then Rails will handle the request.

Pages are removed from the cache simply by deleting them from the public directory. Rails provides the expire_page method and sweepers to help with this.

Sweeping is hard

Suppose you are writing a blogging application and you decide to add page caching. When a post is updated, the cached page that shows the post has to be removed. You write a post sweeper for this:

class PostSweeper < ActionController::Caching::Sweeper
  observe Post

  def after_save(post)
    expire_page(:controller => "post", :action => "show", :id => post.id)
  end
end

But wait… Your blog also has a front page listing the most recent posts. The updated post might be included there, so you need to expire that page too.

def after_save(post)
  expire_page(:controller => "post", :action => "show", :id => post.id)
  expire_page(:controller => "post", :action => "index")
end

Then you realize you also have archive pages and category overviews…

def after_save(post)
  expire_page(:controller => "post", :action => "show", :id => post.id)
  expire_page(:controller => "post", :action => "index")
  expire_page(:controller => "archive", :action => "show", 
    :year => post.published_at.year)
  expire_page(:controller => "archive", :action => "show", 
    :year => post.published_at.year, :month => post.published_at.month)
  expire_page(:controller => "archive", :action => "show", 
    :year => post.published_at.year, :month => post.published_at.month, 
    :day => post.published_at.mday)
  post.categories.each do |category|
    expire_page(:controller => "category", :action => "show", 
      :id => category.id)
  end
end

Ok, but what if a post is destroyed? And what exactly should happen when a category is renamed? And…

When you have an application where a single change can invalidate a large number of pages, the sweepers can get quite complex. It’s easy to forget to expire one or more pages, leading to subtle bugs where old pages are served from a stale cache.

An obvious solution to this would be to just sweep all pages after each change. Sadly, this is not possible with page caching because Rails does not keep a list of cached pages. The files are written directly to the public directory, so there’s no way to cleanly delete them all.

Lazy sweeping

We’ve tried to solve the problem of not being sure which files in the public directory are just cached copies and which pages are static html, by moving all cached pages to a public/cache subdirectory. This seems to work fine for us.

In config/environment.rb, change the page cache directory from the default by adding the following line inside the Rails::Initializer.run block.

config.action_controller.page_cache_directory = RAILS_ROOT+"/public/cache/"

Then change the rewrite rules in the webserver configuration. For lighttpd (config/lighttpd.conf) these should be changed to:

url.rewrite = ( 
  "^/$" => "cache/index.html", 
  "^([^.]+)$" => "cache$1.html" )

For Apache (public/.htaccess) the first two rules probably need to be changed to:

RewriteRule ^$ cache/index.html [QSA]
RewriteRule ^([^.]+)$ cache/$1.html [QSA]

We use the following in app/models/site_sweeper.rb as a single sweeper for all the models in our application.

class SiteSweeper < ActionController::Caching::Sweeper
  observe Post, Category

  def after_save(record)
    self.class::sweep
  end
  
  def after_destroy(record)
    self.class::sweep
  end
  
  def self.sweep
    cache_dir = ActionController::Base.page_cache_directory
    unless cache_dir == RAILS_ROOT+"/public"
      FileUtils.rm_r(Dir.glob(cache_dir+"/*")) rescue Errno::ENOENT
      RAILS_DEFAULT_LOGGER.info("Cache directory '#{cache_dir}' fully swept.")
    end
  end
end

Finally assign the site sweeper to all controllers and actions that may invalidate the cache.

cache_sweeper :site_sweeper, :only => [:add, :update, :destroy

We’ve also added the following script as script/sweep_cache to easily sweep the cache during development.

#!/usr/bin/ruby
require File.dirname(__FILE__) + '/../config/boot'
require File.dirname(__FILE__) + '/../config/environment'
SiteSweeper::sweep

This approach can be extended very nicely for the subdomains as account keys pattern. More on that later.

For the last few months I’ve been using TextMate on my iBook and on the Lil’ Mac at work. I’ve grown accustomed to the various ‘insert snippet’ commands available in TextMate. Unfortunately Vim doesn’t have these commands and I find myself inserting ^Z in my views.

However, Vim configuration is powerful enough to implement these commands. Make sure you’re in command mode and do the following:

:imap <C-z> <lt>%=  %><Left><Left><Left>
:imap <C-x> <lt>%  %><Left><Left><Left>

This will allow you to insert <%= %> and <% %> into your files during insert mode. It is also possible to insert these commands in ~/.vimrc so you don’t have to define them every time you launch vim.

You can find the other mapping syntax rules on the vim.org website and in your vim help files.

As most of you already know, Switchtower is a very nice way to deploy your Rails application.

When I tried to deploy one of Thijs’ applications using Switchtower, I ran into some trouble. Switchtower tried to chmod 666 the log file, but the log file was owned by thijs:thijs.

Jamis Buck asked me to submit this bug to the Rails trac site, so I did just that. To solve the problem for the time being, I added the following to my deploy configuration:

desc "Change permission of the project to avoid problems" + 
    "with source.checkout and symlink"
task :before_update_code do
  sudo "chown -R $USER #{deploy_to}"
end

desc "Set permissions on files so the webserver can work with the files"
task :after_symlink do
  sudo "chgrp -R www-data #{shared_path}/log"
  sudo "chgrp -R www-data #{shared_path}/system"
end

If you want to use this hack, make sure you change www-data to the group your webserver/application server runs under. If you’re running Debian or Ubuntu, this should work. Make sure any other directories the webserver needs to write to, like page cache directories, are also properly chgrp’d.

Test are supposed to be run often, but they only get run if they’re easy to run. When you create a plugin in your Ruby on Rails project, it doens’t get run by default.

And that’s not very easy now, is it? Fortunately there is a standard task to run the plugin tests, which is called `test_plugins’. How do we get this task to run all the time?

RoR allows us to specify our own tasks in /lib/tasks, so we’re going to add a file called `test_plugins_by_default.rake’ to this directory, and we put the following in the file:

desc "Run test:plugins by default."
task :default do
  namespace 'test' do
    Rake::Task[:plugins].invoke
  end
end

Very well sir, but why does this work? The Rake User Guide tells us that a task that is declared multiple times, cumulatively adds it’s actions to the task. So ‘test_plugins’ will now be run in the default task.

A month or so ago I found myself staring at my screen, I had done something stupid. Almost the entire application was finished, but when I tested it on another domain nothing worked. When I started exploring the Firefox javascript window and pasted some errors in Google, I found out I couldn’t do cross-domain XMLHttpRequests. I looked at Thijs, my boss / colleague / friend, and said: ‘We have a little problem.’ After some looking around we found out that Julien Lamarre had solved our problem.

In order to fit this solution into our Rails app, I wrote inject.js and some helpers. When plugin support came out I converted my solution to a plugin, and after some testing it is ready for the world. You can download it here.

Some comments are probably in place. The Inject.Request object doesn’t work exactly like the Ajax.Request object as it always evaluates the response from the server and doesn’t support the same arguments. Please take a look at the code, because it’s the best documentation right now.

The general idea behind the javascript is that it creates an invisible div element in the body element of the page and injects script tags into this div. The URL’s in the src attribute of the script tags are actually calls to Rails controllers and actions. You will have to return javascript from the actions if you want to change the state of the page, play nice and return a ‘text/javascript’ content-type when you do this.