Rspec custom matchers and be_valid_xhtml

Ok. C’mon. Who’s as anal as me when it comes to validating their xhtml? It’s such a beautiful crystaline structure. You don’t want to debalance it by leaving out an angle bracket. The whole thing will fall over.

There’s a nice plugin available at http://www.realityforge.org/svn/code/assert-valid-asset/trunk/ that provides the ability to hit the w3c validator up with your data, but I’m kicking ass and taking names with rspec these days, so I thought I’d wrap it up as a custom matcher for rspec (download here):

[ruby]
class BeValidXhtml
# require ‘action_controller/test_process’
# require ‘test/unit’
require ‘net/http’
require ‘md5′
require ‘ftools’

def initialize
end

# Assert that markup (html/xhtml) is valid according the W3C validator web service.
# By default, it validates the contents of @response.body, which is set after calling
# one of the get/post/etc helper methods. You can also pass it a string to be validated.
# Validation errors, if any, will be included in the output. The input fragment and
# response from the validator service will be cached in the $RAILS_ROOT/tmp directory to
# minimize network calls.
#
# For example, if you have a FooController with an action Bar, put this in foo_controller_test.rb:
#
# def test_bar_valid_markup
# get :bar
# assert_valid_markup
# end
#
MARKUP_VALIDATOR_HOST = ENV['MARKUP_VALIDATOR_HOST'] || ‘validator.w3.org’
MARKUP_VALIDATOR_PATH = ENV['MARKUP_VALIDATOR_PATH'] || ‘/check’
CSS_VALIDATOR_HOST = ENV['CSS_VALIDATOR_HOST'] || ‘jigsaw.w3.org’
CSS_VALIDATOR_PATH = ENV['CSS_VALIDATOR_PATH'] || ‘/css-validator/validator’

@@display_invalid_content = false
cattr_accessor :display_invalid_content

@@auto_validate = false
cattr_accessor :auto_validate

class_inheritable_accessor :auto_validate_excludes
class_inheritable_accessor :auto_validate_includes

def matches?(response)
fn = response.rendered_file
fragment = response.body
return true if validity_checks_disabled?
base_filename = cache_resource(‘markup’,fragment,’html’,fn)

return false unless base_filename
results_filename = base_filename + ‘-results.yml’

begin
response = File.open(results_filename) do |f| Marshal.load(f) end
rescue
response = http.start(MARKUP_VALIDATOR_HOST).post2(MARKUP_VALIDATOR_PATH, “fragment=#{CGI.escape(fragment)}&output=xml”)
File.open(results_filename, ‘w+’) do |f| Marshal.dump(response, f) end
end
markup_is_valid = response['x-w3c-validator-status'] == ‘Valid’
@message = ”
unless markup_is_valid
fragment.split($/).each_with_index{|line, index| message << “#{‘%04i’ % (index+1)} : #{line}#{$/}”} if @@display_invalid_content
@message << XmlSimple.xml_in(response.body)['messages'][0]['msg'].collect{ |m| “Invalid markup: line #{m['line']}: #{CGI.unescapeHTML(m['content'])}” }.join(“\n”)
end
if markup_is_valid
return true
else
return false
end
end

def description
“be valid xhtml”
end

def failure_message
” expected xhtml to be valid, but validation produced these errors:\n #{@message}”
end

def negative_failure_message
” expected to not be valid, but was (missing validation?)”
end

private
def validity_checks_disabled?
ENV["NONET"] == ‘true’
end

def text_to_multipart(key,value)
return “Content-Disposition: form-data; name=\”#{CGI::escape(key)}\”\r\n\r\n#{value}\r\n”
end

def file_to_multipart(key,filename,mime_type,content)
return “Content-Disposition: form-data; name=\”#{CGI::escape(key)}\”; filename=\”#{filename}\”\r\n” +
“Content-Transfer-Encoding: binary\r\nContent-Type: #{mime_type}\r\n\r\n#{content}\r\n”
end

def cache_resource(base,resource,extension,fn)
resource_md5 = MD5.md5(resource).to_s
file_md5 = nil

output_dir = “#{RAILS_ROOT}/tmp/#{base}”
base_filename = File.join(output_dir, fn)
filename = base_filename + extension

parent_dir = File.dirname(filename)
File.makedirs(parent_dir) unless File.exists?(parent_dir)

File.open(filename, ‘r’) do |f|
file_md5 = MD5.md5(f.read(f.stat.size)).to_s
end if File.exists?(filename)

if file_md5 != resource_md5
Dir["#{base_filename}[^.]*”] .each {|f| File.delete(f)}
File.open(filename, ‘w+’) do |f| f.write(resource); end
end
base_filename
end

def http
if Module.constants.include?(“ApplicationConfig”) && ApplicationConfig.respond_to?(:proxy_config)
Net::HTTP::Proxy(ApplicationConfig.proxy_config['host'], ApplicationConfig.proxy_config['port'])
else
Net::HTTP
end
end

end

def be_valid_xhtml
BeValidXhtml.new
end

[/ruby]

Debugging Ruby on Rails with the ruby-debug gem

Remember the breakpointer? It was based on a bug in ruby that was closed in ruby 1.8.5 – when the bug was closed, the handy dandy breakpointer functionality went away. That’s a drag, but there’s an even better way available.

Ok. Lets get going – the first thing we do is install the ruby-debug gem. This uses a ruby C extension, so you’ll need to have your compiler tool chain installed for your platform. (You’re on your own for that one boyo, I ain’t gonna help you).

[shell]
gem install ruby-debug
Need to update 40 gems from http://gems.rubyforge.org
………………………………….
complete
Install required dependency ruby-debug-base? [Yn] y
Select which gem to install for your platform (i686-darwin)
1. ruby-debug-base 0.9.3 (ruby)
2. ruby-debug-base 0.9.3 (mswin32)
3. Skip this gem
4. Cancel installation
> 1
Building native extensions. This could take a while…
Successfully installed ruby-debug-0.9.3
Successfully installed ruby-debug-base-0.9.3
Installing ri documentation for ruby-debug-0.9.3…
Installing ri documentation for ruby-debug-base-0.9.3…
Installing RDoc documentation for ruby-debug-0.9.3…
Installing RDoc documentation for ruby-debug-base-0.9.3…
[/shell]

Ok. Now we have the ruby-debug gem installed we’re gonna rawk it an’ sock it into our rails app. Go and stand in your rails directory and add the following to config/environments/development.rb (and config/test.rb if you want to use it during testing too).

[ruby]
require ‘ruby-debug’
[/ruby]

Now it gets even easier. Just slam a “debugger” statement wherever you think you need one. Let’s experiment with this broken code below:

[ruby]
class PeopleController < ApplicationController

def create
@person = Person.new(params[:personage])
if @person.save
flash[:notice] = 'Person was successfully created.'
redirect_to :action => ‘list’
else
render :action => ‘new’
end
end

end
[/ruby]

Obviously we should be looking for params[:person] here instead of params[:personage]. Instead of saving the parameters that we expect here, we’re going to be saving empty fields, since we’re not passing in any personage hash.

So we’ll just slam in a “debugger” statement:

[ruby]
class PeopleController < ApplicationController

def create
@person = Person.new(params[:personage])
debugger
if @person.save
flash[:notice] = 'Person was successfully created.'
redirect_to :action => ‘list’
else
render :action => ‘new’
end
end

end
[/ruby]

Now when we try and create a new patient we’re going to see that the request doesn’t return in the browser. (I think that’s how the old breakpointer client worked too – I can’t remember). But if we go to the window where our server is running we’ll see something like this:

[ruby]
Completed in 0.02870 (34 reqs/sec) | Rendering: 0.01762 (61%) | DB: 0.00000 (0%) | 200 OK [http://127.0.0.1/people/new]
./script/../config/../app/controllers/people_controller.rb:26 if @person.save
(rdb:1)
[/ruby]

That debugger is open right in the context of where we put the debugger statement. To see what we can do with the debugger we just have to say ‘help’ at the prompt

[ruby]
(rdb:1) help
ruby-debug help v0.9.3
Type ‘help ‘ for help on a specific command

Available commands:
backtrace break catch cont delete display down eval exit finish frame
help irb list method next p pp quit reload restart save script set
step thread tmate trace undisplay up var where

(rdb:1)
[/ruby]

Try them out! Some neeto ones are “tmate” which opens the context in textmate, “irb” which opens an irb session just right in the context just like the old breakpointer client. The other ones mostly work like you’d expect too. You can find a more detailed explanation of the methods here.

rails presenters, REST and fat models

So. I’ve been drinking the kool-aid for the last year or so and trying to build REST-ful type controllers. That’s good I think. Not perfect, but good. As well, I’ve been following Jamis’ advice and doing skinny controllers and fat models. That’s also good. Very good.

The main consequence of all this is that my controllers are now very focused on marshalling and un-marshalling data for the http request and response. Again, I think that’s good. It separates very well the concerns of the domain objects in model and the mechanics of responding to particular types of requests and choosing the type of view to respond with.

Now the problem with this is that I’ve been ending up pushing shitloads of methods that look like this into my models:

[ruby]
# Delegate to BpReading
def number_of_bp_readings_in(period)
bp_readings.number_of_readings_in period
end
[/ruby]

This code means that I can do:
[ruby]
patient.number_of_readings_in last_two_weeks
[/ruby]
in my views.It’s not terrible, and its a minor nod to law of Demeter.

It seems not great to me that I’m pushing all these extra methods into my model just to service my views though. They’re really not methods that belong to my domain model per se. They’re rather methods that only exist to make my views more convenient and to keep long method chains like
[ruby]
patient.bp_readings.number_of_readings_in last_two_weeks
[/ruby]
out of my views.

I used to do this kind of manipulation in my controllers by doing something like:
[ruby]
@number_of_readings = patient.bp_readings.number_of_readings_in last_two_weeks
[/ruby]

I’m so happy with my newfound purity of my controllers, only marshalling and unmarshalling responses to requests, that I don’t want to go back to that. At the same time, I’m not totally happy pushing all those methods into my model. I’m beginning to think that what’s needed is an extra layer of abstraction like a presenter is the way to go, but so far I haven’t seen a presenter package that I find really compelling.

Anyone feel like convincing me of the error of my ways?

arrgh. goddamn you mysql, you asshole.

Fuck. It doesn’t follow the principle of least surprise, it follows the principle of “ARRRGHGH look at me, I’m jumping out of a dark alleyway with a rusty knife”. Goddamn stupid jerk.

Ok. In this post I was complaining about RoR typecasting wrongly. Mea cuppa. It wasn’t RoR, it was mysql which was behaving badly. Rails migrations default :integer type goes to 11, man. I naively assumed that if I amped that up to :limit => 20 I would get a wider integer in mysql. NO! That would be too goddamn sensible. Instead it just pads a bog standard integer out to that width when it returns it.

The. Suck.

Fortunately there’s a plugin. Like always. There’s also been a ticket open on this issue for a while.

There’s actually a drag about having open classes and being able to make plugins so easily. I think it reduces some of the pressure on the core guys to incorporate patches, which is probably a good thing from their point of view, but not neccessarily for the rest of us. I mean I get the impression from the comments on the patch that core thinks this is a worthwhile patch (and not something that should just be closed wontfix), but its been languishing for over a year now. Just sayin is all.

ruby script/server and ‘too many open files’ error

So I’ve been running into problems with my local dev machine lately – every so often it’s been crashing with an exception message like ‘too many open files’. I’ve been content to ignore it until it became a bigger problem, which it did today, when I started full stack integration testing.

After waving a few dead chickens around google, I found this.

I added the small patch to activerecord and it seems to be working properly now.

This issue also seems to have been responsible for < RangeError: 0xfooo is a recycled object> errors that Ruby was emitting occasionally