Musings on RailsConf 2014

This is a bit late, but I wanted to ramble for a bit on RailsConf 2014. I have a few main points that can be summarized as:

  1. TDD isn’t dead, but we’ve opened up a conversation about how useful it is.
  2. RailsConf is less and less about Rails each year and that’s ok.
  3. The Rails/Ruby community is awesome!

TDD isn’t dead

Some people say that DHH declared TDD dead. I disagree. What he did was to open a discussion about testing that we’ve been lacking in the community. The TDD “fad diet” has always troubled me. When TDD proponents quiz a room and ask people how many are using TDD lots of hands go up. Then when they ask if you write your tests first, the hands go down. It makes a point, but it also is a form of shaming. I believe in testing, but I have yet to see a test suite for an app or a gem that I thought was an outstanding example of great code. If it’s out there and I’m just missing it, please let me know. The tools we have now seem to be just barely scratching the surface of what we can do to teach effective testing. I’m thirsty for more testing tips and frameworks that take the confusion out of testing and give us a better way to talk about it and DHH opened the way for that conversation.

RailsConf is less and less about Rails

I’ve been to 3 RailsConfs now. Each one was well worth it. Austin was great as it opened my eyes to the Ruby community and what it has to offer. I learned it’s not possible to ruby without twitter, and that lightning talks rock. There were talks on nearly all aspects of a Rails application, and people even commented on how many were not really about Rails. Even so, talks on coffeescript, backbone, sass and others still seemed dominated the standard Rails talks.

RailsConf 2013 in Portland had Rails as well. But this time it seemed like something was shifting. There were testing tricks, talks on Gotalks about what we can learn from python, and even talks on talks.

And now in 2014, we had a whole track on learning from designers. We also had a track on training and team building and one on playing nicely. There was a talk about saving the world by saving the bees using sinatra. One devoted to databases of all kinds (but just use Postrgre), and another on the tricks our minds play on us.

Overall, I think that signifies that our community is growing, and we have a place for everyone. Which leads to the next section… because…

The Community is Awesome!

Rails is a very open community. We welcome everyone. Yes, there’s always talk about how we get more women, GLBT, and minorities involved, but here’s the thing… we talk about how we can get more women, GLBT and minorities involved! We are not nearly where we need to be yet, but we’re working on it. The more Katrina Owen, Sandi Metz, and Coraline Ehmke get to speak and have a voice, the more women like them will join our community. The diversity of ideas, gender and race can only help us to grow and find new and interesting ways to solve problems.

Community Calendar API

At work, we had a hackathon and developed some sample apps that make use of our new HoopaNow API. HooplaNow is a community calendar in Iowa’s Creative Corridor.

These are still works in progress, but there is a Sinatra app called simply hooplanow_api_example for you Rubyists out there, and a wordpress plugin for those of you who have a wordpress blog, or want to take a peek at some PHP code that uses the API.

Neither is complete, but I would welcome pull requests from anyone who wants to help out.

ActiveRecord: self.field vs field

TL;DR version:
“self.field” is only necessary when you are assigning a new value to a field.

It always confused me why it seemed like sometimes using the field name from the database failed to work in model methods that I wrote. When the method was simple, things would work fine, but then something would happen and it suddenly stopped working. I never quite put together what was going on until now. I would start out with something like this:

# Very simple, overused, contrived example
def full_name
  "#{prefix} #{first_name} #{last_name}"
end

It works beautifully and looks clean. But then, I’d need to add some defaults, like so.

def full_name
  prefix = "Mr." if prefix.blank?
  first_name = "First" if first_name.blank?
  last_name = "Last" if last_name.blank?
  "#{prefix} #{first_name} #{last_name}"
end

At first glance, it looks right, but it’s not. It causes full_name to produce “Mr. First Last”, every time you run it. WTF? Here’s where I would usually overreach and get it horribly wrong. I know that the fields are available as “self.field”, so I make one more change.

def full_name
  self.prefix = "Mr." if self.prefix.blank?
  self.first_name = "First" if self.first_name.blank?
  self.last_name = "Last" if self.last_name.blank?
  "#{self.prefix} #{self.first_name} #{self.last_name}"
end

So, one WTF leads right to another. This code is horrible. It’s ugly and it feels like there should be a better answer. In this example, it’s not so bad, but what usually ends up happening is this type of code gets written in a method that is already fairly ugly, and because of a simple misunderstanding, it goes directly from bad to worse.

Finally having some time, and just being fed up enough, I did some searching and found this. It ends up being fairly easy to understand once you think about it. In ActiveRecord models, the fields are methods. The way Ruby works, writing “field = ” creates a local variable instead of calling the “field=” method on self. It is the very act of assigning the field with a default that causes the issue. So, you can scale it back a little and rewrite the code like this. You only need “self.field” when assigning a new value.

def full_name
  self.prefix = "Mr." if prefix.blank?
  self.first_name = "First" if first_name.blank?
  self.last_name = "Last" if last_name.blank?
  "#{prefix} #{first_name} #{last_name}"
end

Much better.

How to load test your web site or application with siege

Trebuchet
Time to storm the castle!

Your mileage may vary, but this worked well for me. I wanted to test one of our sites under load. Our sites generally get a lot of traffic, so when we move from staging to production we have to keep in mind that we’re about to go from 1 or 2 developers hitting the pages to thousands of people.

I’ve previously used Apache Benchmark (ab) to load test, but it’s fairly limited. It can only test one page. I did a bit of digging and found siege. It was perfect for what I wanted to do. I wanted to submit my site to a heavy load, over a long period of time, and see what happened. I also wanted semi-realistic traffic, and with a few well-typed commands, I was able to create a file that siege can read that contained *exactly* the traffic from our site.

I used this to read the last 1000 hits from our apache logs:

tail -1000 /var/log/apache2/access.log | awk '{print "http://mysite.com" $8}' > /tmp/siege-urls.txt

It’s pretty simple, but I’ll break it down. The following gives me the last 1000 lines from the log. Your log might be in a different location, or named something else. If you want fewer lines, or more lines, change the 1000 to something else.

tail -1000 /var/log/apache2/access.log

Then, pipe the output of the above to awk and print out the url. In our apache logs, the url was in the eighth position. I also appended the site’s domain to the output, since the apache log does not contain that information (at least not in the format I wanted).

| awk '{print "http://mysite.com" $8}'

Lastly, save it to a file for later use.

> /tmp/siege-urls.txt

Next, I used this file to place the site under load. In the example below, I used -i for “internet” mode, where siege randomly reads a line from the file and requests it from the server. I also used only 4 concurrent users or worker threads. If you ramp up the concurrency, you can really put a lot of strain on the server.

siege -i -c 4 -f /tmp/siege-urls.txt

When the siege is underway, you’ll get output that looks like this:

HTTP/1.1 200   0.53 secs:    6926 bytes ==> /some/url?some=params
HTTP/1.1 200   0.54 secs:    7132 bytes ==> /some/other/url?other=params
HTTP/1.1 500   0.13 secs:     521 bytes ==> /some/url?some=params
HTTP/1.1 200   0.64 secs:    7133 bytes ==> /some/other/url?other=params
HTTP/1.1 500   0.13 secs:     521 bytes ==> /some/other/url?other=params
HTTP/1.1 404   0.09 secs:     431 bytes ==> /some/url?some=params

I paid close attention to the 404s and the 500 errors. They indicated that I was getting requests that were erroring out for some reason. Sometimes, those errors are simply bots that grab a hold of old urls and continue to request them. Sometimes, they are cause for concern.

Hitting Control-C ends the siege and you then get output like below.

Lifting the server siege...      done.
Transactions:		         125 hits
Availability:		       88.03 %
Elapsed time:		       29.18 secs
Data transferred:	        1.18 MB
Response time:		        0.41 secs
Transaction rate:	        4.28 trans/sec
Throughput:		        0.04 MB/sec
Concurrency:		        1.77
Successful transactions:         101
Failed transactions:	          17
Longest transaction:	        3.13
Shortest transaction:	        0.08

I also found it useful to look at top and a few other tools on the server while the siege was underway. In our case, I was interested in passenger’s memory consumption, so I used passenger-memory-status and passenger-status.

Photo Credit: Martin AddisonDemonstrating the Trebuchet

Invalid Authenticity Token, IE and Underscores

I spent a couple hours time which I will never get back trying to figure this one out. My rails application worked perfectly fine in Firefox, but IE was throwing an error. That’s not terribly unusual with IE, but this time something was fishy. The error is all over the google. There are several suggested fixes, none of which was working for me. The main one was adding a p3p header to the response that is supposed to to convince IE to allow cookies to be set for your domain. I finally stumbled upon this page that has a subtle reference to not using an underscore in your domain name. Of course, right there in the middle of the subdomain name I had chosen was an underscore. As it turns out underscores are not allowed in hostnames.  Some browsers are just more forgiving than others.

Collection Select

I always fail to remember how to create a simple drop down for my models with a has_many / belongs_to relationship. It’s one of the most basic things you could possibly do with a form, and yet each time I want to hook one up I find myself opening up a tab and googling for it. Well, no more. From now on I’ll say to myself, “Self, you put that on your blog.” So without further ado… collection_select to the rescue!

Let’s say I have a model Nerd that has_many Computers. When I create or edit a computer, I want to select the nerd it belongs_to.


class Nerd
  has_many :computers
end

class Computer
  belongs_to :nerd
end

Then in my view for the computer I’d have this:

<% form_for :computer do |f| %>
  <p>
    <%= f.label :nerd %>
    <%= collection_select(:computer, :nerd_id, Nerd.all, :id, :name) %>
  </p>
<% end %>

Nokogiri and Css Selectors with Namespaces

I’ve been using the css function from nokogiri to navigate xml and grab elements, instead of using xpath.  I love the fact that it’s an option, since css feels to much more natural.  But, today I ran into something I haven’t had to deal with before.  I was parsing some xml from a service when I noticed the xml had some namespaced items, like so.

<xml>
<blah:item1>some text</blah:item1>
<blah:item2>more stuff</blah:item2>
</xml>

So, I tried this, thinking I could get the value back.

node.css("blah:item1")

But, no luck. However, as it turns out, there is a special selector made just for this. Here is the reference on w3.org. The correct code is below:

node.css("blah|item1")