Category: Development

Compile this into that

UPDATE: I made a mistake in thinking that RubyMotion compiled Ruby into Objective C. That was incorrect and is corrected in the comments below.

There’s a new trend I find very interesting. It goes like this; take some code in one language, and compile it into another.

Sometimes it’s a completely new language, like CoffeeScript, or Clojure. Other times it’s an existing language like changing Ruby into Objective C with RubyMotion, or Emscripten, which changes C++ into Javascript.

My initial reaction was that programmers are inherently lazy. We want our syntactic sugar. Those are the things that make languages easy to work with. There is nothing bad about this. Making things easy on ourselves is a good thing. We get more done, and the world is better for it.

Also, I lied before when I said this was a new trend. It’s actually been around a long time. No one writes assembly code anymore. C was a better way of expressing code, so we moved to that, and let the compiler do the work of assembling things into machine code. Compile-this-into-that; C into machine code.

But machine code is inflexible. Write once, run once. So, Java comes along and promises to fix all that with bytecode, and a virtual environment. Compile-this-into-that; Java into bytecode. Interpret with a JVM.

There’s also PHP and Ruby. It’s not strictly the same, but the concept is similar. No compile step, but you simply write the code, and then run through an interpreter, much like you do with Java. The interpreters are written in C. So, it’s not compile-this-into-that, but rather, interpret-this-with-that.

So, if this has been going on for a long time, why am I even bother to write about it? Well, the new trend is adding more and more steps, and I find it slightly disturbing. Compile-this-into-that-and-interpret-with-another. Like I said before if it makes us more productive, I’m all for it.

However, I think we may be writing compilers because it’s easier than evolving the systems we use. Examine that statement for a minute. Writing a *compiler* is easier? Why would that be?

My opinion is that it’s because standards are controlled by committee. Compilers are something one person, or a single group of people acting in concert can do. It’s kind of obvious, but I had to write it all down to get my head around it. We’re writing compilers because we can’t get our systems to evolve fast enough to keep up with our needs. We can’t get systems to evolve fast enough because our systems are bogged down by bureaucracy. Here are two lists companies that work on javascript. Ordinary Members, Associate Members. Many of those companies have competing interests, so progress is slow.

But, for now, it’s what we have. It lets us write in our favorite and most comfortable environment and get the benefit of making that code available in many places. And as long as the compilers write better code in the destination language than we do, why not? Anyone have opinions that differ? Let me know in the comments below, or on Twitter or Google Plus.

 

 

 

Developing on the fly

If you see any weirdness for the next few days, please excuse it. I’m trying out MTV and doing some theme development without testing. The best way to learn something new is to use it for something, so why not make my own personal blog the guinea pig? If my experiment fails, I’ll just revert back to the theme I was using before.

UPDATE

I got it working. I had to make one modification to the core, as I am on shared hosting, and I have an open_basedir restriction. The code uses a folder in /tmp, and on my system, I have no access to it. I worked around it by placing the temporary files inside the plugin directory. Not ideal, but it worked.

There were a few problems with the example themes as well. The simple theme worked ok, but the twentyeleven theme was broken. I made some modifications and submitted them back through github.

Anyway, not everything works, but I’m calling it close enough for now. There’s no commenting (sorry), and search doesn’t work either. I did make the category and tag pages work. Before my changes, they were giving 404 errors.

WordPress as a framework

Found on builtwith.com

I use wordpress for this blog. I also use it at work. WordPress is a wonderful solution when you need a website or a blog and you need it *right now*. That is reflected in the fact that as of now (Oct 22, 2011), it runs 1/2 of the top 10,000 sites on the internet. That blew me away the first time I heard it.

In the last year or so, especially with the addition of custom post types, it has become quite flexible as well. It is fairly easy to pull together a quick plugin to add a new type of content to wordpress. At work, we’ve used custom post types to create things like candidate profiles and video galleries. However, wordpress starts to falter a bit when what you are trying to add doesn’t bear at least a small resemblance to a post.

I’ve heard it said that WordPress is a CMS that is trying to be a framework, so I’m always interested when I hear of projects that attempt to make it even more framework-like. MTV is one of those projects. Some developers at the News Apps Blog Chicago Tribune have recently released a plugin that makes WordPress act more like a true framework.

My main question at this point is the same one they themselves ask in their documentation. “Why?” I can understand it if you are limited to PHP hosting. I can also see it as a way to introduce framework-based programming to developers who are not familiar with frameworks like Ruby on Rails and Django. MVC frameworks can be intimidating, and this could be a way to introduce those concepts in a more familiar environment.

At any rate, it’s intriguing and is worth a longer look, which I will try to do in the coming weeks.

WordPress admin hang after upgrade

It’s 8am on a Tuesday morning, and one of the web sites we maintain has just stopped responding.  It’s not completely unexpected. We just upgraded to WordPress 3.1 last night.

The upgrade went smoothly in our development environment, and after patching a few plugins, we were off and running. The upgrade last night on our production system went off without a hitch as well. My main concern was to make sure the frontend pages still worked, and all the sites in our network looked good. At least they did on the public pages.

But now, in the light of day, something isn’t right. As it turns out, the frontend pages *are* all working. But on one of the sites, the admin interface, is acting weird. The editor who is reporting the issue says the admin is locked up, and sure enough, I try to log in and…nothing. The page just loads and loads. I try again with the same result. And then again, but instead of giving up after 30 seconds, I stop the urge to hit the “reload” button over and over and just let it go. Strangely enough, after a few minutes, the admin interface loads as normal. Two minutes to be exact. I navigate to another page in the admin, wait two minutes, and it loads again. And another, and another, and all of them give the exact same response. The site is not slow, it’s just hanging for two minutes exactly, and then loading the page I requested without issue.

Ok, so my first thought is that it’s a plugin issue. I even find some evidence here and there, to back up that claim. I disable all the plugins on the site. Two minutes later, the plugins are disabled, and the admin still behaves in the same manner. Click, two minutes… click, two minutes.

So next, I call in some help from another developer on the team. We try turning on php debugging, but that doesn’t help, because, first, it takes two minutes between page loads to get anything useful out of the admin, and second, there *is no error.* The pages are not broken, just delayed. Next, he puts a dead simple block of debug code into the admin files of WordPress itself.

<?php die(); ?>

When placed at the top of the file, the admin stops right away. At the bottom or in the middle, it takes two minutes, the page renders (mostly), then dies. After a few minutes of moving the line from one place in the file to the next, and following down a few includes, he’s found the culprit. These simple, innocent-looking lines from wp-admin/admin.php in the core WordPress code.

/**
 * On really small MU installs run the upgrader every time,
 * else run it less often to reduce load.
 *
 * @since 2.8.4b
*/
$c = get_blog_count();
if ( $c <= 50 || ( $c > 50 && mt_rand( 0, (int)( $c / 50 ) ) == 1 ) ) {
  require_once( ABSPATH . WPINC . '/http.php' );
  $response = wp_remote_get( admin_url( 'upgrade.php?step=1' ),
    array( 'timeout' => 120, 'httpversion' => '1.1' ) );
  do_action( 'after_mu_upgrade', $response );
  unset($response);
}

The site having problems is brand new and has fewer than 50 posts. And for some reason, the upgrade db script failed to run last night when the rest of the sites were upgraded. That means, each and every time an admin page loads, it tries to run the database upgrade. And it runs the upgrade in a slightly odd way. Instead of calling a function to perform the upgrade, it does a wp_remote_get, which uses http to call another page on the server. We have a quirk in our production architecture that doesn’t allow the servers sitting behind our load balancer call themselves, so instead of just running the upgrade, it hangs. And it hangs for exactly 120 seconds before giving up. The other sites are not really affected, but even if they are hanging, the code above runs randomly, proportionally to the number of posts they have. And luckily for us, they have a *ton* of posts.

So, is there a lesson here? Not really. It just happens to be a good story. I can try to spin this as a lesson about having your development environments match your production environments as much as possible, but you probably know that already. It’s a luxury that is seldom affordable to have an exact mirror between production and development. But when the bug you run into is *caused* by your production server architecture, how can you avoid it if you have those variations? No really, I’d love to know how. I’ll take any suggestions I can get.

One line of code

You’re always there. You’re that line of code. You are the one that isn’t doing what you’re supposed to. You sneak up and bite me when I least expect it. You make me spend hours and hours tracking you down. Sometimes you’re nested so deep within the system that you can only be triggered when 100 things fall into place. But when the stars align… you are the most important thing in the whole system. Sometimes, you are commented out, or missing all together. Somebody forgot to add you, or forgot to check for side effects when removing you. But even though you are stupid, insignificant, barely-worth-mentioning, only-one-tiny-line-of-code; you are the most critical part of what I am working on at this very moment.

I have only one thing to say to you… You cannot run, you cannot hide. You will be found!

Testing APIs

I’ve been doing a lot of web API testing lately.  There are two main flavors of web APIs:  SOAP and REST.

SOAP is verbose, ugly, and generally awful to work with.  In order to work with SOAP APIs first, you need to have a SOAP library for your language of choice.  Then you have to generate a “driver” specific to the API you are working with.  Next, you have to figure out the commands that are available, and finally, you have to parse though the xml that the calls to the API return.

REST, on the other hand, is simple.  Call a url with query parameters like you would a normal web page.  Get a result in xml, json, or whatever other format the API supports.  Parse it.  Done.  The main benefit is that testing is much easier.  Fire up your friendly neighborhood web browser and paste in the url and look at the xml returned.

But not so fast!  That’s all well and good for reading an API, but what about sending data?  For that, you can sometimes use a web browser too, but not always.  Most REST APIs require POSTs to create objects.  For that, you need a bit more.  If you like GUIs, one option is a Firefox plugin called REST Client that I’ve found works great.  For the command-line there’s cURL.  Here are a couple examples of how to use cURL.


# If the API requires authentication
curl -u user:pass -F "file=@filename_to_send" http://www.example.com/api


# If the API requires you to set the content type and send raw data
curl -H "Content-Type:text/xml" -d @filename_to_send