Community Calendar API

At work, we had a hackathon and developed some sample apps that make use of our new HoopaNow API. HooplaNow is a community calendar in Iowa’s Creative Corridor.

These are still works in progress, but there is a Sinatra app called simply hooplanow_api_example for you Rubyists out there, and a wordpress plugin for those of you who have a wordpress blog, or want to take a peek at some PHP code that uses the API.

Neither is complete, but I would welcome pull requests from anyone who wants to help out.

MTV Follow-up fail

A while back, I wrote about wordpress as a framework, and how I was going to follow up in a few weeks with some thoughts on MTV, which is a project that wants to make wordpress even more framework-like. Well, “a while back” at this point is actually almost an entire year, and I have yet to follow up. Fail. Getting enough time to properly review something is hard, so I’ve instead thrown together a few thoughts that can pretend to be a real review when they grow up.

First, let me say what I like. I love the M in MTV. Using the core models makes working with posts, users, attachments, etc, so much nicer than using the wordpress functions themselves. Because the project was so young when I first tried using it, it could be a bit frustrating to learn that something wasn’t implemented (like avatar images), but over time, that has improved.

I also like being able to create my own models, and MTV keeps storage a detail, so if I don’t want to mix my models into the wp_posts table (like built-in post types have you do), then I can do that.

I understand why it was chosen, and I like that it forces you to use better practices, but I really wish MTV would have stuck with PHP for html templates instead of going with Twig. It makes it nearly impossible to take existing themes and port them to MTV. It’s probably because I come from the Rails community where erb templates are just ruby. I’m sure having a full-featured language at your disposal in the view is exactly what the developers of MTV were trying to avoid, because it leads to misuse, but when you need it and it isn’t there, it’s can be really frustrating.

So, the verdict? Overall I like it. In my opinion, it still needs to mature before it’s ready for a larger audience to adopt it. I also have my own doubts that the wordpress community as a whole would ever adopt MTV or something like it. It would be for its own benefit, but the community seems to embrace its chaos rather than shun it. But that’s probably another blog post all together. It’s also one I probably will never write, as it would just be food for the trolls.

Developing on the fly

If you see any weirdness for the next few days, please excuse it. I’m trying out MTV and doing some theme development without testing. The best way to learn something new is to use it for something, so why not make my own personal blog the guinea pig? If my experiment fails, I’ll just revert back to the theme I was using before.


I got it working. I had to make one modification to the core, as I am on shared hosting, and I have an open_basedir restriction. The code uses a folder in /tmp, and on my system, I have no access to it. I worked around it by placing the temporary files inside the plugin directory. Not ideal, but it worked.

There were a few problems with the example themes as well. The simple theme worked ok, but the twentyeleven theme was broken. I made some modifications and submitted them back through github.

Anyway, not everything works, but I’m calling it close enough for now. There’s no commenting (sorry), and search doesn’t work either. I did make the category and tag pages work. Before my changes, they were giving 404 errors.

WordPress as a framework

Found on

I use wordpress for this blog. I also use it at work. WordPress is a wonderful solution when you need a website or a blog and you need it *right now*. That is reflected in the fact that as of now (Oct 22, 2011), it runs 1/2 of the top 10,000 sites on the internet. That blew me away the first time I heard it.

In the last year or so, especially with the addition of custom post types, it has become quite flexible as well. It is fairly easy to pull together a quick plugin to add a new type of content to wordpress. At work, we’ve used custom post types to create things like candidate profiles and video galleries. However, wordpress starts to falter a bit when what you are trying to add doesn’t bear at least a small resemblance to a post.

I’ve heard it said that WordPress is a CMS that is trying to be a framework, so I’m always interested when I hear of projects that attempt to make it even more framework-like. MTV is one of those projects. Some developers at the News Apps Blog Chicago Tribune have recently released a plugin that makes WordPress act more like a true framework.

My main question at this point is the same one they themselves ask in their documentation. “Why?” I can understand it if you are limited to PHP hosting. I can also see it as a way to introduce framework-based programming to developers who are not familiar with frameworks like Ruby on Rails and Django. MVC frameworks can be intimidating, and this could be a way to introduce those concepts in a more familiar environment.

At any rate, it’s intriguing and is worth a longer look, which I will try to do in the coming weeks.

Hard disk filling up? Check the MySQL binlog.

Last night our monitoring system started throwing errors about a hard drive filling up on one of our servers. Nothing out of the ordinary was going on, but I was doing some maintenance on one of the sites and removing some old spam comments from the system. We use wordpress, and it provides a handy empty spam option, so I figured I’d use that rather than using good old, reliable SQL. That was my first mistake.

When emptying the spam comments, wordpress issues one DELETE statement for each comment. So, not only does it take forever it also had a side effect I was not expecting. It quickly increases the size of the binary logs.

You see, MySQL has a feature called binary logging. It’s an essential feature that serves at least two purposes. One is data recovery. If you have a binary log, you can restore a database from an backup and then tell MySQL to run through the log and bring your db back up to date. The other is replication. The binlogs are used to create a master/slave relationship between two databases. The slave db pulls the binary logs from the master and then can be an exact copy to provide performance benefits and high availability.

Of course, when the alarms started going off, I didn’t suspect that they were in any way related to any of this. Deleting things is supposed to make a database smaller, right? What I eventually discovered was that the binlogs on the server had grown to 46GB. Considering we only had about 2 gigs worth of real data in the system, I had a configuration issue to track down.

It turned out to be simple. Along the way, someone had commented out a line in the my.cnf file.

#expire_logs_days = 14

So, our binlogs had been slowly piling up since that line was commented out… Sometime in early 2010 according to the date on the oldest log. Ouch. Uncommenting it, and restarting MySQL brought the size down considerably. Now we have room to spare again.

Another mystery solved. Now to seek out the next!

WordPress admin hang after upgrade

It’s 8am on a Tuesday morning, and one of the web sites we maintain has just stopped responding.  It’s not completely unexpected. We just upgraded to WordPress 3.1 last night.

The upgrade went smoothly in our development environment, and after patching a few plugins, we were off and running. The upgrade last night on our production system went off without a hitch as well. My main concern was to make sure the frontend pages still worked, and all the sites in our network looked good. At least they did on the public pages.

But now, in the light of day, something isn’t right. As it turns out, the frontend pages *are* all working. But on one of the sites, the admin interface, is acting weird. The editor who is reporting the issue says the admin is locked up, and sure enough, I try to log in and…nothing. The page just loads and loads. I try again with the same result. And then again, but instead of giving up after 30 seconds, I stop the urge to hit the “reload” button over and over and just let it go. Strangely enough, after a few minutes, the admin interface loads as normal. Two minutes to be exact. I navigate to another page in the admin, wait two minutes, and it loads again. And another, and another, and all of them give the exact same response. The site is not slow, it’s just hanging for two minutes exactly, and then loading the page I requested without issue.

Ok, so my first thought is that it’s a plugin issue. I even find some evidence here and there, to back up that claim. I disable all the plugins on the site. Two minutes later, the plugins are disabled, and the admin still behaves in the same manner. Click, two minutes… click, two minutes.

So next, I call in some help from another developer on the team. We try turning on php debugging, but that doesn’t help, because, first, it takes two minutes between page loads to get anything useful out of the admin, and second, there *is no error.* The pages are not broken, just delayed. Next, he puts a dead simple block of debug code into the admin files of WordPress itself.

<?php die(); ?>

When placed at the top of the file, the admin stops right away. At the bottom or in the middle, it takes two minutes, the page renders (mostly), then dies. After a few minutes of moving the line from one place in the file to the next, and following down a few includes, he’s found the culprit. These simple, innocent-looking lines from wp-admin/admin.php in the core WordPress code.

 * On really small MU installs run the upgrader every time,
 * else run it less often to reduce load.
 * @since 2.8.4b
$c = get_blog_count();
if ( $c <= 50 || ( $c > 50 && mt_rand( 0, (int)( $c / 50 ) ) == 1 ) ) {
  require_once( ABSPATH . WPINC . '/http.php' );
  $response = wp_remote_get( admin_url( 'upgrade.php?step=1' ),
    array( 'timeout' => 120, 'httpversion' => '1.1' ) );
  do_action( 'after_mu_upgrade', $response );

The site having problems is brand new and has fewer than 50 posts. And for some reason, the upgrade db script failed to run last night when the rest of the sites were upgraded. That means, each and every time an admin page loads, it tries to run the database upgrade. And it runs the upgrade in a slightly odd way. Instead of calling a function to perform the upgrade, it does a wp_remote_get, which uses http to call another page on the server. We have a quirk in our production architecture that doesn’t allow the servers sitting behind our load balancer call themselves, so instead of just running the upgrade, it hangs. And it hangs for exactly 120 seconds before giving up. The other sites are not really affected, but even if they are hanging, the code above runs randomly, proportionally to the number of posts they have. And luckily for us, they have a *ton* of posts.

So, is there a lesson here? Not really. It just happens to be a good story. I can try to spin this as a lesson about having your development environments match your production environments as much as possible, but you probably know that already. It’s a luxury that is seldom affordable to have an exact mirror between production and development. But when the bug you run into is *caused* by your production server architecture, how can you avoid it if you have those variations? No really, I’d love to know how. I’ll take any suggestions I can get.