And now for a little exercise in armchair software architecture -- the most despicable coder's pastime. Dear non-coding readers: despite its name, this blog is still mostly not about programming. Just skip this post or something. Dear coders, many of you will probably disagree with me. I am not a very good or accomplished coder myself, and you probably should not be taking your advice from me. But then again, I could be right, so keep your mind open.

You might have been aware of the very popular, but uptime-challenged social networking tool called Twitter. They have one of the best problems to have: too many very active users. The site is so popular that it constantly goes down and displays and "over capacity" screen that the users have nicknamed The Fail Whale.

Rapidly writing and displaying short chunks of text with high concurrency on the web is not one of them unsolvable problems in programming. It's not easy, but with right people and tools Twitter could be rewritten inside a month. Twitter founders should do some soul searching. Meanwhile the critical mass has already been reached, the niche for bloggers who want to SMS instead of blogging is big, and even horrible uptime can't this service. I use it myself.

There is a lot of speculation in the blogocube about whether the reason behind the Fail Whale is the wrong choice of technology -- the highly hyped and sexy Ruby on Rails and if it can "scale". Or is it just simple incompetence?

To me Ruby on Rails falls into a class of technologies that are affected by what I call "the VRML syndrome." Basically, if I wait long enough the hype will go away, the recruiters will stop posting job listings requiring 4 years of experience in a 4 month old technology, books as fat as my two fists will stop being published, and I will not have to learn it.

What's the problem with Ruby on Rails? Well, it's the same problem that slightly affects the content management system that I am currently working with (Drupal), and is the reason why I completely gave up using Microsoft web technologies which are saturated with this shit. See, software craptitechts all of a sudden decided that writing CRUD applications is too difficult for regular developers, and complicated GUI tools and frameworks need to be created to help the poor things. CRUD stands for "Create, Read, Update, Delete" and is just a funny way to say "a browser-based application chock-full'o forms".

The default way to build these is to rather simple. You hand-code the html forms, then you write functions or classes to deal with the form input -- validators and SQL queries for creating, updating and deleting. Then you write some code that will query the database and display the saved data in various ways: as pages, xml feeds, etc. None of this is difficult or non-trivial. Bad coders don't do a good job of validation and input sanitizing resulting in the Little Bobby Tables-type situation, but these things are not very hard to learn and there are great libraries for this.

Ruby on Rails makes it very easy to create CRUD apps without hand-coding forms or writing SQL. RoR goes to great lengths to abstract out SQL, not trusting the developers to do it right. SQL is more functional than procedural, and thus a difficult thing for many programmers to grasp, but it's not that hard. Really. SQL is located far enough levels from the machine that abstracting it out becomes a horrible thing due to the Law of Leaky Abstractions. Even when you have full control of SQL queries optimizing them is sometimes hard. When they are hidden by another layer it becomes next to impossible.

In short, RoR makes something that is easy (building CRUD apps) trivial, and something that's hard - optimizing the database layer next to impossible.

In Drupal there are two modules, CCK and Views that allow you to create CRUD entirely through web interfaces. This is a feature that exist in just about every major CMS, it's just that in Drupal it's a little buggier and overcomplicated than necessary. These are fine for small websites and are really useful to amateurs. The problem arises when these are used for high traffic websites.

I think that a lot of people will agree with me that writing HTML and SQL queries using GUI tools is amateur hour. You just can't make a good website with Microsoft Front Page. You can't, you can't, you can't. But in Drupalland it's all of a sudden fine to use Views to build queries for high traffic sites. Well, it's not. Dealing with Views and Views Fast Search has been an ongoing nightmare for me. Hell is not even other people's code in this case. It's other people's Views.

RoR, Views, CCK are one level of abstraction higher than you want to be when building a high performance application. The only way the can be an "Enterprise" tool if your enterprise is a) run by a morons that require 100 changes a day AND b) has very few users. In short, if it's an app for the HR department of a company with 12 employees - knock yourself out. If you are building a public website for millions of people - forget about it.

Your, Deadprogrammer.

P.S. Yes, I know, you can abstract just about everything and reduce your software application to a single button labled "GENERATE MONEY". You have to be a very smart LISP developer for that.

P.P.S. Here are some good books, just for good measure.

Close to the Machine: Technophilia and Its Discontents

Joel on Software: And on Diverse and Occasionally Related Matters That Will Prove of Interest to Software Developers, Designers, and Managers, and to Those Who, Whether by Good Fortune or Ill Luck, Work with Them in Some Capacity

The Best Software Writing I: Selected and Introduced by Joel Spolsky

More Joel on Software: Further Thoughts on Diverse and Occasionally Related Matters That Will Prove of Interest to Software Developers, Designers, and ... Luck, Work with Them in Some Capacity (Pro)

Hackers and Painters: Big Ideas from the Computer Age

My last two weeks were very intense. 12 hour days, a 30 hour coding session, a week in Boston with various very cool nerds, a ride in an Aston Martin, a tour of the Infinite Corridor. Will post more when I'll have more time.

Joel Spolsky wrote about an interesting limitation that he encountered when choosing a cover design for his book:

"And although they would not put a doggie on the cover of my book as I requested, because a certain other book publisher threatens to sue his competitors when they put anything animal like within 90 feet of their covers, their graphic designer worked overtime to create underground cover art called "User Interface Design for Doggies" complete with three golden retrievers, which they framed and sent to me. All in all a classy operation and highly recommended if you're thinking of writing a computer book."

The publisher is, of course, O'Reilly Media. The are famous for publishing computer programming books with engravings of animals on the covers. Like any programmer's, my bookshelf holds a pretty sizable zoo of these critters. The question that always comes to mind is what guides the selection - how the publisher decides which animal to match with which technology. Here's what O'Reilly editors say:

"Our look is the result of reader comments, our own experimentation, and feedback from distribution channels. Distinctive covers complement our distinctive approach to technical topics, breathing personality and life into potentially dry subjects."

Well, with some books it's clear - a spider for a webmaster book and a python for a Python book, for instance. But why does the Perl book have a camel? Wouldn't an oyster make a lot more sense?

Update: Joe Grossberg commented that camel was chosen "because Perl uses camelCase for capitalizing variables". John (website or last name not included) said that "camel was picked for Perl because of the quip that it was a 'horse designed by a committee'". I like John's version much better :)

Joe also started a Wikipedia article on the subject.

One of the more understandable conventions is using Javan animals on Java-related books. For instance, the Java book has a Javan tiger and the JavaScript book has a Javan rhino.

O'Reilly colophons rarely give too much insight into why that particular animal was chosen for the cover, but sometimes you might read between the lines:

"Like the crustaceans after which they are named, crab spiders walk sideways or backwards. They feed on bees and other pollenizing insects, often laying in wait for them by hiding on flowers."

"Both male and female pythons retain vestiges of their ancestral hind legs. The male python uses these vestiges, or spurs, when courting a female"

"Folklore has long held that the horn of the rhinoceros possesses magical and aphrodisiacal powers, and that humans who gain possession of the horns will gain those powers, also."

"Tigers are the largest of all cats, weighing up to 660 pounds and with a body length of up to 9 feet. They are solitary animals, and, unlike lions, hunt alone.
...
There are some tigers, however, who have developed a taste for human flesh. This is a particularly bad problem in an area of India and Bangladesh called the Sunderbans."

The ironic thing is, Javan tigers are extinct and there are only about 100 Javan rhinos remaining. Is that a dig at these languages?

One of the most ironic, yet clearly unintentional choices was that of a stingray for the cover of ASP.NET in a Nutshell.