Search
Popular Articles
CRUD Ain't Hard
Fri, 06/20/2008 - 13:02 — deadprogrammer
And now for a little exercise in armchair software architecture -- the most despicable coder's pastime. Dear non-coding readers: despite its name, this blog is still mostly not about programming. Just skip this post or something. Dear coders, many of you will probably disagree with me. I am not a very good or accomplished coder myself, and you probably should not be taking your advice from me. But then again, I could be right, so keep your mind open. You might have been aware of the very popular, but uptime-challenged social networking tool called Twitter. They have one of the best problems to have: too many very active users. The site is so popular that it constantly goes down and displays and "over capacity" screen that the users have nicknamed The Fail Whale.
Rapidly writing and displaying short chunks of text with high concurrency on the web is not one of them unsolvable problems in programming. It's not easy, but with right people and tools Twitter could be rewritten inside a month. Twitter founders should do some soul searching. Meanwhile the critical mass has already been reached, the niche for bloggers who want to SMS instead of blogging is big, and even horrible uptime can't this service. I use it myself. There is a lot of speculation in the blogocube about whether the reason behind the Fail Whale is the wrong choice of technology -- the highly hyped and sexy Ruby on Rails and if it can "scale". Or is it just simple incompetence? To me Ruby on Rails falls into a class of technologies that are affected by what I call "the VRML syndrome." Basically, if I wait long enough the hype will go away, the recruiters will stop posting job listings requiring 4 years of experience in a 4 month old technology, books as fat as my two fists will stop being published, and I will not have to learn it. What's the problem with Ruby on Rails? Well, it's the same problem that slightly affects the content management system that I am currently working with (Drupal), and is the reason why I completely gave up using Microsoft web technologies which are saturated with this shit. See, software craptitechts all of a sudden decided that writing CRUD applications is too difficult for regular developers, and complicated GUI tools and frameworks need to be created to help the poor things. CRUD stands for "Create, Read, Update, Delete" and is just a funny way to say "a browser-based application chock-full'o forms". The default way to build these is to rather simple. You hand-code the html forms, then you write functions or classes to deal with the form input -- validators and SQL queries for creating, updating and deleting. Then you write some code that will query the database and display the saved data in various ways: as pages, xml feeds, etc. None of this is difficult or non-trivial. Bad coders don't do a good job of validation and input sanitizing resulting in the Little Bobby Tables-type situation, but these things are not very hard to learn and there are great libraries for this. Ruby on Rails makes it very easy to create CRUD apps without hand-coding forms or writing SQL. RoR goes to great lengths to abstract out SQL, not trusting the developers to do it right. SQL is more functional than procedural, and thus a difficult thing for many programmers to grasp, but it's not that hard. Really. SQL is located far enough levels from the machine that abstracting it out becomes a horrible thing due to the Law of Leaky Abstractions. Even when you have full control of SQL queries optimizing them is sometimes hard. When they are hidden by another layer it becomes next to impossible. In short, RoR makes something that is easy (building CRUD apps) trivial, and something that's hard - optimizing the database layer next to impossible. In Drupal there are two modules, CCK and Views that allow you to create CRUD entirely through web interfaces. This is a feature that exist in just about every major CMS, it's just that in Drupal it's a little buggier and overcomplicated than necessary. These are fine for small websites and are really useful to amateurs. The problem arises when these are used for high traffic websites. I think that a lot of people will agree with me that writing HTML and SQL queries using GUI tools is amateur hour. You just can't make a good website with Microsoft Front Page. You can't, you can't, you can't. But in Drupalland it's all of a sudden fine to use Views to build queries for high traffic sites. Well, it's not. Dealing with Views and Views Fast Search has been an ongoing nightmare for me. Hell is not even other people's code in this case. It's other people's Views. RoR, Views, CCK are one level of abstraction higher than you want to be when building a high performance application. The only way the can be an "Enterprise" tool if your enterprise is a) run by a morons that require 100 changes a day AND b) has very few users. In short, if it's an app for the HR department of a company with 12 employees - knock yourself out. If you are building a public website for millions of people - forget about it. Your, Deadprogrammer. P.S. Yes, I know, you can abstract just about everything and reduce your software application to a single button labled "GENERATE MONEY". You have to be a very smart LISP developer for that. Some Items You Might Enjoy:
Running Themes100 Views of the Empire State Building
Architecture
Books
Bread and Circuses
Brown Thumb Adventures
building tops
Coffee and Caffeine
Design
Dreams
Drupal
Family
fashion
Food
Gastronomic Adventures
Happy New Year
Information Carriers
Japan
Japan and Japanophilia
Logos
Looks Like
Materialism
Metablogging
microsoft
music
Natalie E
NYC Subway
NYPD
Odessa Trip
Photography
Pigeons
Programming
rants
Real Estate
Rupert Murdoch
Tolstoys
TT
TV
Unfiled
Unobtainium
WML
Working
|
Subscribe in a reader
Monthly Archives
- August 2008 (4)
- July 2008 (8)
- June 2008 (4)
- May 2008 (10)
- April 2008 (3)
- March 2008 (8)
- February 2008 (2)
- January 2008 (7)
- December 2007 (1)
- November 2007 (10)
- October 2007 (2)
- July 2007 (4)
- June 2007 (3)
- May 2007 (2)
- April 2007 (8)
- March 2007 (8)
- January 2007 (4)
- December 2006 (4)
- November 2006 (5)
- October 2006 (9)
- September 2006 (9)
- August 2006 (16)
- July 2006 (2)
- June 2006 (7)
- May 2006 (5)
- April 2006 (5)
- March 2006 (15)
- February 2006 (11)
- January 2006 (7)
- December 2005 (7)
- November 2005 (5)
- October 2005 (11)
- September 2005 (8)
- August 2005 (17)
- July 2005 (27)
- June 2005 (20)
- May 2005 (28)
- April 2005 (17)
- March 2005 (8)
- February 2005 (16)
- January 2005 (11)
- December 2004 (9)
- November 2004 (9)
- October 2004 (17)
- September 2004 (14)
- August 2004 (32)
- July 2004 (24)
- June 2004 (40)
- May 2004 (22)
- April 2004 (17)
- March 2004 (21)
- February 2004 (6)
- January 2004 (26)
- December 2003 (14)
- November 2003 (10)
- October 2003 (24)
- September 2003 (23)
- August 2003 (25)
- July 2003 (24)
- June 2003 (14)
- May 2003 (16)
- April 2003 (18)
- March 2003 (30)
- February 2003 (21)
- January 2003 (46)
- December 2002 (26)
- November 2002 (35)
- October 2002 (26)
- September 2002 (33)
- August 2002 (43)
- July 2002 (26)
- June 2002 (37)
- May 2002 (22)
Comments
Hey Michael,
Great article. The same discussion has been going on for years and years, not just about CRUD systems, but also other design patterns as well. One example is object oriented vs functional. I don't think the question is about performance, the question is about design. The benefits you gain from using a system severely outweigh what you'd get out of not using it.
... And that's just the beginning..... Yes, you might get better performance if you were to just make it all yourself, but is that something that you really want to do? You'd miss out on all of these benefits. Thoughts?
Although Wikipedia tells me that the first popular patterns book was published in '94, on my memory patterns became a hot topic somewhere in early oughts. Before then people managed to write perfect MVC code without thinking about M, V or C. CRUD is not design pattern, by the way, it's just a type of an application.
I'm not even going to get into the whole object oriented vs functional vs procedural quagmire. I have what you might call some weak opinions on this matter. I'm trying as hard as I can to learn functional programming (and suspect that I'm not smart enough to ever get good at it), and as far as oo and procedural go, I'm ok with both.
As far as the rest goes, here's a quote that I mentioned in a previous post:
"Look, Levin. Programming starts out like it's going to be architecture--all black lines on white paper, theoretical and abstract and spatial and up-in-the-head. Then, right around the time you have to get something fucking working, it has this nasty tendency to turn into plumbing.
"No, no. Lemme think," Harry interrupted himself. "It's more like you're hired as a plumber to work in an old house full of ancient, leaky pipes laid out by some long-gone plumbers who were even weirder than you are. Most of the time you spend scratching your head and thinking: Why the fuck did they do that?"
"Why the fuck did they?" Ethan said.
Which appeared to amuse Harry to no end. "Oh, you know," he went on, laughing hoarsely, "they didn't understand whatever the fuck had come before them, and they just had to get something working in some ridiculous time. Hey, software is just a shitload of pipe fitting you do to get something the hell working. Me," he said, holding up his chewed, nail-torn hands as if for evidence, "I'm just a plumber." "
Programming sometimes starts with architecture and ends with plumbing. Deep abstractions like all kinds of GUI query builders almost always get you in trouble.
Let me answer your nits:
1. Separating business logic from presentation layer is a noble ideal. But in practice -- you know what -- it does not always make maintenance easier. Hunting for a Drupal view is not any easier than hunting for a piece of code. You can at least search the codebase.
2. I'm not arguing against modular code. I'm arguing against reusing other people's bad code. See this article about how and why the Microsoft Excel team wrote its own C compiler.
3. Remember what happens when CCK normalizes its tables? Right? This is a double edged sword. One place to fix things, one place to break things. Fixing one thing sometimes breaks another.
4. Views and CCK do not do something that is difficult. It's just a form creator and a query builder. Queries need to be built by hand and forms need to be coded in modules.
The moral of the story is that you want to use beautiful and well maintained code like the Drupal core, or like Image Cache, let's say. But Views and CCK are amateur hour.
Interesting post - I'll have to go back to the beginning and re-read. But the last paragraph jumped at me.
the only way the can be an "Enterprise" tool if your enterprise is a) run by a morons that require 100 changes a day AND b) has very few users. In short, if it's an app for the HR department of a company with 12 employees - knock yourself out. If you are building a public website for millions of people - forget about it.
It's nitpicking and besides the main point of your post, but I think your definition if 'enterprise tools' is a bit restrictive.
In my experience, an Enterprise tool (or application) is any app that must run 24x7 or the business falls down. I'm the system admin for two of those at work - our EDI application and the bill of material tool.
The latter is instructive. It must be available 24x7, to users in Malaysia, Nampa, Wisconsin and Scotland [1]. Yet it never has more than forty users logged in at a time. The amount of traffic to/from it's web servers is pitiful. But it absolutely must be available all the time: if team leaders can't get at the bits provided by that app we can't make product.
[1] Global conference calls at my company are fun - it's dueling accent time.
Lullabots have thoughts on RAD big traffic sites:
http://www.lullabot.com/blog/dooce-20k-comments-24hrs
A) I read Lullafeed just like everybody else in the industry. B) How's this even relevant to my article?
Drupal core and certain modules are rock solid and beat the pants off of other CMSes cough-Wordpress-cough hands down, especially when you are building a blog. Even the shitty Views and CCK combo is not a huge problem when your site is that straightforward. I still would not use them for something that might get Dooce-level traffic.
I say - Dooce the unnecessary abstraction and you'll be much happier. It's for amateurs.
Post new comment