Scale up or Scale out?
"Normal" applications on which average engineers like you and me get to work on, now have a great variety of architectural choices. Massive Scale-out software architectures have trickled down from Google, Yahoo, Amazon, LinkedIn, Facebook ..<seemingly endless list>... to easily accessible projects under Apache and the like. Hardware has also seen its share of impressive growth. What with super-duper multi-Core CPUs to a million Ops/sec Solid State Drives; Scale-up does not seem like a bad idea at all.
NoSQL is very tempting to us engineers. Why not drink the NoSQL Kool-aid eh? However, there have been many times when I've found myself wondering with a NoSQL hangover and a boat load of challenges that they don't tell you about when you use these kinds of systems.... Do we really need to Scale-up when Scaling-out with good old Disks and CPUs is much much [sic] simpler and cheaper? After all, Intel's SSDs can do 1m ops/sec and there's already 128 hardware Threaded CPUs in the pipeline. In another year or so there will very likely be 100+ Cores. Azul Systems and Sun's Niagara chips have at least in my living memory always had such many-Core systems.
I'm not the only one with these doubts. Scaling-out for average apps just seems like an excuse for bad design. Look at all these bright people playing around with SSDs and big RAM configurations:
- Freebase data that just fits in 32 Gigs of RAM
- Future of RDBMS is RAM Clouds & SSD
- Launching Xbox, Part 2 - SSD Streaming
Until next time..