December 2007 - Posts

• Learning from MySpace

  I wanted to draw attention to another valuable read.  This one comes as a bit of a history of the design and architecture over at MySpace. It's quite an interesting read, and if you watched/read my previous two posts, you will see lots of overlap. Inside MySpace.com I really like how the authors broke the system history down into milestones of User Accounts. Posted Dec 30 2007, 05:22 PM by hoffe with 3 comment(s)

• Building Large Systems at Google

  I watched this video very late last night--way cool.  It's a Google TechTalk about building very large systems. A few things stand out to me: How a large cluster of commodity machines (1000+) can function as a single large fault-tolerant hard drive (GFS) How a large cluster of commodity machines can also function as a single large distributed computation grid I've often read people talking about ultra-large scale systems where the big clusters function as a single large computer, but this video brought that point home. It's pretty cool to think they have abstracted a cluster as a hard drive, then abstracted on top of that to create a huge virtual machine for solving problems (executing algorithms). Enjoy! Posted Dec 30 2007, 05:16 PM by hoffe with no comments

• Infinite Scaling and Distributed Transactions

  I posted a link to this paper a few days agon on one of the mailinglists I subscribe to.  I thought I would share with you guys as well: Life Beyond Distributed Transactions: An Apostate's Opinion This is a really good read from Pat Helland on scalability in really large applications.  It's not terribly new, but it's definitely worth a read if you've not seen it before. Personally, I like his descriptions of vertical and horizontal partitioning, but there's quite a bit of meat there to chew on. Pat Helland, btw, has worked for both Microsoft and Amazon.  He was a key part of the team which designed the DTC. Posted Dec 30 2007, 05:09 PM by hoffe with no comments