El Reg has an exclusive interview with Eisar Lipkovitz, a senior director of engineering at Google, who states that their recent search engine update “Caffeine” moves Google’s back-end indexing system away from MapReduce and onto BigTable. Colossus is apparently their codename for v2 of GFS. Todd Hoffs then provides some further analysis in Google’s Colossus Makes Search Real-time by Dumping MapReduce, and speculates on implementation etc.
So just when everybody and their dog get ready for MapReduce, Google is taking the next step towards a more discrete solutions, where index rebuilds are no more a massive bunch of MapReduce jobs, but happen instantly as the spiders crawl a webpage, through some kind of triggers, as per Todd’s speculation.
It’s just fitting that myNoSQL releases NoSQL Databases Bring Stored Procedures Back In Fashion at the same time… although this is mostly about MapReduce.