M-A-O-L

I don’t think I’ve written about Google’s Megastore yet, so here’s a quick summary of worthwile resources.

Megastore is the data engine supporting the Google Application Engine. It’s a scalable structured data store providing full ACID semantics within partitions but lower consistency guarantees across partitions.

James Hamilton’s take on Google Megastore: The Data Engine Behind GAE. His blog is worth following for people interested in scaling infrastructure in general, not just DBs. Todd Hoff’s write-up is about Google Megastore – 3 Billion Writes and 20 Billion Read Transactions Daily, his blog is about everything High Scalability. And last but not least the Storage Mojo take on Google’s Megastore, from a storage insider.

At times it’s difficult to judge what things are really about. Look at the following announcement by the folks at Nutanix. They got Aster Data, Google File System and Oracle Storage Layer (incl. Exadata, they say) background, but what’s the point of this appliance? Hope we’ll soon here more from them.

The key to Nutanix is virtualization, which provides the abstraction and the additional storage connections necessary to give Nutanix the performance edge it claims. The company is big on solid-state drives for performance and consolidation, but Pandey says that legacy storage systems are limited to the amount of SSDs they can handle. With a virtualized computing layer, however, each virtual server and each physical node provide the requisite housing and connection to an additional SSD. The Nutanix appliance combines both SSDs and hard disk drives to achieve maximum levels of performance and affordability, Pandey said.

via Nutanix Gets $13.2M for Google-like Storage Architecture.

At least somebody read through the seminal papers on NoSQL technologies Google BigTable and Amazon Dynamo, and created 18-20 page summary decks to spare us the reading. Check the two decks below.

• Summary of Google’s Big Table Paper
• Overview of the Amazon Dynamo Paper

There’s movement in the realtime NoSQL world. As GigaOm reports in Yahoo Open-Sources Real-Time MapReduce, Yahoo! is the first to release a large scale implementation of a more realtime oriented NoSQL system (don’t think it’s Hadoop or even MapReduce based), which will allow to query data pretty much as it’s added to the system. Or as the S4 project describes it:

S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data.

All of that on the heels of Google’s changes related to, or at least coinciding with, the introduction of Google Instant.

Following up on Google dumping MapReduce, there are now a couple articles available that shed more light onto that decision and what it means for MapReduce. Go read MapReduce and Hadoop Future and then Google’s Dremel – or, Can MapReduce Itself Handle Fast, Interactive Querying? for additional thoughts on why Google’s decision isn’t the end of MapReduce.

El Reg has an exclusive interview with Eisar Lipkovitz, a senior director of engineering at Google, who states that their recent search engine update “Caffeine” moves Google’s back-end indexing system away from MapReduce and onto BigTable. Colossus is apparently their codename for v2 of GFS. Todd Hoffs then provides some further analysis in Google’s Colossus Makes Search Real-time by Dumping MapReduce, and speculates on implementation etc.

So just when everybody and their dog get ready for MapReduce, Google is taking the next step towards a more discrete solutions, where index rebuilds are no more a massive bunch of MapReduce jobs, but happen instantly as the spiders crawl a webpage, through some kind of triggers, as per Todd’s speculation.

It’s just fitting that myNoSQL releases NoSQL Databases Bring Stored Procedures Back In Fashion at the same time… although this is mostly about MapReduce.

Was war so geheim am Google-Server?

Wieso will Microsoft Facebook kaufen, und als einzige Zugriff auf alle Daten und Informationen haben?

(Via)

Google will der Leim zwischen den Netzwerken sein. Google OpenSocial ist ein Set von APIs mit standardisiertem Zugriff auf all die unterschiedlichen Netzwerke wie LinkedIn, Plaxo, Orkut und Xing, die z.T. nicht mal selbst ein API anbieten.

Die Presse überschlägt sich mit Begeisterung für die Pressekonferenz (in der Reihenfolge des Erscheinens in meinem Feedreader): GigaOm, TechCrunch, TechCrunch mit Screenshots, TechCrunch Vor-Vorankündigung und Heise.

Dabei habe ich auf die Schnelle überhaupt nirgends gesehen, wann das verfügbar sein soll.