maol

musings and one liners

PaaS on Hadoop Yarn

PaaS on Hadoop Yarn – Idea and Prototype is looking at what’s missing to offer PaaS with Hadoop Yarn:

YARN is the next generation Hadoop MapReduce architecture. It is designed to be more flexible architecturally, improve scalability, and achieve a higher resource utilization rate, among other things. Although YARN remains similar to the old Hadoop MapReduce architecture (let’s call it Hadoop MR1), the two are different enough that most components have been rewritten and the same terminology can no longer be used for both.

In short, the Hadoop YARN architecture (or called MR2) splits the two major functions of the JobTracker of MR1 into two separate components – central Resource Manager and Application Master per application. This new architecture not only supports the old MapReduce programming model, but also opens up the possibility of new data processing models

And Monash also recently looked at Yarn.


Posted

in

by