Skip to main content

Mesos: Elastically Scalable Operations, Simplified

Adam Bordelon (Mesosphere, Inc.), Niklas Nielsen (Mesosphere, Inc.)
Average rating: *****
(5.00, 1 rating)

Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications. It can run and manage Apache Hadoop, Apache Spark, MPI, Hypertable, Storm, Chronos, Marathon, and other applications on a dynamically shared pool of nodes. Mesos provides distributed systems primitives that make it easy to build scalable, fault-tolerant frameworks.

The biggest user of Mesos is currently Twitter, where it runs on tens of thousands of cores. Airbnb runs all of their data infrastructure on it, processing petabytes of data. At such large scale, it becomes increasingly important to provide developers with direct access to cluster resources, for scaling and introducing new services. In this way, Mesos speeds development and makes life easier for the data center operator.

This talk will cover:

  • Mesos as a cluster resource manager.
  • Mesos as a distributed systems framework and library.
  • Main takeaways from large-scale adoption.
  • Our vision for future developments in Mesos data center schedulers.

Adam Bordelon

Mesosphere, Inc.

Adam is a distributed systems engineer at Mesosphere and works on Apache Mesos.
Before joining Mesosphere, Adam was lead developer on the Hadoop core team at MapR Technologies, he developed distributed systems for personalized recommendations at Amazon, and he rearchitected the LabVIEW compiler at National Instruments. He completed his Master’s degree at Rice University, building a tool to analyze supercomputer performance data for bottlenecks and anomalies.

Photo of Niklas Nielsen

Niklas Nielsen

Mesosphere, Inc.

Niklas is also a distributed systems engineer at Mesosphere and a committer in the Apache Mesos Open Source project. Before joining Mesosphere, Niklas worked at Adobe as a virtual machine and compiler engineer and completed his Master’s degree doing supercomputer debugger design at Lawrence Livermore National Laboratory.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)