CS-lunsj: "Oivos: Simple and Efficient Distributed Data Processing."
Oivos: Simple and Efficient Distributed Data Processing.
The presentation takes place in meeting room 1, Realfagsbygget.
The paper was presented at the International Conference on High Performance and Communications (HPCC) September 25th - 27th 2008 in DaLian, China. The HPCC-2008 conference is the 10th edition of the highly successful International Conference on High Performance and Communications (HPCC). It provides a forum for engineers and scientists in academia, industry, and government to address the resulting profound challenges and to present and discuss their new ideas, research results, applications and experience on all aspects of high performance computing and communications. Accepted and presented papers will be included into the IEEE Conference Proceedings published by IEEE CS Press.
Conference URL:
http://hpcc08.dlut.edu.cn/index.html
Abstract: Oivos: Simple and Efficient Distributed Data Processing
The complexity of implementing large scale distributed computations has motivated new programming models. Google's MapReduce model has gained widespread use and aims to hide the complex details of data partitioning and distribution, scheduling, synchronization, and fault tolerance. However, our experiences from the enterprise search business indicate that many real-life applications must be implemented as a collection of related MapReduce programs. Since the execution of these programs must be monitored and coordinated externally, several issues concerning scheduling, synchronization, and fault tolerance resurface. To address these limitations, we introduce Oivos; a high-level declarative programming model and its underlying runtime.
We show how Oivos programs may specify computations that span multiple heterogeneous and interdependent data sets, how the programs are compiled and optimized, and how our run-time orchestrates and monitors their distributed execution. Our experimental evaluation reveals that Oivos programs do less I/O and execute significantly faster than the equivalent sequences of MapReduce passes.
Steffen Viken Valvåg started on the Ph.D. program in science (computer science) in late January 2005 and the work should be finished by mid April 2009. He is connected to the WAIF-project (Web of Asynchronous Information Filters) at the Department of Computer Science. His Ph.D. project has the preliminary title: "WAIF Infrastructure". Advisors are Professor Dag Johansen (main advisor), Department of Computer Science, and CTO / Professor II Bjørn Olstad (co-advisor), Fast / NTNU.
----------
The CS-lunch is for Ph.D. students, and it is established for the presentations of accepted conference and journal papers to the other PhD students and the staff at the department. Usually such a presentation will be done before the actual conference presentation. This will give the presenter valuable feedback.
