We present Mesos, a platform for sharing commodity clusters between ...
Updating an index of the web as documents are crawled ...
We present Globule, a collaborative content delivery network developed by ...
Dryad is a general-purpose distributed execution engine for coarse-grain data-parallel ...
Google offers a variety of Internet services that require user ...
The ideal distributed file system would provide all its users ...
BANKS is a system that enables keyword based searches on a relational database. As a paper that was published 10 years ago in ICDE 2002, it has won the most influential paper award for past decade this year at ICDE. Hearty congrats to the teamContinue reading
The buzz about Hadapt and HadoopDB has been around for a while now as it is one of the first systems to combine ideas from two different approaches, namely parallel databases based on a shared-nothing architecture and map-reduce, to address the problem of large scaleContinue reading
One of the aspects you can’t miss even as you just begin reading this paper is the strong scent of functional programming that the design of Spark bears. The use of FP idioms is quite widespread across the architecture of Spark such as the abilityContinue reading
Kafka, a system developed at LinkedIn, is essentially a messaging system that is designed to support aggregation of high throughput log messages arriving from different applications. Why would a traditional messaging system not be a good fit for log processing? Typical enterprise messaging systems tendContinue reading
Windows Azure Storage is a key component of the Windows Azure Cloud platform that offers an infinite disk in the cloud. It’s been in production since November 2008 and is used heavily within Microsoft in addition to being available as a public cloud service. ItContinue reading
The Scandinavian mythology regards Thialfi, a swift runner, as the attendent of Thor, the god of war. Motivated by the swiftness that qualifies Thialfi, was perhaps why the folks at Google named their message delivery system (it delivers notifications at sub-second intervals!) after him. TheContinue reading
This paper from folks at Spotify primarily focuses on how they use P2P techniques in their platform. The service is not web-based, but instead uses a proprietary client and protocol. At the heart of the system is this custom music streaming protocol that is optimizedContinue reading
This paper which appeared in this year’s VLDB talks about the internals of the SQL query engine atop Google’s Map Reduce framework. Its currently used by over 1000 people in Google serving over 10,000 queries each day that span across two data centers, with twoContinue reading
Abstract Distributed processing of real-world graphs is challenging due to their size and the inherent irregular structure of graph computations. We present HipG, a distributed framework that facilitates programming parallel graph algorithms by composing the parallel application automatically from the user-defined pieces of sequential workContinue reading
This paper describes the essential parts of Yahoo’s video delivery system. The Yahoo! Video Platform has a library of over 20 million video assets. From this library, end users make about 30,000,000 requests per day for over 800,000 unique videos, which creates a low ratioContinue reading