Hedwig
Benjamin Reed (Yahoo! Research)
June 7, 2010
ABSTRACT
Hedwig is a large scale cross data center publish/subscribe service developed at Yahoo! Research. We needed a scalable, fault tolerant, publish/subscribe service that has strong delivery and ordering guarantees to maintain the consistency of replicas of datasets in different data centers. We found that we could use ZooKeeper, a coordination service, and BookKeeper, a distributed write-ahead logging service to build such a service. To present Hedwig I will first review two of the services it is built on: ZooKeeper and BookKeeper. I will then present the motivation for Hedwig, review its design, and present its current status.
BIOGRAPHY
Benjamin Reed has worked for almost 2 decades in the industry: from an intern working on CAD/CAM systems, to shipping and receiving applications in OS/2, AIX, and CICS, to operations, to system admin research and Java frameworks at IBM Almaden Research (11 years), until finally arriving at Yahoo! Research (3 years ago) to work on distributed computing problems. His main interests are large scale processing environments and highly available and scalable systems. Dr. Reed’s research project at IBM grew into OSGI, which is now in application servers, IDEs, cars, and mobile phones. While at Yahoo, he has worked also worked on Pig and ZooKeeper, which are Apache projects for which he is a committer. Benjamin has Ph.D. in Computer Science from the University of California, Santa Cruz.
[This video is part of LinkedIn's tech talk series.]
"
No comments:
Post a Comment