0.2 Second before the black holes collide. Image Credit: SXS/LIGO
As early as 2001, the Pegasus team at the Information Sciences Institute (ISI) had embarked upon an effort to create software that would accelerate scientific discovery by enabling scientists to make computations on massive amounts of data.This week, physicists at the Laser Interferometer Gravitational-Wave Observatory (LIGO) announced a long-awaited scientific discovery: detecting gravitational waves as predicted by Einstein’s Theory of Relativity. Pegasus was ever-present behind the scenes.
For the past 15 years, ISI’s Science Automation Technologies research group and the Pegasus software team have been working with LIGO, creating new technologies and frameworks to help physicists automate the analyses of tens of terabytes of data and run millions of workflow jobs that would eventually help prove the existence of gravitational waves. ISI’s Pegasus software, led by distributed computing expert Ewa Deelman (recipient of the 2015 ACM High-performance Parallel and Distributed Computing award) and computer scientists Karan Vahi, Mats Rynge, Rajiv Mayani, Gideon Juve, Rafael Ferreira da Silva along with a cohort of Ph.D. and M.S. students, allows scientists to create workflows, automate data management and job scheduling, and store the results. The software eliminates the need to develop manual commands or scripts, and supports graceful recovery from failures without the fear of losing data or computations. Fittingly, it was originally developed with astronomers in mind and it is named for the constellation.
True to the interdisciplinary nature of ISI, the Science Automation Technologies team has long worked at the intersection of the physical sciences and computer science, innovating and applying computer science to real-world problems. Another key user has been the Southern California Earthquake Center, which uses Pegasus to create a model of seismic hazards in Southern California. This data helps seismologists predict the flow of an earthquake, which is critically important in helping civil engineers ensure human safety. Pegasus has also been leveraged for climate modeling, bioinformatics such as understanding genetic patterns in conditions like schizophrenia and for biologists, the software is even being used to track the extinction of monkeys.
When you ask Deelman about the hundreds of Pegasus projects and millions of workflows all over the world, and if she is at all bothered that she made this software open-sourced versus a for-profit product, she smiles. “Not at all,” she says.
Deelman believes in the role of computer science to enable innovation and make impact.
“It is a great thrill to know that your work can in some way contribute to such amazing discoveries. LIGO’s detection of gravitational waves, for example, is extremely exciting. It’s a discovery that will change the way we view the universe."
Prem Natarajan, the Michael Keston Executive Director of ISI, is understandably proud of the Pegasus team.
“Pegasus’ role in enabling scientific discovery exemplifies the immense impact that ISI researchers have exerted in a wide variety of fields ranging from internetworking and cyber security to artificial intelligence and quantum computing. We congratulate the gravitational-wave discovery team and are thrilled to have had the opportunity to be part of one of the greatest scientific milestones of our times," Natarajan said.
Deelman admits managing such as huge number of workflows and sustaining the software over the last 15 years hasn’t always been easy.
"LIGO challenged the Pegasus workflow management system in terms of scale, reliability and performance. Managing millions of jobs executing on heterogeneous computer systems distributed across the nation and the globe is not an easy task. However, long-term collaborations between domain scientists and computer scientists often result in advances in both domains. The understanding of distributed systems, the algorithms and technologies developed at ISI are greatly enhanced by the fact that they are grounded in real-life problems," Deelman said.