A new Renaissance Computing Institute (RENCI) animation created from a mathematical model shows a black hole moving supersonically through an interstellar gas cloud. This phenomenon often occurs in multiple star systems, where a companion star provides the gas cloud. The gravity of the black hole pulls the gas inward. Early on in the process, a wake forms behind the black hole, much like the wake behind a motorboat. Unlike a motorboat wake, it begins to move back and forth after a while until it whips all the way around the black hole, forming an accretion disk of gas falling into the hole.

What has mystified scientists since 1988 is the seemingly erratic rotation of accretion disks in the computer-generated models. In some cases the model shows the disk rotating for a time in one direction, then suddenly switching directions. The disk’s spin may remain stable for a time and then abruptly reverse direction again. The reversal may repeat several times.

Astrophysicists refer to this accretion disk about face as “flip-flop instability” and have debated its possible causes for years. Some suggest the phenomenon doesn’t actually occur but is a flaw in the model itself. Yet flip-flop instability has shown up in numerous different studies, leading some scientists to suggest it is the cause of stellar flares and bursts of energy that haven’t otherwise been explained.

North Carolina State University researchers John Blondin and T. Chris Pope generated the data used to construct this visualization. Their computer simulations exploit the power of high-performance computers available through the National Science Foundation’s TeraGrid to explore flip-flop instability at an unprecedented level of detail and scientific sophistication. In a paper published June 30 in the Astrophysical Journal, they conclude that the flip-flop instability is real and not an anomaly of computer models.

In the animation, created by Steve Chall of the RENCI’s North Carolina State Engagement Center, the gas cloud swirls around the black hole at the center, creating an accretion disk around the black hole. At first, the disk spins counterclockwise and then very rapidly reverses to a clockwise direction. The background colors in the animation represent pressure, from green for low through blue, violet, red and finally pale yellow for the highest pressure. Spheres emitted from 10 equally spaced sources upstream from the black hole (far right) show the velocities of representative particles in the gas cloud. Pale yellow spheres exhibit the least velocity, through red, violet and up to blue for the fastest-moving particles.

Credits: Numerical simulation: Dr. John Blondin and T. Chris Pope, department of physics, North Carolina State University. Visualization: Steve Chall, Renaissance Computing Institute NC State. This research was supported in part by a grant from the National Science Foundation, by an NC State Undergraduate Research Award to T. Chris Pope, and by computing resources at the Texas Advanced Computing Center.

3D replica of senecavirus, a pathogen discovered several years ago by researchers in Pennsylvania. UC San Francisco researcher Eric Delwart and his colleague Chunlin Wang of Stanford University use the RENCI-developed TeraGrid Science Gateway to access grid computing resources in their search for new viruses. (Credit: Institute for Animal Health, UK)

DNA sequencing and sequence analysis happens daily in many biological sciences laboratories, but analyzing large sets of genetic data increasingly requires computing resources beyond the capabilities of most labs.

That search for the best hardware and software led Eric Delwart, a professor of laboratory medicine at the University of California, San Francisco, and a senior investigator at the Blood Systems Research Institute, and Chunlin Wang, a research associate at the Stanford University Genome Technology Center, to the Renaissance Computing Institute’s (RENCI) Engagement Team and then to the distributed computing resources of the TeraGrid and the Open Science Grid (OSG).

Screen shot of the BLASTMaster interface, part of the RENCI-developed TeraGrid Science Gateway interface that allows researchers to easily use distributed grid resources.

Delwart works with Wang to identify new viruses. The team uses a technique called massively parallel pyrosequencing, which can determine sequences for millions of DNA fragments using high-throughput computing. The resulting DNA sequences are then compared to all the sequences in public sequence databases to identify viral fingerprint sequences. One single sequencing reaction generates massive volumes of data that can take months, even years, to analyze on a small-scale computing cluster.

In an effort to analyze more data more efficiently, Delwart and Wang turned to the RENCI Engagement Team, which participates in the TeraGrid Science Gateways program and the leads the OSG Engagement program. TeraGrid’s Science Gateways aim to bring new communities of users to TeraGrid resources by providing easy access to the TeraGrid’s distributed computing resources. The OSG Engagement program recruits new users from a wide range of disciplines and helps them become users of the distributed computing systems operated and maintained by OSG members.

The effort to find more computing power paid off for the Delwart and Wang: In early May, they used 70,000 CPU hours on TeraGrid and OSG resources to complete in a week a DNA sequence analysis that would have taken over three months on their own lab cluster. The team submitted its jobs to the national resources using a RENCI-developed, Web services-based computational science platform [reference: http://www.teragrid.org/tg09/files/tg09_submission_75.pdf ].

“We created an application that communicates with RENCI’s TeraGrid Science Gateway,” said Jason Reilly, a RENCI senior research software developer. “For the user, it’s very simple—just log in and the application maps the data input to specific tasks to be done. The beauty is you don’t have to submit commands over and over again. You can run hundreds or even thousands of operations and you only have to submit the command once.”

The custom application created by Reilly was dubbed BLASTMaster because it builds on the Basic Local Alignment Search Tool (BLAST) used to search sequence databases. BLASTMaster divides commands into tasks and pushes the work to RENCI’s TeraGrid Science Gateway, which submits, monitors, and manages the compute workload on systems that are part of TeraGrid’s nationwide network of high performance machines, and to OSG machines. After entering the initial commands, the researchers merely had to wait for their results.

3D replica of senecavirus, a pathogen discovered several years ago by researchers in Pennsylvania. UC San Francisco researcher Eric Delwart and his colleague Chunlin Wang of Stanford University use the RENCI-developed TeraGrid Science Gateway to access grid computing resources in their search for new viruses. Credit: Institute for Animal Health, UK

“Large computer farms that we might use are often composed of heterogeneous smaller clusters,” said Wang. “The BLASTMaster tool and a Web services environment is particularly useful to those of us without much experience using compute clusters. It gives us a uniform interface to submit jobs, which greatly enhances our productiveness.”

The sequence analysis work used TeraGrid resources at Purdue University (West Layfayette, IN), OSG resources at RENCI (Chapel Hill, NC) and a cluster in the University of North Carolina at Chapel Hill computer science department supported by the National Institutes of Health. The work has real-world value taken straight from recent headlines about the H1N1 virus.

“Knowing the genomic sequence of a human virus allows for quicker diagnostics to identify infections,” said Delwart. “Quicker diagnostics can lead to more informed decisions on how an emerging virus is spread and how to control it. Knowing the sequence can also help make vaccines or anti-virals against that virus.”
_______________

OSG is a consortium of universities, national laboratories, scientific collaborations and software developers dedicated to meeting the ever-growing computing and data management requirements of scientific researchers. Supported by the U.S. Department of Energy Office of Science and the National Science Foundation, it provides access to its members’ independently owned and managed resources through a common grid infrastructure that uses high performance networks to connect computing systems scattered across the country. For more see http://www.opensciencegrid.org.

TeraGrid is an open scientific discovery infrastructure combining leadership class resources at eleven partner sites to create an integrated, persistent computational resource.  TeraGrid is funded by the National Science Foundation. For more, see http://www.teragrid.org.

The Renaissance Computing Institute (RENCI), a multi-institutional organization, brings together multidisciplinary experts and advanced technological capabilities to address pressing research issues and to find solutions to complex problems that affect the quality of life in North Carolina, our nation and the world. Founded in 2004 as a major collaboration of Duke University, North Carolina State University, the University of North Carolina at Chapel Hill and the state of North Carolina, RENCI is a statewide virtual organization. For more see http://www.renci.org

The TeraGrid Bioportal is a shared, extensible portal environment that brings together more than 100 applications and many standard biological data sets. It provides access to high-end computing resources, including a dedicated cluster and TeraGrid systems across the U.S. Through the Bioportal, biological researchers, students and educators are able to seamlessly access data, resources and applications, compare biological data stored in different formats and remotely collaborate with colleagues.

The portal builds on the success of the North Carolina Bioportal, which was developed by RENCI for use by researchers and educators in North Carolina with seed funding from the University of North Carolina’s Office of the President. Development of the portal for TeraGrid users was supported by the TeraGrid Science Gateways program. RENCI also receives support from the National Institutes of Health for work integrating evolutionary biology and biomedical tools into the Bioportal.

“By incorporating the Bioportal into the TeraGrid infrastructure we are giving a large community of researchers in genomics, proteomics, molecular biology and other fields access to a nationwide network of high-end resources,” said Dan Reed, director of RENCI and principal investigator on the Bioportal TeraGrid Science Gateway project. “These are researchers whose work will have far-reaching impacts. They are uncovering relationships between genetics and human diseases, developing new treatments and even finding cures.”

The Bioportal is an open source framework that builds on emerging grid technologies from the National Science Foundation Middleware Initiative (NMI) and the Open Grid Computing Environment (OGCE) tool suite. Its grid middleware layer is based on Globus, the fundamental open source software for sharing resources and tools across distributed sites, and MyProxy, the grid credential management system. Bioportal resources support a number of common research activities, including database searching, alignment and phylogeny, pattern searching, DNA/RNA analysis and protein analysis. In addition to offering access to TeraGrid resources, the TeraGrid version of Bioportal offers a number of new features, including:

• new versions of widely used open source bioinformatics applications
• automated file format conversion
• enhancements to security policies to align with campus policies.

“The Bioportal project is an excellent example of the how the TeraGrid Science Gateway projects are extending the capabilities of the TeraGrid and providing access to this national cyberinfrastructure facility for a wider community of users,” said TeraGrid Director Charlie Catlett of Argonne National Laboratory and the University of Chicago.

Before the end of the year, RENCI will incorporate a workflow system into the Bioportal that will allow users to chain applications together for multifaceted analyses. Workshops for researchers, educators and students also are planned.

RENCI…Catalyst for Innovation
The Renaissance Computing Institute (RENCI) is a catalyst for innovation in science, engineering, the arts, humanities, and commerce. It fosters multidisciplinary collaborations by leveraging and applying leading edge compute, network, and data information technology resources and capabilities. RENCI is a joint institute of the University of North Carolina at Chapel Hill, Duke University, and North Carolina State University that combines the strengths of these three institutions with the social, business and research opportunities of the Research Triangle Park and the state of North Carolina. For more, see www.renci.org.

About Teragrid
The TeraGrid, sponsored by the National Science Foundation Office of Cyberinfrastructure, is a partnership of people and comprehensive resources that enables discovery in U.S. science and engineering research. Through high-performance network connections, the TeraGrid integrates a distributed set of high capability computational, data management and visualization resources to make U.S. research more productive. With Science Gateway collaborations and education and mentoring programs, the TeraGrid also connects and broadens scientific communities.

Relevant URLs
TeraGrid Bioportal
Official TeraGrid Website
North Carolina Bioportal

Built and implemented over the last four years, the TeraGrid is the world’s most comprehensive distributed cyberinfrastructure for open scientific research. Through high performance network connections, the TeraGrid integrates high performance computers, data resources and tools, and high-end experimental facilities around the country.  The TeraGrid currently serves over 1,600 scientists and engineers who use computational, data management and visualization resources at eight sites across the U.S.

The Renaissance Computing Institute (RENCI, www.renci.org) hosted the meeting. RENCI Director Dan Reed helped develop the TeraGrid project in 2001 when he was director of the National Center for Supercomputing Applications. He coined the term TeraGrid to describe a grid infrastructure capable of moving and analyzing trillions of bytes of data – or terabytes. RENCI contributes to the TeraGrid through the Science Gateways initiative. Science Gateways work to make TeraGrid resources accessible to new  communities of users through common Web portal interfaces. RENCI works with three TeraGrid Science Gateway projects:

• RENCI leads the Bioportal and Biomedicine Science Gateway project, which is developing a portal interface that will give biologists an easy-to-use interface for developing computational models, comparing large datasets including genomic sequences, accessing instruments and collaborating with colleagues. RENCI is adapting its North Carolina Bioportal for the TeraGrid.
• The Linked Environments for Atmospheric Discovery (LEAD) Science Gateway is creating cyberenvironments for mesoscale atmospheric research. RENCI’s work with LEAD focuses on performance monitoring and adaptation and fault tolerance performability and recovery for the LEAD infrastructure.
• RENCI participates in the Open Science Grid, a consortium of U.S. universities and laboratories that works to create a common national grid infrastructure that is open in its architecture, implementation and use.

More science, more successes

As the TeraGrid has matured, it has become an important resource for researchers working on a wide range of scientific problems. Examples include:

• Harvey Newman, a particle physicist at the California Institute of Technology, ran simulations across multiple TeraGrid sites using Gridshell, which provides built-in support for job submission and scheduling, remote I/O redirection, parallel execution of commands/jobs, and inter-script task communication. Newman is investigating the discovery potential of CERN’s CMS experiment at the Large Hadron Collider. The work involves generating, simulating, reconstructing and analyzing tens of millions of proton-proton collisions.
• Klaus Schulten, a University of Illinois molecular biologist, conducted simulations of the activities of nuclear pores, large protein complexes that allow water soluble molecules to pass into a cell’s nucleus but restrict other molecules from passing.
• The MIMD Lattice Calculation (MILC) collaboration, led by Bob Sugar of the University of California at Santa Barbara, continues to be one of the largest users of TeraGrid resources. The group studies quantum chromodynamics, or QCD, which describes the span interactions that bind protons and neutrons together to form the nuclei of atoms.
• In a TeraGrid use that extends beyond science, Rob Shakespeare, a professor in the theatre and drama department at Indiana University, is using TeraGrid resources to create virtual environments to replace traditional theatre sets.

The future: accessibility and interoperability

Grids, according to TeraGrid Director Charlie Catlett, can be much more than a system for linking and accessing powerful computing resources.

“There are growing data collections out there and researchers would be well served if they were integrated and easily accessible,” he said. “There is a need for collaborative services and for work flow systems that can readily incorporate TeraGrid resources. These are all important elements of the TeraGrid infrastructure and of cyberinfrastructure in general. We must extend these capabilities.”

The Science Gateway projects are an important ingredient in extending the capabilities and accessibility of the TeraGrid and new Science Gateway projects that operate seamlessly with the TeraGrid is a goal in the coming year, added Catlett.

In addition, TeraGrid technical staff will continue to implement tools that make collaboration, scheduling and other operations as easy as possible. These include advanced capabilities such as web services enabled by the Globus Toolkit (GT4), new scheduling services, expansion of the high performance, wide area distributed file systems and introduction of a TeraGrid user portal.

More information about TeraGrid facilities, science highlights, science gateways, and other programs is available at www.teragrid.org.

Relevant URLs:
TeraGrid Website
North Carolina Bioportal