SEATTLE, Nov. 9, 2011 – The RENCI/North Carolina booth (#2942) will be one of several on the SC11 show floor to participate in a demonstration that will connect booths in the Washington State Convention Center with large data sets in the U.S. and Europe, creating a distributed, high-speed international data grid that allows researchers to share, store and manage large data sets.

The “Big Data” grid will connect the exhibition booths of DataDirect Networks (DDN, 2304), Karlsruhe Institute of Technology (2534) and RENCI/North Carolina to the Texas Advanced Computing Center (TACC) in Austin, Texas; RENCI (Renaissance Computing Institute) in Chapel Hill, NC; and the Karlsruhe Institute of Technology in Karlsruhe, Germany. The data grid will be built with DDN’s Web Object Scaler (WOS), a hyperscale geo-distributed cloud storage system, and will use the Integrated Rule-Oriented Data System (iRODS), a data software system that manages large, complex data sets by applying management policies to control the execution of all data access and manipulation operations.

WOS is an extremely fast and easy-to-deploy object-oriented cloud storage system that can scale to unprecedented levels while still being managed as a single entity.  It addresses the needs of organizations that have petabytes of data, which need to be archived and shared between multiple data centers.

The Data Intensive Cyber Environments (DICE) research groups at the University of North Carolina at Chapel Hill and the University of California at San Diego develop iRODS, with support from a U.S. National Science Foundation grant. The iRODS@RENCI research group also supports iRODS development. iRODS is the core data management software being deployed  by the DataNet Federation Consortium, an NSF-funded project to prototype a national data management infrastructure with six science and engineering disciplines.

Combining the super fast and scalable WOS with iRODS results in a grid in which data is easily shared, managed and stored in persistent archives, according to Reagan Moore, head of the DICE group at UNC-Chapel Hill and RENCI chief scientist for data grids.

“This international data grid demonstrates how researchers can participate in collaborative research while analyzing massive data collections,” Moore said. “An iRODS-based WOS infrastructure greatly minimizes the effort required to manage and distribute large scientific data sets and make them available for such research.”

Moore and DDN Chief Scientist Dave Fellinger will provide an overview of the WOS-iRODS data grid and a second international data grid connecting Kings College London to RENCI, KEK in Tokyo, Japan; Academia Sinica in Taiwan; and IN2P3 in France.  Their talk takes place at 3:30 p.m. Tuesday, Nov. 15, in the RENCI booth. At 2:30 p.m. Tuesday, also in the RENCI booth, Moore will highlight iRODS and some of the research organizations using iRODS, including NASA, the National Optical Astronomy Observatory, the Australian Research Collaboration Service, and the Texas Digital Libraries.

For more on RENCI at SC11, see RENCI@SC11, Washington State Convention Center, Seattle

Reagan Moore, Ph.D., the principal investigator for the consortium, director of the DICE Center, SILS professor and domain scientist for data management at RENCI.

CHAPEL HILL, NC, Sept. 28, 2011–The National Science Foundation has funded the University of North Carolina at Chapel Hill to lead a multi-institutional team that will build and deploy a prototype national data management infrastructure that addresses some of the key data challenges facing scientific researchers in the digital age.

The infrastructure will support collaborative multidisciplinary research through shared collections, data publication within digital libraries and reference collections within persistent archives.

The NSF awarded nearly $8 million over five years to the DataNet Federation Consortium (DFC), a group that spans seven universities. The DFC will address the data management needs of six science and engineering disciplines: oceanography, hydrology, engineering design, plant biology, cognitive science, and social science. About half the award will support research and development at UNC-Chapel Hill.

The Data Intensive Cyber Environments (DICE) research group in UNC’s School of Information and Library Science (SILS) leads the consortium and RENCI (the Renaissance Computing Institute at UNC-Chapel Hill)  is responsible for  federating the consortium’s diverse data repositories to enable cross-disciplinary research. Federating data involves building a common name space for identifying files, providing a context for file meaning and relevance, providing a common access interface, and developing management policies across the distributed collection.

The DFC will use iRODS, the integrated Rule Oriented Data System, to implement a policy-based data management infrastructure. iRODS, developed by UNC’s DICE Center and DICE researchers at the University of California at San Diego, enforces policies as computer actionable rules to organize distributed data into sharable collections.  Procedures to automate data management functions are cast as computer executable workflows.  Policies control data access, sharing and archiving. Research groups worldwide, including the NASA Center for Climate Simulations, the National Optical Astronomy Observatory, the Australian Research Collaboration Service, and the Texas Digital Libraries, use iRODS technology to manage their research data grids, implement digital libraries, and build persistent archives.

“Excelling in the digital age requires that scientific disciplines and government agencies have the ability to manage the enormous amount of data that are generated each day,” said UNC-Chapel Hill Vice Chancellor for Research Barbara Entwisle. “Scientists can only solve the important problems of our times if they can easily access, share, analyze, and preserve data for future researchers and students. This award is important beyond its dollar amount because it establishes Carolina as the leader in the worldwide research community in taming the data deluge and as the data federation hub for collaborative research. It’s a role that is essential for future discoveries and innovations.”

Experts in the DICE group and at RENCI will work with six NSF-supported national consortia to federate their distributed data repositories and create policies for retention, distribution, access and validation of critical data properties. Those communities are:

• The Ocean Observatories Initiative (OOI), an NSF-funded program led by the University of California at San Diego and the Scripps Institution of Oceanography in San Diego. The OOI researchers use data from environmental sensors to study the physical, chemical, geological and biological variables in the ocean and seafloor.
• The Consortium of Universities for Advancement of Hydrologic Science, Inc.  (CUAHSI) an organization led by the University of South Carolina. CUAHSI includes more than 130 partner organizations, including UNC’s Institute for the Environment, working to advance water science.
• CIBER-U, the Cyber-Infrastructure-Based Engineering Repositories for Undergraduates, an initiative led by Drexel University, which uses digital design repositories to enhance engineering instruction and learning.
• The iPlant Collaborative, a community of researchers and students led by the University of Arizona that is developing an integrated cyberinfrastructure to advance studies of plant biology.
• The Odum Institute for Research in Social Science, an interdisciplinary institute at UNC-Chapel Hill that focuses on teaching and research in the social sciences.
• The Temporal Dynamics of Learning Center (TDLC), an NSF Science of Learning Center based at the University of California at San Diego that studies the role of time and timing in learning in order to improve educational practices.

Arizona State University researchers will participate in the DFC by collaborating on policy-based data management systems and Duke University researchers will collaborate on education and outreach initiatives to broaden the impact of the DFC.

“The data we will work with includes observational data from sensors, experimental and simulation data, engineering designs and both structured and unstructured data,” said Reagan Moore, Ph.D., the principal investigator for the consortium, director of the DICE Center, SILS professor and domain scientist for data management at RENCI.  “The infrastructure we develop will address all stages in the community-based data collection lifecycle, from initial collection formation for a single project, to shared collections across institutions, to formation of data processing pipelines, to publication and long term preservation. We see this as the first step to building a data infrastructure that will accommodate collaborative research, new educational approaches and innovative problem solving in academic institutions, in federal agencies and across national boundaries.”

During the first 18 months of the grant, the consortium will focus on federating the data management cyberinfrastructure for the OOI, CUASHI and CIBER-U. The work will include identifying federation requirements, integrating existing data management systems, deploying a federation hub, and developing policies and procedures for data sharing so that the data collections of these research communities can become the foundation of a national data cyberinfrastructure.

For more information:

iRODS website

DICE Center website

SILS website

The Data Intensive Cyber Environments Center (DICE Center) at the University of North Carolina at Chapel Hill will hold the third annual User Group Meeting for iRODS, the Integrated Rule-Oriented Data System, on “Sustainable Policy-Based Data Management, Sharing, and Preservation.”

The meeting is cosponsored by RENCI and will be held February 17 – 18, 2011, at RENCI’s Europa Center headquarters in Chapel Hill.

The user meeting is an opportunity for the growing iRODS community to participate in sessions on applications of iRODS, discuss sustainability initiatives, and learn about new planned technology development.  The sessions are all focused on helping users implement and extend the new paradigm of sustainable policy-based data management for the sharing and preservation of today’s diverse and rapidly growing digital data collections.

Meeting organizers invite papers and posters on applications of iRODS, client interfaces, integration examples, and more, with submissions due January 17, 2011. More information on the meeting, the call for papers and posters, and registration is available on the iRODS wiki.

iRODS is advanced open source data grid technology for creating shareable “virtual” digital data collections, which can range from personal collections to the largest scales—petabytes of data with hundreds of millions of files—and span sites distributed across the hall or around the globe.

The iRODS User Meeting will bring together users new to iRODS who want to learn how to get the most out of the advanced technology with others already using iRODS and developers in the open source project.

Because iRODS provides key data management features not found in other open source systems, there is rapid growth in use of the software. Responding to this demand, a new collaboration between the iRODS group at RENCI (iRODS@RENCI) and the DICE team is working to expand production support for new iRODS user communities and sustainable development support for features requested by current and future iRODS users. The user meeting will provide an opportunity to interact with both DICE and RENCI staff.

The meeting will feature sessions on:

- New features in iRODS and new versions of the Rule Engine

- Applications of iRODS in data grids, digital libraries and archives

- iRODS interfaces, including Jargon and user-supported clients

- Sustainability plans for iRODS open source software

- User feature request prioritization, and more.

About iRODS

Based on more than a decade of user-driven experience with real-world challenges in a range of applications, core iRODS development is led by the Data Intensive Cyber Environments Center at UNC Chapel Hill and the UC San Diego Institute for Neural Computation (INC), and RENCI, with major collaboration and partners in the U.S. and internationally.

Since its introduction in 2008 the iRODS data grid system has steadily grown in users, with more downloads with each release.  Making use of the significant set of iRODS generic capabilities, projects across the U.S. and international collaborations are applying the versatile iRODS system in a wide range of fields, from sharing data in scientific research collaborations to managing real-time data streams, publishing collections in digital libraries, and preserving electronic records over the long term in archives.

Core development of iRODS is led by the DICE team at UNC-Chapel Hill, UCSD and RENCI. As the iRODS open source community expands, more projects and developers are contributing code with the goal of extending the open source iRODS system for their applications and the benefit of all. Development partners include CC-IN2P3, the Centre de Calcul de l’Institut National de Physique Nucléaire et de Physique des Particules; SHAMAN, the EU Sustaining Heritage Access through Multivalent ArchiviNg project and the University of Liverpool; ARCS, the Australian Research Collaboration Service; UK e-Science; CeRch, the Centre for e-Research at King’s College, London; the KEK High Energy Accelerator Research Organization in Japan, and others.

Core iRODS development has been supported by the National Science Foundation (NSF) and the National Archives and Records Administration (NARA), and most recently, RENCI.

Related Links:

iRODS User Group Meeting 2011
iRODS Integrated Rule-Oriented Data System wiki
Data Intensive Cyber Environments Center (DICE Center)
Renaissance Computing Institute (RENCI)
Data Intensive Cyberinfrastructure Foundation
iRODS Introductory Information

CHAPEL HILL, NC, July 16, 2009—Almost a year after the Data Intensive Cyber Environments (DICE) research group moved from the University of California at San Diego to the University of North Carolina at Chapel Hill, the internationally recognized research group has established deep ties to the Renaissance Computing Institute (RENCI).

Together, DICE and RENCI are working to link data repositories across the state and make them accessible and easy to use by researchers, businesses and government. Through a variety of collaborations, the two groups hope to establish North Carolina as an international leader in solving the pressing problems of managing and sharing today’s deluge of digital data in ways that provide a boost to research, business and education.

DICE members, long of  UCSD’s, San Diego Supercomputer Center, made the cross county trek to UNC Chapel Hill early last fall, bringing with them a research portfolio in excess of $10 million. The researchers hold appointments in Carolina’s nationally recognized School of Information and Library Science (SILS) and also serve as chief scientists at RENCI, with research space at RENCI’s off campus Chapel Hill location.

For more than 10 years the DICE group’s data grid technologies have been used in research projects worldwide to manage large, distributed data collections and support discovery, access, retrieval, replication, archiving, and analysis tasks. The researchers most recently released version 2.1 of iRODS, the open source Integrated Rule-Oriented Data System, which introduced user-settable rules that automate complex management policies, helping users handle today’s mushrooming collections of digital data.

“Having RENCI in North Carolina was one of the main considerations in coming here,” said Reagan Moore, head of the DICE group, director of the new DICE Center at UNC-Chapel Hill and chief scientist for data intensive cyber environments at RENCI. “RENCI works to provide the technology infrastructure that thriving universities and business communities need and a data infrastructure is part of that. Working together, RENCI and DICE can develop the kind of technology infrastructure that opens up all kinds of opportunities.”

RENCI works closely with the DICE Center, which was launched in May with funding from the National Science Foundation, the National Archives and Records Administration and the National Historical Publications and Records Commission. The center draws on leading data management technology, which is advanced and generic enough to have a remarkable array of uses, from helping the National Archives digitally preserve the nation’s historical information to helping digital libraries cope with the ever-increasing size and complexity of digital knowledge to enabling sharing of digital data by large-scale interdisciplinary scientific research collaborations across the nation and the globe.

The center is working to establish an interoperable data center that spans the 17-campus UNC system that allows campus researchers to easily access, search, share and mine vast stores of data. The bulk of the center’s data cache will be located at RENCI, which currently provides about 100 terabytes of data storage to the DICE center. Eventually, said Moore, DICE hopes to have a data infrastructure, supported by RENCI, that provides more than 1 petabyte of data management and storage capacity.

“The indispensable role of digital data across society and the increasing size and complexity of data collections are reaching a critical point,” said Richard Marciano, executive director of the DICE center and RENCI chief scientist for persistent archives and digital preservation. “With the growing need for practical digital data technologies, the new DICE Center is already collaborating with many important projects across UNC-Chapel Hill as well as national and international partners, helping them harness their digital data collections and working with them to efficiently create, share and preserve new knowledge.”

Regional hubs through Data Grid
A premier DICE-RENCI project is the RENCI Data Grid, which links RENCI’s engagement centers at UNC-Chapel Hill, Duke, NC State, UNC Charlotte, UNC Asheville, and East Carolina University as interconnected regional data hubs managed by iRODS. The project started in late 2008 by adding multi-terabyte storage capacity at the RENCI engagement centers and at RENCI’s offices in Chapel Hill. Those data hubs will house RENCI visualization files, the data on which the visualizations are based, and the visual analytics software tools that enable many additional levels of insight into the data. But Data Grid’s potential is much greater, according to Ray Idaszak, RENCI’s director of visualization and collaborative environments.

“Data Grid is the start of an effort by DICE and RENCI to give the state and its universities a data resource that will be a tool used for decision making, for increasing productivity or for examining data in the context of other data,” said Idaszak.
When completed, the Data Grid in action might work like this: Data on development patterns around the North Carolina would be stored at RENCI at UNC Charlotte, where researchers at the RENCI engagement center study urban growth patterns and their implications. An urban planner in eastern North Carolina would be able to access that data as well as the software tools that allow it to be viewed in a visual, intuitive format. Those same researchers also would be able to access coastal floodplain maps and storm surge visualizations stored at other data hubs and to use all of the information to plan sustainable coastal developments.

One of the largest datasets the Data Grid plan to offer will be orthophotographic images—an extremely high resolution photographic map of the state with data points at every six inches across a statewide grid. When overlaid with other datasets—for example, census data or information on the location of healthcare facilities—the orthophotographs become tools useful to emergency managers, city planners, researchers and area businesses.

All of the Data Grid hubs will be operational by fall 2009 and new datasets will be added to the collection as more storage capacity becomes available.

Other projects on which RENCI and DICE collaborate include:
• Distributed Custodial Archival Preservation Environments (DCAPE), a project to build a distributed data preservation environment that meets the needs of archival repositories for trusted archival preservation services.  DCAPE develops preservation policies for state and university archives and cultural institutions, using iRODS to implement and deliver services. Participants include the North Carolina State Archives and the State Library of North Carolina.

• EnginFrame engagement portals, an effort to integrate EnginFrame’s portal environment for using grid-empowered applications on organizational intranets with the RENCI Data Grid. EnginFrame is a major portal for Enabling Grid for e-SciencE (EGEE), the European grid infrastructure for scientists. The collaboration among RENCI, DICE and EnginFrame allows RENCI to improve its Science Gateway portal to the National Science Foundation’s TeraGrid. It will also provide researchers in the UNC system and beyond user-friendly access to distributed data collections available through the Data Grid.

• TUCASI data Infrastructure Project (TIP), an effort to deploy a federated data cyberinfrastructure across the campuses in the Research Triangle area. Funded by the Triangle Universities Center for Advanced Studies, Inc. (TUCASI), the project aims to meet the growing data storage and management needs of UNC-Chapel Hill, Duke and NC State through an interoperable data repository. Eventually, TIP will provide researchers at Triangle area universities with over a petabyte of data storage capacity and may also link to business-operated storage “clouds.” As DICE’s Moore explained, “TIP may serve as a gateway linking business storage clouds and institutional repositories, and will make migration of data between these two worlds much more feasible.”

• Carolina Digital Repository, a project with the UNC-Chapel Hill library to store and provide access to digital collections. The project provides local storage managed by iRODS and a back up of records at RENCI.

• National Archives and Records Administration Transcontinental Persistent Archive Prototype (NARA TPAP). This project to develop, implement and test a nationwide data management infrastructure federates seven independent data grids. RENCI is one of the experimental nodes on the system and works with other NARA TPAP partners to test the persistent archive prototype and the tools and policies which support it.

• DataNet Federation Consortium, a five-year, $20 million project proposal recently submitted to the National Science Foundation that aims to create a nationwide data management infrastructure that will make it easier for researchers across a wide range of science and engineering disciplines to access, use and manage diverse data sets.  The proposal addresses the need to create federated data collections that can be shared through the Internet across disciplines and institutions. If funded, the consortium will build consensus on how to manage, access and store the enormous data files being created daily by sensors and other real-time data streams, medical imaging equipment, genomic studies, statistical studies of populations and more.

More information:
DICE Center: http://dice.unc.edu
RENCI: http://www.renci.org/
SILS: http://sils.unc.edu/
iRODS: http://www.irods.org

Media contact: Karen Green, (919) 445-9648 (office), (919) 619-8213 (mobile), kgreen@renci.org