Data transfer and share

NeSI supports researchers to transfer and share their research data sets, working with research communities, institutions, and REANNZ to improve end-to-end performance and reduce time to solution.

As sensors and instruments become more accessible, the new bottleneck for research is data rather than compute. Leading up to 2019, regular engagement with research communities informed NeSI of growing requirements for a range of data management services, including repositories, access management, reusability, and privacy. Researchers were generating and transferring data at multiple times larger rates than in previous years. In response, NeSI reviewed, revised, and launched a national Data Transfer Platform in early 2019 – operated in partnership with Globus, REANNZ and multiple research institutions – to enable New Zealand researchers to move data to and from NeSI quickly and easily.
 

Globus Data Transfer Nodes (DTNs) were plumbed directly into the new infrastructure platform, enabling access to data on both Māui and Mahuika systems. Globus DTNs provide a common platform for accessing, transferring, and sharing data nationally and internationally.

NeSI has also coordinated with its collaborators to establish DTNs at data storage and research facilities at AgResearch, the University of Auckland, and the University of Otago.

Following the platform’s launch, NeSI hosted three webinars to educate users about the platform’s features and capabilities, and has been actively working to support institutions who want to establish a local endpoint and join the national platform.

 

Establishing new endpoints

At the end of 2019, Manaaki Whenua - Landcare Research began the process of joining the national Data Transfer Platform, expecting to launch their Globus DTN in early 2020 and giving researchers access to 10Gbps transfer via the REANNZ network. NeSI also began conversations with Australian AARNet colleagues to explore options for improving data movement between Australia and New Zealand. By expanding the number of connected research institutions, NeSI’s national Data Transfer Platform will allow for wider support of trans-Tasman workflows and improved collaborations between international research instruments and compute facilities.

 

 

Attribution: 

Building capability for genomics research

In 2019, NeSI initiated work with Genomics Aotearoa to address requirements for storage and hosting of bioinformatics data.

A prototype data repository was developed and launched (with assistance from University of Otago IT staff), followed by a production repository hosted on NeSI infrastructure.

Both repositories utilise Globus for group-based access management and data transfer functionality and fulfil iwi requirements for data to be hosted onshore.

By year’s end, genomic data for five taonga species with cultural significance to Māori was securely stored and available for mediated access.

NeSI staff also began working with Genomics Aotearoa’s Vision Mātauranga Coordinator to develop a permissions workflow, which will include a review of all requests and a consent process through a kahui group of advisors.

Further development of the repository is planned for 2020, including investigation of a metadata catalogue and work towards the goal of making the repository data F.A.I.R. (Findable, Accessible, Interoperable, and Reusable) within a context of indigenous data governance.

 

Photos at right:
1. A Kākāpō on Maud Island. Chris Birmingham, Department of Conservation. 2. A North Island Kōkako. Matt Binns. 3. A juvenile Snapper. Brian Gratwicke. 4. Mānuka flowers in bloom. Wikipedia/Public Domain.