Massive data transfer sets record on REANNZ

A record level of data travelled over the REANNZ network last month as NeSI began moving the first wave of its users’ research data to its new platforms.

Tens of millions of files and hundreds of terabytes of data are being migrated between NeSI’s Pan cluster at the University of Auckland and its new Mahuika supercomputer at NIWA’s High Performance Computing Facility (HPCF) in Wellington.

Over the last few weeks, NeSI support staff have been copying and synchronising the research data on Pan to move it over to the new platform’s storage facilities at the HPCF.

This initial migration step has been progressing smoothly, which is an important milestone for getting users set up on the new platform. Using REANNZ was essential for supporting this massive transfer, as its high capacity, high-speed network has been able to handle the loads with little to no disruption to its regular operations.

REANNZ Chief Executive Officer Nicole Ferguson and Chief Operating Officer Ange Nash took to Twitter to celebrate the record-setting transfer rates. The red areas of the network “weather map” in the tweet below indicate traffic load levels of 60-100%.

A glimpse of the network activity during that week can be seen via REANNZ’s archived weather map animations. For example, the snapshot of 30 May 2018 shows legs of the network between Auckland (AKL) and Wellington (WLG) flickering from green to orange to red, showing the load peaks and lulls throughout the day:
https://weathermap.reannz.co.nz/animations/reannz-2018-05-30.gif

User data will continue to be moved between Auckland and Wellington over these next few weeks as the new clusters undergo final configuration and testing. Once complete, NeSI users will be able to log into the new platform for the first time, where their directories and files will be waiting for them.