Empowering Aotearoa's research ecosystem
Each year NeSI publishes an Annual Review to celebrate the work we do,
the people we work with, and the groundwork we're laying for future directions.
Below is an excerpt from our 2022 Annual Review.
HPC and data analytics are essential tools for modern research. A growing diversity of disciplines require simulations, modelling, data processing, and data analysis. Key investments and service rollouts in 2022 enabled NeSI to modernise and continually improve the performance and fit-for-purpose design of our platforms to meet those needs.
Greener, more powerful computing resources brought online
The majority of Mahuika Extension’s new AMD 3rd Gen EPYC Milan architecture was made available for opt-in use by the end of 2022. The new CPU nodes deliver faster, more powerful computing, and specialised high-memory capabilities that allow rapid simultaneous processing. Designed with energy efficiency in mind, Mahuika Extension’s nodes are 2.5 times more power efficient than Mahuika’s original Broadwell nodes. Production use of the Milan nodes rolled out in early March 2023. As researchers move projects onto these nodes and leverage the improved performance and expanded high-memory capabilities, we look forward to seeing a wider range of communities adopt HPC approaches and build digital skills within their research teams.
Our Flexible HPC platform also came online in late 2022, following months focused on final infrastructure installation, component testing, and core service preparations for onboarding first tenants. Flexible and high performance, this cloud platform is poised to support multiple models of consumption and integration, including on-demand access to shared services. Preliminary service design work was initiated last year to bring clarity to Flexible HPC offerings relevant for Research Software Engineer (RSE) / DevOps* users, starting with a small number of internal users to pilot onboarding processes and materials. Exploration into RSE/DevOps use cases will continue into 2023.
*DevOps is the combination of cultural philosophies, practices, and tools that increases an organisation’s ability to deliver applications and services at a faster pace. See: https://aws.amazon.com/devops/what-is-devops/.
Powering data science approaches
To ensure our platforms remain responsive and high-performing to power researchers’ data-centric and data-intensive projects, we've focused on growing our comprehensive software stack. To date we host 873 environment modules (including libraries) and we're continuously curating our offerings, installing new software packages upon request. We are also updating our existing applications to ensure our users have the tools they need to power their projects. To help researchers put those software tools into practice, last year we offered practical workshops and webinars introducing researchers to machine learning and data science. These ranged from instructional (eg. Jupyter tips ’n tricks) to showcasing real-world domain applications (eg. Xarray & Dask for Geoscience).
Provisioning for efficiency and scale
Unlocked by the Flexible HPC Platform, our Platforms team has been navigating us through a shift to a continuous integration / continuous delivery approach to provisioning. This will better support our abilities to provision HPC resources for efficiency and scale. We are now capable of flexible provisioning (which can take the form of HPC or other formats, such as virtual labs or even supporting services for the clusters), through OpenStack-based virtualization and bare-metal provisioning.
Usage growing across the sector
A diverse community of organisations use NeSI’s advanced computing and data resources to power their research projects. This community includes all of New Zealand’s universities, most of its CRIs, as well as independent research organisations and private companies. The number of organisations with active allocation on NeSI platforms continued to grow in 2022, with returning use from 28 institutions and new activity from Wildlife.ai Trust, Southern Community Laboratories, Humble Bee Bio, and TamoRx.
Collaborating with technology leaders
Our strong working relationship with Hewlett Packard Enterprise (HPE) has helped NeSI ensure its shared HPC assets remain responsive and high-performing. We value having HPE systems engineers Tony Racho and John Sexton based in Wellington and working closely with the NeSI team members at NIWA, where the national supercomputing infrastructure is housed at NIWA’s Greta Point campus.
The Flexible HPC platform builds on OpenStack - an open-source cloud computing framework - as a core component. We are partnering with StackHPC, a Europe-based consultancy that specialises in applying OpenStack to HPC and advanced research computing use-cases. StackHPC have been supporting NeSI engineers with implementation and on-going support of our Flexible HPC platform.
All of our infrastructure relies on and benefits from the advanced research network provided by REANNZ. We collaborate through partnerships, communities, events, and projects.
These are just some of the many partnerships and activities that kept us busy last year.
Click here to read more from our 2022 Annual Review.
If you'd like to receive a printed copy of our review, get in touch.
