| Project challenge | Researcher(s) | Organisation | Consultancy solution |
| New Zealand’s Department of Conservation (DOC) uses a specialised software called Zonation to help its scientists make decisions around conservation strategies and allocation of conservation resources. As part of DOC’s migration onto NeSI’s new platforms, the Zonation software needed to be rebuilt and optimised for running on Mahuika. | Amy Hawcroft | Department of Conservation | Zonation was successfully deployed on Mahuika, following numerous investigations into rebuild options, and a reconfiguration of the code to enable it to run successfully on Mahuika. DOC can now continue to use Zonation as a tool to assist the Department in managing resources and priorities around ecological zone and habitat protection in New Zealand. |
| GNS Science researchers are applying state-of-the-art seismological techniques and numerical modelling to shed light on the underlying mechanism of complex mega-thrust slip behaviour at subduction plate boundaries. Using existing and new seismic datasets, they are developing a dynamical model of crustal deformation that can reproduce the spectrum of fault slip behaviour at the Hikurangi subduction zone. | Yoshihiro Kaneko | GNS Science | NeSI team members helped build the numerical tools required to simulate seismic wave propagation for the Hikurangi subduction zone, and optimised them to run in the most efficient manner possible. This solution has enabled the researchers to reduce their core-hour consumption by 100,000, saving both operating and energy costs – a win-win for both the researchers and NeSI. |
| Researchers at Manaaki Whenua - Landcare Research are developing a processing pipeline for digital soil maps to provide consistency between mapping projects. However, when researchers attempted to run their workflow on Mahuika, it behaved unexpectedly, taking longer and more CPU power than it should. | James Shepherd | Manaaki Whenua - Landcare Research | An audit of the code was conducted, and assistance was provided with additional code development, to improve the workflow’s efficiency and performance on Mahuika. |
| Manaaki Whenua - Landcare Research’s map services (e.g. those available via http://ourenvironment.scinfo.org.nz ) depend on map tile images stored in Berkeley databases, which are populated via Mapserver's Mapcache software. With the increasing number of map layers being served up, the advent of HDPI tiles, and higher-resolution data becoming available, the creation time of these services was becoming too long. | Michael Speth | Manaaki Whenua - Landcare Research | Troubleshooting the Map Cache Builder workflow on NeSI platforms identified opportunities to reduce runtime and improve time to solution. Further troubleshooting and testing of containerisation approaches led to an improved workflow implemented in a Singularity container. Also, significant speedups were achieved by enabling parallel execution in some cases. Overall, the recommendations and solutions provided were successful in helping reduce the map cache building times. |
| Ecological research such as species distribution modelling requires national scale maps of environmental variables. Often these data must be interpolated from point samples of environmental variables. Natural neighbour interpolation provides a potentially useful technique as it is an exact interpolator, it creates a smooth surface free of any discontinuities, it is a local method, and is spatially adaptive. However, efficient and open code does not exist to facilitate the method’s use. | Tom Etherington | Manaaki Whenua - Landcare Research | Consultancy team members began by profiling the code to better understand where time is being spent, and investigated options for improving performance. The code was optimised and can now run over the full domain in a reasonable time, enabling more reliable environmental surfaces to be produced that can then form the basis of future ecological modelling. |
| Massey university researchers are developing a fast, parallelisable Julia code for ultra-cold physics studies. However, recent runs on NeSI showed quite low CPU utilisation and so they were keen to investigate ways to improve their code’s performance, as well as learn more about profiling tools. | Joachim Brand
Ray Yang
Peter Jeszenszki | Massey University | Consultancy team members investigated tools for profiling parallel Julia code on the NeSI platform and shared with the researchers to use for future code development activities. Overall, upskilling around profiling and parallel code development will empower the researchers to make significant scientific contributions in the field of ultra-cold atom quantum many-body physics. |
| NeSI Consultancy team members are part of an international collaboration of scientists and programmers, led by the UK Met Office, to develop LFRic, a next generation weather and climate simulation code to replace and improve upon the Unified Model. Although highly successful, the UM struggles to scale to the resolution required to model convective processes at the heart of cloud formation. | | NIWA | Consultancy team members wrote code to read cubed-sphere data into Paraview and provided an in-situ visualisation solution that allows researchers to plot their data as the simulation proceeds. Also, a new interpolation library (MINT) was developed that preserves the mathematical properties of vector fields in LFRic. The insights from this project are critical as communities, governments, and industry strive to respond to and prepare for future changes in climate. |
| Floods and inundation can be a major threat to communities and infrastructure. Most inundation models resolve all simulated areas with the same resolution, slowing down the program’s run-time, or rely on an engineer on-hand to work on each simulation. NIWA researchers are developing a new model to address these challenges, but needed more effective methods for testing and developing their solution. | Cyprien Bosserelle
Emily Lane
Richard Gorman | NIWA | Visualisation tools were employed to enable faster testing of the researchers’ model, as well as speed up debugging and produce results that are easy to understand in a timely manner. With an adaptive grid and an ability to run smaller-scale simulations on general purpose graphics processing units (GPUs), this model can be used as a building block for future forecasting tools, better weather analysis, and hazard management for emergency services and local councils. |
| Data on New Zealand marine mammal populations and their movements are difficult to obtain, and protecting these species from human activity is difficult without understanding their numbers and behaviour. However, detection and classification of underwater sounds from different sources is a challenging task. Passive acoustics system record large amount of data, and detecting and classifying sounds of interest using standard signal processing techniques request intensive and costly work. | Giacomo Giorli | NIWA | Consultancy team members trained, tested, and assessed the performance of machine learning methods to classify three different sounds produced by echolocating beaked whales. The results and insights gained from this Consultancy may become the basis for a more comprehensive neural network approach applied to underwater surveillance and monitoring. There are also potential commercial applications in the field of marine monitoring. |
| The NZNCM (New Zealand Nested Climate Model) is a tightly coupled global atmosphere-ocean model with an additional nested high resolution regional ocean component around New Zealand. The NZNCM uses the OASIS coupling library for transferring boundary fluxes between the model components. The nested high resolution ocean model receives boundary data using a debug file functionality in OASIS. However, OASIS appends fields to those written to the debug file previously. This made it difficult to read these fields into the ocean component and wastes space. | Erik Behrens
Jonny Williams | NIWA | Consultancy team members carried out important modifications to OASIS to enhance its functionality within the NZESM. The new output option implemented through this Consultancy should simplify correct coupling of the nested ocean model, reducing the chance that inconsistent boundary data is used, and therefore lowering the chance of erroneous results. This enhanced functionality will contribute to an improved Earth System Model to map climate change, allowing government and industry to better plan for New Zealand's future climate. |
| For the past 10 years, James Sneyed has been working on the construction of a multiscale model of saliva secretion, in collaboration with a range of experimentalists in the USA, NZ, Germany, Japan and Australia. He had a working multiscale model of saliva secretion in three spatial dimensions, with the code for that model run in Matlab. The most recent and near-final version of this model, that incorporates intercellular diffusion, needed to be ported onto NeSI’s HPC platform in order to run model tests and simulations that will be critical for further advancement of this project and research. | James Sneyd
Nathan Pages
Elias Siguenza | University of Auckland | Consultancy team members worked with the project’s primary coder to understand the current C++ code, plan a new implementation of the code, and port Matlab additions to the C++ code. From this Consultancy work, the researchers will be able to build a suite of new simulations showing the effects of intercellular diffusion on the dynamics of calcium waves in a parotid acinus. Insights from these simulations are expected to lead to at least one publication. |
| By analysing the brain at different activity states, researchers at the University of Auckland are hoping to find differences in the electroencephalogram (EEG) brain signals between those with Autism Spectrum Disorder and those without. However, the fractal analysis needed for this approach requires vast amounts of computing power – potentially taking weeks or months. | Stephen S Wolfson | University of Auckland | Consultancy team members audited the code to understand where most of the execution time is spent and identify factors affecting performance and memory footprint. Assistance provided through this Consultancy has helped the researchers optimise their code, learn about code profiling, and improve their workflow efficiency to get results faster. |
| Simulating the hundreds of thousands of atomic collisions that occur as a nanowire heats up requires processing power well beyond standard computers. This also means the post-processing stage can take quite some time. While University of Auckland researcher Kannan Ridings had devised a scheme to automate that entire process, he needed assistance with implementing his workflow on Maui. | Kannan Ridings
Shaun Hendy | University of Auckland | Consultancy team members installed the LAMMPS software being used to run the molecular dynamics simulations for this project. They also helped Kannan optimise and deploy his scripts to automate the running of the simulations as well as the post-processing stage, saving dozens of hours of work. Kannan’s work in this area is pioneering in the field and because of the wide-ranging applications of nanotechnology, his work has the potential to affect many other different disciplines as well. |
| Researchers at the University of Auckland are exploring the turbulence patterns of flow over a river bed with ripples, which are sedimentary rhythmic bedforms that develop in sandy environments. Their appearance and development are very complex, and their evolution is still unclear. Before working with NeSI, they could only visualise 2D slices in the 3D data. They wanted to be able to see vortices in 3D as well, particularly along the flow of water. | Chuang Jin
Giovanni Coco | University of Auckland | Consultancy team members helped the researchers develop capability for visualising 3D flow from their HPC simulation and experiment using ParaView visualisation software. As a result, they were able to produce examples of visualisations of "vorticity" vector fields in 3D using the flow data and simulation output. Insights from this work will help improve the predictions of ripple evolution, contributing to a research collaboration with John-Hopkins University in the United States. |
| A significant swath of Canada's western Arctic, as well as parts of eastern Russia are underlain by massive tabular ice, thought to be a remnant of past glaciations. As temperatures warm and rainfall rates increase, the occurrence of erosion has become increasingly common. Researchers had assembled the essential numerical components to build a more accurate landscape evolution model to simulate these changes, however their code took many days to run a simulation of a few decades over a 500x500 grid. In order to push the model out to the century scale, over a 500k x 500k grid, they needed assistance with optimising their code and parallelisation of the core routines. | Jon Tunnicliffe
Steve Kokelj | University of Auckland & collaboration with NWT Geoscience Office, Canada | Many changes were made to the structure of the code to improve robustness, including taking a more object oriented approach, adding a new build system, documentation, testing framework, continuous integration and automatic deploys of releases using online services such as Appveyor and Travis-CI. Future development of the code will be much easier as a result of these changes (and safer too). Optimisations were also carried out in key algorithms and parallel code introduced where possible. These steps lay the foundation for continuing the development of the code to enable simulation of high resolution grids. |
| OpenSees is an open source, community finite element code that simulates the response of buildings to ground shaking. Researchers are currently applying OpenSees to 50x50x50 resolution problems, but would need to increase the resolution to 500x500x50 in order to avoid boundary conditions effects. It was estimated that this would take much longer to run the high resolution case as well as be significantly demanding in memory, which ran the risk of causing the jobs to sit longer in the HPC queue. | Chris McGann | University of Canterbury | Consultancy team members profiled the code to better understand the existing code’s execution time, and see how different builds affected execution performance. Outcomes from the Consultancy included a lift in the researchers’ capabilities to profile and optimise their code. Other NeSI users of OpenSees will also benefit from the resulting optimised version of OpenSees, which has the potential to improve further high impact science aligned with key New Zealand priorities. |
| University of Waikato researcher Alexis Marshall is researching the microbial ecology of marine sediment, analysing its genetic makeup to identify the unique species that make the sediment their home. To do this, she was using the RNA transcriptome reconstructor, Trinity, on Mahuika but her jobs were taking a long time and using a lot of memory. | Alexis Marshall
Maria Rovisco Monteiro | University of Waikato | Consultancy team members investigated parallelisation options for improving Trinity’s efficiency, as well as set up Slurm scripts to automate running the different steps of Trinity as different Slurm jobs, as the different steps have different resource requirements. Their findings and methods were documented in NeSI Support so other NeSI Trinity users could also take advantage of these learnings. Alexis can now can get an assembled data set to ask questions in 48 hours instead of three weeks, and expects findings from this work to be published in early 2020. |
Attribution: Connecting with research communities
