Crashes and quakes at NZ Research Software Engineering Hackathon 2020

The Challenge:
A disrupted year meant software engineers across New Zealand couldn’t get together to share knowledge. How could NeSI help keep NZ’s software development community globally competitive?

The Solution:
The creation of the NZ Research Software Engineering (RSE) Hackathon 2020. Engineers shared ideas over Slack and engaged with a real-world problem in earthquake analysis, all while honing their skills.

The Outcome:
Innovation in predicting core hour usage for earthquake simulations. A thriving local community of engineers took part in the exchange of ideas and a new project that could save lives on NZ’s roads.

COVID-19 presented a unique challenge to researchers in 2020. Lockdown closed many traditional methods of collaboration and learning.

In response, NeSI found a way to counter this isolation, partnering with QuakeCoRE, the New Zealand Centre for Earthquake Resilience, to host a Hackathon as a special lead-up event to its inaugural New Zealand Research Software Engineering Conference.

“We wanted people to engage with each other, so we came up with this challenge. It was more exciting than just throwing people in a room together to talk about machine learning. Community-building was important to us,” said NeSI Research Communities Advisor, Megan Guidry.

The Hackathon brought together software experts to solve QuakeCoRE’s challenge. The problem? Write a program to improve core hour estimations for QuakeCoRE ground quake simulations.

“The driving force wasn’t so much the problem itself, but the need to share expertise across New Zealand. We wanted to elevate the software community’s skills and our own,” said NeSI Scientific Programmer, Alexander Pletzer.

While the main goal was keeping New Zealand’s research community working together while remote, the QuakeCoRE’s problem was a serious one.

Core hours are a measure of one computer processor running a simulation for one hour. NeSI’s Mahuika cluster runs jobs on 8,136 cores, while Maūi has 18,560 cores.

Simulations use thousands of core hours, but when researchers underestimate the number, they may lose the data.

A program that has used up its allocated core hours will shut down, even if it’s only half-done.

But if QuakeCoRE researchers overestimate their jobs too much, they can waste core hours.

Most ground quake simulations are so complex, researchers need to use every core hour to its fullest.

“QuakeCoRE provided a good data set and a clear question about what they wanted to achieve. They wanted to predict the total number of core hours needed for a simulation. They also needed an upper [time] bound for individual jobs,” said NeSI Data Science Engineer, Maxime Rio.

This meant the winning code would predict the average time for simulations and have a buffer for large simulations.

The participants received training data for their program. This data showed QuakeCoRE time predictions on past jobs, and the real time those jobs took. The data included an equation called a quantile loss function which would score the results.

Hackathon winner, Nick Young discussed the way he approached this problem:

“Through trial and error, I found a way to plot the true core hours against the predictions in the training data. I took the range between the true core hours and the time 92 per cent of predictions had guessed, to improve predictions.”

NeSI created a Slack channel for all participants. There, participants could share ideas, troubleshoot problems and point out errors in the training data.

It was this Slack channel where Nick found advice to improve his code. The community spotted strange time figures in the data that were actually failed jobs. Nick took out these figures to improve his code.

“Not everyone created a winning solution, but people got benefits from just being involved in the Slack channel. The spirit of the event was to explore and try something new. It wasn’t really about the best solution,” said Megan.

As part of the prize, Nick won 40 hours of NeSI consultancy on a project of his choice. Nick put his winnings towards trying to predict car crashes around New Zealand:

“The NZ Transport Authority Crash Analysis System data shows data on all crashes since 2000. I want to use that data set to predict where crashes are likely to occur.”

Upgrading roads and placing warning signs in these crash hotspots could save lives. Nick and Maxime are now working together to find these hotspots.

The NeSI Hackathon succeeded in bringing the NZ software engineering community together. It turned a real quake simulation problem into a co-operative problem-solving experiment. The winnings are now going towards making New Zealand roads safer.

Do you have an example of how NeSI support or platforms have supported your work? We’re always looking for projects to feature as a case study. Get in touch by emailing support@nesi.org.nz.