Enhancing understanding and improving prediction of severe weather through spatiotemporal relational learning

Abstract

Severe weather, including tornadoes, thunderstorms, wind, and hail annually cause significant loss of life and property. We are developing spatiotemporal machine learning techniques that will enable meteorologists to improve the prediction of these events by improving their understanding of the fundamental causes of the phenomena and by building skillful empirical predictive models. In this paper, we present significant enhancements of our Spatiotemporal Relational Probability Trees that enable autonomous discovery of spatiotemporal relationships as well as learning with arbitrary shapes. We focus our evaluation on two real-world case studies using our technique: predicting tornadoes in Oklahoma and predicting aircraft turbulence in the United States. We also discuss how to evaluate success for a machine learning algorithm in the severe weather domain, which will enable new methods such as ours to transfer from research to operations, provide a set of lessons learned for embedded machine learning applications, and discuss how to field our technique.

Paper

McGovern, Amy and Gagne II, David J. and Williams, John K. and Brown, Rodger A. and Basara, Jeffrey B. (2014) Enhancing understanding and improving prediction of severe weather through spatiotemporal relational learning. Machine Learning. Vol 95, Issue 1, pages 27-50.

Note: we have updated the code significantly with improved documentation as of July 2014. If you downloaded it before then, please re-download.

Code release

• The SRPT/SRRF code is available as a gzipped tar file (updated July 2014)
  • The older code (only if it is needed)
• The tornadogenesis data is split into two:
  • The raw XML data (compressed to 118M)
  • The schema data needed to load the XML into SRPT/SRRF
  • If your browser has trouble with the XML, the schema is here in zip format
• The turbulence data is split into several parts:
  • The static XML data (compressed)
    • Training data (722K)
    • Testing data (285K)
• The dynamic XML data (compressed)
  • Training data(44M)
  • Test data (12M)
• The schema data needed to load the XML into SRPT/SRRF
• If your browser has trouble with the XML, the schema is here in zip format
• Some of our data sources:
  • The Oklahoma surface observations used in the tornadogenesis data came from the Oklahoma Mesonet
  • The North American Regional Reanalysis (NARR) provided upper air data for the tornadogenesis data
  • The tornado reports came from the Storm Prediction Center's (SPC) tornado reports
  • The turbulence data comes from the National Center for Atmospheric Research (NCAR)