Unsupervised Discovery of Extreme Weather Events Using Universal Representations of Emergent Organization
Adam Rupe, Karthik Kashinath, Nalini Kumar, James P. Crutchfield · arXiv (Cornell University) · 2023
A record of code and data used to produce results for manuscript "Unsupervised Discovery of Extreme Weather Events Using Universal Representations of Emergent Organization" by Adam Rupe, Karthik Kashinath, Nalini Kumar, and James P. Crutchfield. Includes DisCo source code, original run scripts used on the Cori supercomputer at NERSC, LBNL, as well as conda environments with required dependencies. The local causal state segmentation fields computed on Cori are included so that figures and analyses can be reproduced without HPC resources. Contents Data netcdf_data.zipContains CAM5.1 netcdf files with core climate variables. IVT_netcdfs.zipContains netcdf files with Integrated Vapor Transport fields computed from the fields in netcdf_data.zip twodimturb.zipA netcdf file with the vorticity field from two-dimensional free-decay turbulence. reduced_npy_data.zipNumpy arrays of 4x reduced-resolution climate data (integrated vapor field and mid-column velocity components) jupyter_trim.npyNumpy ndarray of integer grayscale of interpolated RGB data from the NASA Cassini spacecraft IVT_alt-result-16.zipLocal causal state segmentation results used as ".../IVT_alt/result-16/fields/" in climate notebooks. IVT-result-8.zipLocal causal state segmentation results used as ".../IVT/result-8/fields/" in climate notebooks. TC_seg_field.npyNumpy ndarray of TECA TC segmentation output. TECA_BARD.zipNetcdf files of TECA BARD AR segmentation output. LCS_1deg_reduced-full-3.zipNumpy arrays of local causal state segmentation output for the low-resolution climate data. turb-result-15.zipNumpy arrays of local causal state segmentation output for the two-dimensional turbulence data. vortex_counts-15.npyOutput of union-find algorithm counting individual vortices from turb-results-15.zip turbulence segmentation output. Notebooks climate_figs.ipynbReproduces majority of climate related figures in the manuscript. climate_old.ipynbReproduces some older figures, some of which are used in Supplementary Information. climate-low-res.ipynbCreates the low-resolution numpy arrays from the CAM5.1 netcdf files, and also creates related figures. extreme_precipitation.ipynbPerforms extreme precipitation analysis. jupyter_figs.ipynbCreates figures for Jupiter atmosphere segmentation. TECA-compare.ipynbCreates figures from TECA segmentations. turbulence_figs.ipynbCreates turbulence figures and vortex decay analyses. Src pdisco.pyPython source code with core DisCo algorithms for distributed reconstruction of local causal states. visuals.pyPython source code for visualizing spacetime fields using matplotlib. Test-single-node turb_small.npySmall sample of turbulence data used to test the DisCo local causal state reconstruction pipeline on a single machine. single-node-turb.ipynbA full pipeline test of local causal state reconstruction that can run on single machine (e.g. a laptop). single-node-turb.pyA python script version of single-node-turb.ipynb. Scripts Collection of python and SLURM run scripts used for experiments run on the Cori supercomputer. Param-logs Logs of parameters and metadata used for various experiments run on the Cori supercomputer. The run numbers used for particular figures or analyses are indicated in the relevant notebook by the data being loaded. Conda environment files disco-deps.ymlSimple list of basic dependencies needed to run DisCo local causal state reconstruction code. disco-env.ymlFull conda environment details for running DisCo code on a single machine at time of this record's publication. disco-cori-env.ymlRecord of conda environment used on the Cori supercomputer for producing the segmentation results shown in the manuscript.