stormtracks API — stormtracks 0.3.2 documentation

### Navigation

- index
- modules |
- next |
- previous |
- stormtracks 0.3.2 documentation »

# stormtracks API

Contents

- stormtracks API
  - Running
    - `stormtracks.download` – Download Necessary Data
    - `stormtracks.run_stormtracks` – Run the Project
  - Main Modules
    - `stormtracks.analysis` – Analysis of results
    - `stormtracks.c20data` – C20 Reanalysis Data
    - `stormtracks.ibtracsdata` – IBTrACS Data
    - `stormtracks.tracking` – Tracking of Features
    - `stormtracks.tracking` – Matching of Tracks
    - `stormtracks.results` – Store and Access Results
  - Utilities
  - Running in Cluster Mode

## Running

### `stormtracks.download` – Download Necessary Data

`stormtracks.download.` `download_full_c20`(year, variables=None)

  Downloads each ensemble member’s values for prmsl, u and v

`stormtracks.download.` `download_full_c20_range`(start_year, end_year, variables=None)

  Downloads each ensemble member’s values for prmsl, u and v in a given range

`stormtracks.download.` `download_grib_c20`(year=2005, month=10, ensemble_member=56)

  Downloads the raw data for one ensemble member.

  Contains all fields for the given ensemble member (over 100 of them).

`stormtracks.download.` `download_ibtracs`()

  Downloads all IBTrACS data

  Downloads compressed tarball from FTP site to settings.DATA_DIR. Decompresses it to settings.DATA_DIR/ibtracs

`stormtracks.download.` `download_mean_c20`(year)

  Downloads mean values for prmsl, u and v

`stormtracks.download.` `download_mean_c20_range`(start_year, end_year)

  Downloads mean values for prmsl, u and v in a given range

### `stormtracks.run_stormtracks` – Run the Project

`stormtracks.run_stormtracks.` `main`(num_ensemble_members=56)

  Entry point into local running of code

  Runs through complete analysis, saves results using a results_manager.

  Parameters: | num_ensemble_members – how many ensemble members to analyse
  Returns: | results_manager that has complete set of results.

## Main Modules

### `stormtracks.analysis` – Analysis of results

class `stormtracks.analysis.` `StormtracksAnalysis`(year, is_setup_logging=False)

  Provides a variety of ways of analysing tracking performance

  Makes extensive use of its results_manager to load/save results. Used by the pyro code to farm out jobs across the cluster. To a large extent replaces the manual analysis steps. :param year: year on which to run analysis :param is_setup_logging: whether to setup logging (useful in ipython)

  `get_good_matches`(ensemble_member, config)

    Either loads or generates (and saves) good_matches

  `get_matching_configs`(**kwargs)

    Allows for easy filtering of current config options

  `get_vort_tracks_by_date`(ensemble_member, config)

    Either loads or generates (and saves) vort_tracks_by_date

  `good_matches_key`(config)

    Returns the good_matches key for the config options

  `list_stats`(ensemble_member=0, sort_on='avgdist', active_configs={})

    Runs through all statistics for the requested ensemble member and compiles stats

    sorts on the requested column and only looks at the active_configs. This makes it easy to e.g. compare all scale 1 or all 850 configuration options.

  `next_best_track`()

    Moves each plotter’s best track on by one

  `plot`()

    Uses each plotter to plot the current scene

  `prev_best_track`()

    Moves each plotter’s best track back by one

  `print_stats`(ensemble_member=0, sort_on='avgdist')

    Prints the stats

  `run_analysis`(ensemble_member, force_regen=False)

    For each set of config options, run a tracking analysis and store the results

  `run_ensemble_matches_analysis`(num_ensemble_members=56, force_regen=False)

    Looks at a particular config and runs a matching algortihm against each track

  `run_individual_analysis`(ensemble_member, config)

    Runs a given analysis based on config dict

  `run_position_analysis`(sort_on='avgdist', active_configs={}, force_regen=False, num_ensemble_members=56)

    Runs a positional analysis on the given sort_on col

    If sort_on is e.g. avg_dist, summed av dist for each of the active configs are calc’d and they are ranked in terms of which is lowest

  `run_wld_analysis`(active_configs={}, num_ensemble_members=56)

    Runs a win/lose/draw analysis on all ensemble members

    If a track from a particular analysis has a lower average dist it is said to have ‘won’ i.e. track for Wilma in pressure_level:850/scale:2/tracker:nearest neighbour has a lower av dist than pl:995/../.. .

  `set_year`(year)

    Sets the year, loading best_tracks and setting up results_manager appropriately

  `setup_analysis`()

    Sets up the current configuration options

  `setup_display`(ensemble_member, active_configs={})

    Sets up plotters for displaying of results

  `vort_tracks_by_date_key`(config)

    Returns the vort_tracks_by_date key for the config options

`stormtracks.analysis.` `analyse_ibtracs_data`(plot=True)

  Adds up a freq. distribution and a yearly total of hurricane-timesteps

`stormtracks.analysis.` `run_ensemble_analysis`(stormtracks_analysis, year, num_ensemble_members)

  Performs a full enesmble analysis on the given year

  Searches through and tries to match all tracks across ensemble members without using any best tracks info.

`stormtracks.analysis.` `run_tracking_stats_analysis`(stormtracks_analysis, year, num_ensemble_members=56)

  Runs a complete tracking analysis, comparing the performance of each configuration option

  Compares performance in a variety of ways, e.g. within pressure level or just scale 1.

### `stormtracks.c20data` – C20 Reanalysis Data

class `stormtracks.c20data.` `C20Data`(start_year, fields='all', smoothing=False, upscaling=False, verbose=True, pressure_level=850, scale_factor=2)

  Class used for accessing data from C20 Reanalysis project.

  This acts as a wrapper around netCDF4.Datasets and makes it easy to view data. Typically it exposes the psl, vort, and vort4 fields for one ensemble member. (vort4 is calculated using a 4th order vorticity calculation.) It will load these fields, along with corresponding maxima (vorticity) and minima (pressure) each time a new date is set for an object.

  Parameters: | start_year – Year from which to take data smoothing – Apply smoothing to data fields upscaling – Upscale data using cubic splines verbose – Prints lots of output pressure_level – Which pressure level (850/995 hPa) to use scale_factor – how much to scale data by

  `close_datasets`()

    Closes all open datasets

  `cvorticity`(u, v)

    Calculates the (2nd order) vorticity by calling into a c function

  `cvorticity4`(u, v)

    Calculates the (4th order) vorticity by calling into a c function

    Algorithm was taken from Walsh’s code

  `first_date`(ensemble_member=0)

    Sets date to the first date of the year (i.e. Jan the 1st)

  `fourth_order_vorticity`(u, v)

    Calculates the (4th order) vorticity using python

    Algorithm was taken from Walsh’s code

  `load_datasets`(year)

    Loads datasets for a given year

    Just sets up the NetCDF4 objects, doesn’t actually load any data apart from lons/lats and dates.

  `next_date`(ensemble_member=0)

    Moves date on by one timestep (6hr)

  `prev_date`(ensemble_member=0)

    Moves date back by one timestep (6hr)

  `set_date`(date, ensemble_member=0)

    Sets date and loads all data for that date

    Will have no effect if there is no difference in date or ensemble_member.

    Parameters: | date – date to load ensemble_member – ensemble member to load
    Returns: | date if successful, otherwise None

  `set_year`(year)

    Sets a year and loads the relevant dataset

  `vorticity`(u, v)

    Calculates the (2nd order) vorticity using python

class `stormtracks.c20data.` `GlobalEnsembleMember`(c20data, ensemble_member=0)

  Wrapper around a C20Data object

  holds state of which ensemble member is currently being analysed

  `set_date`(date)

    Sets the date, does nothing if it is the same as the stored date

  `set_year`(year)

    Change the year of the stored C20Data object

### `stormtracks.ibtracsdata` – IBTrACS Data

class `stormtracks.ibtracsdata.` `IbStormtrack`(year, name)

  Holds info about an IBTrACS best track

class `stormtracks.ibtracsdata.` `IbtracsData`(data_dir=None, verbose=True)

  Class used for accessing IBTrACS data

  Wraps the underlying NetCDF4 files and extracts the information required from them.

  Parameters: | data_dir – directory where IBTrACS NetCDF4 files are held verbose – whether to pring lots of output

  `load_ibtracks_year`(year, basin='NA')

    Loads a given year’s worth of data

    Parameters: | year – year to load basin – which basins to load (‘all’ for all of them)
    Returns: | list of loaded best tracks

  `load_wilma_katrina`()

    Loads only best tracks corresponding to Wilma and Katrina (2005)

### `stormtracks.tracking` – Tracking of Features

class `stormtracks.tracking.` `VortMax`(date, pos, vort)

  Holds key info (date, position, vorticity value) about a vorticity maximum.

  To serialize this class (or any that contain objects of this class) you must make sure next_vortmax/prev_vortmax are None.

  Parameters: | date – date of vortmax pos – position (2 element tuple) vort – vorticity value

  `add_next`(vortmax)

    Used to make doubly linked list of vortmaxes

class `stormtracks.tracking.` `VortMaxTrack`(start_vortmax, ensemble_member)

  Stores a collection of VortMax objects in a list and adds them to a dict that is accessible through a date for easy access.

class `stormtracks.tracking.` `VortmaxFinder`(gem)

  Finds all vortmaxes for a given ensemble member

  Parameters: | gem – GlobalEnsembleMember to use

  `find_vort_maxima`(start_date, end_date, use_upscaled=False)

    Runs over the date range looking for all vorticity maxima

class `stormtracks.tracking.` `VortmaxKalmanFilterTracker`(ensemble_member, Q_mult=0.001, R_mult=0.1)

  Uses a Kalman Filter to try to track vorticity maxima

  The main idea is to use the innovation parameter from the Kalman Filter estimation process as a measure of how likely it is that a vorticity maxima in a subsequent timestep belongs to the same track as a vorticity maxima in the current timestep

  `track_vort_maxima`(vortmax_time_series)

    Uses a generated list of vortmaxes to track them from one timestep to another

    Parameters: | vortmax_time_series – dict of vortmaxes
    Returns: | self.vort_tracks_by_date (OrderedDict)

class `stormtracks.tracking.` `VortmaxNearestNeighbourTracker`(ensemble_member)

  Simple nearest neighbour tracker

  Assumes that the two nearest points from one timestep to another belong to the same track.

  `track_vort_maxima`(vortmax_time_series)

    Uses a generated list of vortmaxes to track them from one timestep to another

    Parameters: | vortmax_time_series – dict of vortmaxes
    Returns: | self.vort_tracks_by_date (OrderedDict)

### `stormtracks.tracking` – Matching of Tracks

class `stormtracks.matching.` `BestTrackMatch`(best_track, store_all_tracks=False)

  Represents one match between many vorticity tracks

  `add_match`(match)

    Checks to see whether the macth should be added, then adds it

    Compares the overlap and cumalitive distance between the match to the current average track, and if they are below a limit adds the match, which involves recalculating the average track for this match, and updating some fields.

    Parameters: | match – match to be added
    Returns: | True if it was added, otherwise False

  `av_dist`()

    Returns the average distance between all vorticity tracks

class `stormtracks.matching.` `CycloneMatch`(best_track, cyclone)

  Represents one match between a best track and a vorticity track

  `av_dist`()

    Returns the average distance between the best and vorticity tracks

class `stormtracks.matching.` `EnsembleMatch`(vort_track, store_all_tracks=False)

  Represents one match between many vorticity tracks

  `add_track`(vort_track)

    Checks to see whether the track should be added, then adds it

    Compares the overlap and cumalitive distance between the track to the current average track, and if they are below a limit adds the track, which involves recalculating the average track for this match, and updating some fields.

    Parameters: | vort_track – vorticitiy track to be added
    Returns: | True if it was added, otherwise False

  `av_dist`()

    Returns the average distance between all vorticity tracks

class `stormtracks.matching.` `Match`(best_track, vort_track)

  Represents one match between a best track and a vorticity track

  `av_dist`()

    Returns the average distance between the best and vorticity tracks

`stormtracks.matching.` `combined_match`(best_tracks, all_matches)

  Uses all best tracks and matches to combine all matches for each best track

  Parameters: | best_tracks – list of best tracks all_matches – all matches to search through
  Returns: | dict of combined_matches (key: best track, value: list of matches)

`stormtracks.matching.` `good_matches`(matches, dist_cutoff=None, overlap_cutoff=6)

  Returns all matches that meet the requirements

  Requirements are that match.av_dist() < dist_cutoff and match.overlap >= overlap_cutoff

`stormtracks.matching.` `match_ensemble_vort_tracks_by_date`(vort_tracks_by_date_list)

  Takes the normal output from a tracker and flattens them to a list

`stormtracks.matching.` `match_vort_tracks_by_date_to_best_tracks`(vort_tracks_by_date, best_tracks)

  Takes all vorticity tracks and best tracks and matches them up

  Parameters: | vort_tracks_by_date – dict with dates as keys and lists of vort tracks as values best_tracks – list of best tracks
  Returns: | array of all matches

### `stormtracks.results` – Store and Access Results

exception `stormtracks.results.` `ResultNotFound`

  Simple exception thrown if result cannot be found in results manager or on disk

class `stormtracks.results.` `StormtracksNumpyResultsManager`(name)

  Super simple key/value store for numpy arrays.

class `stormtracks.results.` `StormtracksResultsManager`(name, cache_loaded=False)

  Manager class that is responsible for loading and saving all python results

  Simple key/value store. Load/saves to settings.OUTPUT_DIR. Saves each result to e.g.:

      settings.OUTPUT_DIR/<self.name>/<year>/<ensemble_member>-<key>.pkl

  (using that as a directory/filename structure).

  `add_result`(year, ensemble_member, result_key, result)

    Adds a given result based on year, ensemble_member and a user chosen result_key

  `compress_year`(year, delete=False)

    Compresses a given year’s dir and then deletes that year

  `decompress_year`(year)

    Decompresses a given year’s tarball

  `delete`(year, ensemble_member, result_key)

    Deletes a specific result from disk

  `delete_year`(year)

    Deletes a year (use with caution!)

  `get_result`(year, ensemble_member, result_key)

    Returns a saved/already loaded result if it exists, or result from disk

    throws a ResultNotFound error if it can’t find the requested result on disk.

  `list_ensemble_members`(year)

    List all results saved for a particular year

  `list_results`(year, ensemble_member)

    List all results saved for a particular year/ensemble_member

  `list_years`()

    List all saved years

  `save`()

    Saves all unsaved results that have been added so far

## Utilities

`stormtracks.utils.utils.` `compress_dir`(data_dir)

  Compresses a given data_dir, taking care to get file names correct

`stormtracks.utils.utils.` `decompress_file`(compressed_file)

  Decompresses a given tarball, taking care to get file names correct

`stormtracks.utils.utils.` `dist`(p1, p2)

  Returns the cartesian distance between two points

`stormtracks.utils.utils.` `find_extrema`(array)

  Takes an array and finds its local extrema.

  Returns an array with 0s for not an extrema, 1s for maxs and -1 for mins and a list of the indices of all maximums and minimums

  N.B. this function is much faster than the above.

`stormtracks.utils.utils.` `find_extrema_slow`(array, print_warning=True)

  Takes an array and finds its local extrema.

  Returns an array with 0s for not an extrema, 1s for maxs and -1 for mins and a list of the indices of all maximums and minimums

`stormtracks.utils.utils.` `geo_dist`(p1, p2)

  Returns the geodesic distance between two points

  p1, p2 should be of the form (lon, lat) in degrees

`stormtracks.utils.utils.` `pairwise`(iterable)

  s -> (s0,s1), (s1,s2), (s2, s3), ...

`stormtracks.utils.utils.` `raster_voronoi`(extrema, maximums, minimums)

  Takes a 2D array and points of max/mins, and returns a 2D array with the voronoi sections filled with different values

`stormtracks.utils.utils.` `upscale_field`(lons, lats, field, x_scale=2, y_scale=2, is_degrees=True)

  Takes a field defined on a sphere using lons/lats and returns an upscaled version, using cubic spline interpolation.

class `stormtracks.utils.kalman.` `KalmanFilter`(F, H)

  Implementation of static Kalman Filter

  Assumes a static model and observation operator.

  Parameters: | F – model to use to update x (np.matrix) H – observation operator (np.matrix)

  `estimate`(x_init, P_init, z, Q, R)

    Perform predict then update

    Parameters: | Q – covariance matrix for uncertainty in model R – covariance matrix for uncertainty in observation
    Returns: | best estimate for x, P

  `predict`(x_init, P_init, Q)

    Predict where x, P will be based on model

  `update`(x, P, z, R)

    Update x, P based on new observation

class `stormtracks.utils.kalman.` `Linear2DKalman`

  Linear 2D version of KalmanFilter

class `stormtracks.utils.kalman.` `RTSSmoother`(F, H)

  Implementation of Rauch-Tung-Striebel smoother

  `process_data`(zs, x, P, Q, R)

    Parameters: | zs – measurements x – initial state P – initial covariance matrix Q – process uncertainty covariance matrix R – measurement uncertainty covariance matrix
    Returns: | self.xs: all smoothed x values, self.Ps: all smoothed P values

## Running in Cluster Mode

- stormtracks Running in Cluster Mode
  - `stormtracks.pyro_cluster.pyro_nameserver` – Runs a Pyro4 Nameserver
  - `stormtracks.pyro_cluster.pyro_starter` – Launches All pyro_workers
  - `stormtracks.pyro_cluster.pyro_manager` – Manages All pyro_workers
  - `stormtracks.pyro_cluster.pyro_ender` – Stops All pyro_workers
  - `stormtracks.pyro_cluster.pyro_worker` – Processes Data
  - `stormtracks.pyro_cluster.pyro_task` – Helps Keep Track of Outstanding Tasks

### Navigation

- index
- modules |
- next |
- previous |
- stormtracks 0.3.2 documentation »