1.1.1. altimetry.data Module — altimetry tools 0 documentation

### Navigation

- index
- modules |
- next |
- previous |
- altimetry tools 0 documentation »
- 1. Description of the modules »

# 1.1.1. altimetry.data Module

This module contains special classes and tools to handle oceanographic data.

Data objects: alti_data class - handles altimetry data: hydro_data class - handles oceanographic (in-situ or remotely sensed) data:

## 1.1.1.1. Data objects:

### 1.1.1.1.1. alti_data class - handles altimetry data:

class `altimetry.data.` `alti_data`(file_pattern, time_range=None, output_is_dict=True, **kwargs)

  An `altimetry.data.hydro_data` object dedicated to handling along-track altimetry data.

  Example : | To load different sets of data, try these : Concatenate a set of AVISO’s MFSTEP NRT along-track data and plot a 2D hovmoller matrix:
      #Define parameters
      trange_str = ['24/09/2012','05/09/2013']
      trange,tdatetime=AT.cnes_convert(trange_str) #convert time range
      alti_pattern = '/path/to/nrt/mfstep/nrt_mfstep_j2_sla_vfec_*.nc'

      #Load data
      alti=alti_data(alti_pattern,verbose=verbose,datatype='DT',time_range=trange,slaext=True) #Load data

      #2D reordering of the data
      alti.reorder()

      #Plot results
      pcolormesh(data.lat,data.cycle,data.sla); show() #plot the hovmoller
  Loads a set of PISTACH L3 5Hz files and create a new SLA variable and slice the object using a given time range :
      #Load data
      alti_pattern = '/path/to/data/PISTACH_L3_Product_NWMED_MLE4_tr*_5hz.nc'
      alti=alti_data(alti_pattern,limit=limit,verbose=verbose)

      alti.create_Variable('sla',                             #new variable name
                           alti.ssh_mss_filtree_21pts,        #data
                           {'time':alti._dimensions['time']}, #set dimensions
                           extend=False)                      #extend option

      #get daily updates of the object
      for date in xrange(21300,21320):

          #get a deep copy of the object, not to erase the whole dataset
          subset=alti.copy(deep=True)

          #update the object with the proper slice
          fg=subset.slice('date', [date,date+1])
          subset.update(fg)

          do_something(subset)
  Loads a set of PISTACH hydro files :
      data=AD.alti_data('%s/*_2PTP*_*.nc' % RepData,verbose=opts.verbose,datatype='RAW',remove_existing=False)
  Load any NetCDF file using `altimetry.tools.nctools.nc` :
      data=AD.alti_data(fout,verbose=opts.verbose,datatype='RAW',transpose=False)

  `__init__`(file_pattern, time_range=None, output_is_dict=True, **kwargs)

    returns a dataset from a single file or multiple concatenated files. cf. `altimetry.data.hydro_data` for further informations

    Parameters: | time_range – get start dates from file names (cf. notes on file names when using this option) kwargs – additionnal keywords to be passed to `altimetry.data.hydro_data.__init__()`

    Note

    Naming convetion should respect AVISO formatting

    - start dates should be the 3rd field from the end
    - satellite name should be the 3rd from the start
    - eg. my_file_sat_20010101_20010108_20010109.nc
  `cycle_list`(*args)

    return the list of cycles contained if the dataset

  `pass_time`()

    Compute the central time for each passes.

    Note

    this must be called AFTER having called `altimetry.data.alti_data.reorder()` as it looks for the CYCLE and RECORD dimensions.

    Note

    The methodology to compute the central time is to interpolate the time along the track at missing points, and then reading the value at point N/2.

  `read`(filename, datatype=None, slaext=False, **kwargs)

    reader method.

    Parameters: | filename – name of the file to load. datatype – choose between DT/NRT/PISTACH/CTOH or other formats to call the corresponding reader. If datatype is : DT or NRT or PISTACH : calls `altimetry.data.alti_data.read_sla()` or `altimetry.data.alti_data.read_slaext()` CTOH : calls `altimetry.data.alti_data.read_CTOH()` else : calls `altimetry.data.alti_data.read_nc()`, based on `altimetry.tools.nctools.nc` object. slaext – force using `altimetry.data.alti_data.read_slaext()`

    Note

    This method is call from `altimetry.data.hydro_data.__init__()` and returns a data structure to be handled by `altimetry.data.hydro_data.update_dataset()`

  `read_CTOH`(filename, params=None, force=False, timerange=None, datatype=None, **kwargs)

    Read AVISO Along-Track SLA regional products

    Return outStr: | Output data structure containing all recorded parameters as specificied by NetCDF file PARAMETER list.
    Author : | Renaud Dussurget

  `read_nc`(filename, **kwargs)

    data reader based on `altimetry.tools.nctools.nc` object.

    Note

    THIS can be VERY powerful!

  `read_sla`(filename, params=None, force=False, timerange=None, datatype=None, **kwargs)

    Read AVISO Along-Track products

    Return outStr: | Output data structure containing all recorded parameters as specificied by NetCDF file PARAMETER list.
    Author : | Renaud Dussurget

  `read_slaext`(filename, params=None, force=False, timerange=None, datatype=None, **kwargs)

    Read AVISO Along-Track SLAEXT regional products

    Return outStr: | Output data structure containing all recorded parameters as specificied by NetCDF file PARAMETER list.
    Author : | Renaud Dussurget

  `reorder`(*args, **kwargs)

    Reorders data vectors in 2D (ie. with dimensions (CYCLE,RECORD)). This is useful to get a hovmoller-type matrix of each variable.

    Example : | To plot a hovmoller for a given variable, do
        .. code-block:: pyhton
    data=alti_data(‘/my/dir/my_files_pattern*.nc’) #concatenate the files data.reorder() #reorder data pcolormesh(data.lat,data.cycle,data.sla); show() #plot the hovmoller

    Note

    This only works for data reprojected along a nominal track.

  `set_sats`()

    set satellite name using (cf. notes on file names in altimetry.data.alti_data.__init__)

  `track_list`(*args)

    return the list of tracks contained if the dataset

### 1.1.1.1.2. hydro_data class - handles oceanographic (in-situ or remotely sensed) data:

class `altimetry.data.` `hydro_data`(file_pattern, limit=None, verbose=1, round=True, zero_2pi=True, output_is_dict=True, flatten=False, **kwargs)

  A base object dedicated to handle oceanographic data (in-situ or remote sensing) with upper level processing methods.

  Note

  This object SHOULD NOT be called directly but through a subclass heritating of it (eg. `altimetry.data.alti_data`)

  `Error`(ErrorMsg)

    raises an exception

  `__init__`(file_pattern, limit=None, verbose=1, round=True, zero_2pi=True, output_is_dict=True, flatten=False, **kwargs)

    Returns the object filled with the data loaded from a single file or a concatenated set of files

    Parameters: | file_pattern – a pattern of files to be globbed (`glob.glob()`) or a list of file names. limit – the limits of the domain to handle ([latmin,lonmin,latmax,lonmax]). verbose – verbosity level on a scale of 0 (silent) to 4 (max verobsity) round – round limits (cf. `altimetry.tools.in_limits()`) zero_2pi – limits goes from 0 to 360 degrees (not -180/180). output_is_dict – data structures are dictionnaries (eg. my_hydro_data.variable[‘data’]). If false uses an object with attributes (eg. my_hydro_data.variable.data).

    Note

    This methodes init all the attributes, then loads the data from files (`altimetry.data.hydro_data.read()`) and appends it to the object (`altimetry.data.hydro_data.update_dataset()`) before checking its content (`altimetry.data.hydro_data.check_variables()`).

    Note

    The method `altimetry.data.hydro_data.read()` MUST be defined (typically by overloading it). This method must return a data structure.

  `__weakref__`

    list of weak references to the object (if defined)

  `check_variables`()

    Forces variables to respect dimensions

  `contour_transect`(x, z, var, xrange=None, zrange=None, xstep=1, zstep=10, vmin=None, vmax=None, marker='.k', **kwargs)

    shows a 2d space-depth section by interpolating the data along the section.

    Note

    This method interpolates using `scipy.interpolate.griddata()` and plots using `matplotlib.pyplot.meshcolorgrid()`

  `copy`(*args, **kwargs)

    Returns a copy of the current data object

    Parameters: | flag – if an argument is provided, this returns an updated copy of current object (ie. equivalent to obj.copy();obj.update(flag)), optimising the memory ( deep (True) – deep copies the object (object data will be copied as well).

  `count` = None

    number of files loaded

  `create_Dim`(name, value)

    Adds a dimension to class.

    Parameters: | name – dimension name value – dimension value

  `create_Variable`(name, value, dimensions, toCreate=None, createDim=None, extend=True)

    create_Variable : This function adds data to `altimetry.data.hydro_data`

    Parameters: | name – name of the parameter to create value – values associated to the variable. Must be a numpy masked_array or a data structure. dimensions – dimensional structure (cf. notes).

    Note

    altimetry tools package handles the NetCDF data using specific structures.

    NetCDF data is structured this way:

        NetCDF_data = {'_dimensions':dimension_structure,  #File dimensions  (COMPULSORY)
                       '_attributes':attribute_structure,  #Global attributes
                       'dimension_1':data_structure,       #Data associated to the dimensions. (COMPULSORY)
                       ...,
                       'variable_1':data_structure,        #Variables
                       ...
                      }

    In standard NetCDF files, dimensions are always associated to a variable. If it is not the case, an array of indices the length of the dimension is generated and a warning is issued.

    Moreover, dimensions MUST be defined to be accepted by `altimetry.tools.nctools.nc` (empty NetCDF files would fail).

    - a dimensional structure should be of the form :

        dimension_structure = {'_ndims':N,        #Attribute setting the number of dimensions.

                               'dims':{'dim_A':A, #Structure containing the name
                                       'dim_B':B, #of the dimensions and their size.
                                       ...,
                                       'dim_N':N
                                      }
                              }

    - an attribute structure is a very simple structure containing the attribute names and values:

        data_structure = {'attribute_1':attribute_1,
                          ...,
                          'attribute_N':attribute_N}

    - a data structure should be of the form :

        data_structure = {'_dimensions':dimension_structure, #dimensions of hte variable (COMPULSORY)
                          'data':data,                       #data associated to the variable (COMPULSORY)

                          'long_name':long_name,             #Variable attributes
                          'units':units,
                          ...
                         }

    DATA and _DIMENSIONS fields are compulsory. Other fields are optional and will be treated as attributes.

    Furthermore, code will have a special look at scale, scale_factor and add_offset while reading and writing data and to _FillValue and missing_value while reading (_FillValue being automatically filled by `NetCDF4.Dataset` when writing)

  `delete_Variable`(name)

    pops a variable from class and delete it from parameter list

    Parameters: | name – name of the parameter to delete

  `dim_list` = None

    array containing the dimensions of each parameter

  `dirname` = None

    Directory name of the file pattern being globbed (`glob.glob()`). Defaulted to current directory

  `extension`(flag=None, round=True)

    returns the limits of the dataset.

    Parameters: | flag – an indexation flag array round – round the limits to the south-west and north-east.

  `fileid` = None

    array of file IDs

  `filelist` = None

    list of files being loaded

  `filelist_count` = None

    number of counted values by files

  `get`(name)

    retunrs a variable

  `get_currentDim`()

    returns the current dimensions of the object

  `get_file`(pattern)

    returns a flag array of the data loaded from a given file pattern

    Parameters: | pattern – pattern to match in the file list.

  `get_object_stats`()

    get some statistics about the whole dataset.

  `get_platform_stats`(id)

    get statistics based on altimetry.data.hydro_data.id

  `get_stats`(flag)

    get some statistics about a part of the dataset

  `in_limits`(limit=None)

    wrapper to `altimetry.tools.in_limits()` based on dataset limits.

  `limit` = None

    limits of the domain : [latmin,lonmin,latmax,lonmax] (default = [-90.,0.,90.,360.])/

    Note

    limits are automatically reset using `altimetry.tools.recale_limits()`

  `map`(flag=None, fname=None, zoom=False, pmap=None, show=True, **kwargs)

    display (or not) a map based on a `altimetry.tools.plot_map` object.

    Parameters: | show – set to False not to show (and neither apply `altimetry.tools.plot_map.setup_map()`)

    Note

    This function creates a `altimetry.tools.plot_map` instance, plot a partion of the dataset using `altimetry.data.hydro_data.plot_track()` and displays it if asked to.

  `message`(MSG_LEVEL, str)

    print function wrapper. Print a message depending on the verbose level

    Parameters: | MSG_LEVEL ({in}{required}{type=int}) – level of the message to be compared with self.verbose
    Example : | To write a message
        self.message(0,'This message will be shown for any verbose level')

  `ncstruct`(**kwargs)

    returns a data structure (dict) of the dataset.

    Parameters: | params – Add this keyword to provide a list of variables to export. Default : all variables contained is self.par_list

  `par_list` = None

    array of parameters

  `platform_summary`(id, col='.k')

    outputs a summary of the statistics for a given platform

  `plot_track`(pmap, flag=None, col='.k', endpoint='*r', endpoint_size=None, title=None, fontsize=8, textcolor='b', ms=5, linewidth=1, **kwargs)

    plot trajectories based on platform IDs

    Parameters: | pmap – a `altimetry.tools.plot_map` instance col – color to be used along the trajectory. If this is an array of values, calls `altimetry.tools.plot_map.scatter()` instead of `altimetry.tools.plot_map.plot()`

    Note

    This method loops on data IDs. Then it calls `altimetry.tools.plot_map.plot()` or `altimetry.tools.plot_map.scatter()` to plot the trajectory and then labels the trajectory using `altimetry.tools.plot_map.text()`

  `plot_track_old`(*args, **kwargs)

    plot a surface map of sampling track

    Warning

    DEPRECATED method!

  `plot_transect`(x, z, var, xrange=None, zrange=None, vmin=None, vmax=None, xstep=1, zstep=10, s=10, edgecolor='none', **kwargs)

    shows a 2d space-depth section plotting point (using `altimetry.tools.plot_map.scatter()`)

    Example : | plot a temperature section along a glider transect

  `pop`(*args, **kwargs)

    This is a wrapper to `altimetry.data.hydro_data.delete_Variable()`

  `push_nc`(*args, **kwargs)

    append a data structure to an exisiting netcdf file

  `read_ArgoNC`(filename, params=None, force=False, dephrange=None, timerange=None, **kwargs)

    An Argo network NetCDF reader

    Return outStr: | Output data stricture (dict) containing all recorded parameters as specificied by NetCDF file PARAMETER list.
    Author : | Renaud Dussurget

  `size` = None

    length of the dataset

  `slice`(param, range, surf=False)

    get a flag for indexing based on values (ange of fixed values).

    Parameters: | param – variable name range – numpy array defining the range of the values. If size(range) == 2 : flag is computed between min and max values of range flag is computed based on equality to range value.

  `summary`(all=False, fig=None, col='.k', legend=None, **kwargs)

    outputs a summary of the whole current dataset

  `time_range`(flag=None)

    time range of the current dataset

    Parameters: | flag – use a flag array to know the time range of an indexed slice of the object

  `time_slice`(timerange, surf=False)

    slice object given a time range

    Parameters: | timerange – rime range to be used.

  `update`(*args, **kwargs)

    Wrapper to `altimetry.data.hydro_data.update_with_slice()`.

  `update_Dim`(name, value)

    update a dimension by appending the number of added elements to the dimensions

        <upddated dimension> = <old dimension> + <number of added elements along this dimension>

  `update_dataset`(dataStr, flatten=False)

    update class with a data structure.

    Parameters: | flatten – use this to automatically flatten variables (squeeze dimensions)

  `update_fid_list`(filename, N)

    update file indices attribute altimetry.data.hydro_data.fileid

  `update_with_slice`(flag)

    update object with a given time slice flag

    Parameters: | array) flag ((boolean) – a flag for indexing data along the ‘’time’’ dimension

  `updated_copy`(flag, deep=True)

    Returns a sliced (updated) copy of current data object

    Summary : | This has the same effect as obj.copy();obj.update(flag) but is much less memory consumming.

    Note

    TypeError could arise if some object attributes are setted outside the `__init__()` function (eg. for data objects derived from `hydro_data`). If this is the case, initialise these attributes within their respective `__init__()`.

  `verbose` = None

    verbosity level on a scale of 0 (silent) to 4 (max verbosity)

  `warning`(MSG_LEVEL, str)

    Wrapper to `warning.warn()`. Returns a warning when verbose level is not 0.

    Parameters: | MSG_LEVEL – level of the message to be compared with self.verbose
    Example : | To issued a warning
        self.warning(1,'Warning being issued)

  `write_nc`(filename, clobber=False, **kwargs)

    write a NetCDF file from current dataset

    Parameters: | kwargs – additional arguments are passed to `altimetry.tools.nctools.nc.write()`

### Navigation

- index
- modules |
- next |
- previous |
- altimetry tools 0 documentation »
- 1. Description of the modules »