cis_interface package — cis_interface 0.3.0 documentation

# cis_interface package

## Subpackages

- cis_interface.communication package
  - Submodules
  - cis_interface.communication.AsciiFileComm module
  - cis_interface.communication.AsciiMapComm module
  - cis_interface.communication.AsciiTableComm module
  - cis_interface.communication.AsyncComm module
  - cis_interface.communication.ClientComm module
  - cis_interface.communication.CommBase module
  - cis_interface.communication.ErrorComm module
  - cis_interface.communication.FileComm module
  - cis_interface.communication.ForkComm module
  - cis_interface.communication.IPCComm module
  - cis_interface.communication.ObjFileComm module
  - cis_interface.communication.PandasFileComm module
  - cis_interface.communication.PickleFileComm module
  - cis_interface.communication.PlyFileComm module
  - cis_interface.communication.RMQAsyncComm module
  - cis_interface.communication.RMQComm module
  - cis_interface.communication.RPCComm module
  - cis_interface.communication.ServerComm module
  - cis_interface.communication.ZMQComm module
  - Module contents
- cis_interface.dataio package
  - Submodules
  - cis_interface.dataio.AsciiFile module
  - cis_interface.dataio.AsciiTable module
  - Module contents
- cis_interface.drivers package
  - Submodules
  - cis_interface.drivers.AsciiFileInputDriver module
  - cis_interface.drivers.AsciiFileOutputDriver module
  - cis_interface.drivers.AsciiMapInputDriver module
  - cis_interface.drivers.AsciiMapOutputDriver module
  - cis_interface.drivers.AsciiTableInputDriver module
  - cis_interface.drivers.AsciiTableOutputDriver module
  - cis_interface.drivers.CMakeModelDriver module
  - cis_interface.drivers.ClientDriver module
  - cis_interface.drivers.ClientRequestDriver module
  - cis_interface.drivers.ClientResponseDriver module
  - cis_interface.drivers.CommDriver module
  - cis_interface.drivers.ConnectionDriver module
  - cis_interface.drivers.Driver module
  - cis_interface.drivers.FileInputDriver module
  - cis_interface.drivers.FileOutputDriver module
  - cis_interface.drivers.GCCModelDriver module
  - cis_interface.drivers.IODriver module
  - cis_interface.drivers.IPCCommDriver module
  - cis_interface.drivers.InputCommDriver module
  - cis_interface.drivers.InputDriver module
  - cis_interface.drivers.LPyModelDriver module
  - cis_interface.drivers.MakeModelDriver module
  - cis_interface.drivers.MatInputDriver module
  - cis_interface.drivers.MatOutputDriver module
  - cis_interface.drivers.MatlabModelDriver module
  - cis_interface.drivers.ModelDriver module
  - cis_interface.drivers.ObjFileInputDriver module
  - cis_interface.drivers.ObjFileOutputDriver module
  - cis_interface.drivers.OutputCommDriver module
  - cis_interface.drivers.OutputDriver module
  - cis_interface.drivers.PandasFileInputDriver module
  - cis_interface.drivers.PandasFileOutputDriver module
  - cis_interface.drivers.PickleFileInputDriver module
  - cis_interface.drivers.PickleFileOutputDriver module
  - cis_interface.drivers.PlyFileInputDriver module
  - cis_interface.drivers.PlyFileOutputDriver module
  - cis_interface.drivers.PythonModelDriver module
  - cis_interface.drivers.RMQAsyncClientDriver module
  - cis_interface.drivers.RMQAsyncCommDriver module
  - cis_interface.drivers.RMQAsyncServerDriver module
  - cis_interface.drivers.RMQClientDriver module
  - cis_interface.drivers.RMQCommDriver module
  - cis_interface.drivers.RMQInputDriver module
  - cis_interface.drivers.RMQOutputDriver module
  - cis_interface.drivers.RMQServerDriver module
  - cis_interface.drivers.RPCCommDriver module
  - cis_interface.drivers.RPCDriver module
  - cis_interface.drivers.ServerDriver module
  - cis_interface.drivers.ServerRequestDriver module
  - cis_interface.drivers.ServerResponseDriver module
  - cis_interface.drivers.ZMQCommDriver module
  - cis_interface.drivers.lpy_model module
  - Module contents
- cis_interface.examples package
  - Module contents
- cis_interface.interface package
  - Submodules
  - cis_interface.interface.CisInterface module
  - cis_interface.interface.PsiInterface module
  - Module contents
- cis_interface.serialize package
  - Submodules
  - cis_interface.serialize.AsciiMapSerialize module
  - cis_interface.serialize.AsciiTableSerialize module
  - cis_interface.serialize.DefaultSerialize module
  - cis_interface.serialize.MatSerialize module
  - cis_interface.serialize.ObjSerialize module
  - cis_interface.serialize.PandasSerialize module
  - cis_interface.serialize.PickleSerialize module
  - cis_interface.serialize.PlySerialize module
  - Module contents

## Submodules

## cis_interface.backwards module

This module allows for backward compatibility.

`cis_interface.backwards.` `bytes2unicode`(b)

  Convert from bytes/unicode/str to unicode.

  Parameters: | b (bytes, unicode, str) – Bytes to be converted into unicode.
  Returns: | String version of input.
  Return type: | unicode/str
  Raises: | `TypeError` – If supplied type cannot be converted to unicode.

`cis_interface.backwards.` `unicode2bytes`(s)

  Convert from bytes/unicode/str to a bytes object.

  Parameters: | s (str, bytes, unicode) – Object to convert to a bytes version.
  Returns: | Bytes version of input.
  Return type: | bytes
  Raises: | `TypeError` – If supplied type cannot be converted to bytes.

## cis_interface.command_line module

`cis_interface.command_line.` `cc_flags`()

  Get the compiler flags necessary for including the interface library in a C or C++ program.

  Returns: | The necessary compiler flags and preprocessor definitions.
  Return type: | list

`cis_interface.command_line.` `ciscc`()

  Compile C/C++ program.

`cis_interface.command_line.` `cisrun`()

  Start a run.

`cis_interface.command_line.` `ld_flags`()

  Get the linker flags necessary for calling functions/classes from the interface library in a C or C++ program.

  Returns: | The necessary library linking flags.
  Return type: | list

## cis_interface.config module

This module imports the configuration for cis_interface.

Todo

Remove reference to environment variables for accessing config options.

class `cis_interface.config.` `CisConfigParser`(defaults=None, dict_type=<class 'collections.OrderedDict'>, allow_no_value=False)

  Bases: `ConfigParser.ConfigParser`

  Config parser that returns None if option not provided on get.

  `get`(section, option, default=None, **kwargs)

    Return None if the section/option does not exist.

    Parameters: | section (str) – Name of section. option (str) – Name of option in section. default (obj, optional) – Value that should be returned if the section and/or option are not found or are an empty string. Defaults to None. **kwargs – Additional keyword arguments are passed to the parent class’s get.
    Returns: | String entry if the section & option exist, otherwise default.
    Return type: | obj

`cis_interface.config.` `cfg_environment`(env=None, cfg=None)

  Set environment variables based on config options.

  Parameters: | env (dict, optional) – Dictionary of environment variables that should be updated. Defaults to os.environ. cfg (`cis_interface.config.CisConfigParser`, optional) – Config parser with options that should be used to update the environment. Defaults to `cis_interface.config.cis_cfg`.

`cis_interface.config.` `cfg_logging`(cfg=None)

  Set logging levels from config options.

  Parameters: | cfg (`cis_interface.config.CisConfigParser`, optional) – Config parser with options that should be used to update the environment. Defaults to `cis_interface.config.cis_cfg`.

## cis_interface.platform module

This module handle platform compatibility issues.

## cis_interface.runner module

This module provides tools for running models using cis_interface.

class `cis_interface.runner.` `CisRunner`(modelYmls, namespace, host=None, rank=0, cis_debug_level=None, rmq_debug_level=None, cis_debug_prefix=None)

  Bases: `cis_interface.tools.CisClass`

  This class handles the orchestration of starting the model and IO drivers, monitoring their progress, and cleaning up on exit.

  Parameters: | modelYmls (list) – List of paths to yaml files specifying the models that should be run. namespace (str) – Name that should be used to uniquely identify any RMQ exchange. host (str, optional) – Name of the host that the models will be launched from. Defaults to None. rank (int, optional) – Rank of this set of models if run in parallel. Defaults to 0. cis_debug_level (str, optional) – Level for CiS debug messages. Defaults to environment variable ‘CIS_DEBUG’. rmq_debug_level (str, optional) – Level for RabbitMQ debug messages. Defaults to environment variable ‘RMQ_DEBUG’. cis_debug_prefix (str, optional) – Prefix for CiS debug messages. Defaults to namespace.

  `namespace`

    str – Name that should be used to uniquely identify any RMQ exchange.

  `host`

    str – Name of the host that the models will be launched from.

  `rank`

    int – Rank of this set of models if run in parallel.

  `modeldrivers`

    dict – Model drivers associated with this run.

  `inputdrivers`

    dict – Input drivers associated with this run.

  `outputdrivers`

    dict – Output drivers associated with this run.

  `serverdrivers`

    dict – The addresses associated with different server drivers.

  `interrupt_time`

    float – Time of last interrupt signal.

  `error_flag`

    bool – True if one or more models raises an error.

  ..todo:: namespace, host, and rank do not seem strictly necessary.

  `all_drivers`

    iterator – For all drivers.

  `cleanup`()

    Perform cleanup operations for all drivers.

  `closeChannels`(force_stop=False)

    Stop IO drivers and join the threads.

    Parameters: | force_stop (bool, optional) – If True, the terminate method is used to stop the drivers. Otherwise, the stop method is used. The stop method will try to exit gracefully while terminate will exit as quickly as possible. Defaults to False.

  `createDriver`(yml)

    Create a driver instance from the yaml information.

    Parameters: | yml (yaml) – Yaml object containing driver information.
    Returns: | An instance of the specified driver.
    Return type: | object

  `createInputDriver`(yml)

    Create an input driver instance from the yaml information.

    Parameters: | yml (yaml) – Yaml object containing driver information.
    Returns: | An instance of the specified driver.
    Return type: | object

  `createModelDriver`(yml)

    Create a model driver instance from the yaml information.

    Parameters: | yml (yaml) – Yaml object containing driver information.
    Returns: | An instance of the specified driver.
    Return type: | object

  `createOutputDriver`(yml)

    Create an output driver instance from the yaml information.

    Parameters: | yml (yaml) – Yaml object containing driver information.
    Returns: | An instance of the specified driver.
    Return type: | object

  `do_client_exits`(model)

    Perform exits for IO drivers associated with a client model.

    Parameters: | model (dict) – Dictionary of model parameters including any associated IO drivers.

  `do_model_exits`(model)

    Perform exits for IO drivers associated with a model.

    Parameters: | model (dict) – Dictionary of model parameters including any associated IO drivers.

  `io_drivers`(model=None)

    Return the input and output drivers for one or all models.

    Parameters: | model (str, optional) – Name of a model that I/O drivers should be returned for. Defaults to None and all I/O drivers are returned.
    Returns: | Access to list of I/O drivers.
    Return type: | iterator

  `loadDrivers`()

    Load all of the necessary drivers, doing the IO drivers first and adding IO driver environmental variables back tot he models.

  `pprint`(*args)

    Print with color.

  `printStatus`()

    Print the status of all drivers, starting with the IO drivers.

  `reset_signal_handler`()

    Reset signal handlers to old ones.

  `run`(signal_handler=None)

    Run all of the models and wait for them to exit.

  `set_signal_handler`(signal_handler=None)

    Set the signal handler.

    Parameters: | signal_handler (function, optional) – Function that should handle received SIGINT and SIGTERM signals. Defaults to self.signal_handler.

  `signal_handler`(sig, frame)

    Terminate all drivers on interrrupt.

  `startDrivers`()

    Start drivers, starting with the IO drivers.

  `terminate`()

    Immediately stop all drivers, beginning with IO drivers.

  `waitModels`()

    Wait for all model drivers to finish. When a model finishes, join the thread and perform exits for associated IO drivers.

`cis_interface.runner.` `get_runner`(models, **kwargs)

  Get runner for a set of models, getting run information from the environment.

  Parameters: | models (list) – List of yaml files containing information on the models that should be run. **kwargs – Additonal keyword arguments are passed to CisRunner.
  Returns: | Runner for the provided models.
  Return type: | CisRunner
  Raises: | `Exception` – If config option ‘namespace’ in ‘rmq’ section not set.

## cis_interface.scanf module

Small scanf implementation.

Python has powerful regular expressions but sometimes they are totally overkill when you just want to parse a simple-formatted string. C programmers use the scanf-function for these tasks (see link below).

This implementation of scanf translates the simple scanf-format into regular expressions. Unlike C you can be sure that there are no buffer overflows possible.

For more information see
  - http://www.python.org/doc/current/lib/node49.html
  - http://en.wikipedia.org/wiki/Scanf
Original code from:
  http://code.activestate.com/recipes/502213-simple-scanf-implementation/

Modified original to make the %f more robust, as well as added %* modifier to skip fields.

Version: 1.3.3

Releases:
  1.0
    2010-10-11
      - Initial release
  1.1
    2010-10-13
      - Changed regex from ‘match’ (only matches at beginning of line) to ‘search’ (matches anywhere in line)
      - Bugfix - ignore cast for skipped fields
  1.2
    2013-05-30
      - Added ‘collapseWhitespace’ flag (defaults to True) to take the search string and replace all whitespace with regex string to match repeated whitespace. This enables better matching in log files where the data has been formatted for easier reading. These cases have variable amounts of whitespace between the columns, depending on the number of characters in the data itself.
  1.3
    2016-01-18
      - Add ‘extractdata’ function.
  1.3.1
    2016-06-23
      - Release to PyPi, now including README.md
Updates by Meagan Lang:
  2018-04-12
    - Added ability to parse components of field specifiers and cope with whitespace padding.

`cis_interface.scanf.` `scanf`(format, s=None, collapseWhitespace=True)
  scanf supports the following formats:
    %c One character %5c 5 characters %d, %i int value %7d, %7i int value with length 7 %f float value %o octal value %X, %x hex value %s string terminated by whitespace

  Examples: >>> scanf(“%s - %d errors, %d warnings”, “/usr/sbin/sendmail - 0 errors, 4 warnings”) (‘/usr/sbin/sendmail’, 0, 4) >>> scanf(“%o %x %d”, “0123 0x123 123”) (66, 291, 123)

  If the parameter s is a file-like object, s.readline is called. If s is not specified, stdin is assumed.

  The function returns a tuple of found values or None if the format does not match.

`cis_interface.scanf.` `scanf_compile`(format, collapseWhitespace=True)

  Translate the format into a regular expression

  For example: >>> format_re, casts = _scanf_compile(‘%s - %d errors, %d warnings’) >>> print format_re.pattern (S+) - ([+-]?d+) errors, ([+-]?d+) warnings

  Translated formats are cached for faster reuse

## cis_interface.schema module

class `cis_interface.schema.` `ComponentSchema`(schema_type, subtype_attr=None, **kwargs)

  Bases: `dict`

  Schema information for one component.

  Parameters: | schema_type (str) – The name of the component. subtype_attr (str, optional) – The attribute that should be used to log subtypes. Defaults to None. **kwargs – Additional keyword arguments are entries in the component schema.

  `append`(comp_cls, subtype=None)

    Append component class to the schema.

    Parameters: | comp_cls (class) – Component class that should be added. subtype (str, tuple, optional) – Key used to identify the subtype of the component type. Defaults to subtype_attr if one was provided, otherwise the subtype will not be logged.

  `append_rules`(new)

    Add rules from new class’s schema to this one.

    Parameters: | new (dict) – New schema to add.

  `class2subtype`

    dict – Mapping from class to list of subtypes.

  `classes`

    list – All available classes for this schema.

  `subtype2class`

    dict – Mapping from subtype to class.

  `subtypes`

    list – All subtypes for this schema type.

class `cis_interface.schema.` `SchemaDumper`(stream, default_style=None, default_flow_style=None, canonical=None, indent=None, width=None, allow_unicode=None, line_break=None, encoding=None, explicit_start=None, explicit_end=None, version=None, tags=None)

  Bases: `yaml.dumper.Dumper`

  SafeDumper for schema that includes tuples and Schema classes.

  `represent_ComponentSchema`(data)
  `represent_SchemaRegistry`(data)
  `represent_python_tuple`(data, **kwargs)
  `yaml_representers` = {None: <unbound method SafeRepresenter.represent_undefined>, <class 'cis_interface.schema.SchemaRegistry'>: <unbound method SchemaDumper.represent_SchemaRegistry>, <class 'cis_interface.schema.ComponentSchema'>: <unbound method SchemaDumper.represent_ComponentSchema>, <type 'bool'>: <unbound method SafeRepresenter.represent_bool>, <type 'classobj'>: <unbound method Representer.represent_name>, <type 'complex'>: <unbound method Representer.represent_complex>, <type 'float'>: <unbound method SafeRepresenter.represent_float>, <type 'function'>: <unbound method Representer.represent_name>, <type 'int'>: <unbound method SafeRepresenter.represent_int>, <type 'list'>: <unbound method SafeRepresenter.represent_list>, <type 'long'>: <unbound method Representer.represent_long>, <type 'dict'>: <unbound method SafeRepresenter.represent_dict>, <type 'builtin_function_or_method'>: <unbound method Representer.represent_name>, <type 'module'>: <unbound method Representer.represent_module>, <type 'NoneType'>: <unbound method SafeRepresenter.represent_none>, <type 'set'>: <unbound method SafeRepresenter.represent_set>, <type 'str'>: <unbound method Representer.represent_str>, <type 'tuple'>: <unbound method SchemaDumper.represent_python_tuple>, <type 'type'>: <unbound method Representer.represent_name>, <type 'unicode'>: <unbound method Representer.represent_unicode>, <type 'datetime.datetime'>: <unbound method SafeRepresenter.represent_datetime>, <type 'datetime.date'>: <unbound method SafeRepresenter.represent_date>}

class `cis_interface.schema.` `SchemaLoader`(stream)

  Bases: `yaml.loader.SafeLoader`

  SafeLoader for schema that includes tuples.

  `construct_python_tuple`(node)
  `yaml_constructors` = {None: <unbound method SafeConstructor.construct_undefined>, u'tag:yaml.org,2002:binary': <unbound method SafeConstructor.construct_yaml_binary>, u'tag:yaml.org,2002:bool': <unbound method SafeConstructor.construct_yaml_bool>, u'tag:yaml.org,2002:float': <unbound method SafeConstructor.construct_yaml_float>, u'tag:yaml.org,2002:int': <unbound method SafeConstructor.construct_yaml_int>, u'tag:yaml.org,2002:map': <unbound method SafeConstructor.construct_yaml_map>, u'tag:yaml.org,2002:null': <unbound method SafeConstructor.construct_yaml_null>, u'tag:yaml.org,2002:omap': <unbound method SafeConstructor.construct_yaml_omap>, u'tag:yaml.org,2002:pairs': <unbound method SafeConstructor.construct_yaml_pairs>, 'tag:yaml.org,2002:python/tuple': <unbound method SchemaLoader.construct_python_tuple>, u'tag:yaml.org,2002:seq': <unbound method SafeConstructor.construct_yaml_seq>, u'tag:yaml.org,2002:set': <unbound method SafeConstructor.construct_yaml_set>, u'tag:yaml.org,2002:str': <unbound method SafeConstructor.construct_yaml_str>, u'tag:yaml.org,2002:timestamp': <unbound method SafeConstructor.construct_yaml_timestamp>}

class `cis_interface.schema.` `SchemaRegistry`(registry=None, required=None)

  Bases: `dict`

  Registry of schema’s for different integration components.

  Parameters: | registry (dict, optional) – Dictionary of registered components. Defaults to None and the registry will be empty. required (list, optional) – Components that are required. Defaults to [‘comm’, ‘file’, ‘model’, ‘connection’]. Ignored if registry is None.
  Raises: | `ValueError` – If registry is provided and one of the required components is missing.

  `class2conntype`

    dict – Mapping from connection class to comm classes & direction.

  `class2filetype`

    dict – Mapping from communication class to filetype.

  `class2language`

    dict – Mapping from ModelDriver class to programming language.

  `conntype2class`

    dict – Mapping from comm classes & direction to connection class.

  `filetype2class`

    dict – Mapping from filetype to communication class.

  classmethod `from_file`(fname)

    Create a SchemaRegistry from a file.

    Parameters: | fname (str) – Full path to the file the schema should be loaded from.

  `language2class`

    dict – Mapping from programming language to ModelDriver class.

  `load`(fname)

    Load schema from a file.

    Parameters: | fname (str) – Full path to the file the schema should be loaded from.

  `save`(fname)

    Save the schema to a file.

    Parameters: | fname (str) – Full path to the file the schema should be saved to. schema (dict) – cis_interface YAML options.

  `validator`

    Compose complete schema for parsing yaml.

class `cis_interface.schema.` `SchemaValidator`(*args, **kwargs)

  Bases: `cerberus.validator.Validator`

  Class for validating the schema.

  `cis_type_order` = ['list', 'string', 'integer', 'boolean', 'function']
  `coercers` = ('boolean', 'function', 'integer', 'list', 'string')
  `default_setters` = ()
  `normalization_rules` = {'coerce': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}, 'default': {'nullable': True}, 'default_setter': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ()}]}, 'purge_unknown': {'type': 'boolean'}, 'rename': {'type': 'hashable'}, 'rename_handler': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}
  `rules` = {'allof': {'type': 'list', 'logical': 'allof'}, 'allow_unknown': {'oneof': [{'type': 'boolean'}, {'validator': 'bulk_schema', 'type': ['dict', 'string']}]}, 'allowed': {'type': 'list'}, 'anyof': {'type': 'list', 'logical': 'anyof'}, 'coerce': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}, 'default': {'nullable': True}, 'default_setter': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ()}]}, 'dependencies': {'validator': 'dependencies', 'type': ('dict', 'hashable', 'list')}, 'empty': {'type': 'boolean'}, 'excludes': {'type': ('hashable', 'list'), 'schema': {'type': 'hashable'}}, 'forbidden': {'type': 'list'}, 'items': {'validator': 'items', 'type': 'list'}, 'keyschema': {'forbidden': ['rename', 'rename_handler'], 'validator': 'bulk_schema', 'type': ['dict', 'string']}, 'max': {'nullable': False}, 'maxlength': {'type': 'integer'}, 'min': {'nullable': False}, 'minlength': {'type': 'integer'}, 'noneof': {'type': 'list', 'logical': 'noneof'}, 'nullable': {'type': 'boolean'}, 'oneof': {'type': 'list', 'logical': 'oneof'}, 'purge_unknown': {'type': 'boolean'}, 'readonly': {'type': 'boolean'}, 'regex': {'type': 'string'}, 'rename': {'type': 'hashable'}, 'rename_handler': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}, 'required': {'type': 'boolean'}, 'schema': {'anyof': [{'validator': 'schema'}, {'validator': 'bulk_schema'}], 'type': ['dict', 'string']}, 'type': {'validator': 'type', 'type': ['string', 'list']}, 'validator': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ()}]}}, {'type': 'string', 'allowed': ()}]}, 'valueschema': {'forbidden': ['rename', 'rename_handler'], 'validator': 'bulk_schema', 'type': ['dict', 'string']}}
  `types_mapping` = {'binary': TypeDefinition(name='binary', included_types=(<type 'str'>, <type 'bytearray'>), excluded_types=()), 'boolean': TypeDefinition(name='boolean', included_types=(<type 'bool'>,), excluded_types=()), 'date': TypeDefinition(name='date', included_types=(<type 'datetime.date'>,), excluded_types=()), 'datetime': TypeDefinition(name='datetime', included_types=(<type 'datetime.datetime'>,), excluded_types=()), 'dict': TypeDefinition(name='dict', included_types=(<class '_abcoll.Mapping'>,), excluded_types=()), 'float': TypeDefinition(name='float', included_types=(<type 'float'>, (<type 'int'>, <type 'long'>)), excluded_types=()), 'function': TypeDefinition(name='function', included_types=<type 'function'>, excluded_types=()), 'integer': TypeDefinition(name='integer', included_types=((<type 'int'>, <type 'long'>),), excluded_types=()), 'list': TypeDefinition(name='list', included_types=(<class '_abcoll.Sequence'>,), excluded_types=(<type 'basestring'>,)), 'number': TypeDefinition(name='number', included_types=((<type 'int'>, <type 'long'>), <type 'float'>), excluded_types=(<type 'bool'>,)), 'set': TypeDefinition(name='set', included_types=(<type 'set'>,), excluded_types=()), 'string': TypeDefinition(name='string', included_types=<type 'basestring'>, excluded_types=())}
  `validation_rules` = {'allof': {'type': 'list', 'logical': 'allof'}, 'allow_unknown': {'oneof': [{'type': 'boolean'}, {'validator': 'bulk_schema', 'type': ['dict', 'string']}]}, 'allowed': {'type': 'list'}, 'anyof': {'type': 'list', 'logical': 'anyof'}, 'dependencies': {'validator': 'dependencies', 'type': ('dict', 'hashable', 'list')}, 'empty': {'type': 'boolean'}, 'excludes': {'type': ('hashable', 'list'), 'schema': {'type': 'hashable'}}, 'forbidden': {'type': 'list'}, 'items': {'validator': 'items', 'type': 'list'}, 'keyschema': {'forbidden': ['rename', 'rename_handler'], 'validator': 'bulk_schema', 'type': ['dict', 'string']}, 'max': {'nullable': False}, 'maxlength': {'type': 'integer'}, 'min': {'nullable': False}, 'minlength': {'type': 'integer'}, 'noneof': {'type': 'list', 'logical': 'noneof'}, 'nullable': {'type': 'boolean'}, 'oneof': {'type': 'list', 'logical': 'oneof'}, 'readonly': {'type': 'boolean'}, 'regex': {'type': 'string'}, 'required': {'type': 'boolean'}, 'schema': {'anyof': [{'validator': 'schema'}, {'validator': 'bulk_schema'}], 'type': ['dict', 'string']}, 'type': {'validator': 'type', 'type': ['string', 'list']}, 'validator': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ()}]}}, {'type': 'string', 'allowed': ()}]}, 'valueschema': {'forbidden': ['rename', 'rename_handler'], 'validator': 'bulk_schema', 'type': ['dict', 'string']}}
  `validators` = ()

`cis_interface.schema.` `clear_schema`()

  Clear global schema.

`cis_interface.schema.` `create_schema`()

  Create a new schema from the registry.

`cis_interface.schema.` `get_schema`(fname=None)

  Return the cis_interface schema for YAML options.

  Parameters: | fname (str, optional) – Full path to the file that the schema should be loaded from. If the file dosn’t exist, it is created. Defaults to _schema_fname.
  Returns: | cis_interface YAML options.
  Return type: | dict

`cis_interface.schema.` `inherit_schema`(orig, key, value, **kwargs)

  Create an inherited schema, adding new value to accepted ones for dependencies.

  Parameters: | orig (dict) – Schema that will be inherited. key (str) – Field that other fields are dependent on. value (str) – New value for key that dependent fields should accept. **kwargs – Additional keyword arguments will be added to the schema with dependency on the provided key/value pair.
  Returns: | New schema.
  Return type: | dict

`cis_interface.schema.` `init_registry`()

  Initialize the registries and schema.

`cis_interface.schema.` `init_schema`(fname=None)

  Initialize global schema.

`cis_interface.schema.` `load_schema`(fname=None)

  Return the cis_interface schema for YAML options.

  Parameters: | fname (str, optional) – Full path to the file that the schema should be loaded from. If the file dosn’t exist, it is created. Defaults to _schema_fname.
  Returns: | cis_interface YAML options.
  Return type: | dict

`cis_interface.schema.` `register_component`(component_class)

  Decorator for registering a class as a yaml component.

`cis_interface.schema.` `str_to_function`(value)

  Convert a string to a function.

  Parameters: | value (str, list) – String or list of strings, specifying function(s). The format should be “<package.module>:<function>” so that <function> can be imported from <package>.
  Returns: | Callable function.
  Return type: | func

## cis_interface.tools module

This modules offers various tools.

class `cis_interface.tools.` `CisClass`(name, uuid=None, working_dir=None, timeout=60.0, sleeptime=0.01, **kwargs)

  Bases: `logging.LoggerAdapter`

  Base class for CiS classes.

  Parameters: | name (str) – Class name. uuid (str, optional) – Unique ID for this instance. Defaults to None and is assigned. working_dir (str, optional) – Working directory. If not provided, the current working directory is used. timeout (float, optional) – Maximum time (in seconds) that should be spent waiting on a process. Defaults to 60. sleeptime (float, optional) – Time that class should sleep for when sleep is called. Defaults to 0.01. **kwargs – Additional keyword arguments are assigned to the extra_kwargs dictionary.

  `name`

    str – Class name.

  `uuid`

    str – Unique ID for this instance.

  `sleeptime`

    float – Time that class should sleep for when sleep called.

  `longsleep`

    float – Time that the class will sleep for when waiting for longer tasks to complete (10x longer than sleeptime).

  `timeout`

    float – Maximum time that should be spent waiting on a process.

  `working_dir`

    str – Working directory.

  `errors`

    list – List of errors.

  `extra_kwargs`

    dict – Keyword arguments that were not parsed.

  `sched_out`

    obj – Output from the last scheduled task with output.

  `logger`

    logging.Logger – Logger object for this object.

  `suppress_special_debug`

    bool – If True, special_debug log messages are suppressed.

  `as_str`(obj)

    Return str version of object if it is not already a string.

    Parameters: | obj (object) – Object that should be turned into a string.
    Returns: | String version of provided object.
    Return type: | str

  `check_timeout`(key=None, key_level=0)

    Check timeout for the calling function/method.

    Parameters: | key (str, optional) – Key for timeout that should be checked. Defaults to None and is set by the calling class and function/method (See timeout_key). key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that called start_timeout. Higher values use classes and function/methods further up in the stack. Defaults to 0.
    Raises: | `KeyError` – If there is not a timeout registered for the specified key.

  `debug_log`()

    Turn on debugging.

  `display`(msg='', *args, **kwargs)

    Print a message, no log.

  `dummy_log`(*args, **kwargs)

    Dummy log function that dosn’t do anything.

  `error`

    Log an error level message.

  `exception`

    Log an exception level message.

  `get_timeout_key`(key_level=0, key_suffix=None)

    Return a key for a given level in the stack, relative to the function calling get_timeout_key.

    Parameters: | key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that is 2 steps higher in the stack. Higher values use classes and function/methods further up in the stack. Defaults to 0. key_suffix (str, optional) – String that should be appended to the end of the generated key. Defaults to None and is ignored.
    Returns: | Key identifying calling object and method.
    Return type: | str

  `name`

    Name of the class object.

  `printStatus`()

    Print the class status.

  `print_encoded`(msg, *args, **kwargs)

    Print bytes to stdout, encoding if possible.

    Parameters: | msg (str, bytes) – Message to print. *args – Additional arguments are passed to print. **kwargs – Additional keyword arguments are passed to print.

  `process`(msg, kwargs)

    Process logging message.

  `raise_error`(e)

    Raise an exception, logging it first.

    Parameters: | e (Exception) – Exception to raise.
    Raises: | The provided exception.

  `reset_log`()

    Resetting logging to prior value.

  `sched_task`(t, func, args=[], kwargs={}, store_output=False)

    Schedule a task that will be executed after a certain time has elapsed.

    Parameters: | t (float) – Number of seconds that should be waited before task is executed. func (object) – Function that should be executed. args (list, optional) – Arguments for the provided function. Defaults to []. kwargs (dict, optional) – Keyword arguments for the provided function. Defaults to {}. store_output (bool, optional) – If True, the output from the scheduled task is stored in self.sched_out. Otherwise, it is not stored. Defaults to False.

  `sleep`(t=None)

    Have the class sleep for some period of time.

    Parameters: | t (float, optional) – Time that class should sleep for. If not provided, the attribute ‘sleeptime’ is used.

  `special_debug`

    Log debug level message contingent of supression flag.

  `start_timeout`(t=None, key=None, key_level=0, key_suffix=None)

    Start a timeout for the calling function/method.

    Parameters: | t (float, optional) – Maximum time that the calling function should wait before timeing out. If not provided, the attribute ‘timeout’ is used. key (str, optional) – Key that should be associated with the timeout that is created. Defaults to None and is set by the calling class and function/method (See get_timeout_key). key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that called start_timeout. Higher values use classes and function/methods further up in the stack. Defaults to 0. key_suffix (str, optional) – String that should be appended to the end of the generated key. Defaults to None and is ignored.
    Raises: | `KeyError` – If the key already exists.

  `stop_timeout`(key=None, key_level=0, key_suffix=None, quiet=False)

    Stop a timeout for the calling function method.

    Parameters: | key (str, optional) – Key for timeout that should be stopped. Defaults to None and is set by the calling class and function/method (See timeout_key). key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that called start_timeout. Higher values use classes and function/methods further up in the stack. Defaults to 0. key_suffix (str, optional) – String that should be appended to the end of the generated key. Defaults to None and is ignored. quiet (bool, optional) – If True, error message on timeout exceeded will be debug log. Defaults to False.
    Raises: | `KeyError` – If there is not a timeout registered for the specified key.

  `timeout_key`

    str – Key identifying calling object and method.

  `verbose_debug`(*args, **kwargs)

    Log a verbose debug level message.

class `cis_interface.tools.` `CisPopen`(cmd_args, forward_signals=True, **kwargs)

  Bases: `subprocess.Popen`

  Uses Popen to open a process without a buffer. If not already set, the keyword arguments ‘bufsize’, ‘stdout’, and ‘stderr’ are set to 0, subprocess.PIPE, and subprocess.STDOUT respectively. This sets the output stream to unbuffered and directs both stdout and stderr to the stdout pipe. In addition this class overrides Popen.kill() to allow processes to be killed with CTRL_BREAK_EVENT on windows.

  Parameters: | args (list, str) – Shell command or list of arguments that should be run. forward_signals (bool, optional) – If True, flags will be set such that signals received by the spawning process will be forwarded to the child process. If False, the signals will not be forwarded. Defaults to True. **kwargs – Additional keywords arguments are passed to Popen.

  `kill`(*args, **kwargs)

    On windows using CTRL_BREAK_EVENT to kill the process.

class `cis_interface.tools.` `CisThread`(name=None, target=None, args=(), kwargs=None, daemon=False, group=None, **cis_kwargs)

  Bases: `threading.Thread`, `cis_interface.tools.CisClass`

  Thread for CiS that tracks when the thread is started and joined.

  `lock`

    threading.RLock – Lock for accessing the sockets from multiple threads.

  `start_event`

    threading.Event – Event indicating that the thread was started.

  `terminate_event`

    threading.Event – Event indicating that the thread should be terminated. The target must exit when this is set.

  `atexit`()

    Actions performed when python exits.

  `before_start`()

    Actions to perform on the main thread before starting the thread.

  `cleanup`()

    Actions to perform to clean up the thread after it has stopped.

  `main_terminated`

    bool – True if the main thread has terminated.

  `run`(*args, **kwargs)

    Continue running until terminate event set.

  `set_started_flag`()

    Set the started flag for the thread to True.

  `set_terminated_flag`()

    Set the terminated flag for the thread to True.

  `start`(*args, **kwargs)

    Start thread and print info.

  `terminate`(no_wait=False)

    Set the terminate event and wait for the thread to stop.

    Parameters: | no_wait (bool, optional) – If True, terminate will not block until the thread stops. Defaults to False and blocks.
    Raises: | `AssertionError` – If no_wait is False and the thread has not stopped after the timeout.

  `unset_started_flag`()

    Set the started flag for the thread to False.

  `unset_terminated_flag`()

    Set the terminated flag for the thread to False.

  `wait`(timeout=None, key=None)

    Wait until thread finish to return using sleeps rather than blocking.

    Parameters: | timeout (float, optional) – Maximum time that should be waited for the driver to finish. Defaults to None and is infinite. key (str, optional) – Key that should be used to register the timeout. Defaults to None and is set based on the stack trace.

  `was_started`

    bool – True if the thread was started. False otherwise.

  `was_terminated`

    bool – True if the thread was terminated. False otherwise.

class `cis_interface.tools.` `CisThreadLoop`(*args, **kwargs)

  Bases: `cis_interface.tools.CisThread`

  Thread that will run a loop until the terminate event is called.

  `after_loop`()

    Actions performed after the loop.

  `before_loop`()

    Actions performed before the loop.

  `on_main_terminated`(dont_break=False)

    Actions performed when 1st main terminated.

    Parameters: | dont_break (bool, optional) – If True, the break flag won’t be set. Defaults to False.

  `run`(*args, **kwargs)

    Continue running until terminate event set.

  `run_loop`(*args, **kwargs)

    Actions performed on each loop iteration.

  `set_break_flag`()

    Set the break flag for the thread to True.

  `set_loop_flag`()

    Set the loop flag for the thread to True.

  `terminate`(*args, **kwargs)

    Also set break flag.

  `unset_break_flag`()

    Set the break flag for the thread to False.

  `unset_loop_flag`()

    Set the loop flag for the thread to False.

  `wait_for_loop`(timeout=None, key=None)

    Wait until thread enters loop to return using sleeps rather than blocking.

    Parameters: | timeout (float, optional) – Maximum time that should be waited for the thread to enter loop. Defaults to None and is infinite. key (str, optional) – Key that should be used to register the timeout. Defaults to None and is set based on the stack trace.

  `was_break`

    bool – True if the break flag was set.

  `was_loop`

    bool – True if the thread was loop. False otherwise.

class `cis_interface.tools.` `TimeOut`(max_time, key=None)

  Bases: `object`

  Class for checking if a period of time has been elapsed.

  Parameters: | max_time (float, bbol) – Maximum period of time that should elapse before ‘is_out’ returns True. If False, ‘is_out’ will never return True. Providing 0 indicates that ‘is_out’ should immediately return True. key (str, optional) – Key that was used to register the timeout. Defaults to None.

  `max_time`

    float – Maximum period of time that should elapsed before ‘is_out’ returns True.

  `start_time`

    float – Result of time.time() at start.

  `key`

    str – Key that was used to register the timeout.

  `elapsed`

    float – Total time that has elapsed since the start.

  `is_out`

    bool – True if there is not any time remaining. False otherwise.

`cis_interface.tools.` `check_locks`()

  Check for locks in lock registry that are locked.

`cis_interface.tools.` `check_sockets`()

  Check registered sockets.

`cis_interface.tools.` `check_threads`()

  Check for threads that are still running.

`cis_interface.tools.` `cis_atexit`()

  Things to do at exit.

`cis_interface.tools.` `eval_kwarg`(x)

  If x is a string, eval it. Otherwise just return it.

  Parameters: | x (str, obj) – String to be evaluated as an object or an object.
  Returns: | Result of evaluated string or the input object.
  Return type: | obj

`cis_interface.tools.` `get_CIS_MSG_MAX`()

  Get the maximum message size for the default comm.

`cis_interface.tools.` `get_default_comm`()

  Get the default comm that should be used for message passing.

`cis_interface.tools.` `is_ipc_installed`()

  Determine if the IPC libraries are installed.

  Returns: | True if the IPC libraries are installed, False otherwise.
  Return type: | bool

`cis_interface.tools.` `is_zmq_installed`(check_c=True)

  Determine if the libczmq & libzmq libraries are installed.

  Returns: | True if both libraries are installed, False otherwise.
  Return type: | bool

`cis_interface.tools.` `kill`(pid, signum)

  Kill process by mapping signal number.

  Parameters: | pid (int) – Process ID. signum (int) – Signal that should be sent.

`cis_interface.tools.` `locate_path`(fname, basedir='/')

  Find the full path to a file using where on Windows.

`cis_interface.tools.` `popen_nobuffer`(*args, **kwargs)

  Uses Popen to open a process without a buffer. If not already set, the keyword arguments ‘bufsize’, ‘stdout’, and ‘stderr’ are set to 0, subprocess.PIPE, and subprocess.STDOUT respectively. This sets the output stream to unbuffered and directs both stdout and stderr to the stdout pipe.

  Parameters: | args (list, str) – Shell command or list of arguments that should be run. forward_signals (bool, optional) – If True, flags will be set such that signals received by the spawning process will be forwarded to the child process. If False, the signals will not be forwarded. Defaults to True. **kwargs – Additional keywords arguments are passed to Popen.
  Returns: | Process that was started.
  Return type: | CisPopen

`cis_interface.tools.` `print_encoded`(msg, *args, **kwargs)

  Print bytes to stdout, encoding if possible.

  Parameters: | msg (str, bytes) – Message to print. *args – Additional arguments are passed to print. **kwargs – Additional keyword arguments are passed to print.

`cis_interface.tools.` `sleep`(interval)

  Sleep for a specified number of seconds.

  Parameters: | interval (float) – Time in seconds that process should sleep.

## cis_interface.units module

`cis_interface.units.` `is_unit`(ustr)

  Determine if a string is a valid unit.

  Parameters: | ustr – String representation to test.
  Returns: | True if the string is a valid unit. False otherwise.
  Return type: | bool

## cis_interface.yamlfile module

`cis_interface.yamlfile.` `bridge_io_drivers`(yml, iodict)

  Create connection from input/output driver.

  Parameters: | dict – YAML entry for model output that should be modified. iodict (dict) – Log of all input/outputs.

`cis_interface.yamlfile.` `cdriver2filetype`(driver)

  Convert a connection driver to a file type.

  Parameters: | driver (str) – The name of the connection driver.
  Returns: | The corresponding file type for the driver.
  Return type: | str

`cis_interface.yamlfile.` `link_model_io`(existing)

  Link I/O drivers back to the models they communicate with.

  Parameters: | existing (dict) – Dictionary of existing components.
  Returns: | Dictionary with I/O drivers added to models.
  Return type: | dict

`cis_interface.yamlfile.` `load_yaml`(fname)

  Parse a yaml file defining a run.

  Parameters: | fname (str) – Path to the yaml file.
  Returns: | Contents of yaml file.
  Return type: | dict

`cis_interface.yamlfile.` `parse_component`(yml, ctype, existing=None)

  Parse a yaml entry for a component, adding it to the list of existing components.

  Parameters: | yml (dict) – YAML dictionary for a component. ctype (str) – Component type. This can be ‘input’, ‘output’, ‘model’, or ‘connection’. existing (dict, optional) – Dictionary of existing components. Defaults to empty dict.
  Raises: | `TypeError` – If yml is not a dictionary. `ValueError` – If dtype is not ‘input’, ‘output’, ‘model’, or ‘connection’. `ValueError` – If the component already exists.
  Returns: | All components identified.
  Return type: | dict

`cis_interface.yamlfile.` `parse_connection`(yml, existing)

  Parse a yaml entry for a connection between I/O channels.

  Parameters: | yml (dict) – YAML dictionary for a connection. existing (dict) – Dictionary of existing components.
  Raises: | `RuntimeError` – If the ‘input’ entry is not a model output or file. `RuntimeError` – If neither the ‘input’ or ‘output’ entries correspond to model I/O channels.
  Returns: | Updated log of all entries.
  Return type: | dict

`cis_interface.yamlfile.` `parse_model`(yml, existing)

  Parse a yaml entry for a model.

  Parameters: | yml (dict) – YAML dictionary for a model. existing (dict) – Dictionary of existing components.
  Returns: | Updated log of all entries.
  Return type: | dict

`cis_interface.yamlfile.` `parse_yaml`(files)

  Parse list of yaml files.

  Parameters: | files (str, list) – Either the path to a single yaml file or a list of yaml files.
  Raises: | `ValueError` – If the yml dictionary is missing a required keyword or has an invalid value. `RuntimeError` – If one of the I/O channels is not initialized with driver information.
  Returns: | Dictionary of information parsed from the yamls.
  Return type: | dict

`cis_interface.yamlfile.` `prep_connection`(yml, iodict)

  Prepare yaml connection entry for parsing with schema.

  Parameters: | yml (dict) – YAML entry for connection that will be modified. iodict (dict) – Log of all input/outputs.

`cis_interface.yamlfile.` `prep_model`(yml, iodict)

  Prepare yaml model entry for parsing with schema.

  Parameters: | yml (dict) – YAML entry for model that will be modified. iodict (dict) – Log of all input/outputs.

`cis_interface.yamlfile.` `prep_yaml`(files)

  Prepare yaml to be parsed by Cerberus using schema including covering backwards compatible options.

  Parameters: | files (str, list) – Either the path to a single yaml file or a list of yaml files.
  Returns: | YAML ready to be parsed using schema.
  Return type: | dict

`cis_interface.yamlfile.` `rwmeth2filetype`(rw_meth)

## Module contents

This package provides a framework for integrating models across languages such that they can be run simultaneously, passing input back and forth.