Metadata-Version: 1.1
Name: humilis
Version: 1.5.8
Summary: AWS cloudformation-based deployment framework
Home-page: http://github.com/humilis/humilis
Author: German Gomez-Herrero
Author-email: german@findhotel.net
License: MIT
Description: humilis
        =======
        
        |Build Status| |PyPI|
        
        Helps you deploy AWS infrastructure with
        `Cloudformation <https://aws.amazon.com/cloudformation/>`__.
        
        This project is originally based on the
        `cumulus <https://github.com/cotdsa/cumulus>`__ project. See
        `CUMULUS_LICENSE <https://github.com/humilis/humilis/blob/master/CUMULUS_LICENSE>`__
        for license information.
        
        Installation
        ============
        
        Install the `AWS CLI <https://aws.amazon.com/cli/>`__:
        
        ::
        
           pip install awscli
        
        Configure the AWS CLI:
        
        ::
        
           aws configure
        
        ``humilis`` will use whatever credentials you introduced when
        configuring your AWS CLI installation.
        
        You can now install the latest “stable” version of ``humilis``:
        
        ::
        
           pip install humilis
        
        or the development version if you prefer that:
        
        ::
        
           pip install git+https://github.com/humilis/humilis
        
        After installation you need to configure humilis. To configure globally
        for your system:
        
        ::
        
           humilis configure
        
        The command above will store and read the configuration options from
        ``~/.humilis.ini``. You can also store the configuration in a
        ``.humilis.ini`` file stored in your current working directory by using:
        
        ::
        
           humilis configure --local
        
        ``humilis`` will always read the configuration first from a
        ``.humilis.ini`` file under your current work directory. If it is not
        found then it will read it from your system global config file
        ``~/.humilis``.
        
        Development environment
        =======================
        
        Assuming you have
        `virtualenv <https://virtualenv.readthedocs.org/en/latest/>`__
        installed:
        
        ::
        
           make develop
        
           . .env/bin/activate
        
        Testing
        =======
        
        At the moment, most tests are integration tests with the AWS SDK. This
        means that you will need to `set up your
        system <http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html>`__
        to access AWS resources if you want to run the test suite.
        
        ::
        
           py.test tests
        
        Quickstart
        ==========
        
        Define your infrastructure environment following the examples in the
        `examples
        directory <https://github.com/humilis/humilis/tree/master/examples>`__.
        Then to create the environment:
        
        ::
        
           humilis create examples/humilis-firehose.yaml
        
        To update the environment after it has been deployed:
        
        ::
        
           humilis update examples/humilis-firehose.yaml
        
        And to delete it:
        
        ::
        
           humilis delete examples/humilis-firehose.yaml
        
        Humilis environments
        ====================
        
        A ``humilis`` environment is just a collection of cloudformation stacks
        that are required for an application. Instead of having a monolytic CF
        template for your complete application, ``humilis`` allows you to define
        infrastructure *layers* that are combined into an *environment*. Each
        ``humilis`` layer translates exactly into one CF template (therefore
        into one CF stack after the layer is deployed).
        
        Breaking a complex infrastructure environment into smaller layers has at
        least two obvious advantages:
        
        -  **Easier to maintain**. It’s easier to maintain a simple layer that
           contains just a bunch of `CF
           resources <http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html>`__
           than serve a well-defined purpose.
        
        -  **Easier to reuse**. You should strive to define your infrastructure
           layers in such a way that you can reuse them across various
           environments. For instance, many projects may require a base layer
           that defines a VPC, a few subnets, a gateway and some routing tables,
           and maybe a (managed) NAT. You can define a humilis layer with those
           resources and have a set of layer parameters (e.g. the VPC CIDR) that
           will allow you to easily reuse it across environments.
        
        Environment anatomy
        -------------------
        
        An environment *definition file* is a
        `yaml <https://en.wikipedia.org/wiki/YAML>`__ document that specifies
        the list of layers that form your enviroment. The file should be named
        as your environment. That is, for environment ``my-app-environment`` the
        environment description file should be called
        ``my-app-environment.yaml``. The contents of the environment definition
        should be organized as follows:
        
        ::
        
           ---
           my-app-environment:
               description:
                   A description of what this environment is for
               layers:
                   # The layers that you environment requires. They will be deployed in the
                   # same order as you list them. Note that you can also pass parameters 
                   # to a layer (more on that later).
                   - {layer: name_of_first_layer, layer_param: layer_value}
                   - {layer: name_of_second_layer}
                   - {layer: name_of_third_layer}
        
        Layer anatomy
        -------------
        
        Anything associated to a given layer must be stored in a directory with
        the same name as the layer, within the same directory where the
        environment *definition file* is located. If we consider the
        ``my-app-environment`` environment we used above then your directory
        tree should look like this:
        
        ::
        
           .
           ├── my-app-environment.yaml
           ├── name_of_first_layer
           │   ├── meta.yaml
           │   └── resources.yaml
           ├── name_of_second_layer
           │   ├── meta.json
           │   └── meta.yaml
           └── name_of_third_layer
               ├── resources.json.j2
               └── resources.yaml.j2
        
        A layer must contain at least two files:
        
        -  ``meta.yaml``: Meta information about the layer such as a
           description, and layer parameters.
        -  ``resources.yaml``: Basically a CF template with the resources that
           the layer contains.
        
        Those two files can also be in ``.json`` format (``meta.json`` and
        ``resources.json``). Or you can add the extension ``.j2`` if you want
        the files to be pre-processed with the
        `Jinja2 <http://jinja.pocoo.org/>`__ template compiler.
        
        Below an example of how a layer ``meta.yaml`` may look like:
        
        ::
        
           ---
           meta:
               description:
                   Creates a VPC, that's it
               parameters:
                   vpc_cidr:
                       description: The CIDR block of the VPC
                       value: 10.0.0.0/16
        
        Above we declare only one layer parameter: ``vpc_cidr``. ``humilis``
        will make pass that parameter to Jinja2 when compiling any template
        contained in the layer. So the ``resources.yaml.j2`` for that same layer
        may look like this:
        
        ::
        
           ---
           resources:
               VPC:
                   Type: "AWS::EC2::VPC"
                   Properties:
                       CidrBlock: {{ vpc_cidr }}
        
        References
        ==========
        
        You can use references in your ``meta.yaml`` files to refer to thing
        other than resources within the same layer (to refer to resources within
        a layer you can simply use Cloudformation’s
        `Ref <http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html>`__
        or
        `GetAtt <http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-getatt.html>`__
        functions). Humilis references are used by setting the value of a layer
        parameter to a dict that has a ``ref`` key. Below an a ``meta.yaml``
        that refers to a resource (with a logical name ``VPC``) that is
        contained in another layer (called ``vpc_layer``):
        
        ::
        
           ---
           meta:
               description:
                   Creates an EC2 instance in the vpc created by the vpc layer
               dependencies:
                   - vpc
               parameters:
                   vpc:
                       description: Physical ID of the VPC where the instance will be created
                       value:
                           ref: 
                               parser: layer
                               parameters:
                                   layer_name: vpc_layer
                                   resource_name: VPC
        
        Every reference must have a ``parser`` key that identifies the parser
        that should be used to parse the reference. There are also two optional
        keys:
        
        -  ``parameters``: allows you to pass parameters to the reference
           parser. You can pass either named parameters (as a dict) or
           positional arguments (as a list).
        
        -  ``priority``: the parsing priority. Parameters with a lower value in
           ``priority`` will be parsed before parameters with a higher value.
           This allows some reference parsers to refer internally to other
           parameters within the same layer. For example, the ``lambda`` parser,
           when parsing templated lambda code, it uses previously parsed layer
           parameters as template parameters.
        
        More information on the reference parsers that are bundled with humilis
        below.
        
        Available reference parsers
        ---------------------------
        
        ``layer_resource`` references
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        
        ``layer_resource`` references allow you to refer to the physical ID of a
        resource that is part of another layer.
        
        **Parameters**:
        
        -  ``layer_name``: The name of the layer you are referring to
        -  ``resource_name``: The logical name of the layer resource
        
        **Example**:
        
        Consider the following environment definition:
        
        ::
        
           ---
           my-environment:
               description:
                   Creates a VPC with a NAT in the public subnet
               layers:
                   - {layer: vpc}
                   - {layer: nat}
        
        Obviously the ``nat`` layer that takes care of deploying the NAT in the
        public subnet will need to know the physical ID of that subnet. You
        achieve this by declaring a ``layer_resource`` reference in the
        ``meta.yaml`` for the ``nat`` layer:
        
        ::
        
           ---
           meta:
               description:
                   Creates a managed NAT in the public subnet of the NAT layer
               parameters:
                   subnet_id:
                       description:
                           The physical ID of the subnet where the NAT will be placed
                       value:
                           ref:
                               parser: layer_resource
                               parameters:
                                   layer_name: vpc
                                   # The logical name of the subnet in the vpc layer
                                   resource_name: PublicSubnet
        
        When parsing ``meta.yaml`` humilis will replace this:
        
        ::
        
           ref:
               parser: layer_resource
               parameters:
                   layer_name: vpc
                   # The logical name of the subnet in the vpc layer
                   resource_name: PublicSubnet
        
        with the physical ID you need (something like ``subnet-bafa90cd``). You
        can then use this physical ID in the ``resources.yaml.j2`` section of
        the ``nat`` layer:
        
        ::
        
           {# Pseudo-content of layers/nat/resources.yaml.j2 #}
           resources:
               {# An Elastic IP reservation that will be associated to the NAT #}
               NatEip:
                 Type: 'AWS::EC2::EIP'
                 Properties: {}
               {# Custom resource deploying the NAT #}
               NatGateway:
                 Type: 'Custom::NatGateway',
                 Properties:
                   {# The ARN of the Lambda function backing the custom resource #}
                   ServiceToken: 'arn:aws:lambda:eu-west-1:XXXX:function:CreateNatGateway'
                   {# Here we use the subnet_id reference defined in meta.yaml #}
                   SubnetId: {{subnet_id}}
                   AllocationId:
                       Ref: NatEip
        
        ``environment_resource`` references
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        
        ``environment_output`` references allow you to refer to resources that
        belong to other humilis environments.
        
        **Parameters**:
        
        -  ``environment_name``: The name of the environment you are referring
           to
        -  ``layer_name``: The name of the layer you are referring to
        -  ``resource_name``: The logical name of the layer resource
        
        ``layer_output`` references
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~
        
        ``layer_output`` references allow you to refer to outputs produced by
        another layer.
        
        **Parameters**:
        
        -  ``layer_name``: The name of the layer you are referring to
        -  ``output_name``: The logical name of the output parameter
        
        In general you should prefer using ``layer_output`` references over
        ``layer_resource`` references. The output parameters produced by a layer
        define an informal *layer interface* that is more likely to remain
        constant than the logical names of resources within a layer.
        
        ``boto3`` references
        ~~~~~~~~~~~~~~~~~~~~
        
        ``boto3`` references define arbitrary calls to
        `boto3facade <https://github.com/InnovativeTravel/boto3facade>`__. The
        latter is just a simpler facade interface on top of
        `boto3 <https://github.com/boto/boto3>`__.
        
        **Parameters**:
        
        -  ``service``: The AWS service, e.g. \ ``ec2`` or ``cloudformation``.
           Note that only only AWS services that have a facade in
           `boto3facade <https://github.com/InnovativeTravel/boto3facade>`__ are
           supported.
        -  ``call``: The corresponding facade method,
           e.g. \ ``get_ami_by_name``. The value of this parameter must be a
           dictionary with a ``method`` key (the name of the facade method to
           invoke) and an optional ``args`` key (the parameters to pass to the
           facade method). Best to look at the example below to understand how
           this works.
        -  ``output_attribute``: Optional. If provided the reference parser will
           return the value of this attribute from the object returned by the
           facade method.
        
        Below an example of a layer that uses a ``boto3`` reference:
        
        ::
        
           ---
           meta:
               description:
                   Creates an EC2 instance using a named AMI
               # More stuff omitted for brevity
               ami:
                   description: The AMI to use when launching the EC2 instance
                   value:
                       ref:
                           parser: boto3
                           parameters:
                               service: ec2
                               call:
                                   method: get_ami_by_name
                                   args:
                                       - test-ami
                               output_attribute: id
        
        ``humilis`` will parse the reference using this code:
        
        ::
        
           # Import the Ec2 facade
           from boto3facade.ec2 import Ec2
        
           # Create a facade object
           ec2_facade = Ec2()
        
           # Make the call
           ami = ec2_facade.get_ami_by_name('test-ami')
        
           # Extract the requested attribute
           ref_value = ami.id
        
        ``file`` references
        ~~~~~~~~~~~~~~~~~~~
        
        ``file`` references allow you to refer to a local file. The file will be
        uploaded to S3 and the reference will evaluate to the corresponding S3
        path.
        
        **Parameters**:
        
        -  ``path``: The path to the file, relative to the layer root directory.
        
        ``lambda`` references
        ~~~~~~~~~~~~~~~~~~~~~
        
        ``lambda`` references allow you to refer to some Python code in your
        local machine. If your code follows some simple conventions ``humilis``
        will take care of building a `deployment
        package <http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html>`__
        for you, uploading it to S3, and the reference will evaluate to the S3
        path of the deployment package.
        
        **Parameters**:
        
        -  ``path``: Path to either a completely self-contained ``.py`` file, or
           to the root directory of your lambda code. In the latter case your
           code needs to follow some simple conventions for this to work. More
           information below.
        
        -  ``dependencies``: A list of dependencies to be included in the Lambda
           deployment package. Dependencies may be either pip installable
           packages, or paths to local Python packages or modules, or paths to
           local ``requirements`` files.
        
        **Example**:
        
        ::
        
           ref:
               parser: lambda
               parameters:
                   # Path to the root directory containing your lambda code
                   path: dummy_function
                   dependencies:
                       # The Lambda code requires Pypi's pyyaml
                       - pyyaml
                       # It also requires a local package in this path
                       - mycode/mypkgdir
                       # And this local module
                       - mycode/mymodule.py
        
        which will evaluate to a S3 path such as:
        
        ::
        
           s3://[bucket_name]/[environment_name]/[stage_name]/[func_name]-[commithash].zip
        
        **Code conventions**:
        
        Following the example above, the contents of the layer responsible of
        deploying the ``dummy_function`` lambda may look like this:
        
        ::
        
           .
           ├── dummy_function
           │   ├── dummy_function.py
           │   └── setup.py
           ├── meta.yaml
           ├── outputs.yaml.j2
           └── resources.yaml.j2
        
        Basically all your code needs to be included under directory
        ``dummy_function``. In this case there is only one file:
        ``dummy_function.py``. External dependencies need to be specified in
        your ``setup.py``.
        
        ``secret`` references
        ~~~~~~~~~~~~~~~~~~~~~
        
        ``secret`` references retrieve a secret using Python’s
        `keyring <https://pypi.python.org/pypi/keyring>`__ module.
        
        **Parameters**:
        
        -  ``service``: The name of the service the secret is associated to.
        
        -  ``key``: The key (e.g. the username) that identifies the secret.
        
        **Example**:
        
        ::
        
           ref:
               parser: secret
               parameters: {"service": "mysqldb", "key": "adminuser"}
        
        Custom Jinja2 filters
        ---------------------
        
        Humilis defines the following `custom Jinja2
        filters <http://jinja.pocoo.org/docs/dev/templates/#filters>`__:
        
        -  ``uuid``: A random UUID. Example: ``{{''|uuid}}``.
        -  ``password(length=8)``: A random password. Example:
           ``{{10|password}}``.
        
        .. |Build Status| image:: https://travis-ci.org/humilis/humilis.svg?branch=master
           :target: https://travis-ci.org/humilis/humilis
        .. |PyPI| image:: https://img.shields.io/pypi/v/humilis.svg?style=flat
           :target: https://pypi.python.org/pypi/humilis
        
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 3
