Lamvery
=======

|Build Status| |Coverage| |Scrutinizer Code Quality| |Version|

Description
===========

User-friendly deploy and management tool for AWS Lambda function.

Why user-friendly?
~~~~~~~~~~~~~~~~~~

The format of the configuration file is ``YAML``\ (with ``Jinja2``)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

-  ``YAML`` is user-friendly than ``JSON``
-  We can avoid some redundant contents by ``Jinja2`` template engine

Additional features that are not in the standard Lambda functions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

-  Passing the environment variables
-  Passing the confidential information use ``KMS`` encryption
-  Passing the confidential files use ``KMS`` encryption
-  And more

More useful features for deploying and invoking our functions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

-  We can build, configure, deploy, rollback and invoke with a single
   command
-  We can rollback **correctly** to the previous version of the alias
-  Deply(build) hooks
-  And more

More useful features for using and managing the related services
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

-  API Gateway (Swagger)
-  CloudWatch Events
-  CloudWatch Logs
-  And more

Requirements
============

-  Python2.7
-  pip

Recommends
==========

-  virtualenv
   **Automatically collect the lightweighted and compiled libraries in
   the virtualenv environment.**

Installation
============

PyPI
----

.. code:: sh

    pip install lamvery

Apt
---

.. code:: sh

    echo "deb https://dl.bintray.com/willyworks/deb trusty main" | sudo tee -a /etc/apt/sources.list
    sudo apt-get update
    sudo apt-get install lamvery
    export PATH=/opt/lamvery/bin:$PATH

Yum
---

.. code:: sh

    echo "
    [bintraybintray-willyworks-rpm]
    name=bintray-willyworks-rpm
    baseurl=https://dl.bintray.com/willyworks/rpm/centos/\$releaserver/\$basearch/
    gpgcheck=0
    enabled=1
    " | sudo tee -a /etc/yum.repos.d/bintray-willyworks-rpm.repo
    sudo yum install lamvery
    export PATH=/opt/lamvery/bin:$PATH

Setup and configuration
=======================

First,

.. code:: sh

    lamvery init

| And then edit your ``.lamvery.yml`` like so.
| The configuration is written in YAML syntax with ``jinja2`` template.
| Environment variables are stored in the ``env`` variable.

General settings (default: ``.lamvery.yml``)
--------------------------------------------

.. code:: yml

    profile: private
    region: us-east-1
    versioning: true
    default_alias: test
    clean_build: false
    configuration:
      name: lamvery-test
      runtime: python2.7
      role: {{ env['AWS_LAMBDA_ROLE'] }}
      handler: lambda_function.lambda_handler
      description: This is sample lambda function.
      timeout: 10
      memory_size: 128
      environment_variables:
        MYENV: foobar
      vpc_config:
        subnets:
        - subnet-cadf2993
        security_groups:
        - sg-4d095028

profile
~~~~~~~

The name of a profile to use. If not given, it depends on ``boto3``.

region
~~~~~~

| The region name of your environment.
| If you doesn't set this option, it depends on ``boto3``.

versioning
~~~~~~~~~~

Enable the function versioning.

default\_alias
~~~~~~~~~~~~~~

The alias when it has not been specified in the ``-a`` or ``--alias``
option.

clean\_build
~~~~~~~~~~~~

Build the archive(zip) in the temporary area.

configuration
~~~~~~~~~~~~~

-  | name
   | The name of your function.

-  | runtime
   | The runtime environment for the Lambda function you are uploading.
   | Currently, ``lamvery`` supports ``python2.7`` and ``nodejs``.

-  | role
   | The Amazon Resource Name (ARN) of the IAM role for your function.

-  | handler
   | The function within your code that Lambda calls to begin execution.

-  | description
   | The description of your function.

-  | timeout
   | The function execution time(seconds) at which Lambda should
     terminate the function.

-  | memory\_size
   | The amount of memory for your function environment.

-  | environment\_variables
   | Environment variables.

-  vpc\_config
   The VPC configurations for the function to access resources in your
   VPC.
-  subnets
   The Subnet ids in your VPC.
-  | security\_groups
   | The SecurityGroup ids in your VPC.

CloudWatch Events settings (default: ``.lamvery.event.yml``)
------------------------------------------------------------

.. code:: yml

    rules:
    - name: foo
      description: bar
      schedule: 'rate(5 minutes)'
      targets:
      - id: test-target-id
        input:
          this:
          - is: a
          - sample: input

rules
~~~~~

CloudWatch Event Rules.

-  | NAME
   | The name of CloudWatch Event Rule.

-  | description
   | The description of CloudWatch Event Rule.

-  | schedule
   | The schedule expression of CloudWatch Event Rule.

-  | disabled
   | When this setting is true, change the state of CloudWatch Event
     Rule to ``DISABLED``.
   | default: ``false``

-  targets
   The targets of CloudWatch Event Rule.
-  id
   The unique target assignment ID.
-  input
   Arguments passed to the target.
-  | input\_path
   | The value of the JSONPath that is used for extracting part of the
     matched event when passing it to the target.

*``input`` and ``input_path`` are mutually-exclusive and optional
parameters of a target.*

Secret informations (default: ``.lamvery.secret.yml``)
------------------------------------------------------

.. code:: yml

    key_id: {{ env['AWS_KMS_KEY_ID'] }}
    cipher_texts:
      foo: CiC4xW9lg7HaxaueeN+d9yJMyY1uw1i7tYVvQz9I8+e2UBKXAQEBAgB4uMVvZYOx2sWrnnjfnfciTMmNbsNYu7WFb0M/SPPntlAAAABuMGwGCSqGSIb3DQEHBqBfMF0CAQAwWAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAzWTJWk/69T8NTBquoCARCAK2Hg2de71hzwjiMKkfMSG2G1Olj1EjxajS+3PsFVTPZ91Oi/AjR1aMqAI8U=

key\_id
~~~~~~~

The ID of your encryption key on KMS.

cipher\_texts
~~~~~~~~~~~~~

The name and cipher texts to pass to the lambda function.

secret\_files
~~~~~~~~~~~~~

The filename and the encrypted contents to pass to the lambda function.

Excluded patterns from the archive (default: ``.lamvery.exclude.yml``)
----------------------------------------------------------------------

.. code:: yml

    - ^\.lamvery\.yml$
    - ^\.lamvery\.event\.yml$
    - ^\.lamvery\.secret\.yml$
    - ^\.lamvery\.exclude\.yml$

Exclude files or directories using regular expression.

Action hooks (default: ``.lamvery.hook.yml``)
---------------------------------------------

.. code:: yml

    build:
      pre:
      - pip install -r requirements.txt -t ./
      post: []

build
~~~~~

The hooks for ``build``\ (and ``deploy``)

-  | pre
   | The commands that run before building.

-  | post
   | The commands that run after building.

API Gateway integration (default: ``.lamvery.api.yml``)
-------------------------------------------------------

.. code:: yml

    api_id: myipugal74
    stage: dev
    cors:
      origin: '*'
      methods:
      - GET
      - OPTION
      headers:
      - Content-Type
      - X-Amz-Date
      - Authorization
      - X-Api-Key
    configuration:
      swagger: '2.0'
      info:
        title: Sample API
      schemes:
      - https
      paths:
        /:
          get:
            produces:
            - application/json
            parameters:
            - name: sample
              in: query
              required: false
              type: string
            responses:
              '200':
                description: 200 response
                schema:
                  $ref: '#/definitions/Sample'
      definitions:
        Sample:
          type: object

api\_id
~~~~~~~

| The id of your REST API.
| This is written on automatically when you deployed your API with the
  ``-w`` option.

stage
~~~~~

The name of the stage in API Gateway.

cors
~~~~

| CORS (Cross-Origin Resource Sharing) options.
| If you did not set the ``x-amazon-apigateway-integration`` option,
  these are set automatically.

-  origin
   For the response's header that named "Access-Control-Allow-Origin".
-  headers
   For the response's header that named "Access-Control-Allow-Headers".
-  methods
   For the response's header that named "Access-Control-Allow-Methods".

configuration
~~~~~~~~~~~~~

| The settings of your APIs written in the ``Swagger`` format.
| If you did not set the ``x-amazon-apigateway-integration`` option,
  these are set automatically.

Commands
========

build
~~~~~

-  Build and archive your code and libraries to
   ``<your-function-name>.zip``
-  Store the secret informations to the archive

.. code:: sh

    lamvery build [-e <env-name>=<env-value>]

deploy
~~~~~~

-  Build and deploy your code and libraries
-  Store the secret informations to the archive
-  Update configuration of the function
-  Set alias to a version of the function

.. code:: sh

    lamvery deploy [-e <env-name>=<env-value>] [-a <alias>]

rollback
~~~~~~~~

-  Rollback to the previous version of the function

**You must do one of the following to use this command.**

-  Deploy with ``publish(-p,--publish)`` and ``alias(-a, --alias)``
   options.
-  Turn on(true) ``versioning`` and set a value to ``default_alias`` in
   the configuration file.

::

    lamvery rollback [-a <alias>]

set-alias
~~~~~~~~~

-  Set alias to a version of the function

.. code:: sh

    lamvery set-alias -a <alias> -v <alias-version>

encrypt
~~~~~~~

-  Encrypt a text value using KMS

.. code:: sh

    lamvery encrypt [-s] -n <secret-name> <secret-value>

encrypt-file
~~~~~~~~~~~~

-  Encrypt a file using KMS

.. code:: sh

    lamvery encrypt [-s] -n <filename> <local-file-path>

decrypt
~~~~~~~

-  Decrypt the secret information using KMS

.. code:: sh

    lamvery decrypt -n <secret-name>

events
~~~~~~

-  Apply CloudWatch Events settings

.. code:: sh

    lamvery events [-k] [-a <alias>]

invoke
~~~~~~

-  Invoke the function and output result

.. code:: sh

    lamvery invoke [-a <alias>] [-v <version>] '{"foo": "bar"}'

or

.. code:: sh

    lamvery invoke [-a <alias>] [-v <version>] path/to/input.json

logs
~~~~

-  Watch the log events of the function

.. code:: sh

    lamvery logs [-f] [-F <filter>] [-s <start-time-string>] [-i <interval-seconds>]

api
~~~

-  Manage your APIs

.. code:: sh

    lamvery api [-n] [-r] [-s <stage-name>] [-w]

generate
~~~~~~~~

-  Generate skeleton files

.. code:: sh

    lamvery generate -k <kind-of-file>

Options
-------

``-a`` or ``--alias``
~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the
  ``deploy``,\ ``set-alias``,\ ``invoke``,\ ``rollback``,\ ``events``,\ ``api``
  commands.
| Alias for a version of the function.

``-c`` or ``--conf-file``
~~~~~~~~~~~~~~~~~~~~~~~~~

| This option needed by all commands.
| Specify the configuration file.
| default: ``.lamvery.yml``

``-d`` or ``--dry-run``
~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the
  ``deploy``,\ ``alias``,\ ``rollback``,\ ``events``,\ ``api`` commands.
| Output the difference of configuration and the alias without updating.

``-s`` or ``--single-file``
~~~~~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``archive`` and ``deploy`` command.
| Archive only the lambda function file, so you can inline edit in the
  AWS Management Console.

``-l`` or ``--no-libs``
~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``archive`` and ``deploy`` command.
| Archive without all libraries.

``-n`` or ``--secret-name``
~~~~~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``encrypt`` and ``decrypt`` commands.
| The name of the secret value.

``-p`` or ``--publish``
~~~~~~~~~~~~~~~~~~~~~~~

This option is only needed by the ``deploy`` command. Publish the
version as an atomic operation.

``-k`` or ``--keep-empty-events``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This option is only needed by the ``events`` command. Keep the empty
CloudWatch Event Rule that does not have CloudWatch Event Target.

``-s`` or ``--store``
~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``encrypt`` command.
| Store encripted value to configuration file (default:
  ``.lamvery.yml``).
| Requires the ``-n`` or ``--secret-name`` option.

``-v`` or ``--version``
~~~~~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``set-alias``,\ ``invoke``,\ ``rollback``
  commands.
| Version of the function.

``-f`` or ``--follow``
~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``logs`` command.
| Watch the log events and updates the display (like ``tail -f``).

``-F`` or ``--filter``
~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``logs`` command.
| Filtering pattern for the log messages.

``-i`` or ``--interval``
~~~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``logs`` command.
| Intervals(seconds) to watch the log events.

``-s`` or ``--start``
~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``logs`` command.
| Time to start the log events watching.
| Examples: ``yesterday``,\ ``"-1 h"``, ``"2016-01-01"``,
  ``"2016-01-01 10:20:30"``

``-t`` or ``--target``
~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``set-alias`` command.
| The alias of the version that is targeted for setting alias.

``-e`` or ``--env``
~~~~~~~~~~~~~~~~~~~

| This option is needed by the ``archive`` and ``deploy`` commands.
| Environment variables that pass to the function.
| **This option can be used repeatedly to pass multiple variables.**
| Examples: ``FOO=BAR``

``-r`` or ``--remove``
~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``api`` command.
| Remove your APIs.

``-s`` or ``--stage``
~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``api`` command.
| The name of the stage in API Gateway.

``-w`` or ``--write-id``
~~~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``api`` command.
| Write the id of your API to the configuration file (default:
  ``.lamvery.api.yml``)

``-n`` or ``--no-integrate``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``api`` command.
| Without automatic integration.

``-k`` or ``--kind``
~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``generate`` command.
| The kind of the file.
| Allowed values: ``function``

``-n`` or ``--name``
~~~~~~~~~~~~~~~~~~~~

| This option is only needed by the ``encrypt-file`` command.
| The name of the secret file.

How to use the confidential information in the lambda function
==============================================================

1. Create key on KMS
~~~~~~~~~~~~~~~~~~~~

See:
https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html

2. Create IAM role for lambda function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Policy example:

.. code:: json

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "kms:Decrypt"
                ],
                "Resource": [
                    "arn:aws:kms:us-east-1:<your-account-number>:key/<your-key-id>"
                ]
            }
        ]
    }

3. Set the key-id to your configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Configuration example:

-  .lamvery.yml

.. code:: yml

    profile: default
    region: us-east-1
    versioning: false
    default_alias: null
    configuration:
      name: sample_lambda_function
      runtime: python2.7 # or nodejs
      role: arn:aws:iam::000000000000:role/lambda_basic_execution
      handler: lambda_function.lambda_handler
      description: This is sample lambda function.
      timeout: 10
      memory_size: 128

-  .lamvery.secret.yml

.. code:: yml

    key_id: xxxx-yyyy-zzzz # <-here!
    cipher_texts: {}
    secret_files: {}

4. Encrypt and store the confidential information to your configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

    lamvery encrypt -s -n foo "This is a secret"

5. Write your function
~~~~~~~~~~~~~~~~~~~~~~

Generate the skeleton function by this command.

.. code:: sh

    lamvery generate -k function

Code example:

-  Python

.. code:: py

      import lamvery

      def lambda_handler(event, context):
          print(lamvery.secret.get('foo'))

-  Node.js

.. code:: js

    var lamvery = require('./lamvery.js');

    exports.lambda_handler = function(event, context) {
        lamvery.secret.get('foo', function(err, data) {
            console.log(data);
        });
    }

6. Deploy your function
~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

    lamvery deploy

7. Invoke your function
~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

    lamvery invoke {}

Result example:

::

    START RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d Version: $LATEST
    This is a secret
    END RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d

How to use the confidential file in the lambda function
=======================================================

1. Create key on KMS
~~~~~~~~~~~~~~~~~~~~

See:
https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html

2. Create IAM role for lambda function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Policy example:

.. code:: json

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "kms:Decrypt"
                ],
                "Resource": [
                    "arn:aws:kms:us-east-1:<your-account-number>:key/<your-key-id>"
                ]
            }
        ]
    }

3. Set the key-id to your configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Configuration example:

-  .lamvery.yml

.. code:: yml

    profile: default
    region: us-east-1
    versioning: false
    default_alias: null
    configuration:
      name: sample_lambda_function
      runtime: python2.7 # or nodejs
      role: arn:aws:iam::000000000000:role/lambda_basic_execution
      handler: lambda_function.lambda_handler
      description: This is sample lambda function.
      timeout: 10
      memory_size: 128

-  .lamvery.secret.yml

.. code:: yml

    key_id: xxxx-yyyy-zzzz # <-here!
    cipher_texts: {}
    secret_files: {}

4. Encrypt and store the confidential file to your configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

    lamvery encrypt-file -s -n foo.txt /path/to/local/confidential/file

5. Write your function
~~~~~~~~~~~~~~~~~~~~~~

Generate the skeleton function by this command.

.. code:: sh

    lamvery generate -k function

Code example:

-  Python

.. code:: py

      import lamvery

      def lambda_handler(event, context):
          print(open(lamvery.secret.file('foo.txt'), 'r').read())

-  Node.js

.. code:: js

    var lamvery = require('./lamvery.js');

    exports.lambda_handler = function(event, context) {
        lamvery.secret.file('foo.txt', function(err, path) {
            fs.readFile(path, 'utf-8', function(err, txt) {
                console.log(txt);
            });
        });
    }

6. Deploy your function
~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

    lamvery deploy

7. Invoke your function
~~~~~~~~~~~~~~~~~~~~~~~

Command example:

.. code:: sh

    lamvery invoke {}

Result example:

::

    START RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d Version: $LATEST
    This is a secret file
    END RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d

Development
-----------

-  Source hosted at `GitHub <https://github.com/marcy-terui/lamvery>`__
-  Report issues/questions/feature requests on `GitHub
   Issues <https://github.com/marcy-terui/lamvery/issues>`__

Pull requests are very welcome! Make sure your patches are well tested.
Ideally create a topic branch for every separate change you make. For
example:

1. Fork the repo
2. Create your feature branch (``git checkout -b my-new-feature``)
3. Commit your changes (``git commit -am 'Added some feature'``)
4. Push to the branch (``git push origin my-new-feature``)
5. Create new Pull Request

Authors
-------

Created and maintained by `Masashi
Terui <https://github.com/marcy-terui>`__ (marcy9114@gmail.com)

License
-------

MIT License (see
`LICENSE <https://github.com/marcy-terui/lamvery/blob/master/LICENSE>`__)

.. |Build Status| image:: https://img.shields.io/travis/marcy-terui/lamvery/master.svg
   :target: http://travis-ci.org/marcy-terui/lamvery
.. |Coverage| image:: https://img.shields.io/coveralls/marcy-terui/lamvery.svg
   :target: https://coveralls.io/github/marcy-terui/lamvery
.. |Scrutinizer Code Quality| image:: https://scrutinizer-ci.com/g/marcy-terui/lamvery/badges/quality-score.png?b=master
   :target: https://scrutinizer-ci.com/g/marcy-terui/lamvery/?branch=master
.. |Version| image:: https://img.shields.io/pypi/v/lamvery.svg
   :target: https://pypi.python.org/pypi/lamvery
