# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['serverbackups', 'serverbackups.examples', 'serverbackups.templates']

package_data = \
{'': ['*']}

install_requires = \
['PyYAML>=5.0.0,<6.0.0',
 'click>=7.1.0,<7.2.0',
 'colored>=1.4.0,<1.5.0',
 'prettyprinter>=0.18.0,<0.19.0']

entry_points = \
{'console_scripts': ['serverbackups = mreschke.serverbackups.init:cli']}

setup_kwargs = {
    'name': 'mreschke-serverbackups',
    'version': '1.0.7',
    'description': 'Config driven server backups that feel like a simple backup.py script',
    'long_description': '# Server Backups\n\nConfig driven server backups that feel like a simple backup.py script\n\n* PyPi: https://pypi.org/project/mreschke-serverbackups\n* Github: https://github.com/mreschke/server-backups\n\n\n# Installation\n\n```bash\npip install mreschke-serverbackups\n```\n\n\n# Usage\n\nAll backups are initiated from a simple `backups.py` file.  Create this file with:\n\n```bash\ncd /usr/local/bin\nserverbackups init-cli-script\n```\n\nThis `backups.py` can hold your server backup definitions as a dictionary, fully self contained.\nOr you can choose to use `yaml` files in `/etc/serverbackups` to separate each server in its own `/etc/serverbackups/config.d` file.\nTo initialize a stock `/etc/serverbackups` directory run\n\n```bash\nserverbackups init-directory\n\n# Defaults to /etc/serverbackups.  Optionally, set a different directory\nserverbackups init-directory --path /etc/someother/dir\n```\n\nRead below for how to modify the configs.  Once you have a good configuration setup you can run the\nbackups like so\n\n```bash\n# Run all servers AND custom python code\n./backups.py run --all\n\n# Run all servers with NO custom python code\n./backups.py run --servers\n\n# Run a single server (no custom python code)\n./backups.py run --server=myserver\n\n# Run all servers defined in the same cluster (no custom python code)\n./backups.py run --cluster=webfarm\n\n# Run only custom code in backups.py, NOT the server definition arrays\n./backups.py run --custom\n```\n\n\n\n# Basic Idea\n\nThe basic idea is to replace your quick and simple backup `bash` script with a more\ncomprehensive but still simple and readable `backups.py` file.  This file can backup common items\nsuch as files and databases using `rsync` hard-link style snapshots in a simple\nconfig driven manner.  Instead of hacking bash scripts to rsync properly you can use\na simple config instead.  At the end of `backups.py` after all backups have executed\nfrom this simple dictionary/yaml definition you can still code your own functionality\nright in python as you normally would in a bash script.  This gives you the power\nof a script with the added benefit if easy config driven common backup routines.\n\nWhether you are using inline `backups.py` dictionaries to define your server or\nusing `/etc/serverbackups/config.d/*` files the concept of how the system works\nis the same.\n\nYou define an array of server to backup.  Each server has many options including\nsource, destination, files to backup and exclude, databases to backup and pre/post\nscripts to run.\n\n\n## Backup Strategy\n\nI use `rsync` and `ssh` for all backups local or remote.  To cut down on storage costs I am utilizing\nrsync hard-link style snapshots.  This means your snapshot folder looks full and complete each day (not\na diff or incremental look) but in actuality most of those files are hard linked to previous snapshots.\nThis means the storage used on the backup server is only the size of all unique files combined + the ones\npreviously deleted.  Total storage is NOT every days worth of files, that would be HUGE.\nI adapted the code from http://anouar.adlani.com/2011/12/how-to-backup-with-rsync-tar-gpg-on-osx.html\nto accomplish this.  The trick is in the `rsync --hard-links --link-dest` parameters.  This storage saving\ntrick allows you to have a generous purge strategy, keeping years worth of weekly, monthly and yearly backups\nwithout sacrificing storage.\n\n\n## Defaults\n\nMost of the settings for a single server are redundant and can therefore be\ndeduplicated in to what I call a `defaults` config.  The `defaults` config\nis `merged` with **each** server definition in your array.  This keeps your server definition\nsmall and clean.  `Defaults` and each of your server configs are deep merged which\nallows you to override small pieces of the `defaults` for each server as needed.\n\nDefaults are completely **optional** and are there to help cut down on redundant configuration.  If\nyou are simply backing up a single server, just define the entire complete server options and forget\na \'defaults\' definition.\n\n**NOTICE - Sensible Defaults Applied**\n\nIf you do not define your own \'defaults\', a sensible \'defaults\' are still applied to keep even a single\nserver definition less verbose.  Run `serverbackups show-builtin-defaults` to see the `defaults` that are\nstill being applied here.  For example you don\'t need to define the prune rates, ssh port or ssh key\nif the `defaults` are acceptable.\n\nBecause of the `defaults` system, your actual server definitions can be much smaller.  For example, here\nis a minimal server definition that inherits from `defaults`\n\nIn yaml format\n```yaml\nmyloadbalancer:\n  source: {ssh: {host: haproxy.example.com}}\n  backup:\n    files:\n      exclude:\n        - /var/log/haproxy_info.log\n```\n\n\n## Servers\n\nEach server is defined in a python dictionary or yaml file in `/etc/serverbackups/config.d`\n**ending in .yml**.  Before the backups run each server is `deep merged` with your `defaults` config to\ncreate a full and rich server definition.\n\n\n## Files\n\nThe `files` section of each server definition allows you to backup files and folders obeying the `exclude`\narray.  The only difference between `common` and `extra` is that `common` was meant to be used in the `defaults`\nsection while `extra` was there just for you to add to each of your servers.  Technically you can still add\n`common` to your server as arrays are APPENDED during the `default deep merge algorithm`.  Exclude is also\nappended during merge so you can add more excludes per server while maintaining those you defined in `defaults`.\n\n\n## Scripts\n\nThe script section of each server definition allows you to run arbitrary bash scripts on the destination.\nThe `output` of the scripts is optional.  If present, the output is saved to the snapshot folder.  This is\nhandy to backup the output of `dpkg -l` command for example.\n\nThere are 2 script sections, `preScripts` and `postScripts`.  The pre scripts run before any other backups\nfor that particular server.  Post scripts run after all backups.\n\nExample.  I want to backup my `gitlab` server.  I really don\'t want to export my repositories and dig into\ntheir postgres database manually.  Gitlab provides a command `gitlab-backup create` that does it all for me\nand saves the backups to `/var/opt/gitlab/backups`.  Using a pre script to run this command before your `files`\ndefinition grabs `/var/opt/gitlab/backups` is a good example of a pre script.  Gitlab does not clean up their\nown backups so using a post script to remove `/var/opt/gitlab/backups/*.gz` is a good example of a post script.\nBoth of which can either store the output of the command to the snapshot, or ignore output with `output=None`.\n\n\n\n\n## Push Me Pull You\n\nServerbackups can run\n* `local -> local`,\n* `local -> remote` (push via SSH and rsync)\n* `remote -> local` (pull via SSH and rsync)\n\nIf you just want to backup your laptop to your server, probably the `local -> remote` is simplest.  If you\nuse `serverbackups` in a corporate environment like I do you probably want a single backup server that\n**pulls** backups from all your servers and VMS.  Pulling has the advantage of not having to put your\nSSH keys on the backup server.  So all VMs cannot SSH into the backup server.  Pulling means the backup\nserver needs access to all VMs but not the other way around.\n\nIn my corporate environment I have a single backup server which has SSH access into the `root` account\non every server and VM.  I use the file based `/etc/serverbackups` configuration.  Each server has\na file in `/etc/serverbackups/config.d/someserver.yml`.  Using single files for each server helps\nmanage the backups when you have hundreds of servers.  For best results, keep these backup files in ansible for revision control and easy deployments.\n\n\n# Examples\n\n\n## Standalone backups.py\n\nIf you want to define your entire backup plan including the `defaults` config in a single backup.py\n\n```bash\ncd /usr/local/bin\nserverbackups init-cli-script\nchmod a+x backups.py\n```\n\nEdit this new file and adjust line #1 (the shebang to your python3 location\n\nEdit the `defaults` and `servers` dictionary adding all your servers as needed.\n\n\n\n\n\nCreates the following stubbed example\n\n```python\n#!/usr/bin/env python\nfrom datetime import date\nfrom mreschke.serverbackups import Backups, cli, log\n\n# Configure logger\nlog.init({\n    \'console\': {\n        \'level\': \'INFO\',  # DEBUG, INFO, WARNING, ERROR, CRITICAL\n    },\n    \'file\': {\n        \'enabled\': True,\n        \'file\': \'/tmp/backups-{}.log\'.format(date.today().strftime(\'%Y-%m-%d\')),\n    }\n})\n\n# Set per server defaults.  Each server can override parts of this default.\ndefaults = {\n    \'enabled\': True,\n    \'cluster\': None,\n\n    # Keep all daily up to X days back, last snapshot in a week X weeks back, last snapshot in a month X months back...\n    \'prune\': {\n        \'keepDaily\': 30,\n        \'keepWeekly\': 24,\n        \'keepMonthly\': 60,\n        \'keepYearly\': 10,\n    },\n\n    # Rsync options\n    \'rsync\': {\n        \'verbose\': True,\n    },\n\n    # Source server connection details\n    \'source\': {\n        \'location\': \'local\',  # local, ssh,\n        \'ssh\': {\n            # Only used with location=ssh\n            \'host\': \'server.example.com\',  # or IP address,\n            \'port\': 22,\n            \'user\': \'root\',\n            \'key\': \'~/.ssh/id_rsa\',\n        },\n    },\n\n    # Destination server connection details\n    \'destination\': {\n        \'location\': \'local\',  # local, ssh,\n        \'path\': \'/mnt/backups\',\n        \'ssh\': {\n            # Only used with location=ssh\n            \'host\': \'server.example.com\',  # or IP address,\n            \'port\': 22,\n            \'user\': \'root\',\n            \'key\': \'~/.ssh/id_rsa\',\n        },\n    },\n\n    # Backup items\n    \'backup\': {\n        # Pre scripts run on destination before this servers backups, a good place to prep files for backup\n        \'preScripts\': {\n            \'gitlab\': {\'script\': \'sudo gitlab-backup create\', \'output\': \'gitlab-backup.txt\', \'enabled\': False},\n        },\n\n        # Files to backup on the destination.  Use extra in your actual server definition for extra files.\n        # Arrays are merged with defaults, so you could append to common and exclude as well\n        \'files\': {\n            \'common\': [\n                \'/etc/\',\n                \'/usr/local/bin/\',\n                \'/root/\',\n                \'/home/\',\n                \'/var/log/\',\n                \'/var/spool/cron/\',\n            ],\n            \'extra\': [],\n            \'exclude\': [\n                \'lost+found\',\n                \'.Trash-1000\',\n            ],\n        },\n\n        # MySQL Databases and Tables\n        # Host and port are relative inside the server itself via ssh, not remote\n        \'mysql\': {\n            \'enabled\': False,\n            \'mysqlCmd\': \'mysql\',\n            #\'mysqlCmd\': \'docker exec -i mysql mysql\',\n            \'dumpCmd\': \'mysqldump\',\n            \'dumpFlags\': \'--quick --single-transaction --flush-logs\',\n            #\'dumpCmd\': \'docker exec -i mysql mysqldump\',\n            \'host\': \'127.0.0.1\',\n            \'port\': 3306,\n            \'user\': \'root\',\n            \'password\': \'unknown\',\n            # All databases minus exclusions\n            \'dbs\': \'*\',\n            # A list of databases (all tables)\n            # \'dbs\': [\'db1\', \'db2\'],\n            # A list of database with a selection of tables\n            # \'dbs\': [\n            #     {\'name\': \'db1\', \'tables\': [\'table1\', \'table2\']},\n            #     {\'name\': \'db2\', \'tables\': \'*\'},\n            # ],\n            \'excludeDbs\': [\n                \'information_schema\',\n                \'performance_schema\',\n                \'lost+found\',\n                \'#mysql50#lost+found\',\n            ],\n        },\n\n        # Post scripts run on destination after this servers backups, a good place to cleanup files\n        \'postScripts\': {\n            \'dpkg\': {\'script\': \'dpkg -l | gzip\', \'output\': \'dpkg.gz\', \'enabled\': False},\n        },\n    },\n}\n\n# Define each server to backup (each will be merged with defaults config)\nservers = {\n    \'myserver.example.com\': {\n        \'enabled\': True,\n        \'cluster\': \'localhost\',\n        # Example of local -> local backup\n        \'source\': {\n            \'location\': \'local\',\n        },\n        \'destination\': {\n            \'location\': \'local\',\n        },\n        \'backup\': {\n            \'preScripts\': {\n                #\'gitlab\': {\'enabled\': True}\n            },\n            \'mysql\': {\n                \'enabled\': True,\n                \'mysqlCmd\': \'mysql\',\n                #\'mysqlCmd\': \'docker exec -i mysql mysql\',\n                \'dumpCmd\': \'mysqldump\',\n                \'dumpFlags\': \'--quick --single-transaction --flush-logs\',\n                #\'dumpCmd\': \'docker exec -i mysql mysqldump\',\n                \'host\': \'127.0.0.1\',\n                \'port\': 3306,\n                \'user\': \'root\',\n                \'password\': \'password\',\n                \'dbs\': \'*\',\n            },\n            \'postScripts\': {\n                # Export dpkg package list of Linux Debian\n                \'dpkg\': {\'enabled\': True},\n                # Example of backing up postgres database\n                #\'fusionauth\': {\'script\': \'cd /tmp/ && sudo -u postgres pg_dump fusionauth | gzip\', \'output\': \'fusionauth.sql.gz\', \'enabled\': True},\n                # Example of capturing script output to snapshot\n                #\'test2\': {\'script\': \'ls -Hhal /etc/\', \'output\': \'etc.txt\', \'enabled\': True},\n            }\n        },\n    },\n    \'myserver2.example.com\': ...\n}\n\n# Initialize Backup Class\n# If you are defining servers in a dictionary above\n#backups = Backups(servers=servers)\n\n# If you are defining both servers and defaults above\nbackups = Backups(servers=servers, defaults=defaults)\n\n# If you are using a config.d directory (/etc/serverbackups by default)\n#backups = Backups()\n#backups = Backups(config_path=\'/etc/alternative/dir\')\n\n# Run Backups\ncli.start(backups)\n\n\n# Custom backups\nif not cli.allowcustom(): cli.handle(); exit()\nlog.header("Running custom backups from " + __file__)\n#################################################\n# Add your custom backups code here\n#################################################\nprint(\'custom stuff here, any python code you want\')\n\n\n# Handle CLI requests\ncli.handle()\n```\n\nRunning `backups.py` can be read at the top of this readme.\n\n\n## Directory based /etc/serverbackups/config.d\n\nIf you have a lot of servers, a single backup.py with a massive server dictinary soon becomes a bit\nunwieldy.  As an alternative you can define your `defaults` and `server` configs as separate files in a\ndirectory (normally `/etc/serverbackups`)\n\n\nCreate the directory with a example `defaults.yml` and example `config.d/myserver.example.com` file\n\n```bash\nserverbackups init-directory\n\n# Defaults to /etc/serverbackups.  Optionally, set a different directory\nserverbackups init-directory --path /etc/someother/dir\n```\n\nYou still need a `backups.py` which initiates the backups, see the top of the readme to create one.\n\nNow your `backups.py` will look something like this\n\n```python\n#!/usr/bin/env python\nfrom datetime import date\nfrom mreschke.serverbackups import Backups, cli, log\n\n# Configure logger\nlog.init({\n    \'console\': {\n        \'level\': \'INFO\',  # DEBUG, INFO, WARNING, ERROR, CRITICAL\n    },\n    \'file\': {\n        \'enabled\': True,\n        \'file\': \'/tmp/backups-{}.log\'.format(date.today().strftime(\'%Y-%m-%d\')),\n    }\n})\n\n# Initialize Backup Class\nbackups = Backups()\n\n# Or if using an alternate directory\n#backups = Backups(config_path=\'/etc/alternative/dir\')\n\n# Run Backups\ncli.start(backups)\n\n\n# Custom backups\nif not cli.allowcustom(): cli.handle(); exit()\nlog.header("Running custom backups from " + __file__)\n#################################################\n# Add your custom backups code here\n#################################################\nprint(\'custom stuff here, any python code you want\')\n\n\n# Handle CLI requests\ncli.handle()\n```\n\nRunning `backups.py` can be read at the top of this readme.\n\n\n\n## As a python module/package\n\nServerbackups can also be used right inside your own python code as a module.  No need for a `backups.py`\nthough you can still optionally use `/etc/serverbackups` style files if needed.\n\nImagine as part of your own python program, you simply want to backup a file or database.  Using `serverbackups`\nas a module import can help with this.\n\n\nExample of a simple\n```python\nfrom datetime import date\nfrom mreschke.serverbackups import Backups, log\n\n# Optionally configure logger, but this app may have its own log settings\nlog.init({\n    \'console\': {\n        \'level\': \'INFO\',  # DEBUG, INFO, WARNING, ERROR, CRITICAL\n    },\n    \'file\': {\n        \'enabled\': True,\n        \'file\': \'/tmp/backups-{}.log\'.format(date.today().strftime(\'%Y-%m-%d\')),\n    }\n})\n\n# In this example I am NOT defining a defaults config because I may just be\n# backing up a single server.  Defaults are used to deduplicate common items\n# in multiple servers.  If you only have one server, just define the entire config\n# in the server itself\n\n# NOTE sensible defaults are still applied to keep even a single server definition\n# less verbose.  Run `serverbackups show-builtin-defaults` to see the defaults that are\n# still being applied here\n\n# Define each server to backup (each will be merged with defaults config)\n# Source and Destination default to \'local\'\nservers = {\n    \'myserver.example.com\': {\n        \'destination\': {\'path\': \'/mnt/Backups\'},\n        \'backup\': {\n            \'files\': {\n                \'common\': [\n                    \'/etc/\',\n                ],\n                \'exclude\': [\n                    \'lost+found\',\n                    \'.Trash-1000\',\n                ],\n            },\n        },\n    },\n}\n\n# Initialize Backup Class\n# If you are defining servers in a dictionary above, without a separate defaults\nbackups = Backups(servers=servers)\n\n# If you are defining both servers and defaults above\n#backups = Backups(servers=servers, defaults=defaults)\n\n# If you are using a config.d directory (/etc/serverbackups by default)\n#backups = Backups()\n#backups = Backups(config_path=\'/etc/alternative/dir\')\n\n# Run backups\n# Can just run them all\nbackups.run()\n\n# Or can flip through each and add your own logic per server if needed\n#for server in backups.servers:\n    #backups.run(server)\n```\n\n\n# Pruning\n\nRsync hard-link style snapshots are pruned every time the backups run.\n\nThe `prune` server or defaults config determines the prune rates\n```python\n# Keep all daily up to X days back, last snapshot in a week X weeks back, last snapshot in a month X months back...\n\'prune\': {\n    \'keepDaily\': 30,\n    \'keepWeekly\': 24,\n    \'keepMonthly\': 60,\n    \'keepYearly\': 10,\n},\n```\n\nLets say we are running the backups **twice** a day.  The above configuration means we keep each daily\nbackup for 30 days (meaning 60 backups since 2 a day).\n\nKeep the LAST/LATEST weekly backup.  This means the last backup on Saturday of each week for 24 weeks.\n\nKeep the LAST/LATEST monthly backup.  This means the last backup on the 31st of each month for 60 months (5 years)\n\nKeep the LAST/LATEST yearly backup.  This all 12/31 backups of each year will be saved.\n\nThe default prune settings may seem like a waste of storage, but read my `Backup Strategy` notes above.  Using\nrsync hard-link style snapshots saves space by using unix hard links to deduplicate files efficiently.\n',
    'author': 'Matthew Reschke',
    'author_email': 'mail@mreschke.com',
    'maintainer': None,
    'maintainer_email': None,
    'url': 'http://github.com/mreschke/server-backups',
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'entry_points': entry_points,
    'python_requires': '>=3.5,<4.0',
}


setup(**setup_kwargs)
