# ECScmdb

# CLI Version

    cmdb:     ecscmdb(1.7.2), ecspylibs(1.1.27)
    cmdbdiff: ecscmdb(1.7.2), ecspylibs(1.1.27)

# Summary of commands

    cmdb:     Download the devices in the OpenManage database into a
              spreadsheet, one work sheet for each device.

    cmdbdiff: Compare two cmdb spreadsheets for differences.

# Assumptions

The ecscmdb user needs to know:

* how to run commands in a Linux environment.
* how to install python packages in a virtualenv.
* how to run python commands in a virtualenv.
* how to configure yaml files.
* some basic understanding of jinja2 variable used in the yaml file.

# Installation

    It is best to create a dedicated python virtualenv to install
    ecscmdb into so as not to interfere with any other python packages
    installed on your system.

    # virtualenv .venv
    # . .venv/bin/activate
    # pip install ecscmdb

# Setup

## cmdb

    Before cmdb can be used to gather OpenManage data, two config
    files and one password files must be created.  They are both
    located in the directory etc/ecscmdb, which is relative to the
    virtualenv directory ecscmdb was installed in.  The config files
    can be in yaml, toml, or json formats.  Only yaml format will be
    used in this documentation.  The two config files are cmdb.yml and
    filter.yml.

### cmdb.yml

    cmdb.yml is a yaml file. The yaml file can have multiple "config" 
    sections defined within it.  There must be at lease one config section.
    Each config secion has the following fields:
    
    name:       The name of the config section.

    pw:         The location of the cmdb password file relative to 
                virtualenv base directory cmdb was installed in.

    log_level:  The python log level cmdb will run at.  Must be one of 
                DEBUG, INFO, WARNING, ERROR, CRITICAL, or TRACE.

    log_dir:    The location of the cmdb log directory relative to 
                virtualenv base directory cmdb was installed in. 

    OME_Login:  The User ID of the OpenManage server that will be used 
                to gather the data.

    poolsize:   The number of python ProcessPoolExecutor pools that will
                be used to gather the OpenManage data.

    filter:     The location of the cmdb filter file.  The filter
                file, explained in detail below, scans the incoming
                OpenManage, and only stores the required columns and
                rows of the data.  
    
    output:     The location of the cmdb output file that the cmdb 
                spreadsheet will be written to.  The name of the file can
                include the jinja2 variable "{{ TOD }}" to include the 
                current date and time.

    In addition to the config section(s), the yaml file also must include 
    the yaml variable "section:" which indicates which of the config sections
    to process.

### cmdb.yml example file
```
Here is an example cmdb.yml file with two config sections.  Because the
"section:" variable is set to 2 in this example the second section, with 
name "Configuration file Testing", will be used.

---
section: 2
config:
  - name: Configuration file Production.
    pw: etc/ecscmdb/cmdb.pw
    log_level: warning
    log_dir: log
    OME_Login: cmdb
    poolsize: 12
    filter: etc/ecscmdb/filter.yml
    output: output/OpenManage-cmdb.{{ TOD }}.xlsx
  - name: Configuration file Testing.
    pw: etc/ecscmdb/cmdb.pw
    log_level: TRACE
    log_dir: log
    OME_Login: cmdb
    poolsize: 24
    filter: etc/ecscmdb/filter.yml
    output: output/OpenManage-test-cmdb.{{ TOD }}.xlsx
...
```

### filter.yml

    filter.yml is a yaml file. The yaml file can have multiple "config" 
    sections defined within it.  There must be at lease one config section.
    Each config secion has the following fields:
    
    name:           The name of the config section.
    
    column_header:  The name of the column used to match the "row" labels.
    
    rename_columns: A dictionary of "Old Name": "New Name" used to rename
                    Column lines.
    
    sort_columns:   A list of column names to sort the each worksheet with.
    
    filters:        A list of Row:Columns:Update to include from the 
                    OpenManage database into each created worksheet.  The
                    Update option is optional.
	
        Row:        The names of the row to look for matching columns in
                    within the OpenManage database data.
	
        Columns:    A list of matching Columns names for the given Row to
                    include.
	
        Update:     A dictional entry of "Column Name": "Function Name".
                    If the Column Name is part of the data precented from
                    the OpenManage database, then the cell for Row:Column
                    will be modified using the Function Name provided.
                    The Function is defined in the python library
                    updatecells.py that is part of the ECScmdb packages.
                    This should be the only python library that needs to
                    be modified.

    In addition to the config section(s), the yaml file also must
    include the yaml variable "section:" which indicates which of the
    config sections to process.

### filter.yml example file
```
Here is an example filter.yml file with one config section.  Because the
"section:" variable is set to 1 in this example the first section, with 
name "Partial", will be used.

---
section: 1
config:
  - name: Partial
    column_header: InventoryType
    rename_columns:
      Ports: "Mac Address"

    sort_columns:
      - InventoryType
      - DeviceDescription
      - DeviceName
      - InstanceId
      - NicId
      - CapabilityType.CapabilityId
      - CapabilityType.Description
      - CapabilityType.IdOwner
      - CapabilityType.Name
      - DiskNumber
      - DnsName
      - EndDate
      - EntitlementId
      - Id
      - IdOwner
      - LicenseDescription
      - LicenseType.LicenseId
      - LicenseType.Name
      - Name
      - Number
      - SasAddress
      - SerialNumber
      - ServiceTag
      - SlotNumber
      - SoldDate

    filters:
      -
        row: serverProcessors
        columns:
          - InstanceId
          - InventoryType
          - ModelName
          - NumberOfCores
      -
        row: serverFcCards
        columns:
          - DeviceDescription
          - DeviceName
          - InventoryType
          - VirtualWwn
          - VirtualWwpn
          - Wwn
          - Wwpn
      -
        row: serverOperatingSystems
        columns:
          - Hostname
          - InventoryType
          - OsName
      -
        row: serverArrayDisks
        columns:
          - BusType
          - DiskNumber
          - InventoryType
          - MediaType
          - ModelNumber
          - SerialNumber
          - Size
      -
        row: serverMemoryDevices
        columns:
          - DeviceDescription
          - InstanceId
          - InventoryType
          - ManufacturerDate
          - Name
          - SerialNumber
          - Size
          - Speed
          - TypeDetails
      -
        row: serverNetworkInterfaces
        columns:
          - InventoryType
          - NicId
          - Ports
        update:
          Ports: update_ports
      -
        row: chassisSlotsList
        columns:
          - DnsName
          - InventoryType
          - Model
          - Name
          - Number
          - Occupied
          - ServiceTag
      -
        row: chassisControllerList
        columns:
          - InventoryType
          - Name
      -
        row: deviceManagement
        columns:
          - DnsName
          - EndPointAgents
          - InstrumentationName
          - InventoryType
          - IpAddress
          - MacAddress
          - ManagementId
          - ManagementType.Description
          - ManagementType.Name
        update:
          EndPointAgents: update_end_point_agents
      -
        row: nicInformation
        columns:
          - Description
          - DeviceId
          - InventoryType
          - Ipv4_address
          - Mac_address
          - Subnet_mask
...
```

### cmdb.pw
    cmdb.pw is a python AES encripted password file.  It must contain the
    following two entries: 

    1. A key that matches the OME_Login variable from the cmdb.yml
       file. The value of the entry is the password for that user.

    2. A key named "OMEHost" (case-sensitive string).  The value of
       the entry is the URL for the OpenManage host.

### Adding or updating an entry in the cmdb.pw file

    Execute "cmdb --add=STRING", where STRING is the key to be created
    or updated.  cmdb will then prompt for the value for the given
    key.  The entries are encripted and stored in the cmdb.pw file.

### Example 

    # cmdb --add=xyzzy
    Enter password for user 'xyzzy':
    #

### Deleting an entry from the cmdb.pw file

    Execute "cmdb --delete=STRING", where STRING is the key to be
    deleted.  The entries will be deleted without prompting.

### Example

    # cmdb --delete=xyzzy
    #

### List the keys or keys/values stored n the cmdb.pw file

    Execute "cmdb --list" to list all of the keys in the cmdb.pw file,
    or "cmdb --list --verbose" to list both the keys and the values
    for each key.

### Examples

    # cmdb --list
    # cmdb --list --verbose

## cmdbdiff

    Before cmdbdiff can be used to compare two OpenManage
    spreadsheets, created using the cmdb command, the file
    cmdbdiff.yml must be created.  This file is located in the
    directory etc/ecscmdb, which is relative to the virtualenv
    directory ecscmdb was installed in.

### cmdbdiff.yml

    cmdbdiff.yml is a yaml file. The yaml file can have multiple
    "config" sections defined within it.  There must be at least one
    config section in it.  In addition to the config section(s), the
    yaml file also must include the yaml variable "section:" which
    indicates which of the config sections to process.st be at lease
    one config section.  Each config secion has the following fields:

    name:                       The name of the configuration section 
                                (optional).
    admin:                      The Admin managing cmdbdiff.
        name:                   The Admin name.
        email:                  The Admin email address.
        phone:                  The Admin phone number.
    log_level:                  The python log level cmdb will run at.  
                                Must be one of DEBUG, INFO, WARNING, ERROR, 
                                CRITICAL, or TRACE.
    log_dir:                    The location of the cmdb log directory 
                                relative to virtualenv base directory cmdb 
                                was installed in.
    report:                     The location of the cmdbdiff report file 
                                that cmdbdiff writes to.  The name of the 
                                file can include the jinja2 variable 
                                "{{ TOD }}" to include the current date and 
                                time.
    email:                      The email section indicates who should 
                                receive a copy of the cmdbdiff report.
        subject:                The subject line of the report email.
        from:                   The name and email address of the user 
                                sending the report.
            name:               The sending name.
            email:              The sending email address.
        to:                     A list of zero or more "to" email addresses.
            name:               The outgoing name.
            email:              The outgoing email address.
        cc:                     A list of zero or more "cc" email addresses. 
            name:               The outgoing name.
            email:              The outgoing email address.
        text:                   The text of the email that is sent.
            with_changes:       The text of the email being if there are 
                                changes between the spreadsheets being 
                                checked.
            without_changes:    The text of the email being if there are no 
                                changes between the spreadsheets being 
                                checked. 

    In addition to the config section(s), the yaml file also must include 
    the yaml variable "section:" which indicates which of the config sections
    to process.

### jinja2 variables used in the cmdbdiff.yml file

```
There are several jinja2 variables that can be used in the cmdbdiff.yml 
configuration file.  Here is a list with a description of their values:

TOD:            The current Date and Time.
DATE:           The currect Date.
TIME:           The current Time.
HOST:           The Host name the report was generated on.
VERSION:        The Version of the cmdbdiff program.
SPREADSHEET1:   The name of the first spreadsheet being checked.
WORKSHEETS1:    The number of Worksheets in the spreadsheet 1.
SPREADSHEET2:   The name of the second spreadsheet being checked.
WORKSHEETS2:    The number of Worksheets in the spreadsheet 2.
REPORT:         The name of the report file (from config.report).
ADMINNAME:      The name of the cmdbdiff Admin (from config.admin.name).
ADMINEMAIL:     The email address of the cmdbdiff Admin (from 
                config.admin.email).
ADMINPHONE:     The phone number of the cmdbdiff Admin (from 
                config.admin.phone).
```

### cmdbdiff.yml example file

```
Here is an example cmdb.yml file with one config sections.  The "section:" 
variable is set to 1 in this example the second section, with name 
"Configuration file Production", will be used.

---
section: 1
config:
  - name: Configuration file Production.
    admin:
      name: Bill Jones
      email: BJ@example.com
      phone: 999.555.1234
    log_level: warning
    log_dir: log
    report: report/OpenManage-cmdbdiff-{{ TOD }}.xlsx
    email:
      subject: CMDB Diff Report generated on {{ DATE }} at {{ TIME }} on server {{ HOST }}.
      from:
        name: CMDB Diff Report
        email: CMDB-Diff-Report@{{ HOST }}
      to:
        -
          name: John Smith
          email: JS@example.com
        -
          name: Steve Thomas
          email: ST@example.com
      cc:
        -
          name: Tim George
          email: TG@example.com
      text:
        with_changes:

          "\nCMDB Diff report, Version {{ VERSION }}, generated on {{ DATE
          }} at {{ TIME }} on server {{ HOST }}.\n

          There are three attached spreadsheets in this report:\n

          1) {{ SPREADSHEET1 }}, is the previous CMDB spreadsheet with
          {{ WORKSHEETS1 }} worksheets.\n

          2) {{ SPREADSHEET2 }}, is an currect CMDB spreadsheet with {{
          WORKSHEETS2 }} worksheets.\n

          3) {{ REPORT }}, lists the changes between {{ SPREADSHEET1 }}
          and {{ SPREADSHEET2 }}.\n

          The first worksheet in {{ REPORT }} lists a summary of all
          changes. The remaining worksheets in {{ REPORT }}, if any, are
          copies of the worksheets in {{ SPREADSHEET2 }} that are
          different from {{ SPREADSHEET1 }}.\n

          Contact {{ ADMINNAME }} at email address {{ ADMINEMAIL }} or
          phone number {{ ADMINPHONE }} if you have questions or
          concerns regarding any of the spreadsheets generated in this
          report.\n

          Thanks!\n"

        without_changes:

          "\nCMDB Diff report, Version {{ VERSION }}, generated on {{ DATE
          }} at {{ TIME }} on server {{ HOST }}.\n

          The CMDB Diff report found no changes between {{
          SPREADSHEET1 }} and {{ SPREADSHEET1 }} on {{ DATE }}.\n

          There are three attached spreadsheets in this report:\n

          1) {{ SPREADSHEET1 }}, is the previous CMDB spreadsheet with
          {{ WORKSHEETS1 }} worksheets.\n

          2) {{ SPREADSHEET2 }}, is an currect CMDB spreadsheet with {{
          WORKSHEETS2 }} worksheets.\n

          3) {{ REPORT }}, lists the changes between {{ SPREADSHEET1 }}
          and {{ SPREADSHEET2 }}.\n

          Contact {{ ADMINNAME }} at email address {{ ADMINEMAIL }} or
          phone number {{ ADMINPHONE }} if you have questions or
          concerns regarding any of the spreadsheets generated in this
          report.\n

          Thanks!\n"

```

# CLI Help text

## # cmdb --help

    Program to download the data from the OpenManage DB and build a spreadsheet.

    Some default option values listed below can be overridden within the
    configuration file.

    Usage:
      cmdb [-v] [-L LEVEL] [--LOG=DIR] [-F] [-c CONFIG] [-s SECTION] [-o OUTPUT] [-p PWFILE] [-D] [-P SIZE]
      cmdb [-vl] [-L LEVEL] [--LOG=DIR] [-c CONFIG] [-s SECTION] [-a ID]... [-d ID]... [-p PWFILE] [-D]
      cmdb (-h | --help | -V | --version)

      There are no required options.

    Options:
      -h, --help                     Show this help message and exit.
      -V, --version                  Show version information and exit.
      -F, --full                     Show all data, no filtering.
      -c CONFIG, --config=CONFIG     The configuration file.
                                     Default: "/home/tom/Run/ECScmdb/Testing/etc/ecscmdb/cmdb.yml"
      -s SECTION, --section=SECTION  The configuration file version (default
                                     defined within the configuration file).
      -o OUTPUT, --output=OUTPUT     Output file or directory.
                                     Default: "/home/tom/Run/ECScmdb/Testing/output/OpenManage-cmdb.2023-02-08-18-14-30.xlsx"
      -p PWFILE, --pw=PWFILE         The password file.  This file is used when a
                                     login to a website or webpage is required.
                                     Default: "/home/tom/Run/ECScmdb/Testing/etc/ecscmdb/cmdb.pw"
      -l, --list                     List all of the IDs in the password file and
                                     exit.  If both the --list and --verbose
                                     options are included, list both IDs and
                                     Passwords and exit.
      -a ID, --add=ID                Add (or update) an ID and Password and exit.
                                     Program will prompt for the Password to be
                                     saved to the password file.
      -d ID, --delete=ID             Delete an ID (if it exists) from the
                                     password file and exit.
      -v, --verbose                  Print verbose messages.
      -L LEVEL, --log=LEVEL          Print log messages at log value LEVEL.
                                     Valid levels are: TRACE, DEBUG, INFO, WARNING,
                                     ERROR, and CRITICAL.
                                     Default: WARNING
      --LOG=DIR                      Log directory.
                                     Default: "/home/tom/Run/ECScmdb/Testing/log/cmdb.log"
      -D, --dryrun                   Only print out what would be done.
      -P SIZE, --poolsize=SIZE       Call OpenManage using pools of size SIZE.
                                     Default: set by the OS.

## # cmdbdiff --help

    Program to analyze two spreadsheets for differences.

    Some default option values listed below can be overridden within the initialization file.

    Usage:
      cmdbdiff [-v] [-L LEVEL] [--LOG=DIR] [-c CONFIG] [-s SECTION] [-r REPORT] [-D] SPREADSHEET1 SPREADSHEET2
      cmdbdiff (-h | --help | -V | --version)

      Variables SPREADSHEET1 and SPREADSHEET2 are required, all other parameters are optional.

    Options:
      -h, --help                          Show this help message and exit.
      -V, --version                       Show version information and exit.
      -c CONFIG, --config=CONFIG          The configuration file.
                                          Default: "/home/tom/Run/ECScmdb/Testing/etc/ecscmdb/cmdbdiff.yml"
      -s SECTION, --section=SECTION       The configuration file version (default
                                          defined within the configuration file).
      -r REPORT, --report=REPORT          Report directory or file.
      -v, --verbose                       Print verbose messages.
      -L LEVEL, --log=LEVEL               Print log messages at log value LEVEL.
                                          Valid levels are: TRACE, DEBUG, INFO, WARNING,
                                          ERROR, and CRITICAL.
                                          Default: "WARNING"
      --LOG=DIR                           Log Directory,
                                          Default: "/home/tom/Run/ECScmdb/Testing/log/cmdbdiff.log"
      -D, --dryrun                        Only print out what would be done.
 
# Git information
[GIT Home][CMDB],
[README File][README],
[LICENSE File][LICENSE],
[ChangeLog File][CHANGELOG],
[pyproject.toml File][PYPROJECT].

# About me
[My contact information][About Me].

[CMDB]: https://git.wayne.edu/ECS_Projects/ECScmdb
[README]: https://git.wayne.edu/ECS_Projects/ECScmdb/-/blob/master/README.md
[LICENSE]: https://git.wayne.edu/ECS_Projects/ECScmdb/-/blob/master/LICENSE.txt
[CHANGELOG]: https://git.wayne.edu/ECS_Projects/ECScmdb/-/blob/master/ChangeLog
[PYPROJECT]: https://git.wayne.edu/ECS_Projects/ECScmdb/-/blob/master/pyproject.toml
[About Me]: https://About.Me/Thomas.R.Stevenson
