# couchdiscover

Maintainer: Joe Black <joeblack949@gmail.com>
Repository: https://www.github.com/joeblackwaslike/couchdiscover

Other repositories:

Images:

## Description

Utilizes the Kubernetes and CouchDB 2.0 clustering API's for automating the process of creating a CouchDB 2.0 Cluster. The reqirements here vary significantly compared to the predecessor BigCouch.

This module has an entrypoint stub called `couchdiscover` that will be created upon installation with setuptools.

This tool is meant to be used in a kubernetes cluster as a sidecar container.

## Usage

Create a secret to hold the couchdb credentials.  
*Names and keys don't matter here as long as they match those used in the petset spec* 
Use and modify the the following commands to generate the base64 encoded data needed for this step.

```bash
# base64 encode a username
echo -n admin | base64
# generate and base64 encode a password
python -c 'import os,base64; print base64.b64encode(os.urandom(32))' | base64
```

```yaml
# couchdb-creds.yaml
apiVersion: v1
kind: Secret
metadata:
  name: couchdb-creds
type: Opaque
data:
  couchdb.user: [encoded-username-from-above]
  couchdb.pass: [encoded-password-from-above]
```

Create another secret for the erlang-cookie.  This isn't required but is highly suggested.

```bash
# generate and base64 encode a password
python -c 'import os,base64; print base64.b64encode(os.urandom(64))' | base64
```

```yaml
# erlang-cookie.yaml
apiVersion: v1
kind: Secret
metadata:
  name: erlang-cookie
type: Opaque
data:
  erlang.cookie: [encoded-erlang-cookie-from-above]
```

Now create both secrets in kubernetes

```bash
kubectl create -f couchdb-creds.yaml
kubectl create -f erlang-cookie.yaml
```

Create a configmap holding the configration for CouchDB.
*This isn't a necessary step but you will need to set your environment values in the petset inline during the last step*

Note: I have included alot of different environment variable hooks in my CouchDB image so that all important configuration information can be manipulated at container run.

```yaml
# couchdb-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: couchdb-config
  labels:
    app: couchdb
data:
  erlang.threads: '25'
  couchdb.log-level: info
  couchdb.require-valid-user: 'false'
  couchdb.shards: '4'
  couchdb.replicas: '3'
  couchdb.read-quorum: '1'
  couchdb.write-quorum: '2'
  
  couchdiscover.log-level: info
```

Now create the configmap in kubernetes

```bash
kubectl create -f couchdb-config.yaml
```

Create a headless service for the petset that will tolerate unready endpoints
* *This is required for proper dns resolution*
* *Names here don't matter but need to match what is used in the petset spec*

```yaml
# couchdb-service-headless.yaml
apiVersion: v1
kind: Service
metadata:
  name: couchdb
  annotations:
    service.alpha.kubernetes.io/tolerate-unready-endpoints: 'true'
spec:
  clusterIP: None
  selector:
    app: couchdb
  ports:
  - name: data
    protocol: TCP
    port: 5984
  - name: admin
    protocol: TCP
    port: 5986
```

```bash
kubectl create -f couchdb-service-headless.yaml
```

Create a normal service in order to obtain a clusterIP and load balance requests across couch servers.*Name doesn't matter but will effect how you discover the service through DNS*

```yaml
# couchdb-service-balanced.yaml
apiVersion: v1
kind: Service
metadata:
  name: couchdb-bal
spec:
  selector:
    app: couchdb
  ports:
  - name: data
    protocol: TCP
    port: 5984
  - name: admin
    protocol: TCP
    port: 5986
```

```bash
kubectl create -f couchdb-service-balanced.yaml
```

Create the petset for CouchDB
*Name's don't matter here but make sure things match with the other specs you're creating*

```yaml
# couchdb-petset.yaml
apiVersion: apps/v1alpha1
kind: PetSet
metadata:
  name: couchdb
spec:
  serviceName: couchdb
  replicas: 3
  template:
    metadata:
      labels:
        app: couchdb
      annotations:
        pod.alpha.kubernetes.io/initialized: 'true'
    spec:
      terminationGracePeriodSeconds: 30
      containers:
      - name: couchdb
        image: callforamerica/couchdb-app:latest
        ports:
        - name: data
          protocol: TCP
          containerPort: 5984
        - name: admin
          protocol: TCP
          containerPort: 5986
        env:
        - name: COUCHDB_ADMIN_USER
          valueFrom:
            secretKeyRef:
              name: couchdb-creds
              key: couchdb.user
        - name: COUCHDB_ADMIN_PASS
          valueFrom:
            secretKeyRef:
              name: couchdb-creds
              key: couchdb.pass
        - name: ERLANG_COOKIE
          valueFrom:
            secretKeyRef:
              name: erlang-cookie
              key: erlang.cookie
        - name: ERLANG_THREADS
          valueFrom:
            configMapKeyRef:
              name: couchdb-config
              key: erlang.threads
        - name: COUCHDB_LOG_LEVEL
          valueFrom:
            configMapKeyRef:
              name: couchdb-config
              key: couchdb.log-level
        - name: COUCHDB_REQUIRE_VALID_USER
          valueFrom:
            configMapKeyRef:
              name: couchdb-config
              key: couchdb.require-valid-user
        - name: COUCHDB_SHARDS
          valueFrom:
            configMapKeyRef:
              name: couchdb-config
              key: couchdb.shards
        - name: COUCHDB_REPLICAS
          valueFrom:
            configMapKeyRef:
              name: couchdb-config
              key: couchdb.replicas
        - name: COUCHDB_READ_QUORUM
          valueFrom:
            configMapKeyRef:
              name: couchdb-config
              key: couchdb.read-quorum
        - name: COUCHDB_WRITE_QUORUM
          valueFrom:
            configMapKeyRef:
              name: couchdb-config
              key: couchdb.write-quorum
        resources:
          requests:
            cpu: 2
            memory: 2Gi
          limits:
            cpu: 2
            memory: 2Gi
        readinessProbe:
          httpGet:
            path: /
            port: 5984
          initialDelaySeconds: 10
          timeoutSeconds: 3
          successThreshold: 1
          failureThreshold: 5
        livenessProbe:
          httpGet:
            path: /_up
            port: 5984
          initialDelaySeconds: 15
          timeoutSeconds: 3
          successThreshold: 1
          failureThreshold: 5
        volumeMounts:
        - name: couchdb-data
          mountPath: /volumes/couchdb/data
        imagePullPolicy: IfNotPresent
      - name: couchdb-discover
        image: callforamerica/couchdb-discover:latest
        env:
        - name: LOG_LEVEL
          valueFrom:
            configMapKeyRef:
              name: couchdb-config
              key: couchdiscover.log-level
        imagePullPolicy: IfNotPresent
      restartPolicy: Always
  volumeClaimTemplates:
  - metadata:
      name: couchdb-data
      annotations:
        volume.alpha.kubernetes.io/storage-class: anything
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 8Gi
```

Create the petset in kubernetes

```bash
kubectl create -f couchdb-petset.yaml
```

### IMPORTANT NOTES:

* If you do not have dynamic volume provisioning setup or enabled in your kubernetes cluster, you will need to comment or modify the `volumeClaimTemplates` and `volumeMounts` keys from the above yaml.  Setting up Dynamic Volume Provisioning it **way** outside the scope of this README.  

* You will need to make sure the annotation value for `storage-class` in `volumeClaimTemplates` matches the configured storage-class in your cluster.

* Adjust the above `resource` `requests` and `limits` to your needs.

* Adjust the `readinessProbe` and `livenessProbe` settings to your needs.

* Expected cluster size will be retrieved from `spec.replicas` in the petset above.  This value will be used to trigger the cluster finish action of the cluster_setup endpoint. 

* To override the expected cluster size, add an additional environment variable to the **`couchdb`** container (not couchdiscover) named: `COUCHDB_CLUSTER_SIZE` with the value of the expected size of your cluster.  It should be rare that this is necessary though.  Keep in mind that by doing this your cluster will not be ready to store or retrieve information until you reach the number of nodes specified in `COUCHDB_CLUSTER_SIZE`.  Until that point you will see continuous errors in your CouchDB logs complaining about missing databases.


## Environment variables used by couchdiscover:

### `couchdb` container:
* `COUCHDB_ADMIN_USER`: username to use when enabling the node, required.
* `COUCHDB_ADMIN_PASS`: password to use when enabling the node, required.
* `ERLANG_COOKIE`: cookie value to use as the `.erlang.cookie`, not required, fails back to insecure cookie value when not set.
* `COUCHDB_CLUSTER_SIZE`: not required, overrides the value of `spec.replicas` in the petset, should rarely be necessary to set. Don't set unless you know what you're doing.

### `couchdiscover` container:
* `LOG_LEVEL`: logging level to output container logs for.  Defaults to `INFO`, most logs are either INFO or WARNING level.


## How information is discovered

In order to best use something that is essentially "zero configuration," it helps to understand how the necessary information is obtained from the environment and api's. 

1. Initially a great deal of information is obtained by grabbing the hostname of the container that's part of a petset and parsing it.  This is how the namespace is determined, how hostnames are calculated later, the name of the petset to look for in the api, the name of the headless service, the node name, the index, whether a node is master or not, etc.

2. The kubernetes api is used to grab the petset and entrypoint objects. The entrypoint object is parsed to obtain the `hosts` list.  Then the petset is parsed for the ports, then the environment is resolved, fetching any externally referenced configmaps or secrets that are necessary.  Credentials are resolved by looking through the environment for the keys: `COUCHDB_ADMIN_USER`, `COUCHDB_ADMIN_PASS`.  Finally the expected cluster size is set to the number of replicas in the fetched petset.  You can override this as detailed in the above notes section, but should be completely unnecessary for most cases.


## Main logic

The main logic is performed in the `manage` module's `ClusterManager` object's `run` method.  I think most of it is relatively straighforward.

```python
# couchdiscover.manage.ClusterManager
def run(self):
    """Main logic here, this is where we begin once all environment
    information has been retrieved."""
    log.info('Starting couchdiscover: %s', self.couch)
    if self.couch.disabled:
        log.info('Cluster disabled, enabling')
        self.couch.enable()
    elif self.couch.finished:
        log.info('Cluster already finished')
        self.sleep_forever()

    if self.env.first_node:
        log.info("Looks like I'm the first node")
        if self.env.single_node_cluster:
            log.info('Single node cluster detected')
            self.couch.finish()
    else:
        log.info("Looks like I'm not the first node")
        self.couch.add_to_master()
        if self.env.last_node:
            log.info("Looks like I'm the last node")
            self.couch.finish()
        else:
            log.info("Looks like I'm not the last node")
    self.sleep_forever()
```
