{
  "code": 200,
  "status": 20000,
  "data": {
    "title": "Writing a Datasette CLI plugin that mostly duplicates an existing command",
    "description": "",
    "url": "https://til.simonwillison.net/datasette/plugin-modifies-command",
    "content": "My new [datasette-gunicorn](https://datasette.io/plugins/datasette-gunicorn) plugin adds a new command to Datasette - `datasette gunicorn` - which mostly replicates the existing `datasette serve` command but with a few differences.\n\nI learned some useful tricks for modifying and extending existing [Click](https://click.palletsprojects.com/) commands building this plugin.\n\nHere's the relevant section of code, with some extra comments (the [full code is here](https://github.com/simonw/datasette-gunicorn/blob/0.1/datasette_gunicorn/__init__.py)):\n\nimport click\nfrom datasette import hookimpl\n\n\\# These options do not work with 'datasette gunicorn':\ninvalid\\_options \\= {\n    \"get\",\n    \"root\",\n    \"open\\_browser\",\n    \"uds\",\n    \"reload\",\n    \"pdb\",\n    \"ssl\\_keyfile\",\n    \"ssl\\_certfile\",\n}\n\ndef serve\\_with\\_gunicorn(\\*\\*kwargs):\n    \\# Avoid a circular import error running the tests:\n    from datasette import cli\n\n    workers \\= kwargs.pop(\"workers\")\n    port \\= kwargs\\[\"port\"\\]\n    host \\= kwargs\\[\"host\"\\]\n    \\# Need to add back default kwargs for everything in invalid\\_options:\n    kwargs.update({invalid\\_option: None for invalid\\_option in invalid\\_options})\n    kwargs\\[\"return\\_instance\"\\] \\= True\n    ds \\= cli.serve.callback(\\*\\*kwargs)\n    \\# ds is now a configured Datasette instance\n    asgi \\= StandaloneApplication(\n        app\\=ds.app(),\n        options\\={\n            \"bind\": \"{}:{}\".format(host, port),\n            \"workers\": workers,\n        },\n    )\n    asgi.run()\n\n@hookimpl\ndef register\\_commands(cli):\n    \\# Get a reference to the existing \"datasette serve\" command\n    serve\\_command \\= cli.commands\\[\"serve\"\\]\n    \\# Create a new list of params, excluding any in invalid\\_options\n    params \\= \\[\n        param for param in serve\\_command.params if param.name not in invalid\\_options\n    \\]\n    \\# This is the longer way of constructing a new Click option, as an alternative\n    \\# to using the @click.option() decorator\n    params.append(\n        click.Option(\n            \\[\"-w\", \"--workers\"\\],\n            type\\=int,\n            default\\=1,\n            help\\=\"Number of Gunicorn workers\",\n            \\# Causes \\[default: 1\\] to show in the option help\n            show\\_default\\=True,\n        )\n    )\n    gunicorn\\_command \\= click.Command(\n        name\\=\"gunicorn\",\n        params\\=params,\n        callback\\=serve\\_with\\_gunicorn,\n        short\\_help\\=\"Serve Datasette using Gunicorn\",\n        help\\=\"Start a Gunicorn server running to serve Datasette\",\n    )\n    \\# cli is the Click command group passed to this plugin hook by\n    \\# Datasette - this is how we add the \"datasette gunicorn\" command:\n    cli.add\\_command(gunicorn\\_command, name\\=\"gunicorn\")\n\nHere's the documentation for the [register\\_commands() plugin hook](https://docs.datasette.io/en/stable/plugin_hooks.html#register-commands-cli). It is passed `cli` which is a Click command group.\n\n`cli.add_command(...)` can then be used to register additional commands - in this case the `datasette gunicorn` one.\n\nI want that command to take _almost_ the same options as the existing `datasette serve` command - which is defined [here in the Datasette codebase](https://github.com/simonw/datasette/blob/0.62/datasette/cli.py#L468-L498).\n\nSo... I start by creating a copy of those options. But there are a few options which don't make sense for my new command (see [this issue](https://github.com/simonw/datasette-gunicorn/issues/2)). So I filtered those out with a list comprehension:\n\nparams \\= \\[\n    param for param in serve\\_command.params if param.name not in invalid\\_options\n\\]\n\nI did need one extra option: a `-w/--workers` integer specifying the number of workers that should be started by Gunicorn.\n\nHere's the [relevant Click documentation](https://click.palletsprojects.com/en/8.1.x/api/#click.Option). I defined it like this:\n\nparams.append(\n    click.Option(\n        \\[\"-w\", \"--workers\"\\],\n        type\\=int,\n        default\\=1,\n        help\\=\"Number of Gunicorn workers\",\n        \\# Causes \\[default: 1\\] to show in the option help\n        show\\_default\\=True,\n    )\n)\n\nI defined the new `gunicorn` command like this:\n\ngunicorn\\_command \\= click.Command(\n    name\\=\"gunicorn\",\n    params\\=params,\n    callback\\=serve\\_with\\_gunicorn,\n    short\\_help\\=\"Serve Datasette using Gunicorn\",\n    help\\=\"Start a Gunicorn server running to serve Datasette\",\n)\ncli.add\\_command(gunicorn\\_command, name\\=\"gunicorn\")\n\nThe `short_help` is shown in the list of commands displayed by `datasette --help`.\n\nThe `help` is shown at the top of the list of options when you run `datasette gunicorn --help`.\n\nThe most important argument here is `callback=` - this is the function which will be executed when the user types `datasette gunicorn ...`.\n\nHere's a partial implementation of that function:\n\ndef serve\\_with\\_gunicorn(\\*\\*kwargs):\n    \\# Avoid a circular import error running the tests:\n    from datasette import cli\n\n    workers \\= kwargs.pop(\"workers\")\n    port \\= kwargs\\[\"port\"\\]\n    host \\= kwargs\\[\"host\"\\]\n    \\# Need to add back default kwargs for everything in invalid\\_options:\n    kwargs.update({invalid\\_option: None for invalid\\_option in invalid\\_options})\n    kwargs\\[\"return\\_instance\"\\] \\= True\n    ds \\= cli.serve.callback(\\*\\*kwargs)\n    \\# ds is now a configured Datasette instance\n    asgi \\= StandaloneApplication(\n        app\\=ds.app(),\n        options\\={\n            \"bind\": \"{}:{}\".format(host, port),\n            \"workers\": workers,\n        },\n    )\n    asgi.run()\n\nThe `**kwargs` passed to that function are the options and argumenst that have been extracted from the command line by Click.\n\nIn this case, I know I'm going to be calling the existing `serve` function from Datasette. `cli.serve` is the Click decorated version, but `cli.serve.callback()` is the original function I defined in my own Datasette source code (linked above).\n\nThat function in Datasette takes a list of keyword arguments, which I need to pass through.\n\nThe `kwargs` passed to `serve_with_gunicorn()` are not quite right - remember, I removed some options earlier, and I also added a `workers` option that `serve()` doesn't know how to handle.\n\nSo I pop `workers` off the dictionary, and I add `\"name\": None` keys for the `invalid_options` that I previously filtered out.\n\nOne last trick: my `serve()` function [here in Datasette](https://github.com/simonw/datasette/blob/0.62/datasette/cli.py#L468-L498) has an extra `return_instance` keyword argument, which can be used to shortcut that function and return the configured Datasette instance instead of starting the server.\n\nI originally built this to help with unit tests, but this is also exactly what I need for this particular plugin! I set that to true to get back a configured Datasette object instance, which I can then use to serve the application using Gunicorn.\n\nCreated 2022-10-22T17:58:11-07:00 \u00b7 [Edit](https://github.com/simonw/til/blob/main/datasette/plugin-modifies-command.md)",
    "usage": {
      "tokens": 1720
    }
  }
}
