Introduction and examples — confattr v1.6.0 documentation

# Introduction and examples

## Config and ConfigFile

`confattr` (config attributes) is a python library to make applications configurable. This library defines the `Config` class to create attributes which can be changed in a config file. It uses the descriptor protocol to return it’s value when used as an instance attribute.

    from confattr import Config

    class Car:

    	speed_limit = Config('traffic-law.speed-limit', 50, unit='km/h')

    	def __init__(self) -> None:
    		self.speed = 0

    	def accelerate(self, value: int) -> None:
    		new_speed = self.speed + value
    		if new_speed > self.speed_limit:
    			raise ValueError('you are going too fast')

    		self.speed = new_speed

If you want to access the Config object itself you need to access it as a class attribute:

    	def print_config(self) -> None:
    		print('{key}: {val}'.format(key=type(self).speed_limit.key, val=self.speed_limit))

You load a config file with a `ConfigFile` object. You should provide a callback function with `set_ui_callback()` which informs the user if the config file contains invalid lines. This callback function takes a `Message` object as argument. You can format it automatically by converting it to a str, e.g. with `str(msg)`. Among other attributes this object also has a `notification_level` (or `lvl` for short) which should be used to show messages of different severity in different colors. By default only `ERROR` messages are reported but you should pass a `Config` to `notification_level` when instantiating a `ConfigFile` object so that the users of your application can change that. When you load a config file with `ConfigFile.load()` all `Config` objects which are set in the config file are updated automatically.

It is recognized automatically that the setting `traffic-law.speed-limit` has an integer value. A value given in a config file is therefore automatically parsed to an integer. If the parsing fails the user interface callback function is called.

    if __name__ == '__main__':
    	from confattr import ConfigFile, NotificationLevel
    	config_file = ConfigFile(appname='example',
    		notification_level=Config('notification-level', NotificationLevel.ERROR))
    	config_file.load()

    	# Print error messages which occurred while loading the config file.
    	# In this easy example it would have been possible to register the callback
    	# before calling load but in the real world the user interface
    	# will depend on the values set in the config file.
    	# Therefore the messages are stored until a callback is added.
    	config_file.set_ui_callback(lambda msg: print(msg))

    	c1 = Car()
    	print('speed_limit: %s' % c1.speed_limit)

Warning

> Make sure that all `Config` instances have been instantiated before creating a `ConfigFile` instance. All settings defined later cannot be used in that config file. On the other hand, `ConfigFile.load()` must be called before using the value of any `Config` instances.

Therefore I recommend to:

1. Always define `ConfigFile` instances as class attributes or as global variables before you call any functions, never as local variables.

2. Always put the required imports at the top of a file, never inside of a function or method.

3. Instantiate the `ConfigFile` at the beginning of your main method/function.

Given the following config file (the location of the config file is determined by `ConfigFile.iter_config_paths()`):

    set traffic-law.speed-limit = 30

The script will give the following output:

    speed_limit: 30

You can save the current configuration with `ConfigFile.save()` if you want to write it to the default location or with `ConfigFile.save_file()` if you want to specify the path yourself.

    	filename = config_file.save()
    	print('configuration was written to %s' % filename)

This will write the following file:

    # Data types
    # ----------
    # int:
    #   An integer number in python 3 syntax, as decimal (e.g. 42),
    #   hexadecimal (e.g. 0x2a), octal (e.g. 0o52) or binary (e.g.
    #   0b101010). Leading zeroes are not permitted to avoid confusion
    #   with python 2's syntax for octal numbers. It is permissible to
    #   group digits with underscores for better readability, e.g.
    #   1_000_000.

    # notification-level
    # ------------------
    # one of info, error
    set notification-level = error

    # traffic-law.speed-limit
    # -----------------------
    # an int in km/h
    set traffic-law.speed-limit = 30

## Quickstart

The `quickstart` module defines several commonly needed `Config` instances and the `ConfigManager` class which instantiates two `ConfigFile` instances with different notification levels as shown in a later example and which provides a `create_argument_parser()` method to create an `argparse.ArgumentParser` with an improved `HelpFormatter` and options to print the application version and config related options.

If you are unhappy with the keys of the settings defined by `quickstart` (or any third party library which uses `confattr`) you can change them by calling `Config.push_key_changer()` before importing the module and `Config.pop_key_changer()` after importing the module. Or by changing the `key` attribute: `ConfigManager.editor.key = "text-editor"`. Changing the `key` attribute is only possible before any `ConfigFile` or `ConfigManager` objects have been instantiated.

For consistency with the predefined settings I recommend to start the help of settings with an upper case letter and end it on a full stop.

As when using `ConfigFile` directly all `Config` instances must be created before instantiating `ConfigManager`.

    '''
    Print a greeting.
    '''

    APP_NAME = 'test-app'
    __version__ = '1.0.0'

    from confattr import Config, Message
    from confattr.quickstart import ConfigManager

    class App:

    	greeting = Config('greeting', "hello world", help=
    		"This setting is defined in the example and "
    		"is not a standard setting defined by quickstart.")

    	def __init__(self, config_manager: ConfigManager) -> None:
    		self.cfg = config_manager
    		self.ui_notifier = self.cfg.ui_notifier
    		self.cfg.load()

    	def main(self, greeting: 'str|None' = None) -> None:
    		if greeting is None:
    			greeting = self.greeting

    		self.init_ui()
    		self.cfg.set_ui_callback(self.show_message)
    		self.mainloop(greeting)

    	def init_ui(self) -> None:
    		pass

    	def mainloop(self, greeting: str) -> None:
    		self.ui_notifier.show_info(greeting)

    	def show_message(self, msg: Message) -> None:
    		print(msg)

    def main(largs: 'list[str]|None' = None) -> None:
    	cfg = ConfigManager(APP_NAME, __version__, __doc__)
    	p = cfg.create_argument_parser()
    	p.add_argument('greeting', nargs='?', help="how you want to be greeted")

    	args = p.parse_args(largs)
    	app = App(cfg)
    	app.main(args.greeting)

    if __name__ == '__main__':
    	main()

Calling this script with

    test-app

would print

    hello world

Calling this script with

    test-app --help

would print

    usage: test-app [-h] [-H] [-v] [-e] [-E] [-c CONFIG] [greeting]

    Print a greeting.

    positional arguments:
      greeting              how you want to be greeted

    options:
      -h, --help            show this help message and exit
      -H, --help-config     show the config help and exit
      -v, --version         show the version and exit
      -e, --edit-config     edit the config file and exit
      -E, --update-and-edit-config
                            rewrite the config file, open it for editing
                            and exit
      -c CONFIG, --config CONFIG
                            use this config file instead of the default

Calling this script with

    test-app --edit

would open the following config file in a text editor. This text editor can be chosen by the user with the editor setting.

    # Data types
    # ----------
    # command with alternatives:
    #   One or more commands separated by ||. The first command where the
    #   program is installed is executed. The other commands are ignored.

    #   If the command name starts with a '$' it is interpreted as an
    #   environment variable containing the name of the program to be
    #   executed. This is useful to make use of environment variables like
    #   EDITOR. If the environment variable is not set the program is
    #   considered to not be installed and the next command is tried
    #   instead.

    #   The command is executed without a shell so redirection or wildcard
    #   expansion is not possible. Setting environment variables and
    #   piping like in a POSIX shell, however, are implemented in python
    #   and should work platform independently. If you need a shell write
    #   the command to a file, insert an appropriate shebang line, make
    #   the file executable and set this value to the file.

    # optional existing directory:
    #   The path to an existing directory or an empty str to use a default
    #   path.

    # str:
    #   A text. If it contains spaces it must be wrapped in single or
    #   double quotes.

    # editor
    # ------
    # a command with alternatives
    # The editor to be used when opening the config file.
    set editor = "'$EDITOR' fn||rifle fn||vim fn||nano fn||vi fn"

    # greeting
    # --------
    # a str
    # This setting is defined in the example and is not a standard setting
    # defined by quickstart.
    set greeting = 'hello world'

    # include.home
    # ------------
    # an optional existing directory
    # The directory where the include command looks for relative paths. An
    # empty string is the default, the directory where the config file is
    # located.
    set include.home = ''

    # notification-level.config-file
    # ------------------------------
    # one of info, error
    set notification-level.config-file = error

    # notification-level.user-interface
    # ---------------------------------
    # one of info, error
    set notification-level.user-interface = info

Calling this script with

    test-app --help-config

would print

    The first existing file of the following paths is loaded:
    - ~/.config/test-app/config

    This can be influenced with the following environment variables:
    - XDG_CONFIG_HOME
    - XDG_CONFIG_DIRS
    - TEST_APP_CONFIG_PATH
    - TEST_APP_CONFIG_DIRECTORY
    - TEST_APP_CONFIG_NAME

    You can also use environment variables to change the values of the
    settings listed under `set` command. The corresponding environment
    variable name is the name of the setting in all upper case letters
    with dots, hypens and spaces replaced by underscores and prefixed with
    "TEST_APP_".

    Lines in the config file which start with a `"` or `#` are ignored.

    The config file may contain the following commands:

    set
    ===
    usage: set [--raw] key1=val1 [key2=val2 ...]
           set [--raw] key [=] val

    Change the value of a setting.

    In the first form set takes an arbitrary number of arguments, each
    argument sets one setting. This has the advantage that several
    settings can be changed at once. That is useful if you want to bind a
    set command to a key and process that command with
    ConfigFile.parse_line() if the key is pressed.

    In the second form set takes two arguments, the key and the value.
    Optionally a single equals character may be added in between as third
    argument. This has the advantage that key and value are separated by
    one or more spaces which can improve the readability of a config file.

    You can use the value of another setting with %other.key% or an
    environment variable with ${ENV_VAR}. If you want to insert a literal
    percent character use two of them: %%. You can disable expansion of
    settings and environment variables with the --raw flag.

    data types:

      command with alternatives:
        One or more commands separated by ||. The first command where
        the program is installed is executed. The other commands are
        ignored.

        If the command name starts with a '$' it is interpreted as an
        environment variable containing the name of the program to be
        executed. This is useful to make use of environment variables
        like EDITOR. If the environment variable is not set the
        program is considered to not be installed and the next command
        is tried instead.

        The command is executed without a shell so redirection or
        wildcard expansion is not possible. Setting environment
        variables and piping like in a POSIX shell, however, are
        implemented in python and should work platform independently.
        If you need a shell write the command to a file, insert an
        appropriate shebang line, make the file executable and set
        this value to the file.

      optional existing directory:
        The path to an existing directory or an empty str to use a
        default path.

      str:
        A text. If it contains spaces it must be wrapped in single or
        double quotes.

    settings:

      editor:
        a command with alternatives

        The editor to be used when opening the config file.

      greeting:
        a str

        This setting is defined in the example and is not a standard
        setting defined by quickstart.

      include.home:
        an optional existing directory

        The directory where the include command looks for relative
        paths. An empty string is the default, the directory where the
        config file is located.

      notification-level.config-file:
        one of info, error

      notification-level.user-interface:
        one of info, error

    include
    =======
    usage: include path

    Load another config file.

    This is useful if a config file is getting so big that you want to
    split it up or if you want to have different config files for
    different use cases which all include the same standard config file to
    avoid redundancy or if you want to bind several commands to one key
    which executes one command with ConfigFile.parse_line().

    positional arguments:
      path  The config file to load. Slashes are replaced with the
            directory separator appropriate for the current operating
            system. If the path contains a space it must be wrapped in
            single or double quotes.

    echo
    ====
    usage: echo [-l {info,error}] [-r] msg [msg ...]

    Display a message.

    Settings and environment variables are expanded like in the value of a
    set command.

    positional arguments:
      msg                   The message to display

    options:
      -l {info,error}, --level {info,error}
                            The notification level may influence the
                            formatting but messages printed with echo are
                            always displayed regardless of the
                            notification level.
      -r, --raw             Do not expand settings and environment
                            variables.

    help
    ====
    usage: help [cmd]

    Display help.

    positional arguments:
      cmd  The command for which you want help

## Using the values of settings or environment variables

Consider the following example:

    from confattr import Config, ConfigFile

    class App:
    	s = Config('s', 'hello world')
    	i = Config('i', 42, unit='')
    	b = Config('b', True)
    	l = Config('l', [1, 2, 3], unit='')
    	d = Config('d', dict(a=1, b=2, c=3), unit='')
    a = App()

    config_file = ConfigFile(appname='exampleapp')
    config_file.set_ui_callback(print)

To simplify the writing of this example I will directly pass the lines that I would otherwise write into a config file to `ConfigFile.parse_line()`.

You can use the value of a setting in the `set` command by wrapping the key of the setting in percent characters. You can also specify a format_spec.

    config_file.parse_line('set s = "%i% = 0x%i:02X%"')
    assert a.s == '42 = 0x2A'

If you want to apply the format spec to the string representation of the value instead of on the value itself put an exclamation mark in front of the colon. You can also use `!r`, `!s` and `!a` like in an f-string.

    config_file.parse_line('set s = "[%b!:<5%] ..."')
    assert a.s == '[true ] ...'

You can use a format_spec to convert a bool to an int.

    config_file.parse_line('set i = %b:d%')
    assert a.i == 1

The values are expanded when executing the set command so the order in which you set the settings is important.

    config_file.parse_line('set s="i was %i%" i=2 s="%s%, i is %i%"')
    assert a.s == 'i was 1, i is 2'

You can use the expansion of settings to add new values to a collection.

    config_file.parse_line('set l="%l%,4"')
    assert a.l == [1, 2, 3, 4]

    config_file.parse_line('set d="%d%,d:4"')
    assert a.d == dict(a=1, b=2, c=3, d=4)

And you can use the format_spec defined in the `List()`, `Set()` and `Dict()` classes to remove elements from collections. (Exceptions raised by `expand_value()` are caught in `ConfigFile` and reported via the callback registered with `ConfigFile.set_ui_callback()`.)

    # select all elements except for the second element
    # i.e. all elements except for index 1
    config_file.parse_line('set l="%l:[:1,2:]%"')
    assert a.l == [1, 3, 4]

    # select all elements except for key b
    config_file.parse_line('set d="%d:{^b}%"')
    assert a.d == dict(a=1, c=3, d=4)

    # select the elements for keys c and d
    # and assign the value of key a to z
    config_file.parse_line('set d="%d:{c,d}%,z:%d:[a]%"')
    assert a.d == dict(c=3, d=4, z=1)

If you want to insert a literal percent sign use `%%`. Alternatively, if you don’t want to expand any settings or environment variables, you can insert the flag `--raw` (or `-r` for short) before the key.

    config_file.parse_line('set s="%i%%%"')
    assert a.s == '2%'

You can access environment variables like in a POSIX shell, although not all expansion features are supported and the curly braces are mandatory. For more information see the description of `ConfigFile.expand_env_match()`.

    config_file.parse_line('set s="hello ${HELLO:-world}"')
    assert a.s == 'hello world'

Please note that a list containing exactly one empty string is syntactically equivalent to an empty list and will be loaded as an empty list.

## Config file syntax

I have looked at the config files of different applications trying to come up with a syntax as intuitive as possible. Two extremes which have heavily inspired me are the config files of vim and ranger.

I am using `shlex.split(line, comments=True)` to split the lines so quoting and inline comments work similar to bash although there are minor differences, e.g. a `#` in an argument must be escaped or quoted.

Lines starting with a `"` or `#` are ignored.

The `set` command has two different forms. I recommend to not mix them in order to improve readability. Both forms support the expansion of settings and environment variables in the value, see the previous example.

- `set key1=val1 [key2=val2 ...]` (inspired by vimrc)

  `set` takes an arbitrary number of arguments, each argument sets one setting.

  Has the advantage that several settings can be changed at once. This is useful if you want to bind a set command to a key and process that command with `ConfigFile.parse_line()` if the key is pressed.

  If the value contains one or more spaces it must be quoted. `set greeting='hello world'` and `set 'greeting=hello world'` are equivalent.

- `set key [=] val` (inspired by ranger config)

  `set` takes two arguments, the key and the value. Optionally a single equals character may be added in between as third argument.

  Has the advantage that key and value are separated by one or more spaces which can improve the readability of a config file.

  If the value contains one or more spaces it must be quoted: `set greeting 'hello world'` and `set greeting = 'hello world'` are equivalent.

I recommend to not use spaces in key names so that they don’t need to be wrapped in quotes.

Further commands:

- `include filename`

  Load another config file. If `filename` is a relative path it is relative to the directory of the config file it appears in.

  This command is explained in more detail in this example.

- `echo message`

  Display a message. Settings and environment variables in the message are expanded just like in the value of a `set` command (see the previous example).

- `help [command]`

  Display a list of available commands if no arguments are given or the help for the specified command if an argument is passed.

  Just like the other commands the output is passed to the callback registered with `ConfigFile.set_ui_callback()`.

## Allowed key names

As first argument of any `Config` instance you must provide a `key` which is used to specify the setting in a config file.

This key must not contain an `=` sign because that is used to separate the value from the key in the `set` command.

I recommend to stick to the following characters:

- letters

- digits

- hyphens

- dots to group several settings together

I recommend to not use spaces and other special characters to avoid the need to wrap the key in quotes.

I recommend to not internationalize the keys so that config files do not break if a user changes the language. This also gives users of different languages the possibility to exchange their config files.

I recommend to name the keys in English.

## Different values for different objects

A `Config` object always returns the same value, regardless of the owning object it is an attribute of:

    from confattr import Config

    class Car:
    	speed_limit = Config('traffic-law.speed-limit', 50, unit='km/h')

    c1 = Car()
    c2 = Car()

    print(c1.speed_limit, c2.speed_limit)
    c2.speed_limit = 30  # don't do this, this is misleading!
    print(c1.speed_limit, c2.speed_limit)

Output:

    50 50
    30 30

If you want to have different values for different objects you need to use `MultiConfig` instead. This requires the owning object to have a special attribute called `config_id`. All objects which have the same `config_id` share the same value. All objects which have different `config_id` can have different values (but don’t need to have different values).

    import enum
    from confattr import Config, MultiConfig, ConfigId, ConfigFile, NotificationLevel

    class Color(enum.Enum):
    	RED = 'red'
    	YELLOW = 'yellow'
    	GREEN = 'green'
    	BLUE = 'blue'
    	WHITE = 'white'
    	BLACK = 'black'

    class Car:

    	speed_limit = Config('traffic-law.speed-limit', 50, unit='km/h')
    	color = MultiConfig('car.color', Color.BLACK)

    	def __init__(self, config_id: ConfigId) -> None:
    		self.config_id = config_id

    if __name__ == '__main__':
    	config_file = ConfigFile(appname='example',
    		notification_level=Config('notification-level', NotificationLevel.ERROR))
    	config_file.load()

    	config_file.set_ui_callback(lambda msg: print(msg))

    	cars = []
    	for config_id in MultiConfig.config_ids:
    		cars.append(Car(config_id))
    	cars.append(Car(ConfigId('another-car')))

    	for car in cars:
    		print('color of %s: %s' % (car.config_id, car.color))

Given the following config file:

    set traffic-law.speed-limit = 30

    [alices-car]
    set car.color = red

    [bobs-car]
    set car.color = blue

It creates the following output:

    color of alices-car: Color.RED
    color of bobs-car: Color.BLUE
    color of another-car: Color.BLACK

`another-car` gets the default color black as it is not set in the config file. You can change this default color in the config file by setting it before specifying a config id or after specifying the special config id `general` (`Config.default_config_id`). Note how this adds `general` to `MultiConfig.config_ids`.

    set traffic-law.speed-limit = 30
    set car.color = white

    [alices-car]
    set car.color = red

    [bobs-car]
    set car.color = blue

Creates the following output:

    color of general: Color.WHITE
    color of alices-car: Color.RED
    color of bobs-car: Color.BLUE
    color of another-car: Color.WHITE

## MultiConfig.reset

For normal `Config` instances you can restore a certain state of settings by calling `ConfigFile.save(comments=False)` (when you have the state that you want to restore later on) and `ConfigFile.load()` (where you want to restore the saved state).

This is not enough when you are using `MultiConfig` instances. Consider the following example:

    from confattr import MultiConfig, ConfigId, ConfigFile

    class Widget:

    	greeting = MultiConfig('greeting', 'hello world')

    	def __init__(self, name: str) -> None:
    		self.config_id = ConfigId(name)

    config_file = ConfigFile(appname='example')
    config_file.set_ui_callback(lambda msg: print(msg))

    w1 = Widget('w1')
    assert w1.greeting == 'hello world'

    config_file.save(comments=False)
    w1.greeting = 'hey you'
    assert w1.greeting == 'hey you'

    #MultiConfig.reset()   # This is missing
    config_file.load()
    assert w1.greeting == 'hello world'   # This fails!

The last assert fails because when saving the config no value for `w1` has been set yet. It is just falling back to the default value “hello world”. The saved config file is therefore:

    set greeting = 'hello world'

After the config was saved the value for `w1` is changed to “hey you”. When loading the config the default value is restored to `hello world` (which makes no difference because it has never been changed) but the value for `w1` is not changed because there is no value for `w1` in the config file.

The solution is to call `MultiConfig.reset()` before loading the config.

## Settings without default value

Sometimes there is no sane default value, several values are equally likely and if a wrong value is set the application won’t work at all. In those cases you can use `ExplicitConfig` which throws an exception if the user does not explicitly set a value in the config file.

    from confattr import ExplicitConfig, ConfigFile

    class Bus:

    	bitrate = ExplicitConfig('bus.bitrate', int, unit='')

    config_file = ConfigFile(appname='test')
    config_file.set_ui_callback(print)
    config_file.load()

    bus = Bus()
    print(f"bitrate: {bus.bitrate}")  # This throws a TypeError if bus.bitrate has not been set in the config file

## Include

Consider a backup application which synchronizes one or more directory pairs. The following code might be a menu for it to choose which side should be changed:

    from confattr import Config, MultiConfig, ConfigId, ConfigFile, NotificationLevel, Message
    import enum
    import urwid

    CMD_QUIT = 'quit'
    CMD_TOGGLE = 'toggle'
    CMD_IGNORE = 'ignore'

    urwid.command_map['q'] = CMD_QUIT
    urwid.command_map[' '] = CMD_TOGGLE
    urwid.command_map['i'] = CMD_IGNORE

    class Direction(enum.Enum):
    	SRC_TO_DST = ' > '
    	DST_TO_SRC = ' < '
    	IGNORE = ' | '
    	TWO_WAY = '<->'

    class DirectoryPair:

    	path_src = MultiConfig('path.src', '')
    	path_dst = MultiConfig('path.dst', '')
    	direction = MultiConfig('direction', Direction.SRC_TO_DST)

    	def __init__(self, config_id: ConfigId) -> None:
    		self.config_id = config_id

    	def toggle_direction(self) -> None:
    		if self.direction is Direction.SRC_TO_DST:
    			self.direction = Direction.DST_TO_SRC
    		else:
    			self.direction = Direction.SRC_TO_DST

    	def ignore(self) -> None:
    		self.direction = Direction.IGNORE

    class DirectoryPairWidget(urwid.WidgetWrap):  # type: ignore [misc]  # Class cannot subclass "WidgetWrap" (has type "Any") because urwid is not typed yet

    	def __init__(self, dirs: DirectoryPair) -> None:
    		self.model = dirs
    		self.widget_src = urwid.Text(dirs.path_src)
    		self.widget_dst = urwid.Text(dirs.path_dst)
    		self.widget_direction = urwid.Text('')
    		self.update_direction()
    		widget = urwid.Columns([self.widget_src, (urwid.PACK, self.widget_direction), self.widget_dst])
    		widget = urwid.AttrMap(widget, None, App.ATTR_FOCUS)
    		super().__init__(widget)

    	def selectable(self) -> bool:
    		return True

    	def keypress(self, size: 'tuple[int, ...]', key: str) -> 'str|None':
    		if not super().keypress(size, key):
    			return None  # pragma: no cover  # WidgetWrap does not consume any key presses

    		cmd = self._command_map[key]
    		if cmd == CMD_TOGGLE:
    			self.model.toggle_direction()
    			self.update_direction()
    		elif cmd == CMD_IGNORE:
    			self.model.ignore()
    			self.update_direction()
    		else:
    			return key

    		return None

    	def update_direction(self) -> None:
    		self.widget_direction.set_text(' %s ' % self.model.direction.value)

    class App:

    	ATTR_ERROR = NotificationLevel.ERROR.value
    	ATTR_INFO = NotificationLevel.INFO.value
    	ATTR_FOCUS = 'focus'

    	PALETTE = (
    		(ATTR_ERROR, 'dark red', 'default'),
    		(ATTR_INFO, 'dark blue', 'default'),
    		(ATTR_FOCUS, 'default', 'dark blue'),
    	)

    	notification_level = Config('notification-level', NotificationLevel.ERROR,
    		help = {
    			NotificationLevel.ERROR: 'show errors in the config file',
    			NotificationLevel.INFO: 'additionally show all settings which are changed in the config file',
    		}
    	)

    	def __init__(self) -> None:
    		self.config_file = ConfigFile(appname='example-include', notification_level=type(self).notification_level)
    		self.config_file.load()
    		self.directory_pairs = [DirectoryPair(config_id) for config_id in MultiConfig.config_ids]

    		self.body = urwid.ListBox([DirectoryPairWidget(dirs) for dirs in self.directory_pairs])
    		self.status_bar = urwid.Pile([])
    		self.frame = urwid.Frame(self.body, footer=self.status_bar)
    		self.config_file.set_ui_callback(self.on_config_message)

    	def run(self) -> None:
    		urwid.MainLoop(self.frame, palette=self.PALETTE, input_filter=self.input_filter, unhandled_input=self.unhandled_input, handle_mouse=False).run()

    	def on_config_message(self, msg: Message) -> None:
    		markup = (msg.notification_level.value, str(msg))
    		widget_options_tuple = (urwid.Text(markup), self.status_bar.options('pack'))
    		self.status_bar.contents.append(widget_options_tuple)
    		self.frame.footer = self.status_bar

    	def input_filter(self, keys: 'list[str]', raws: 'list[int]') -> 'list[str]':
    		self.status_bar.contents.clear()
    		Message.reset()
    		return keys

    	def unhandled_input(self, key: str) -> bool:
    		cmd = urwid.command_map[key]
    		if cmd == CMD_QUIT:
    			raise urwid.ExitMainLoop()
    		self.on_config_message(Message(NotificationLevel.ERROR, 'undefined key: %s' % key))
    		return True

    if __name__ == '__main__':
    	app = App()
    	app.run()

Let’s assume there are many more settings how to synchronize a pair of directories than just the direction. You might want to use the same synchronization settings for several directory pairs. You can write these settings to a separate config file and include it for the corresponding directory pairs:

main config file: config

    [documents]
    set path.src = ~/documents
    set path.dst = /media/usb1/documents
    include mirror

    [music]
    set path.src = ~/music
    set path.dst = /media/usb1/music
    include mirror

    [pictures]
    set path.src = ~/pictures
    set path.dst = /media/usb1/pictures
    include two-way

included config file: mirror

    set direction = src-to-dst

included config file: two-way

    set direction = two-way

This produces the following display:

    ~/documents              >  /media/usb1/documents
    ~/music                  >  /media/usb1/music
    ~/pictures              <-> /media/usb1/pictures

The config id of the included file starts with the value of the config id that the including file has at the moment of calling `include`. Otherwise the pattern shown above of reusing a config file for several config ids would not be possible.

If the included file changes the config id the config id is reset to the value it had at the beginning of the include when reaching the end of the included file. Otherwise changing an included file might unexpectedly change the meaning of the main config file or another config file which is included later on.

It is possible to change this default behavior by using `include --reset-config-id-before filename` or `include --no-reset-config-id-after filename`.

## Generating help

You can generate a help with `ConfigFile.write_help()` or `ConfigFile.get_help()`.

`ConfigFile.get_help()` is a wrapper around `ConfigFile.write_help()`. If you want to print the help to stdout `config_file.write_help(HelpWriter(None))` would be more efficient than `print(config_file.get_help())`. If you want to display the help in a graphical user interface you can implement a custom `FormattedWriter` which you can pass to `ConfigFile.write_help()` instead of parsing the output of `ConfigFile.get_help()`.

    from confattr import ConfigFile, Config, DictConfig, MultiConfig
    from enum import Enum, auto

    class Color(Enum):
    	RED = auto()
    	GREEN = auto()
    	BLUE = auto()

    Config('answer', 42, unit='', help={
    	42: 'The answer to everything',
    	23: '''
    		The natural number following 22
    		and preceding 24
    	''',
    })
    DictConfig('color', dict(foreground=Color.RED, background=Color.GREEN))
    MultiConfig('greeting', 'hello world', help='''
    	A text that may be displayed
    	if your computer is in a good mood.
    ''')

    if __name__ == '__main__':
    	config_file = ConfigFile(appname='exampleapp')
    	print(config_file.get_help())

Assuming the above file was contained in a package called `exampleapp` it would output the following:

    The first existing file of the following paths is loaded:
    - /home/username/.config/exampleapp/config
    - /etc/xdg/exampleapp/config

    This can be influenced with the following environment variables:
    - XDG_CONFIG_HOME
    - XDG_CONFIG_DIRS
    - EXAMPLEAPP_CONFIG_PATH
    - EXAMPLEAPP_CONFIG_DIRECTORY
    - EXAMPLEAPP_CONFIG_NAME

    You can also use environment variables to change the values of the
    settings listed under `set` command. The corresponding environment
    variable name is the name of the setting in all upper case letters
    with dots, hypens and spaces replaced by underscores and prefixed with
    "EXAMPLEAPP_".

    Lines in the config file which start with a `"` or `#` are ignored.

    The config file may contain the following commands:

    set
    ===
    usage: set [--raw] key1=val1 [key2=val2 ...]
           set [--raw] key [=] val

    Change the value of a setting.

    In the first form set takes an arbitrary number of arguments, each
    argument sets one setting. This has the advantage that several
    settings can be changed at once. That is useful if you want to bind a
    set command to a key and process that command with
    ConfigFile.parse_line() if the key is pressed.

    In the second form set takes two arguments, the key and the value.
    Optionally a single equals character may be added in between as third
    argument. This has the advantage that key and value are separated by
    one or more spaces which can improve the readability of a config file.

    You can use the value of another setting with %other.key% or an
    environment variable with ${ENV_VAR}. If you want to insert a literal
    percent character use two of them: %%. You can disable expansion of
    settings and environment variables with the --raw flag.

    data types:

      int:
        An integer number in python 3 syntax, as decimal (e.g. 42),
        hexadecimal (e.g. 0x2a), octal (e.g. 0o52) or binary (e.g.
        0b101010). Leading zeroes are not permitted to avoid confusion
        with python 2's syntax for octal numbers. It is permissible to
        group digits with underscores for better readability, e.g.
        1_000_000.

      str:
        A text. If it contains spaces it must be wrapped in single or
        double quotes.

    application wide settings:

      answer:
        an int

        42: The answer to everything
        23: The natural number following 22 and preceding 24

      color.background:
        one of red, green, blue

      color.foreground:
        one of red, green, blue

    settings which can have different values for different objects:
      You can specify the object that a value shall refer to by
      inserting the line `[config-id]` above. `config-id` must be
      replaced by the corresponding identifier for the object.

      greeting:
        a str

        A text that may be displayed if your computer is in a good
        mood.

    include
    =======
    usage: include [--reset-config-id-before | --no-reset-config-id-after]
                   path

    Load another config file.

    This is useful if a config file is getting so big that you want to
    split it up or if you want to have different config files for
    different use cases which all include the same standard config file to
    avoid redundancy or if you want to bind several commands to one key
    which executes one command with ConfigFile.parse_line().

    By default the loaded config file starts with which ever config id is
    currently active. This is useful if you want to use the same values
    for several config ids: Write the set commands without a config id to
    a separate config file and include this file for every config id where
    these settings shall apply.

    After the include the config id is reset to the config id which was
    active at the beginning of the include because otherwise it might lead
    to confusion if the config id is changed in the included config file.

    positional arguments:
      path                  The config file to load. Slashes are replaced
                            with the directory separator appropriate for
                            the current operating system. If the path
                            contains a space it must be wrapped in single
                            or double quotes.

    options:
      --reset-config-id-before
                            Ignore any config id which might be active
                            when starting the include
      --no-reset-config-id-after
                            Treat the included lines as if they were
                            written in the same config file instead of the
                            include command

    echo
    ====
    usage: echo [-l {info,error}] [-r] msg [msg ...]

    Display a message.

    Settings and environment variables are expanded like in the value of a
    set command.

    positional arguments:
      msg                   The message to display

    options:
      -l {info,error}, --level {info,error}
                            The notification level may influence the
                            formatting but messages printed with echo are
                            always displayed regardless of the
                            notification level.
      -r, --raw             Do not expand settings and environment
                            variables.

    help
    ====
    usage: help [cmd]

    Display help.

    positional arguments:
      cmd  The command for which you want help

The help is formatted on two levels:

1. `argparse.HelpFormatter` does the merging of lines, wrapping of lines and indentation. It formats the usage and all the command line arguments and options. Unfortunately “All the methods provided by the class are considered an implementation detail” according to it’s doc string. The only safe way to customize this level of formatting is by handing one of the predefined standard classes to the `formatter_class` parameter of the `ConfigFile` constructor:

   - `argparse.HelpFormatter`

   - `argparse.RawDescriptionHelpFormatter`

   - `argparse.RawTextHelpFormatter`

   - `argparse.ArgumentDefaultsHelpFormatter`

   - `argparse.MetavarTypeHelpFormatter`

   Additionally I provide another subclass `confattr.utils.HelpFormatter` which has a few class attributes for customization which I am trying to keep backward compatible. So you can subclass this class and change these attributes. But I cannot guarantee to always support the newest python version.

   If you want any more customization take a look at the source code but be prepared that you may need to change your code with any future python version.

2. `FormattedWriter` is intended to do stuff like underlining sections and inserting comment characters at the beginning of lines (when writing help to a config file). This package defines two subclasses: `ConfigFileWriter` which is used by default in `ConfigFile.save()` and `HelpWriter` which is used in `ConfigFile.get_help()`.

   If you want to customize this level of formatting implement your own `FormattedWriter` and override `ConfigFile.get_help()` or `ConfigFile.save_to_open_file()` to use your class.

## Custom data type for regular expressions

The following example defines a custom data type for regular expressions. Using this instead of a normal `str` has the following advantages:

- It is called a “regular expression” instead of a “str” which tells the user that this is a regular expression.

- It provides help for the user.

- `ConfigFile.load()`/`ConfigFile.parse_line()` will do the error handling for you. If the user enters a syntactically incorrect value then `re.compile()` throws an exception which will be caught and the error message is reported to the user by calling the callback which has been (or will be) passed to `ConfigFile.set_ui_callback()`.

    import re
    from collections.abc import Callable

    Regex: 'Callable[[str], re.Pattern[str]]'
    class Regex:  # type: ignore [no-redef]

    	type_name = 'regular expression'
    	help = '''
    	A regular expression in python syntax.
    	You can specify flags by starting the regular expression with `(?aiLmsux)`.
    	https://docs.python.org/3/library/re.html#regular-expression-syntax
    	'''

    	def __init__(self, pattern: str) -> None:
    		self._compiled_pattern: 're.Pattern[str]' = re.compile(pattern)

    	def __getattr__(self, attr: str) -> object:
    		return getattr(self._compiled_pattern, attr)

    	def __str__(self) -> str:
    		return self._compiled_pattern.pattern

    	def __repr__(self) -> str:
    		return f'{type(self).__name__}({self._compiled_pattern.pattern!r})'

This definition is actually included in `confattr.types` so you can easily use it like this:

    from confattr import Config, ConfigFile
    from confattr.types import Regex

    class App:

    	greeting = Config('greeting', Regex(r'(?i)(hello|hi)\b'), help='determine whether the user is polite or not')

    	def __init__(self) -> None:
    		self.config_file = ConfigFile(appname='example')
    		self.config_file.set_ui_callback(print)
    		self.config_file.load()

    	def save(self) -> None:
    		self.config_file.save()

    	def main(self) -> None:
    		inp = input('>>> ')
    		if self.greeting.match(inp):
    			print('nice to meet you')
    		else:
    			print('you are rude')

    if __name__ == '__main__':
    	a = App()
    	a.main()

Similarly there is also a `confattr.types.CaseInsensitiveRegex` class which compiles the regular expression with the `re.I` flag.

When you save the configuration like this

    	a.save()

the saved file looks like this:

    # Data types
    # ----------
    # regular expression:
    #   A regular expression in python syntax. You can specify flags by
    #   starting the regular expression with `(?aiLmsux)`.
    #   https://docs.python.org/3/library/re.html#regular-expression-syntax

    # greeting
    # --------
    # a regular expression
    # determine whether the user is polite or not
    set greeting = '(?i)(hello|hi)\b'

Custom data types to be used as a `value` of a setting must fulfill the following conditions:

- Return a string representation suitable for the config file from `__str__()`.

- Accept the return value of `__str__()` as argument for the constructor to create an equal object.

- Have a `help` attribute to give the user a description of the data type. Alternatively the help can be provided via `Primitive.help_dict`.

It can have a `type_name` attribute to specify how the type is called in the config file. If it is missing it is derived from the class name.

For more information on the supported data types see `Config`.

## Custom data type for paths

Similar to the regex example you can also define a custom data type to store paths. There is a `OptionalExistingDirectory` included in `confattr.types` but probably you will need to define a different type because paths differ too much from use case to use case:

- Does it point to files or directories?

- Does the target need to exist?

- Does an external drive need to be mounted? If so, at which point?

- Does a UUID need to be replaced by a mount point?

A very simple definition could look like this. Note how `os.path.expanduser()` is used to convert this type to a str. This is something that you will probably want in any case.

    import os

    class Path:

    	type_name = 'path'
    	help = 'The path to a file or directory.'

    	def __init__(self, value: str) -> None:
    		self.raw = value

    	def __str__(self) -> str:
    		return self.raw

    	def __repr__(self) -> str:
    		return '%s(%r)' % (type(self).__name__, self.raw)

    	def expand(self) -> str:
    		return os.path.expanduser(self.raw)

## Adding new commands to the config file syntax

You can extend this library by defining new commands which can be used in the config file. All you need to do is subclass `ConfigFileCommand` and implement the `run()` method. Additionally I recommend to provide a doc string explaining how to use the command in the config file. The doc string is used by `get_help()` which may be used by an in-app help. Optionally you can set `name` and `aliases` and implement the `save()` method.

Alternatively `ConfigFileArgparseCommand` can be subclassed instead, it aims to make the parsing easier and avoid redundancy in the doc string by using the `argparse` module. You must implement `init_parser()` and `run_parsed()`. You should give a doc string describing what the command does. In contrast to `ConfigFileCommand` `argparse` adds usage and the allowed arguments to the output of `get_help()` automatically.

For example you may want to add a new command to bind keys to whatever kind of command. The following example assumes urwid as user interface framework.

    import argparse
    from collections.abc import Sequence
    import urwid
    from confattr import ConfigFileArgparseCommand, ConfigFile, Config, NotificationLevel, Message

    class Map(ConfigFileArgparseCommand):

    	'''
    	bind a command to a key
    	'''

    	def init_parser(self, parser: argparse.ArgumentParser) -> None:
    		parser.add_argument('key', help='http://urwid.org/manual/userinput.html#keyboard-input')
    		parser.add_argument('cmd', help='any urwid command')

    	def run_parsed(self, args: argparse.Namespace) -> None:
    		urwid.command_map[args.key] = args.cmd
    		self.ui_notifier.show_info(f'map {args.key} {args.cmd!r}')

    if __name__ == '__main__':
    	# config
    	choices = Config('choices', ['vanilla', 'strawberry'])
    	urwid.command_map['enter'] = 'confirm'
    	config_file = ConfigFile(appname='map-exp', notification_level=Config('notification-level', NotificationLevel.ERROR))
    	config_file.load()

    	# show errors in config
    	palette = [(NotificationLevel.ERROR.value, 'dark red', 'default')]
    	status_bar = urwid.Pile([])
    	def on_config_message(msg: Message) -> None:
    		markup = (msg.notification_level.value, str(msg))
    		widget_options_tuple = (urwid.Text(markup), status_bar.options('pack'))
    		status_bar.contents.append(widget_options_tuple)
    		if 'frame' in globals():
    			frame._invalidate()
    	config_file.set_ui_callback(on_config_message)

    	def input_filter(keys: 'list[str]', raw: 'list[int]') -> 'list[str]':
    		status_bar.contents.clear()
    		Message.reset()
    		return keys

    	# a simple example app showing check boxes and printing the user's choice to stdout
    	def key_handler(key: str) -> bool:
    		cmd = urwid.command_map[key]
    		if cmd == 'confirm':
    			raise urwid.ExitMainLoop()
    		on_config_message(Message(NotificationLevel.ERROR, f'key {key!r} is not mapped'))
    		return True
    	checkboxes = [urwid.CheckBox(choice) for choice in choices.value]
    	frame = urwid.Frame(urwid.Filler(urwid.Pile(checkboxes)), footer=status_bar)
    	urwid.MainLoop(frame, palette=palette, input_filter=input_filter, unhandled_input=key_handler).run()

    	for ckb in checkboxes:
    		print(f'{ckb.label}: {ckb.state}')

Given the following config file it is possible to move the cursor upward and downward with `j` and `k` like in vim:

    map j 'cursor down'
    map k 'cursor up'
    map q 'confirm'

The help for the newly defined command looks like this:

    print(ConfigFile(appname='example').command_dict['map'].get_help())

    usage: map key cmd

    bind a command to a key

    positional arguments:
      key  http://urwid.org/manual/userinput.html#keyboard-input
      cmd  any urwid command

(All subclasses of `ConfigFileCommand` are saved in `ConfigFileCommand.__init_subclass__()` and can be retrieved with `ConfigFileCommand.get_command_types()`. The `ConfigFile` constructor uses that if `commands` is not given.)

## Writing custom commands to the config file

The previous example has shown how to define new commands so that they can be used in the config file. Let’s continue that example so that calls to the custom command `map` are written with `ConfigFile.save()`.

All you need to do for that is implementing the `ConfigFileCommand.save()` method.

`should_write_heading` is True if there are several commands which implement the `save()` method.

Experimental support for type checking `**kw` has been added in mypy 0.981. `SaveKwargs` depends on `typing.TypedDict` and therefore is not available before Python 3.8.

    import argparse
    import enum
    import typing
    if typing.TYPE_CHECKING:
    	from typing_extensions import Unpack  # This will hopefully be replaced by the ** syntax proposed in https://peps.python.org/pep-0692/
    	from confattr import SaveKwargs

    import urwid
    from confattr import ConfigFileArgparseCommand, FormattedWriter, ConfigFile, SectionLevel

    class Map(ConfigFileArgparseCommand):

    	'''
    	bind a command to a key
    	'''

    	def init_parser(self, parser: argparse.ArgumentParser) -> None:
    		parser.add_argument('key', help='http://urwid.org/manual/userinput.html#keyboard-input')
    		parser.add_argument('cmd', help='any urwid command')

    	def run_parsed(self, args: argparse.Namespace) -> None:
    		urwid.command_map[args.key] = args.cmd

    	def save(self, writer: FormattedWriter, **kw: 'Unpack[SaveKwargs]') -> None:
    		if self.should_write_heading:
    			writer.write_heading(SectionLevel.SECTION, 'Key bindings')

    		for key, cmd in sorted(urwid.command_map._command.items(), key=lambda key_cmd: str(key_cmd[1])):
    			if isinstance(cmd, enum.Enum):
    				cmd = cmd.value
    			quoted_key = self.config_file.quote(key)
    			quoted_cmd = self.config_file.quote(cmd)
    			writer.write_command(f'map {quoted_key} {quoted_cmd}')

    if __name__ == '__main__':
    	ConfigFile(appname='example').save()

However, `urwid.command_map` contains more commands than the example app uses so writing all of them might be confusing. Therefore let’s add a keyword argument to write only the specified commands:

    import argparse
    import enum
    import typing
    from collections.abc import Sequence

    import urwid
    from confattr import ConfigFileArgparseCommand, FormattedWriter, ConfigFile, SectionLevel

    if typing.TYPE_CHECKING:
    	from typing_extensions import Unpack  # This will hopefully be replaced by the ** syntax proposed in https://peps.python.org/pep-0692/
    	from confattr import SaveKwargs

    	class MapSaveKwargs(SaveKwargs, total=False):
    		urwid_commands: 'Sequence[str]'

    class Map(ConfigFileArgparseCommand):

    	'''
    	bind a command to a key
    	'''

    	def init_parser(self, parser: argparse.ArgumentParser) -> None:
    		parser.add_argument('key', help='http://urwid.org/manual/userinput.html#keyboard-input')
    		parser.add_argument('cmd', help='any urwid command')

    	def run_parsed(self, args: argparse.Namespace) -> None:
    		urwid.command_map[args.key] = args.cmd

    	def save(self, writer: FormattedWriter, **kw: 'Unpack[MapSaveKwargs]') -> None:
    		if self.should_write_heading:
    			writer.write_heading(SectionLevel.SECTION, 'Key bindings')

    		commands = kw.get('urwid_commands', sorted(urwid.command_map._command.values()))
    		for cmd in commands:
    			for key in urwid.command_map._command.keys():
    				if urwid.command_map[key] == cmd:
    					if isinstance(cmd, enum.Enum):
    						cmd = cmd.value
    					quoted_key = self.config_file.quote(key)
    					quoted_cmd = self.config_file.quote(cmd)
    					writer.write_command(f'map {quoted_key} {quoted_cmd}')

    if __name__ == '__main__':
    	urwid_commands = [urwid.CURSOR_UP, urwid.CURSOR_DOWN, urwid.ACTIVATE, 'confirm']
    	mapkw: 'MapSaveKwargs' = dict(urwid_commands=urwid_commands)
    	kw: 'SaveKwargs' = mapkw
    	config_file = ConfigFile(appname='example')
    	config_file.save(**kw)

This produces the following config file:

    # ============
    # Key bindings
    # ============
    map up 'cursor up'
    map down 'cursor down'
    map ' ' activate
    map enter activate

If you don’t care about Python < 3.8 you can import `SaveKwargs` normally and save a line when calling `ConfigFile.save()`:

    kw: SaveKwargs = MapSaveKwargs(urwid_commands=...)
    config_file.save(**kw)

## Customizing the config file syntax

If you want to make minor changes to the syntax of the config file you can subclass the corresponding command, i.e. `Set` or `Include`.

For example if you want to use a `key: value` syntax you could do the following.

I am setting `name` to an empty string (i.e. `DEFAULT_COMMAND`) to make this the default command which is used if an unknown command is encountered. This makes it possible to use this command without writing out it’s name in the config file.

    from confattr import ParseException, ConfigId, FormattedWriter, SectionLevel
    from confattr.configfile import Set

    import typing
    if typing.TYPE_CHECKING:
    	from confattr import SaveKwargs
    	from typing_extensions import Unpack
    	from collections.abc import Sequence

    class SimpleSet(Set, replace=True):

    	name = ''

    	SEP = ':'

    	def run(self, cmd: 'Sequence[str]') -> None:
    		ln = self.config_file.context_line
    		if self.SEP not in ln:
    			raise ParseException(f'missing {self.SEP} between key and value')
    		key, value = ln.split(self.SEP)
    		value = value.lstrip()
    		self.parse_key_and_set_value(key, value)

    	def save_config_instance(self, writer: FormattedWriter, instance: 'Config[object]', config_id: 'ConfigId|None', **kw: 'Unpack[SaveKwargs]') -> None:
    		# this is called by Set.save
    		if kw['comments']:
    			self.write_config_help(writer, instance)
    		value = self.config_file.format_value(instance, config_id)
    		#value = self.config_file.quote(value)  # not needed because run uses line instead of cmd
    		writer.write_command(f'{instance.key}{self.SEP} {value}')

    if __name__ == '__main__':
    	from confattr import Config, ConfigFile
    	color = Config('favorite color', 'white')
    	subject = Config('favorite subject', 'math')
    	config_file = ConfigFile(appname='example')
    	config_file.load()
    	config_file.set_ui_callback(lambda msg: print(msg))
    	print(color.value)
    	print(subject.value)

Then a config file might look like this:

    favorite color: sky blue
    favorite subject: computer science

Please note that it’s still possible to use the `include` command. If you want to avoid that use

    from confattr import ConfigFileCommand
    from confattr.configfile import Include
    ConfigFileCommand.delete_command_type(Include)

If you want to make bigger changes like using JSON you need to subclass `ConfigFile`.

## Auto completion with prompt_toolkit

(This example uses prompt_toolkit as user interface, the next example uses urwid.)

`confattr` can also be used as backend for a command line interface. In that case you call `ConfigFile.parse_line()` directly with the input that you get from the user.

You can define custom commands as shown in Adding new commands to the config file syntax and if you use different `ConfigFile` objects for the config file and for the command line interface you can specify different commands to be available with the `commands` parameter.

In order to make a command line usable, however, it also needs an auto completion. The most interesting part of this example is therefore probably the `ConfigFileCompleter` class which acts an adapter between the suggestions generated by `ConfigFile.get_completions()` and the way how prompt_toolkit wants to have them.

I am using different `ConfigFile` objects for the config file and for the command line in order to have different settings and default values for the `notification_level`. I am passing `show_line_always` to the `ConfigFile` which is responsible for the command line so that the input line is not repeated in every response. Note how I am patching the `config_file` of the `include` command to always use the `ConfigFile` object for the config file, no matter whether it is called from the command line or from the config file. Since version 1.1.0 you can use the `quickstart` module to create the `ConfigFile` objects with configurable notification levels, see the quickstart example.

    import os
    import argparse
    from collections.abc import Iterator

    from confattr import Config, DictConfig, ConfigFile, Message, NotificationLevel, ConfigFileArgparseCommand, Primitive
    from confattr.configfile import Echo as OriginalEcho

    from prompt_toolkit import print_formatted_text, PromptSession
    from prompt_toolkit.formatted_text import FormattedText
    from prompt_toolkit.completion import Completer, Completion, CompleteEvent
    from prompt_toolkit.document import Document
    from prompt_toolkit.styles import ANSI_COLOR_NAMES, NAMED_COLORS

    APPNAME = 'auto-completion-example'

    class Color(Primitive[str]):

    	type_name = 'color'

    	DEFAULT = 'default'
    	CHOICES = (DEFAULT,) + tuple(ANSI_COLOR_NAMES) + tuple(c.lower() for c in NAMED_COLORS.keys())

    	def __init__(self) -> None:
    		super().__init__(str, allowed_values=self.CHOICES, type_name=self.type_name)

    colors = DictConfig('color', {NotificationLevel.ERROR: 'ansired', NotificationLevel.INFO: Color.DEFAULT}, type=Color())

    class ConfigFileCompleter(Completer):

    	def __init__(self, config_file: ConfigFile) -> None:
    		super().__init__()
    		self.config_file = config_file

    	def get_completions(self, document: Document, complete_event: CompleteEvent) -> 'Iterator[Completion]':
    		start_of_line, completions, end_of_line = self.config_file.get_completions(document.text, document.cursor_position)
    		for word in completions:
    			yield Completion(start_of_line + word.rstrip(os.path.sep), display=word, start_position=-document.cursor_position)

    class Quit(ConfigFileArgparseCommand):

    	def init_parser(self, parser: argparse.ArgumentParser) -> None:
    		pass

    	def run_parsed(self, args: argparse.Namespace) -> None:
    		raise EOFError()

    class Echo(OriginalEcho, replace=True):

    	def init_parser(self, parser: argparse.ArgumentParser) -> None:
    		parser.add_argument('-c', '--color', choices=Color.CHOICES)
    		parser.add_argument('msg', nargs=argparse.ONE_OR_MORE)

    	def run_parsed(self, args: argparse.Namespace) -> None:
    		msg = ' '.join(self.config_file.expand(m) for m in args.msg)
    		colored_print(args.color, msg)

    def colored_print(color: 'str|None', msg: str) -> None:
    		if color and color != Color.DEFAULT:
    			print_formatted_text(FormattedText([(color, str(msg))]))
    		else:
    			print_formatted_text(str(msg))

    def main() -> None:
    	# creating 2 ConfigFile instances with different notification level filters
    	notification_level_config = Config('notification-level.config-file', NotificationLevel.ERROR)
    	notification_level_cli = Config('notification-level.cli', NotificationLevel.INFO)
    	config_file = ConfigFile(appname=APPNAME, notification_level=notification_level_config)
    	cli = ConfigFile(appname=APPNAME, notification_level=notification_level_cli, show_line_always=False)
    	cli.command_dict['include'].config_file = config_file
    	config_file.load()

    	# show errors in config
    	def on_config_message(msg: Message) -> None:
    		color = colors.get(msg.notification_level)
    		colored_print(color, str(msg))
    	config_file.set_ui_callback(on_config_message)
    	cli.set_ui_callback(on_config_message)

    	# main user interface
    	p: 'PromptSession[str]' = PromptSession('>>> ', completer=ConfigFileCompleter(cli))
    	while True:
    		Message.reset()
    		try:
    			cli.parse_line(p.prompt())
    		except EOFError:
    			break

    if __name__ == '__main__':
    	main()

## Auto completion with urwid

This example is pretty similar to the previous example but it uses urwid instead of prompt_toolkit.

Unfortunately urwid does not provide a widget with auto completion. There is urwid_readline but it’s auto completion feature is not powerful enough to work well with this library. Instead this example implements a custom `EditWithAutoComplete` widget.

    from confattr import Config, ConfigFile, Message, NotificationLevel, ConfigFileArgparseCommand
    import argparse
    import urwid
    urwid.command_map['esc'] = 'quit'
    urwid.command_map['tab'] = 'complete cycle forward'
    urwid.command_map['shift tab'] = 'complete cycle backward'
    urwid.command_map['ctrl u'] = 'clear before'
    urwid.command_map['ctrl k'] = 'clear after'

    APPNAME = 'auto-completion-example'

    class EditWithAutoComplete(urwid.Edit):  # type: ignore [misc]  # Class cannot subclass "Edit" (has type "Any")

    	def __init__(self, caption: str, edit_text: str, *, config_file: ConfigFile) -> None:
    		super().__init__(caption, edit_text)
    		self.config_file = config_file
    		self.completions: 'list[str]|None' = None
    		self.completion_index = -1

    	def keypress(self, size: 'tuple[int]', key: str) -> 'str|None':
    		if not super().keypress(size, key):
    			self.on_change()
    			return None
    		cmd = self._command_map[key]
    		if cmd == 'complete cycle forward':
    			self.complete_cycle(+1)
    		elif cmd == 'complete cycle backward':
    			self.complete_cycle(-1)
    		elif cmd == 'clear before':
    			self.set_edit_text(self.edit_text[self.edit_pos:])
    			self.set_edit_pos(0)
    		elif cmd == 'clear after':
    			self.set_edit_text(self.edit_text[:self.edit_pos])
    			self.set_edit_pos(len(self.edit_text))
    		else:
    			return key

    		return None

    	def on_change(self) -> None:
    		self.completions = None
    		self.completion_index = -1

    	def complete_cycle(self, direction: int) -> None:
    		if self.completions is None:
    			self.default_line = self.edit_text
    			self.default_pos = self.edit_pos
    			self.start_of_line, self.completions, self.end_of_line = self.config_file.get_completions(self.edit_text, self.edit_pos)
    		self.completion_index += direction
    		if self.completion_index < -1:
    			self.completion_index = len(self.completions) - 1
    		elif self.completion_index >= len(self.completions):
    			self.completion_index = -1
    		if self.completion_index == -1:
    			self.set_edit_text(self.default_line)
    			self.set_edit_pos(self.default_pos)
    		else:
    			completed = self.start_of_line + self.completions[self.completion_index]
    			self.set_edit_text(completed + self.end_of_line)
    			self.set_edit_pos(len(completed))

    class Quit(ConfigFileArgparseCommand):

    	def init_parser(self, parser: argparse.ArgumentParser) -> None:
    		pass

    	def run_parsed(self, args: argparse.Namespace) -> None:
    		raise urwid.ExitMainLoop()

    def main() -> None:
    	# creating 2 ConfigFile instances with different notification level filters
    	notification_level_config = Config('notification-level.config-file', NotificationLevel.ERROR)
    	notification_level_cli = Config('notification-level.cli', NotificationLevel.INFO)
    	config_file = ConfigFile(appname=APPNAME, notification_level=notification_level_config)
    	cli = ConfigFile(appname=APPNAME, notification_level=notification_level_cli, show_line_always=False)
    	cli.command_dict['include'].config_file = config_file
    	config_file.load()

    	# show errors in config
    	palette = [(NotificationLevel.ERROR.value, 'dark red', 'default')]
    	status_bar = urwid.Pile([])
    	def on_config_message(msg: Message) -> None:
    		markup = (msg.notification_level.value, str(msg))
    		widget_options_tuple = (urwid.Text(markup), status_bar.options('pack'))
    		status_bar.contents.append(widget_options_tuple)
    		frame_widget._invalidate()

    	# main user interface
    	def keypress(key: str) -> None:
    		cmd = urwid.command_map[key]
    		if cmd == 'activate':
    			Message.reset()
    			status_bar.contents.clear()
    			if cli.parse_line(edit.edit_text):
    				edit.set_edit_text('')
    	edit = EditWithAutoComplete('>>> ', '', config_file=cli)
    	main_widget = urwid.Filler(urwid.LineBox(edit))
    	frame_widget = urwid.Frame(main_widget, footer=status_bar)

    	config_file.set_ui_callback(on_config_message)
    	cli.set_ui_callback(on_config_message)
    	urwid.MainLoop(frame_widget, unhandled_input=keypress, palette=palette, handle_mouse=False).run()

    if __name__ == '__main__':
    	main()

## Opening the config file in a text editor

You can use `confattr.types.SubprocessCommandWithAlternatives.editor()` to create a command for opening the config file. It respects the `EDITOR`/`VISUAL` environment variables on non-Windows systems and includes some fall backs if the environment variables are not set.

`config_file.save()` returns the file name of the config file.

You can pass a context manager factory to `context` in case you need to `stop()` an urwid screen before opening the file and `start()` it again after closing the file.

    from confattr import Config, ConfigFile
    from confattr.types import SubprocessCommandWithAlternatives as Command

    class App:

    	editor = Config('editor', Command.editor(visual=False),
    	                help="The editor to be used when opening the config file")

    	def __init__(self) -> None:
    		self.config_file = ConfigFile(appname='example-app')
    		self.config_file.load()

    	def edit_config(self) -> None:
    		self.editor \
    			.replace(Command.WC_FILE_NAME, self.config_file.save(if_not_existing=True)) \
    			.run(context=None)

    if __name__ == '__main__':
    	app = App()
    	app.edit_config()

## Config without classes

If you want to use `Config` objects without custom classes you can access the value via the `Config.value` attribute:

    from confattr import Config, ConfigFile

    backend = Config('urwid.backend', 'auto', allowed_values=('auto', 'raw', 'curses'))

    config_file = ConfigFile(appname='example')
    config_file.load()
    config_file.set_ui_callback(lambda msg: print(msg))

    print(backend.value)

Given the following config file (the location of the config file is determined by `ConfigFile.iter_config_paths()`):

    set urwid.backend = curses

The script will give the following output:

    curses

## Testing your application

I recommend doing static type checking with mypy in strict mode and dynamic testing with pytest. tox can run both in a single command and automatically handles virtual environments for you. While you can configure tox to run your tests on several specific python versions you can also simply use `py3` which will use whatever python 3 version you have installed. For packaging and publishing your application I recommend flit over the older setuptools because flit is much more intuitive and less error prone.

For dynamic testing you need to consider two things:

1. Your application must not load a config file from the usual paths so that the tests always have the same outcome no matter which user is running them and on which computer. You can achieve that by setting one of the attributes `ConfigFile.config_directory` or `ConfigFile.config_path` or one of the corresponding environment variables `APPNAME_CONFIG_DIRECTORY` or `APPNAME_CONFIG_PATH` in the setup of your tests.

   In pytest you can do this with an auto use fixture. tmp_path creates an empty directory for you and monkeypatch cleans up for you after the test is done. If all of your tests are defined in a single file you can define this fixture in that file. Otherwise the definition goes into conftest.py.

       import pytest
       import pathlib
       from confattr import ConfigFile

       @pytest.fixture(autouse=True)
       def reset_config(tmp_path: pathlib.Path, monkeypatch: pytest.MonkeyPatch) -> None:
       	monkeypatch.setattr(ConfigFile, 'config_directory', str(tmp_path))

2. Your tests need to change settings in order to test all possibilities but all settings which have been changed in a test must be reset after each test so that the tests always have the same outcome no matter whether they are executed all together or alone.

   Of course you could just save a config file in the setup and load it (with `env=False`) in the teardown (and don’t forget to call MultiConfig.reset). But keep in mind that you may have many settings and many tests and that they may become more in the future. It is more efficient to let monkeypatch clean up only those settings that you have changed.

   Let’s assume we want to test our car from the first example:

       from sut import Car
       import pytest

       def test_car_accelerate(monkeypatch: pytest.MonkeyPatch) -> None:
       	monkeypatch.setattr(Car.speed_limit, 'value', 10)

       	c1 = Car()
       	c1.accelerate(5)
       	c1.accelerate(5)
       	with pytest.raises(ValueError):
       		c1.accelerate(5)

       	assert c1.speed == 10

   If we want to change the value of a `MultiConfig` setting like in this example for a specific object we would use `monkeypatch.setitem()` to change `MultiConfig.values`:

       from sut import Car, Color
       from confattr import ConfigId
       import pytest

       def test_car_color(monkeypatch: pytest.MonkeyPatch) -> None:
       	monkeypatch.setattr(Car.color, 'value', Color.WHITE)
       	monkeypatch.setitem(Car.color.values, ConfigId('alices-car'), Color.BLUE)
       	monkeypatch.setitem(Car.color.values, ConfigId('bobs-car'), Color.GREEN)

       	alices_car = Car(ConfigId('alices-car'))
       	bobs_car = Car(ConfigId('bobs-car'))
       	another_car = Car(ConfigId('another-car'))

       	assert alices_car.color is Color.BLUE
       	assert bobs_car.color is Color.GREEN
       	assert another_car.color is Color.WHITE

## Environment variables

Settings can be changed via environment variables, too. For example if you have an application called `example-app` with the following code

    from confattr import Config, ConfigFile
    greeting = Config('ui.greeting', 'hello world')
    ConfigFile(appname='example-app').load()
    print(greeting.value)

and you call it like this

    EXAMPLE_APP_UI_GREETING='hello environment' example-app

it will print

    hello environment

For the exact rules how the names of the environment variables are created are described in `ConfigFile.get_env_name()`.

Environment variables which start with the name of the application but do not match a setting (and are not one those listed below) or have an invalid value are reported as `ERROR` to the callback registered with `ConfigFile.set_ui_callback()`.

Furthermore this library is influenced by the following environment variables:

- `XDG_CONFIG_HOME` defines the base directory relative to which user-specific configuration files should be stored on Linux. [1] [2]

- `XDG_CONFIG_DIRS` defines the preference-ordered set of base directories to search for configuration files in addition to the `XDG_CONFIG_HOME` base directory on Linux. The directories in `XDG_CONFIG_DIRS` should be separated with a colon. [1] [2]

- `APPNAME_CONFIG_PATH` defines the value of `ConfigFile.config_path`. [2] [3]

- `APPNAME_CONFIG_DIRECTORY` defines the value of `ConfigFile.config_directory`. [2] [3]

- `APPNAME_CONFIG_NAME` defines the value of `ConfigFile.config_name`. [2] [3]

[1] (1,2)

is not queried directly but outsourced to platformdirs, xdgappdirs or appdirs, see `ConfigFile.get_app_dirs()`. See also https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html#variables.

[2] (1,2,3,4,5)

influences `ConfigFile.iter_config_paths()`

[3] (1,2,3)

where `APPNAME` is the value of `appname` which is passed to the constructor of `ConfigFile` but in all upper case letters and hyphens, dots and spaces replaced by underscores, see `ConfigFile.get_env_name()`.