Introduction and examples — confattr v0.5.1 documentation

# Introduction and examples

## Config and ConfigFile

`confattr` (config attributes) is a python library to make applications configurable. This library defines the `Config` class to create attributes which can be changed in a config file. It uses the descriptor protocol to return it’s value when used as an instance attribute.

    from confattr import Config

    class Car:

    	speed_limit = Config('traffic-law.speed-limit', 50, unit='km/h')

    	def __init__(self) -> None:
    		self.speed = 0

    	def accelerate(self, value: int) -> None:
    		new_speed = self.speed + value
    		if new_speed > self.speed_limit:
    			raise ValueError('you are going too fast')

    		self.speed = new_speed

If you want to access the Config object itself you need to access it as a class attribute:

    	def print_config(self) -> None:
    		print('{key}: {val}'.format(key=type(self).speed_limit.key, val=self.speed_limit))

You load a config file with a `ConfigFile` object. In order to create a ConfigFile instance you need to provide a callback function which informs the user if the config file contains invalid lines. This callback function takes two arguments: (1) a `NotificationLevel` which says whether the notification is an error or an information and (2) the message to be presented to the user, either a `str` or a `BaseException`. When you load a config file with `ConfigFile.load()` all `Config` objects which are set in the config file are updated automatically.

It is recognized automatically that the setting `traffic-law.speed-limit` has an integer value. A value given in a config file is therefore automatically parsed to an integer. If the parsing fails the user interface callback function is called.

    if __name__ == '__main__':
    	from confattr import ConfigFile
    	config_file = ConfigFile(appname=__package__)
    	config_file.load()

    	# Print error messages which occurred while loading the config file.
    	# In this easy example it would have been possible to register the callback before calling load
    	# but in the real world the user interface will depend on the values set in the config file.
    	# Therefore the messages are stored until a callback is added.
    	config_file.set_ui_callback(lambda lvl, msg: print(msg))

    	c1 = Car()
    	print('speed_limit: %s' % c1.speed_limit)

Given the following config file (the location of the config file is determined by `ConfigFile.iter_config_paths()`):

    set traffic-law.speed-limit = 30

The script will give the following output:

    speed_limit: 30

You can save the current configuration with `ConfigFile.save()` if you want to write it to the default location or with `ConfigFile.save_file()` if you want to specify the path yourself.

    	filename = config_file.save()
    	print('configuration was written to %s' % filename)

This will write the following file:

    # traffic-law.speed-limit
    # -----------------------
    # an int in km/h
    set traffic-law.speed-limit = 30

## Config file syntax

I have looked at the config files of different applications trying to come up with a syntax as intuitive as possible. Two extremes which have heavily inspired me are the config files of vim and ranger. Quoting and inline comments work like in a POSIX shell (except that there is no difference between single quotes and double quotes) as I am using `shlex.split()` to split the lines.

The `set` command has two different forms. I recommend to not mix them in order to improve readability.

- `set key1=val1 [key2=val2 ...]` (inspired by vimrc)

  `set` takes an arbitrary number of arguments, each argument sets one setting.

  Has the advantage that several settings can be changed at once. This is useful if you want to bind a set command to a key and process that command with `ConfigFile.parse_line()` if the key is pressed.

  If the value contains one or more spaces it must be quoted. `set greeting='hello world'` and `set 'greeting=hello world'` are equivalent.

- `set key [=] val` (inspired by ranger config)

  `set` takes two arguments, the key and the value. Optionally a single equals character may be added in between as third argument.

  Has the advantage that key and value are separated by one or more spaces which can improve the readability of a config file.

  If the value contains one or more spaces it must be quoted: `set greeting 'hello world'` and `set greeting = 'hello world'` are equivalent.

I recommend to not use spaces in key names so that they don’t need to be wrapped in quotes.

It is possible to include another config file:

- `include filename`

  If `filename` is a relative path it is relative to the directory of the config file it appears in.

## Different values for different objects

A `Config` object always returns the same value, regardless of the owning object it is an attribute of:

    from confattr import Config

    class Car:
    	speed_limit = Config('traffic-law.speed-limit', 50, unit='km/h')

    c1 = Car()
    c2 = Car()

    print(c1.speed_limit, c2.speed_limit)
    c2.speed_limit = 30  # don't do this, this is misleading!
    print(c1.speed_limit, c2.speed_limit)

Output:

    50 50
    30 30

If you want to have different values for different objects you need to use `MultiConfig` instead. This requires the owning object to have a special attribute called `config_id`. All objects which have the same `config_id` share the same value. All objects which have different `config_id` can have different values (but don’t need to have different values).

    import enum
    from confattr import Config, MultiConfig, ConfigId, ConfigFile

    class Color(enum.Enum):
    	RED = 'red'
    	YELLOW = 'yellow'
    	GREEN = 'green'
    	BLUE = 'blue'
    	WHITE = 'white'
    	BLACK = 'black'

    class Car:

    	speed_limit = Config('traffic-law.speed-limit', 50, unit='km/h')
    	color = MultiConfig('car.color', Color.BLACK)

    	def __init__(self, config_id: ConfigId) -> None:
    		self.config_id = config_id

    if __name__ == '__main__':
    	config_file = ConfigFile(appname=__package__)
    	config_file.load()

    	config_file.set_ui_callback(lambda lvl, msg: print(msg))

    	cars = []
    	for config_id in MultiConfig.config_ids:
    		cars.append(Car(config_id))
    	cars.append(Car(ConfigId('another-car')))

    	for car in cars:
    		print('color of %s: %s' % (car.config_id, car.color))

Given the following config file:

    set traffic-law.speed-limit = 30

    [alices-car]
    set car.color = red

    [bobs-car]
    set car.color = blue

It creates the following output:

    color of alices-car: Color.RED
    color of bobs-car: Color.BLUE
    color of another-car: Color.BLACK

`another-car` gets the default color black as it is not set in the config file. You can change this default color in the config file by setting it before specifying a config id or after specifying the special config id `general` (`Config.default_config_id`). Note how this adds `general` to `MultiConfig.config_ids`.

    set traffic-law.speed-limit = 30
    set car.color = white

    [alices-car]
    set car.color = red

    [bobs-car]
    set car.color = blue

Creates the following output:

    color of general: Color.WHITE
    color of alices-car: Color.RED
    color of bobs-car: Color.BLUE
    color of another-car: Color.WHITE

## Custom data types

It is possible to use custom data types. You can use that for example to avoid repeating a part of help which applies to several settings:

    import re
    from confattr import Config, ConfigFile

    class Regex(str):

    	type_name = 'regular expression'
    	help = 'https://docs.python.org/3/library/re.html#regular-expression-syntax'

    class Parser:

    	re_mount_output = Config('udisksctl.mount-output-pattern', Regex(r'^.*?(?P<mountpath>/(\S+/)*[^/]+?)\.?$'),
    		help='a regular expression to parse the output of `udisksctl mount`. Must contain a named group called "mountpath".')
    	re_unlock_output = Config('udisksctl.unlock-output-pattern', Regex(r'^.*?(?P<unlockpath>/(\S+/)*[^/]+?)\.?$'),
    		help='a regular expression to parse the output of `udisksctl unlock`. Must contain a named group called "unlockpath".')

    	def compile_regex(self) -> None:
    		'''
    		This must be called every time after the config file has been loaded.
    		'''
    		self.reo_mount_output = re.compile(self.re_mount_output)
    		self.reo_unlock_output = re.compile(self.re_unlock_output)

    if __name__ == '__main__':
    	ConfigFile(appname=__package__).save()

This exports the following config file:

    # Data types
    # ----------
    # - regular expression
    #   https://docs.python.org/3/library/re.html#regular-expression-syntax

    # udisksctl.mount-output-pattern
    # ------------------------------
    # a regular expression
    # a regular expression to parse the output of `udisksctl mount`. Must contain a named group called "mountpath".
    set udisksctl.mount-output-pattern = '^.*?(?P<mountpath>/(\S+/)*[^/]+?)\.?$'

    # udisksctl.unlock-output-pattern
    # -------------------------------
    # a regular expression
    # a regular expression to parse the output of `udisksctl unlock`. Must contain a named group called "unlockpath".
    set udisksctl.unlock-output-pattern = '^.*?(?P<unlockpath>/(\S+/)*[^/]+?)\.?$'

`__str__()` must return a string representation suitable for the config file and the constructor must create an equal object if it is passed the return value of `__str__()`. This is fulfilled by inheriting from `str`. `type_name` is a special str attribute which specifies how the type is called in the config file. If it is missing it is derived from the class name. `help` is a special str attribute which contains a description which is printed in the config file. If it is missing the doc string is used instead.

`confattr.types` defines several such types, including the above definition of `Regex`.

For more information on the supported data types see `Config`.

## Adding new commands to the config file syntax

You can extend this library by defining new commands which can be used in the config file. All you need to do is subclass `ConfigFileCommand` and implement the `ConfigFileCommand.run()` method. Additionally I recommend to provide a doc string explaining how to use the command in the config file. The doc string is used by `get_help()` which may be used by an in-app help. Optionally you can set `ConfigFileCommand.name` and `ConfigFileCommand.aliases` and implement the `ConfigFileCommand.save()` method.

Alternatively `ConfigFileArgparseCommand` can be subclassed instead, it aims to make the parsing easier and avoid redundancy in the doc string by using the `argparse` module. You must implement `init_parser()` and `run_parsed()`. Note that `init_parser()` is a class method. You should give a doc string describing what the command does. In contrast to `ConfigFileCommand` `argparse` adds usage and the allowed arguments to the output of `ConfigFileArgparseCommand.get_help()` automatically.

For example you may want to add a new command to bind keys to whatever kind of command. The following example assumes urwid as user interface framework.

    import argparse
    from collections.abc import Sequence
    import urwid
    from confattr import ConfigFileArgparseCommand, ConfigFile, Config, NotificationLevel

    class Map(ConfigFileArgparseCommand):

    	'''
    	bind a command to a key
    	'''

    	@classmethod
    	def init_parser(cls) -> None:
    		cls.parser.add_argument('key', help='http://urwid.org/manual/userinput.html#keyboard-input')
    		cls.parser.add_argument('cmd', help='any urwid command')

    	def run_parsed(self, args: argparse.Namespace) -> None:
    		urwid.command_map[args.key] = args.cmd

    if __name__ == '__main__':
    	# config
    	choices = Config('choices', ['vanilla', 'strawberry'])
    	urwid.command_map['enter'] = 'confirm'
    	config_file = ConfigFile(appname=__package__)
    	config_file.load()

    	# show errors in config
    	palette = [(NotificationLevel.ERROR.value, 'dark red', 'default')]
    	status_bar = urwid.Pile([])
    	def on_config_message(lvl: NotificationLevel, msg: 'str|BaseException') -> None:
    		markup = (lvl.value, str(msg))
    		widget_options_tuple = (urwid.Text(markup), status_bar.options('pack'))
    		status_bar.contents.append(widget_options_tuple)
    	config_file.set_ui_callback(on_config_message)

    	# a simple example app showing check boxes and printing the user's choice to stdout
    	def key_handler(key: str) -> None:
    		cmd = urwid.command_map[key]
    		if cmd == 'confirm':
    			raise urwid.ExitMainLoop()
    	checkboxes = [urwid.CheckBox(choice) for choice in choices.value]
    	frame = urwid.Frame(urwid.Filler(urwid.Pile(checkboxes)), footer=status_bar)
    	urwid.MainLoop(frame, palette=palette, unhandled_input=key_handler).run()

    	for ckb in checkboxes:
    		print(f'{ckb.label}: {ckb.state}')

Given the following config file it is possible to move the cursor upward and downward with `j` and `k` like in vim:

    map j 'cursor down'
    map k 'cursor up'
    map q 'confirm'

The help for the newly defined command looks like this:

    print(Map.get_help())

    usage: map key cmd

    bind a command to a key

    positional arguments:
      key  http://urwid.org/manual/userinput.html#keyboard-input
      cmd  any urwid command

(All subclasses of `ConfigFileCommand` are saved in `ConfigFileCommand.__init_subclass__()` and can be retrieved with `ConfigFileCommand.get_command_types()`. The `ConfigFile` constructor uses that if `commands` is not given.)

## Writing custom commands to the config file

The previous example has shown how to define new commands so that they can be used in the config file. Let’s continue that example so that calls to the custom command `map` are written with `ConfigFile.save()`.

All you need to do for that is implementing the `ConfigFileCommand.save()` method.

Experimental support for type checking `**kw` has been added in mypy 0.981. `SaveKwargs` depends on `typing.TypedDict` and therefore is not available before Python 3.8.

    import argparse
    import typing
    if typing.TYPE_CHECKING:
    	from typing_extensions import Unpack  # This will hopefully be replaced by the ** syntax proposed in https://peps.python.org/pep-0692/
    	from confattr import SaveKwargs

    import urwid
    from confattr import ConfigFileArgparseCommand, ConfigFile

    class Map(ConfigFileArgparseCommand):

    	'''
    	bind a command to a key
    	'''

    	@classmethod
    	def init_parser(cls) -> None:
    		cls.parser.add_argument('key', help='http://urwid.org/manual/userinput.html#keyboard-input')
    		cls.parser.add_argument('cmd', help='any urwid command')

    	def run_parsed(self, args: argparse.Namespace) -> None:
    		urwid.command_map[args.key] = args.cmd

    	def save(self, f: typing.TextIO, **kw: 'Unpack[SaveKwargs]') -> None:
    		for key, cmd in sorted(urwid.command_map._command.items(), key=lambda key_cmd: str(key_cmd[1])):
    			quoted_key = self.config_file.quote(key)
    			quoted_cmd = self.config_file.quote(cmd)
    			print(f'map {quoted_key} {quoted_cmd}', file=f)

    if __name__ == '__main__':
    	ConfigFile(appname=__package__).save()

However, `urwid.command_map` contains more commands than the example app uses so writing all of them might be confusing. Therefore let’s add a keyword argument to write only the specified commands:

    import argparse
    from collections.abc import Sequence
    import typing

    import urwid
    from confattr import ConfigFileArgparseCommand, ConfigFile

    if typing.TYPE_CHECKING:
    	from typing_extensions import Unpack  # This will hopefully be replaced by the ** syntax proposed in https://peps.python.org/pep-0692/
    	from confattr import SaveKwargs

    	class MapSaveKwargs(SaveKwargs, total=False):
    		urwid_commands: 'Sequence[str]'

    class Map(ConfigFileArgparseCommand):

    	'''
    	bind a command to a key
    	'''

    	@classmethod
    	def init_parser(cls) -> None:
    		cls.parser.add_argument('key', help='http://urwid.org/manual/userinput.html#keyboard-input')
    		cls.parser.add_argument('cmd', help='any urwid command')

    	def run_parsed(self, args: argparse.Namespace) -> None:
    		urwid.command_map[args.key] = args.cmd

    	def save(self, f: typing.TextIO, **kw: 'Unpack[MapSaveKwargs]') -> None:
    		commands = kw.get('urwid_commands', sorted(urwid.command_map._command.values()))
    		for cmd in commands:
    			for key in urwid.command_map._command.keys():
    				if urwid.command_map[key] == cmd:
    					quoted_key = self.config_file.quote(key)
    					quoted_cmd = self.config_file.quote(cmd)
    					print(f'map {quoted_key} {quoted_cmd}', file=f)

    if __name__ == '__main__':
    	urwid_commands = [urwid.CURSOR_UP, urwid.CURSOR_DOWN, urwid.ACTIVATE, 'confirm']
    	mapkw: 'MapSaveKwargs' = dict(urwid_commands=urwid_commands)
    	kw: 'SaveKwargs' = mapkw
    	config_file = ConfigFile(appname=__package__)
    	config_file.save(**kw)

This produces the following config file:

    map up 'cursor up'
    map down 'cursor down'
    map ' ' activate
    map enter activate

If you don’t care about Python < 3.8 you can import `SaveKwargs` normally and save a line when calling `ConfigFile.save()`:

    kw: SaveKwargs = MapSaveKwargs(urwid_commands=...)
    config_file.save(**kw)

## Customizing the config file syntax

If you want to make minor changes to the syntax of the config file you can subclass the corresponding command, i.e. `Set` or `Include`.

For example if you want to use a `key: value` syntax you could do the following.

I am setting `name` to an empty string (i.e. `confattr.DEFAULT_COMMAND`) to make this the default command which is used if an unknown command is encountered. This makes it possible to use this command without writing out it’s name in the config file.

    from confattr import Set, ParseException, ConfigId

    import typing
    if typing.TYPE_CHECKING:
    	from confattr import ParseSplittedLineKwargs, SaveKwargs
    	from typing_extensions import Unpack
    	from collections.abc import Sequence

    class SimpleSet(Set, replace=True):

    	name = ''

    	SEP = ':'

    	def run(self, cmd: 'Sequence[str]', **kw: 'Unpack[ParseSplittedLineKwargs]') -> None:
    		ln = kw['line']
    		if self.SEP not in ln:
    			raise ParseException(f'missing {self.SEP} between key and value')
    		key, value = ln.split(self.SEP)
    		value = value.lstrip()
    		self.parse_key_and_set_value(key, value)

    	def save_config_instance(self, f: typing.TextIO, instance: 'Config[object]', config_id: 'ConfigId|None', **kw: 'Unpack[SaveKwargs]') -> None:
    		# this is called by Set.save
    		if kw['comments']:
    			self.write_help(f, instance)
    		value = self.format_value(instance, config_id)
    		#value = self.config_file.quote(value)  # not needed because run uses line instead of cmd
    		ln = f'{instance.key}{self.SEP} {value}\n'
    		f.write(ln)

    if __name__ == '__main__':
    	from confattr import Config, ConfigFile
    	color = Config('favorite color', 'white')
    	subject = Config('favorite subject', 'math')
    	config_file = ConfigFile(appname=__package__)
    	config_file.load()
    	config_file.set_ui_callback(lambda lvl, msg: print(msg))
    	print(color.value)
    	print(subject.value)

Then a config file might look like this:

    favorite color: sky blue
    favorite subject: computer science

Please note that it’s still possible to use the `include` command. If you want to avoid that use

    from confattr import ConfigFileCommand, Include
    ConfigFileCommand.delete_command_type(Include)

If you want to make bigger changes like using JSON you need to subclass `ConfigFile`.

## Config without classes

If you want to use `Config` objects without custom classes you can access the value via the `Config.value` attribute:

    from confattr import Config, ConfigFile

    backend = Config('urwid.backend', 'auto', allowed_values=('auto', 'raw', 'curses'))

    config_file = ConfigFile(appname=__package__)
    config_file.load()
    config_file.set_ui_callback(lambda lvl, msg: print(msg))

    print(backend.value)

Given the following config file (the location of the config file is determined by `ConfigFile.iter_config_paths()`):

    set urwid.backend = curses

The script will give the following output:

    curses

## Environment variables

This library is influenced by the following environment variables:

- `XDG_CONFIG_HOME` defines the base directory relative to which user-specific configuration files should be stored. [1] [2]

- `XDG_CONFIG_DIRS` defines the preference-ordered set of base directories to search for configuration files in addition to the `XDG_CONFIG_HOME` base directory. The directories in `XDG_CONFIG_DIRS` should be separated with a colon. [1] [2]

- `CONFATTR_FILENAME` defines the value of `ConfigFile.FILENAME`. [2]

[1] (1,2)

is not queried directly but outsourced to platformdirs, xdgappdirs or appdirs, see `ConfigFile.get_app_dirs()`. See also https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html#variables.

[2] (1,2,3)

influences `ConfigFile.iter_config_paths()`