How do *you* solve the configuration problem

[magnus0re]

I’ve worked with or touched many configurable systems.
Some are home-brew, some are open source tools, and all have some kind of configuration in one file format or the other…
*.edn, *.yaml, *.xml, *.json, (jsonnet) *.config. *.cfg, *.ini, *.js, *.py… the list goes on. Don’t get me started on kubernetes yaml and terrorform!
It can even be as simple as an opts argument to a function.

I’ve even worked with projects that had multiple types of config files…
I don’t want to remember those dark times… anyways.

Some issues that results from a configuration.

1. What the options are are not obvious, not documented, or really hard to find in the documentation.
   A documentation issue. But perhaps it is a deeper problem as well?
2. What the values of the options can be is a mystery
   That is to say, what is the valid matrix of options that the option parameters can be in. if option A is X, for example, option B is not allowed to be Y. Or even, what is the valid subset of strings that can go into this json field???
3. How do I change my configuration so that it works with your new update of the software?
   For example, in version 1, the default of X is 512, but in version 2, the default is 256. However, it had never been required (or documented) in the settings file (true story, this broke something at work once).

So I’m really curious to see how you solve these three problems.

• documenting the available options; how do you communicate to the users of your software/code/libraries how to properly work with it, and how do you enable them to do so?
• what options are incompatible. How do you describe this?
• what options need to be changed in the new version

I’m thinking that in an ideal world, I would have a piece of the code that generated a valid configuration, and also updated old configurations to a new one. But this can imply inescapable pitfalls. It feels kind of related to the expression problem.

I feel that a config file is like a puzzle piece that needs to fit just right, and it is easy to write them in such a manner that it becomes an impossible puzzle with exponentially many options and combinations of options.

Maybe someone already made a software for this, with config migrations, and all that good stuff , if so, please tell me about it.

spec might work for some parts of this problem, but definitely not all…

It’s difficult to google this subject for me, because all I get is ‘How to choose your configuration file format’ type thing.

I would like to hear your experience, and your way of dealing with it… maybe there is some wonderful insight out there.

[seabass]

I think that, to some extent, this can solved completely with good, human-written documentation. System administration tools, for instance, often have one or more manual pages included with the program. Libraries often also have good documentation, but only on the function-level - that makes some of the more complex interactions between options harder to express.

The manual for GNU Wget is a lovely example of an extensive and well written manual - it’s current 76 pages long and every option is documented with at least a paragraph, and with regard to both the kind of nuances that you describe and the ‘big-picture’ usage:

‘--config=FILE’
Specify the location of a startup file you wish to use instead of the default one(s).
Use –no-config to disable reading of config files. If both –config and –no-config are
given, –no-config is ignored

This probably isn’t the wonderful insight you were hoping for, but I think bog-standard manuals have their place even for a language with function docstrings!