yaml configuration: the best of a bad bunch

Over the years, I’ve worked on a variety of systems, each with its own approach to configuration. From system.properties in Java apps to wp-config.php in WordPress, .ini and .toml files, JSON, and, of course, environment variables. None of these solutions are perfect. When dealing with application configuration you try to aim for a balance between: readability, maintainability, and flexibility. These qualities can often be at odds with each other.

After years of using various methods for configuration I have eventually settled on what I believe to be the most pragmatic approach. Combining YAML for structured configuration with environment variables. I’ve found this hybrid method aligns with modern development practices, especially in containerised environments, while remaining developer-friendly.

Golang’s built-in os.ExpandEnv function can handle basic environment variable expansion, but it lacks support for default values when variables are undefined. To address this limitation, I extended the functionality in my YAML configuration library to provide a fallback behaviour, ensuring graceful handling of missing environment variables.

My Approach: YAML + Enhanced `os.ExpandEnv` #

Using YAML as the base configuration format allows me to define structured defaults that are easy to read, maintain, and version-control. Environment variables then handle dynamic values for environment-specific customisations, with custom logic to support default values for additional flexibility.

Why YAML is Superior to Environment Variables #

While environment variables are popular thanks to methologies like Twelve-Factor applications and containerised runtimes, they often create hidden dependencies when developers use os.Getenv directly within code. This practice can obscure configuration requirements and make application behavior unpredictably dependent on environment state.

I believe YAML offers a more robust approach through. Maintaining all configuration in a single, structured file loaded at startup, a YAML configuration file ensures settings are visible, traceable, and maintainable. A YAML configuration file establishes a clear boundary between configuration and code, enabling consistent behavior, simplified validation, and more effective testing.

Of course environment variables remain valuable for specific use cases like managing secrets and environment-specific overrides. I believe however, YAML proves more effective as the primary configuration method.

YAML with Placeholders #

Here’s an example of a YAML configuration file:

log:
  level: "info"

database:
  postgres:
    url: "postgres://postgres:postgres@127.0.0.1:5432/default?sslmode=disable"

web:
  host: 127.0.0.1
  port: 8974

This file might be named config.local.yaml to clearly indicate it’s for local development. It provides sensible defaults that work out of the box for other developers.

In production, we can easily override these values using environment variables, as per the Twelve-Factor App principles. For instance:

log:
  level: "$LOG_LEVEL"

database:
  postgres:
    url: "$DATABASE_URL"

web:
  host: "$WEB_HOST"
  port: "$WEB_PORT"

LOG_LEVEL=debug
DATABASE_URL=postgres://prod_user:prod_pass@db.example.com:5432/prod
WEB_PORT=80

This approach works well for dynamic settings like credentials or deployment-specific URLs. However, some configuration variables—such as the web port—rarely change once defined. Keeping these “static defaults” in the YAML file avoids overloading the environment with unnecessary variables.

To merge YAML configuration with environment variables, I use os.ExpandEnv. However, this function has a key limitation.

Why `os.ExpandEnv` Falls Short #

While os.ExpandEnv is excellent for basic environment variable substitution, it doesn’t support fallback values. If an environment variable isn’t defined, the placeholder is simply replaced with an empty string.

This limitation can lead to brittle configurations where missing environment variables cause unexpected behaviour. For example, if DATABASE_URL isn’t set in the example above, it will result in an invalid database configuration.

To address this, I extended the behaviour of os.ExpandEnv to support default values using a ${ENV_NAME:default} syntax.

This is where some custom logic and using os.LookupEnv can be combined together to provide even more flexibility.

Enhanced Logic for Default Values #

Here’s the function I use to handle both $ENV_NAME and ${ENV_NAME:default} placeholders:

// parseEnv replaces $ENV_NAME and ${ENV_NAME:default} placeholders.
// Default values are only supported with ${ENV_NAME:default}.
func parseEnv(input string) string {
	// Regex to match both $ENV_NAME and ${ENV_NAME:default}
	re := regexp.MustCompile(`\$(\w+)|\$\{(\w+)(?::([^}]*))?\}`)

	return re.ReplaceAllStringFunc(input, func(match string) string {
		parts := re.FindStringSubmatch(match)
		if len(parts) == 0 {
			return match // No match, return as-is
		}

		// Check if it's $ENV_NAME or ${ENV_NAME:default}
		if parts[1] != "" {
			// $ENV_NAME style
			varName := parts[1]
			if value, found := os.LookupEnv(varName); found {
				return value
			}
			return "" // If not found, replace with an empty string
		}

		// ${ENV_NAME:default} style
		varName := parts[2]
		defaultValue := parts[3]
		if value, found := os.LookupEnv(varName); found {
			return value
		}
		return defaultValue // Fallback to default value if not found
	})
}

How It Works #

$ENV_NAME: This is replaced by the corresponding environment variable’s value, or an empty string if not set.
${ENV_NAME}: This is replaced by the corresponding environment variable’s value, or an empty string if not set.
${ENV_NAME:default}: This adds a fallback mechanism. If the variable isn’t set, the specified default value is used instead.

Example #

Given the following environment variables:

LOG_LEVEL=debug

The configuration file:

log:
  level: "${LOG_LEVEL:info}"

database:
  postgres:
    url: "${DATABASE_URL:postgres://localhost:5432/default}"

web:
  host: "${WEB_HOST:127.0.0.1}"
  port: "${WEB_PORT:8080}"

Would result in:

log:
  level: "debug"

database:
  postgres:
    url: "postgres://localhost:5432/default"

web:
  host: "127.0.0.1"
  port: "8080"

Benefits of This Approach #

This proposed solution combines structured YAML files with environment variables to create a robust configuration system. YAML provides a version-controlled foundation with clear defaults, while environment variables enable flexible overrides for different deployment environments. Extending Golang’s environment variable handling with custom default logic, we ensure graceful fallbacks while maintaining clarity. This hybrid approach delivers both the stability of structured configuration with flexibility needed for modern deployment scenarios.

Trade-Offs #

While this configuration approach offers a balance of flexibility and structure, it does comes with notable trade-offs. The custom parsing logic will increases the maintenance overhead. The system may also require additional validation steps to ensure configuration completeness and correctness.

Conclusion: A Balanced Approach #

For me, combining YAML’s readability and structure with the flexibility of environment variables—and augmenting Go’s os.LookupEnv to support default values—creates a configuration system that balances developer productivity and operational needs.

It is not a perfect solution, and there are limitations to this approach, but it strikes a balance that has proven reliable and effective across a variety of projects.

For a more complete solution for parsing YAML configuration, I’ve created my own library for loading, parsing, and validating configurations. If you’re interested in checking it out, you can find it on my GitHub: aranw/yamlcfg.

Update: an unexpected bug #

After using my updated YAML Configuration library in production I came across an unexpected bug where configuration values that contained a dollar $ symbol would trigger the environment variable parsing logic unexpectedly.

I wrote a follow up post regarding this and the fix ultimatelly deciding to remove support for $ENV_VAR style variables.

You can checkout the post here.

My Approach: YAML + Enhanced os.ExpandEnv #