1. Introduction to Viper
Understanding the need for a configuration solution in Go applications
To build reliable and maintainable software, developers need to separate configuration from application logic. This allows you to adjust the application's behavior without changing the codebase. A configuration solution enables this separation by facilitating the externalization of configuration data.
Go applications can greatly benefit from such a system, especially as they grow in complexity and face various deployment environments, such as development, staging, and production. Each of these environments may require different settings for database connections, API keys, port numbers, and more. Hardcoding these values can be problematic and error-prone, as it leads to multiple code paths to maintain different configurations and increases the risk of sensitive data exposure.
A configuration solution like Viper addresses these concerns by providing a unified approach that supports diverse configuration needs and formats.
Overview of Viper and its role in managing configurations
Viper is a comprehensive configuration library for Go applications, aiming to be the de facto solution for all configuration needs. It aligns with the practices stipulated in the Twelve-Factor App methodology, which encourages storing configuration in the environment to achieve portability between execution environments.
Viper plays a pivotal role in managing configurations by:
- Reading and unmarshaling configuration files in various formats such as JSON, TOML, YAML, HCL, and more.
- Overriding configuration values with environment variables, thus adhering to the external configuration principle.
- Binding and reading from command line flags to allow dynamic setting of configuration options at runtime.
- Allowing defaults to be set within the application for configuration options not provided externally.
- Watching for changes in configuration files and live reloading, providing flexibility and reducing downtime for config changes.
2. Installing and Setup
Installing Viper using Go modules
To add Viper to your Go project, ensure that your project is already using Go modules for dependency management. If you already have a Go project, you most likely have a go.mod file at the root of your project. If not, you can initialize Go modules by running the following command:
go mod init <module-name>
Replace <module-name> with your project's name or path. Once you have Go Modules initialized in your project, you can add Viper as a dependency:
go get github.com/spf13/viper
This command will fetch the Viper package and record its version in your go.mod file.
Initializing Viper in a Go project
To start using Viper within your Go project, you first need to import the package and then create a new instance of Viper or use the predefined singleton. Below is an example of how to do both:
package main
import (
"fmt"
"github.com/spf13/viper"
)
func main() {
// Using the Viper singleton, which is pre-configured and ready to use
viper.SetDefault("serviceName", "My Awesome Service")
// Alternatively, creating a new Viper instance
myViper := viper.New()
myViper.SetDefault("serviceName", "My New Service")
// Accessing a configuration value using the singleton
serviceName := viper.GetString("serviceName")
fmt.Println("Service Name is:", serviceName)
// Accessing a configuration value using the new instance
newServiceName := myViper.GetString("serviceName")
fmt.Println("New Service Name is:", newServiceName)
}
In the code above, SetDefault is used to define a default value for a configuration key. The GetString method retrieves a value. When you run this code, it prints out both the service names we configured using both the singleton instance as well as the new instance.
3. Reading and Writing Configuration Files
Working with configuration files is a core feature of Viper. It allows your application to externalize its configuration so that it can be updated without needing to recompile the code. Below, we'll explore setting up various configuration formats and show how to read from and write to these files.
Setting up configuration formats (JSON, TOML, YAML, HCL, etc.)
Viper supports several configuration formats such as JSON, TOML, YAML, HCL, etc. To start, you must set the name and the type of the configuration file Viper should look for:
v := viper.New()
v.SetConfigName("app") // Configuration file name without the extension
v.SetConfigType("yaml") // or "json", "toml", "yml", "hcl", etc.
// Configuration file search paths. Add multiple paths if your
// configuration file location varies.
v.AddConfigPath("$HOME/.appconfig") // Typical UNIX user config location
v.AddConfigPath("/etc/appconfig/") // UNIX system-wide configuration path
v.AddConfigPath(".") // The working directory
Reading from and writing to configuration files
Once the Viper instance knows where to look for the configuration files and what to look for, you can ask it to read the configuration:
if err := v.ReadInConfig(); err != nil {
if _, ok := err.(viper.ConfigFileNotFoundError); ok {
// Config file was not found; ignore if desired or handle otherwise
log.Printf("No config file found. Using default values, and/or environment variables.")
} else {
// Config file was found but another error was encountered
log.Fatalf("Error reading config file, %s", err)
}
}
To write modifications back to the config file, or to create a new one, Viper offers several methods. Here's how you write the current configuration to a file:
err := v.WriteConfig() // Writes current config to predefined path set by `v.SetConfigName` and `v.AddConfigPath`
if err != nil {
log.Fatalf("Error writing config file, %s", err)
}
Establishing default configuration values
Default values serve as fallbacks in case a key has not been set in the configuration file or by environment variables:
v.SetDefault("ContentDir", "content")
v.SetDefault("LogLevel", "debug")
v.SetDefault("Database.Port", 5432)
// A more complex data structure for the default value
viper.SetDefault("Taxonomies", map[string]string{
"tag": "tags",
"category": "categories",
})
4. Managing Environment Variables and Flags
Viper is not just limited to configuration files—it can also manage environment variables and command-line flags, which is particularly useful when dealing with environment-specific settings.
Binding environment variables and flags to Viper
Binding environment variables:
v.AutomaticEnv() // Automatically search for environment variable keys that match with Viper's keys
v.SetEnvPrefix("APP") // Prefix for environment variables to distinguish them from others
v.BindEnv("port") // Bind the PORT environment variable (e.g., APP_PORT)
// You can also match environment variables with different names to keys in your app
v.BindEnv("database_url", "DB_URL") // This tells Viper to use the value of DB_URL environmental variable for the "database_url" configuration key
Binding flags using pflag, a Go package for flag parsing:
var port int
// Define a flag using pflag
pflag.IntVarP(&port, "port", "p", 808, "Port for the application")
// Bind the flag to a Viper key
pflag.Parse()
if err := v.BindPFlag("port", pflag.Lookup("port")); err != nil {
log.Fatalf("Error binding flag to key, %s", err)
}
Handling environment-specific configurations
An application often needs to operate differently in various environments (development, staging, production, etc.). Viper can consume configuration from environment variables that can override settings in the config file, allowing for environment-specific configurations:
v.SetConfigName("config") // The default configuration filename
// Configuration may be overridden by environment variables
// with the prefix APP and the rest of the key in uppercase
v.SetEnvPrefix("APP")
v.AutomaticEnv()
// In a production environment, you may use the APP_PORT environment variable to override the default port
fmt.Println(v.GetString("port")) // Output will be the value of APP_PORT if set, else the value from the config file or the default
Remember to handle the differences between environments within your application code, if needed, based on the configurations loaded by Viper.
5. Remote Key/Value Store Support
Viper provides robust support for managing application configuration using remote key/value stores such as etcd, Consul, or Firestore. This allows configurations to be centralized and dynamically updated across distributed systems. Additionally, Viper enables secure handling of sensitive configuration through encryption.
Integrating Viper with Remote Key/Value Stores (etcd, Consul, Firestore, etc.)
To start using Viper with remote key/value stores, you need to perform a blank import of the viper/remote package in your Go application:
import _ "github.com/spf13/viper/remote"
Let's look at an example integrating with etcd:
import (
"log"
"github.com/spf13/viper"
_ "github.com/spf13/viper/remote"
)
func initRemoteConfig() {
viper.SetConfigType("json") // Set the type of the remote configuration file
viper.AddRemoteProvider("etcd", "http://127...1:4001", "/config/myapp.json")
err := viper.ReadRemoteConfig() // Attempt to read the remote config
if err != nil {
log.Fatalf("Failed to read remote config: %v", err)
}
log.Println("Successfully read remote configuration")
}
func main() {
initRemoteConfig()
// Your application logic here
}
In this example, Viper connects to an etcd server running on http://127...1:4001 and reads the configuration located at /config/myapp.json. When working with other stores like Consul, replace "etcd" with "consul" and adjust the provider-specific parameters accordingly.
Managing Encrypted Configurations
Sensitive configurations, like API keys or database credentials, should not be stored in plain text. Viper allows encrypted configurations to be stored in a key/value store and decrypted in your application.
To use this feature, ensure encrypted settings are stored in your key/value store. Then leverage Viper's AddSecureRemoteProvider. Here is an example of utilizing this with etcd:
import (
"log"
"github.com/spf13/viper"
_ "github.com/spf13/viper/remote"
)
func initSecureRemoteConfig() {
const secretKeyring = "/path/to/secret/keyring.gpg" // Path to your keyring file
viper.SetConfigType("json")
viper.AddSecureRemoteProvider("etcd", "http://127...1:4001", "/config/myapp.json", secretKeyring)
err := viper.ReadRemoteConfig()
if err != nil {
log.Fatalf("Unable to read remote config: %v", err)
}
log.Println("Successfully read and decrypted remote configuration")
}
func main() {
initSecureRemoteConfig()
// Your application logic here
}
In the above example, AddSecureRemoteProvider is used, specifying the path to a GPG keyring that contains the keys necessary for decryption.
6. Watching and Handling Config Changes
One of Viper's powerful features is its ability to monitor and respond to configuration changes in real time, without restarting the application.
Monitoring Configuration Changes and Re-reading Configurations
Viper uses the fsnotify package to watch for changes to your configuration file. You can setup a watcher to trigger events whenever the configuration file changes:
import (
"log"
"github.com/fsnotify/fsnotify"
"github.com/spf13/viper"
)
func watchConfig() {
viper.WatchConfig()
viper.OnConfigChange(func(e fsnotify.Event) {
log.Printf("Config file changed: %s", e.Name)
// Here you can read the updated configuration if necessary
// Perform any action such as reinitializing services or updating variables
})
}
func main() {
viper.SetConfigName("myapp")
viper.AddConfigPath(".")
err := viper.ReadInConfig()
if err != nil {
log.Fatalf("Error reading config file, %s", err)
}
watchConfig()
// Your application logic here
}
Triggers for Updating Configurations in a Running Application
In a running application, you may want to update configurations in response to various triggers, such as a signal, a time-based job, or an API request. You can structure your application to refresh its internal state based on Viper's re-read configurations:
import (
"os"
"os/signal"
"syscall"
"time"
"log"
"github.com/spf13/viper"
)
func setupSignalHandler() {
signalChannel := make(chan os.Signal, 1)
signal.Notify(signalChannel, syscall.SIGHUP) // Listening for SIGHUP signal
go func() {
for {
sig := <-signalChannel
if sig == syscall.SIGHUP {
log.Println("Received SIGHUP signal. Reloading configuration...")
err := viper.ReadInConfig() // Re-read the configuration
if err != nil {
log.Printf("Error re-reading config: %s", err)
} else {
log.Println("Configuration reloaded successfully.")
// Reconfigure your application based on the new configuration here
}
}
}
}()
}
func main() {
viper.SetConfigName("myapp")
viper.AddConfigPath(".")
err := viper.ReadInConfig()
if err != nil {
log.Fatalf("Error reading config file, %s", err)
}
setupSignalHandler()
for {
// Application's main logic
time.Sleep(10 * time.Second) // Simulate some work
}
}
In this example, we're setting up a handler to listen for a SIGHUP signal. When received, Viper reloads the configuration file and the application should then update its configurations or state as necessary.
Always remember to test these configurations to ensure your application can handle dynamic updates gracefully.