Last Updated on October 21, 2024 by jt

Introduction

Situation

Most of our applications are supported by external services such as a database server, an SMS gateway, or services such as PayPal. Such services can exist in more than one environment i.e development and production environments. If we want to connect to the production environment we must pass through the development environment first. This implies that as we develop our applications there would be a need to switch between these environments. This is because configuration data, such as connection settings are unique per environment.

Problem

In this article, let us assume that we are developing a payment service that connects to an external payment provider.  The merchant account values are not the same in the production and development environment. This means that each time we switch environments, we must modify the values and recompile the code which is not good.

Solution

The best solution to our problem is to externalize our application configuration data. This way we do not need to recompile our code each time we switch environments. We will be able to override configuration settings values with less effort. Our Spring application will be able to read merchant account values from external sources such as environment variables, command-line arguments, and property files.

Configuration Data Externalization

Property-Sources

There are various ways of externalizing configuration data of a Spring application. We can use environment variables, property files (i.e in YAML format or with .properties file extension), and command-line arguments to specify our application properties. We can also store the property files in custom locations and then tell our Spring application where to search for them.

Property Files

By default Spring application loads properties from the application.properties or application.yml from the following locations listed below in the order of precedence (i.e properties file in locations higher in the list override those defined in lower locations) and adds them to the environment:

1. config subdirectory of the current directory
2. current directory
3. config package in the classpath
4. on the classpath root

The default configuration file name is application. We can specify a different name if we want by using the environment property key spring.config.name. See the example below we have overridden the default Spring configuration name to new_name.

spring.config.name=newname

Custom Location

We can externalize the application property or YAML files using the environment property spring.config.location. We can set its value to point to custom locations which can be anywhere and by doing so we will be overriding the default locations. See example below:

spring.config.location={path_to_configuration_file/directory}

Note: When we want to specify a directory location then we must make sure that the value of spring.config.location  ends with / (for example  spring.config.location=classpath:/config/) and that the configuration file name is the default. It is also possible to specify additional locations to be searched before the default locations using the property key spring.config.additional-location.

spring.config.additional-location={path_to_configuration_file/directory}

Spring Boot also supports wildcard locations and this feature becomes useful in environments where there are multiple sources of config properties i.e in Kubernetes environments. For example, if you have some Redis configuration and some MySQL configuration, you might want to keep those two pieces of configuration separate, while requiring that both those are present in an application.properties that the app can bind to. This might result in two separate application.properties files mounted at different locations such as /config/redis/application.properties and /config/mysql/application.properties. In such a case, having a wildcard location of config/*/ will result in both files being processed.

File Formats

The application properties file can be in YAML format or can have a file extension of .properties. If we place these two property files in the same configuration folder, the application.properties file will take precedence over the application.yml file. The following code snippet shows our merchant account settings when defined in each type of property file.

application.properties

merchantaccount.name=Maureen Sindiso Mpofu
merchantaccount.username=momoe
merchantaccount.code=771222279
merchantaccount.number=100
merchantaccount.currency=ZWL
server.port: 9092

application.yml

merchantaccount:
 name: Maureen Sindiso Mpofu
 username: momoe
 code: MCA1230
 number: 771222279
 currency: ZWL

YAML vs .properties file

YAML  is a human-friendly data serialization standard and is commonly used in configuration files. It is a superset of JSON and is very convenient when specifying hierarchical configuration data. We prefer YAML files because they are clearer and readable especially when compared to the .properties file and besides its readability, it has other features that are very useful like type safety, etc.

To load the YAML file our Spring Application requires the SnakeYAML library on the classpath. In the example code provided we used the Spring Boot’s starters, hence there is no need to include the library on the classpath.

Multiple profiles

A YAML allows us to specify multiple profiles in a single configuration file whereas with .property file we may need to provide a configuration file for each profile. Let us look at the example below.

a) YAML file:

application.yml

spring:
 profiles:
   active: development
---
spring:
 profiles: development
merchantaccount:
 name: Maureen Sindiso Mpofu
 username: momoe
 code: MCA1230
 number: 771222279
 currency: ZWL
server:
 port: 9090
---
spring:
 profiles: production
server:
 port: 9093
merchantaccount:
 name: Maureen Sindiso Mpofu
 username: momoe
 code: MCA1234
 number: 771222279
 currency: ZWD

b) .properties file:

In case of a .properties file if we want to define two profiles then we need to create an individual configuration file for each profile. The name of each configuration file must be sufficed with a -{profile_name}.properties. See the example below for our development and production application profiles.

application-development.properties

merchantaccount.name=Maureen Sindiso Mpofu
merchantaccount.username=momoe
merchantaccount.code=771222279
merchantaccount.number=100
merchantaccount.currency=ZWL
server.port: 9092

application-production.properties

merchantaccount.name=Maureen Sindiso Mpofu
merchantaccount.username=momoe
merchantaccount.code=MCA1234
merchantaccount.number=771222279
merchantaccount.currency=ZWD
server.port: 9093

We may need the default application.properties if there are properties that are common to both the development and production profiles.

application.properties

spring.profiles.active=development
#default port number
server.port=9091

The sample configuration data in the code snippets above have set the development profile as the active profile. Therefore during application start-up, the property values defined in this profile will take precedence over the production profile. However, let us remember that we can still override the profile-specific settings using the command line arguments.

You can read more about profiles in this Spring Profile post.

Readability

YAML supports lists and maps as hierarchical properties and compared to the .properties file, the YAML version is more readable. Imagine we want to set up connection settings for live and test environments, we will first set the connection names as a list and then map the connection name to its corresponding URL using a Map, as indicated below. Let us see how YAML simplifies this configuration as compared to the .properties file.

application.yml

connection:
 names:
   - test
   - live
 addresses:
   test: http://host/test
   live: http://host/live

application.properties

#list
connection.names[0]=test
connection.names[1]=live
#map
connection.addresses.test=http://host/test
connection.addresses.live= http://host/live

We have provided test cases of verifying the mappings in the test packages of the example code of this article.

Command-Line Arguments

When we pass a command-line argument, the Spring application converts it to a property and adds it to the Spring Environment. We can use these command-line arguments to configure our application settings, for example, the following command line args will override the application server port defined in any other property source. If we are running our application using maven command or java command we will still get the same output.

Using the maven command: 

$mvn spring-boot:run -Dspring-boot.run.arguments="--spring.profiles.active=production"

JVM command:

$java -jar target/app.jar --spring.profiles.active=production

It is also possible to pass multiple arguments at the same time. Using the example above we will pass one more property, server port as shown below.

Maven command (space separated): 

$mvn spring-boot:run -Dspring-boot.run.arguments="--spring.profiles.active=production --server.port=8089"

Using JVM command:

$java -jar target/app.jar --spring.profiles.active=production  --server.port=8089

Environment Variables

Given that we are not able to alter our property values through the command line then we can take advantage of the environment variables. Spring application can read from the environment variables and at start-up, Spring application looks for an environment variable named SPRING_APPLICATION_JSON which can carry a set of properties using inline JSON. We can experiment by overriding the connection addresses defined in our properties file by following the steps below.

Let us open our terminal and run the following command. The command sets the environment variables of our application by overriding the connection settings.

$export SPRING_APPLICATION_JSON='{"connection":{"addresses":{"test":"http://localhost/payments/pre-prod1","live":"http://192.168.123.23/payments/prod1"}}}'

Then let us run our application

$java -jar -Dspring.profiles.active=development target/app.jar

Output:

When we check our log, we will notice that the connection addresses in the development profile were overridden and the values in the JSON file that we passed through the environment variable took precedence.

Property Injection

There are various ways we can use to inject property values into our application from the property sources. We can use @Value annotation, the Spring’s Environment abstraction or we can bind these values to a structured object annotated with @ConfigurationProperties.

@Value

We can use this method if we have a few properties but it is not recommended if we have many properties. Let us imagine if the merchant account had more than twenty properties, then it means we were going to specify@Value annotation twenty times. The code snippet below shows how we can use this annotation to inject a property value into our application.

@Value(“${propertyName}”)

It is important to make sure that the property name of the @Value matches with the one specified in the property sources.

@ConfigurationProperties

If we have a number of properties, we can group them together and map them to a POJO class. This will provides us with a structured and type-safe object that we can inject anywhere in our application. So instead of using the @Value annotation, the property values can be retrieved using the getter methods of this POJO.

@Data
@Component
@ConfigurationProperties(prefix = "merchantaccount")
public class MerchantAccount {
  private String name;
  private String username;
  private String code;
  private int number;
  private String currency;
}

The POJO class must be annotated with @ConfigurationProperties and @Component as indicated above. The prefix value specified in the annotation must be the same as the prefix of property defined inside the application.yml file as indicated above.

application.yml

merchantaccount:
 name: Maureen Sindiso Mpofu
 username: momoe
 code: MCA1230
 number: 771222279
 currency: ZWL

It is important to note that we can also use the @ConfigurationProperties annotation for mapping lists and maps as indicated below:

@ConfigurationProperties(prefix = "connection")
@Component
@Data
public class ConnectionSettings {
   List<String> names;
   Map<String, String> addresses;
}

Order of Precedence of Configuration Data

It is possible to have multiple property sources in a Spring Boot application. Therefore it is important to be aware of the property source that will take precedence over others. For instance, if we have provided configuration for our application using an application.yml file and during application execution we decide to pass command line arguments then the property values in the application.yml file will be overridden by the ones specified in the command line arguments.

The order of property sources provided below is used by Spring Boot 2.2.x.  A property source higher up in the list takes precedence over the ones below it.

1. Global settings properties in the $HOME/.config/spring-boot folder when devtools is active
2. @TestPropertySource annotations on your tests.
3. properties attribute on your tests. Available on @SpringBootTest and the test annotations for testing a particular slice of your application.
4. Command-line arguments
5. Properties from SPRING_APPLICATION_JSON (inline JSON embedded in an environment variable or system property)
6. ServletConfig init parameters
7. ServletContext init parameters
8. JNDI attributes from java:comp/env.
9. Java System properties i.e System.getProperties()
10. OS environment variables.
11. A RandomValuePropertySource that has properties only in random.*.
12. Profile-specific application properties outside of your packaged jar (application- {profile}.properties and YAML variants)
13. Profile-specific application properties packaged inside your jar (application- {profile}.properties and YAML variants)
14. Application properties outside of your packaged jar (application.properties and YAML variants)
15. Application properties packaged inside your jar (application.properties and YAML variants)
16. @PropertySource annotations on your @Configuration classes. Please note that such property sources are not added to the Environment until the application context is being refreshed. This is too late to configure certain properties such as logging.* and spring.main.* which are read before refresh begins
17. Default properties (specified by setting SpringApplication.setDefaultProperties) 

Conclusion

 It recommended to externalize our configuration data and if there are many configuration properties, then we can group them into a simple Java class and use the @ConfigurationProperties annotation to structure our configuration and make it type-safe. However, the biggest challenge with externalizing configuration is on the part of ensuring that the deployed application runs with the correct configuration. Therefore it is important to be careful when setting up an application that uses different property sources for different environments. The sample code for this article is found here on GitHub.