///////////////////////////////////////////////////////////////////////////////

    Copyright (c) 2018, 2019 Oracle and/or its affiliates. All rights reserved.

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.

///////////////////////////////////////////////////////////////////////////////

:javadoc-base-url-api: {javadoc-base-url}?io/helidon/config

= Additional Supported Formats and Sources
:description: Helidon config supported formats and sources
:keywords: helidon, config
:toc: preamble
:toclevels: 2

Helidon Config provides several extension modules that support other configuration
 formats (parsers) and sources. This section describes how to add these modules
to your build and how to use them from your application.

== Introduction
This document describes the additional config formats and sources the Helidon
config system supports and how to include them and use them in your project. In each
case you need to add module dependencies to your project and, in some cases,
write your application accordingly.

== Additional Config Formats and Parsers

=== Automatic Media Type and File Type Handling
With each of the parsers described here, your application can either

1. explicitly add a parser of the correct implementation to the `Config.Builder`, or
2. rely on Java service loading and the config system's matching of file types and
media types to parsers.

If your application creates a `Config.Builder` with parser services _disabled_ 
(see link:{javadoc-base-url-api}/Config.Builder.html#disableParserServices--[`disableParserServices`] 
then that builder will not find the Java services for the various parsers and so
will be unable to match the file type or media type of sources with the corresponding
parser automatically. So if you want to use automatic type 
matching with a given builder, do not invoke `Config.Builder.disableParserServices()`.

=== YAML [[Config-ModuleYaml]]

==== Maven Coordinates

Add the following dependency in your project:

[source,xml]
.Config YAML Dependency in `pom.xml`
----
<dependencies>
    <dependency>
        <groupId>io.helidon.config</groupId>
        <artifactId>helidon-config-yaml</artifactId>
    </dependency>
</dependencies>
----

[source,java]
.Config YAML Dependency in `module-info.java`
----
module myModule {
    requires io.helidon.config.yaml;
}
----

==== Using the YAML Parser
The YAML parser handles the following media type:

- `application/x-yaml` - YAML format (file type `.yaml`)

[source,java]
.Automatic selection
----
Config config = Config.create(classpath("application.yaml")); // <1>
----

<1> The config system automatically maps the file type `.yaml` to the media type `application/x-yaml`
which the Helidon YAML parser matches.

[source,java]
.YAML parser specified - no file type on source
----
Config config = Config.create(classpath("my-config")                                    // <1>
                                    .parser(YamlConfigParserBuilder.buildDefault())); // <2>
----

<1> The media type of the source `my-config` is unknown, so the config system
cannot choose a parser automatically.
<2> The config system will parse the resource `my-config` on the runtime classpath 
using the YAML parser instance created by the
link:{javadoc-base-url-api}/yaml/YamlConfigParserBuilder.html[`YamlConfigParserBuilder`].
The `buildDefault()` method creates a config parser with default behavior.

[source,java]
.Media type specified
----
Config config = Config.create(classpath("my-config")                     // <1>
                                    .mediaType("application/x-yaml")); // <2>
----

<1> The media type of the source `my-config` is unknown, so the config system
cannot choose a parser automatically.
<2> Specifying the media type for the config source allows the config system to
use its matching algorithm with the available parsers to choose a parser for that type.

[source,java]
.YAML parser specified because parser services disabled 
----
Config config = Config.builder(classpath("application.yaml"))
        .disableParserServices()                                  // <1>
        .addParser(YamlConfigParserBuilder.buildDefault())        // <2>
        .build();
----

<1> Disables automatic parser lookup and registration.
<2> Explicit registration of the YAML parser is therefore required.

=== HOCON/JSON [[Config-ModuleHocon]]
The Helidon HOCON config module handles sources in the 
HOCON and JSON formats.

==== Maven Coordinates
Add the following dependency in your project:

[source,xml]
.Config HOCON Dependency in `pom.xml`
----
<dependencies>
    <dependency>
        <groupId>io.helidon.config</groupId>
        <artifactId>helidon-config-hocon</artifactId>
    </dependency>
</dependencies>
----

[source,java]
.Config HOCON Dependency in `module-info.java`
----
module myModule {
    requires io.helidon.config.hocon;
}
----

==== Using the HOCON/JSON Parser

The parser handles the following media types:

- `application/hocon` - HOCON format (file type `.conf`)
- `application/json` - JSON format (file type `.json`)

[source,java]
.Automatic selection
----
Config config = Config.create(classpath("application.conf")); // <1>
----

<1> The config system automatically maps the file type `.conf` to the media type `application/hocon
which the Helidon HOCON parser matches. 

The same module and parser supports file type `.json` and the media type 
 `application/json`.

[source,java]
.HOCON parser specified - no file type on source
----
Config config = Config.create(classpath("my-config")                                     // <1>
                                    .parser(HoconConfigParserBuilder.buildDefault())); // <2>
----

<1> the media type of the source `my-config`is unknown, so the config system cannot 
choose a parser automatically.
<2> The config system will parse the resource `my-config` using the HOCON parser created
by the link:{javadoc-base-url-api}/hocon/HoconConfigParserBuilder.html[HoconConfigParserBuilder].
The `buildDefault()` method creates a config parser with default behavior.

[source,java]
.Media type specified
----
Config config = Config.create(classpath("my-config")                    // <1>
                                    .mediaType("application/hocon")); // <2>
----

<1> The media type of the source `my-config` is unknown, so the config system
cannot choose a parser automatically.
<2> Specifying the media type for the config source allows the config system to 
use its matching algorithm with the available parsers to choose a parser for that
type.

[source,java]
.HOCON parser specified because parser services disabled
----
Config config = Config.builder(classpath("application.conf"))
        .disableParserServices()                                  // <1>
        .addParser(HoconConfigParserBuilder.buildDefault())       // <2>
        .build();
----

<1> Disables automatic parser lookup and registration.
<2> Explicit registration of the HOCON parser is therefore required.

[source,java]
.Customized HOCON parser
----
Config config = Config.builder(classpath("application.conf"))
        .disableParserServices()
        .addParser(HoconConfigParserBuilder.create()              // <1>
                           .disableResolving()                    // <2>
                           .build())                              // <3>
        .build();
----

<1> Creates new instance of the parser builder.
<2> Disables resolution of substitutions.
(See the link:https://github.com/lightbend/config/blob/master/HOCON.md#substitutions[HOCON documentation].)
<3> Builds a new instance of the HOCON config parser.

You can also specify
link:https://github.com/lightbend/config/blob/master/config/src/main/java/com/typesafe/config/ConfigResolveOptions.java[`ConfigResolveOptions`]
using the `HoconConfigParserBuilder.resolveOptions` method.

== Additional Config Source Types

=== Etcd [[Config-ModuleEtcd]]

The Helidon Etcd config module supports reading configuration from a specified
 Etcd key.

==== Maven Coordinates
Add the following dependency to your project:
[source,xml]
.Config Etcd Dependency in `pom.xml`
----
<dependencies>
    <dependency>
        <groupId>io.helidon.config</groupId>
        <artifactId>helidon-config-etcd</artifactId>
    </dependency>
</dependencies>
----

[source,java]
.Config Etcd Dependency in `module-info.java`
----
module myModule {
    requires io.helidon.config.etcd;
}
----

==== Using the Etcd Config Source
To read configuration from an Etcd source, your application uses the
link:{javadoc-base-url-api}/etcd/EtcdConfigSourceBuilder.html[`EtcdConfigSourceBuilder`].

[source,java]
.Use Etcd config source
----
Config config = Config.create(
        EtcdConfigSourceBuilder                             // <1>
                .create(URI.create("http://my-etcd:2379"),  // <2>
                      "/config.yaml",                       // <3>
                      EtcdConfigSourceBuilder.EtcdApi.v3)); // <4>
----

<1> Use the factory method `EtcdConfigSourceBuilder.create` to initialize the builder.
<2> Specify the Etcd endpoint address.
<3> Specify the Etcd key of the configuration document.
<4> Version of the Etcd API to use; `v2` and `v3` are supported.

The config system will use the <<Config-ModuleYaml,YAML parser>> automatically in this example
because the file type of the key is `.yaml`.

The `EtcdConfigSourceBuilder` class extends 
link:{javadoc-base-url-api}/spi/AbstractParsableConfigSource.Builder.html[`AbstractParsableConfigSource.Builder`]
and so supports the usual settings on config sources.

==== Monitoring for Source Changes
The Etcd support includes a polling strategy designed for an etcd config source.

[source,java]
.Use Etcd config source
----
Config config = Config.create(
        EtcdConfigSourceBuilder
                .create(URI.create("http://my-etcd:2379"), "/config.yaml", EtcdApi.v3)
                .pollingStrategy(EtcdWatchPollingStrategy::new));                    // <1>
----

<1> Use the etcd-specific polling strategy.


==== Loading Meta-configuration via Etcd
The config system can load information about config sources from 
<<config/06_advanced-configuration.adoc#Config-Advanced-Config-MetaConfig,meta-configuration>>
rather than requiring your application to construct the builder. To read
meta-configuration from an Etcd source set the following required properties
for the source:

* `type` to `etcd`, or `class` to `io.helidon.config.etcd.EtcdConfigSourceBuilder`
* `uri` (type `URI`) - Etcd endpoint URI.
* `key` (type `String`) - Etcd key that is associated with the configuration.
* `api` (type `EtcdConfigSourceBuilder.EtcdApi`, i.e. `v2` or `v3`) - Etcd API
 version.

Other optional `properties` are inherited from
 `AbstractParsableConfigSource.Builder`. (see
link:{javadoc-base-url-api}/spi/AbstractParsableConfigSource.Builder.html#init-io.helidon.config.Config-[javadoc])

[source,java]
.Load Config from meta-configuration
----
Config config = Config.loadSourcesFrom(classpath("config-meta-etcd.yaml"));
----

[source,YAML]
.Meta-config `config-meta-etcd.yaml` for the etcd source
----
sources:
    - type: "etcd"                                                 # <1>
      properties:
          uri: "http://my-etcd:2379"                               # <2>
          key: "/config.yaml"                                      # <2>
          api: "v3"                                                # <2>
          polling-strategy:
              class: "io.helidon.config.etcd.EtcdWatchPollingStrategy" # <3>

----

<1> `etcd` config source type
<2> Etcd source-specific (mandatory) `properties`: `uri`, `key` and `api`.
<3> Polling strategy `EtcdWatchPollingStrategy` is automatically initialized by
 specified mandatory `properties`.

=== git [[Config-ModuleGit]]
The Helidon git config module supports reading configuration from a git
 repository.

==== Maven Coordinates
Add the following dependency to your project:
[source,xml]
.Config git Dependency in `pom.xml`
----
<dependencies>
    <dependency>
        <groupId>io.helidon.config</groupId>
        <artifactId>helidon-config-git</artifactId>
    </dependency>
</dependencies>
----

[source,java]
.Config git Dependency in `module-info.java`
----
module myModule {
    requires io.helidon.config.git;
}
----

==== Using the git Config Source
To read configuration from a git source, your application uses the 
link:{javadoc-base-url-api}/git/GitConfigSourceBuilder.html[`GitConfigSourceBuilder`].

[source,java]
.Use git config source
----
Config config = Config.create(
        GitConfigSourceBuilder
                .create("application.conf")                                           // <1>
                    .uri(URI.create("https://github.com/okosatka/test-config.git")) // <2>
                    .directory(Paths.get("/config"))                                // <3>
                    .branch("dev"));                                                // <4>
----

<1> Use the factory method `GitConfigSourceBuilder.create` to initialize the builder
 with a mandatory  path to the configuration file.
<2> Specify the git repository URI.
<3> Specify a directory where the git repository is already cloned or it will be cloned.
<4> Specify the git branch.

Note that the config system will use the <<Config-ModuleHocon,HOCON parser>> in 
this example because the file type is `.conf`. Recall that for this to work the 
HOCON config module must be on module-path or classpath.

The `GitConfigSourceBuilder` supports the usual source builder properties because
 it extends
link:{javadoc-base-url-api}/spi/AbstractParsableConfigSource.Builder.html[`AbstractParsableConfigSource.Builder`].

==== Monitoring for Source Changes
Your application can monitor changes to a configuration loaded from a git source
associating the `regular` built-in polling strategy with the source.
[source,java]
.Use of git config source with polling strategy
----
Config config = Config.create(
        GitConfigSourceBuilder
                .create("application.conf")
                .uri(URI.create("https://github.com/okosatka/test-config.git"))
                .pollingStrategy(PollingStrategies.regular(Duration.ofMinutes(5)))); // <1>
----
<1> Use `PollingStrategies.regular(Duration duration)` to monitor for config changes.

You can also implemention your own polling strategy by implementing
link:{javadoc-base-url-api}/spi/PollingStrategy.html[`PollingStrategy`]. See
the <<config/05_mutability-support.adoc,mutability support>> and
<<config/07_extensions.adoc#Config-SPI-PollingStrategy,polling strategy>> discussions.

==== Loading Meta-configuration via git

The config system can load information about config sources from meta-configuration 
rather than requiring your application to construct the builder. To read 
meta-configuration from a git config source set the following properties for the source:

* `type` to `git` or `class` to `io.helidon.config.git.GitConfigSourceBuilder` 
* `path` (type `String`) - Relative path to the configuration file in repository.
* `uri` (type `URI`) - URI to the git repository.
* `directory` (type `Path`) - Directory with a cloned repository, by default 
 a temporary directory.
* `branch` (type `String`) - git branch (default is `master`).

The meta-configuration must set the `path` and one of `uri` or `directory`. 
Other optional `properties` are inherited from
 `AbstractParsableConfigSource.Builder` (see
 link:{javadoc-base-url-api}/spi/AbstractParsableConfigSource.Builder.html#init-io.helidon.config.Config-[javadoc])

[source,java]
.Load Config from meta-configuration
----
Config config = Config.loadSourcesFrom(classpath("config-meta-git.yaml"));
----

[source,YAML]
.Meta-config `config-meta-git.yaml` for the git source
----
sources:
    - type: "git"                                            # <1>
      properties:
          path: "application.conf"                           # <2>
          uri: "https://github.com/okosatka/test-config.git" # <2>
          directory: "/config"                               # <2>
          branch: "dev"                                      # <2>
          polling-strategy:
              type: "regular"                                # <3>
              properties:
                  interval: "PT5M"                           # <3>

----

<1> `git` config source type
<2> git source-specific properties: `path`, `uri`, `directory` and `branch`.
<3> Polling strategy `regular` with an interval, in
 `Duration` format, of 5 minutes in this example.
