///////////////////////////////////////////////////////////////////////////////

    Copyright (c) 2018, 2019 Oracle and/or its affiliates. All rights reserved.

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.

///////////////////////////////////////////////////////////////////////////////

:javadoc-base-url-api: {javadoc-base-url}?io/helidon/config

= Extensions
:description: Helidon config extensions
:keywords: helidon, config

Developer-provided extensions influence how the config system behaves.

The <<config/01_introduction.adoc,config system introduction>> explains the design of the config
system and how its parts work together to read and parse config data, convert it
to Java types, fine-tune the look-up of config data, and reload and
reprocess data when it changes. _Config extensions_ provided by the application
modify and expand the way the config system performs these steps. 

== Introduction
Each config extension implements one of the interfaces defined in the Configuration SPI:

// TODO 
//Once our asciidoc processing handles labeled lists, uncomment the following
//and use it instead of the bulleted list which follows:
//`ConfigSource`:: Loads raw configuration data from a given type of source and
//delegates to a `ConfigParser`, producing the in-memory data structure which
//represents the loaded and parsed configuration.
//`ConfigParser`:: Translates configuration content in a given format into the
//corresponding internal config data structures.
//`OverrideSource`:: Provides key/value pairs which override config values loaded
//from any `ConfigSource`, given the key but _ignoring_ the original value.
//`ConfigFilter`:: Transforms config `String` values returned from any value-type
//`Config` node, given the key _and_ the original value.
//`ConfigMapperProvider`:: Provides one or more ``ConfigMapper``s each of which
//converts a `Config` object tree to a Java type specific to the application.
//`PollingStrategy`:: Implements a custom technique for notifying the Config system
//when the data underlying a `ConfigSource` or `OverrideSource` has changed.
* `ConfigSource` - Loads raw configuration data from a given type of source and
delegates to a `ConfigParser`, producing the in-memory data structure which
represents the loaded and parsed configuration.
* `ConfigParser` - Translates configuration content in a given format into the
corresponding internal config data structures.
* `OverrideSource` - Provides key/value pairs which override config values loaded
from any `ConfigSource`, given the key and _ignoring_ the original value.
* `ConfigFilter` - Transforms config `String` values returned from any value-type
`Config` node, given the key _and_ the original value.
* `ConfigMapperProvider` - Provides one or more ``ConfigMapper``s each of which
converts a `Config` object tree to a Java type specific to the application.
* `PollingStrategy` - Implements a custom technique for notifying the Config system
when the data underlying a `ConfigSource` or `OverrideSource` has changed.

The config system itself implements several of these SPIs, as noted in the sections
below.

=== About Priority [[priority-info]]
The config system invokes extensions of a given type in priority order. 
Developers can express the relative 
importance of an extension by annotating the implementation class with
`@javax.annotation.Priority`. The default value is 100. 
A _lower_ priority value represents _greater_ importance. The sections below for
each interface tell which SPIs support `@Priority`.

== ConfigSource SPI [[Config-SPI-ConfigSource]]

The config system includes built-in support for several types of sources
(for example, Java `String`, `Readable`, `Properties`, and `Map`
objects - see link:{javadoc-base-url-api}/ConfigSources.html[`ConfigSources`]).
Implement a link:{javadoc-base-url-api}/spi/ConfigSource.html[`ConfigSource`] to
load raw configuration data from a type of source that the config system does
not already support.

[plantuml, config/spi-ConfigSource, png, title="ConfigSource SPI", align="center"]
----
together {
    interface ConfigSource {
        + void init(ConfigContext context)
        + Optional<ObjectNode> load()
        ..
        + String description()
        ..
        + Flow.Publisher<Optional<ObjectNode>> changes()
    }

    interface ConfigContext {
        Optional<ConfigParser> findParser(String mediaType)
    }
}

ConfigSource -[hidden]> ConfigContext

together {
    interface List<E>
    interface Map<K,V>
    interface ConfigNode

    Map -[hidden]> ConfigNode
    ConfigNode -[hidden]> List
}

ConfigSource -[hidden]--> Map

together {
    interface ConfigNode.ValueNode extends ConfigNode
    interface ConfigNode.ListNode extends ConfigNode, List
    interface ConfigNode.ObjectNode extends ConfigNode, Map

    interface ConfigNode.ValueNode
    note bottom: extends Supplier<String>

    interface ConfigNode.ListNode
    note bottom: extends List<ConfigNode>

    interface ConfigNode.ObjectNode
    note bottom: extends Map<String, ConfigNode>
}
----

The interfaces `ConfigNode`, `ObjectNode`, `ValueNode` and
 `ListNode` represent the in-memory data structure for loaded and parsed configuration data.

//TODO example [P2] Issue #29


== ConfigParser SPI [[Config-SPI-ConfigParser]]

The parsing step converts config data in some format into the corresponding
in-memory representation of config ``ObjectNode``s. The config system can already parse 
several data formats (for example 
Java `Properties`, YAML, and HOCON). Implement the 
link:{javadoc-base-url-api}/spi/ConfigParser.html[`ConfigParser`] SPI to allow the
config system to handle additional formats.

[plantuml, config/spi-ConfigParser, png, title="ConfigParser SPI", align="center"]
----
interface ConfigParser {
    + Set<String> getSupportedMediaTypes()
    + ObjectNode parse(Content content)
}

interface ConfigParser.Content {
    + String getMediaType()
    + Instant getTimestamp()
    ..
    + Readable asReadable()
    + void close()
}

ConfigParser -[hidden]> ConfigParser.Content
----

The `ConfigParser.Content` interface defines operations on the content that is to
to be parsed by a `ConfigParser` implementation:

// TODO
// When our asciidoc impl supports labeled lists replace what follows with the
// commmented doc below.
* `getStamp()` - Returns a stamp (for example, a time stamp) that is different for
different values of the content.
* `getMediaType()` - Reports the media type of the content.
* `asReadable()` - Provides a `Readable` and `Autocloseable` object from which
the content can be read.

//`getStamp()`:: Returns a stamp (for example, a time stamp) that is different for
//different values of the content.
//`getMediaType()`:: Reports the media type of the content.
//`asReadable()`:: Provides a `Readable` and `Autocloseable` object from which
//the content can be read.

The application can register parsers for a builder by invoking `Config.Builder#addParser(ConfigParser)`.
The config system also uses the Java service loader mechanism to load automatically, 
for all builders, any parsers listed in the 
`META-INF/services/io.helidon.config.spi.ConfigParser` resource on the
runtime classpath. Prevent autoloading of parsers for a given builder
by invoking `Config.Builder#disableParserServices()`.

`ConfigParser` accepts `@Priority`. See <<priority-info, About Priority>>.

[source,java]
.Example custom parser implementation listed in `META-INF/services/io.helidon.config.spi.ConfigParser`
----
myModule.MyConfigParser
----

[source,java]
.Example custom parser definition in `module-info.java`
----
module myModule {
    requires transitive io.helidon.config;
    provides io.helidon.config.spi.ConfigParser with myModule.MyConfigParser;
}
----

//TODO example [P2] Issue #29


== OverrideSource SPI [[Config-SPI-OverrideSource]]

When the application retrieves a configuration value the config system first uses
the relevant config sources and filters. It then applies any _overrides_ the
application has provided. Each override has:

* a `Predicate<Config.Key>` (a boolean-valued function that operates on 
the config key), and 
* a replacement, _overriding_, `String` value the config system should use if the predicate
evaluates to true.

To furnish overrides to the config system, implement the 
link:{javadoc-base-url-api}/spi/OverrideSource.html[`OverrideSource`] SPI one or 
more times and pass instances of those implementations to the config builder's
link:{javadoc-base-url-api}/Config.Builder.html#overrides-java.util.function.Supplier-[`overrides`]
method. The config system will apply the overrides returned from each 
`OverrideSource` to each config key requested from a `Config` that is based on 
that `Config.Builder`.

[plantuml, config/spi-OverrideSource, png, title="OverrideSource SPI", align="center"]
----
interface OverrideSource {
    + Optional<OverrideSource.OverrideData> load() throws ConfigException
}
----

//TODO example [P2] Issue #29


== ConfigFilter SPI [[Config-SPI-ConfigFilter]]

Before returning a `String` from `Config.value()` the config system applies any 
_filters_ set up on the `Config.Builder` used to create the config tree that
contains the config node of interest. The application provides filters as 
implementations of the 
link:{javadoc-base-url-api}/spi/ConfigFilter.html[`ConfigFilter`] interface.
Each filter is a function which accepts a `Config.Key` and an input `String` value
and returns a `String` value the config system should use for that key going forward.
The filter can return the original value or return some other value. 

The application registers filters and filter providers by passing `ConfigFilter` 
implementations to one of the config builder
link:{javadoc-base-url-api}/Config.Builder.html[`addFilter` methods]. The config 
system also uses the Java service loader mechanism to load 
additional filters automatically, for all builders, using 
the service interface described in the following table. Prevent a given 
builder from using the auto-loaded filters by invoking the
link:{javadoc-base-url-api}/Config.Builder.html#disableFilterServices--[`disableFilterServices`]
method.

.Config SPI Interfaces for Filtering
|===
|Interface |Method |Usage

|link:{javadoc-base-url-api}/spi/ConfigFilter.html[`ConfigFilter`]

Accepts `@Priority`. See <<priority-info, About Priority>>.
|`String apply(Config.Key key, String stringValue);`
|Accepts a key and the corresponding `String` value and 
returns the `String` which the config system should use for that key.
|===

=== Initializing Filters
The `ConfigFilter` JavaDoc describes multiple methods for adding filters to a
`Config.Builder`. Some accept a `ConfigFilter` directly and some accept a provider 
function which, when passed a `Config` instance, returns a `ConfigFilter`.

*_Neither a `ConfigFilter` nor a provider function which furnishes one should 
access the `Config` instance passed to the provider function._*

Instead, implement the `ConfigFilter.init(Config)` method on the filter. The config
system invokes the filters' `init` methods according to the filters' `@Priority` 
order. 

Recall that whenever any code invokes `Config.get`, the `Config` instance
invokes the `apply` method of _all_ registered filters. By the time the application
retrieves config this way the config system will have run the `init` method on all
the filters. _But note that when a filter's `init` method invokes `Config.get`, the 
`init` methods of lower-priority filters will not yet have run._

[plantuml, config/spi-ConfigFilter, png, title="ConfigFilter SPI", align="center"]
----
interface ConfigFilter {
    + String apply(Config.Key key, String stringValue)
}
----

//TODO example [P2] Issue #29


== ConfigMapperProvider SPI [[Config-SPI-ConfigMapperProvider]]

The config system provides built-in mappings from `String` values to various Java 
types. (See link:{javadoc-base-url-api}/ConfigMappers.html[`ConfigMappers`].) 

To handle mappings to other types the application can register
custom mappers with the config system by implementing the 
link:{javadoc-base-url-api}/spi/ConfigMapperProvider.html[`ConfigMapperProvider`]
 SPI.

Such providers return a map, with entries in which:

* the key is the Java type (a `Class` object) the mapper produces, and
* the value is a `ConfigMapper` that converts the config in-memory
data structure into the type in the key.

The provider may also implement other methods for finer tuned conversion mechanisms:

* `genericTypeMappers()` returns a map with entries for specific `GenericType` conversions,
    for example when the provider supports only mapping for `GenericType<Map<String, Integer>>`
* `mapper(Class)` returns a conversion function (optional) that converts a config node
    to the typed instance (if supported by this provider)
* `mapper(GenericType)` returns a conversion function (optional) that coverts a config node
    to the GenericType (if supported by this provider) - for example in case this provider supports
    any Map<String, ?> type, such as `Map<String, Integer>` and `Map<String, Double>`

The config conversion system works as follows:

For `Config.as(Class)`:

1. Check whether a conversion function exists for the class requested (from method `mappers()`).
2. Check whether a conversion function is provided by any `ConfigMapperProvider` with method `mapper(Class)`.
3. Check whether a conversion function exists for a generic type for the class requested (from method `genericTypeMappers`).
4. Check whether a conversion function is provided by any `ConfigMapperProvider` with method `mapper(GenericType)` for
    a generic type for the class requested.

For `Config.as(GenericType)` - the first two steps are skipped.

The config system also uses the Java `ServiceLoader` mechanism to load automatically,
for all builders, any mappers returned by the providers listed in the 
`META-INF/services/io.helidon.config.spi.ConfigMapperProvider` resource on the
runtime classpath. The application can prevent autoloading of mappers for a
given builder by invoking `Config.Builder#disableMapperServices()`. Note
that the built-in mappers described in `ConfigMappers` still operate.

Mapper providers accept `@Priority`. See <<priority-info, About Priority>>. 

[plantuml, config/spi-ConfigMapperProvider, png, title="ConfigMapperProvider SPI", align="center"]
----
interface ConfigMapperProvider {
    + Map<Class<?>, Function<Config, ?>> getMappers()
    + Map<GenericType<?>, BiFunction<Config, ConfigMapper, ?>> genericTypeMappers()
    + <T> Optional<Function<Config, T>> mapper(Class<T> type)
    + <T> Optional<BiFunction<Config, ConfigMapper, T>> mapper(GenericType<T> type)
}
----


A mapper provider can specify a `@javax.annotation.Priority`.
If no priority is explicitly assigned, the value of `100` is assumed.

[source,java]
.Reference custom mapper provider implementation in `META-INF/services/io.helidon.config.spi.ConfigMapperProvider`
----
myModule.MyConfigMapperProvider
----

[source,java]
.Reference custom mapper provider implementation in `module-info.java`
----
module myModule {
    requires transitive io.helidon.config;
    provides io.helidon.config.spi.ConfigMapperProvider with myModule.MyConfigMapperProvider;
}
----

//TODO example [P2] Issue #29


== PollingStrategy SPI [[Config-SPI-PollingStrategy]]

Once it loads a `Config` tree from ``ConfigSource``s the config
system does not itself change the in-memory `Config` tree. Even so, the
underlying data available via the tree's ``ConfigSource``s can change.
Implementations of link:{javadoc-base-url-api}/spi/PollingStrategy.html[`PollingStrategy`] 
informs other interested code when changes to that underlying data might have 
occurred.

In implementations of `PollingStrategy` the `#ticks()` method
returns a `Flow.Publisher` of ``PollingEvent``s to which the
application or the ``ConfigSource``s themselves can subscribe. Generally,
each event is a hint to the subscriber that
it should check to see if any of the underlying config data it relies on
has changed. Note that a ``PollingStrategy``'s publication of an
event does not necessarily guarantee that the underlying data has in fact
changed, although this might be true for some `PollingStrategy`
implementations.

The config system offers polling strategies for periodic time-based
checks and for a file watcher. Often an application can create a config source
simply by using one of the methods on `ConfigSources` (for example,
`ConfigSources#file(path)` to get a builder and then invoke `pollingStrategy` 
passing one of the predefined strategies. But the application can implement
its own `PollingStrategy` and set it on the config source builder instead.

[plantuml, config/spi-PollingStrategy, png, title="PollingStrategy SPI", align="center"]
----
interface PollingStrategy {
    + Flow.Publisher<PollingEvent> ticks()
}

interface PollingStrategy.PollingEvent {
    + Instant getTimestamp()
}

PollingStrategy -[hidden]> PollingStrategy.PollingEvent
----

`The PollingStrategy` `ticks()` method returns a `Publisher` of ``PollingEvent``s. 
Each event becomes available as the particular `PollingStrategy` publishes it.
Depending on the implementation of the polling strategy, such events might 
indicate that the underlying source data _has_ changed or that it _might have_ changed.
In either case the subscribers to the publisher are notified. If the `ConfigSource`
itself subscribes to the publisher, for example, then it might choose to reload 
the underlying data when its subscriber receives an event.

//TODO example [P2] Issue #29

== RetryPolicy SPI [[Config-SPI-RetryPolicy]]

The builder for each `ConfigSource` and `OverrideSource` accepts a 
link:{javadoc-base-url-api}/spi/RetryPolicy.html[`RetryPolicy`] 
governing if and how the source should deal with failures loading the underlying 
data. 

A retry policy accepts a function, the invocation of which the policy will
govern according to its own implementation. 
Applications can use the predefined policies in 
link:{javadoc-base-url-api}/RetryPolicies.html[`RetryPolicies`], such as 
`RetryPolicies.justCall` which simply invokes the function without any retry.
That class also exposes a builder for constructing a time-based retry policy, 
with several parameters:

.Parameters Controlling Built-in `RetryPolicy`
|===
|Parameter |Usage |Default

|`delay` |Initial delay between calls to the function | 200 ms
|`delayFactor` |Multiplier applied to `delay` on each successive call | 2
|`callTimeout` |Time limit for each individual call of the function | 500 ms
|`overallTimeout` |Limit for the total elapsed time attempting to 
call the function successfully, including delays between calls | 2 s
|===

The actual delay between function call starts as `delay` and changes by the factor
`delayFactor` on each successive attempt.

Note that the job of each retry policy is to call the provided function 
successfully. As such, the policy must perform the first attempt as well
as any retries.

[plantuml, config/spi-RetryPolicy, png, title="RetryPolicy SPI", align="center"]
----
interface RetryPolicy {
    + <T> T execute(Supplier<T> call)
    + boolean cancel(boolean mayInterruptIfRunning)
}
----

The application can try to cancel the overall execution of a `RetryPolicy` by invoking
the `RetryPolicy#cancel(boolean mayInterruptIfRunning)` method. Ideally the retry policy
implementation should be able to abort the execution of the retry policy, even while
a function call is in progress, but the policy must respond to cancels between
function calls. In either case `cancel` returns `true` if the retry was aborted
without a successful call to the function, and `false` otherwise, including if 
the function call had already completed successfully or had previously been 
successfully canceled.

//TODO example [P2] Issue #29

