Skip to content

Conversation

@anuraaga
Copy link
Contributor

This is an alternative to #3743 and #3714. I like it better

I realized that many uses don't need to use SPI to configure autoconfiguration - they're calling OpenTelemetrySdkAutoConfiguration.inintialize() and could instead just configure a builder which is simpler, and should be more flexible than an interface that only presents configuration through default no-op methods. We can remove some of the initialize overloads too.

The SPI that accepts the builder can still be exposed for downstream of downstream - for example, a javaagent end user could still implement an extension module with an implementation of the SPI if they need to themselves configure auto-configuration.

@anuraaga anuraaga requested a review from a user October 15, 2021 08:30
@anuraaga anuraaga requested a review from Oberon00 as a code owner October 15, 2021 08:30
@anuraaga anuraaga marked this pull request as draft October 15, 2021 08:32
@anuraaga anuraaga force-pushed the autoconfiguration-builder branch from 007c21e to 8078d88 Compare October 15, 2021 08:35
Copy link
Member

@mateuszrzeszutek mateuszrzeszutek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would the usage in the agent look like? Would we still have to maintain a custom ConfigProperties implementation?
Or would it look something like that:

OpenTelemetrySdkAutoConfiguration.builder()
    .addPropertySource(config::getAllProperties)
    .setResultAsGlobal(true)
    .newOpenTelemetrySdk()

?

public OpenTelemetrySdkAutoConfigurationBuilder addPropertySource(
Supplier<Map<String, String>> propertiesSupplier) {
requireNonNull(propertiesSupplier, "propertiesSupplier");
this.propertiesSupplier = mergeProperties(this.propertiesSupplier, propertiesSupplier);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

propertiesSupplier does not seem to be used after it's set in this method

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I didn't wire this in yet for the draft, despite it being what you're most interested in :) But yes it would apply to default properties same as the other PR

Copy link
Contributor Author

@anuraaga anuraaga Oct 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way for the javaagent, it should be something like addPropertySource(new FileProperties(filePath)) and there shouldn't be any custom implementation of ConfigProperties. Does it sound like it'll work?

@jkwatson
Copy link
Contributor

I think this is a good idea. It allows a single SPI to configure pretty much anything that would need to be configured, without exploding the amount of SPIs. Do you think we could get rid of some of the existing ones with this approach?

@jack-berg
Copy link
Member

It'd be cool if: 1. We had this customizer SPI. 2. our exporters adopted a convention of having a toBuilder() method so that you could do something like this to easily enhance the configuration done by autoconfiguration:

autoConfiguration.addSpanExporterCustomizer(spanExporter -> {
            if (spanExporter instanceof OtlpGrpcSpanExporter) {
              OtlpGrpcSpanExporter otlpGrpcSpanExporter = (OtlpGrpcSpanExporter) spanExporter;
              return otlpGrpcSpanExporter.toBuilder()
                  .addHeader("foo", "bar")
                  .build();
            }
            return spanExporter;
          });

@jkwatson
Copy link
Contributor

It'd be cool if: 1. We had this customizer SPI. 2. our exporters adopted a convention of having a toBuilder() method so that you could do something like this to easily enhance the configuration done by autoconfiguration:

autoConfiguration.addSpanExporterCustomizer(spanExporter -> {
            if (spanExporter instanceof OtlpGrpcSpanExporter) {
              OtlpGrpcSpanExporter otlpGrpcSpanExporter = (OtlpGrpcSpanExporter) spanExporter;
              return otlpGrpcSpanExporter.toBuilder()
                  .addHeader("foo", "bar")
                  .build();
            }
            return spanExporter;
          });

We'd definitely need to make sure that the creation of an exporter didn't start it, though, so this would work. I'm looking at you, Prometheus exporter!

@anuraaga
Copy link
Contributor Author

Want to still add a test with a decorating span exporter, will do it after #3768. Otherwise think this is ok for another look

* @deprecated Use the information retrievable from {@link
* DefaultOpenTelemetrySdkAutoConfigurationBuilder#newResource()}.
*/
@Deprecated
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sure there must have been one but I can't remember the use case for this method. Perhaps it's not needed anymore thanks to addResourceCustomizer?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't find the context. Here's the PR where the first method was added to get the auto configured resource.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found the original issue

#2858

I think I remember it now but wonder if it's actually a use case in practice. I tend to like autoconfiguration least of most people :) But I've never imagined myself using that class when manually configuring an SDK - manual is manual

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a specific ask from someone in the community, IIRC. I think from someone who wanted to leverage the resource autoconfig, but otherwise do manual setup.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm - let me see if anyone complains about it being deprecated :)

if (sampler != null) {
tracerProviderBuilder.setSampler(samplerCustomizer.apply(configureSampler(sampler, config)));
if (sampler == null) {
sampler = "parentbased_always_on";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean the sampler does not currently default to parentbased_always_one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The builder itself defaults to it - but we should allow all components to go through customizers, so now we can't rely on the builder's default. While builder and env var default always should match up, I guess there's a bit of readability improvement having the string here by matching up with the spec's wording more closely.

* Returns a new {@link Resource} configured with the settings of this {@link
* OpenTelemetrySdkAutoConfigurationBuilder}.
*/
Resource newResource();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its a bit strange that OpenTelemetrySdkAutoconfigurationBuilder is the interface the applications use to customize autoconfigure (via the OpenTelemetrySdkAutoConfigurationCustomizer SPI), but also exposes methods to obtain a resource newResource(), and get a new opentelemetry sdk instnace newOpenTelemetrySdk().

It also feels weird to have a builder that doesn't have a build method. And that OpenTelemetrySdkAutoConfigurationBuilder doesn't produce OpenTelemetrySdkAutoConfiguration instances.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah agree on it feeling weird but I don't think we can put the interfaces in the SPI package otherwise if returning something built. Maybe it doesn't matter and they could just be in the autoconfigure module?

FWIW having a build would just make it .build().newOpenTelemetry() etc, the build I guess doesn't do much in practice here since what we want is the SDK.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about:

  • Change OpenTelemetrySdkAutoConfigurationCustomizer => AutoConfigurationCustomizerProvider
  • Change OpenTelemetrySdkAutoconfigurationBuilder => AutoConfigurationCustomizerBuilder
  • Remove newOpenTelemetrySdk() and newResource() methods
  • Add AutoConfigurationCustomizer build().
  • Add AutoConfigurationCustomizer interface defined as:
public interface AutoConfigurationCustomizer {
  ConfigProperties getConfig();
  TextMapPropagator customizePropagator(TextMapPropagator textMapPropagator);
  Resource customizeResource(Resource resource);
  Sampler customizeSampler(Sampler sampler);
  SpanExporter customizeSpanExporter(SpanExporter spanExporter);
  boolean shouldSetGlobal();
}

Many SPIs implementations of AutoConfigurationCustomizerProvider can participate in building the AutoConfigurationCustomizerBuilder. The aggregate of all customization is a AutoConfigurationCustomizer instance. By default if not customizer providers are specified via SPI, all methods of AutoConfigurationCustomizer are just identity implementations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intent of the PR is to not be SPI-only - many usages, especially those in an actual distribution or framework, not the end user app, don't need to use SPI and I think the ergonomics are better (we already have the three-element initialize method, and need to add ClassLoader as well for #3634 so definitely need a builder). The SPI accepting the builder is mostly for end users to customize a distribution (javaagent, lambda layer) despite not actually invoking autoconfiguration themselves.

I think the current API in this PR makes sense in general, the reason to run this code is to get a configured SDK so there isn't a big need for a build method. Perhaps the names are just misleading - OpenTelemetrySdkConfigurer, AutoConfiguredSdkBuilder, or any other naming changes that could help?

* @deprecated Use the information retrievable from {@link
* DefaultOpenTelemetrySdkAutoConfigurationBuilder#newResource()}.
*/
@Deprecated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't find the context. Here's the PR where the first method was added to get the auto configured resource.

Copy link
Contributor Author

@anuraaga anuraaga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jack-berg for the nudge in another direction. I've rewritten this PR significantly, and ended up deprecating every public method we currently have :)

The crux of the issue seemed to be that we have three independent entry points to auto-configured things, even though they are tightly coupled, and in practice we're really only supposed to auto-configure once. Instead I created a holder object for any auto configured components we may need to expose to users (just Resource and SDK for now) and this lent itself better to the standard builder pattern.

if (sampler != null) {
tracerProviderBuilder.setSampler(samplerCustomizer.apply(configureSampler(sampler, config)));
if (sampler == null) {
sampler = "parentbased_always_on";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The builder itself defaults to it - but we should allow all components to go through customizers, so now we can't rely on the builder's default. While builder and env var default always should match up, I guess there's a bit of readability improvement having the string here by matching up with the spec's wording more closely.

@codecov
Copy link

codecov bot commented Oct 20, 2021

Codecov Report

Merging #3753 (65403bc) into main (559d0b2) will increase coverage by 0.02%.
The diff coverage is 85.85%.

Impacted file tree graph

@@             Coverage Diff              @@
##               main    #3753      +/-   ##
============================================
+ Coverage     89.28%   89.31%   +0.02%     
- Complexity     3956     3983      +27     
============================================
  Files           473      475       +2     
  Lines         12307    12387      +80     
  Branches       1207     1210       +3     
============================================
+ Hits          10988    11063      +75     
- Misses          908      914       +6     
+ Partials        411      410       -1     
Impacted Files Coverage Δ
...toconfigure/OpenTelemetrySdkAutoConfiguration.java 61.90% <23.07%> (-20.45%) ⬇️
...nfigure/AutoConfiguredOpenTelemetrySdkBuilder.java 92.59% <92.59%> (ø)
...java/io/opentelemetry/api/GlobalOpenTelemetry.java 91.66% <100.00%> (+0.55%) ⬆️
.../autoconfigure/AutoConfiguredOpenTelemetrySdk.java 100.00% <100.00%> (ø)
...try/sdk/autoconfigure/DefaultConfigProperties.java 91.56% <100.00%> (+0.20%) ⬆️
...lemetry/sdk/autoconfigure/EnvironmentResource.java 100.00% <100.00%> (ø)
...figure/OpenTelemetryResourceAutoConfiguration.java 83.33% <100.00%> (-16.67%) ⬇️
...try/sdk/autoconfigure/PropagatorConfiguration.java 100.00% <100.00%> (ø)
...y/sdk/autoconfigure/SpanExporterConfiguration.java 95.00% <100.00%> (+0.33%) ⬆️
...sdk/autoconfigure/TracerProviderConfiguration.java 100.00% <100.00%> (ø)
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 559d0b2...65403bc. Read the comment docs.

* Sets the {@link ConfigProperties} to use when resolving properties for auto-configuration.
* {@link #addPropertySupplier(Supplier)} will have no effect if this method is used.
*/
public AutoConfiguredSdkBuilder setConfig(ConfigProperties config) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mateuszrzeszutek With addPropertySupplier, do you think we can just remove this method?

* <p>Multiple calls will execute the customizers in order.
*/
@Override
public AutoConfiguredOpenTelemetrySdkBuilder addResourceCustomizer(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cyrille-leclerc I think you'll be able to use this to merge in the default for namespace / name. If it's very common to do so we may even provide a helper in the future

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @anuraaga. I have a question please.

On one hand, it is clear to me from the javadocs that AutoConfiguredOpenTelemetrySdkCustomizer#addPropertiesSupplier(Supplier<Map<String, String>> propertiesSupplier) uses the precedence "system properties > environment variables > supplier properties".

On the other hand, I am not clear on the precedence of AutoConfiguredOpenTelemetrySdkBuilder#addResourceCustomizer(BiFunction<? super Resource, ConfigProperties, ? extends Resource> resourceCustomizer), maybe it's because I don't mentally represent what is a "BiFunction".

❓ Is #addResourceCustomizer intended to provide sensible defaults that are overridden by environment variables ...? (Exemple: default "service.name=jenkins")
❓ Is #addResourceCustomizer intended to provide overrides that take precedence on system properties...? (Exemple: jenkins plugin configuration defines "service.name='Jenkins Team Accounting'")
❓ Or can it do both?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say there is no explicit intent for this method, it is the final call on programatically determining the Resource. It means that requirements should be satisfiable as needed (unless I am missing any :-)

For example, either of these could be implemented based on what behavior you wanted

if (resource.get(ResourceAttributes.SERVICE_NAME) == null) {
  resource = resource.toBuilder().put(ResourceAttributes.SERVICE_NAME, "jenkins").build();
}
resource = resource.toBuilder().put(ResourceAttributes.SERVICE_NAME, "jenkins").build();

The former allows user configuration to take precedence, the latter doesn't, whichever is needed could be passed here.

Copy link
Member

@cyrille-leclerc cyrille-leclerc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This improvement is very interesting for author of integrations like what we do on the Jenkins Otel integration or the Maven Otel integration.
I should be able to test the API proposed by this PR against the Jenkins Otel integration configuration logic at the end of this week if it helps.

* <p>Multiple calls will execute the customizers in order.
*/
@Override
public AutoConfiguredOpenTelemetrySdkBuilder addResourceCustomizer(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @anuraaga. I have a question please.

On one hand, it is clear to me from the javadocs that AutoConfiguredOpenTelemetrySdkCustomizer#addPropertiesSupplier(Supplier<Map<String, String>> propertiesSupplier) uses the precedence "system properties > environment variables > supplier properties".

On the other hand, I am not clear on the precedence of AutoConfiguredOpenTelemetrySdkBuilder#addResourceCustomizer(BiFunction<? super Resource, ConfigProperties, ? extends Resource> resourceCustomizer), maybe it's because I don't mentally represent what is a "BiFunction".

❓ Is #addResourceCustomizer intended to provide sensible defaults that are overridden by environment variables ...? (Exemple: default "service.name=jenkins")
❓ Is #addResourceCustomizer intended to provide overrides that take precedence on system properties...? (Exemple: jenkins plugin configuration defines "service.name='Jenkins Team Accounting'")
❓ Or can it do both?

}

/**
* Adds a {@link Function} to invoke with the default autoconfigured {@link TextMapPropagator} to
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could there be a problem of copy paste of javadoc? This method doesn't refer to "TextMapPropagator".

}

/**
* Adds a {@link Function} to invoke with the default autoconfigured {@link TextMapPropagator} to
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could there be a problem of copy paste of javadoc? This method doesn't refer to "TextMapPropagator".

return () -> {
Map<String, String> merged = new HashMap<>();
merged.putAll(first.get());
merged.putAll(second.get());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it correct that the property otel.resource.attributes that is itself composed of properties is not benefiting of a nested merge? Is it correct that:

  • if first["otel.resource.attributes"]="service.name=Jenkins Team Accounting" and second["otel.resource.attributes"]="service.namespace=Production"
  • then mergeProperties(first, second) returns {"otel.resource.attributes"="service.name=Jenkins Team Accounting"}?

@cyrille-leclerc
Copy link
Member

This improvement is very interesting for author of integrations like what we do on the Jenkins Otel integration or the Maven Otel integration.
I should be able to test the API proposed by this PR against the Jenkins Otel integration configuration logic at the end of this week if it helps.

}

/**
* Adds a {@link Function} to invoke with the default autoconfigured {@link TextMapPropagator} to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these should all be s/Function/BiFunction/ I think.

@jkwatson
Copy link
Contributor

I think this is really powerful. It's definitely not "easy mode", but anyone who is going to be writing custom SPIs for configuring the SDK is probably not an "easy mode" user.

Any thoughts to exposing the other pieces of the SDK on the AutoConfiguredOpenTelemetrySdk instance? I guess if someone asks to have only the tracing SDK (or metrics, or whatever) exposed, it would be an easy place to stick it.

A few naming bikesheds to be painted, but other than that, this looks excellent, and seems to cover all the bases, including being extensible for metrics/logging in the future.

@anuraaga
Copy link
Contributor Author

Any thoughts to exposing the other pieces of the SDK on the AutoConfiguredOpenTelemetrySdk instance?

Anything in mind? OpenTelemetrySdk provides a getter for the tracer provider and eventually for meter provider or similar. Don't think I see something to add to it but there could be

@jkwatson
Copy link
Contributor

Any thoughts to exposing the other pieces of the SDK on the AutoConfiguredOpenTelemetrySdk instance?

Anything in mind? OpenTelemetrySdk provides a getter for the tracer provider and eventually for meter provider or similar. Don't think I see something to add to it but there could be

Ah. yes. I had forgotten that we exposed these. 👍🏽

Copy link
Contributor

@jkwatson jkwatson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge this and ruminate on precise naming as we go forward.

public interface AutoConfigurationCustomizer {

/**
* Adds a {@link Function} to invoke with the default autoconfigured {@link TextMapPropagator} to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bunch of Function references still in here to be replaced with BiFunction

@cyrille-leclerc
Copy link
Member

cyrille-leclerc commented Oct 26, 2021

I tested successfully the new AutoConfiguredOpenTelemetrySdkBuilder on the Jenkins OpenTelemetry plugin, it's a great simplification of my code. THANK YOU:

Here is a detailed feedback:

  • The AutoConfiguredOpenTelemetrySdkBuilder#addResourceCustomizer() is straightforward to provides overrides.
  • The ResourceProvider SPI to define the default settings of the component that embeds the Otel SDK has a small learning curve because it's a different programming pattern from the other patterns of the AutoConfiguredOpenTelemetrySdkBuilder but it was straightforward once I understood this pattern. A big benefit is that it's consistent with the Otel Resources Extension so it's intuitive and there are many of examples. See JenkinsResourceProvider
  • AutoConfiguredOpenTelemetrySdkBuilder#addPropertiesSupplier() works well to provide overrides. However, I still had to keep some boilerplate because the default value of "otel.traces.exporter" is "otlp". See Jenkins Otel Plugin code here
    • ❓ Did I miss an API to change the default of "otel.traces.exporter" to "none"?
  • AutoConfiguredOpenTelemetrySdk#getConfig() and AutoConfiguredOpenTelemetrySdk#getResource() are very smart to provide feedback to the user on which congiuration on the configuration that has actually been loaded.

@cyrille-leclerc
Copy link
Member

Hello again.
I successfully tested the proposed Auto Configured SDK Builder with the Maven OpenTelemetry Extension:

A challenge, similar to what we face with the Jenkins OpenTelemetry Plugin, is the boilerplate needed to change the default Otel SDK exporter from "otlp" to "none" if no OTLP exporter endpoint is specified.

A noteworthy improvement is that the ResourceProvider SPI now works with the OTEL SDK loaded through the isolated classloader used by Maven Plexus for extensions. Looking at the Otel SDK code, I don't know why this behavior changed.

See simplified code here:

@anuraaga
Copy link
Contributor Author

anuraaga commented Oct 27, 2021

@cyrille-leclerc Sorry for not replying earlier. In the unit test in this PR I set the default exporter to none using addPropertiesSupplier

https://github.com/open-telemetry/opentelemetry-java/pull/3753/files#diff-9a2c5a13fb22cf145c1812dcd314273ae90af9f2298a68c12f4e814e955734eaR106

It seems to work but does that pattern not work for you? That method is meant to override default config values.

Edit: Oh I see now, you want to change the default based on the config. Does the span exporter customizer work for that?

@cyrille-leclerc
Copy link
Member

cyrille-leclerc commented Oct 27, 2021

Thanks @anuraaga , no problem.

Edit: Oh I see now, you want to change the default based on the config. Does the span exporter customizer work for that?

I'm sorry but I'm not sure to understand how the span exporter customizer is supposed to work.

My need is that if no otel configuration is provided when starting my otel native tool, then the Otel SDK should turn to a "noop" instrumentation. I only want the Otel SDK to export traces if it is explicitly configured to do so.

Today, if no configuration is defined, the Otel Auto Configured SDK will export traces on GRPC OTLP to "http://localhost:4317".

I would like the default behavior to change so that if none of OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_TRACES_ENDPOINT is defined, then OTEL_TRACES_EXPORTER defaults to nonerather than otlp.

Here is a table to describe the change I would like to apply:

o.traces.exporter o.exporter.otlp.endpoint o.exporter.otlp.traces.endpoint Otel Trace Exporter
desired - actual
#not specified# #not specified# #not specified# actual=otlp/grpc http://localhost:4317
---
desired=none
#not specified# http://my-endpoint.example.com #not specified# actual=otlp/grpc http://my-endpoint.example.com
---
desired=#same as actual#
#not specified# #not specified# http://my-endpoint.example.com actual=otlp/grpc http://my-endpoint.example.com
---
desired=#same as actual#

Did I miss something in the API to do so?

@anuraaga
Copy link
Contributor Author

In the span exporter customizer, you could read the config and if endpoint isn't set return a noop exporter.

Though I don't know if I recommend this. I understand for builds making export default to off makes sense, so many unrelated people may run the build. But then, to debug a build performance problem, someone can just set OTEL_TRACES_EXPORTER=otlp and start a backend locally to get debugging. To keep track with a remote backend, set both OTEL_TRACES_EXPORTER=otlp and the endpoint.

For the remote backend case it's an extra variable but doesn't seem so bad. It allows the configuration to still be more consistent with the standard one. What do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants