Skip to content

S3 Source Connector throwing exception (Unable to instantiate request handler chain for client.) #1376

Open
@sunil-chormale-polestar

Description

I am using the s3 source connector to dump data from the s3 to kafka using kafka-connect-aws-s3-assembly-7.4.4.jar, Locally everything is working fine I am getting the data in the kafka, but with same configs on the EC2 box I am getting following exception.

software.amazon.awssdk.core.exception.SdkClientException: Unable to instantiate request handler chain for client. Listed request handler ('software.amazon.awssdk.core.internal.interceptor.HttpChecksumRequiredInterceptor') does not implement the interface software.amazon.awssdk.core.interceptor.ExecutionInterceptor API.
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111)
	at software.amazon.awssdk.core.interceptor.ClasspathInterceptorChainFactory.createExecutionInterceptor(ClasspathInterceptorChainFactory.java:134)
	at software.amazon.awssdk.core.interceptor.ClasspathInterceptorChainFactory.createExecutionInterceptorFromResource(ClasspathInterceptorChainFactory.java:95)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at software.amazon.awssdk.core.interceptor.ClasspathInterceptorChainFactory.createExecutionInterceptorsFromClasspath(ClasspathInterceptorChainFactory.java:64)
	at software.amazon.awssdk.core.interceptor.ClasspathInterceptorChainFactory.getGlobalInterceptors(ClasspathInterceptorChainFactory.java:58)
	at software.amazon.awssdk.core.client.builder.SdkDefaultClientBuilder.resolveExecutionInterceptors(SdkDefaultClientBuilder.java:520)
	at software.amazon.awssdk.core.client.builder.SdkDefaultClientBuilder.finalizeConfiguration(SdkDefaultClientBuilder.java:329)
	at software.amazon.awssdk.core.client.builder.SdkDefaultClientBuilder.syncClientConfiguration(SdkDefaultClientBuilder.java:199)
	at software.amazon.awssdk.services.s3.DefaultS3ClientBuilder.buildClient(DefaultS3ClientBuilder.java:37)
	at software.amazon.awssdk.services.s3.DefaultS3ClientBuilder.buildClient(DefaultS3ClientBuilder.java:26)
	at software.amazon.awssdk.core.client.builder.SdkDefaultClientBuilder.build(SdkDefaultClientBuilder.java:164)
	at io.lenses.streamreactor.connect.aws.s3.auth.AwsS3ClientCreator$.$anonfun$make$19(AwsS3ClientCreator.scala:84)
	at scala.util.Try$.apply(Try.scala:217)
	at io.lenses.streamreactor.connect.aws.s3.auth.AwsS3ClientCreator$.$anonfun$make$18(AwsS3ClientCreator.scala:84)
	at scala.util.Either.flatMap(Either.scala:360)
	at io.lenses.streamreactor.connect.aws.s3.auth.AwsS3ClientCreator$.$anonfun$make$7(AwsS3ClientCreator.scala:83)
	at scala.util.Either.flatMap(Either.scala:360)
	at io.lenses.streamreactor.connect.aws.s3.auth.AwsS3ClientCreator$.$anonfun$make$2(AwsS3ClientCreator.scala:55)
	at scala.util.Either.flatMap(Either.scala:360)
	at io.lenses.streamreactor.connect.aws.s3.auth.AwsS3ClientCreator$.make(AwsS3ClientCreator.scala:48)
	at io.lenses.streamreactor.connect.aws.s3.source.S3SourceTask.createClient(S3SourceTask.scala:53)
	at io.lenses.streamreactor.connect.aws.s3.source.S3SourceTask.createClient(S3SourceTask.scala:31)
	at io.lenses.streamreactor.connect.cloud.common.source.CloudSourceTask.$anonfun$make$2(CloudSourceTask.scala:130)
	at flatMap @ io.lenses.streamreactor.connect.cloud.common.source.CloudSourceTask.$anonfun$make$2(CloudSourceTask.scala:130)
	at flatMap @ io.lenses.streamreactor.connect.cloud.common.source.CloudSourceTask.$anonfun$make$1(CloudSourceTask.scala:129)
	at flatMap @ io.lenses.streamreactor.connect.cloud.common.source.CloudSourceTask.make(CloudSourceTask.scala:128)
	at flatMap @ io.lenses.streamreactor.connect.cloud.common.source.CloudSourceTask.start(CloudSourceTask.scala:83)

Here is my config file for s3-connector:

{
  "name": "S3-Source-Connector",
  "config": {
    "connector.class": "io.lenses.streamreactor.connect.aws.s3.source.S3SourceConnector",
    "tasks.max": "1",
    "connect.s3.aws.region": "us-east-1",
    "connect.s3.aws.access.key": "my-access-key",
    "connect.s3.aws.secret.key": "my-secret-key",
    "connect.s3.bucket": "test-bucket",
    "connect.s3.kcql": "INSERT INTO test-topic SELECT * FROM test-data-sync-service-qe:test-topic-2/1",
    "connect.s3.poll.interval": "60000",
    "connect.s3.aws.auth.mode": "Credentials",
    "connect.s3.source.partition.extractor.type": "hierarchical",
    "key.converter": "org.apache.kafka.connect.storage.StringConverter",
    "value.converter": "com.polestar.CustomConverter"
  }
}

I am using this kafka: kafka_2.12-3.8.0
Not sure why this is happening is there any dependency conflicts happening?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions