Skip to content

Conversation

@moolen
Copy link
Member

@moolen moolen commented Oct 30, 2025

Summary

Overview

The v2 provider enables out-of-process providers using gRPC, allowing a single provider codebase to expose multiple APIs (e.g., AWS SecretsManager, ParameterStore, ECR, STS) without requiring modifications to existing v1 provider implementations.

Open Issues

  • figure out options for mTLS authentication
  • research/document alternatives and their trade offs (such as: autodiscovery, referencing provider CRs directly)
  • referent authentication
  • provider-specific cmd flags (in compatibility with v1 providers)
    • favour a ENV-based approach for provider specific features. Some of the features could/should be part of the CRD.
  • gRPC client instantiation & scalability & testing strategy
  • TBD

ADR Deliverables

  • Controller Flow Diagram v1 vs v2
    • Adapter: why & how
    • Provider v2 internals
    • autogen of main.go
  • client_manager & gRPC connection pooling
  • mTLS, autodiscovery & credential management, threat model & trust boundaries
  • deployment of provider
  • command line flags & provider configuration
  • testing strategy
eso-secretstore-v2-SecretStorev2_Provider drawio (2)

Flow Diagram

graph TB
    subgraph "ESO Controller (In-Process)"
        A[ExternalSecret Controller] -->|"GetProviderSecretData()"| B[Client Manager]
        B -->|"Check storeRef.kind == Provider"| C{Is v2 Provider?}
        C -->|Yes| D[Create gRPC Client]
        C -->|No| E[Use v1 Provider In-Process]
        D --> F[V2ClientWrapper<br/>v1→v2 Adapter]
        F -->|"Implements<br/>esv1.SecretsClient"| G[gRPC Client]
    end
    
    subgraph "gRPC Communication"
        G -->|"GetSecretRequest<br/>{ProviderRef, RemoteRef}"| H[mTLS Connection]
    end
    
    subgraph "Provider Server (Out-of-Process)"
        H --> I[AdapterServer<br/>v2→v1 Adapter]
        I -->|"1. Parse ProviderRef<br/>(apiVersion + kind)"| J{Resolve GVK}
        J -->|"SecretsManager"| K[AWS v1 Provider]
        J -->|"ParameterStore"| K
        J -->|"ECRAuthToken"| K
        J -->|"STSSessionToken"| K
        K -->|"2. Fetch CR<br/>(e.g., SecretsManager)"| M[v1.SyntheticStore]
        M -->|"4. Call v1 provider"| N[provider.NewClient]
        N -->|"5. Get secret"| O[AWS API]
        O -->|"6. Return secret data"| H
    end

    style A fill:#e1f5ff
    style F fill:#ffe1f5
    style I fill:#fff5e1
    style K fill:#e1ffe1
Loading

1. Client-Side: v2 → v1 Adapter (In-Process)

How ExternalSecret Controller Uses gRPC Clients

In externalsecret_controller_secret.go, the reconciler uses the Client Manager to obtain provider clients:

// We MUST NOT create multiple instances of a provider client (mostly due to limitations with GCP)
// Clientmanager keeps track of the client instances
// that are created during the fetching process and closes clients
// if needed.
mgr := secretstore.NewManager(r.Client, r.ControllerClass, r.EnableFloodGate)

Client Manager: Creating gRPC Clients

When a SecretStoreRef has kind: Provider, the manager creates a gRPC client:

// Get returns a provider client from the given storeRef or sourceRef.secretStoreRef
// while sourceRef.SecretStoreRef takes precedence over storeRef.
// Do not close the client returned from this func, instead close
// the manager once you're done with recinciling the external secret.
func (m *Manager) Get(ctx context.Context, storeRef esv1.SecretStoreRef, namespace string, sourceRef *esv1.StoreGeneratorSourceRef) (esv1.SecretsClient, error) {
	if storeRef.Kind == "Provider" {
		return m.getV2ProviderClient(ctx, storeRef.Name, namespace)
	}
	if sourceRef != nil && sourceRef.SecretStoreRef != nil {
		storeRef = *sourceRef.SecretStoreRef
	}
	store, err := m.getStore(ctx, &storeRef, namespace)
	if err != nil {
		return nil, err
	}
	// check if store should be handled by this controller instance
	if !ShouldProcessStore(store, m.controllerClass) {
		return nil, errors.New("can not reference unmanaged store")
	}
	// when using ClusterSecretStore, validate the ClusterSecretStore namespace conditions
	shouldProcess, err := m.shouldProcessSecret(store, namespace)
	if err != nil {
		return nil, err
	}
	if !shouldProcess {
		return nil, fmt.Errorf(errClusterStoreMismatch, store.GetName(), namespace)
	}

	if m.enableFloodgate {
		err := assertStoreIsUsable(store)
		if err != nil {
			return nil, err
		}
	}
	return m.GetFromStore(ctx, store, namespace)
}

The getV2ProviderClient method:

  1. Fetches the Provider resource
  2. Creates a gRPC connection with TLS
  3. Wraps it with V2ClientWrapper (the v2→v1 adapter)
// Create gRPC client
grpcClient, err := grpc.NewClient(address, tlsConfig)
if err != nil {
	return nil, fmt.Errorf("failed to create gRPC client for Provider %q: %w", providerName, err)
}

// Convert ProviderReference to protobuf format
providerRef := &pb.ProviderReference{
	ApiVersion: provider.Spec.Config.ProviderRef.APIVersion,
	Kind:       provider.Spec.Config.ProviderRef.Kind,
	Name:       provider.Spec.Config.ProviderRef.Name,
	Namespace:  provider.Spec.Config.ProviderRef.Namespace,
}

// Wrap with V2ClientWrapper
wrappedClient := adapter.NewV2ClientWrapper(grpcClient, providerRef, namespace)

V2ClientWrapper: Implementing v1.SecretsClient

The wrapper adapts the gRPC v2.Provider interface to the v1 SecretsClient interface:

// V2ClientWrapper wraps a v2.Provider (gRPC client) and exposes it as an esv1.SecretsClient.
// This allows v2 providers to be used with the existing client manager infrastructure.
type V2ClientWrapper struct {
	v2Provider      v2.Provider
	providerRef     *pb.ProviderReference
	sourceNamespace string
}

// Ensure V2ClientWrapper implements SecretsClient interface
var _ esv1.SecretsClient = &V2ClientWrapper{}

// NewV2ClientWrapper creates a new wrapper that adapts a v2.Provider to esv1.SecretsClient.
func NewV2ClientWrapper(v2Provider v2.Provider, providerRef *pb.ProviderReference, sourceNamespace string) esv1.SecretsClient {
	return &V2ClientWrapper{
		v2Provider:      v2Provider,
		providerRef:     providerRef,
		sourceNamespace: sourceNamespace,
	}
}

// GetSecret retrieves a single secret from the provider.
func (w *V2ClientWrapper) GetSecret(ctx context.Context, ref esv1.ExternalSecretDataRemoteRef) ([]byte, error) {
	return w.v2Provider.GetSecret(ctx, ref, w.providerRef, w.sourceNamespace)
}

gRPC Client: Making RPC Calls

The gRPC client converts v1 types to protobuf and makes RPC calls:

// GetSecret retrieves a single secret from the provider via gRPC.
func (c *grpcProviderClient) GetSecret(ctx context.Context, ref esv1.ExternalSecretDataRemoteRef, providerRef *pb.ProviderReference, sourceNamespace string) ([]byte, error) {
	c.log.V(1).Info("getting secret via gRPC",
		"key", ref.Key,
		"version", ref.Version,
		"property", ref.Property,
		"connectionState", c.conn.GetState().String(),
		"providerRef", providerRef,
		"sourceNamespace", sourceNamespace)

	// Check connection state before call
	state := c.conn.GetState()
	if state != connectivity.Ready && state != connectivity.Idle {
		c.log.Info("connection not ready, attempting to reconnect",
			"state", state.String(),
			"target", c.conn.Target())
	}

	// Create context with timeout
	ctx, cancel := context.WithTimeout(ctx, defaultTimeout)
	defer cancel()

	// Convert v1 reference to protobuf message
	pbRef := &pb.ExternalSecretDataRemoteRef{
		Key:              ref.Key,
		Version:          ref.Version,
		Property:         ref.Property,
		DecodingStrategy: string(ref.DecodingStrategy),
		MetadataPolicy:   string(ref.MetadataPolicy),
	}

	// Make gRPC call with provider reference
	req := &pb.GetSecretRequest{
		RemoteRef:       pbRef,
		ProviderRef:     providerRef,
		SourceNamespace: sourceNamespace,
	}

	c.log.V(1).Info("calling GetSecret RPC",
		"target", c.conn.Target(),
		"timeout", defaultTimeout.String())

	resp, err := c.client.GetSecret(ctx, req)
	if err != nil {
		c.log.Error(err, "GetSecret RPC failed",
			"key", ref.Key,
			"connectionState", c.conn.GetState().String(),
			"target", c.conn.Target())
		return nil, fmt.Errorf("failed to get secret via gRPC: %w", err)
	}

	c.log.V(1).Info("GetSecret RPC succeeded",
		"key", ref.Key,
		"valueLength", len(resp.Value))

	return resp.Value, nil
}

2. Multiple APIs via ProviderReference Mapping

Separate CRDs for Each AWS Service

The AWS v2 provider exposes separate Kubernetes Custom Resources for different services:

var (
	// GroupVersion is group version used to register these objects
	GroupVersion = schema.GroupVersion{Group: "provider.external-secrets.io", Version: "v2alpha1"}

	// SecretsManagerKind is the kind name used for SecretsManager resources.
	SecretsManagerKind = "SecretsManager"

	// SchemeBuilder is used to add go types to the GroupVersionKind scheme
	SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion}

	// AddToScheme adds the types in this group-version to the given scheme.
	AddToScheme = SchemeBuilder.AddToScheme
)

Example: SecretsManager CRD:

// SecretsManager is the Schema for AWS Secrets Manager provider configuration
type SecretsManager struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   SecretsManagerSpec   `json:"spec,omitempty"`
	Status SecretsManagerStatus `json:"status,omitempty"`
}

Future expansion will include ParameterStore, ECRAuthToken, STSSessionToken, etc., all served by the same gRPC server process.


3. Server-Side: v1 → v2 Adapter (Out-of-Process)

AdapterServer: Mapping ProviderRef to v1 Clients

The gRPC server uses AdapterServer to map incoming ProviderReference (apiVersion + kind) to v1 provider implementations:

// AdapterServer wraps a v1 provider and exposes it as a v2 gRPC service.
// This allows existing v1 provider implementations to be used in the v2 architecture.
type AdapterServer struct {
	pb.UnimplementedSecretStoreProviderServer
	kubeClient client.Client

	// we support multiple v1 providers, so we need to map the v2 provider
	// with apiVersion+kind to the corresponding v1 provider
	resourceMapping ProviderMapping
	specMapper      SpecMapper
}

type ProviderMapping map[schema.GroupVersionKind]esv1.ProviderInterface

// maps a provider reference to a SecretStoreSpec
// which is used to create a synthetic store for the v1 provider.
type SpecMapper func(ref *pb.ProviderReference) (*esv1.SecretStoreSpec, error)

// NewAdapterServer creates a new AdapterServer that wraps a v1 provider.
func NewAdapterServer(kubeClient client.Client, resourceMapping ProviderMapping, specMapping SpecMapper) *AdapterServer {
	return &AdapterServer{
		kubeClient:      kubeClient,
		resourceMapping: resourceMapping,
		specMapper:      specMapping,
	}
}

Resolving Provider from ProviderReference

The server resolves the v1 provider based on GVK:

func (s *AdapterServer) resolveProvider(ref *pb.ProviderReference) (esv1.ProviderInterface, error) {
	if ref == nil {
		return nil, fmt.Errorf("provider reference is nil")
	}

	splitted := strings.Split(ref.ApiVersion, "/")
	if len(splitted) != 2 {
		return nil, fmt.Errorf("invalid api version: %s", ref.ApiVersion)
	}
	group := splitted[0]
	version := splitted[1]

	key := schema.GroupVersionKind{
		Group:   group,
		Version: version,
		Kind:    ref.Kind,
	}
	v1Provider, ok := s.resourceMapping[key]
	if !ok {
		return nil, fmt.Errorf("resource mapping not found for %q", key)
	}
	return v1Provider, nil
}

func (s *AdapterServer) getClient(ctx context.Context, ref *pb.ProviderReference, namespace string) (esv1.SecretsClient, error) {
	if ref == nil {
		return nil, fmt.Errorf("request or remote ref is nil")
	}

	spec, err := s.specMapper(ref)
	if err != nil {
		return nil, fmt.Errorf("failed to map provider reference to spec: %w", err)
	}
	// TODO: support cluster scoped Provider
	store, err := NewSyntheticStore(spec, namespace)
	if err != nil {
		return nil, fmt.Errorf("failed to create synthetic store: %w", err)
	}
	provider, err := s.resolveProvider(ref)
	if err != nil {
		return nil, fmt.Errorf("failed to resolve provider: %w", err)
	}
	return provider.NewClient(ctx, store, s.kubeClient, namespace)
}

GetSecret RPC Handler

The server receives GetSecret requests and delegates to v1 providers:

// GetSecret retrieves a single secret from the provider.
func (s *AdapterServer) GetSecret(ctx context.Context, req *pb.GetSecretRequest) (*pb.GetSecretResponse, error) {
	if req == nil || req.RemoteRef == nil {
		return nil, fmt.Errorf("request or remote ref is nil")
	}
	client, err := s.getClient(ctx, req.ProviderRef, req.SourceNamespace)
	if err != nil {
		return nil, fmt.Errorf("failed to get client: %w", err)
	}
	defer client.Close(ctx)

	// Convert protobuf remote ref to v1 remote ref
	ref := esv1.ExternalSecretDataRemoteRef{
		Key:      req.RemoteRef.Key,
		Version:  req.RemoteRef.Version,
		Property: req.RemoteRef.Property,
	}
	if req.RemoteRef.DecodingStrategy != "" {
		ref.DecodingStrategy = esv1.ExternalSecretDecodingStrategy(req.RemoteRef.DecodingStrategy)
	}
	if req.RemoteRef.MetadataPolicy != "" {
		ref.MetadataPolicy = esv1.ExternalSecretMetadataPolicy(req.RemoteRef.MetadataPolicy)
	}

	value, err := client.GetSecret(ctx, ref)
	if err != nil {
		return nil, fmt.Errorf("failed to get secret: %w", err)
	}

	return &pb.GetSecretResponse{
		Value: value,
	}, nil
}

AWS Provider Main: Single Process, Multiple APIs

The AWS provider's main function sets up the mapping:

v1Provider := awsv1.NewProvider()
adapterServer := adapter.NewAdapterServer(kubeClient, adapter.ProviderMapping{
	schema.GroupVersionKind{
		Group:   awsv2alpha1.GroupVersion.Group,
		Version: awsv2alpha1.GroupVersion.Version,
		Kind:    awsv2alpha1.SecretsManagerKind,
	}: v1Provider,
}, func(ref *pb.ProviderReference) (*v1.SecretStoreSpec, error) {
	if ref.Kind != awsv2alpha1.SecretsManagerKind {
		return nil, fmt.Errorf("unsupported provider kind: %s", ref.Kind)
	}
	var awsProvider awsv2alpha1.SecretsManager
	err := kubeClient.Get(context.Background(), client.ObjectKey{
		Namespace: ref.Namespace,
		Name:      ref.Name,
	}, &awsProvider)
	if err != nil {
		return nil, err
	}
	return &v1.SecretStoreSpec{
		Provider: &v1.SecretStoreProvider{
			AWS: &v1.AWSProvider{
				Service:           v1.AWSServiceSecretsManager,
				Auth:              awsProvider.Spec.Auth,
				Role:              awsProvider.Spec.Role,
				Region:            awsProvider.Spec.Region,
				AdditionalRoles:   awsProvider.Spec.AdditionalRoles,
				ExternalID:        awsProvider.Spec.ExternalID,
				SecretsManager:    awsProvider.Spec.SecretsManager,
				SessionTags:       awsProvider.Spec.SessionTags,
				TransitiveTagKeys: awsProvider.Spec.TransitiveTagKeys,
				Prefix:            awsProvider.Spec.Prefix,
			},
		},
	}, nil
})

To add ParameterStore, you'd simply extend the mapping:

schema.GroupVersionKind{
    Group:   awsv2alpha1.GroupVersion.Group,
    Version: awsv2alpha1.GroupVersion.Version,
    Kind:    "ParameterStore",
}: v1Provider,  // Same v1 provider instance!

And update the specMapper to handle the new Kind.


Example manifest

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: database-credentials
  namespace: external-secrets-system
spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: Provider  # <-- 
    name: aws-provider
  target:
    name: creds  
  data:
  - secretKey: creds
    remoteRef:
      key: app-credentials
---
apiVersion: external-secrets.io/v1
kind: Provider
metadata:
  name: aws-provider
  namespace: external-secrets-system
spec:
  config:
    address: provider-aws.external-secrets-system.svc:8080
    providerRef:
      apiVersion: provider.external-secrets.io/v2alpha1
      kind: SecretsManager
      name: aws-sm
      namespace: external-secrets-system
---
apiVersion: provider.external-secrets.io/v2alpha1
kind: SecretsManager
metadata:
  name: aws-sm
  namespace: external-secrets-system
spec:
  region: eu-central-1
  auth: {}

@github-actions github-actions bot added size/xl kind/documentation Categorizes issue or PR as related to documentation. kind/dependency dependabot and upgrades component/github-actions and removed size/xl labels Oct 30, 2025
@moolen moolen marked this pull request as draft October 30, 2025 14:19
Signed-off-by: Moritz Johner <[email protected]>
@sonarqubecloud
Copy link

sonarqubecloud bot commented Nov 6, 2025

Quality Gate Failed Quality Gate failed

Failed conditions
3 Security Hotspots
7.5% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/github-actions kind/dependency dependabot and upgrades kind/documentation Categorizes issue or PR as related to documentation. size/xl size/xs

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants