tfx.v1.components.Evaluator

A TFX component to evaluate models trained by a TFX Trainer component.

Inherits From: BaseBeamComponent, BaseComponent, BaseNode

tfx.v1.components.Evaluator(
    examples: tfx.v1.types.BaseChannel,
    model: Optional[tfx.v1.types.BaseChannel] = None,
    baseline_model: Optional[tfx.v1.types.BaseChannel] = None,
    feature_slicing_spec: Optional[Union[tfx.v1.proto.FeatureSlicingSpec, tfx.v1.dsl.experimental.RuntimeParameter]] = None,
    fairness_indicator_thresholds: Optional[Union[List[float], tfx.v1.dsl.experimental.RuntimeParameter]] = None,
    example_splits: Optional[List[str]] = None,
    eval_config: Optional[tfma.EvalConfig] = None,
    schema: Optional[tfx.v1.types.BaseChannel] = None,
    module_file: Optional[str] = None,
    module_path: Optional[str] = None
)

Used in the notebooks

Used in the tutorials

Component outputs contains:

  • evaluation: Channel of type standard_artifacts.ModelEvaluation to store the evaluation results.
  • blessing: Channel of type `standard_artifacts.ModelBlessing' that contains the blessing result.

See the Evaluator guide for more details.

examples A BaseChannel of type standard_artifacts.Examples, usually produced by an ExampleGen component. required
model A BaseChannel of type standard_artifacts.Model, usually produced by a Trainer component.
baseline_model An optional channel of type 'standard_artifacts.Model' as the baseline model for model diff and model validation purpose.
feature_slicing_spec Deprecated, please use eval_config instead. Only support estimator. evaluator_pb2.FeatureSlicingSpec instance that describes how Evaluator should slice the data.
fairness_indicator_thresholds Optional list of float (or RuntimeParameter) threshold values for use with TFMA fairness indicators. Experimental functionality: this interface and functionality may change at any time. to additional documentation for TFMA fairness indicators here.
example_splits Names of splits on which the metrics are computed. Default behavior (when example_splits is set to None or Empty) is using the 'eval' split.
eval_config Instance of tfma.EvalConfig containg configuration settings for running the evaluation. This config has options for both estimator and Keras.
schema A Schema channel to use for TFXIO.
module_file A path to python module file containing UDFs for Evaluator customization. This functionality is experimental and may change at any time. The module_file can implement following functions at its top level. def custom_eval_shared_model( eval_saved_model_path, model_name, eval_config, **kwargs, ) -> tfma.EvalSharedModel: def custom_extractors( eval_shared_model, eval_config, tensor_adapter_config, ) -> List[tfma.extractors.Extractor]:
module_path A python path to the custom module that contains the UDFs. See 'module_file' for the required signature of UDFs. This functionality is experimental and this API may change at any time. Note this can not be set together with module_file.

outputs Component's output channel dict.

Methods

with_beam_pipeline_args

with_beam_pipeline_args(
    beam_pipeline_args: Iterable[Union[str, placeholder.Placeholder]]
) -> 'BaseBeamComponent'

Add per component Beam pipeline args.

Args
beam_pipeline_args List of Beam pipeline args to be added to the Beam executor spec.

Returns
the same component itself.

with_node_execution_options

with_node_execution_options(
    node_execution_options: utils.NodeExecutionOptions
) -> typing_extensions.Self

POST_EXECUTABLE_SPEC None
PRE_EXECUTABLE_SPEC None