tfx_bsl.public.tfxio.TensorAdapter

A TensorAdapter converts a RecordBatch to a collection of TF Tensors.

tfx_bsl.public.tfxio.TensorAdapter(
    config: tfx_bsl.public.tfxio.TensorAdapterConfig
)

The conversion is determined by both the Arrow schema and the TensorRepresentations, which must be provided at the initialization time. Each TensorRepresentation contains the information needed to translates one or more columns in a RecordBatch of the given Arrow schema into a TF Tensor or CompositeTensor. They are contained in a Dict whose keys are the names of the tensors, which will be the keys of the Dict produced by ToBatchTensors().

TypeSpecs() returns static TypeSpecs of those tensors by their names, i.e. if they have a shape, then the size of the first (batch) dimension is always unknown (None) because it depends on the size of the RecordBatch passed to ToBatchTensors().

It is guaranteed that for any tensor_name in the given TensorRepresentations self.TypeSpecs()[tensor_name].is_compatible_with( self.ToBatchedTensors(...)[tensor_name])

Sliced RecordBatches and LargeListArray columns having null elements backed by non-empty sub-lists are not supported and will yield undefined behaviour.

Methods

OriginalTypeSpecs

OriginalTypeSpecs() -> Dict[str, tf.TypeSpec]

Returns the origin's type specs.

A TFXIO 'Y' may be a result of projection of another TFXIO 'X', in which case then 'X' is the origin of 'Y'. And this method returns what X.TensorAdapter().TypeSpecs() would return.

May equal to self.TypeSpecs().

Returns: a mapping from tensor names to tf.TypeSpecs.

ToBatchTensors

ToBatchTensors(
    record_batch: pa.RecordBatch, produce_eager_tensors: Optional[bool] = None
) -> Dict[str, Any]

Returns a batch of tensors translated from record_batch.

Args
record_batch	input RecordBatch.
produce_eager_tensors	controls whether the ToBatchTensors() produces eager tensors or ndarrays (or Tensor value objects). If None, determine that from whether TF Eager mode is enabled.

Raises
RuntimeError	when Eager Tensors are requested but TF is not executing eagerly.
ValueError	when Any handler failed to produce a Tensor.

TypeSpecs

TypeSpecs() -> Dict[str, tf.TypeSpec]

Returns the TypeSpec for each tensor.