Known 2.2 image version issues and limitations:
- Data Lineage is not available since Spark 3.5 does not support lineage data collection.
- Legacy agents are not installed in 2.2 image version clusters.
monitoring-agent-defaults
are not available unless the Ops Agent
is installed. Note: OSS metrics and logging are
available for Dataproc components.
- Logging or monitoring for third-party applications are not available unless the Ops Agent is installed.
Notes:
- The source code to image 2.2 libraries that are licensed under Reciprocal
and Restricted licenses is available at the
/usr/local/share/google/dataproc/third-party-sources
path on Dataproc cluster VMs. - The following Hudi procedures
are known to not work on a Hudi table backed by the Cloud Storage file system:
run_clustering
run_compaction