24 Pull requests merged by 13 people

• [Spark] Make physical name assignment case-insensitive on the logical field names

  #4405 merged Apr 11, 2025

• [Kernel]Keep track of all file size related stats correctly in CRC

  #4328 merged Apr 11, 2025

• Adding a Spark config to force Delta Sharing return delta format

  #4401 merged Apr 11, 2025

• [Branch-4.0-kernel] Upgrade Hadoop to 3.4.0

  #4400 merged Apr 11, 2025

• [Kernel] Read the fileSizeHistogram from CRCInfo and propagate it through transaction metrics

  #4403 merged Apr 11, 2025

• [Spark] Implement CatalogOwnedTableFeature

  #4394 merged Apr 11, 2025

• [Spark] [Catalog-Owned] Use _staged_commits/ folder for staged/unbackfilled commit files

  #4399 merged Apr 11, 2025

• [Spark] Improve DeltaVersionsNotContiguousException error message & add extra logging

  #4358 merged Apr 11, 2025

• [Kernel] Add removeFileSizeInBytes to transaction metrics and increment when removes are committed

  #4398 merged Apr 10, 2025

• [Kernel] Add comment

  #4395 merged Apr 10, 2025

• [Kernel] Add Universal format TableConfig.

  #4375 merged Apr 9, 2025

• [Kernel][icebergWriterCompatV1] Allow legacy features to be in the table protocol if they are inactive

  #4386 merged Apr 9, 2025

• [Spark] Add option to DeltaParquetFileFormat to allow missing Row Tracking metadata fields

  #4387 merged Apr 9, 2025

• Minor improvements to Delta streaming source

  #4388 merged Apr 9, 2025

• [SPARK] Fix uniqueFileActionGroupBy in CloneTableSuiteBase.scala

  #4393 merged Apr 9, 2025

• [SPARK] Actualize sqlState for delta feature drop errors

  #4380 merged Apr 9, 2025

• [Kernel] Support creating remove actions for Iceberg metadata conversion

  #4369 merged Apr 9, 2025

• [Kernel][IcebergWriterCompatV1] Update column mapping so physicalName='col-{columnId}' and enforce this as part of our checks

  #4383 merged Apr 9, 2025

• [Spark] Minor refactor to IcebergTable util

  #4385 merged Apr 8, 2025

• [Kernel][icebergWriterCompatV1] Block disabling the icebergWriterCompatV1 table property

  #4379 merged Apr 8, 2025

• [Kernel] Support creating add actions for Iceberg metadata conversion

  #4368 merged Apr 7, 2025

• [UniForm] Refactoring IcebergTransaction

  #4373 merged Apr 7, 2025

• Unify and Improve error message for Delta source schema tracking

  #4378 merged Apr 7, 2025

• [SPARK] Do not remove EXISTS_DEFAULT on every metadata update

  #4308 merged Apr 7, 2025

8 Pull requests opened by 7 people

• [WIP][Kernel] add dvs as a part of DataFileStatus and then use it during append actions

  #4376 opened Apr 7, 2025

• [Spark ]Add config to disable schema tracking for type widening

  #4377 opened Apr 7, 2025

• [PROTOCOL RFC] Catalog-owned Tables

  #4382 opened Apr 7, 2025

• [Spark] Make MERGE source checkpoint materialization a SQL operation

  #4390 opened Apr 9, 2025

• [SPARK][WIP] Refactor error test suites to use `checkError()`

  #4392 opened Apr 9, 2025

• [Kernel] Schema validation for clustering columns

  #4402 opened Apr 11, 2025

• [Kernel]Try loading crc file on conflict resolution

  #4406 opened Apr 11, 2025

• [Kernel]Add domain metadata to CRC Info

  #4407 opened Apr 11, 2025

4 Issues closed by 3 people

• [BUG] Integer and Long Types Appear as 'Unknown' in S3 Bucket Data Table With AWS Athena When Using AWS Glue Catalog

  #4397 closed Apr 10, 2025

• [BUG] Import for this method is missing.

  #4226 closed Apr 10, 2025

• [Feature Request][Kernel] Support Universal Format Table Configuration

  #4374 closed Apr 9, 2025

• [Protocol Change Request] Delta Coordinated Commits

  #2598 closed Apr 7, 2025

6 Issues opened by 5 people

• [Feature Request] [Kernel] Update exception message for gaps in the delta log to provide context of catalogOwned feature

  #4404 opened Apr 11, 2025

• [BUG] Empty Results When Querying Delta Lake Table from AWS Athena

  #4396 opened Apr 10, 2025

• [BUG] CDF stream reading from latest is not working as expected

  #4391 opened Apr 9, 2025

• [BUG] [Core] [Spark] Not able to rename/drop special column when enable dataSkippingStatsColumns

  #4389 opened Apr 9, 2025

• [BUG] [Spark] No Streaming query statistics generated when write Protocol Version >= 4

  #4384 opened Apr 8, 2025

• [PROTOCOL RFC] Catalog-owned Tables

  #4381 opened Apr 7, 2025

16 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Add support for writing log compaction files

  #4334 commented on Apr 12, 2025 • 58 new comments

• [Spark] Resolves #1679 issue glue catalog

  #2310 commented on Apr 8, 2025 • 1 new comment

• [BUG] [Spark] Java code using delta share client throws java.lang.NoSuchMethodError: 'org.apache.spark.sql.types.StructType org.apache.spark.sql.delta.DeltaTableUtils$.removeInternalWriterMetadata

  #4254 commented on Apr 7, 2025 • 0 new comments

• Not null invariant incorrectly fails on non-nullable field inside a nullable struct

  #860 commented on Apr 7, 2025 • 0 new comments

• [BUG][Spark] Deletion Vectors can cause `AMBIGUOUS_REFERENCE` errors on MERGE

  #2943 commented on Apr 7, 2025 • 0 new comments

• [Feature Request][Kernel] Support IcebergWriterCompatV1 in Kernel

  #4289 commented on Apr 9, 2025 • 0 new comments

• [BUG] removeInternalMetadata

  #4297 commented on Apr 10, 2025 • 0 new comments

• [PROTOCOL RFC] Support other columnar file formats such as `Vortex`

  #4213 commented on Apr 10, 2025 • 0 new comments

• [Feature Request] Support for remote shuffle services in OptimizedWrite

  #4343 commented on Apr 10, 2025 • 0 new comments

• [Feature Request][Spark] Support VACUUM LITE in user-facing API

  #4259 commented on Apr 10, 2025 • 0 new comments

• [Feature Request] Generalize isUnityCatalog concept for general REST TableCatalog Usage

  #4306 commented on Apr 12, 2025 • 0 new comments

• [Feature Request] [Kernel] Delete a partition or delete Parquet files from a table

  #3988 commented on Apr 13, 2025 • 0 new comments

• [BUG][Spark] Unable to read columns which has `DEFAULT` value set during table creation

  #4370 commented on Apr 14, 2025 • 0 new comments

• [PySpark] Pass scala version as optional argument to configure_spark_with_delta_pip

  #3505 commented on Apr 7, 2025 • 0 new comments

• [Spark] Fix metadata cleanup by retaining a checkpoint before the cutoff window. Alternative fix

  #4146 commented on Apr 8, 2025 • 0 new comments

• Add RFC for IcebergWriterCompatV1

  #4285 commented on Apr 10, 2025 • 0 new comments