Is this your first time submitting a feature request?

• I have read the expectations for open source contributors
• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing dbt-spark functionality, rather than a Big Idea better suited to a discussion

Describe the feature

Delta Lake 2.0 supports dropping of columns on delta tables if the table has certain tblproperties (https://delta.io/blog/2022-08-29-delta-lake-drop-column/) so we may want to support the on_schema_change = 'sync_all_columns' config when columns are removed from the source (compared to the target).

Describe alternatives you've considered

A user could probably achieve this today by rewriting some of the relevant built-in macros.

Who will this benefit?

Previously a schema change (specifically a column removed from source vs target) will result in an exception:

Primarily this will bring it up to par with other adapters's behaviour for on_schema_change without having users to implement their own overrides for the relevant helper macros (e.g. alter_relation_add_remove_columns() and family).

Are you interested in contributing this feature?

Sure

Anything else?

We did not support this behaviour back in #229 because delta could not drop columns - this is now supported (albeit with the necessary tblproperties applied).

Probably the way to do this is to retrieve the tblproperties of the target - and then decide whether to raise or not (or perhaps warn with a suggestion "in order to drop columns, please alter the tblproperties, ...".

We probably want this in dbt-databricks too. The behaviour in the adapter is the same as it is here.