Description

This pull request introduces support for two new column types from the Iceberg table format: string and binary. These types are increasingly common in data warehousing scenarios, and their inclusion in sqlalchemy-redshift will enhance compatibility with Iceberg tables and improve the user experience for those working with Redshift and Iceberg.

This PR is specifically tailored to add/improve interactions with Spectrum tables in Iceberg format.

Changes

• Added a new custom type ICEBERG_STRING that maps Iceberg's string type to Redshift/Postgre's VARCHAR type with a default max length.
• Added a new custom type ICEBERG_BINARY that maps Iceberg's binary type to SQLAlchemy's LargeBinary type.
• Included unit tests
• Updated the documentation to reflect the new type support.

Motivation

As Iceberg tables become more prevalent, it is important for sqlalchemy-redshift to evolve and support the diverse data types used in these tables. The lack of native support for string and binary types can lead to challenges when querying Iceberg tables, and this PR aims to address those challenges.

Backward Compatibility

These changes are fully backward-compatible as they introduce new types without altering the existing functionality of the dialect.

Additional Notes

I look forward to feedback from the maintainers and community on this PR. If there are any additional changes or enhancements needed, I am open to discussion and collaboration.

Todos

• MIT compatible
• Tests
• Documentation
• Updated CHANGES.rst