-
Notifications
You must be signed in to change notification settings - Fork 231
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ADAP-518] Convert information to a dict
#751
Comments
dict
dict
Thanks for kicking this off @Fokko 🏆 I started a As you create more issues for this, just let me know and we'll add them to that tasklist. |
@dbeatty10 Thanks! Much appreciated |
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days. |
Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers. |
Is this your first time submitting a feature request?
Describe the feature
In preparation for having three-part identifiers
catalog.schema.table
(#755), I would like to change theinformation
attribute on theSparkRelation
into adict
:dbt-spark/dbt/adapters/spark/relation.py
Lines 36 to 37 in cb41ab0
Describe alternatives you've considered
The current way is unmaintainable with the regex that extracts useful information from the big blob of text. Also, I noticed that the types are missing currently:
Who will this benefit?
Mostly the developers because it is hard to maintain right now, and hard to extend the current situation
Are you interested in contributing this feature?
Yes!
Anything else?
I wanted to add the database to the configuration. In Spark, this is called a catalog: https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.Catalog.html
Since Spark 3.0 it can discover tables/views from multiple catalogs, such as a Hive or Glue catalog. I would love to add this, but this refactor needs to be done first and I also want to keep the PRs concise.
The text was updated successfully, but these errors were encountered: