Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redshift Multi-AZ cannot use STL scan #11922

Closed
WiseLin1125 opened this issue Nov 22, 2024 · 5 comments
Closed

Redshift Multi-AZ cannot use STL scan #11922

WiseLin1125 opened this issue Nov 22, 2024 · 5 comments
Labels
bug Bug report ingestion PR or Issue related to the ingestion of metadata

Comments

@WiseLin1125
Copy link

WiseLin1125 commented Nov 22, 2024

Describe the bug
Redshift Multi-AZ cannot use STL scan
DataHub version: v0.13.2
Redshift configuration: multi-AZ

To Reproduce
Steps to reproduce the behavior:

  1. Go to Redshift Query in query editor v2
  2. Execute this command

SELECT * FROM stl_insert LIMIT 10;

  1. And you will get

ERROR: permission denied for relation stl_insert [ErrorId: 1-67402a42-124aa79d44733fe95b73d753]

Expected behavior
It should be queried successfully.
Or could you please help to guide how to modify it in the source code? EX: Using other SYS_* views to modify?

Screenshots
Screenshot 2024-11-22 at 3 10 08 PM

Desktop (please complete the following information):
Chrome browser

Additional context
This issue will impact the Redshift ingestion feature.
You can find the AWS official document here https://docs.aws.amazon.com/redshift/latest/mgmt/overview-multi-az.html and which mentioned You can't use STL, SVCS, SVL, SVV, STV views with Multi-AZ deployments as they only support system monitoring views (SYS_* views). Change your monitoring queries to use system monitoring views (SYS_* views).

@WiseLin1125 WiseLin1125 added the bug Bug report label Nov 22, 2024
@WiseLin1125
Copy link
Author

The version is v0.13.2

@RyanHolstien RyanHolstien added the ingestion PR or Issue related to the ingestion of metadata label Nov 26, 2024
@WiseLin1125
Copy link
Author

@RyanHolstien do you have any other ideas for adjusting it in advance? Due to we are currently using it and we need the data lineage for Redshift.

@WiseLin1125
Copy link
Author

Discussed with the DataHub Team:
The workaround solution is to set:

is_serverless: True

This solution is yet to be confirmed and will be tested later.

@WiseLin1125
Copy link
Author

@treff7es Thanks for your solution. is_serverless can fix this issue. 👍

@WiseLin1125
Copy link
Author

WiseLin1125 commented Dec 11, 2024

It works with the
is_serverless
solution
https://datahubproject.io/docs/generated/ingestion/sources/redshift/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug report ingestion PR or Issue related to the ingestion of metadata
Projects
None yet
Development

No branches or pull requests

2 participants