Is there an existing issue for this?
Current Behavior
Branches over the last 100th are not recorded
Expected Behavior
All branches are recorded
CloudQuery (redacted) config
kind: source
spec:
name: "github"
path: "cloudquery/github"
version: "v8.2.1"
# all tables: https://www.cloudquery.io/docs/plugins/sources/aws/tables
tables:
- github_repositories
- github_repository_branches
skip_tables:
- github_repository_dependabot_alerts
- github_releases
- github_repository_dependabot_secrets
- github_repository_sboms
- github_repository_keys
destinations: ["postgresql"]
spec:
access_token: "${GITHUB_PERSONAL_ACCESS_TOKEN}"
Steps To Reproduce
Run a sync with a repo that has over 100 branches.
CloudQuery (redacted) logs
No errors in log
CloudQuery version
5.11.0
Additional Context
Because Page is set to NextPage before checking if it’s equal to LastPage, LastPage will never be queried.
This issue appears to be present in most of the Github fetch functions.
|
opts.ListOptions.Page = resp.NextPage |
|
if opts.ListOptions.Page == resp.LastPage { |
|
break |
|
} |
Pull request (optional)
Is there an existing issue for this?
Current Behavior
Branches over the last 100th are not recorded
Expected Behavior
All branches are recorded
CloudQuery (redacted) config
Steps To Reproduce
Run a sync with a repo that has over 100 branches.
CloudQuery (redacted) logs
No errors in log
CloudQuery version
5.11.0
Additional Context
Because
Pageis set toNextPagebefore checking if it’s equal toLastPage,LastPagewill never be queried.This issue appears to be present in most of the Github fetch functions.
cloudquery/plugins/source/github/resources/services/repositories/branches.go
Lines 41 to 44 in 2947506
Pull request (optional)