Hi @froyo-saif, what Backstage version are you using and what's the error message?

Hello @camilaibs,
This is the error and I have a doubt like Backstage requires catalog-info.yaml to fetch groups too or how it is done?

[1] 2024-12-06T06:27:04.541Z catalog error GitlabOrgDiscoveryEntityProvider:yourProviderId refresh failed, Error: GraphQL errors: [{"message":"Unexpected end of document","locations":[]}] GraphQL errors: [{"message":"Unexpected end of document","locations":[]}] target=GitlabOrgDiscoveryEntityProvider:yourProviderId class=GitlabOrgDiscoveryEntityProvider taskId=GitlabOrgDiscoveryEntityProvider:yourProviderId:refresh taskInstanceId=a9215320-1a21-4613-b1fb-d58e8b63f29c stack=Error: GraphQL errors: [{"message":"Unexpected end of document","locations":[]}]
[1]     at GitLabClient.listDescendantGroups (C:\Users\SKH9QP\Downloads\newdevportal\scania-dev-portal\node_modules\@backstage\plugin-catalog-backend-module-gitlab\src\lib\client.ts:213:15)
[1]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[1]     at GitlabOrgDiscoveryEntityProvider.refresh (C:\Users\SKH9QP\Downloads\newdevportal\scania-dev-portal\node_modules\@backstage\plugin-catalog-backend-module-gitlab\src\providers\GitlabOrgDiscoveryEntityProvider.ts:367:17)
[1]     at Object.fn (C:\Users\SKH9QP\Downloads\newdevportal\scania-dev-portal\node_modules\@backstage\plugin-catalog-backend-module-gitlab\src\providers\GitlabOrgDiscoveryEntityProvider.ts:342:19)        
[1]     at <anonymous> (C:\Users\SKH9QP\Downloads\newdevportal\scania-dev-portal\node_modules\@backstage\backend-defaults\src\entrypoints\scheduler\lib\PluginTaskSchedulerImpl.ts:154:19)
[1]     at TaskWorker.fn (C:\Users\SKH9QP\Downloads\newdevportal\scania-dev-portal\node_modules\@backstage\backend-defaults\src\entrypoints\scheduler\lib\PluginTaskSchedulerImpl.ts:151:9)
[1]     at TaskWorker.runOnce (C:\Users\SKH9QP\Downloads\newdevportal\scania-dev-portal\node_modules\@backstage\backend-defaults\src\entrypoints\scheduler\lib\TaskWorker.ts:156:13)
[1]     at <anonymous> (C:\Users\SKH9QP\Downloads\newdevportal\scania-dev-portal\node_modules\@backstage\backend-defaults\src\entrypoints\scheduler\lib\TaskWorker.ts:76:31)

Hi @froyo-saif, this is a community-maintained module, let me loop in one of the module authors (@elaine-mattos ) to see if she can help you 🙂 .

Hey @camilaibs thank you for the support

Hi @froyo-saif ,

I'll take a look at it later today. Meanwhile, could you please provide the configuration in your app-config .yaml file?

Hello @elaine-mattos
Here is app-config.yaml file

`app:
title: Developer Portal
baseUrl: http://localhost:3000

organization:
name: Local

backend:
baseUrl: http://localhost:7007
listen:
port: 7007
csp:
connect-src: ["'self'", 'http:', 'https:']
cors:
origin: http://localhost:3000
methods: [GET, HEAD, PATCH, POST, PUT, DELETE]
credentials: true
database:
client: better-sqlite3
connection: ':memory:'

integrations:
github:
- host: github.com
token: token

gitlab:
- host: gitlab.example.com
baseUrl: http://gitlab.example.com
apiBaseUrl: http://gitlab.example.com/api/v4
# If the repo is private, add a token as well
token: token

proxy:

techdocs:
builder: 'local'
generator:
runIn: 'local'
publisher:
type: 'local'

auth:
providers:
guest: {}

scaffolder:

catalog:
import:
entityFilename: catalog-info.yaml
pullRequestBranchName: backstage-integration
rules:
- allow: [Component, System, API, Group, Resource, Location, Template]
providers:
gitlab:
yourProviderId:
host: gitlab.example.com # GitLab Personal Instance
orgEnabled: true
#branch: main # Optional: The branch to scan, defaults to 'main'
#fallbackBranch: master # Optional: Fallback to master if no default branch
#skipForkedRepos: false # Optional: Whether to skip forked repositories (false means include forks)
group: devtools
relations:
- INHERITED # Include members of parent groups in subgroups
#- DESCENDANTS # Include members of subgroups in parent groups
- SHARED_FROM_GROUPS
# Optional: Group or subgroup within GitLab
#restrictUsersToGroup: true
entityFilename: catalog-info.yaml # Optional: Default is 'catalog-info.yaml'
#projectPattern: '[\s\S]*' # Optional: Filter for specific projects (e.g., 'myproject')
#excludeRepos: [] # Optional: List of repositories to exclude
schedule: # Schedule for periodic discovery
frequency: { minutes: 10 } # How often the provider scans GitLab (every 30 minutes)
timeout: { minutes: 3 } # Timeout for each scan

locations:
# Existing locations
- type: url
target: https://github.com/froyo-saif/backstage-experiments/blob/main/catalog-info.yaml
- type: url
target: https://github.com/froyo-saif/scania-devtools/blob/main/catalog-info.yaml
- type: url
target: https://github.com/froyo-saif/backstage-cloud/blob/main/catalog-info.yaml
- type: file
target: template.yaml
rules:
- allow: [Template]
- type: file
target: ../../examples/template/template.yaml
rules:
- allow: [Template]
- type: file
target: template2.yaml
rules:
- allow: [Template]
- type: file
target: template3.yaml
rules:
- allow: [Template]
- type: file
target: streetlights-api.yaml
rules:
- allow: [API]
- type: file
target: newapi.yaml
rules:
- allow: [API]
- type: file
target: ../app/src/assets/githubrepos-api.yaml
rules:
- allow: [API]

logging:
level: debug

`

And there is one more issue regarding groups
This is the error

[1] 2024-12-10T06:44:08.248Z catalog info Scanned 3413 groups and processed 0 groups target=GitlabOrgDiscoveryEntityProvider:yourProviderId class=GitlabOrgDiscoveryEntityProvider taskId=GitlabOrgDiscoveryEntityProvider:yourProviderId:refresh taskInstanceId=f26a2bee-f116-487c-9b63-9e1ab2b5ba00

Hi @froyo-saif ,

To answer your question, no the plugin doesn't require a catalog-info.yaml to ingest the groups, just the config in your app-config file.
Unfortunately this is the only answer I have for you right now. I do have some questions tho...

1. You mentioned you are using a self-hosted instance right? Does your GitLab token have the read_api scope setup?
2. Does the group have any members?

The error you see comes from the GitLab Group.groupMembers GraphQL query. You can see the query's documentation here: https://docs.gitlab.com/ee/api/graphql/reference/#groupgroupmembers.
You can also try an run this query using the playground that can be found in your local instance under the link https://your-gitlab-instance/-/graphql-explorer
This is the query performed by the plugin:

{
  group(fullPath: "devtools") {
    groupMembers(
      first: 100
      relations: [DIRECT, INHERITED,  SHARED_FROM_GROUPS]
    ) {
      nodes {
        user {
          id
          username
          publicEmail
          name
          state
          webUrl
          avatarUrl
        }
      }
      pageInfo {
        endCursor
        hasNextPage
      }
    }
  }
}

Could you give it a try and let us know if you see the users being returned?

I tried to run the query and got this as output as you said
{
"data": {
"group": {
"groupMembers": {
"nodes": [
{
"user": {
"id": "gid://gitlab/User/14038",
"username": "ARUJU5",
"publicEmail": "",
"name": "Adam Ruth",
"state": "active",
"webUrl": "https://gitlab.example.com/ARUJU5",
"avatarUrl": "https://secure.gravatar.com/avatar/8b0b25bd0a15a15583be241c8627bc17f359cce03801b43f3f6352eb6cf7a0ea?s=80&d=identicon"
}
},
{
"user": {
"id": "gid://gitlab/User/8967",
"username": "AJADO9",
"publicEmail": "",
"name": "Andreas Jansson",
"state": "active",
"webUrl": "https://gitlab.example.com/AJADO9",
"avatarUrl": "/uploads/-/system/user/avatar/8967/avatar.png"
}
},
{
"user": {
"id": "gid://gitlab/User/13200",
"username": "SKH9QP",
"publicEmail": null,
"name": "Saif Khan",
"state": "active",
"webUrl": "https://gitlab.example.com/SKH9QP",
"avatarUrl": "https://secure.gravatar.com/avatar/04dc02173e362695529b97f66363eb2ee94a95ec287b51927aa05b641cfd4831?s=80&d=identicon"
}
},
{
"user": {
"id": "gid://gitlab/User/8165",
"username": "pshh8y",
"publicEmail": "",
"name": "Punit",
"state": "blocked",
"webUrl": "https://gitlab.example.com/pshh8y",
"avatarUrl": "https://secure.gravatar.com/avatar/f9e833b04372cf5e0d94d525a0dd107f53606762f5e89bdda810da793d81e8db?s=80&d=identicon"
}
},
{
"user": {
"id": "gid://gitlab/User/9418",
"username": "AKAQRO",
"publicEmail": null,
"name": "guest3",
"state": "blocked",
"webUrl": "https://gitlab.example.com/SSSS12",
"avatarUrl": "https://secure.gravatar.com/avatar/f74446c196224c90e45ee5407c765b4d0b056dfa34a2032b3b8267860351341b?s=80&d=identicon"
}
},
{
"user": {
"id": "gid://gitlab/User/9158",
"username": "DSU3SO",
"publicEmail": null,
"name": "guest2",
"state": "deactivated",
"webUrl": "https://gitlab.example.com/DSU3SO",
"avatarUrl": "https://secure.gravatar.com/avatar/9d57ef92b6a5992ff5f059b4a8e56e08047eab182e230882e44f3502de415f0d?s=80&d=identicon"
}
},
{
"user": {
"id": "gid://gitlab/User/251",
"username": "mskvb0",
"publicEmail": "[email protected]",
"name": "guest",
"state": "active",
"webUrl": "https://gitlab.example.com/mskvb0",
"avatarUrl": "/uploads/-/system/user/avatar/25/avatar.png"
}
}
],
"pageInfo": {
"endCursor": "eyJpZCI6IjQwMjcifQ",
"hasNextPage": false
}
}
}
}
}

Added descendants to relations too and got the users with blocked and deactivated
Is the error because of blocked or deactivated?
If it is because of it, how can we change it

@elaine-mattos

Hi @froyo-saif ,

thanks! Could you please try running this one and see if you get the descendant groups back?

{
  group(fullPath: "devtools") {
    descendantGroups(first: 100) {
      nodes {
        id
        name
        description
        fullPath
        visibility
        parent {
          id
        }
      }
      pageInfo {
        endCursor
        hasNextPage
      }
    }
  }
}

Now I dont get any users as earlier it was but now groups are visible
Like this:
{
"id": "gid://gitlab/Group/51003",
"name": "Saif-Labs",
"description": "",
"fullPath": "devtools/labs/saif-labs",
"visibility": "private",
"parent": {
"id": "gid://gitlab/Group/3223"
}
So, for the users that are blocked or deactivated or some mails were null so that can be the probable errors behind this ?
And if the errors persists with that blocked or deactivated users how should I proceed and where should I change
and I found this https://gitlab.com/gitlab-org/gitlab/-/merge_requests/133398
@elaine-mattos

Any update @elaine-mattos