Using authentication tokens in the requests via PyGithub looks easy, however it appears to me that kernelci doesn't allow access with personal access tokens or the "fine-grained" ones.

It looks it may need some actions from the organization owner. @gctucker: Can you help with this?

So what kind of tokens is required?

To summarize, right now we are using "anonymous" requests which are severely throttled, but we need to implement authenticated Github API calls.
We need to generate Application(under KernelCI organization), which will have it's own key (.pem file), and this key will be used on each run to generate temporary token for authenticated API calls. This will allow to use much higher quota for API calls.
We need to make sure .pem file stored securely, do not add it to plaintext git.

• Generate application and pem key for it, document procedure
• Generate unique token using this pem key and verify github calls are authenticated
• Create PR, where you are integrate token generation and use to deploy scripts, but make sure if pem file is invalid or doesnt exist it will fallback to using old, unauthenticated calls, like before
• PR merged

We could add the certificate to kernelci-project/secrets as an encrypted file.

Below is the detailed information to achieve authenticated Github API calls.

1. Create a Github App
   Follow the instructions from https://docs.github.com/en/apps/creating-github-apps/setting-up-a-github-app/creating-a-github-app to create Github App for the Organization.
   To summarize, navigate to below path from your Github account and create a new github app:

• Settings -> Your organizations -> Developer settings -> GitHub Apps -> New GitHub App
• Add GitHub App name and Description.
• Homepage URL can be linked to the KernelCI organization page.
• Navigate to Permissions and make sure to provide Contents permission. At the moment Read-only access will work as we use Github API just to checkout repositories.

2. Generate a private key
   Now, that we have created a github app, create a private key and store the .pem file securely.

3. Ways of authentication using an app
   Now the authentication can be done in multiple ways as described here https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/about-authentication-with-a-github-app
   The documentation states that "Authenticating as an app installation lets your app access resources that are owned by the user or organization that installed the app" which fits our case. Hence, we should go for "Authentication as an app installation".

4. Install the Github App
   In the case of a personal Github account, the app can be installed by navigating to the below path (It should be the same for organization as well).
   Settings -> Developer settings -> Github Apps -> Install App
   The app can be installed by providing "All repositories" or "Only select repositories" access depending on our use case.

5. Generate a JWT
   Now, generate a JWT using the .pem file.
   The python script https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-json-web-token-jwt-for-a-github-app#example-using-python-to-generate-a-jwt can be used for it. The location of .pem file and App ID will need to be provided to the script. The App ID can be fetched from Settings -> Developer Settings -> Github Apps -> Name of the app. "App ID" should be visible on the page.
   Note that the token will be valid for 10 min.

6. Get the installation ID
   We can use app/installations endpoint to get the installation ID that we want to authenticate as. Please check out the details here. https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/authenticating-as-a-github-app-installation#generating-an-installation-access-token
   The sample curl command is as below. Please replace JWT with the actual token.
   The response will contain the installation ID as "id" field in the dictionary.

curl -i -H "Authorization: Bearer JWT" -H "Accept: application/vnd.github.v3+json" https://api.github.com/app/installations

7. Generate installation access token using JWT
   Use the below command to generate an installation access token. Make sure to replace JWT and INSTALLATION_ID with actual values. Note that the access token will be valid for 1 hr.

curl -i -X POST -H "Authorization: Bearer JWT" -H "Accept: application/vnd.github.v3+json" https://api.github.com/app/installations/INSTALLATION_ID/access_tokens

8. Now the access token can be used in kernelci-deploy/kernelci/__init__.py to get the Github API handler.

github.Github(installation_access_token)

I have created a test app https://github.com/apps/test-github-app-kernelci and verified the above procedure.

Below are some commands that worked for the test app:

• Get app installation ID:

$ curl -i -H "Authorization: Bearer JWT" -H "Accept: application/vnd.github.v3+json" https://api.github.com/app/installations
[
  {
    "id": 36870146,
    "account": {
      "login": "JenySadadia",
      "id": 11505157,
      "node_id": "MDQ6VXNlcjExNTA1MTU3",
      "avatar_url": "https://avatars.githubusercontent.com/u/11505157?v=4",
      "gravatar_id": "",
      "url": "https://api.github.com/users/JenySadadia",
      "html_url": "https://github.com/JenySadadia",
      "followers_url": "https://api.github.com/users/JenySadadia/followers",
      "following_url": "https://api.github.com/users/JenySadadia/following{/other_user}",
      "gists_url": "https://api.github.com/users/JenySadadia/gists{/gist_id}",
      "starred_url": "https://api.github.com/users/JenySadadia/starred{/owner}{/repo}",
      "subscriptions_url": "https://api.github.com/users/JenySadadia/subscriptions",
      "organizations_url": "https://api.github.com/users/JenySadadia/orgs",
      "repos_url": "https://api.github.com/users/JenySadadia/repos",
      "events_url": "https://api.github.com/users/JenySadadia/events{/privacy}",
      "received_events_url": "https://api.github.com/users/JenySadadia/received_events",
      "type": "User",
      "site_admin": false
    },
    "repository_selection": "selected",
    "access_tokens_url": "https://api.github.com/app/installations/36870146/access_tokens",
    "repositories_url": "https://api.github.com/installation/repositories",
    "html_url": "https://github.com/settings/installations/36870146",
    "app_id": 324278,
    "app_slug": "test-github-app-kernelci",
    "target_id": 11505157,
    "target_type": "User",
    "permissions": {
      "contents": "write",
      "metadata": "read"
    },
    "events": [

    ],
    "created_at": "2023-04-26T12:25:58.000Z",
    "updated_at": "2023-04-26T13:24:39.000Z",
    "single_file_name": null,
    "has_multiple_single_files": false,
    "single_file_paths": [

    ],
    "suspended_by": null,
    "suspended_at": null
  }
]

• Get installation access token:

$ curl -i -X POST -H "Authorization: Bearer JWT" -H "Accept: application/vnd.github.v3+json" https://api.github.com/app/installations/36870146/access_tokens
HTTP/2 201 
server: GitHub.com
date: Thu, 27 Apr 2023 09:45:41 GMT
content-type: application/json; charset=utf-8
content-length: 208
cache-control: public, max-age=60, s-maxage=60
vary: Accept
etag: "d442da0bb374c1a42822b56a10fb7c89b8138baaad77c40c7d7d5aa9b99e9e95"
x-github-media-type: github.v3; format=json
x-github-api-version-selected: 2022-11-28
access-control-expose-headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
access-control-allow-origin: *
strict-transport-security: max-age=31536000; includeSubdomains; preload
x-frame-options: deny
x-content-type-options: nosniff
x-xss-protection: 0
referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin
content-security-policy: default-src 'none'
vary: Accept-Encoding, Accept, X-Requested-With
x-github-request-id: C562:7BC7:1C4740:1F2255:644A4445

{
  "token": "ghs_*",
  "expires_at": "2023-04-27T10:45:41Z",
  "permissions": {
    "contents": "write",
    "metadata": "read"
  },
  "repository_selection": "selected"
}

X-Rate-Limit-Limit for authenticated calls is 5000 requests per hour whereas for unauthenticated calls is 60 requests per hour.
This can be verified by using github.enable_console_debug_logging(), a method provided by PyGithub.

@mgalka Could you please help to create a github app for KernelCI organization as I am not authorized for it?