Skip to content

[FEATURE] Add support for GitHub Models #1719

Closed
@jdubois

Description

GitHub Models is a new feature provided by GitHub, which can be found at https://github.com/marketplace/models

It allows to access a variety of models using a GitHub API key, and they are running behind the scenes on Azure OpenAI.

It makes it easy for all GitHub users to use those models, and to support them here is my proposal:

  • Create a new "langchain4j-github-models" Maven module
  • Have this module depend on "langchain4j-azure-open-ai": as it's the same system underneath, that should simplify a lot testing and dependency management
  • Implement a specific mechanism for authentication (it's just a bearer token), and use the endpoint "https://models.inference.ai.azure.com" -> this should be most of the work
  • Have a list of supported models to help people configure their model names

If all goes well, the resulting API would be something like:

GitHubModelsChatModel.builder()
                .aipKey(gitHubApiKey)
                .deploymentName(GitHubModelsCahtModelName.PHI_3_5_MINI_INSTRUCT)
                .logRequestsAndResponses(true)
                .build();

Once this is all done, I'll try to have this in the official documentation, when you click on Get Started at https://github.com/marketplace/models/azureml/Phi-3-5-mini-instruct as they have currently 0 Java documentation.

Metadata

Assignees

No one assigned

    Labels

    P2High priorityenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions