add stepfun models #5283
add stepfun models #5283
Conversation
|
@forrestlinfeng is attempting to deploy a commit to the NextChat Team on Vercel. A member of the Team first needs to authorize it. |
WalkthroughThe recent updates enhance the application by integrating support for a new service provider, Stepfun. This includes routing API requests, authentication, and client interactions specifically tailored for Stepfun. New components for configuration and localization have been added, while existing structures were modified to accommodate the additional functionality seamlessly. Overall, these changes enrich user experience and expand the application’s capabilities. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant APIHandler
participant Auth
participant StepfunAPI
Client->>APIHandler: Request to Stepfun
APIHandler->>Auth: Authenticate request
Auth-->>APIHandler: Return auth status
alt Auth success
APIHandler->>StepfunAPI: Forward request
StepfunAPI-->>APIHandler: Return response
APIHandler-->>Client: Send response
else Auth failure
APIHandler-->>Client: Return 401 Unauthorized
end
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Outside diff range, codebase verification and nitpick comments (2)
app/store/access.ts (1)
46-48: Add a comment forDEFAULT_STEPFUN_URL.Consider adding a comment to explain the purpose of
DEFAULT_STEPFUN_URLfor better maintainability and clarity.+// Default URL for Stepfun service based on the build mode const DEFAULT_STEPFUN_URL = isApp ? DEFAULT_API_HOST + "/api/proxy/stepfun" : ApiPath.Stepfun;app/config/server.ts (1)
65-67: Document the new environment variables.Consider documenting the
STEPFUN_URLandSTEPFUN_API_KEYenvironment variables to ensure they are correctly set and used in different environments.+// Environment variables for Stepfun service STEPFUN_URL?: string; STEPFUN_API_KEY?: string;
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (11)
- app/api/[provider]/[...path]/route.ts (2 hunks)
- app/api/auth.ts (1 hunks)
- app/api/stepfun.ts (1 hunks)
- app/client/api.ts (5 hunks)
- app/client/platforms/stepfun.ts (1 hunks)
- app/components/settings.tsx (3 hunks)
- app/config/server.ts (3 hunks)
- app/constant.ts (9 hunks)
- app/locales/cn.ts (1 hunks)
- app/locales/en.ts (1 hunks)
- app/store/access.ts (4 hunks)
Additional comments not posted (25)
app/api/[provider]/[...path]/route.ts (2)
34-35: Check routing logic for Stepfun API.The case for
ApiPath.Stepfunhas been added to the switch statement. Ensure that this logic correctly routes requests to the Stepfun API.
10-10: Ensure thestepfunHandleris correctly implemented.The import statement for
stepfunHandlerhas been added. Verify that thestepfunHandleris correctly implemented in thestepfunmodule and adheres to the expected interface.Verification successful
The
stepfunHandleris correctly implemented.The
handlefunction in thestepfunmodule is implemented with the expected functionality. It handles requests appropriately, including authentication and error handling. No issues were found with its implementation.
- Location:
app/api/stepfun.ts(lines 17-41)Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the implementation of `stepfunHandler`. # Test: Search for the `stepfunHandler` function implementation. Expect: A valid function definition. ast-grep --lang typescript --pattern $'function stepfunHandler($_, $_) { $$$ }'Length of output: 80
Script:
#!/bin/bash # Search for the `handle` function in the `stepfun` module to verify its implementation. ast-grep --lang typescript --pattern $'function handle($_, $_) { $$$ }'Length of output: 39852
app/api/auth.ts (1)
88-90: EnsurestepfunApiKeyis configured correctly.The
ModelProvider.Stepfuncase assignssystemApiKeytoserverConfig.stepfunApiKey. Verify thatstepfunApiKeyis correctly configured in the server configuration.app/api/stepfun.ts (2)
17-41: Review the Stepfun API request handling logic.The
handlefunction manages the request flow, including authentication and error handling. Ensure that the logic is robust and correctly handles different HTTP methods and errors.
43-131: Check the request function for potential improvements.The
requestfunction handles the proxying of requests to the Stepfun API. Consider the following:
- Timeout Handling: The timeout is set to 10 minutes. Verify if this duration is appropriate for the expected use case.
- Error Handling: Ensure that errors are logged and handled appropriately.
- Headers Management: The function modifies headers, such as removing
www-authenticate. Verify that these changes align with the intended behavior.app/store/access.ts (3)
175-177: LGTM! EnsureisValidStepfunis correctly integrated.The
isValidStepfunfunction looks good. Ensure it is correctly integrated into the overall validation logic.
199-200: EnsureisValidStepfunis correctly invoked.The
isValidStepfuncheck is added to theisAuthorizedfunction. Verify that it is correctly invoked in all necessary contexts.
114-116: EnsurestepfunApiKeyis securely handled.The
stepfunApiKeyis initialized as an empty string. Ensure that it is securely handled and not logged or exposed inappropriately.Verification successful
stepfunApiKeyis securely handled.The
stepfunApiKeydoes not appear to be logged or exposed through network requests or other logging mechanisms in the codebase. No occurrences were found indicating insecure handling.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check for any logging or exposure of `stepfunApiKey`. # Test: Search for any console logs or exposures of `stepfunApiKey`. Expect: No occurrences. rg --type javascript 'console\.log\(.*stepfunApiKey.*\)'Length of output: 93
Script:
#!/bin/bash # Description: Check for any logging or exposure of `stepfunApiKey`. # Test: Search for any console logs or exposures of `stepfunApiKey`. Expect: No occurrences. rg 'console\.log\(.*stepfunApiKey.*\)' --glob '*.js'Length of output: 54
Script:
#!/bin/bash # Description: Check for any network requests or other logging mechanisms involving `stepfunApiKey`. # Test: Search for any network requests containing `stepfunApiKey`. Expect: No occurrences. rg 'fetch\(.*stepfunApiKey.*\)' --glob '*.js' # Test: Search for any other logging methods containing `stepfunApiKey`. Expect: No occurrences. rg 'logger\.log\(.*stepfunApiKey.*\)' --glob '*.js'Length of output: 101
app/config/server.ts (1)
204-206: LGTM! Ensure server-side configuration is secure.The server-side configuration for Stepfun looks good. Ensure that sensitive information, such as API keys, is handled securely.
app/client/platforms/stepfun.ts (1)
248-250: Implement model listing logic.The
modelsmethod currently returns an empty array. Implement logic to list available models from the Stepfun service.Verification successful
Implement model listing logic.
The
modelsmethod currently returns an empty array, and there is no implementation found in the codebase. Please implement the logic to list available models from the Stepfun service as suggested in the review comment.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check if model listing logic is implemented. # Test: Search for implementation of model listing logic. Expect: Implementation details. rg --type javascript 'StepfunApi.*models'Length of output: 78
Script:
#!/bin/bash # Description: Check if model listing logic is implemented. # Test: Search for implementation of model listing logic in JavaScript files. rg --type js 'StepfunApi.*models'Length of output: 35
Script:
#!/bin/bash # Description: Check for any implementation of the models method in JavaScript files. # Test: Search for any models method implementation. rg --type js 'models'Length of output: 21
app/client/api.ts (4)
16-16: Import statement for StepfunApi looks good.The import is necessary for integrating the Stepfun API into the
ClientApiclass.
132-134: Integration of Stepfun in ClientApi constructor is correct.The addition of
ModelProvider.Stepfuncase correctly initializes theStepfunApi.
Line range hint
223-238: Header configuration for Stepfun is correctly implemented.The addition of
isStepfunand the branch forstepfunApiKeyensures proper header setup for Stepfun.
306-307: Stefun integration in getClientApi is correctly implemented.The addition of
ServiceProvider.Stepfuncase correctly returns a newClientApiinstance for Stepfun.app/constant.ts (7)
28-28: STEPFUN_BASE_URL constant addition is correct.The constant is necessary for interacting with the Stepfun API.
59-59: ApiPath enum update for Stepfun is correct.The addition of
Stepfunensures integration into the existing API path structure.
116-116: ServiceProvider enum update for Stepfun is correct.The addition of
Stepfunensures it is recognized as a service provider.
140-140: ModelProvider enum update for Stepfun is correct.The addition of
Stepfunensures it is recognized as a model provider.
214-217: Stepfun constant addition is correct.The constant provides structured access to Stepfun's API endpoints.
348-357: stepfunModels array addition is correct.The array facilitates the management and access of Stepfun models within the application.
459-469: Inclusion of stepfunModels in DEFAULT_MODELS is correct.This inclusion ensures that Stepfun models are available alongside other existing models.
app/locales/cn.ts (1)
397-406: Localization entries for Stepfun are correctly added.These entries enhance localization support for Chinese-speaking users interacting with the Stepfun API.
app/locales/en.ts (1)
381-390: Localization entries for Stepfun look good!The new entries for Stepfun's API key and endpoint are consistent with the existing localization structure and provide clear guidance for users.
app/components/settings.tsx (2)
1047-1086: Stepfun configuration component is well-integrated!The Stepfun configuration component follows the established pattern for other providers, ensuring consistency in user experience. Input fields are appropriately labeled and connected to the access store.
1624-1624: Stepfun component integration is correct!The Stepfun configuration component is correctly integrated into the settings, ensuring it is displayed only when Stepfun is the selected provider.
| async chat(options: ChatOptions) { | ||
| const messages: ChatOptions["messages"] = []; | ||
| for (const v of options.messages) { | ||
| const content = getMessageTextContent(v); | ||
| messages.push({ role: v.role, content }); | ||
| } | ||
|
|
||
| const modelConfig = { | ||
| ...useAppConfig.getState().modelConfig, | ||
| ...useChatStore.getState().currentSession().mask.modelConfig, | ||
| ...{ | ||
| model: options.config.model, | ||
| providerName: options.config.providerName, | ||
| }, | ||
| }; | ||
|
|
||
| const requestPayload: RequestPayload = { | ||
| messages, | ||
| stream: options.config.stream, | ||
| model: modelConfig.model, | ||
| temperature: modelConfig.temperature, | ||
| presence_penalty: modelConfig.presence_penalty, | ||
| frequency_penalty: modelConfig.frequency_penalty, | ||
| top_p: modelConfig.top_p, | ||
| // max_tokens: Math.max(modelConfig.max_tokens, 1024), | ||
| // Please do not ask me why not send max_tokens, no reason, this param is just shit, I dont want to explain anymore. | ||
| }; | ||
|
|
||
| console.log("[Request] openai payload: ", requestPayload); | ||
|
|
||
| const shouldStream = !!options.config.stream; | ||
| const controller = new AbortController(); | ||
| options.onController?.(controller); | ||
|
|
||
| try { | ||
| const chatPath = this.path(Stepfun.ChatPath); | ||
| const chatPayload = { | ||
| method: "POST", | ||
| body: JSON.stringify(requestPayload), | ||
| signal: controller.signal, | ||
| headers: getHeaders(), | ||
| }; | ||
|
|
||
| // make a fetch request | ||
| const requestTimeoutId = setTimeout( | ||
| () => controller.abort(), | ||
| REQUEST_TIMEOUT_MS, | ||
| ); | ||
|
|
||
| if (shouldStream) { | ||
| let responseText = ""; | ||
| let remainText = ""; | ||
| let finished = false; | ||
|
|
||
| // animate response to make it looks smooth | ||
| function animateResponseText() { | ||
| if (finished || controller.signal.aborted) { | ||
| responseText += remainText; | ||
| console.log("[Response Animation] finished"); | ||
| if (responseText?.length === 0) { | ||
| options.onError?.(new Error("empty response from server")); | ||
| } | ||
| return; | ||
| } | ||
|
|
||
| if (remainText.length > 0) { | ||
| const fetchCount = Math.max(1, Math.round(remainText.length / 60)); | ||
| const fetchText = remainText.slice(0, fetchCount); | ||
| responseText += fetchText; | ||
| remainText = remainText.slice(fetchCount); | ||
| options.onUpdate?.(responseText, fetchText); | ||
| } | ||
|
|
||
| requestAnimationFrame(animateResponseText); | ||
| } | ||
|
|
||
| // start animaion | ||
| animateResponseText(); | ||
|
|
||
| const finish = () => { | ||
| if (!finished) { | ||
| finished = true; | ||
| options.onFinish(responseText + remainText); | ||
| } | ||
| }; | ||
|
|
||
| controller.signal.onabort = finish; | ||
|
|
||
| fetchEventSource(chatPath, { | ||
| ...chatPayload, | ||
| async onopen(res) { | ||
| clearTimeout(requestTimeoutId); | ||
| const contentType = res.headers.get("content-type"); | ||
| console.log( | ||
| "[OpenAI] request response content type: ", | ||
| contentType, | ||
| ); | ||
|
|
||
| if (contentType?.startsWith("text/plain")) { | ||
| responseText = await res.clone().text(); | ||
| return finish(); | ||
| } | ||
|
|
||
| if ( | ||
| !res.ok || | ||
| !res.headers | ||
| .get("content-type") | ||
| ?.startsWith(EventStreamContentType) || | ||
| res.status !== 200 | ||
| ) { | ||
| const responseTexts = [responseText]; | ||
| let extraInfo = await res.clone().text(); | ||
| try { | ||
| const resJson = await res.clone().json(); | ||
| extraInfo = prettyObject(resJson); | ||
| } catch {} | ||
|
|
||
| if (res.status === 401) { | ||
| responseTexts.push(Locale.Error.Unauthorized); | ||
| } | ||
|
|
||
| if (extraInfo) { | ||
| responseTexts.push(extraInfo); | ||
| } | ||
|
|
||
| responseText = responseTexts.join("\n\n"); | ||
|
|
||
| return finish(); | ||
| } | ||
| }, | ||
| onmessage(msg) { | ||
| if (msg.data === "[DONE]" || finished) { | ||
| return finish(); | ||
| } | ||
| const text = msg.data; | ||
| try { | ||
| const json = JSON.parse(text); | ||
| const choices = json.choices as Array<{ | ||
| delta: { content: string }; | ||
| }>; | ||
| const delta = choices[0]?.delta?.content; | ||
| const textmoderation = json?.prompt_filter_results; | ||
|
|
||
| if (delta) { | ||
| remainText += delta; | ||
| } | ||
| } catch (e) { | ||
| console.error("[Request] parse error", text, msg); | ||
| } | ||
| }, | ||
| onclose() { | ||
| finish(); | ||
| }, | ||
| onerror(e) { | ||
| options.onError?.(e); | ||
| throw e; | ||
| }, | ||
| openWhenHidden: true, | ||
| }); | ||
| } else { | ||
| const res = await fetch(chatPath, chatPayload); | ||
| clearTimeout(requestTimeoutId); | ||
|
|
||
| const resJson = await res.json(); | ||
| const message = this.extractMessage(resJson); | ||
| options.onFinish(message); | ||
| } | ||
| } catch (e) { | ||
| console.log("[Request] failed to make a chat request", e); | ||
| options.onError?.(e as Error); | ||
| } |
There was a problem hiding this comment.
Enhance error handling in chat method.
The chat method could benefit from more robust error handling, especially for network errors and response parsing.
try {
const chatPath = this.path(Stepfun.ChatPath);
const chatPayload = {
method: "POST",
body: JSON.stringify(requestPayload),
signal: controller.signal,
headers: getHeaders(),
};
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
REQUEST_TIMEOUT_MS,
);
if (shouldStream) {
// Streaming logic
} else {
const res = await fetch(chatPath, chatPayload);
clearTimeout(requestTimeoutId);
if (!res.ok) {
throw new Error(`Request failed with status ${res.status}`);
}
const resJson = await res.json();
const message = this.extractMessage(resJson);
options.onFinish(message);
}
} catch (e) {
console.error("[Request] failed to make a chat request", e);
options.onError?.(e as Error);
}Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| async chat(options: ChatOptions) { | |
| const messages: ChatOptions["messages"] = []; | |
| for (const v of options.messages) { | |
| const content = getMessageTextContent(v); | |
| messages.push({ role: v.role, content }); | |
| } | |
| const modelConfig = { | |
| ...useAppConfig.getState().modelConfig, | |
| ...useChatStore.getState().currentSession().mask.modelConfig, | |
| ...{ | |
| model: options.config.model, | |
| providerName: options.config.providerName, | |
| }, | |
| }; | |
| const requestPayload: RequestPayload = { | |
| messages, | |
| stream: options.config.stream, | |
| model: modelConfig.model, | |
| temperature: modelConfig.temperature, | |
| presence_penalty: modelConfig.presence_penalty, | |
| frequency_penalty: modelConfig.frequency_penalty, | |
| top_p: modelConfig.top_p, | |
| // max_tokens: Math.max(modelConfig.max_tokens, 1024), | |
| // Please do not ask me why not send max_tokens, no reason, this param is just shit, I dont want to explain anymore. | |
| }; | |
| console.log("[Request] openai payload: ", requestPayload); | |
| const shouldStream = !!options.config.stream; | |
| const controller = new AbortController(); | |
| options.onController?.(controller); | |
| try { | |
| const chatPath = this.path(Stepfun.ChatPath); | |
| const chatPayload = { | |
| method: "POST", | |
| body: JSON.stringify(requestPayload), | |
| signal: controller.signal, | |
| headers: getHeaders(), | |
| }; | |
| // make a fetch request | |
| const requestTimeoutId = setTimeout( | |
| () => controller.abort(), | |
| REQUEST_TIMEOUT_MS, | |
| ); | |
| if (shouldStream) { | |
| let responseText = ""; | |
| let remainText = ""; | |
| let finished = false; | |
| // animate response to make it looks smooth | |
| function animateResponseText() { | |
| if (finished || controller.signal.aborted) { | |
| responseText += remainText; | |
| console.log("[Response Animation] finished"); | |
| if (responseText?.length === 0) { | |
| options.onError?.(new Error("empty response from server")); | |
| } | |
| return; | |
| } | |
| if (remainText.length > 0) { | |
| const fetchCount = Math.max(1, Math.round(remainText.length / 60)); | |
| const fetchText = remainText.slice(0, fetchCount); | |
| responseText += fetchText; | |
| remainText = remainText.slice(fetchCount); | |
| options.onUpdate?.(responseText, fetchText); | |
| } | |
| requestAnimationFrame(animateResponseText); | |
| } | |
| // start animaion | |
| animateResponseText(); | |
| const finish = () => { | |
| if (!finished) { | |
| finished = true; | |
| options.onFinish(responseText + remainText); | |
| } | |
| }; | |
| controller.signal.onabort = finish; | |
| fetchEventSource(chatPath, { | |
| ...chatPayload, | |
| async onopen(res) { | |
| clearTimeout(requestTimeoutId); | |
| const contentType = res.headers.get("content-type"); | |
| console.log( | |
| "[OpenAI] request response content type: ", | |
| contentType, | |
| ); | |
| if (contentType?.startsWith("text/plain")) { | |
| responseText = await res.clone().text(); | |
| return finish(); | |
| } | |
| if ( | |
| !res.ok || | |
| !res.headers | |
| .get("content-type") | |
| ?.startsWith(EventStreamContentType) || | |
| res.status !== 200 | |
| ) { | |
| const responseTexts = [responseText]; | |
| let extraInfo = await res.clone().text(); | |
| try { | |
| const resJson = await res.clone().json(); | |
| extraInfo = prettyObject(resJson); | |
| } catch {} | |
| if (res.status === 401) { | |
| responseTexts.push(Locale.Error.Unauthorized); | |
| } | |
| if (extraInfo) { | |
| responseTexts.push(extraInfo); | |
| } | |
| responseText = responseTexts.join("\n\n"); | |
| return finish(); | |
| } | |
| }, | |
| onmessage(msg) { | |
| if (msg.data === "[DONE]" || finished) { | |
| return finish(); | |
| } | |
| const text = msg.data; | |
| try { | |
| const json = JSON.parse(text); | |
| const choices = json.choices as Array<{ | |
| delta: { content: string }; | |
| }>; | |
| const delta = choices[0]?.delta?.content; | |
| const textmoderation = json?.prompt_filter_results; | |
| if (delta) { | |
| remainText += delta; | |
| } | |
| } catch (e) { | |
| console.error("[Request] parse error", text, msg); | |
| } | |
| }, | |
| onclose() { | |
| finish(); | |
| }, | |
| onerror(e) { | |
| options.onError?.(e); | |
| throw e; | |
| }, | |
| openWhenHidden: true, | |
| }); | |
| } else { | |
| const res = await fetch(chatPath, chatPayload); | |
| clearTimeout(requestTimeoutId); | |
| const resJson = await res.json(); | |
| const message = this.extractMessage(resJson); | |
| options.onFinish(message); | |
| } | |
| } catch (e) { | |
| console.log("[Request] failed to make a chat request", e); | |
| options.onError?.(e as Error); | |
| } | |
| try { | |
| const chatPath = this.path(Stepfun.ChatPath); | |
| const chatPayload = { | |
| method: "POST", | |
| body: JSON.stringify(requestPayload), | |
| signal: controller.signal, | |
| headers: getHeaders(), | |
| }; | |
| // make a fetch request | |
| const requestTimeoutId = setTimeout( | |
| () => controller.abort(), | |
| REQUEST_TIMEOUT_MS, | |
| ); | |
| if (shouldStream) { | |
| // Streaming logic | |
| } else { | |
| const res = await fetch(chatPath, chatPayload); | |
| clearTimeout(requestTimeoutId); | |
| if (!res.ok) { | |
| throw new Error(`Request failed with status ${res.status}`); | |
| } | |
| const resJson = await res.json(); | |
| const message = this.extractMessage(resJson); | |
| options.onFinish(message); | |
| } | |
| } catch (e) { | |
| console.error("[Request] failed to make a chat request", e); | |
| options.onError?.(e as Error); | |
| } |
There was a problem hiding this comment.
@forrestlinfeng, great to hear that you've made the update! If you have any more questions or need further assistance, feel free to ask. 😊
|
Your build has completed! |
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- app/client/platforms/stepfun.ts (1 hunks)
Files skipped from review as they are similar to previous changes (1)
- app/client/platforms/stepfun.ts
how to authorize NextChat?? |
1 participant
💻 变更类型 | Change Type
🔀 变更说明 | Description of Change
📝 补充信息 | Additional Information
Summary by CodeRabbit
New Features
Bug Fixes
Documentation
Chores