VLMs: major clean up 🧼 #34502
Open
VLMs: major clean up 🧼 #34502
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
We can remove it after updating the files on the hub and that mean we also need to change warning to error so users have chance to see what is the reason for failure. I think the earliest we can remove is next release, because the blocking PR will prob be merged next week. After that I will take time to update all hub configs. Maybe then we'll wait for the blocking PR and remove all deprecation warnings? |
|
Sounds good, let's wait a bit! |
4 tasks
This was referenced Dec 6, 2024
|
@ArthurZucker i think this can be review now :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
We have updated all the configs for VLMs on the hub so this PR removes legacy path for models, as it has been there for already 3 releases from v4.44. Also it fixes some stuff that broke on the way, like generating from only text input in LLaVA models
For Video-LLaVA the hub configs cannot be updated as the hub owner has been silent for several mmonths already. And since there is only one model with such architecture, we can hardcode the default values for
patch_numand also remove the legacy pathfixes #34824, fixes #35169