vision-and-language

Here are 256 public repositories matching this topic...

salesforce / LAVIS

LAVIS - A One-stop Library for Language-Vision Intelligence

deep-learning salesforce image-captioning deep-learning-library vision-framework vision-and-language multimodal-deep-learning multimodal-datasets vision-language-transformer vision-language-pretraining visual-question-anwsering

Updated Nov 18, 2024
Jupyter Notebook

aishwaryanr / awesome-generative-ai-guide

Star

A one stop repository for generative AI research updates, interview resources, notebooks and much more!

awesome awesome-list interview-questions vision-and-language notebook-jupyter large-language-models llms generative-ai

Updated Dec 11, 2024

salesforce / ALBEF

Star

Code for ALBEF: a new vision-language pre-training method

representation-learning weakly-supervised-learning image-text vision-and-language contrastive-learning

Updated Sep 20, 2022
Python

om-ai-lab / OmDet

Star

Real-time and accurate open-vocabulary end-to-end object detection

object-detection vision-and-language zero-shot-object-detection open-vocabulary

Updated Sep 6, 2024
Python

open-mmlab / Multimodal-GPT

Star

Multimodal-GPT

transformer llama gpt flamingo multimodal vision-and-language gpt-4

Updated Jun 4, 2023
Python

dandelin / ViLT

Star

Code for the ICML 2021 (long talk) paper: "ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision"

vision-and-language

Updated Apr 3, 2024
Python

roboflow / maestro

Star

streamline the fine-tuning process for multimodal models: PaliGemma, Florence-2, and Qwen2-VL

transformers vqa objectdetection captioning fine-tuning multimodal vision-and-language phi-3-vision paligemma florence-2

Updated Dec 9, 2024
Python

om-ai-lab / OmAgent

Star

A Language and Multimodal Agents Framework for Smart Device and More

agent workflow framework vision-and-language large-language-models smart-hardware multimodal-agent gpt-4o

Updated Dec 12, 2024
Python

NVlabs / prismer

Star

The implementation of "Prismer: A Vision-Language Model with Multi-Task Experts".

vqa image-captioning language-model multi-task-learning vision-and-language multi-modal-learning vision-language-model

Updated Jan 17, 2024
Python

yuewang-cuhk / awesome-vision-language-pretraining-papers

Star

Recent Advances in Vision and Language PreTrained Models (VL-PTMs)

bert vision-and-language multimodal-deep-learning pretraining vl-ptms

Updated Aug 19, 2022

microsoft / Oscar

Star

Oscar and VinVL

vqa image-captioning oscar vision-and-language pre-training image-text-search vinvl

Updated Aug 28, 2023
Python

llm-jp / awesome-japanese-llm

Star

日本語LLMまとめ - Overview of Japanese LLMs

japanese generative-model japanese-language language-models language-model generative-models multimodal vision-and-language vision-language foundation-models large-language-models llm llms generative-ai large-language-model vision-language-model japanese-llm japanese-language-model llm-japanese

Updated Dec 8, 2024
TypeScript

X-modaler is a versatile and high-performance codebase for cross-modal analytics(e.g., image captioning, video captioning, vision-language pre-training, visual question answering, visual commonsense reasoning, and cross-modal retrieval).

image-captioning video-captioning visual-question-answering vision-and-language cross-modal-retrieval pretraining tden

Updated Feb 27, 2023
Python

OFA-Sys / ONE-PEACE

Star

A general representation model across vision, audio, language modalities. Paper: ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities

representation-learning multimodal vision-and-language contrastive-loss vision-language vision-transformer foundation-models audio-language

Updated Oct 6, 2024
Python

rhymes-ai / Aria

Star

Codebase for Aria - an Open Multimodal Native MoE

multimodal vision-and-language mixture-of-experts

Updated Dec 12, 2024
Jupyter Notebook

26hzhang / DL-NLP-Readings

Star

My Reading Lists of Deep Learning and Natural Language Processing

machine-learning natural-language-processing reinforcement-learning deep-learning robotics paper language-model bibtex-references commonsense vision-and-language source-code-link

Updated Apr 30, 2022
TeX

mbzuai-oryx / groundingLMM

Star

[CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.

vision-and-language lmm foundation-models vision-language-model llm-agent