Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

Mobile-Agent: The Powerful Mobile Device Operation Assistant Family

InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output

Reasoning in Large Language Models: Papers and Resources, including Chain-of-Thought and OpenAI o1 🍓

Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

A family of lightweight multimodal models.

[CVPR2024] The code for "Osprey: Pixel Understanding with Visual Instruction Tuning"

✨✨Woodpecker: Hallucination Correction for Multimodal Large Language Models. The first work to correct hallucinations in MLLMs.

Agent S: an open agentic framework that uses computers like a human

[ECCV2024] Grounded Multimodal Large Language Model with Localized Visual Tokenization

EAGLE: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders

This project is the official implementation of 'LLMGA: Multimodal Large Language Model based Generation Assistant', ECCV2024 Oral

Custom ComfyUI nodes for Vision Language Models, Large Language Models, Image to Music, Text to Music, Consistent and Random Creative Prompt Generation

NeurIPS 2024 Paper: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train your own 8B/14B LLaVA-training-like MLLM on RTX3090/4090 24GB.

🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).

Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Pre-training Dataset and Benchmarks

Awesome_Multimodel is a curated GitHub repository that provides a comprehensive collection of resources for Multimodal Large Language Models (MLLM). It covers datasets, tuning techniques, in-context learning, visual reasoning, foundational models, and more. Stay updated with the latest advancement.

[NeurIPS'24 Spotlight] EVE: Encoder-Free Vision-Language Models