Vision-Language Modelを活用ã—ãŸã€Œè¦‹ãŸç›®ãŒè¿‘ã„商å“ã€ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰æ”¹å–„ã®å–り組㿠| メルカリエンジニアリングã“ã‚“ã«ã¡ã¯ã€ãƒ¡ãƒ«ã‚«ãƒªã®AI/LLMãƒãƒ¼ãƒ ã§æ©Ÿæ¢°å¦ç¿’エンジニアをã—ã¦ã„ã‚‹arr0wã¨shoã§ã™ï¼ 本テックブãƒã‚°ã§ã¯ã€Vision-Language Modelã®ä¸€ã¤ã§ã‚ã‚‹SigLIP [1]ã‚’ã€ãƒ¡ãƒ«ã‚«ãƒªã®å•†å“データ（Image-Text Pairs）ã§ãƒ•ã‚¡ã‚¤ãƒ³ãƒãƒ¥ãƒ¼ãƒ‹ãƒ³ã‚°ã—ã€ãƒ¡ãƒ«ã‚«ãƒªã®å•†å“ç”»åƒEmbeddingã®æ€§èƒ½ã‚’大幅ã«æ”¹å–„ã—ãŸãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã«ã¤ã„ã¦ç´¹ä»‹ã—ã¾ã™ã€‚ 今回作æˆã—ãŸSigLIPã®æ€§èƒ½ã‚’評価ã™ã‚‹ãŸã‚ã«ã€å•†å“詳細ページã®ã€Œè¦‹ãŸç›®ãŒè¿‘ã„商å“ã€ã®ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰æ©Ÿèƒ½ã§A/Bテストを実施ã—ã¾ã—ãŸã€‚ ã“ã®ã€Œè¦‹ãŸç›®ãŒè¿‘ã„商å“ã€ã®ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰æ©Ÿèƒ½ã¯ã€ç¤¾å†…ã§ã¯Similar Looksã¨å‘¼ã°ã‚Œã¦ã„ã¾ã™ã€‚作æˆã—ãŸãƒ¢ãƒ‡ãƒ«ã‚’Similar Looksã®é¡žä¼¼ç”»åƒæ¤œç´¢ã«é©ç”¨ã—ã€æ—¢å˜ãƒ¢ãƒ‡ãƒ«ã¨ã®æ¯”較ã®ãŸã‚ã®A/Bテストを行ã„ã¾ã—ãŸã€‚ ãã—ã¦ã€ãã®çµæžœã¨ã—ã¦ã€ä¸»è¦ãªKPIã«ãŠã„ã¦ä»¥ä¸‹ã®ã‚ˆã†ãªé¡•è‘—ãªæ”¹å–„ãŒç¢ºèªã§ãã¾ã—ãŸã€‚
日本語ã«ã‚‚対応ã™ã‚‹ãƒžãƒ«ãƒãƒ¢ãƒ¼ãƒ€ãƒ«AIモデルSigLIPã§ã‚¼ãƒã‚·ãƒ§ãƒƒãƒˆç”»åƒåˆ†é¡žã‚’試ã™
Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models
https://molmo.allenai.org/blog
https://molmo.allenai.org/blog?utm_campaign=Data%20Points&utm_source=hs_email&utm_medium=email
OpenAI Platform
GitHub - hkproj/pytorch-paligemma: Coding a Multimodal (Vision) Language Model from scratch in PyTorch with full explanation: https://www.youtube.com/watch?v=vAmKB7iPkWw
{{#tags}}- {{label}}
{{/tags}}