A search engine in 80 lines of Python
February 05, 2024 · 26 mins · 4727 words Discussion on HackerNews. Last September I hopped on board with Wallapop as a Search Data Scientist and since then part of my work has been working with Solr, an open-source search engine based on Lucene. I’ve got the basics of how a search engine works, but I had this itch to understand it even better. So, I rolled up my sleeves and decided to build one fr
注目ã®æ—¥æœ¬èªžå¤§è¦æ¨¡ãƒ¢ãƒ‡ãƒ«LLM-JP 13Bを試ã™(JSONåã‘ãŸ)｜shi3z
å›½ç«‹æƒ…å ±å¦ç ”究所ãŒä½œã£ãŸLLM-JP 13BãŒå…¬é–‹ã•ã‚Œã¦ã„ãŸã®ã§æ—©é€Ÿè©¦ã—ã¦ã¿ãŸã€‚色々ãªãƒ¢ãƒ‡ãƒ«ãŒå…¬é–‹ã•ã‚Œã¦ã„ã‚‹ãŒã€å…¨éƒ¨ç››ã‚Šã£ã½ã„ 「llm-jp/llm-jp-13b-instruct-full-jaster-dolly-oasst-v1.0ã€ã‚’試ã™ã€‚ マシンã¯ã‚‚ã¡ã‚んドスパラ製Memeplexマシン(A6000x2)ã 。 import torch from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("llm-jp/llm-jp-13b-instruct-full-jaster-dolly-oasst-v1.0") model = AutoModelForCausalLM.from_pretrained("llm-jp/llm-jp-13b
都立大 自然言語処ç†ç ”究室 - 自然言語処ç†ã‚’独習ã—ãŸã„人ã®ãŸã‚ã«
æ±äº¬éƒ½ç«‹å¤§å¦è‡ªç„¶è¨€èªžå‡¦ç†ç ”究室（å°ç”ºç ”）ã«å…¥å¦ã™ã‚‹äººãŸã¡ã¯ã€å…¥å¦å¾Œã«è‡ªç„¶è¨€èªžå‡¦ç†ãƒ»æ©Ÿæ¢°å¦ç¿’・プãƒã‚°ãƒ©ãƒŸãƒ³ã‚°ã®åŸºç¤Žå‹‰å¼·ä¼šã‚’è¡Œãªã†ãŸã‚ã€ç‰¹ã«äº‹å‰ã«å¦ç¿’ã™ã‚‹ã“ã¨ã¯ã‚ã‚Šã¾ã›ã‚“ãŒã€å…¥å¦å‰ã«å‹‰å¼·ã—ã¦ãŠã„ãŸã»ã†ãŒã‚ˆã„ã“ã¨ã¯ã‚ã‚‹ã‹ã€ã¨å•ã„åˆã‚ã›ãŒã‚ã‚‹ã®ã§ã€è‡ªç„¶è¨€èªžå‡¦ç†ã®ç‹¬ç¿’用ã®æƒ…å ±ã‚’æ›¸ã„ã¦ãŠãã¾ã™ã€‚（主ã«æƒ…å ±ç³»ä»¥å¤–ã®å¦éƒ¨ç”Ÿã‚’対象ã«ã—ã¦ã„ã¾ã™ã€‚） è¿‘å¹´ã®æ·±å±¤å¦ç¿’ã®ç™ºå±•ã«ä¼´ã£ã¦ãƒ‹ãƒ¥ãƒ¼ãƒ©ãƒ«æ©Ÿæ¢°ç¿»è¨³ã‚’ã¯ã˜ã‚ã¨ã—ãŸæ·±å±¤å¦ç¿’ã®ç ”究をã—ãŸã„ã¨ã„ã†äººãŒã†ã¡ã®å—験希望者ã®å¤§åŠã§ã™ãŒã€æ·±å±¤å¦ç¿’ã®ç ”究をã—ãŸã„人ã¯ãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°ã¨æ•°å¦ã¨è‹±èªžã®ã™ã¹ã¦ãŒã‚る程度ã§ããªã‘ã‚Œã°ã„ã‘ã¾ã›ã‚“。ã©ã‚Œã‹1ã¤ã§ã‚‚足りãªã„ã‚‚ã®ãŒã‚ã‚Œã°ã€ã»ã¨ã‚“ã©ç ”究ã§ããªã„ã¨æ€ã£ã¦ãã ã•ã„。ã“れらã¯ç¾åœ¨ã»ã¼æœªç¿’ã®äººãŒå…¥å¦ã—ã¦ã‹ã‚‰å‹‰å¼·ã—ã¦è¿½ã„ã¤ã‘ã‚‹ã‚‚ã®ã§ã¯ãªã„ã®ã§ã€æ·±å±¤å¦ç¿’ã®ç ”究ãŒã—ãŸã„ã€ã¨ã„ã†å ´åˆã¯ã‚ˆãよã考ãˆãŸã»ã†ãŒã„ã„ã§ã™ã€‚æ±å¤§ä¸å±±ç ”ã®æ·±å±¤å¦ç¿’ã®ç ”究ãŒã‚„ã‚ŠãŸã„人
1
ランã‚ング
ランã‚ング
ランã‚ング
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
処ç†ã‚’実行ä¸ã§ã™
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
{{#tags}}- {{label}}
{{/tags}}