[B! encoding] yassã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

yass id:yass

encodingã«é–¢ã™ã‚‹yassã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (27)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Apacheãƒªãƒãƒ¼ã‚¹ãƒ—ãƒã‚ã‚·ã«ãŠã‘ã‚‹ã‚¹ãƒ©ãƒƒã‚·ãƒ¥ã¨%2Få•é¡Œã¯ã¾ã£ãŸãƒã‚¤ãƒ³ãƒˆ - YOMON8.NET
Apacheã¨%2Fã§ã‚’èª¿ã¹ã‚‹ã¨æ²¢å±±å‡ºã¦ãã‚‹å†…å®¹ãªã®ã§ã™ãŒã€Apacheã¯URLã‹ã‚‰å–å¾—ã•ã‚Œã‚‹PATH_INFOå¤‰æ•°ã« / ã®ã‚¨ãƒ³ã‚³ãƒ¼ãƒ‰ã§ã‚ã‚‹ %2F ãŒã‚ã‚‹ã¨404ã‚’è¿”ã—ã¾ã™ã€‚ ã“ã‚Œè‡ªä½“ã¯AllowEncodedSlashesã¨ã„ã†ãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ã‚’Onã«ã—ã¦ã‚„ã‚‹ã¨å‡¦ç†ã§ãã‚‹ã‚ˆã†ã«ãªã‚Šã¾ã™ã€‚ <VirtalHost> ... AllowEncodedSlashes on ... </VirtalHost> AllowEncodedSlashes off %2F ã¯404ã‚¨ãƒ©ãƒ¼ AllowEncodedSlashes on %2F ã¯ / ã«ãƒ‡ã‚³ãƒ¼ãƒ‰ ã“ã‚Œã§ã™ãŒã€ãƒ‡ã‚³ãƒ¼ãƒ‰ã—ãªã„ã§å¾Œã‚ã®WEBã‚µãƒ¼ãƒã«æ¸¡ã—ã¦æ¬²ã—ã„ã¨ã„ã†è¦ä»¶ã«åˆã†é¸æŠžè‚¢ãŒã‚ã‚Šã¾ã›ã‚“ã€‚ã“ã‚ŒãŒä¸€ã¤ã®ãƒãƒžã‚Šãƒã‚¤ãƒ³ãƒˆã§ã—ãŸã€‚ ä»Šæ›¸ã„ã¦ã„ã‚‹æ™‚ç‚¹ã§ã¯æ—¥æœ¬èªžã®ãƒžãƒ‹ãƒ¥ã‚¢ãƒ«ã«ã¯ã“ã®äºŒã¤ã®ã‚ªãƒ—ã‚·ãƒ§ãƒ³ã¾ã§ã—ã‹æ›¸ã‹ã‚Œã¦ã„ãªã„ã®ã§ã™ãŒã€è‹±èªžã«ãƒžãƒ‹ãƒ¥ã‚¢ãƒ«è¦‹ã‚‹ã¨
yass 2017/11/02
apache

proxy

encoding
ãƒªãƒ³ã‚¯
Senary - Wikipedia
yass 2014/09/07
" Base 36 is therefore the most compact case-insensitive alphanumeric numeral system using ASCII characters "

ascii

encoding
ãƒªãƒ³ã‚¯
Excelã§UTF-8ã‚¨ãƒ³ã‚³ãƒ¼ãƒ‡ã‚£ãƒ³ã‚°ã•ã‚ŒãŸCSVãƒ•ã‚¡ã‚¤ãƒ«ã‚’é–‹ãæ–¹æ³• - å°ã•ã„é ƒã¯ã‚¨ãƒ©å‘¼å¸
photo credit: gonzalo_ar via photopin cc ã¯ã˜ã‚ã« ã“ã®ã‚¨ãƒ³ãƒˆãƒªã§ã¯ã€Excelã§UTF-8ã‚¨ãƒ³ã‚³ãƒ¼ãƒ‡ã‚£ãƒ³ã‚°ã•ã‚ŒãŸCSVãƒ•ã‚¡ã‚¤ãƒ«ã‚’é–‹ãæ–¹æ³•ã‚’ç´¹ä»‹ã—ã¾ã™ã€‚ Excel VBAã®ãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°ã®ãƒ„ãƒœã¨ã‚³ãƒ„ãŒã‚¼ãƒƒã‚¿ã‚¤ã«ã‚ã‹ã‚‹æœ¬â€•æœ€åˆã‹ã‚‰ãã†æ•™ãˆã¦ãã‚Œã‚Œã°ã„ã„ã®ã«!Excel2007/2003å¯¾å¿œposted with amazlet at 12.12.23ç«‹å±± ç§€åˆ© ç§€å’Œã‚·ã‚¹ãƒ†ãƒ å£²ã‚Šä¸Šã’ãƒ©ãƒ³ã‚ãƒ³ã‚°: 1,346 Amazon.co.jp ã§è©³ç´°ã‚’è¦‹ã‚‹ çµè«– ä»¥ä¸‹ã®ã‚ˆã†ãªï¼“ã¤ã®æ–¹æ³•ãŒã‚ã‚Šã¾ã™ã€‚ æ‹¡å¼µåã‚’csvã‹ã‚‰txtã«å¤‰æ›´ã—ã¦ã‹ã‚‰é–‹ãã€‚ CSVãƒ•ã‚¡ã‚¤ãƒ«ã‚’BOMã¤ãã®UTF-8ãƒ•ã‚¡ã‚¤ãƒ«ã«ä¿å˜ã—ãªãŠã—ã¦ã‹ã‚‰é–‹ãã€‚ CSVãƒ•ã‚¡ã‚¤ãƒ«ã®ä¿å˜å½¢å¼ã‚’ANSIã«å¤‰æ›´ã—ã¦ã‹ã‚‰é–‹ãã€‚ Excelã¯CSVãƒ•ã‚¡ã‚¤ãƒ«ã‚’Shift_JISã§é–‹ã“ã†ã¨ã™ã‚‹ Excelã¯ã€CSVãƒ•ã‚¡ã‚¤ãƒ«
yass 2013/06/10
Excel

UTF-8

encoding
ãƒªãƒ³ã‚¯
List::FrontCode - naoyaã®ã¯ã¦ãªãƒ€ã‚¤ã‚¢ãƒªãƒ¼
å…ˆæ—¥ Array::Gap ã¨ã„ã† Variable Byte Codes ã«ã‚ˆã‚‹æ•´åˆ—æ¸ˆã¿æ•´æ•°ã®åœ§ç¸®ã®å®Ÿè£…ã‚’ä½œã‚Šã¾ã—ãŸã€‚(id:naoya:20080906:1220685978) ä»Šæ—¥ã¯ Front Coding ã‚’ä½¿ã£ãŸåŒã˜ã‚ˆã†ãªåœ§ç¸®ãƒªã‚¹ãƒˆã‚¯ãƒ©ã‚¹ã€List::FrontCode ã‚’ä½œã£ã¦ã¿ã¾ã—ãŸã€‚Front Coding ã¯è¾žæ›¸å¼é †ã«æ•´åˆ—æ¸ˆã¿ã®æ–‡å—åˆ—ãƒªã‚¹ãƒˆãªã©ã‚’åœ§ç¸®ã™ã‚‹æ‰‹æ³•ã§ã™ã€‚WEB+DB PRESS Vol.42 ã®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ &ãƒ‡ãƒ¼ã‚¿æ§‹é€ ã®è¨˜äº‹ã§ PFI ã®å²¡é‡ŽåŽŸã•ã‚“ã«ã‚ˆã‚‹è§£èª¬ãŒã‚ã£ãŸã®ã§ã€ãã‚Œã‚’å‚è€ƒã«å®Ÿè£…ã—ã¾ã—ãŸã€‚ Front Coding Front Coding ã¯ http://www.hoge.jp http://www.hoge.jp/a.htm http://www.hoge.jp/index.htm http://www.fuga.com/ http://www.
yass 2013/04/21
front coding

encoding

compression

algorithm

string

diff

delta encoding
ãƒªãƒ³ã‚¯
GitHub - MarkusH/MiGBase64: Forked from http://sourceforge.net/projects/migbase64/ on version 2.2
yass 2013/03/02
base64

encoding

java
ãƒªãƒ³ã‚¯
Shitf_JISã¨CP932ã¨MS932ã¨Windows-31Jã‚’æ˜”è©±é¢¨ã«èªžã£ã¦ã¿ãŸ - å‡¶æ‚ªãƒ¢ãƒŠãƒ‰
è¦ç´„ã™ã‚‹ã¨ ã“ã‚“ãªæ„Ÿã˜ Shift_JIS â‰’ CP932 ï¼ Windows-31J ï¼ MS932 ã“ã†ã„ã†è¨€ã„æ–¹ã‚‚ã§ãã‚‹ ä»•æ§˜ï¼šShift_JIS å®Ÿè£…ï¼šCP932, Windows-31J Javaã§ã®ã¿ä½¿ã‚ã‚Œã‚‹éƒ½åˆä¸Šã®åå‰ åˆ¥åï¼šMS932 æœ¬ç·¨ ã‚€ã‹ã—ã‚€ã‹ã—ã€ã‚ã‚‹ã¨ã“ã‚ã«ã€ŒShift_JISã€ã¨ã„ã†åã®æ–‡å—ã‚¨ãƒ³ã‚³ãƒ¼ãƒ‡ã‚£ãƒ³ã‚°ãŒãŠã£ãŸãã†ãªã€‚é‡Œã®æ‘é•·ã€Œãƒžã‚¤ã‚¯ãƒã‚½ãƒ•ãƒˆãŸã‚“ã€ã¯ã„ã¡æ—©ãShift_JISã®æ‰ã«ç›®ã‚’ã¤ã‘ãŸã€‚ãƒžã‚¤ã‚¯ãƒã‚½ãƒ•ãƒˆãŸã‚“ã¯Shift_JISã‚’è‚²ã¦ã‚‹ã“ã¨ã«ã—ã€æ„›ç§°ã¨ã—ã¦ã€Žã‚³ãƒ¼ãƒ‰ãƒšãƒ¼ã‚¸932(CP932)ã€ã¨å‘¼ã‚“ã§ãŠã£ãŸãã†ãªã€‚ãã‚Œã‹ã‚‰ã¾ã‚‚ãªãã—ã¦ã€Shift_JISã¯MS-DOSã¨ã„ã†OSã«ã®ã›ã¦å£²ã‚‰ã‚Œã¯ã˜ã‚ã‚‹ã‚ˆã†ã«ãªã£ãŸã‚“ã˜ã‚ƒã€‚ åŒã˜é‡Œã®ã€ŒIBMãŸã‚“ã€ã¯ãã®MS-DOSã‚’ã¤ã‹ã£ã¦ãƒžãƒ«ãƒã‚¹ãƒ†ãƒ¼ã‚·ãƒ§ãƒ³5550ã¨ã„ã†ãƒ‘ã‚½ã‚³ãƒ³ã‚’ã“ã—ã‚‰ãˆãŸã€‚IBMãŸã‚“ã¯ã‚‚ã£ã¨ä¾¿åˆ©ã«ãªã‚‹ã‚ˆã†ã«ã¨
yass 2013/03/01
Windows

encoding

cp932
ãƒªãƒ³ã‚¯
30åˆ†ã§ã‚ã‹ã‚‹é«˜æ€§èƒ½ãªåœ§ç¸®ç¬¦å·vertical code - EchizenBlog-Zwei
æ¤œç´¢ã‚¨ãƒ³ã‚¸ãƒ³ã®è»¢ç½®ã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ãªã©ãƒ‡ãƒ¼ã‚¿åˆ—ã‚’å°ã•ã„ãƒ‡ãƒ¼ã‚¿ã‚µã‚¤ã‚ºã§æŒãŸã›ãŸã„ã€ã¨ã„ã†çŠ¶æ³ãŒã‚ã‚‹ã€‚ã“ã†ã„ã†å ´åˆåœ§ç¸®ç¬¦å·ã‚’ä½¿ã†ã®ãŒä¸€èˆ¬çš„ã§unaryç¬¦å·ã‚„gammaç¬¦å·ã€deltaç¬¦å·ãªã©æ§˜ã€…ãªç¨®é¡žãŒã‚ã‚‹ã€‚ åœ§ç¸®ç¬¦å·ã®ä¸ã§ã‚¤ãƒã‚ªã‚·ãªã®ãŒvertical code(vcode)ã€‚ã“ã‚Œã¯å²¡é‡ŽåŽŸ(@hillbig)æ°ã«ã‚ˆã£ã¦ææ¡ˆã•ã‚ŒãŸåœ§ç¸®ç¬¦å·ã§å˜ç´”ãªä»•çµ„ã¿ã§deltaç¬¦å·ä¸¦ã®æ€§èƒ½ã‚’èª‡ã£ã¦ã„ã‚‹ã€‚ æœ¬è¨˜äº‹ã§ã¯vcodeã®ãƒã‚¤ãƒ³ãƒˆã‚’çµžã£ã¦30åˆ†ã§ã‚ã‹ã‚‹ã‚ˆã†ã«è§£èª¬ã—ã¦ã¿ã‚‹ã€‚ vcodeã¯æœ¬æ£šã«æœ¬ã‚’ä¸¦ã¹ã‚‹ä½œæ¥ã‚’é€£æƒ³ã™ã‚‹ã¨ç†è§£ã—ã‚„ã™ã„ã€‚æœ¬æ£šã¯äºˆã‚é«˜ã•ãŒæ±ºã¾ã£ã¦ã„ã‚‹ã®ã§å…¨ã¦ã®æœ¬ãŒå…¥ã‚‹ã‚ˆã†ãªæœ¬æ£šã‚’ç”¨æ„ã™ã‚‹ã€‚ã¤ã¾ã‚Š ã¨ã„ã†ã‚ˆã†ãªã‚‚ã®ã‚’æƒ³åƒã™ã‚‹ã€‚ ã“ã®æœ¬æ£šã¯8å†Šã®æœ¬ãŒä¸¦ã‚“ã§ã„ã‚‹ãŒå·¦ã‹ã‚‰5å†Šç›®ã®æœ¬ãŒä»–ã‚ˆã‚Šã‚‚èƒŒãŒé«˜ã„ã€‚ã“ã®ãŸã‚5å†Šç›®ã®æœ¬ã«åˆã‚ã›ã¦èƒŒã®é«˜ã„æœ¬æ£šãŒå¿…è¦ã«ãªã‚‹ã€‚ã ãŒä»–ã®æœ¬ã¯5å†Šç›®ã®æœ¬ã»ã©ã«èƒŒãŒé«˜ããªã„ã®ã§ã€5å†Šç›®ãŒ
yass 2012/11/17
compression

algorithm

bit

encoding
ãƒªãƒ³ã‚¯
What Every Programmer Absolutely, Positively Needs to Know About Encodings and Character Sets to Work With Text
There are 95 human readable characters specified in the ASCII table, including the letters A through Z both in upper and lower case, the numbers 0 through 9, a handful of punctuation marks and characters like the dollar symbol, the ampersand and a few others. It also includes 33 values for things like space, line feed, tab, backspace and so on. These are not printable per se, but still visible in
yass 2012/11/13
ascii

encoding
ãƒªãƒ³ã‚¯
Î³ç¬¦å·ã€Î´ç¬¦å·ã€ã‚´ãƒãƒ ç¬¦å·ã«ã‚ˆã‚‹åœ§ç¸®åŠ¹æžœ - naoyaã®ã¯ã¦ãªãƒ€ã‚¤ã‚¢ãƒªãƒ¼
é€šå¸¸ã®æ•´æ•°ã¯ 32 ãƒ“ãƒƒãƒˆã¯ 4 ãƒã‚¤ãƒˆã®å›ºå®šé•·ã«ã‚ˆã‚‹ãƒã‚¤ãƒŠãƒªç¬¦å·ã§ã™ãŒã€å°ã•ãªæ•°å—ãŒãŸãã•ã‚“å‡ºç¾ã—ã€å¤§ããªæ•°å—ã¯ã»ã¨ã‚“ã©å‡ºç¾ã—ãªã„ã¨ã„ã†ç¢ºçŽ‡åˆ†å¸ƒã®ã‚‚ã¨ã§ã¯ç„¡é§„ãªãƒ“ãƒƒãƒˆãŒç›®ç«‹ã¡ã¾ã™ã€‚ Variable Byte Code (Byte Aligned ç¬¦å·ã¨ã‚‚å‘¼ã°ã‚Œã¾ã™) ã¯æ•´æ•°ã®ç¬¦å·åŒ–æ‰‹æ³•ã®ä¸€ã¤ã§ã€ã“ã®ç„¡é§„ã‚’å¹¾åˆ†è§£æ¶ˆã—ã¾ã™ã€‚è©³ã—ãã¯ Introduction to Information Retrieval (ä»¥ä¸‹ IIR) ã®ç¬¬5ç« ã«æŽ²è¼‰ã•ã‚Œã¦ã„ã¾ã™ã€‚(http://nlp.stanford.edu/IR-book/html/htmledition/variable-byte-codes-1.html ã§å…¬é–‹ã•ã‚Œã¦ã„ã¾ã™) Variable Byte Code ã¯ãã®åã®é€šã‚Šãƒã‚¤ãƒˆãƒ¬ãƒ™ãƒ«ã®å¯å¤‰é•·ç¬¦å·ã§ã€1ãƒã‚¤ãƒˆã®å…ˆé 1ãƒ“ãƒƒãƒˆã‚’ continuation ãƒ“ãƒƒãƒˆã¨ã—ã¦æ‰±ã„ã€ç¶šã 7 ãƒ“ãƒƒãƒˆ
yass 2012/11/12
" ã‚´ãƒãƒ ç¬¦å·ã¯ãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ãƒ•ãƒªãƒ¼ãªÎ³ç¬¦å·ã‚„Î´ç¬¦å·ã¨ã¯ç•°ãªã‚Šã€ç¬¦å·é•·ã‚’èª¿æ•´ã™ã‚‹ãŸã‚ã®ãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ãŒå¿…è¦ã«ãªã‚Šã¾ã™ã€‚ã“ã®ãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ã¯è»¢ç½®ã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ã®å ´åˆã€å…¨ä½“ã®æ–‡æ›¸æ•°ã‚„å˜èªžã®æ•°ã‹ã‚‰æ±‚ã‚ã‚‹ã“ã¨ãŒã§ãã¾ã™ "

integer

encoding

gamma coding

delta coding

compression

vByte

algorithm

Golomb coding
ãƒªãƒ³ã‚¯
Information Retrieval â€“ Addenda: Index Compression
Group VarInt In Chapter 6, we discussed the byte-aligned vByte method as an example of an index compression technique with high decoding performance. After we had written the chapter, Jeff Dean gave a keynote talk at WSDM 2009 in which he discussed a few compression algorithms used by Google, including the byte-aligned Group VarInt method. Group VarInt is named after VarInt (variable-size integer
yass 2012/11/11
group varint encoding

integer

encoding

comparison

vByte

simple9
ãƒªãƒ³ã‚¯
GitHub - stuhood/gvi: group varint encoding
yass 2012/11/11
" A Java implementation of group varint encoding."

integer

encoding

group varint encoding

java

varint

compression
ãƒªãƒ³ã‚¯
Number Encoding
yass 2012/11/11
integer

encoding

group varint encoding

vByte
ãƒªãƒ³ã‚¯
Looking for more details about "Group varint encoding/decoding" presented in Jeff's slides
That's referring to "variable integer encoding", where the number of bits used to store an integer when serialized is not fixed at 4 bytes. There is a good description of varint in the protocol buffer documentation. It is used in encoding Google's protocol buffers, and you can browse the protocol buffer source code. The CodedOutputStream contains the exact encoding function WriteVarint32FallbackTo
yass 2012/11/11
group varint encoding

integer

encoding

ProtocolBuffers

varint
ãƒªãƒ³ã‚¯
å…¨æ–‡æ¤œç´¢ã‚¨ãƒ³ã‚¸ãƒ³ Miniseã‚’ãƒªãƒªãƒ¼ã‚¹ + WEB+DBã§å…¨æ–‡æ¤œç´¢ã®ç‰¹é›†è¨˜äº‹ - DO++
å…¨æ–‡æ¤œç´¢ã‚¨ãƒ³ã‚¸ãƒ³ã® Minise: MIni Search Engineã‚’ãƒªãƒªãƒ¼ã‚¹ã—ã¾ã—ãŸï¼Ž ã“ã®ã‚¨ãƒ³ã‚¸ãƒ³ã¯å…¨æ–‡æ¤œç´¢ã®åŸºæœ¬çš„ãªæ©Ÿèƒ½ã‚’ã‚µãƒãƒ¼ãƒˆã—ãŸã‚‚ã®ã§ï¼Œç´¢å¼•æ‰‹æ³•ã¯é€æ¬¡æ¤œç´¢ï¼ˆç´¢å¼•ç„¡ï¼‰ï¼ŒN-gramï¼Œè»¢ç½®ãƒ•ã‚¡ã‚¤ãƒ«ï¼ŒæŽ¥å°¾è¾žé…åˆ—ã‚’ã‚µãƒãƒ¼ãƒˆã—ã¦ãŠã‚Šï¼Œãã“ãã“æœ€é©åŒ–ã‚’è¡Œã£ã¦ã¾ã™ï¼ŽWikipediaæ—¥æœ¬èªžç‰ˆã‚’å®Ÿé¨“ã§ä½¿ã£ãŸã‚‚ã®ã§20ä¸‡æ–‡æ›¸ã§æ§‹ç¯‰æ™‚é–“ãŒ500ç§’å‰å¾Œï¼Œæ¤œç´¢æ™‚é–“ãŒä¸€ã‚¯ã‚¨ãƒªã‚ãŸã‚Šæ•°msã¨ãªã£ã¦ã„ã¾ã™ï¼Ž BSDãƒ©ã‚¤ã‚»ãƒ³ã‚¹ã§å…¬é–‹ã—ã¦ã„ã¾ã™ï¼Ž å‰²ã‚Šãã£ã¦ï¼Œæ©Ÿèƒ½ã‚’çµžã£ã¦ã‚·ãƒ³ãƒ—ãƒ«ãªæ§‹æˆã«ã—ã¦ã„ã¾ã™ã®ã§æ”¹é€ ã—ãŸã‚Šã—ã‚„ã™ã„ã‚ˆã†ã«ãªã£ã¦ã„ã¾ã™ã€‚ã¾ã ã€ãƒ‰ã‚ãƒ¥ãƒ¡ãƒ³ãƒˆã¯ãªã„ã§ã™ãŒã€C++ APIã¨ã—ã¦åˆ©ç”¨ã—ã‚„ã™ã„ã‚ˆã†ã«ã‚‚ãªã£ã¦ã„ã¾ã™ã®ã§ã€ç ”ç©¶ç”¨é€”ãªã©ã§æ–°ã—ã„ç´¢å¼•ã‚„ãƒ©ãƒ³ã‚ãƒ³ã‚°ã¨ã‹ã§ã®åˆ©ç”¨ã‚‚æƒ³å®šã—ã¦ã„ã¾ã™ï¼ˆå®Ÿéš›ã«ç ”ç©¶ç”¨ã§ä½¿ã£ã¦ã¾ã™ï¼‰ï¼Ž --- ä»Šå›žã®å…¨æ–‡æ¤œç´¢ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã‚’é–‹ç™ºã™ã‚‹æ©Ÿä¼šã«ãªã£ãŸã®ãŒï¼Œç§ãŒæ‹…å½“ã—ãŸä»Šæœˆå·ã®WEB+
yass 2012/11/11
full text search

encoding

Index
ãƒªãƒ³ã‚¯
[IR] Google WSDM'09è¬›æ¼”ã§è¿°ã¹ã‚‰ã‚Œã¦ã„ã‚‹ç¬¦å·åŒ–æ–¹å¼ã‚’å®Ÿè£…ã—ã¦ã¿ãŸ - tsubosakaã®æ—¥è¨˜
MGå‹‰å¼·ä¼šã®å¾Œã«id:sleepy_yoshiã•ã‚“ã«æ•™ãˆã¦ã‚‚ã‚‰ã£ãŸWSDM 2009ã«ãŠã‘ã‚‹è¬›æ¼”"Challenges in Building Large-Scale Information Retrieval Systems"ã§è¿°ã¹ã‚‰ã‚Œã¦ã„ã‚‹ç¬¦å·åŒ–æ–¹å¼ã®Group Varint Encodingã‚’å®Ÿè£…ã—ã¦ã¿ãŸã€‚ è³‡æ–™ è¬›æ¼”ã‚¹ãƒ©ã‚¤ãƒ‰ ã‚¹ãƒ©ã‚¤ãƒ‰ã®æ—¥æœ¬èªžã«ã‚ˆã‚‹è§£èª¬è¨˜äº‹ æ•´æ•°ã®ç¬¦å·åŒ–æ–¹å¼ è»¢ç½®ã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ãªã©ã§æ–‡ç« ç•ªå·ã®ãƒªã‚¹ãƒˆã‚’å‰ã®å€¤ã¨ã®å·®åˆ†ã§è¡¨ã™ãªã©ã®æ–¹æ³•ã‚’ç”¨ã„ã‚‹ã¨å‡ºç¾ã™ã‚‹ã€ã»ã¨ã‚“ã©ã®å€¤ã¯å°ã•ãªå€¤ã¨ãªã‚‹ãŸã‚ã“ã‚Œã‚’4ãƒã‚¤ãƒˆä½¿ã£ã¦è¡¨ç¾ã™ã‚‹ã®ã¯è¨˜æ†¶å®¹é‡ã®ç„¡é§„ã§ã‚ã‚‹ã€‚ ã“ã®ãŸã‚Varint Encodingã€ã‚¬ãƒ³ãƒžç¬¦å·ã€ãƒ‡ãƒ«ã‚¿ç¬¦å·ã€Rice Codingã€Simple 9ã€pForDeltaãªã©æ§˜ã€…ãªç¬¦å·åŒ–æ–¹å¼ãŒææ¡ˆã•ã‚Œã¦ã„ã‚‹ã€‚ã“ã®ã†ã¡Varint Encodingã¯å®Ÿè£…ãŒæ‰‹è»½ãªã“ã¨ã‹ã‚‰ã‚ˆãç”¨ã„ã‚‰ã‚Œã¦
yass 2012/11/11
compression

google

integer

encoding

group varint encoding

varint
ãƒªãƒ³ã‚¯
ç¬¬11å›žã€€è»¢ç½®ç´¢å¼•ã®åœ§ç¸® | gihyo.jp
ã¯ã˜ã‚ã« ç¬¬2å›žã§ã€ç´¢å¼•ã¯å¤šãã®å ´åˆåœ§ç¸®ã•ã‚Œã¦ã„ã‚‹ã“ã¨ã«è¨€åŠã—ã¾ã—ãŸã€‚ã¾ãŸç¬¬7å›žã§ã¯ã€ç´¢å¼•æ§‹ç¯‰æ™‚ã«ã©ã®éƒ¨åˆ†ã§ç´¢å¼•ã‚’åœ§ç¸®ã™ã‚Œã°ã‚ˆã„ã‹ã‚’ç–‘ä¼¼ã‚³ãƒ¼ãƒ‰ã‚’ç”¨ã„ã¦èª¬æ˜Žã—ã¾ã—ãŸã€‚ä»Šå›žã¯ã€è»¢ç½®ç´¢å¼•ã®å…·ä½“çš„ãªåœ§ç¸®æ–¹æ³•ã«ã¤ã„ã¦èª¬æ˜Žã—ã¦ã„ãã¾ã™ã€‚ åœ§ç¸®ã®ç›®çš„ ä¸è¦æ¨¡ã‹ã‚‰å¤§è¦æ¨¡ãªç´¢å¼•ã®å ´åˆã€è»¢ç½®ãƒªã‚¹ãƒˆã¯éžå¸¸ã«é•·ããªã‚Šã€æ¤œç´¢æ™‚ã«ã¯ãƒ‡ã‚£ã‚¹ã‚¯ã‹ã‚‰ã®å¤§é‡ã®ãƒ‡ãƒ¼ã‚¿ã®èªã¿å–ã‚ŠãŒè¡Œã‚ã‚Œã¾ã™ã€‚è»¢ç½®ç´¢å¼•ï¼ˆã‚’ç”¨ã„ãŸæ¤œç´¢ã‚¨ãƒ³ã‚¸ãƒ³ï¼‰ã§ã¯ã€ã“ã‚Œã«ã‚ˆã‚‹æ¤œç´¢å‡¦ç†æ™‚é–“ã®å¢—åŠ ã‚’é˜²ããŸã‚ã«ã€è»¢ç½®ãƒªã‚¹ãƒˆã‚’åœ§ç¸®ã—ãƒ‡ã‚£ã‚¹ã‚¯ã‹ã‚‰ã®èªã¿è¾¼ã¿æ™‚é–“ã®çŸç¸®ã‚’å›³ã‚Šã¾ã™ã€‚ ã“ã®å ´åˆã€åœ§ç¸®ã•ã‚ŒãŸè»¢ç½®ãƒªã‚¹ãƒˆã‚’ãƒ‡ã‚£ã‚¹ã‚¯ã‹ã‚‰èªã¿è¾¼ã¿ã•ã‚‰ã«å¾©å…ƒå‡¦ç†ã‚’è¡Œã†å¿…è¦ãŒã‚ã‚Šã¾ã™ãŒã€é€šå¸¸ã¯æ¬¡ã®ã‚ˆã†ã«ãªã‚Šã¾ã™ã€‚ ã“ã‚Œã¯ã€è¿‘å¹´ã®CPUã¨ãƒ‡ã‚£ã‚¹ã‚¯ã®é€Ÿåº¦å·®ãŒå¤§ãã„ãŸã‚ã€ä¸»ã«CPUã«ãŠã‘ã‚‹å‡¦ç†ã§ã‚ã‚‹å¾©å…ƒå‡¦ç†ãŒé«˜é€Ÿã«è¡Œãˆã‚‹ã“ã¨ã«ã‚ˆã‚‹ã‚‚ã®ã§ã™ã€‚ã‚ˆã£ã¦ã€åœ§ç¸®ã¨ã„ã†ã¨å®¹é‡ã‚’ç¯€ç´„ã®æ„å›³ã§ä½¿ã†ã“ã¨ãŒå¤šã„ã¨æ€ã„ã¾
yass 2012/11/11
PForDelta

compression

encoding

gamma coding

simple9

vByte

inverted index

integer
ãƒªãƒ³ã‚¯
Google Code Archive - Long-term storage for Google Code Project Hosting.
Code Archive Skip to content Google About Google Privacy Terms
yass 2012/11/11
compression

integer

encoding

PForDelta

simple9

vByte
ãƒªãƒ³ã‚¯
Simple-9ã«ã¤ã„ã¦è§£èª¬ - tsubosakaã®æ—¥è¨˜
å‰å›žã«å¼•ãç¶šãè»¢ç½®ã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ã®åœ§ç¸®ã‚’å®Ÿè£…ã—ã¦ã¿ã‚‹ã€‚ä»Šå›žç´¹ä»‹ã™ã‚‹ã®ã¯[2]ã§ææ¡ˆã•ã‚Œã¦ã„ã‚‹Simple-9ã¨ã„ã†ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã§ã‚ã‚‹ã€‚ Simple-9ã¯32bitã®wordã«ã§ãã‚‹ã ã‘æ•°å—ã‚’è©°ã‚ã¦ã„ãã¨ã„ã†åœ§ç¸®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã§ã‚ã‚‹ã€‚ä¾‹ãˆã°2bitã®æ•°ãŒ16å€‹ãªã‚‰ã‚“ã§ã„ã‚Œã°32bitã§è¡¨ç¾ã§ãã‚‹ã€‚ã—ã‹ã—ã€å®Ÿéš›ã¯å¤§ãã„æ•°å—ã‚‚å‡ºç¾ã™ã‚‹ãŸã‚æ•°å—ã®é•·ã•ã®æƒ…å ±ã‚‚æ ¼ç´ã™ã‚‹å¿…è¦ãŒã‚ã‚‹ã€‚Simple-9ã§ã¯4bitã‚’ç”¨ã„ã¦æ®‹ã‚Šã®28bitãŒã©ã†è©°ã‚ã‚‰ã‚Œã¦ã„ã‚‹ã‹ã‚’è¡¨ã™ã€‚ 28bitã®è¡¨ã—æ–¹ã¨ã—ã¦ã¯ ä¸Šä½bit ç¬¦å·ã®å€‹æ•° ç¬¦å·ã®ãƒ“ãƒƒãƒˆé•· 0000 28 1 0001 14 2 0010 9 3 0011 7 4 0100 5 5 0101 4 7 0110 3 9 0111 2 14 1000 1 28 ã®9é€šã‚ŠãŒã‚ã‚Šã€ã“ã‚ŒãŒSimple-9ã®åå‰ã®ç”±æ¥ã¨ãªã£ã¦ã„ã‚‹ã€‚ ä¾‹ãˆã° ( 3 , 5 , 0 , 0 ,
yass 2012/11/11
" 32bitã®wordã«ã§ãã‚‹ã ã‘æ•°å—ã‚’è©°ã‚ã¦ã„ã / ä¾‹ãˆã°2bitã®æ•°ãŒ16å€‹ãªã‚‰ã‚“ã§ã„ã‚Œã°32bitã§è¡¨ç¾ã§ãã‚‹ / æ•°å—ã®é•·ã•ã®æƒ…å ±ã‚‚æ ¼ç´ã™ã‚‹å¿…è¦ãŒã‚ã‚‹ã€‚Simple-9ã§ã¯4bitã‚’ç”¨ã„ã¦æ®‹ã‚Šã®28bitãŒã©ã†è©°ã‚ã‚‰ã‚Œã¦ã„ã‚‹ã‹ã‚’è¡¨ã™ "

integer

encoding

simple9

compression
ãƒªãƒ³ã‚¯
http://hpc.isti.cnr.it/
yass 2012/11/10
integer

encoding

algorithm

PForDelta

simple9

delta coding

comparison

gamma coding

vByte
ãƒªãƒ³ã‚¯
å·®åˆ†ç¬¦å·åŒ– - Wikipedia
å·®åˆ†ç¬¦å·åŒ–ï¼ˆã•ã¶ã‚“ãµã”ã†ã‹ã€è‹±: Delta encodingï¼‰ã¨ã¯ã€ãƒ‡ãƒ¼ã‚¿ã®æ ¼ç´ã‚„è»¢é€ã‚’å®Œå…¨ãªãƒ•ã‚¡ã‚¤ãƒ«ã¨ã—ã¦ã§ã¯ãªãã€ã‚·ãƒ¼ã‚±ãƒ³ã‚·ãƒ£ãƒ«ãªãƒ‡ãƒ¼ã‚¿ã®å·®åˆ†ã®å½¢å¼ã§è¡Œã†æ–¹å¼ã§ã‚ã‚‹ã€‚ç‰¹ã«å¤‰æ›´å±¥æ´ã®ä¿å˜ã‚’ç›®çš„ã¨ã™ã‚‹å ´åˆï¼ˆã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆãªã©ï¼‰ã€å·®åˆ†ç¬¦å·åŒ–ã¯å·®åˆ†åœ§ç¸®ï¼ˆè‹±: Delta compressionï¼‰ã¨ã‚‚å‘¼ã°ã‚Œã‚‹ã€‚ãƒ‡ãƒ«ã‚¿ç¬¦å·åŒ–ã€ãƒ‡ãƒ«ã‚¿åœ§ç¸®ã¨ã‚‚å‘¼ã°ã‚Œã‚‹ãŒã€ãƒ‡ãƒ«ã‚¿ç¬¦å·ã¨ã¯ç•°ãªã‚‹ã€‚ ä¾‹ãˆã°UNIXã®ãƒ•ã‚¡ã‚¤ãƒ«æ¯”è¼ƒãƒ¦ãƒ¼ãƒ†ã‚£ãƒªãƒ†ã‚£ã§ã‚ã‚‹ diff ãªã©ã§ã€Œå·®åˆ†ã€ã¾ãŸã¯ã€Œãƒ‡ãƒ«ã‚¿ã€ã‚’ä½œæˆã—ã€å€‹åˆ¥ã«ãƒ•ã‚¡ã‚¤ãƒ«ã¨ã—ã¦è¨˜éŒ²ã™ã‚‹ã€‚å·®åˆ†ã¯ä¸€èˆ¬ã«å…ƒã®ãƒ•ã‚¡ã‚¤ãƒ«ã‚ˆã‚Šã‚‚å°ã•ã„ã®ã§ã€å·®åˆ†ç¬¦å·åŒ–ã«ã‚ˆã£ã¦ãƒ‡ãƒ¼ã‚¿ã®å†—é•·æ€§ã‚’å¤§å¹…ã«å‰Šæ¸›ã§ãã‚‹ã€‚ä¸€é€£ã®å·®åˆ†ãƒ•ã‚¡ã‚¤ãƒ«ã®æ–¹ãŒå„ãƒãƒ¼ã‚¸ãƒ§ãƒ³ã®ãã®ã¾ã¾ã®ãƒ•ã‚¡ã‚¤ãƒ«ç¾¤ã‚ˆã‚Šã‚‚å¤§å¹…ã«è¨˜éŒ²å®¹é‡ãŒç¯€ç´„ã§ãã‚‹ã€‚ è«–ç†çš„è¦³ç‚¹ã‹ã‚‰è¨€ãˆã°ã€2ã¤ã®ãƒ‡ãƒ¼ã‚¿ã®å·®åˆ†ãŒã‚ã‚Œã°ã€ä¸€æ–¹ã®ãƒ‡ãƒ¼ã‚¿ã‹ã‚‰ã‚‚ã†ä¸€æ–¹ã®ãƒ‡ãƒ¼ã‚¿ã‚’å¾—ã‚‹ã“ã¨ãŒã§ãã‚‹
yass 2012/11/10
algorithm

delta encoding

encoding

integer
ãƒªãƒ³ã‚¯
1 2 æ¬¡ã®ãƒšãƒ¼ã‚¸