[B! mmap] yassã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

yass id:yass

mmapã«é–¢ã™ã‚‹yassã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (25)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

read(2) vs mmap(2) ã®è¿·ä¿¡
ã“ã®è¨˜äº‹ã¯ã€ã‚·ã‚§ãƒ«ã‚¹ã‚¯ãƒªãƒ—ãƒˆã®è¨˜äº‹ã‚ˆã‚Šã‚‚å‰ã«èªã‚“ã ã‚ˆã†ãªæ°—ãŒã™ã‚‹ã€‚åŒã˜ãã‚‰ã„å¤ã„è¨˜äº‹ã ã‘ã‚Œã©ã€ã¡ã‚‡ã£ã¨æ›¸ã„ã¦ã¿ã‚‹ã€‚è¨˜äº‹ã¸ã®åè«–ã¯ã„ãã¤ã‹æ¤œç´¢ã™ã‚‹ã¨è¦‹ã¤ã‹ã£ã¦ã€ãŸã¶ã‚“ã¿ã‚“ãªçŸ¥ã£ã¦ã„ã‚‹ã“ã¨ãªã®ã ã¨æ€ã†ã‘ã‚Œã©ã€ã¾ã¨ã¾ã£ã¦æ›¸ã‹ã‚Œã¦ã„ã‚‹æ–‡ç« ã¯ãªã„ã¿ãŸã„ã€‚ tl;dr read(2) ã¨ mmap(2) ã®æ€§èƒ½å·®ã«çµ¶å¯¾çš„ãªå›žç”ã¯ãªã„ã€‚ã©ã¡ã‚‰ã‹ä¸€æ–¹ãŒå¸¸ã«é«˜é€Ÿã ã¨ä¸»å¼µã—ã¦ã„ã‚‹æ–‡ç« ã¯ã€æ ¹æ‹ ãŒè¨¼æ‹ ã¨ã¨ã‚‚ã«æ˜Žç¢ºã«æ›¸ã‹ã‚Œã¦ã„ãªã„é™ã‚Šä¿¡ç”¨ã—ãªã„ã»ã†ãŒè‰¯ã„ã€‚ ãƒ¡ãƒ¢ãƒªã‚³ãƒ”ãƒ¼ã®ã‚³ã‚¹ãƒˆãŒé«˜ã‹ã£ãŸæ™‚ä»£ã¨ã€L1ã‚ãƒ£ãƒƒã‚·ãƒ¥ãŒå·¨å¤§ã«ãªã£ã¦ãƒ¡ãƒ¢ãƒªã‚³ãƒ”ãƒ¼ã®ã‚³ã‚¹ãƒˆãŒä½Žããªã£ãŸæ™‚ä»£ã¨ã€SMPãŒä¸€èˆ¬çš„ã«ãªã£ã¦ãƒ¡ãƒ¢ãƒªã®ãƒžãƒƒãƒ”ãƒ³ã‚°å‡¦ç†ã®ã‚³ã‚¹ãƒˆãŒé«˜ããªã£ãŸæ™‚ä»£ã¨ã§ã€ã“ã®æ€§èƒ½å·®ã¯é »ç¹ã«å…¥ã‚Œæ›¿ã‚ã£ã¦ã„ã‚‹ã€‚å°‘ãªãã¨ã‚‚ã‚¹ãƒ«ãƒ¼ãƒ—ãƒƒãƒˆã¨ãƒ¬ã‚¤ãƒ†ãƒ³ã‚·ã‚’åˆ†ã‘ãªã„ã§åˆ†æžã§ãã‚‹ã‚‚ã®ã§ã¯ãªã„ã€‚ ã¾ãšå½“è©²è¨˜äº‹ã«ã¯æŠ€è¡“çš„ãªé–“é•ã„ãŒã„ãã¤ã‹ã‚ã‚‹ã€‚ ã€Œmmap()ã¯ãƒ¦ãƒ¼ã‚¶ãƒ©ãƒ³ãƒ‰
yass 2017/01/13
mmap

syscall

TLB
ãƒªãƒ³ã‚¯
Mappedbus: A library for low latency IPC between multiple Java processes/JVMs
A library for low latency IPC between multiple Java processes/JVMs Mappedbus is a Java based high throughput, low latency message bus, using either a memory mapped file or shared memory as transport Mappedbus was inspired by Java Chronicle with the main difference that it's designed to efficiently support multiple writers â€“ enabling use cases where the order of messages produced by multiple proces
yass 2015/07/03
java

mmap

ipc
ãƒªãƒ³ã‚¯
StefanKarpinski/bam Â· GitHub - Bam: fast, simple TSV key-value server
yass 2015/01/31
kvs

http

mmap

perfect hash

tsv

minimal perfect hash
ãƒªãƒ³ã‚¯
ç¶š:memory mapped file ãŒæ¶ˆã›ãªã„ - odz buffer
memory mapped file ãŒæ¶ˆã›ãªã„ã®ç¶šãã€‚ ãªã‚“ã‹ã€Java APIãƒ‰ã‚ãƒ¥ãƒ¡ãƒ³ãƒˆã®FileChannel#map ã®é …ç›®ã‚’è‰¯ãè¦‹ã‚‹ã¨ã€ ãƒãƒƒãƒ•ã‚¡ã¨ã“ã®ãƒãƒƒãƒ•ã‚¡ãŒç¤ºã™ãƒžãƒƒãƒ”ãƒ³ã‚°ã¯ã€ãƒãƒƒãƒ•ã‚¡è‡ªä½“ãŒã‚¬ãƒ™ãƒ¼ã‚¸ã‚³ãƒ¬ã‚¯ãƒˆã•ã‚Œã‚‹ã¾ã§æœ‰åŠ¹ã§ã™ã€‚ ã¨ã‹æ›¸ã„ã¦ã‚„ãŒã‚‹ã€‚ã§ã€è©¦ã—ã« File#delete ã‚’ call ã™ã‚‹å‰ã«æ˜Žç¤ºçš„ã« GC ã™ã‚‹ã‚ˆã†ã«ã—ã¦ã¿ãŸã€‚ File = new File("foo.txt"); FileInputStream in = new FileInputStream(file); FileChannel channel = in.getChannel(); MappedByteBuffer buffer = channel.map(FileChannel.MapMode.READ_ONLY, 0, file.length()); channel.close(); b
yass 2015/01/03
" è¦ã™ã‚‹ã«ãƒžãƒƒãƒ”ãƒ³ã‚°ã‚’ç¢ºå®Ÿã«é–‹æ”¾ã™ã‚‹ã«ã¯å¼·åˆ¶çš„ã«Â GCÂ ã—ã‚ã¨ã€‚"

mmap

java
ãƒªãƒ³ã‚¯
ç´°ã‹ã™ãŽã¦ä¼ã‚ã‚‰ãªã„mdbm
ãŠã¾ã‘è©±ã¨ã—ã¦ã€mdbmã¯Linear Hashingã¨å‘¼ã°ã‚Œã‚‹ãƒãƒƒã‚·ãƒ¥ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã®å½±éŸ¿ã‚’å¼·ãå—ã‘ã¦ã„ã¾ã™ã€‚ Linear Hashingã®è©³ç´°ã¯wikipediaã‚’ã”è¦§ãã ã•ã„ã€‚ http://en.wikipedia.org/wiki/Linear_hashing ã“ã®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã«ã‚ˆã‚Šmdbmã¯ã€æ‰±ã†ãƒ‡ãƒ¼ã‚¿ã‚µã‚¤ã‚ºãŒå¤§ãããªã‚Œã°ã€å‹•çš„ã«HashTableã‚’æ‹¡å¤§ã™ã‚‹ã“ã¨ãŒã§ãã‚‹éžå¸¸ã«ä¾¿åˆ©ãªç‰¹æ€§ã‚’æŒã£ã¦ã„ã¾ã™ã€‚ ã—ã‹ã—ã€å†·é™ã«ãªã£ã¦è€ƒãˆã¦ã¿ã¦ã¿ã¾ã—ã‚‡ã†ã€‚ã“ã®Linear Hasingã®ç®¡ç†ç”¨ã®ãƒ†ãƒ¼ãƒ–ãƒ«ã‚’èµ°æŸ»ã™ã‚‹è¨ˆç®—ã‚³ã‚¹ãƒˆã¯å¯èƒ½ãªã‚‰é¿ã‘ã‚‹ã¹ãã§ã™ã€‚ mdbmã‚’ã¯ã˜ã‚ã€å¤šãã®KVSã§ã¯æœ€çµ‚çš„ãªãƒ‡ãƒ¼ã‚¿ã®ã‚µã‚¤ã‚ºã®äºˆæƒ³ãŒã¤ãã®ã§ã‚ã‚Œã°ã€ã‚ã‚‰ã‹ã˜ã‚å¤§ãã‚ã®ã‚µã‚¤ã‚ºã§ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ãƒ•ã‚¡ã‚¤ãƒ«ã‚’ä½œæˆã™ã‚‹æ–¹ãŒå¥½ã¾ã—ã„ã§ã—ã‚‡ã†ã€‚ ã“ã®è¾ºã®è©±ã«èˆˆå‘³ãŒã‚ã‚Šã¾ã—ãŸã‚‰ã€ã‚³ãƒ¼ãƒ‰ã®ã€Œhashval_to_pagenum()ã€
yass 2014/06/23
" mdbmã¯Linear Hashingã¨å‘¼ã°ã‚Œã‚‹ãƒãƒƒã‚·ãƒ¥ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã®å½±éŸ¿ã‚’å¼·ãå—ã‘ã¦ã„ã¾ã™ / ã“ã®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã«ã‚ˆã‚Šmdbmã¯ã€æ‰±ã†ãƒ‡ãƒ¼ã‚¿ã‚µã‚¤ã‚ºãŒå¤§ãããªã‚Œã°ã€å‹•çš„ã«HashTableã‚’æ‹¡å¤§ã™ã‚‹ã“ã¨ãŒã§ãã‚‹éžå¸¸ã«ä¾¿åˆ©ãªç‰¹æ€§ã‚’æŒã£ã¦ã„ã¾ã™ "

kvs

yahoo

mdbm

tlb

hugepage

mmap

tmpfs

linear hashing

cdb
ãƒªãƒ³ã‚¯
Intro to IPC | Interprocess Communication
Interprocess communication is defined as follows on Wikipedia: InÂ computing,Â inter-process communicationÂ (IPC) is a set of methods for the exchange of data among multipleÂ threadsÂ in one or moreÂ processes. Processes may be running on one or more computers connected by aÂ network. More or less it is exactly what you would expect, the transfer of data between processes. This may soundÂ straight forward
yass 2014/05/08
ipc

process

mmap

pipe
ãƒªãƒ³ã‚¯
Off Heap concurrent counter
Concurrent counter are part of almost every system, it is used to collect data, thread synchronization etc. Java has good support of heap based counter. There could be use case when you need counter that can be shared between processor. How to build inter process countersDatabaseThis is first option that comes to mind, database sequence is counter that can be used by multiple process. All concurre
yass 2014/04/20
java

offheap

concurrency

mmap

CAS

counter
ãƒªãƒ³ã‚¯
ArrayList Using Memory Mapped File
yass 2014/04/20
java

mmap

list
ãƒªãƒ³ã‚¯
Memory performance comparison in java - Senior Java Developer
yass 2014/04/19
" If you need a large set of data (more than 2GB) and you're using linux - choice SHM without any doubts.Â "

java

offheap

mmap

benchmark

unsafe

tmpfs
ãƒªãƒ³ã‚¯
HDFSãŒé«˜é€Ÿã«ï¼Ÿmmapã«ã‚ˆã‚‹zero-copyã§ã®èªã¿è¾¼ã¿
é€šå¸¸ã€ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã¯read()ãªã©ã®ã‚·ã‚¹ãƒ†ãƒ ã‚³ãƒ¼ãƒ«çµŒç”±ã§ãƒ•ã‚¡ã‚¤ãƒ«ã‚’èªã¿å‡ºã—ã¾ã™ã€‚ ã“ã®HDFS-4953ã¯mmap()ã‚·ã‚¹ãƒ†ãƒ ã‚³ãƒ¼ãƒ«ã‚’ä½¿ç”¨ã™ã‚‹ã“ã¨ã§ã€èªã¿å–ã‚Šæ™‚ã«ã‹ã‹ã‚‹ã‚ªãƒ¼ãƒãƒ¼ãƒ˜ãƒƒãƒ‰ã‚’æ¸›ã‚‰ãã†ã¨ã„ã†ã‚‚ã®ã®ã‚ˆã†ã§ã™ã€‚ å‚è€ƒã¾ã§ã«ã€é€šå¸¸ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ãŒãƒ•ã‚¡ã‚¤ãƒ«ã‚’èªã¿å‡ºã™å ´åˆã€ä»¥ä¸‹ã®ã‚ˆã†ãªãƒ•ãƒãƒ¼ã§ã‚«ãƒ¼ãƒãƒ«ã‹ã‚‰ã®èªã¿è¾¼ã¿å‡¦ç†ãŒè¡Œã‚ã‚Œã¾ã™ã€‚ ã‚¢ãƒ—ãƒªã‹ã‚‰ã®èªã¿è¾¼ã¿è¦æ±‚ v fread()ãªã© (stdlib) v read()ã‚·ã‚¹ãƒ†ãƒ ã‚³ãƒ¼ãƒ«(glibc) v ï¼ˆä»¥ä¸‹ã‚«ãƒ¼ãƒãƒ«ç©ºé–“ï¼‰ v sys_read() v vfs_read() v â€¦. å‚è€ƒè³‡æ–™ï¼šãƒšãƒ¼ã‚¸ã‚ãƒ£ãƒƒã‚·ãƒ¥ã®ãƒ¡ãƒ¢ P.12 ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã‹ã‚‰ã®èªã¿å‡ºã—è¦æ±‚ã«ã‚ˆã‚Šã‚·ã‚¹ãƒ†ãƒ ã‚³ãƒ¼ãƒ«ãŒå‘¼ã°ã‚Œã‚‹ã®ã¯ä¸Šè¨˜ã®é€šã‚Šã§ã™ãŒã€å•é¡Œã¨ãªã‚‹ã®ã¯ã€ read()ãŒé »ç¹ã«å‘¼ã³ã ã•ã‚Œã‚‹å ´åˆã€ã‚³ãƒ³ãƒ†ã‚ã‚¹ãƒˆã‚¹ã‚¤ãƒƒãƒãŒå¤šãç™ºç”Ÿã—ã¦ã‚³ã‚¹ãƒˆãŒã‹ã‹ã‚‹â€»ã‚³ã‚¹ãƒˆã®è¨ˆæ¸¬ã«ã¯ s
yass 2014/02/20
hadoop

hdfs

mmap
ãƒªãƒ³ã‚¯
http://www.tbray.org/ongoing/When/200x/2003/03/22/Binary
yass 2013/10/30
" better use of your CPU's I-cache "

binary search

mmap

cache
ãƒªãƒ³ã‚¯
ãƒ¡ãƒ¢ãƒªç®¡ç†
ãƒ¡ãƒ¢ãƒªãƒžãƒƒãƒ—ãƒ‰ãƒ•ã‚¡ã‚¤ãƒ« ã‚ªãƒšãƒ¬ãƒ¼ãƒ†ã‚£ãƒ³ã‚°ã‚·ã‚¹ãƒ†ãƒ http://funini.com/kei/mmap/ ä»Šæ—¥ã®æµã‚Œ (12/10) ï®ãƒ‡ã‚–ã‚¹ã‚¯ã®è©±ã®æ®‹ã‚Š ïµãƒ‡ã‚–ã‚¹ã‚¯ã‚’é«˜é€Ÿã«ä½¿ã†å·¥å¤« ï®ãƒ¡ãƒ¢ãƒªã¨ãƒ‡ã‚–ã‚¹ã‚¯ã®ç°¡å˜ãªã¾ã¨ã‚ ï®ãƒ¡ãƒ¢ãƒªãƒžãƒƒãƒ—ãƒˆãƒ»ãƒ•ã‚”ã‚—ãƒ« (mmap) http://funini.com/kei/mmap/ ï®ãƒ‡ã‚–ã‚¹ã‚¯ (ãƒãƒ¼ãƒ‰ãƒ‡ã‚–ã‚¹ã‚¯, DVDãªã©) ïµé›»æºã‚’åˆ‡ã£ã¦ã‚‚æ¶ˆãˆãªã„ ïµç‰©ç†çš„ã«ã¯ã‚·ãƒªãƒ³ãƒ€ãƒ»ãƒ–ãƒãƒƒã‚¯ã«åˆ†ã‹ã‚Œã¦ã„ã‚‹ ïµOSã«ã‚ˆã£ã¦æŠ½è±¡åŒ–ã•ã‚Œï¼Œãƒ•ã‚”ã‚—ãƒ«å˜ä½ã§ãƒ‡ãƒ¼ã‚¿ ã‚’ç®¡ç†ã§ãã‚‹ ï®ã‚•ã‚¯ã‚»ã‚¹ã¯ãƒ¡ãƒ¢ãƒªã«æ¯”ã¹ã¦é…ã„ ïµâ†’é«˜é€ŸåŒ–ã™ã‚‹å·¥å¤« ãƒ‡ã‚£ã‚¹ã‚¯ã«ã¤ã„ã¦ (å‰å›žã®ç¶šã) http://funini.com/kei/mmap/ é€£ç¶šã—ãŸé ˜åŸŸã¸ã®å‰²ã‚Šå½“ã¦ ï®ä¸€åº¦ã«èªã¿å‡ºã™ã®ã«éƒ½åˆã®è‰¯ã„ãƒ–ãƒãƒƒã‚¯ (ä¾‹: åŒã˜ã‚·ãƒªãƒ³ãƒ€(å††å‘¨)å†…ã®å…¨ãƒ–ãƒãƒƒã‚¯)ã« ãƒ•ã‚”ã‚—ãƒ«ã®é€£ç¶šã—ãŸé ˜åŸŸã‚’å‰²ã‚Šå½“ã¦ã‚‹ ïµcf. ã„
yass 2013/10/26
mmap
ãƒªãƒ³ã‚¯
GitHub - gstrauss/mcdb: mcdb - fast, reliable, simple code to create and read constant databases
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
yass 2013/10/25
" mcdb (mmap constant database) is originally based on the cdb package, a: "fast, reliable, simple package for creating and reading constant databases." mcdb is almost 33% faster, provides support for use in threaded programs, and supports databases larger than 4 GB."

cdb

mcdb

mmap

embedded
ãƒªãƒ³ã‚¯
The Design of A Big, Fast, and Persistent Queue - Abstraction Builder
yass 2013/10/19
" Suppose the average message size is 1KB, then big queue can concurrently producing and consuming 166K message per second on a commodity machine / the throughput is only limited by disk IO bandwidth. / automatic paging & swapping algorithm, only most-recently accessed data is kept in memory. "

queue

mmap

io

file

cache

LRU
ãƒªãƒ³ã‚¯
sortdb - static key value database
As dataset sizes grow, you need to buy larger/faster database machines, or shard data across multiple machines. sortdb is a database server we have written to help fill a specific need for performant access to static data. One observation about many datasets is that there is often a small dataset that is heavily updated, and a larger older related dataset that does not change. It is this second ca
yass 2013/10/18
database

http

kvs

csv

sort

binary search

mmap

cdb
ãƒªãƒ³ã‚¯
mmapã®ã»ã†ãŒreadã‚ˆã‚Šé€Ÿã„ã¨ã„ã†è¿·ä¿¡ã«ã¤ã„ã¦ - kazuhoã®ãƒ¡ãƒ¢ç½®ãå ´
@ITã«ä»¥ä¸‹ã®ã‚ˆã†ãªè¨˜äº‹ãŒå‡ºã¦ã€ ä»Šå›žã‹ã‚‰ã—ã°ã‚‰ãã®é–“ã¯ã€ã¾ã£ãŸãé€†ã®ä¾‹ã€ã¤ã¾ã‚Šä½¿ã†ã¨ãƒ—ãƒã‚°ãƒ©ãƒ ã®å‡¦ç†æ€§èƒ½ãŒä¸ŠãŒã‚‹ã¨ã„ã†ã‚·ã‚¹ãƒ†ãƒ ã‚³ãƒ¼ãƒ«ã‚’ç´¹ä»‹ã—ã¦ã„ãã€‚ã‚·ã‚¹ãƒ†ãƒ ã‚³ãƒ¼ãƒ«ã‚’å‘¼ã¶å›žæ•°ã¯å°‘ãªã„æ–¹ãŒå‡¦ç†æ€§èƒ½ã¯é«˜ããªã‚‹ã¨ã„ã†åŽŸå‰‡ã¯å¤‰ã‚ã‚‰ãªã„ãŒã€å‘¼ã³å‡ºã—ã¦ãŠãã¨å‡¦ç†æ€§èƒ½ãŒå‘ä¸Šã™ã‚‹ã‚·ã‚¹ãƒ†ãƒ ã‚³ãƒ¼ãƒ«ã¨ã„ã†ã‚‚ã®ãŒå˜åœ¨ã™ã‚‹ã®ã ã€‚ã“ã†ã—ãŸã‚·ã‚¹ãƒ†ãƒ ã‚³ãƒ¼ãƒ«ã‚’ä½¿ã‚ãªã„ã§ã„ã‚‹ã“ã¨ã¯ã€ã¨ã¦ã‚‚ã‚‚ã£ãŸã„ãªã„ã€‚ ä»Šå›žç´¹ä»‹ã™ã‚‹ã‚·ã‚¹ãƒ†ãƒ ã‚³ãƒ¼ãƒ«ã¯ã€Œmmap(2)ã€ã ã€‚ã“ã“ã§ã¯è©³ã—ãä»•çµ„ã¿ã‚’è§£èª¬ã—ãªã„ãŒã€mmap(2)ã¯ã€ãƒ—ãƒã‚°ãƒ©ãƒ ã®å‡¦ç†æ€§èƒ½ã«å¿…ãšè‰¯ã„å½±éŸ¿ã‚’ä¸Žãˆã‚‹ã€‚ ã‚„ã¯ã‚Šã‚ã£ãŸï¼Ÿ é«˜é€ŸåŒ–ã«åŠ¹ãã‚·ã‚¹ãƒ†ãƒ ã‚³ãƒ¼ãƒ« (1/2)ï¼šçŸ¥ã£ã¦ãƒˆã‚¯ã™ã‚‹ã‚·ã‚¹ãƒ†ãƒ ã‚³ãƒ¼ãƒ«ï¼ˆ3ï¼‰ - ï¼ IT ãã‚Œã‚’çœŸã«å—ã‘ãŸã®ã‹ã€ã€ŒGoè¨€èªžã§mmapã‚·ã‚¹ãƒ†ãƒ ã‚³ãƒ¼ãƒ«ã‚’ä½¿ã£ãŸãƒ•ã‚¡ã‚¤ãƒ«èªã¿è¾¼ã¿ã®é«˜é€ŸåŒ–æ¤œè¨Žã¨Cè¨€èªžã®ã‚³ãƒ³ãƒ‘ã‚¤ãƒ©ã®è©± - ryochack.blogã€ã®ã‚ˆã†ãªãƒ–ãƒã‚°ã‚¨ãƒ³ãƒˆãƒªã‚‚
yass 2013/10/10
" æ£ã—ã„ã‚³ãƒ¼ãƒ‰ã‚’æ›¸ã‘ã°ã€ã‚·ãƒ¼ã‚±ãƒ³ã‚·ãƒ£ãƒ«ã‚¢ã‚¯ã‚»ã‚¹ã‚’è¡Œã†ã‚±ãƒ¼ã‚¹ã«ãŠã„ã¦ read(2) ãŒ mmap(2) ã‚ˆã‚Šå¤§å¹…ã«é…ã„ã¨ã„ã†ã“ã¨ã¯ã€ã¾ãšèµ·ã“ã‚‰ãªã„ã€‚ã‚€ã—ã‚ã€read(2) ã®ã»ã†ãŒ mmap(2) ã‚ˆã‚Šã‚‚é€Ÿããªã‚‹ã¨ã„ã†ã‚±ãƒ¼ã‚¹ã‚‚å®Ÿéš›ã«ã¯å¤šã„ã€‚"

mmap

linux

kernel
ãƒªãƒ³ã‚¯
Map<D,B>
Map<D,B>Map DB provides Java Maps, Sets, Lists, Queues and other collections backed by off-heap or on-disk storage. It is a hybrid between java collection framework and embedded database engine. It is free and open-source under Apache license.
yass 2013/07/30
java

offheap

database

memory

hashtable

mapdb

mmap

collection
ãƒªãƒ³ã‚¯
SQLite Now Faster With Memory Mapped I/O - Phoronix
SQLite Now Faster With Memory Mapped I/O Written by Michael Larabel in Free Software on 21 May 2013 at 04:31 PM EDT. 5 Comments SQLite 3.7.17 was released yesterday. What makes this new release of the popular lightweight SQL database software noteworthy is that it introduces support for memory-mapped I/O. Rather than using read/write system calls for dealing with database disk files, SQLite 3.7.17
yass 2013/05/24
sqlite

mmap
ãƒªãƒ³ã‚¯
GitHub - xerial/larray: Large off-heap arrays and mmap files for Scala and Java
LArray can create arrays with more than 2G(2^31) entries. 2^31 -1 (2G) is the limitation of the default Java/Scala array size, because these arrays use 32-bit signed integer (int) as indexes. LArray uses long type indexes of 64-bit signed integers to resolve this limitation. For example, the entire human genome data (3GB) can be stored in LArray. LArray can be released immediately from the memory.
yass 2013/05/03
" A library for managing large off-heap arrays that can hold more than 2G (2^31) entries in Java and Scala. Notably LArray is disposable by calling LArray.free or you can let GC automatically release the memory. LArray can be used to create an mmap (memory-mapped file) whose size is more than 2GB. "

java

offheap

memory

array

gc

scala

unsafe

mmap
ãƒªãƒ³ã‚¯
Day-4 : Too Big to Fail: Top Tips for Massive, Mission-Critical Enteprise Applications - è°·æœ¬ å¿ƒ in ã›ã‚éƒ¨å±‹
å¤§è¦æ¨¡ãƒŸãƒƒã‚·ãƒ§ãƒ³ã‚¯ãƒªãƒ†ã‚£ã‚«ãƒ«ç³»ã‚·ã‚¹ãƒ†ãƒ ã®ãŸã‚ã®Tipsã€‚ ãŸã£ãŸ1åˆ†é…ã‚ŒãŸã ã‘ã§ã€æº€å¸ã«ãªã£ã¦ã¦åº§ã‚Œãªã„ãã‚‰ã„ã®äººæ°—ã‚»ãƒƒã‚·ãƒ§ãƒ³ã§ã—ãŸã€‚ å†…å®¹çš„ã«ã‚‚ã€çŸ¥ã‚‰ãªã‹ã£ãŸTipsãŒå¤šã„ã€å¾—ã‚‹ã‚‚ã®ã®å¤šã„ã‚»ãƒƒã‚·ãƒ§ãƒ³ã§ã—ãŸã€‚ ã“ã®ã‚»ãƒƒã‚·ãƒ§ãƒ³ã§å¯¾è±¡ã«ã™ã‚‹ã‚·ã‚¹ãƒ†ãƒ ã¯ã€ãƒ¡ãƒ¢ãƒªãŒ16ã€œ100GBãã‚‰ã„ã€ ã‚¹ãƒ¬ãƒƒãƒ‰ãŒ10ã€œ100ãã‚‰ã„åŒæ™‚ã«å‹•ãã‚ˆã†ãªè¦æ¨¡ã®ã‚‚ã®ã€‚ 1. Heapã‚’æŠ‘ãˆã‚‹ãŸã‚ã«ã€Composed OOPS (COOPS) ã‚’ä½¿ã„ãªã•ã„ã€‚ æ–¹æ³• : -XX:+UseCompressedOOPS åŠ¹æžœ : ãƒ’ãƒ¼ãƒ—ã«é•·ãæ®‹ã£ã¦ã„ã‚‹æƒ…å ±ã‚’åœ§ç¸®ã™ã‚‹ã€‚å®Ÿéš›2.76GB â†’ 2.27GBã«ãªã£ãŸã€‚ 2. 1ã‚¹ãƒ¬ãƒƒãƒ‰ã§ï¼ˆ1ãƒªã‚¯ã‚¨ã‚¹ãƒˆã§ï¼‰ãŸãã•ã‚“ã®ã‚ªãƒ–ã‚¸ã‚§ã‚¯ãƒˆã‚’ç”Ÿæˆã™ã‚‹å ´åˆã¯ã€NUMAã‚’ä½¿ã„ãªã•ã„ æ–¹æ³• : -XX:+UserNUMA åŠ¹æžœ : GCã®åŠ¹çŽ‡ãŒæ”¹å–„ã™ã‚‹ 3. å…±æœ‰ãƒ¡ãƒ¢ãƒªã‚’è‰¯ã„ãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹ã§èªã¿å‡ºã—ãŸã„ãª
yass 2013/05/03
java

ByteBuffer

heap

performance

gc

hprof

mmap
ãƒªãƒ³ã‚¯
1 2 æ¬¡ã®ãƒšãƒ¼ã‚¸