注目コメント算出アルゴリズムの一部にLINEヤフー株式会社の「建設的コメント順位付けモデルAPI」を使用しています
David, I understood "25M + page/day" was total for 5 machines, as you wrote "(these numbers are t... David, I understood "25M + page/day" was total for 5 machines, as you wrote "(these numbers are totals of all 5 instances combined)". Did you mean 25M+ page/day/instance? If so, my "100 threads" comment is pointless. Please disregard it. I'm running broad crawl that captures everything linked: images, script, CSS, PDF, Excel, ... even mpeg4 videos. Our Heritrix 3 runs on 8GB memory + 4 core virtua
2012/06/22 リンク