pixivã®ãƒ‡ãƒ¼ã‚¿ã‚’拾ã£ã¦ãã‚‹Rubyã®ã‚³ãƒ¼ãƒ‰ã‚’書ãã¾ã—㟠- shimobayashiã®æ—¥è¨˜
自分ãŒå¿…è¦ã¨ã—ã¦ã„る分ã—ã‹å®Ÿè£…ã—ã¦ã„ãªã„ã®ã§ã€èª°ã‹ã¡ã‚ƒã‚“ã¨å®Ÿè£…ã—ã¦ãã‚Œã¾ã›ã‚“ã‹ãェ。 #!ruby -Ku require 'rubygems' require 'mechanize' require 'uri' module WWW class Mechanize alias_method :_get, :get def get(*args) sleep 1 puts "get: #{args.join(', ')}" _get(*args) end end end class Pixiv attr_reader :bookmark_new_illust def initialize(pixiv_id, pass) @agent = WWW::Mechanize.new login(pixiv_id, pass) BookmarkNewIllust.agent = @agent Mem
jsAutoPageScraperã¨ã„ã†ã®ã‚’作ã£ãŸ - snippets from shinichitomita’s journal
http://coderepos.org/share/wiki/jsAutoPageScraper æ¦‚è¦ ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ãƒ¬ãƒƒãƒˆé–‹ç™ºãªã©ã«ãŠã„ã¦ã€JavaScriptã§ã®HTMLスクレーピングを支æ´ã—ã¾ã™ã€‚検索çµæžœãªã©ã®è¤‡æ•°ã®ãƒ¬ã‚³ãƒ¼ãƒ‰ã‚’JavaScriptã®é…列ã«å¤‰æ›ã—ã¦å–å¾—ã§ãã¾ã™ã€‚ページングを利用ã—ã¦ã„るサイトã§ã¯ã€æ„è˜ã™ã‚‹ã“ã¨ãªã複数ページã«ã¾ãŸãŒã£ã¦ãƒ¬ã‚³ãƒ¼ãƒ‰ã‚’å–å¾—ã§ãã¾ã™ã€‚ 特徴 クãƒã‚¹ãƒ–ラウザã§å‹•ä½œã—ã¾ã™ã€‚ スクレーピングã™ã‚‹è¦ç´ ã®æŒ‡å®šã«ã¯XPathを使ã„ã¾ã™ã€‚IEã¨ã‹Safari2ã®å ´åˆã¯JavaScript-XPathã‚’ãƒãƒ¼ãƒ‰ã—ã¦ä½¿ã„ã¾ã™ã€‚ AutoPagerizeã¨ã‹LDRizeã®SITEINFOå½¢å¼ï¼ˆä¸€éƒ¨ï¼‰ã‚’利用ã§ãã¾ã™ã€‚ 使ã„æ–¹ 1. jsAutoPageScraperã‚’codereposã‹ã‚‰ãƒã‚§ãƒƒã‚¯ã‚¢ã‚¦ãƒˆã—ã€é©å½“ãªã‚µãƒ¼ãƒã«ã‚¢ãƒƒãƒ—ã—ã¾ã™ svn co http://svn.
[php][PEAR]Services_MixiAPIã¨XML_Feed_Parserを使ã£ã¦ã¿ãŸ | ã†ãˆã¡ã‚‡ã“ï¼ ã¼ã‚ã
id:shimookaã•ã‚“ãŒæŒã¡ãƒã‚¿ã‚’放出ã—ãŸã¨ã®ã“ã¨ã§ã€CodeReposã«è¿½åŠ ã•ã‚ŒãŸã‚ˆã†ãªã®ã§ã€Services_MixiAPIを使ã£ã¦ã¿ã¾ã—ãŸã€‚ ã¨ã‚Šã‚ãˆãšã€å‚考ã«ã—ãŸã‚µã‚¤ãƒˆã‚’å…ˆã«ã¾ã¨ã‚ã¦ãŠãã¾ã™ã€‚ æŒã¡ãƒã‚¿æ”¾å‡º – Do You PHP ã¯ã¦ãª mixiã®APIãŒå¢—ãˆã¦ã„ãŸã®ã§ã€Services_MixiAPI作ã£ã¦ã¿ãŸ – Do You PHP ã¯ã¦ãª Services_MixiAPI（APIドã‚ュメント） 2007-11-13 – doublenegativeã®æ—¥è¨˜ PHP: POSIX – Manual PHP: uniqid – Manual PEAR :: Manual :: XML_Feed_Parser（PEARã®æ—¥æœ¬èªžãƒ‰ã‚ュメント） XML_Feed_Parserã§dc:subjectè¦ç´ ã‚’èªã¿è¾¼ã‚€ – ã»ãã„りょã™ã‘ã®ã»ã‚ã±ã’ PHP: DOMDocument->
![[php][PEAR]Services_MixiAPIã¨XML_Feed_Parserを使ã£ã¦ã¿ãŸ | ã†ãˆã¡ã‚‡ã“ï¼ ã¼ã‚ã](https://cdn-ak-scissors.b.st-hatena.com/image/square/fb139b64147ac6479c2c1a8cf00ce623f85aedf9/height=288;version=1;width=512/http%3A%2F%2Flabs.uechoco.com%2Fblog%2Fimages%2Ft%2FServices_MixiAPI_01.jpg)
PHPã§ã®ã‚¹ã‚¯ãƒ¬ã‚¤ãƒ”ングã«å½¹ç«‹ã¤ãƒ©ã‚¤ãƒ–ラリ - (DxD)∞
APIãŒæä¾›ã•ã‚Œã¦ã„ãªã„サービスã‹ã‚‰æ¬²ã—ã„æƒ…å ±ã ã‘ã‚’å–å¾—ã™ã‚‹ã«ã¯ã€HTMLãªã©ã‹ã‚‰è‡ªå‰ã§ã‚¹ã‚¯ãƒ¬ã‚¤ãƒ”ングを行ã†ã—ã‹ã‚ã‚Šã¾ã›ã‚“。PHPã§ã®ã‚¹ã‚¯ãƒ¬ã‚¤ãƒ”ングã«å½¹ç«‹ã¤ãƒ©ã‚¤ãƒ–ラリãªã©ã‚’ã¾ã¨ã‚ã¦ã¿ã¾ã—ãŸã€‚ Perlã‚„Rubyã«ã¯è‰²ã€…ã¨ä¾¿åˆ©ãã†ãªã‚‚ã®ãŒè¦‹ã¤ã‹ã‚‹ã®ã§ã™ãŒã€PHPã«ã¯ãªã‹ãªã‹ã“ã‚Œã¨ã„ã£ãŸã‚‚ã®ãŒãªã„ã§ã™ã。 Webスクレイピングライブラリ HTMLScraping HTMLã‚’XML化ã—ã¦DOMã‚„XPathã§æ“作ã§ãるクラス。主ã«HTTP_Request+HTMLParser(XML_HTMLSax3ã‚’å«ã‚€)/Tidy+Cache_Liteã¨ã„ã†æ§‹æˆã§ã€ã‚¹ã‚¯ãƒ¬ã‚¤ãƒ”ングã«å¿…è¦ãªã‚‚ã®ãŒä¸€é€šã‚Šæƒã£ã¦ã„る。ライセンスã¯LGPL他。 WebScraper シンプルãªæ±Žç”¨ã‚¹ã‚¯ãƒ¬ã‚¤ãƒ”ングクラス。HTTP_Client+HTMLParser(XML_HTMLSax3ã‚’å«ã‚€)ã¨ã„ã†æ§‹æˆã§ã€XPathã§è¦ç´ を抽出ã§ã
scrAPI Cheat Sheet
Selectors *any element Ean element of type E E.warningan E element whose class is "warning" (the document language specifies how class is determined). E#myidan E element with ID equal to "myid". E[foo]an E element with a "foo" attribute E[foo="bar"]an E element whose "foo" attribute value is exactly equal to "bar" E[foo~="bar"]an E element whose "foo" attribute value is a list of space-separated v
John Resig - Bringing the Browser to the Server
This weekend I took a big step in upping the ante for JavaScript as a Language. At some point last Friday evening I started coding and didn’t stop until sometime mid-Monday. The result is a good-enough browser/DOM environment, written in JavaScript, that runs on top of Rhino; capable of running jQuery, Prototype, and MochiKit (at the very least). The implications of this are phenomenal, and I’m no
GoogleCodeSearchã§æŒ‡å®šURL以下ã®ã‚³ãƒ¼ãƒ‰ã‚’検索ã™ã‚‹ - ã¯ã“ã¹ã«ã£ã ♨
ã¯ã“ã¹ã«ã£ã# - GoogleCodeSearchã§ãƒ–ãƒã‚°ä¸Šã®ã‚³ãƒ¼ãƒ‰ã‚’表示ã§è§¦ã‚ŒãŸã‚ˆã†ã«ã€GoogleCodeSearchã§ã€ã‚ã‚‹URL以下ã®ã‚½ãƒ¼ã‚¹ã‚³ãƒ¼ãƒ‰ã‚’å–å¾—ã™ã‚‹ã®ã¯å°‘々メンドウã§ã™ã€‚ ã—ã‹ã—ã€å°‘ã—検索クエリを工夫ã™ã‚‹ã¨ã€ãã“ãã“æ£ç¢ºã«ã‚ã‚‹URI以下ã®ã‚½ãƒ¼ã‚¹ã‚³ãƒ¼ãƒ‰ã‚’見ã¤ã‘ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ãŸã¨ãˆã°ã€ä»¥ä¸‹ã®ã‚ˆã†ãªã‚ーワードを用ã„ã¦GoogleCodeSearchã§æ¤œç´¢ã™ã‚‹ã¨ã€ã“ã®ãƒ–ãƒã‚°ã®è¨˜äº‹ã‚’見ã¤ã‘ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ ^hakobe932/ package:"http://d.hatena.ne.jp"検索çµæžœã‚’見るã¨ã€ã“ã®ãƒ–ãƒã‚°ã®ã‚³ãƒ¼ãƒ‰ãŒè¡¨ç¤ºã•ã‚Œã¦ã„ã‚‹ã®ãŒã‚ã‹ã‚Šã¾ã™ã€‚packageã§ãƒ‰ãƒ¡ã‚¤ãƒ³ã‚’ã—ã¼ã‚‹ã®ã¨ã€æ£è¦è¡¨ç¾ã‚’用ã„ã¦URLパス以外ãŒãƒžãƒƒãƒã—ãªã„よã†ã«ã€è¡¨ç¾ã‚’é™å®šã—ã¦ã„ã¾ã™ã€‚ ã“ã®æ©Ÿèƒ½ãŒç°¡å˜ã«åˆ©ç”¨ã§ãるよã†ã«ãƒ–ックマークレットを作ã£ã¦ã¿ã¾ã—ãŸã€‚Google Code Search b
B10[mg]: Scraping Yahoo! Search with Web::Scraper
Yet another non-informative, useless blog As seen on TV! Scraping websites is usually pretty boring and annoying, but for some reason it always comes back. Tatsuhiko Miyagawa comes to the rescue! His Web::Scraper makes scraping the web easy and fast. Since the documentation is scarce (there are the POD and the slides of a presentation I missed), I'll post this blog entry in which I'll show how to
Web::Scraper ? SlideShare
1. Practical Web Scraping with Web::Scraper Tatsuhiko Miyagawa [email_address] Six Apart, Ltd. / Shibuya Perl Mongers YAPC::Europe 2007 Vienna 4. abbreviation Acme::Module::Authors Acme::Sneeze Acme::Sneeze::JP Apache::ACEProxy Apache::AntiSpam Apache::Clickable Apache::CustomKeywords Apache::DefaultCharset Apache::GuessCharset Apache::JavaScript::DocumentWrite Apache::No404Proxy Apache::Profiler
1
ランã‚ング
ランã‚ング
ランã‚ング
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
{{#tags}}- {{label}}
{{/tags}}