[B! html][parse] uchiuchiyamaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

uchiuchiyama id:uchiuchiyama

htmlã¨parseã«é–¢ã™ã‚‹uchiuchiyamaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (7)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Ruby Scraping - FrontPage
Rubyã«ã‚ˆã‚‹Web Scrapingãƒ©ã‚¤ãƒ–ãƒ©ãƒªã®æƒ…å ±ã‚’ã¾ã¨ã‚ã‚‹ãŸã‚ã®Wikiã§ã™ã€‚ HpricotHTMLã‚’ã€ŒRubyã‚‰ã—ãã€æ‰±ã†ãƒ©ã‚¤ãƒ–ãƒ©ãƒª MechanizeWebã‚µã‚¤ãƒˆã¸è‡ªå‹•ã§ã‚¢ã‚¯ã‚»ã‚¹ã™ã‚‹ãŸã‚ã®ãƒ©ã‚¤ãƒ–ãƒ©ãƒª scRUBYt!DSLã‚’ä½¿ã£ã¦ç°¡å˜ã«ã‚¹ã‚¯ãƒ¬ã‚¤ãƒ”ãƒ³ã‚°ã‚’è¡Œã†ãƒ©ã‚¤ãƒ–ãƒ©ãƒª feedalizerhtmlã‹ã‚‰RSSãƒ•ã‚£ãƒ¼ãƒ‰ã‚’ä½œã‚‹ã®ã«å½¹ç«‹ã¤ãƒ©ã‚¤ãƒ–ãƒ©ãƒª scrAPIãƒ‘ãƒ¼ã‚µã‚’å®šç¾©ã™ã‚‹ã“ã¨ã§HTMLã‚’è§£æžã™ã‚‹ãƒ©ã‚¤ãƒ–ãƒ©ãƒª ã‚¦ã‚§ãƒ–ã‚µã‚¤ãƒˆã‹ã‚‰å¿…è¦ãªãƒ‡ãƒ¼ã‚¿ã‚’æŠ½å‡ºã™ã‚‹ã“ã¨ã€‚(Scrape = å‰Šã‚Šå–ã‚‹) ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã«ã‚ˆã£ã¦ã¯ã€å—ä¿¡ã—ãŸãƒ‡ãƒ¼ã‚¿ã®è§£æžã ã‘ã§ãªããƒ‡ãƒ¼ã‚¿ã®é€ä¿¡ã«ã‚‚å¯¾å¿œã—ã¦ã„ã‚‹ã€‚ ä¾‹ï¼š RSSã‚’é…ä¿¡ã—ã¦ã„ãªã„ã‚¦ã‚§ãƒ–ã‚µã‚¤ãƒˆã®HTMLã‚’ã‚¹ã‚¯ãƒ¬ã‚¤ãƒ”ãƒ³ã‚°ã—ã¦é‡Žè‰¯RSSã‚’ä½œã‚‹ Googleã®æ¤œç´¢çµæžœã‚’ã‚¹ã‚¯ãƒ¬ã‚¤ãƒ”ãƒ³ã‚°ã—ã¦è‡ªå‹•ã§Googleæ¤œç´¢ã™ã‚‹ã‚¹ã‚¯ãƒªãƒ—ãƒˆã‚’æ›¸ã ãƒ–ãƒã‚°ã®æŠ•ç¨¿ãƒšãƒ¼ã‚¸ã‚’è§£æžã—ã¦ã€ã‚³ãƒžãƒ³ãƒ‰ãƒ©ã‚¤ãƒ³ã‹ã‚‰
uchiuchiyama 2007/10/10
ruby

html

parse

tool
ãƒªãƒ³ã‚¯
http://people.ucsc.edu/~merphant/wiki2html.html
uchiuchiyama 2007/05/25
wikipedia

wiki

parse

html

tool

web
ãƒªãƒ³ã‚¯
Hpricot ã‹ã‚‰ãƒ†ã‚ã‚¹ãƒˆã‚’å–ã‚Šå‡ºã™ - nazokingã®ãƒ–ãƒã‚°
scrAPIã‚ˆã‚Šã‚‚ä½¿ã„ã‚„ã™ã„æ„Ÿã˜ã®Hpricotã§ã™ãŒã€ã€ŒinnerTextã€ãŒä¸Šæ‰‹ãHTMLã‚¨ãƒ³ãƒ†ã‚£ãƒ†ã‚£ãƒ¼ã‚’æˆ»ã—ã¦ãã‚Œãªã„ã®ã§ã€é•ã†ãƒ¡ã‚½ãƒƒãƒ‰ã‚’ã¤ã‘ã¦ã¿ã¾ã—ãŸã€‚ require "rubygems" require 'hpricot' class Hpricot::Elem def [](a) CGI.unescapeHTML(get_attribute(a)) end def to_text r = [] traverse_text{|text| case text when Hpricot::CData r << text.content else r << CGI.unescapeHTML(text.inner_text.gsub("\n"," ").gsub(/ +/," ").strip) end } r.join end end hp = Hpricot('<html><bog
uchiuchiyama 2007/02/25
rails

tips

hbto

html

parse
ãƒªãƒ³ã‚¯
é€²åŒ–ã™ã‚‹â€œWebã‚¹ã‚¯ãƒ¬ã‚¤ãƒ”ãƒ³ã‚°â€æŠ€è¡“ã®ä¸–ç•Œ â€• ï¼ IT
2007/02/20 Webã‚µãƒ¼ãƒ“ã‚¹ã®APIã‚„RSSãƒ•ã‚£ãƒ¼ãƒ‰ã‚’ä½¿ã£ã¦è¤‡æ•°ã‚µã‚¤ãƒˆã®ã‚µãƒ¼ãƒ“ã‚¹ã‚„æƒ…å ±ã‚’ãƒžãƒƒã‚·ãƒ¥ã‚¢ãƒƒãƒ—â€•â€•ã€‚ã“ã‚Œã¯Web2.0ãŒåŒ…å«ã™ã‚‹ã„ãã¤ã‹ã®æ¦‚å¿µã®ã†ã¡ã€æœ€ã‚‚é‡è¦ãªã‚‚ã®ã®1ã¤ã ã€‚Amazon.comã‚„Googleã€Yahoo!ã€æ¥½å¤©ã¨ã„ã£ãŸå¤§æ‰‹Webã‚µã‚¤ãƒˆã¯ã€RESTã‚„SOAPã‚’ç”¨ã„ãŸAPIã‚’å…¬é–‹ã—ã¦ãŠã‚Šã€ã•ã¾ã–ã¾ãªã‚µãƒ¼ãƒ“ã‚¹æä¾›è€…ã‚„å€‹äººãŒAPIã‚’é€šã—ã¦å„ç¨®ã‚µãƒ¼ãƒ“ã‚¹ã‚’åˆ©ç”¨ã—ã¦ã„ã‚‹ã€‚ãã®ä¸€æ–¹ã€ä¸–ã®ä¸ã®Webã‚µã‚¤ãƒˆã®å¤§å¤šæ•°ã¯Web1.0çš„ãªHTMLã‚„CGIãƒ•ã‚©ãƒ¼ãƒ ã—ã‹æä¾›ã—ã¦ã„ãªã„ã®ãŒç¾å®Ÿã ã€‚ã“ã†ã—ãŸèƒŒæ™¯ã‹ã‚‰Web1.0ã‚µã‚¤ãƒˆã‹ã‚‰æ§‹é€ åŒ–ã•ã‚ŒãŸãƒ‡ãƒ¼ã‚¿ã‚’å¼•ã£å¼µã‚Šå‡ºã™â€œWebã‚¹ã‚¯ãƒ¬ã‚¤ãƒ”ãƒ³ã‚°â€æŠ€è¡“ãŒæ€¥é€Ÿã«ç™ºå±•ã—ã¦ãã¦ã„ã‚‹ã‚ˆã†ã ã€‚ HTMLã‚’XMLåŒ–ã—ã€XPathã§é–¢é€£ãƒ‡ãƒ¼ã‚¿ã ã‘ã‚’æŠ½å‡º ä¾‹ãˆã°ä¾¡æ ¼æƒ…å ±ã‚µã‚¤ãƒˆã§ã¯è£½å“åã‹ã‚‰ä¾¡æ ¼ãŒç°¡å˜ã«èª¿ã¹ã‚‰ã‚Œã‚‹ãŒã€Webã‚µãƒ¼ãƒã‹ã‚‰æä¾›ã•ã‚Œã‚‹ã®ã¯ã€è£½å“åã‚„ä¾¡æ ¼ã«ãƒ¬
uchiuchiyama 2007/02/21
web

ruby

html

xml

parse
ãƒªãƒ³ã‚¯
[ruby] Rubyã§HTMLã¨Webã‚’æ“ä½œã™ã‚‹ãŸã‚ã®ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã€Hpricotã¨WWW::Mechanize - Greenbear Diary (2007-02-05)
â– [ruby] Rubyã§HTMLã¨Webã‚’æ“ä½œã™ã‚‹ãŸã‚ã®ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã€Hpricotã¨WWW::Mechanize ä»Šæ—¥ã¯ã€Rubyã§Webã‚µã‚¤ãƒˆã‚’è§£æžã™ã‚‹ã¨ãã«å¼·ã„å‘³æ–¹ã¨ãªã‚‹ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã€Hpricotã¨WWW::Mechanizeã‚’ç´¹ä»‹ã—ã¾ã™ã€‚ ã©ã¡ã‚‰ã‚‚éžå¸¸ã«å¼·åŠ›ãªã®ã§ã€è¦šãˆã¦ãŠã„ã¦æã¯ãªã„ã‚ˆï¼ ä»¥ä¸‹ã§ã¯ã¾ãšHpricotã§HTMLã‚’è§£æžãƒ»ç·¨é›†ã™ã‚‹æ–¹æ³•ã«ã¤ã„ã¦è§£èª¬ã—ã¾ã™ã€‚ æ¬¡ã«ã€ã€Œã¯ã¦ãªãƒ€ã‚¤ã‚¢ãƒªãƒ¼ã®è‡ªå‹•æ›´æ–°ã€ã‚’ä¾‹ã«WWW::Mechanizeã®ä½¿ã„æ–¹ã‚’è§£èª¬ã—ã¾ã™ã€‚ Hpricot Hpricotã¯HTMLã‚’è§£æžã™ã‚‹ãŸã‚ã®ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã§ã™ã€‚ ä¾‹ãˆã°ã€Œã‚ã‚‹ãƒšãƒ¼ã‚¸ã®ãƒªãƒ³ã‚¯ã ã‘ã‚’å…¨éƒ¨æŠœãå‡ºã—ãŸã„ã€ã¨æ€ã£ãŸã¨ãã€ã©ã†ã—ã¾ã™ã‹ï¼ŸscrAPIã‚’ä½¿ã†ï¼Ÿã§ã‚‚scrAPIã¯ã‚„ã£ã±ã‚Š ã¡ã‚‡ã£ã¨ä½¿ã„ãŸã„ã ã‘ãªã®ã«ãƒ‘ãƒ¼ã‚¶(Scrape)ç”¨ã®ã‚¯ãƒ©ã‚¹ã‚’å®šç¾©ã™ã‚‹ã®ãŒé¢å€’ãªã‚“ã ã‚ˆãï¼ Hpricotãªã‚‰ã€ãŸã£ãŸã“ã‚Œã ã‘ã§
uchiuchiyama 2007/02/18
ruby

html

library

parse

web
ãƒªãƒ³ã‚¯
Greenbear Laboratory - Hpricot Showcase-Ja
GitHubindexHello source: index.md View on github | Report issue Generated by middleman 3.1.6. Powered by Ruby 2.2.2.
uchiuchiyama 2007/01/21
web

rails

parse

html

reference
ãƒªãƒ³ã‚¯
Lessons From Hpricot
HTML processing, C parser, yada, yada, yada. What lesson did I actually learn from Hpricot? Abuse the / operator! Why, the Lucky Stiff, presents this neat bit of code: doc = Hpricot.parse(File.read("index.html")) (doc/:p/:a).each do |link| p link.attributes end Nice! That's hot. Let's run with this for a moment and add / navigation to hashes. class Hash def /(key) self[key] end end people = {:toph
uchiuchiyama 2006/08/20
ruby

html

parse

tips
ãƒªãƒ³ã‚¯
1