はてなブックマーク

A few Googlers at WWW 2010 had a paper, "Beyond Position Bias: Examining Result Attractiveness as a Source of Presentation Bias in Clickthrough Data" (PDF), that explores how much people tend to click on eye-catching search results rather than seeking the most relevant search results. The work itself was pretty simple -- just looking at how bolding title and abstract terms changes clickthough rate

A remarkably detailed paper, "Web-Scale User Modeling for Targeting" (PDF), will be presented at WWW 2012 that gives many insights into how Yahoo does personalized advertising. In summary, the researchers describe a system used in production at Yahoo that does daily builds of large user profiles. Each profile contains tens of thousands of features that summarize the interests of each user from the

In a paper at the recent RecSys 2010 conference, "The YouTube Video Recommendation System" (ACM), eleven Googlers describe the system behind YouTube's recommendations and personalization in detail. The most interesting disclosure in the paper is that YouTube has switched from their old recommendation algorithm based on random walks to a new one based on item-to-item collaborative filtering. Item-t

A googol of Googlers published a paper at VLDB 2008, "Google's Deep-Web Crawl" (PDF), that describes how Google pokes and prods at web forms to see if it can find things to submit in the form that yield interesting data from the underlying database. An excerpt from the paper:This paper describes a system for surfacing Deep-Web content; i.e., pre-computing submissions for each HTML form and adding

Google Fellow Jeff Dean gave a fun talk last week at University of Washington Computer Science titled "Research Challenges Inspired by Large-Scale Computing at Google". The talk is a "collection of problems we think are interesting/difficult" and, since it is coming from Jeff, has a heavy bias toward infrastructure problems. The talk starts with energy efficiency in large scale clusters. Jeff poin

I had the great pleasure of giving a talk today on practical issues in personalization and recommendations for the Data Mining (CS345) class at Stanford taught by Anand Rajaraman and Jeff Ullman. The slides from my talk are available in two versions. The first version is the talk I actually gave; make sure to read the notes pages for the slides, or it will be difficult to follow. The second versio

It is hard to tell from the limited documentation available, but the Pig project at Yahoo Research seems to have a lot in common with Sawzall at Google. Both are high level programming languages targeting massively parallel processing across huge clusters. From the Pig project page:We are creating infrastructure to support ad-hoc analysis of very large data sets. Parallel processing is the name of

Laurie Burkitt at the Seattle PI reports that Googler Christophe Bisciglia is directing a "Google 101" class in the Computer Science department at University of Washington. The class explores programming and problem solving on massively parallel distributed systems (such as the one Google happens to have). The class is actually called CSE 490h. The lecture slides and class tutorials are publicly a

I came across an interesting VLDB 2005 paper, "C-Store: A Column-oriented DBMS" (PDF). What attracted me to this paper, other than that Mike Stonebraker is lead author, was that the goals seem to have a lot in common with what appeared to motivate Google's BigTable. C-Store is column-oriented (values for a column are stored contiguously) instead of row-oriented like most databases. It is optimized

Misinformation and disinformation are the biggest problems on the internet. To solve a problem, you need to understand the problem. In Algorithms and Misinformation: Why Wisdom of the Crowds Failed the Internet and How to Fix It, I claim that the problem is not that misinformation exists, but that so many people see it. I explain why algorithms amplify scams and propaganda, how it easily can happe

Google VP Marissa Mayer just spoke at the Web 2.0 Conference and offered tidbits on what Google has learned about speed, the user experience, and user satisfaction. Marissa started with a story about a user test they did. They asked a group of Google searchers how many search results they wanted to see. Users asked for more, more than the ten results Google normally shows. More is more, they said.

I spent several more years at Amazon. Amazon grew and grew. Amazon expanded from a tiny online bookstore into an online superstore, selling books, music, videos, software, video games, electronics, toys, hardware, clothing, jewelry, and much, much more. In later years, I went on to lead the software team in the Personalization group. We did great things. I am proud of everything we accomplished. B

In a world with infinite storage, bandwidth, and CPU power Google is hosting an analyst day today. I found skimming the 94 slide presentation (PPT, PDF alternative) to be interesting and worthwhile. In particular, I liked slide 19, 20, and 31, all of which makes it clear that Google isn't losing its wide-eyed optimism. Slide 31 says that Google's philosophy to new product development is "no constr