It takes effort to explain the significance of a new product when the immediate benefit to consumers may not be so obvious, and the awkwardly named “pipes” from Yahoo! is no exception. The product name is taken from the world of UNIX where a pipe is a conduit for the transfer of data between applications, while with the Yahoo product it is a conduit for data between web services. In a basic form Yahoo! Pipes allows you to take data from one or more sources and to bring it together, for example – to aggregate a group of feeds.
But Yahoo! Pipes goes beyond what just pipes are and what pipes do though as the application provides functions (or as they are called in the app – modules) that will perform a variety of different actions. There are modules available to prompt the user for input (a variety of input types), different operators to count, loop, cut, count, sort and merge data along with a variety of string and date functions. Because of this already broad base of available functions, Yahoo! Pipes is more akin to a shell scripting environment for the web rather than just a simple conduit between applications. It works like a visual procedural programming language with the output of the process dropping out at the bottom, in the form of text output, RSS, SMS alerts of even JSON. You can use feeds, user input or other pipes as input.
The beauty of the application is with its simplicity – a user can take any sources, user input requests or the above mentioned module and drag+drop them into place and then connect the pipes. Within minutes I had built an application (also known as a pipe, they should probably change the name as not everything can be a pipe) that would search for ‘Techcrunch’ in a variety of feeds, bring that data together, sort it and filter it for unique results. I saved the application and published it, from where I can now execute it at any time and receive the output in a variety of formats. I can take a copy of an existing pipe (application, argggh) and use it as a base template for my own pipe and I can browse an existing library of pipes.
Pipes can take any feed as input, and combined with the already available list of functions proves to be very powerful – my mind is still buzzing thinking about all that can be done with Pipes. I think some of the terminology needs to be cleared up, there needs to be a better introduction on the main page – but besides that this product is fantastic. It was inevitable that such a product would be released, and it is very good for Yahoo! that they managed to be the first of the big web companies to release such a product. The fact that they include Google Base as a default source in Pipes shows that the web is much more about interoperability than the desktop ever was or ever will be.
See Anil Dash, Tim O’Reilly and Jeremey Zawodny for more.
