This document discusses how to create pipes using Yahoo Pipes to aggregate and filter content from multiple web sources. It provides examples of basic pipes that: 1. Pull in an RSS feed using the Fetch Feed block and output it unchanged 2. Combine content from multiple sources using the Union block 3. Allow users to input search terms using a Text Input block to dynamically query Google News feeds The document demonstrates how blocks like Fetch Feed, Filter, Union, and URL Builder can be combined to select relevant content from multiple web sources and feeds, and construct dynamic search queries from user-supplied text. It emphasizes that pipes allow easy redistribution and aggregation of web content.

1. Rewiring the WebWith Yahoo Pipes“It’s all about me” edition Tony Hirst

3. How do you merge content from multiple web searches and add it your own personal page? How do you quickly and easily plot your photos or a list of addresses contained in a spreadsheet onto an interactive map? How do you automatically find Youtube movies related to a news story? http://pipes.yahoo.com

5. A. Pipes are like construction blocks that can take content from one location on the web, process it in some way, and then republish it to where you need it to be. You don't even need to create your own in order to get started – just use someone else’s… B. To create your own Yahoo Pipes you’ll need to login with a Yahoo ID. C. Once you’re logged in, click Create Pipe to get started. Just because you aren’t a programmer doesn’t mean you shouldn’t be able to get computers to do what you want them to do.Yahoo Pipes helps you rewire the web, moving content from where it currently is to where you need it to be, in the form that you require.

7. D. This is the central canvas, the place where you build your pipe out of simple building blocks or even other pipes. E. The debug area allows you to preview the output of any part of pipe to make sure that each block is doing exactly what you expect. F. Building blocks are listed in this menu area. Drag individual blocks onto the canvas, or click the + sign to add them to the canvas automatically. G. Many great works of art may be untitled, but it can be hard to tell them apart by name alone. Give your pipe a meaningful name by clicking in this tab. Drag and drop is the name of the game (with as little typing as possible required).Wire the building blocks together and see what happens as content flows through the pipe.Are you ready to give it a go?

8. Use the blank pages to write your own notes, ideas for how to use each pipe, or sketches for how to modify or extend each pipe…

9. The Simplest Pipe

11. 0. Wires are used to take the output of one part of the pipe and feed it to the next. Content flows out of the connector on the bottom of a block and into the connector on the top of another block. Click on a connector and drag the mouse, and the wire will follow. Make a connection by dropping the wire onto a connector on another block. 1. Many websites and search engine publish content via an RSS or Atom feed, often denoted by the symbol shown opposite. The Fetch Feed block can be used to pull that content into a Pipe from the feed’s web address. In this case, we pull in content from an RSS feed from my blog that has been republished via Feedburner. 2. The output block is always the final block in a pipe. Content is pulled into the pipe via an input blocks such as Fetch Feed and published to the outside world via a Pipe Output block. From the very beginning, Yahoo Pipes was designed as a tool to help content flow around the web.The first step is to get web content into the Pipes environment……and the second is to learn how to publish it back to world.

13. 3. A preview of the output of the pipe. If the pipe emits valid RSS, you should be able to see highlights of the first few elements of that web feed here. 4. Content came into the pipe via an RSS feed, so it’s only natural that we should be able to republish it as RSS too. 5. Pipes are also about sharing and reuse. Clone a pipe (either one of yours, or one of someone else’s) in order to grab a copy of it to keep for ever, or that you can edit to create a modified version of the pipe. 6. Each pipe has its own web address. You can give your pipe a nice URL if you want to. Every Yahoo Pipe has its own personal web page.This page previews the output of the pipe and provides a range of ways publishing that content to the web.But that’s not all – because every pipe can also be used inside other pipes…

15. Multiple Sources(and that includes other pipes)

17. 7. The My Pipes menu option contains a list of pipes I have cloned or made that I can include in a new pipe, just like any other pipes block. 8. Content from multiple sources can be pulled together (that is, aggregated) within a single pipe. Clicking the + button in the Fetch Feed block allows you import content from several different web addresses. This technique can be used as is to extend The Simplest Pipe directly. 9. The Union block, which can be found in the operators menu, allows you to merge the content from different sources. To join more sources together, just wire the output of one Union block to the input of another. My online identity is distributed across multiple websites. Now can I pull all my personal feeds together in one place.Although quick and easy to build, this sort of pipe provides a powerful tool for merging content from different sources.So what are you going to draw together?

19. 10. Clicking on a block highlights it and turns it orange. Selecting a block in this way means we can inspect the output of the block in the preview window. 11. Yahoo Pipes process lists of items. In a search results feed, each item will contain details for a single search result. In a feed from a news website or blog, each news story or blog post will be a separate item. In this preview, each item corresponds to a single presentation from my Slideshare account. Each item contains a list of elements. The title element provides the text used to identify each item. The link element contains the web address for the original webpage of each presentation item. Probe the internals of a pipe to see what’s going on.Inspect the first few items that flow out of a block to check their basic, elemental properties.And then start putting those properties to work…

21. Selecting What’s Relevant

23. 12. I created this pipe by Copying the Multiple Sources pipe. 13. The * that is added to the end of the Pipe’s name shows that changes have been made to the pipe since it was last saved. 14. If you want to rewire part of a pipe, move the cursor over a connector and a pair of scissors will appear: you may now cut the connecting wire and then rewire pipe as required. In this case, I am going to add a Filter block between the Union and Pipe Output blocks. 15. The Filter block, which can be found in the Operators menu, allows you to select which items you do – or do not – what to pass through the block based one or more properties of each item. Rules can be added as required by clicking +. Items may be Permitted to flow through the Filter, or Blocked (i.e. prevented) from passing through it, based on whether any single rule is true or whether all the rule conditions are met. Sometimes we want to be selective about what we let flow out of a pipe.And sometimes we want to stop particular items from making it through.What would you focus your pipe’s attention on?

25. 16. The preview output of the pipe shows that only items that met the Filter criteria are passed through. 17. If you have a Google personal page, you can add a widget to it based on the output of a Pipe. Whenever the the content in the feeds defined as inputs to the pipe change, the pipe’s output will reflect that change (subject in this case to the Filter conditions.) So if I write a post about a new presentation on my blog, a link to it will now appear on my Google personal page, courtesy of this pipe and The Simplest Pipe before it.. The best thing about good pipework is that you shouldn’t have to see it – it does its job discreetly.So if you want a pipe to deliver its results somewhere more convenient – like your Google or Yahoo Personal page – it can do.

27. Search Me...

29. 18. The URL Builder makes it easy to construct new URLs from old ones. For example, copy the web address for the RSS feed from a Google News search into the Base text area and the individual parts of the URI will be separated out. 19. Turn the pipe’s homepage into a web form by allowing the user to enter a search term of their own using a Text Input block. Set a default value to demonstrate what the pipe does. 20. Look for the part of the URL that contains your original query, and wire the output of the Text Input to corresponding connector. 21. Having constructed a URL for an RSS feed from Google News that uses a user supplied search term, we can now fetch the feed with a Fetch Feed block. Sometimes it’s useful to be able to let the user of a pipe specify some search terms to be used by a pipe.If search terms or topic terms appear in the web address of an RSS feed, we can create our own search engine.Remember, a pipe can fetch more than one feed…

32. INDEX M Multiple sources 8, 9 F Fetch Feed 1, 8, 21 U Union 9