A while back I had a neat idea to build a service that would allow me to select a list of Craigslist feeds for a location (or locations) and aggregate it into a single feed. You could add any of the categories from any of the cities just by going to a website and it’d do all the work for you. However, I’ve found a much easier solution using Yahoo! Pipes.
If you’re unfamiliar with Pipes, it’s a way of aggregating data from a ton of different sources and pulling it into one channel. This is a perfect way to build a single RSS feed from multiple feeds. Here’s a diagram of what the finished product looks like:
Build Your Pipe
The basic idea is to create multiple instances of the Fetch Feed entity and join them together with a Union. You’ll want to create a Fetch Feed entity for each of the categories you want to aggregate, such as electronics, games/toys, etc. Simply click the output button from the feed entity and drag it to an input on the union entity to join the two. You can pipe multiple unions together if you need to scale the diagram.
Once you’ve joined your feeds into a union, you should add a Sort entity to sort the results by date in descending order. Drag the Sort entity to the sandbox and connect its input with the output from the union. Then, select “item.pubDate” as the sort field and “Descending” as the order (as shown in the diagram). Once you’ve done that, you can send its output to the input of the Pipe Output entity.
Run Your Pipe
When that’s all finished, save your pipe. At the top of the screen there will be a link labeled “Run Pipe…”, so click on that to get the results. After it’s been run, click More Options and select Get as RSS. Copy the URL and add that to your feed reader. Voila, you’ve now got a single feed for your city on Craigslist.



1 comment so far ↓
November 14th, 2008 at 9:38 am
THAT is so cool, I can’t contain my excitement! Really… Kind’a geeky, I know. But that’s something I’ve been wondering how to do for a while…
Thanks!
Andrew