Tutorial: Creating a News Aggregator
Applies to: Functional Programming
Published: January 2010
Summary: This tutorial creates a web page for aggregating news using RSS feeds. The application is implemented using F# asynchronous workflows and asynchronous controllers to avoid the unnecessary blocking of threads.
This topic contains the following sections.
This article is associated with Real World Functional Programming: With Examples in F# and C# by Tomas Petricek with Jon Skeet from Manning Publications (ISBN 9781933988924, copyright Manning Publications 2009, all rights reserved). No part of these chapters may be reproduced, stored in a retrieval system, or transmitted in any form or by any means—electronic, electrostatic, mechanical, photocopying, recording, or otherwise—without the prior written permission of the publisher, except in the case of brief quotations embodied in critical articles or reviews.
The web application created in this tutorial is a very basic example of a mashup. It obtains data from several other web sites (using RSS feeds), aggregates the data, and displays the summary to the user. The application created in this tutorial is fairly simple. It downloads the latest world news items from three newspapers and displays them in three columns. A screenshot of the application is shown in Figure 1.
Despite its simplicity, the application demonstrates an important type of web application. The value of mashups is that they give a better view on the aggregated data. For example, the application could use machine learning to automatically group related news together. F# would be a perfect fit for this type of application, but this tutorial doesn't go that far.
The web application is created using ASP.NET MVC 3. It uses F# to implement the functionality (downloading and aggregating news) and C# to implement the views (generating the HTML markup sent to the user). The project can be created using a template described in the following tutorial:
This step of the tutorial explains how to implement the model of a news aggregator web page. The model asynchronously downloads RSS feeds and formats them into F# records.
This tutorial shows how to use asynchronous F# functions for downloading news from an asynchronous controller provided by ASP.NET MVC and how to render news using the Razor view engine.
This tutorial demonstrates how to create a web application that downloads data from other web sites and aggregates them. More complicated web applications usually also store some data in database. More information about working with databases using ADO.NET as well as LINQ to SQL can be found in the following articles:
To download the code snippets shown in this article, go to http://code.msdn.microsoft.com/Chapter-5-Bulding-Data-ec639934
This article is based on Real World Functional Programming: With Examples in F# and C#. Book chapters related to the content of this article are:
Book Chapter 7: “Designing data-centric programs” explains how to design applications that are primarily designed to work with or create a data structure. This is very often the design used by ASP.NET MVC models.
Book Chapter 12: “Sequence expressions and alternative workflows” contains detailed information on processing in-memory data (such as lists and seq<'T> values) using higher-order functions and F# sequence expressions.
Book Chapter 13: “Asynchronous and data-driven programming” explains how asynchronous workflows work and uses them to write an interactive script that downloads a large dataset from the Internet.
Previous article: Tutorial: Creating a Web Project in F# Using an Online Template
Next article: Step 1: Downloading RSS Feeds