Writing a data-centric oriented web application using HTML5 local storage

By Dariusz Parys, Developer Evangelist, Microsoft

A lot of people still consider JavaScript a "toy language", despite the fact that it has received much attention lately and there are quite a number of high quality libraries and frameworks available for creating production-ready applications. This article will follow the creation of a basic business oriented application. The application is meant as a starting point for digging deeper into the existing options and to experience how much fun coding quality JavaScript can be.

The basic application that was developed alongside this article catalogues products and divides them into categories. Those categories as well as the products can be created, updated and deleted. In addition to this typical CRUD approach, there are other standard tasks that will be handled: internationalization, validation of input and controlling the application via the keyboard. One of the most important aspects of the app is that it will use HTML5 local storage in order to allow offline editing.

Description: http://html5.democode.de/article/products-overview.png

Karhu - products overview list

How to start

So how does one go about writing a business oriented JavaScript application? One well established way is by using a Model-View-Controller structure. This has been successfully employed in frameworks like Ruby on Rails, Django or ASP.NET MVC. It allows for a strict structure and the separation of the concerns of the application like view and business logic. This is especially important with JavaScript as it is really easy to write confusing code, doing everything in one place. I often have to remind myself not to do that and to try and write clean, readable and reusable code. There are several frameworks built for a MVC structure. Some of the most notable ones are Backbone.js, Eyeballs.js and Sammy.js.

For this example application I used Sammy.js, primarily because I already know it but also because it is small, well written, tested, and does all the things I need to get started. It does not provide you with an implicit MVC structure but it allows you to easily build upon its base. The only dependency it currently has is jQueryand that is a library I personally use for DOM manipulation anyway. The directory structure I started with looks like this:

- public
  - js
      app.js
    + controllers
    + models
    + helpers
    + views
  + templates
  - vendor
    - sammy
        sammy.js
    - jquery
        jquery.js

In templates I put all the template files that may be rendered through the JavaScript code and in viewsall the JavaScript code relevant for rendering those templates.

The application file

The actual Sammy.js application is created in app.js - here the controllers are loaded and their routes are initialized. I tend to namespace all the variables (controllers, models, etc.) I create. In this case I chose to call this namespace karhu.

First we start loading plugins such as Mustachewhich is a template rendering engine. Then we initialize helpers (karhu.ApplicationHelper) and controllers (karhu.Products). Once the app is defined and all the DOM elements are loaded, you can run the sammy app and direct it to the initial route which is the index of all products.

Writing tests

Before showing you how the products controller works and displays all the products I want to briefly go into how the quality of JavaScript applications can be greatly increased through testing. As I went about developing the example application before every major step I first wrote an acceptance test to ensure that the code is actually working. This also prevents regressions, guaranteeing everything I wrote before is also still functioning correctly. For more complex code, I write unit tests and try to cover most of the cases that can happen when running the code. For writing acceptance tests one of the easiest and most readable ways is using Capybara with Selenium. In the first scenario we can test if we can see a list of products:

Feature: Products
In order to know which products I have
  As a user
  I want to see a list of products
  Scenario: list products
    Given a category "Trees" with the description "Plants"
      And a product "Oak" with the description "Brown"
               and the price "232.00€"
               that is valid to "12/20/2027"
               and belongs to the category "Trees"
      And a product "Birch" with the description "White"
               and the price "115.75€"
               that is valid to "03/01/2019"
               and belongs to the category "Trees"
    When I go to the start page
    Then I should see "Trees"
      And I should see "Oak"
      And I should see "Brown"
      And I should see "232.00€"
      And I should see "12/20/2027"
      And I should see "Birch"

Once headless browsers like Phantomjs are available as Capybara drivers, it will probably make sense to use those instead of Selenium as they are a lot faster.

For unit testing there are a lot of different possibilities. I used to work with jspec, since it is similar to Ruby's rspec which I have used before. Recently that has been deprecated in favor of jasmine, so I've used that here. It works quite well and brings a raketask that allows it to easily run alongside the acceptance tests. One of the unit tests for the example application looks like this:

Defining controllers

Once I finish the scenario I start with the controller, which is very simple when just starting out:

At the moment there is only one route defined, which is a GET on the #/productsroute. The callback will be run once the location hash in the URL changes to /products. So if you append the route to your URL (like http://localhost:4567/index.html#/products) the attached callback will be executed. The same will happen when the application is just started, because we defined in our app.jsthat the initial path points to the same route.

Debugging controllers/products.js with Internet Explorer F12 Developer Tools

Inside the route we retrieve the categories and products via helpers that just do a basic AJAX GET request to our backend. Once we have retrieved this data we map it to JavaScript objects and then render those objects inside the index.mustachetemplate. This will render them in the <div id="main"> HTML tag, which was defined as the root element_selector in the app.jsfile.

Defining models

We need to map the data to JavaScript objects so we can associate the products with the category they belong to and can render the name of the category alongside the product, which looks like this:

We extend the object with all the attributes of the product and then we attach the category of the product to the object. attachCategoryis kept inside the closure to make it a private function. Important to note in this code is the use of the underscore functions which are provided by Underscore.js. Underscore defines helpers for enumerables and helps one to write easy to read, concise code.

Interacting with the product model in the web app

Rendering templates

In the above case, we do not need an additional view layer object because the rendering logic is very basic - it is just iterating over the objects of the products we created and displaying the attributes of each, including the category name we attached beforehand. The logic-free mustache template that will be rendered looks like this:

Rendered HTML output from the above mustache template

Moving model specific controller code into the model

It is a matter of taste how much responsibility a controller is given and how much can be refactored into model code. If I want to write the above code in a more model-centric fashion, I could do something like this:

Controller

Model

Standard tasks and difficulties

There are several tasks that are very common to web development and will be recurrent when working on JavaScript applications. I want to explain how I solved those tasks and which problems I encountered. As usual, there are several different ways to approach an issue.

Authentication

Most applications, including ours, add basic security by making the user log in before using the application. Since HTTP is stateless, we need to resend the authentication with every request. We can accomplish this by saving a token when first logging in and then using that for every request thereafter. The way I chose to do that was to save a token in local storage once the user had logged in successfully and send that token as a header attached to the XMLHttpRequest. The code to do this looks similar to the following. It is stashed in a backend model which is used by the helpers I mentioned earlier:

X-Karhu-Authentication included in HTTP request

One case is that we already have a token, the other is that the user just logged in and there is a user(name) and a password available. Either way we attach the token or user/password combination as a header and if the request is successful, we know that the user is authenticated successfully. Otherwise the backend will just return an error. This approach was relatively straight forward to implement and the only issue I encountered was that the code became a little complex and unreadable. To fix this I refactored the helpers in to separate model. Abstracting the requests into a backend model is quite common, as, for example, seen in the Backbone.js library where it is a core part of the library. Authentication code is often unique per-application and always depends on the backend and what it expects the frontend to send.

I18n

A very common library for internationalization in JavaScript applications is jquery.global.js. It provides us with methods to format numbers and dates and enables us to translate strings using a dictionary for the current locale. Once we have loaded this dictionary, which is a simple JavaScript object with keys and translated values, the only thing we need to pay attention to is formatting numbers and dates. A sensible place to do that is in the models before rendering the objects to the templates, in the product model it would look something like this:

Language switched to German

Validation

One of the benefits of developing in JavaScript is we have the opportunity to give the user real-time feedback. It makes sense to use this potential to validate the data before it is sent to the backend. Note that it is still necessary to validate the data in the backend as well since there might be requests that are not using the frontend. There is a common jQuery library for validations, jquery.validate.js. The library uses a set of rules on a form and shows errors on the appropriate input fields if the content does not match the provided rules. It makes sense to structure those validation rules into the models we already have, so every model has a validationsfunction which returns the rules. This is similar to Backbone.js but different from Eyeballs.js where the validations work in a more Rails-like fashion. Here is how the validations for our category model could look:

Validation error on creating a new category

Validation goes further. Navigation away from unsubmitted forms is prohibited. The user has to submit a valid data entry or cancel proactively data entry.

Red flash notification warns user on unsubmitted form data

Caching objects for offline editing

This is the most central and complex part of the application. All objects need to be cached ahead of time so that once we are offline they can be correctly sorted, paginated and filtered. There needs to be a queue for all actions being done before the objects are cached, so that those actions can be applied to the objects as soon as they are cached. There also needs to be a second queue that is filled once we actually are offline, so that when we are back online, everything that was done offline can be patched through to the backend.

App response when taken offline

There are several issues that need to be addressed outside of the already complicated caching and queuing process. For example, when an object is created when offline, it cannot be updated or deleted without further code because it does not have an id. I worked around that for now by simply disallowing those actions for objects created while offline. Another issue was that categories created while offline cannot be used for creating products; here again the reason being that they do not have an id yet. I simply do not display those categories in the list of available categories for creating a product. Some of those problems might be solved by working with temporary ids and by rearranging the offline queue.

In addition, the available partials and template need to be cached. This can either be done through a cache manifest as defined in HTML5 if the targeted browser group supports it, or simply through loading the partials and putting them into local storage. This is quite simple with Sammy.js and looks something like this:

Integration into Windows

Internet Explorer 9 is great in running HTML5 applications. Further it gives web applications the ability to natively integrate into the Windows 7 Taskbar, enhance the application with the possibility to show notifications, integrate navigation and offer jump list support. The integration of jump lists is straightforward, in its simplest form just a declaration of meta tag attributes. This is exactly the approach used in Karhu which makes it easier for your users to access. Direct jump list tasks to go to the views add product, add categories, products overviewand categories overview. The following code snippet demonstrates how to declare a simple jump list task:

You can learn all about pinning and Windows 7 integration at Build My Pinned Site, which has additional ideas for your web app such as browser notifications and dynamic jumplists through JavaScript.  It’s simple to add more functionality to your web app in a few extra minutes.

Conclusion

At this point, all of the requirements for the example application were implemented. With enough time one could also handle the issues mentioned above. Tasks like authentication, internationalization and handling business logic need to be coded independent of the frameworks and libraries, which are really just a starting point.

If you always write tests and pay attention to the structure of the application, writing JavaScript production-ready applications that can continue to evolve is, in my opinion, possible and a goal well worth investing in. Getting started is easy, however, it is important to keep checking for a clean code base and refactoring if necessary. If these requirements are met, JavaScript gives you the opportunity to write very elegant and maintainable applications.

The code of the example application can be found on codeplex. Other starting points for getting a better understanding of the subject are the MSDN JavaScript Language Reference.aspx), the documentations of the aforementioned libraries and frameworks, other example applications and, of course, JavaScript books.

And to get a detailed overview on how to pin your HTML app / site to the Windows taskbar have a look at the website Build My Pinned Site.

About the authors

Frank Prößdorf

Frank is a freelance web programmer and co-founder of NotJustHostingwho loves working with Ruby and JavaScript. His passion is discovering and playing around with new technologies. He regularly supports open source software as he is excited about the opportunity to contribute and share code and ideas. If you don't meet him drinking coffee and coding away, he is probably outside or out of the country. Besides his work Frank enjoys traveling, sailing and playing tennis.

Frank's Github profile

Frank's website

Dariusz Parys

Dariusz is a developer evangelist at Microsoft Germany doing web development with a strong focus on upcoming technologies. Currently he is writing a lot of JavaScript and HTML5. You can contact Dariusz through his blog downtocode.net