ListEditor: A Useful XML Web Service
April 21, 2000
I find myself using this little XML ASP tool over and over for all sorts of things. ListEditor is an ASP application that gives a live collaborative editable view over a list of items. It isn't designed to scale to www.microsoft.com usage scenarios. It is a low-tech solution that is useful for small teams of people who want to maintain simple, shared lists. This article includes a good amount of source code, which you may find useful.
The ListEditor is a general-purpose set of Active Server Pages (ASP) scripts that provide an editable list view on any XML file on the same Web server—so long as that XML file contains a special kind of simple inline schema that describes the shape of the list items. The user interface is built using DHTML.
You can check out a demo of this application by downloading the source files and copying them to your local intranet server. (The demo requires Internet Explorer 5 or later.) The sample list is a collation of feature requests received via the public newsgroup news://msnews.microsoft.com/microsoft.public.xml. When you enter the address http://localhost/listeditor/edit.asp?file=feedback.xml, you should see something like this:
You can sort the columns by clicking on the column headings. You can select a subset of rows to display by entering some filter expression. For example, if you enter "file" and press tab, you will see all items that contain the word "file" somewhere in the text of the item. You can also customize the view by selecting which columns you want displayed. The richer your schema, the more interesting your view becomes. All of this user interface so far is done with no round trips back to the server, which is a good way to offload cycles from your poor overworked Web servers.
You can make changes by first selecting a row, and then selecting the cell that you want to edit. Either an edit box or a drop-down list box will appear. You type in your changes and then click outside the edit box to see your changes incorporated in the list. When you are done, click the SAVE button. Your changes will be sent back to the server to be merged into the master document on the server.
If you try to update the same cell as someone else at exactly the same time, your update may fail—in which case the save process will report a conflict. You will then have to click the SYNC button and resolve the conflict before saving again.
Where it gets interesting is the "polling" feature. At the bottom of the window, a polling interval input box defines the number of seconds between polling events. When this polling value is nonzero, the page will ping the server asking for updates at the specified interval. As other users make changes, those changes are highlighted on your machine automatically.
You can also edit the schema by clicking the Edit Schema button at the bottom of the list. More on this below.
A more complete help file is available with the application.
The ListEditor is made up of the following files:
|Edit.asp||This is the main enty point to the application. Notice that the URL used to edit the above XML list is http://…/edit.asp?file=feedback.xml.|
|Feedback.xml||This is the list that I built from the microsoft.public.xml newsgroup. It happens to be in the same location as the edit.asp, but this doesn't have to be the case; it could live somewhere else on the same server.|
|Global.asa||This uses ASP Application state to maintain the current change log for distributing updates to the current set of clients.|
|List.css||The simple style sheet gives a nice orange table design.|
|Merge.asp||This ASP file takes a set of <update> elements and applies them on the master document stored on the server.|
|Merge.js||This is the companion file for merge.asp; it runs on the client side to merge the changes received from the server.|
|MergeSchema.asp||This ASP file merges changes to the schema—adding and removing columns, renaming columns, and so forth.|
|MimeDefault.xsl||This is a simple style sheet used by the SHOW XML and SHOW XSL buttons.|
|NextId.asp||This ASP file returns a unique ID for a new row when the user clicks the ADD button. This is synchronized across all users so that no two users try to grab the same row ID.|
|Reset.asp||This clears the current update log, which is stored in Application state. This is for the administrator to use.|
|Save.asp||This is the entry point for saving updates from a specific client and adding them to the change log in Application state. It also checks for conflicts.|
|SaveSchema.asp||This is the entry point for saving updates to the schema and adding them to the change log in Application state. It also checks for conflicts.|
|SchemaView.xsl||This is the XSL style sheet used when editing the schema.|
|Sync.asp||This sends down the current set of changes from a given timestamp. Clients use this to manually sync up when they are not polling automatically.|
|Template.xml||This is an empty list used for creating new lists. Try the URL http://.../edit.asp?file=template.xml to see what this looks like and to edit it!|
|Usage.htm||The help file.|
|User.asp||This ASP file does user authentication to get the username, which is then sent to the client so you know who's changing your list.|
The XML Format
The XML source for the feedback.xml file looks like this:
<?xml version="1.0" encoding="windows-1252"?> <list next="129" timestamp="2" schemastamp="1"> <Schema next="6"> <ElementType name="item"> <element id="s1" type="id"/> <element id="s2" type="area" values="ADO,DTD,DOM,Parser,MimeViewer, DSO,XSL,XLink,Schemas,XPath,Interop"/> <element id="s3" type="name"/> <element id="s4" type="description"/> </ElementType> </Schema> <item><id>1</id><area>ADO</area><name>XML String</name> <description>Get recordset into XML in-memory.</description></item> <item><id>32</id><area>ADO</area><name>Custom Schema</name> <description>Get ADO to spit out XML according to custom schemas</description></item> ... </list>
The entire list of features is 13,900 bytes, which downloads pretty quickly even at home over a slow modem.
The inline schema is not a real XML-Data or W3C Schema. It is just a minimal schema required to make the editor work. The editor requires an ID on every row. When a new row is created, the NextId.asp script takes the "next" ID from the <list> element and assigns this to the new row. When a new column is added, it takes the next schema ID from the <Schema> element and prefixes it with "s".
The editor supports one outer <ElementType>, which defines the row. The row can contain both attributes and elements. For example, the ID could have been defined as an attribute instead of an element.
The editor can support "sparse columns." For example, when you add a new column, it does not add the corresponding child elements to all the rows. Instead, it lazily creates the child elements as they are populated with non-empty values.
To get the table-view display, an XSL style sheet is generated on the client from the schema, allowing you to edit the schema without having to worry about also updating the XSL. This is done by the GenXSL() function in the utils.js file, which walks the children of the <ElementType> in the schema and generates table column headings for each <element> or <attribute> it finds. It then generates an <xsl:for-each select="..."> statement containing a template for the table rows, and it looks up the customized-view check boxes to see which columns to include in the style sheet. Each cell contains an <xsl:apply-templates/> statement instead of <xsl:value-of/>, so that you can use HTML markup—such as bold, anchors, lists, and so forth—inside the table cells.
Editing a Value
To illustrate how it all works, I will describe the way a simple cell value change happens. Suppose I want to edit the description of the following item:
First, I select the row. At this point the HandleMouseDown() function is called in utils.js, and it highlights the selected row and saves it in a variable. When I click in the description column, HandleMouseDown() notices the row is already selected, and it calls EditCell(). EditCell() calls GetNodeFromCell(), which uses the first column in the table and the item name defined in the schema to assemble an XPath expression that is guaranteed to find the XML node associated with that row. It does not use the row index, because the table could be sorted—in which case the row index is useless.
EditCell() looks up the schema for the specific node to see whether it defines a values attribute for that column. For example the Area column defines the values "ADO, DTD, DOM, Parser, MimeViewer, DSO, XSL, XLink, Schemas, XPath, Interop". If it does define values, EditCell() creates a drop-down list box containing those values, with the current value selected; otherwise, it displays an edit box containing the current value, then it positions this list box or edit box over the top of the table cell with some tricky DHTML code. In this case, there are no fixed values for the description field so the result on screen is this:
I can now edit this text directly. Let's say I want to put a comment in there about MDAC 2.5, which has this requested feature. I type away—and then I click somewhere outside this edit box and I see the following:
Behind the scenes, the Commit() function was called to store my edits back into the local XML document. The original XmlNode was attached to the edit box as an expando property, so Commit() simply grabbed that and compared the before and after values. Noticing that the values had changed, it updated the node, updated the table cell, and stored away a little <update> element describing this change in another local XML document called the "updatedoc". When I click the PENDING UPDATES button, I see the following:
<updates schemastamp="1"> <update type="modify" location="//item[id='1']" by="clovett" name="description"> <old>Get recordset into XML in-memory</old> <new>Get recordset into XML in-memory (MDAC 2.5 supports save into Istream which means you can save into an IXMLDOMDocument directly).</new> </update> </updates>
This is a self-contained description of the change—containing both the old and new values for the changed cell which can later be transmitted to the server for merging with the master document. It is only 319 bytes which makes for an efficient upload.
Saving the Change
So far, no other user knows that I'm doing any of this. At this point, I can decide to abort all my local changes and just navigate away from the page or hit F5, or I can save my change. To save changes, ListEditor uses the Microsoft.XMLHTTP control to HTTP POST the updatedoc to the Save.asp script on the server.
The Save.asp script first calls the Application.Lock() function to serialize Save operations, then checks to see if the schema has changed since the user last synced. If the schema has changed, Save.asp aborts the save and tells the user that the schema is out of sync. (Merging changes after the schema has changed is more difficult—but in most cases, it can be done. I just decided it wasn't worth the effort to write all that code for what I considered an edge case at the time. Extra points to the first person who posts the code for this).
If the schemas are in sync, Save.asp loads the master document off disk and calls the Merge() function in the included Merge.asp file. The Merge() function walks the <update> elements and applies the changes to loaded master document. In this case, it is of type "modify" so Merge() calls the Modify() function passing the node that it located using the XPath expression contained in the update-gram. First, Modify() checks to see whether the node exists already; if not, it creates the node in the right location based on the relative location of the matching element in the schema.
Next, it compares the old values. If the old values do not match, then the Merge() function fails and an error is returned to the client. This means someone else updated this same node after you last synched up. If the old values do match, Merge() replaces the old children of the updated node with the new children. (It does this instead of using the .text property, so that it preserves any HTML markup, such as <B> or <A HREF="...">, that you may have entered into the cell manually).
Similar code exists for Insert and Delete. If the Merge returns OK, Save.asp adds the updates to the shared update log in the Application state on the server, updates the timestamp in the master document, and saves the document back to disk. Then it releases the application lock and returns a result string to the client.
Other users can manually sync to your change by clicking the SYNC button, or they can turn on automatic polling. In either case, they will see the following updated display on their screen. This tells them that the user "clovett" changed the description of item ID 1.
This is all done by the SyncUpdates() function in the Merge.js file. SyncUpdates() uses the Microsoft.XMLHTTP control to do an HTTP GET on the Sync.asp script on the server, passing the local timestamp attribute value as a URL parameter. Sync.asp does a selectSingleNode on the shared update document object with the XPath expression:
"updates/update[@timestamp>" + timestamp+"]"
Sync.asp sends any matching <update> elements down to the client. The client then basically executes the exact same code as the server to merge the changes into the local document. When it's all done, Sync.asp calls UpdateDisplay() to redraw the table view.
The pretty colorization is done by attaching an updated-by="clovett" attribute to the <description> node during merging. The XSL style sheet transfers these attributes to the HTML table cells. ShowUpdates() is then called to walk all the rows and cells looking for these attributes. If it finds them, it assigns a color to the user name and highlights the cell in that color. It also displays a legend showing all the user names and their associated colors. When a lot of users are updating the same list, the list can get rather colorful.
Editing the Schema
The cool thing about editing the schema is that it uses a lot of the same code. When you click on the EDIT SCHEMA, button you will see something like this:
The SchemaEditor.asp page, a clone of the Edit.asp page, re-routes the save operations to the SaveSchema.asp. To enable editing, SchemaEditor.asp simply cooks up a "schema for editing the schema," which looks like this:
<Schema> <ElementType name='element'> <attribute type='id'/> <attribute type='type'/> <attribute type='values'/> </ElementType> </Schema>
This defines three columns—all stored as attributes. When you edit this, the same kind of <update> elements are generated. The only difference is that they are saved using the MergeSchema.asp page, which must update both the schema and the existing rows when necessary.
Adding a New Column
Suppose I want to add a new column called "status," which I'll use to communicate the status of each feature request. I select the name column, click INSERT, and enter the type name "status" with the values "Idea, Scheduled, Done." This results in three <update> elements , an insert element, and two modify elements, as follows:
<updates schemastamp="0"> <update type="insert" location="//ElementType" by="clovett" id="s8" before="//element[@id='s3']"> <element id="s8" type="" values="" /> </update> <update type="modify" location="//element[@id='s8']" by="clovett" name="@type"> <old></old> <new>status</new> </update> <update type="modify" location="//element[@id='s8']" by="clovett" name="@values"> <old></old> <new>Idea, Scheduled, Done</new> </update> </updates>
When I click SAVE, other users who are polling for updates on the Edit.asp page will automatically see a new empty column called status appear on their screens. I can go back to the main Edit.asp view and edit the status column values; a drop-down list of choices will appear, because I specified predefined values in the schema, as follows:
Similarly, if I delete a column, the column will disappear. Be careful with this one because it will also remove all the matching columns in the data as well. You can also rename a column and it will rename all the matching columns in the data for you. The code that does all this is in MergeSchema.asp. It's only 195 lines.
One thing that is not hooked up in the SchemaEditor.asp page is the ability to show multiple users editing the page. This would be an interesting exercise for the reader.
Even though ListEditor is a relatively low-tech little application, it's really easy to move around from server to server, and it provides an easy-to-use UI that offloads cycles from your server to your rich clients. I've used ListEditor for a while now, so it also should be pretty free of bugs.
Some follow-on ideas for improving this editor are:
- Make the merge-changes code more tolerant about changes in the schema.
- Make the schema editor collaborative, so that it also shows updates from other users.
- Add code to help resolve merge conflicts, rather than having to resolve them manually.
- Provide richer choices for the inline schema language, such as data type support—or, even better, support real XML-Data inline schemas.
- Richer editing UI using an HTML rich-text editor.
- Save the change log to disk, and provide multi-level undo.
- Generalize the schema and editor to allow XML Hierarchy, then enable display by different groupings.
- Hook up spell checking, either on the client or as a Web service.
- Provide more Web service-oriented admin stuff for managing all your lists on the server—creating new lists, moving lists, finding all the lists you own, and so on.
Chris Lovett is a program manager for Microsoft's XML team.