Implementing Smart Tags with Enterprise Applications
This content is no longer actively maintained. It is provided as is, for anyone who may still be using these technologies, with no warranties or claims of accuracy with regard to the most recent product version or service release.
John R. Durant
Architechs
January 2002
Applies to:
Microsoft® Office XP
Summary: How to implement smart tags with emphasis on data caching, performance, and security. (8 printed pages)
Contents
Introduction
Performance
Security
Conclusion
About the Author
Introduction
In the Planning Smart Tags With Enterprise Applications article, some guidelines are given that lead to the proper conception of the smart tag solution. The focus is on thinking about the size of lists, the amount of updating needed, and good candidates for data sources. These are vital considerations when you are putting together a strategy for an enterprise smart tag implementation. But, this is only part of what goes into building a solid foundation for your implementation. Once you have figured out what type of smart tag you are going to implement, you need to know how it should be built, how it works under the hood. In this article, our focus shifts to the actual implementation of smart tags. It seeks to answer questions such as:
- What are some of the security implications related to a smart tag implementation in an enterprise?
- How do I handle security credentials in my smart tags?
- How can I limit trips to my data sources?
- What is the best way to store data on client workstations?
- How can I make my smart tags perform better?
In order to answer these and other questions, this article has been divided into two sections: Data Caching and Performance, and Security. Each section focuses on providing guidelines and best practices. In order to fully profit from the information in this article, you should read the Smart Tag SDK documentation and the Planning Smart Tags With Enterprise Applications article first.
Performance
In the Planning Smart Tags With Enterprise Applications article just mentioned, we put a lot of emphasis on how to select and access a data source for your smart tags. The assumption is that databases are undoubtedly a common source for smart tags. But, once I have established access to the data source, how should terms be handled? Imagine the impact of tens of thousands of clients hitting a database server (or cluster) to retrieve very large lists. Not only the database server but also the network and other resources can become overwhelmed by such loads. The same holds true for a Web service or some other data source. Equally improbable is that you would allow thousands of workstations to connect to a database server in a simple two-tier architecture to download small and large amounts of data. Three-tier client-server computing addresses many of the well-known limitations that appear in two tier client-server systems as they try to scale beyond the departmental level. These limitations include performance, security, and management issues. Simply put, you need to minimize trips to data sources, keep the amount of data downloaded to clients as small and focused as possible, and allow for a more manageable and efficient architecture. But how?
Data Caching
One way to help meet these performance requirements is to combine the advantages of both data caching and term filtering. Data caching has to do with storing data on the client so that trips to the data sources are limited. But, storing hundreds and thousands of terms that users will rarely if ever need seems rather senseless. The key then is to determine what users are really up to. You need to find out what range of data users most likely need. For example, out of the list of hundreds of thousands of invoices our company has sent over the years, how many does a user really want to use in a typical work task? We'll look at the best ways to filter the data a little later on, but for now, assume you have whittled the necessary list of data from several hundred thousand records down to a fraction.
When you have scoped the data range, your work is cut out for you. You are in familiar territory because, despite the fact that you are developing a smart tag solution, which is a new technology, you are really trying to develop an n-tier database-driven application. In this enterprise scenario, you would not typically create a two-tier database application with client applications connecting directly to the database. You would most likely implement some middle tier application layer that fields client application requests, processes some business logic, and interacts with the database server.
In summary, you need a layer that can get the subset to the client workstation and let your smart tag DLL work with that list. This middle tier layer can be a traditional COM-based application or, in a more innovative vein, you could implement a Web service. Whether you choose to implement the Web service using the SOAP toolkit or using Microsoft® Visual Studio® .NET is up to you. Either way, you can create a Web service that receives requests from clients, including required security credentials, and passes them along to a database server of some kind. It could also retrieve data from another Web service, a LOB application, or from a Web site and then return the data to the calling workstation.
One way to focus the data subset for users is to use the power of the Active Directory®, user authentication, and group organization. If your data source is an application that can implement integrated Microsoft Windows® authentication, things get simpler. For example, if you are using Microsoft® SQL Server™ with integrated Windows Authentication, you can build stored procedures or views to which users in a finance department or other group are given permissions. So, you can create a security group that represents the people in the finance department who handle invoices (or some other group of people who require similar permissions) and give that group permissions to the stored procedure. As users load the custom DLL, a request is sent to your middle tier which in turn invokes specific stored procedures. As the connection is made to the database server, the credentials of the user are passed along to the SQL Server before invoking the stored procedures. In this example, your stored procedures could return the invoice numbers from just the past thirty days or even more recent if need be. Without a great deal more effort, you could even filter the returned records even more to a list containing only the past month of invoices for a specific user rather than a group. Again, because you have developed an n-tier application, you can add more sophisticated business logic to the middle-tier and make the request for data very refined.
Once the records are returned to the middle tier layer, they are passed back up to the client and stored there. On the client, you have options for how the data range should be cached. The data can be stored in an XML document or data island. Using XML makes a lot of sense as the data can be easily consumed by future smart tags and programs. Also, it can be more easily transformed, formatted, and manipulated. If you are using a Web service to field the request for data and return the data range to the client, the service relies so completely on XML to transfer messages between the client and the server, storing XML on the client could not be easier. Taking the clever cue from MOSTL files, you can cache data for a custom DLL in an XML file as well. In other words, if you were to create your own DLL that works in a fashion similar to MOFL.DLL, you can use an XML file to store the recognition list, actions, update information, and so forth.
Moving from a simple two-tier architecture to one that implements a middle tier to field requests, retrieve, and then return data solves another dilemma: managing changes in underlying data sources. For example, imagine you have swapped out one database server for a new one, perhaps a migration from Sybase to Microsoft SQL Server 2000. In a more primitive two-tier architecture where your smart tag DLL contains all of the connection information, and so forth, you will need to update the smart tag DLL throughout the enterprise. This is one area where using a Web service can save the day.
Not only are Web services many times easier to implement, deploy, and maintain than a DCOM or other remote application architecture, they are easier to develop and understand as they receive requests and respond to clients over HTTP on your network. Simply put, they are implemented via IIS and an ISAPI DLL. As such, you can take advantage of the security features of IIS and Windows authentication (more on that in the section on security). The point is that by using a middle-tier you need not adjust the smart tag DLLs that handle recognition and actions as database servers are migrated or as passwords change. In summary, the middle tier allows you to keep the code in the smart tag DLL focused on its main objective: recognizing text, matching it up with specific types, and handling special actions invoked by the user.
Using this middle application services tier brings up a few other things that need to be considered. First, keep in mind how dependent your smart tag infrastructure is upon the performance and availability of the middle tier. This application layer has to be ready at all times and it should not take long to do its work. Create a layer that is lean and efficient, and place it in a network context that allows it to service its most likely customers. For instance, placing your middle tier on a slow or highly trafficked network segment will obviously reduce its effectiveness. You will probably want to create a cluster of servers for the middle tier and implement a load balancing strategy. Chances are you already have such a strategy implemented for servers that act as data sources for enterprise applications, and you may already have one implemented for custom n-tier applications.
Pattern-Matching
Caching data is a great way to minimize the impact on your network, servers, and client workstations. As words are recognized, your smart tag DLL checks against a small, focused, and highly relevant list. However, caching is great as long as you are not trying to cache inordinately large lists. One way to achieve efficiency and accuracy in your smart tag is to use filtering. Filtering means that your smart tag DLL takes a good hard look at the text passed to the Recognizer before looking in any lists at all. For example, if a user types a number, how does the smart tag know if it needs to immediately go check lists and databases and see if the number exists? It makes no sense at all to kick up a lot of dust in our data sources unless we know what the number really means. Is a user just telling a friend how many times they have been to a Neil Young concert or does the number represent an actual order ID?
The more intelligent your filtering, the more likely it is your smart tag will sit quietly before springing into action to retrieve the right information and propose the right actions. Pattern-matching is one way to filter text. This amounts to comparing the target text with a known textual and looking for a match. You can use regular expressions to filter the text passed to the Recognizer before it is compared against the recognition term list. Use of the RegExp object (available via the VBScript_RegExp_55 library) is well documented in MSDN, but the basic implementation begins when you set a pattern. Then, check whether the text passed to the Recognizer fits the pattern. If it does, make a call to an application services layer (as discussed in the previous section) to retrieve more information. So, if we know that orders are always in form of three digits followed by a hyphen and then four more digits like this, ###-####, we can find out if the target text is in this pattern. If so, we can retrieve the order number and supplemental data from a data source via our middle tier. The number of times the middle tier will need to field requests can thus be greatly reduced, and the smart tag host application will commit smart tags only when the likelihood of text being of a certain type is very high.
Another way of filtering is to look not only at a number or word itself, but you can look at the text that surrounds it. For example, employee IDs may always be written as "EMPID 2343434." Most companies have conventions such as these for purchase orders, invoices, employee records, accounts receivable, and so on. Another example would be for projects. Your organization may list projects in a specific way such as PRJ 00003 MN. In these examples, the pattern of the numbers themselves do not tell us much about the nature of the data. For example, project numbers could be three characters or maybe 10, and other things, like an invoice or order number, could have the same number of characters. However, the first prefix characters indicate that we are working with a project, and the last two tell us that the project is being implemented in Minnesota. Using this information, we can know that when we go looking for additional data, we do not need to worry about projects for any other state.
Using pattern-matching in this way means that the smart tag DLL may need no database access at all, even via a middle-tier. So, if our smart tag has recognized PRJ 00003 MN as a project number, it can then request the current project status and project team member list in response to an action. To be sure, there is a risk that the project number was incorrectly identified, such as when a user types an invalid project number in the correct pattern, and so the subsequently requested project data comes back empty. You should code your smart tags to gracefully respond to these errors. In an enterprise setting, you are continually confronted with decisions that require trading performance for security, accuracy for performance, and so on, and implementing smart tags is no different. You need to decide if it is worth having occasional mismatches so that the smart tags hit your databases less frequently.
One way to strike a balance between accuracy and performance is to implement both pattern-matching and some client-side data caching. Taking up our project number example above, this would mean caching a list of active projects or some other subset of all project numbers on the client workstation. Chances are very likely that users will be composing documents relating to active projects that will complete very soon, so eliminating completed, postponed, or cancelled projects from the locally cached list will probably not bother too many people. Then you can still use pattern-matching to find out if actual project numbers are being typed before looking in the local list. If the item is not in the locally cached list, you can allow the smart tag to search more broadly.
In this composite approach, pattern-matching is done on the client, and there are other algorithms in the tier that contains your business logic. In this business logic tier you can filter based on dates, aggregated values, and whatever data elements are important to your organization. You can also filter based on the security credentials of the user who has requested the data. This leads us to another main area of consideration when implementing enterprise smart tags: security.
Security
In the Smart Tag Installation and Security for Microsoft Office XP article, the basics of smart tag security are explained. These deal mostly with macro-level security and settings within the host application itself. But smart tag security extends beyond these things. For example, what is the best way to handle authentication for term list data sources? How is security handled when I invoke certain actions that work with corporate applications and other resources?
Integrated Windows Authentication
In the section above, some attention was given to the use of Integrated Windows Authentication for handling authentication and permission control when connecting to data sources. This works well for applications like SQL Server, Microsoft Exchange Server, Microsoft SharePoint™ Portal Server, and Microsoft Commerce Server whose authentication services are woven into the Microsoft Windows 2000 security model. Clients running Microsoft Office XP can pass their credentials to a Web server or other .NET Enterprise Server for authentication and gain access to resources. Use integrated authentication where possible because it makes your smart tag DLL and its supporting application tiers easier to maintain and deploy.
Non-Integrated Windows Authentication
While integrating with Windows 2000 security is the best way to handle authentication in most instances, some applications, like a proprietary LOB application, may not support integration. Hence, they handle security on their own. As stated before, in most cases, you should avoid having your smart tag DLL connect directly to a data source. Normally, a middle tier of the application infrastructure should handle this for you. However, if you need your smart tag DLL to have direct access to connection information in cases like this, you have a few options. First, you could embed security credentials right in your smart tag DLL. The good part is that it hides the connection information from the user so that usernames and passwords cannot be discovered. But, while this technique will provide access, it makes your application harder to maintain. If credentials change, the DLL needs to be re-deployed.
One way to accommodate changes to credentials is to create a security COM component that maintains authentication information. This component is then implemented in your smart tag DLL or in one of its application tiers. When credentials change, only the support component DLL needs to be deployed anew. You can use the techniques listed in the Deploying Smart Tags with Enterprise Applications article to deploy both the smart tag and any support components. One of the disadvantages of this technique is that you still need to re-deploy the component on all workstations any time changes are made to the security credentials. The same is true if the underlying data source is changed, as when a database is moved to a newly installed server. You would also need to re-deploy the component if it uses the server name in the connection string. Most importantly, this type of implementation presupposes that the DLL on the client workstation is going to connect to the database and is a move away from the n-tier application architecture advocated in this article.
If you really must store connection information on the client, you can use an external data file for the data source and other connection information. Use XML to mark up the file and place the requisite data source and authentication data in it. You should encrypt the data in the file or encrypt the entire XML file. The smart tag DLL should have a routine to decrypt the contents of the XML file so that the connection information can be read. The advantage to this technique is that any time the connection information is changed, the simple XML can be updated, and the new connection will be used. Additionally, other client applications that require the database connection information could use the same XML file.
There is no doubt that updating an XML file on client workstations is many times easier than updating a DLL. With a file, you do not run any risks of registry conflicts and so forth. To update the file, use a Web-based mechanism the way the MOSTL tool does. Inside your custom smart tag DLL, embed logic that checks for an updated XML file periodically just as the MOSTL XML files does. You can then place the XML file on a Web server and have it automatically downloaded to the client workstation where the new information can be decrypted and used by the smart tag application.
Web Services and Authentication
Using Web services as the middle-tier can make the security implementation a lot easier. As mentioned earlier, Web services on Windows are implemented with IIS, so you should get familiar with how to configure IIS in a secure environment. For example, disabling anonymous access on your Web directory in IIS seems like a good place to start, and it's probably a good idea to avoid basic authentication as well, because the user names and passwords will get shipped across the network in clear text. Using NTLM offers some nice advantages because all of the Windows authentication credentials are passed from client to server using the encrypted NTLM protocol. You could use Kerberos as well, but this presumes that all of your clients are running Windows 2000.
So, let's see how using NTLM would work if you implement your middle-tier as a Web service. If the client workstation supports NTLM and the Web service is configured to support NTLM authentication, any anonymous Web request failing due to permissions will result in automatic use of the NTLM protocol to send a username and encrypted password from the client to the service. The client request will then be reprocessed, using the client's user information. The user account obtained from the client is what the user chose when logging onto the workstation. As this account, including its Microsoft Window NT®/Windows 2000 domain, must be a valid account on the IIS server running the Web service, NTLM authentication is most useful in an intranet environment, where the client and server machines are in the same, or trusted, domains.
If the user has not been granted permission to use the Web service, the WSDL will not be visible to the client. Once authenticated by IIS and the Web service, the client credentials can be used when accessing databases or other data sources. The Web service could choose to implement different security on the server. So, while a user may not be a member of the Administrators group, the code in the application layer could perform certain tasks permitted only to administrators. In a .NET Web service, your application can impersonate the identity of the client on whose behalf the application is operating. Studying other .NET topics such as role-based security, code access security, and the use of config.web files will be of great benefit as you move to using .NET Web services in your enterprise smart tag implementations.
Locking Down the Registry
Smart tags use the registry for some of their settings, such as whether they are enabled or not, which applications can use the smart tag component, and where to look for new smart tags and actions. In most cases, you will not want users changing this data, but some of the settings, such as enabling and disabling the smart tag, can be done via the dialog box in the host application. One way to keep users from altering these settings is to lock down registry access by using system policies as part of your enterprise administration.
Digital Certificates
Digital certificates have been discussed elsewhere as a way to make sure that only safe smart tags are installed on a client workstation. In other words, only smart tags with a valid certificate from a trusted source can be installed on the workstation. While this is a great line of defense for protecting clients from maliciously designed smart tags, it can also be used as a way of authenticating users for access to resources. For example, when an action is invoked, some special credentials may be needed for the action to succeed. In a case where the resource has no access to the security credentials provided by the client's login information, a digital certificate is convenient alternative.
Using a certificate from a trusted Certificate Authority (CA), credentials can be passed that will then allow the resource application to disclose data or initiate some other operation. Using certificates can be used when invoking actions, and it can be used for connections made to data sources when retrieving recognizable data. Each client workstation can have a unique certificate, and this may be useful in a situation where each user has very refined security needs. On the other hand, you can deploy the same certificate on multiple workstations to allow similar authentication on a variety of workstations.
Implementing certificates comes with some costs. There are administrative tasks such as interacting with the CA and acquiring the certificates, or setting up a private certificate server. Then, they must be deployed. In a large enterprise, this is a big job. Fortunately, there are third-party software applications specifically designed to make this job easier for systems administrators, or Microsoft Systems Management Server can be used to deploy the certificates.
Conclusion
In this article, we have touched on a number of factors for consideration in implementing an enterprise smart tag. Ultimately, you want your smart tags to deliver the maximum end-user value for the least cost of performance and administration (see the Deploying Smart Tags with Enterprise Applications article for more on administration and deployment). In an enterprise setting, a smart tag will need to connect to and work with a potentially wide variety of data sources and systems. As an enterprise smart tag developer, think beyond the scope of the workstation and consider issues like network performance, network security, database security, enterprise application interfaces and exposure to create smart tags that can leverage the power of immense corporate wealth. Following the guidelines in this paper and implementing solid, general application development practices will make your smart tags reliable, robust, and welcomed by your final audience, the end-user, without dramatically impacting the enterprise network and application infrastructure.
For more product information, see the Microsoft Office Web site.
For more technical information, see the Microsoft Office Developer Center.
About the Author
John R. Durant independently specializes in collaboration and Web technologies. He provides technical and project leadership, authorship, and speaking services. He earned a Bachelor of Arts degree and a Master of Arts degree at Brigham Young University, and a Master of Arts degree at the University of Minnesota. He is a Microsoft Certified Solution Developer, Microsoft Certified Systems Engineer, Microsoft Certified Database Administrator, and a Microsoft Certified trainer. Readers may reach John at jrd@architechsIT.com.