Architecting a Knowledge-Management System

Ranju Saroch and Jean Barmash

December 2007

Summary: One key factor for the success of a knowledge-management system is to plan it around a critical real-world issue. It will also be successful if its future users can frame for themselves the goals and objectives of the system. (7 printed pages)

Contents

Introduction
Reinventing the Wheel
The Role of the Architect
Selecting the Knowledge-Management Platform
Initial Prototype
Customizing the KMS: Business Processes
Technology Customization
Conclusion
Critical-Thinking Questions
Sources
Glossary
 

It's not what you know, but whether you can find it.
–Ranju Saroch and Jean Barmash

Introduction

It was late at night on a Saturday, and the client was waiting. We had to come up with a proposal, quickly. Our company had created similar proposals for this client, but the senior partner who had those proposals was off on a surgical leave. We had no way to access those examples that would make our job easier. This was not the first time that something like this had happened. Many times, we would find ourselves wading through endless e-mail threads, scrambling to determine who owned critical documentation from a project that was executed months, or even years, earlier. This was symptomatic of how our company worked: There was no central place to store information; it was spread among various project team's hard drives, and hard to track. As a result, we reinvented a lot of wheels.

Reinventing the Wheel

A few of us who did not care to be in the "reinventing the wheel" business decided that we needed a centralized system to manage our shared corporate knowledge. We knew that information that was stored on the platters of countless hard drives constituted our firm's most valuable asset; but, without efficient access, so much of it was as good as lost. This was doubly true for consulting organizations such as ours, where people jumped from project to project, needed to ramp up quickly for every new situation, and yet might find themselves without enough time to create proper documentation or other assets that they might need to get their job done.

We knew, coming in, that we were facing a big task; but we had to start somewhere, so we started with two modest goals:

· Searching for past proposals—Past proposals tended to exist on people's desktops or shared drives, which made searching for a particular proposal very difficult. We wanted our system to help us easily find past proposals to help in future proposal writing.

· Collaboration through communities of practice—Because people seem constantly to be traveling or working at client locations, they would need effective access to a wide variety of knowledge. They would also need to collaborate with their colleagues. We wanted to organize their knowledge into easily identifiable communities of practice. These communities would provide colleagues with up-to-date information about each other's whereabouts, as well as enable them to collaborate on various projects from far-off places.

Because this was the first time that we had ever worked on a knowledge-management system (KMS), we talked with people from other companies who had already done what we had set out to do, to get the benefit of their experience. We learned that many products existed to address this issue, and that most of our efforts would be focused on selecting a system and customizing it to fit our organizational needs. Most of our customization activities would involve integrating the chosen KMS with our internal systems, configuring search engines, setting up security, and importing information from existing data stores.

On the project-management side, we learned that our biggest challenge would be getting support, commitment, and a separate budget from top management, especially in a consulting organization such as ours. The second biggest challenge would be to lower resistance from other employees, as a KMS would change the way in which they would work. Because of the expected lack of budget and corporate initiative, those who worked on the KMS would probably consider it overhead work. We secured funding for our KMS by selling it as a corporate-wide initiative. Additionally, we found champions within each department that would help us gather requirements and work through the adoption process.

The Role of the Architect

We worked in a team, together with a project manager and an information architect. My role as system architect was to make sure that we collected our technical requirements, produced a high-level system architecture and design, created an integration architecture, and determined infrastructure and scalability characteristics. I also would have to provide an estimate for how long our implementation might take.

People in different departments had different ideas of what these systems should do for them, from both functional and technical perspectives. A few departments already had homegrown efforts under way to implement Wiki portals, networks of shared file folders, and even relational database systems. We catalogued every one of these internal efforts, which often started out as small projects and grew into isolated silos of their own, where they would remain until some larger effort would subsume them. It was our job to integrate and centralize these systems into a single, common platform.

First, we needed to learn more about the kind of systems that were currently available in the market. After choosing an appropriate system, we planned to deploy a scaled prototype. Our goal would be to prove the value of such a system to management and, thus, secure additional funding for full development and deployment. We would demonstrate that our colleagues were already managing their knowledge; but, without the desired system in place, their efforts were not succeeding as well as any of us would have liked.

Selecting the Knowledge-Management Platform

Many of the knowledge-management systems that we reviewed offered more functionality than we had originally sought, but which we decided could be valuable to us in the longer term. The most supported were:

· Document management. Provides for the tracking and storing of electronic documents and/or images of paper documents, and directly addresses issues of document storage, retrieval, filing, security, archival, retention, distribution, workflow, and creation.

· Collaboration tools. Allow people who are involved in a common task to achieve their goals by providing e-mail, calendaring, text chat, Wiki, workflow management, and so on.

· Wikis. Represent a type of Web site that allows visitors to add, remove, and edit available content easily—sometimes, without the need for registration. This ease of interaction and operation makes a Wiki an effective tool for collaborative authoring.

· Federated search. Enables a user to search multiple independent, discretely mounted data sources or databases through a single search query. Federated search engines broadcast the user's query to a group of disparate data stores (documents, relational databases, file shares), merge the results, and then present them to the user in a succinct and unified format, and with minimal duplication.

· Web portals. Allow for quick and easy access to the information that is stored in the knowledge-management system over the Web and on the corporate intranet.

· Blogs. Are either company-hosted or personal Web sites that give employees the opportunity to communicate ideas in an informal environment. A typical blog combines text, images, and links to other blogs, Web pages, and other media that are related to its topic, as well as the opportunity for readers to leave comments in an interactive forum.

· Personalization. The ability to tailor data automatically to specific user characteristics or preferences. For instance, top Web portals allow site visitors to customize their home—or "My"—pages with their preferred physical location and choice of news categories, local weather reports, and other features.

On the technical-architecture side, I started to realize that these systems offered so much functionality that we might be able to use them one day to start developing unique Internet applications. I focused on evaluating the following key features:

· Pluggable security for single sign-on (SSO)

· Security capabilities of the product

· Ease of integration to existing databases

· Configurable federated search

· Portal capabilities, for use as intranet

· Content-management capabilities

· Overall extensibility of the product

· Whether the product required development skills already present in our organization

Some of the capabilities that these systems provided might not have been immediately useful to us, but they could have proved useful later as our users became more sophisticated in their use of—and expectations for—the system.

Initial Prototype

The initial prototype was made for the Business Development department. They often received many requests from people searching for past proposals and other related documentation. Sometimes, it was difficult to reply quickly, which frustrated those who requested the proposals. The Business Development team wanted us to create a database that would be easily searchable by other departments, so that they could find what they needed quickly and without having to involve Business Development team members directly. We worked with a business analyst to generate search requirements. We determined the type of searches that could be important to users, and identified metadata that was likely to be associated with proposals. The Business Development team wanted to be able to conduct searches based on a number of criteria, such as a client's name, industry, deal size, proposed technologies, and whether the proposal was a success or failure. Our information architect redefined the taxonomy for the metadata to maximize search effectiveness. We also augmented the property pages of the documents in our sample, in order to add the metadata. Proposal managers defined the metadata for each proposal, before loading them into the database.

The Business Development team also wanted us to create a collaboration area where people who were working on a particular proposal could collaborate through discussion boards, chat, and track the progress of active proposals. The collaboration area would act as a central repository for all proposal documents, taking the place that was currently occupied by shared hard drives. The goal was to use as much out-of-the-box functionality as the KMS could provide—with more customization to follow, should the project go forward.

Wireframes (static prototypes) were created for the search queries and data-entry pages for the metadata to elicit approval for the design. The initial prototype was scaled for 100 proposals. We hired a consultant to help set up and customize the software, which helped us finish our prototype within the one month that we allotted ourselves for this effort. We lowered our risk of failure by using a subject-matter expert, who was provided to us by the Business Development team, to help us define the metadata for each proposal. The success of this prototype was crucial, as it would serve as the basis for seeking additional funding to further implement the KMS solution within the organization. The prototype was released on time to the Business Development team and upper management; it worked well, and it was well received. Our team was very excited. We were finally seeing our vision become reality.

A decision was made to go forward with the initiative, and we received more funding to bring this to fruition company-wide. We also received feedback on our prototype, including ways in which we could improve the KMS in the final implementation. For example, the people who migrated the documents into the prototype found our metadata entry screen cumbersome. Thus, we had to come up with a way to make that task easier. To solve this problem, we developed a spreadsheet-like interface in which people could configure metadata for multiple documents at the same time. In addition, we created a program that would upload documents automatically. Similarly, we were asked to improve our integration with commonly used word-processing applications, integrate this into our corporate single sign-on system, and improve our searches. We also received feedback suggesting that we organize our data taxonomy differently.

Customizing the KMS: Business Processes

As soon as we got the go-ahead, it was back to the drawing board. We had several challenges—some technical, some on the business side. We needed to make sure that we understood our requirements, and that we had properly mapped the technology to the needs of the business. After all, that is what architects do.

We developed an information architecture, or a way in which to set up our implementation so that the information would be organized in a way that made the most sense to our company. We were a matrix organization—organized along both business verticals (automotive, telecommunications, financial services) and functional groups (development, consulting, and sales). This meant that we had to give each knowledge area two classes of metadata. In addition to the information architecture, we were tasked with managing the complete information life cycle, which included capturing, organizing, and archiving documents, proposals, and all other information.

As a result of our efforts, we began to witness important changes that occurred to business processes within our company and, even, in some cases, to the corporate culture itself. Knowledge-management systems are not very useful in environments that are overly secretive or overly competitive, so we had to find ways to reduce these barriers. We also needed to provide our colleagues with incentives to share their information with one another. One way in which we handled this was to appoint, for each department, a person who was responsible for organizing knowledge for that department. It was up to that person to figure out how they wanted to obtain and manage that knowledge, with our support.

Technology Customization

As soon as we knew what we wanted our information architecture to look like, it was time to implement the solution on top of the product that we chose. We created a flexible document classification that allowed users to search for—and navigate quickly to—their area of particular interest. For example, "automotive training" could be found associated with both "training" and with the automotive business unit.

We still needed to integrate all of the existing legacy knowledge-sharing systems into our own. To minimize the impact on our day-to-day business, we decided to let each department plan its own migration schedule. Slowly but surely, however, we search-enabled relational databases one by one and integrated one legacy system after another.

In addition to proposal documents that were the focus of our prototype, we also started to add other document types and the metadata that was associated with them, such as project plans, technical-architecture documents, design specifications, requirements, and so on. After that, we tackled search issues. We configured our search engine to index not only all documents that were stored within the system's repository, but also those documents that were found on departmental file shares, Wikis, and intranet Web sites—at least, until they were be migrated over to KMS itself. Additionally, we provided the ability to constrain searches to a specific knowledge domain. For example, we allowed the user to search only within the Telecommunications department, or only within HR.

Information security is important to many organizations, especially ones that have sensitive customer data. Therefore, security was one of the most important things on the minds of all of our managers. We had to integrate with the SSO and entitlement systems that our company was using. Our KMS platform supported custom extensions to the built-in security model, and we were able to take advantage of that. We set up group security, and then had each department designate a person to manage its security. We had another interesting problem wherein the search engines would find documents that people were not entitled to access. In most cases, it was decided that the search results should still be returned, and that the user could request access from the owner of the site. However, a few departments—namely, accounting and legal—asked us to remove the restricted results from the corporate-wide searches.

Another challenge was to take much of the information from the hard drives and other systems, and import it into the KMS document repository. A few departments wanted to stop using their file shares immediately, so we had to provide import functionality early on in development and deployment. We extended some base functionality, and came up with a reusable framework that other departments would later be able to incorporate into their own legacy solutions.

Of course, we had our share of the normal deployment issues—hardware complications, as well as storage, reliability, and backup problems—but, fortunately, our IT staff was on board to take care of most of those issues.

Conclusion

From a small skunk works, this grew into a huge project, and it led to promotions for a few of us who had originally shown the initiative. One key success factor was that our knowledge-management system was planned around a specific, critical issue that was faced by the company. We were also successful because its future users framed the goals and objectives of the system for themselves. This helped to encourage participation from the bottom up, which encouraged the continual migration of existing knowledge into the system, and helped to improve its value to our company. As data became more organized, people found new uses for it, and even started to build business-intelligence applications on top of it—such as reports and executive dashboards, workflows, and even Web applications that took advantage of the platform that we had chosen. Over time, our system became an application-development platform with wide implications.

Critical-Thinking Questions

· Why would a company need a system for managing knowledge?

· How is knowledge managed at your organization? Is there a centralized repository in place, or do you engage in impromptu sharing?

· How would you organize your project documentation, so that it is convenient for you?

· How do you see your company's business processes changing, if you implemented a knowledge-management system?

· For a knowledge-management system to be successful, the technical architect should select the KMS based on what criteria?

· What is the ROI of the knowledge-management system? How would you go about measuring it?

Sources

· [Abhishek 2005] Abhishek M. "Knowledge Management: An Initiative." The Code Project: The Scrapbook. 2005. (Accessed January 9, 2007.)

· [Journal of Knowledge Management Practice 2005] Abdullah, Rusli, and Mohd Hasan Selamat (Universiti Putra Malaysia); Sahibudin, Shamsul, and Rose Alinda Alias (Universiti Teknologi Malaysia). "A Framework for Knowledge Management System Implementation in Collaborative Environment for Higher Learning Institution." Journal of Knowledge Management Practice. 2005. (Accessed January 9, 2007.)

· [KMA Insights 2003] "What Is a Knowledge Management System and Why Would I Need One?" KMA Insights. 2003. (Accessed January 9, 2007.)

· [Moore 2006] Mitchell Moore, Meg. "The Next Wave of Knowledge Management." Microsoft Midsize Business Center. 2006. (Accessed January 9, 2007.)

· [Wikipedia 2007a] Wikipedia contributors. "Document management system." Wikipedia, the Free Encyclopedia. (Cited January 15, 2007.)

· [Wikipedia 2007b] Wikipedia contributors. "Community of practice." Wikipedia, the Free Encyclopedia. (Cited January 15, 2007.)

Glossary

Blogs—Found on either internal or external company and personal Web sites, blogs give employees the opportunity to espouse ideas in an informal environment. A typical blog combines text, images, and links to other blogs, Web pages, and other media that are related to its topic. The ability for readers to leave comments in an interactive format is an important part of many blogs.

Collaboration tools—Tools that allow people who are involved in a common task to achieve their goals by using mail, calendaring, text chat, Wiki, workflow systems, and so forth.

Communities of practice—A process of social learning that results from people collaborating over time on a topic of common interest. [Wikipedia 2007b]

Federated search—Federated search enables a user to search multiple independent, discretely mounted data sources or databases through one search query. The large volume of documents that are housed in proprietary databases is not open to traditional Internet search engines, unless the documents are exposed through a Web site. Federated searching transforms a query and broadcasts it to a group of disparate data stores (documents, relational databases, file shares) with the appropriate syntax—merging the results that are collected, and presenting them in a succinct and unified format, and with minimal duplication.

Personalization—The ability to tailor pages to individual users' characteristics or preferences, such as portals that allow site visitors to customize the page with selected news categories, local weather reports, and other features.

Web portal—Web portals can be integrated with information on knowledge-management system with the Web.

Wiki—A type of Web site that allows the visitors themselves to add, remove, and otherwise edit or change some available content easily—sometimes, without the need for registration. This ease of interaction and operation makes a Wiki an effective tool for collaborative authoring.

About the authors

Ranju Saroch works as a Technical Architect with ICF International at Fairfax, VA. She has more than 12 years of experience in technical-architecture design and development for Web applications and in applying usability engineering. Ranju has designed Web-based applications, using object-oriented analysis and design methodologies such as UML.

Jean Barmash is a Senior Consultant/Trainer at Infusion Development, where he is involved in architecture, project management, and hands-on development. As an integration consultant and technical lead for Trilogy, he worked on several large automotive projects. While working as principal software engineer for Symantec, Jean helped develop a product that focused on business-intelligence solutions for the data center. Well-versed in both .NET and Java, he is currently focused on Web business-intelligence solutions.

This article was published in Skyscrapr, an online resource provided by Microsoft. To learn more about architecture and the architectural perspective, please visit skyscrapr.net.