Export (0) Print
Expand All

SharePoint Developer Building Blocks: Technologies for Creating SharePoint Applications (Part 1 of 2)

SharePoint 2010

Summary:  Read a high-level overview of development for Microsoft SharePoint 2010and learn about the SharePoint developer building blocks in this two-part series.

You can use various languages, frameworks, object models, and tools to build scalable distributed web applications on the SharePoint 2010 development platform. It can be difficult for a developer new to SharePoint to know what to use to implement various scenarios, or even what is possible. This article contains a high-level overview of SharePoint as a development platform, compares and contrasts SharePoint development to other kinds of development, and lists the SharePoint Developer Building Blocks, which are the technologies that you use to build a SharePoint application. Perhaps the biggest challenge for you as a SharePoint developer is to understand what your choices are for building blocks, and to select the correct options that meet your architectural and design goals.

From a high level, it is useful to examine SharePoint development from several different viewpoints, such as the following:

  • How SharePoint development compares to and contrasts with other forms of development.

    • Scalable loosely coupled ASP.NET application development

    • Database application development

    • Traditional rich client development

  • What languages and frameworks must developers know?

  • How do document formats such as Open XML affect your application design? What new scenarios are possible?

This article presumes that the decision to develop on SharePoint has been made. This is the position that many developers are in. In some cases, developers are assigned to a project that is built on SharePoint. In other cases, developers are proactively developing their skills.

From a high level, the developer architecture of SharePoint 2010 consists of several layers of technology. It is an Microsoft ASP.NET application that is built on SQL Server, and runs on Windows Server 2008 or Windows Server 2008 R2. The following diagram shows how several of the more important building blocks use underlying systems to provide necessary functionality.

Figure 1. Diagram of key building blocks and underlying systems

Diagram of key building blocks and systems

It can be a bit daunting to see the breadth of technologies that a SharePoint developer must know. When I jumped into SharePoint development, I started from a spot similar to many other developers who enter SharePoint development. I was a competent developer with in-depth experience in some areas, shallower experience in other areas, and a cursory knowledge of many other areas.

I was competent in the following technologies:

  • Microsoft Visual C++

  • Microsoft Visual C#

  • Microsoft .NET Framework

  • LINQ

  • SQL

  • XML

  • Open XML

I had to deepen my knowledge in the following technologies:

  • ASP.NET

  • ASP.NET themes and skins

  • HTML and CSS

  • ECMAScript (JavaScript, JScript)

  • ASP.NET Web Parts

  • Internet Information Services (IIS)

Although I understood the architecture, as a developer I needed to learn the details of the following technologies:

  • SharePoint object model

  • SharePoint managed client object model

  • Microsoft Silverlight and XAML

  • Workflows

  • Windows Communication Foundation (WCF) and web services

  • SharePoint REST APIs

  • Excel Services

  • Windows Identity Foundation (WIF) and claims-based identity

My point is that even though I am an experienced developer, I still faced a significant learning curve. Each of these technologies requires some amount of study. The basic concepts are not difficult, but learning enough to be proficient can be a lengthy process.

In this article, I share my thoughts about various aspects of SharePoint development. I relate aspects of SharePoint development to previous experiences. What was similar to previous development environments that I worked in? What was different? I also discuss how I think of SharePoint development. What comprises a SharePoint application? What are my options for building one? What is the big picture, and where do I have to drill down when I implement a particular scenario? My goal is to outline the key aspects that you must know to be effective at SharePoint development.

To summarize, SharePoint Server is a system with the following capabilities:

  • Sites. Websites both inside and outside the firewall.

  • Communities. Collaboration through wikis, blogs, and more.

  • Content.Enterprise Content Management, Web content management, andpublishing systems.

  • Search. SharePoint products and technologies include a search engine that you can customize for your needs.

  • Insights. Business intelligence.

  • Composite Applications. Includes data access, user interface, processes, and infrastructure.

SharePoint products and technologies provide a scalable architecture that can be appropriate for several users, or 100,000 employees of a large corporation. For websites outside the firewall, SharePoint Server 2010 for Internet Sites enables you to build scalable, flexible websites that you can maintain with the ease of maintaining a SharePoint site.

In most cases, when you build a SharePoint application, you are adding additional capabilities to the core functionality. You may want to streamline the user interface for some important users. You may want to provide some custom capabilities in several Web Parts for a particular vertical market such as construction, law, or manufacturing.

In some cases, you may build a stand-alone application that makes programmatic use of SharePoint capabilities. The user may never see the SharePoint user interface, or even be aware that the application is developed by using SharePoint as part of its infrastructure.

Customization can be light or deep. Your whole customization may consist of a single Web Part that enables some scenario in your group or team. Or you may be building a complex commercial application that has data flow and processes. You may have integration (deep or shallow) with external customer relationship management (CRM) or enterprise resource planning (ERP) systems.

SharePoint integration with the Microsoft Office 2010 system is important. You can take advantage of this integration to build solutions in such a way that—regardless of whether the users are in an Office client application or in a web browser—the features of your application are discoverable and convenient for them. I discuss where SharePoint development intersects with Office 2010 client development through this article.

It is useful to examine SharePoint development from three points of view:

  • Building scalable applications for the .NET Framework

  • Building database applications

  • Building traditional rich client applications

Comparing SharePoint Applications to Scalable .NET Web Applications

You can examine SharePoint development from the perspective of a developer who builds highly scalable, loosely coupled web applications that run on large server farms. These applications must handle hundreds or thousands of page views per minute.

At its core, SharePoint is based on ASP.NET, and runs on IIS, and can have multiple front-end web servers that handle load balancing. SQL Server provides the integrity, scalability, robustness, and security of data and documents that you store in SharePoint sites. The following are some important ways that scalability affects SharePoint development.

  • API Design. Scalability drives certain characteristics of the programming interfaces of SharePoint. It is easier to understand these programming interfaces when you understand that their design facilitates scalability. For example, the Managed Client Object Model, although very similar in abstractions to the SharePoint Foundation server-side object model, is more complex because it enables you to explicitly control when you retrieve data or content from the server.

  • Solution Design. Scalability influences how you design a solution built on SharePoint. You have to avoid designs that cause unnecessary computation or querying activity on the server. You have to write applications that do not consume more resources than they should. For instance, this means appropriately using Collaborative Application Markup Language Core Schemas and LINQ to SharePoint to query for list items.

  • Best Practices. Scalability is behind certain programming approaches and issues that are best practices of SharePoint development. For example, certain objects in the SharePoint object model have unmanaged data associated with them. Therefore, you must understand and follow the rules around object disposal. Similarly, there are several best practices to consider when you are working with large lists in SharePoint. If you do not follow those rules, you can adversely affect the server farm. For more information, see Best Practices with SharePoint Foundation and Best Practices with SharePoint Server. See also Best Practices: Using Disposable Windows SharePoint Services Objects and Disposing Objects. You can improve your code reviews with an automated tool. See Automate SharePoint Dispose() code reviews with SPDisposeCheck for more information.

These are issues that are identical or similar to the issues that developers face who are building highly scalable web applications. I heard a story about a SharePoint developer who wrote code to iterate over all documents in his site collections at set intervals and assemble information to be presented in a tree control. This worked fine in his test environment. However, the code design presented a performance problem with real-world quantities of documents and list items.

Scalability affects solution design in two distinct ways:

  • You have to build applications that are distributable, and that work correctly when they are deployed on multiple front-end web servers. For example, you could build a small Create/Retrieve/Update/Delete web service for Microsoft Business Connectivity Services (BCS) (see Business Connectivity Services) that stores data in a local XML file. However, that does not work when it is deployed on a load-balancing server farm.

  • You must build applications that perform well. For example, unless you are certain that a list will contain few list items, do not iterate over them by using the object model; use LINQ to SharePoint instead, and give SharePoint an opportunity to optimize.

For more information about how development for SharePoint is similar to and differs from ASP.NET development, see Glide Path for ASP.NET Developers. See also SharePoint 2010 Development for ASP.NET Developers.

Best Practices with SharePoint Foundation contains guidance that helps avoid pitfalls that can adversely affect performance. This includes guidance around object disposal, event receivers, large folders and lists, and code performance optimization.

Comparing SharePoint Applications to Database Applications

Database application development is one of the good vantage points for viewing SharePoint development. Custom lists in a SharePoint site have much in common with database tables. You can define columns in lists, with rich metadata about those columns. Further, SharePoint lists can effectively have foreign keys defined so that you can model more interesting scenarios that contain related data. SharePoint provides cascade and restrict behavior for delete operations. You can create these lists programmatically or declaratively, or you can write programs that work with lists that users defined. These lists can be visible or hidden. You can use the security features of SharePoint to restrict access.

One aspect of SharePoint that is a direct parallel to SQL databases is that you use a non-procedural declarative query language to retrieve data. However, instead of using SQL, you use LINQ to SharePoint or Collaborative Application Markup Language (CAML), which you write using XML.

SharePoint integration with database technology is deep. You can use Business Connectivity Services to consume databases, web services, and almost any data source. These data sources are represented as external content types.

An interesting characteristic of the data capabilities of SharePoint is that it does not have transactional guarantees. For example, you cannot insert a list item in one table and update a list item in another table and be guaranteed that neither or both will occur. SharePoint is not intended for use as a platform to implement transactional systems. Instead, such transactional systems should be implemented in external databases that can provide the appropriate guarantees. You can then present that data in SharePoint by using Business Connectivity Services. As you design SharePoint applications, you must take this characteristic into account.

An important difference between the data capabilities of SharePoint and traditional database development is that SharePoint lists (comparable to database tables) are not necessarily rectangular. In SharePoint, a content type defines the fields that make up a list item (comparable to a row) in a list. This can be thought of as the schema for a list item. A SharePoint list can contain list items of more than one content type. The following figure represents a SharePoint list that contains items of two content types: LABOR and MATERIAL.

Figure 2. Non-rectangular SharePoint list

Nonrectangular SharePoint List

This affects how you design and develop applications that consume list items and the fields in list items. If you must have rectangular data, you can define lists so that they can contain only one content type. If you enable non-rectangular data, when you iterate through list items, you must check for the content type and alter the behavior of your code accordingly.

Another interesting characteristic is that because of internal implementation details, querying SharePoint using LINQ to SharePoint or CAML is much slower than querying tables using SQL. You do not want to create a design where some list unintentionally grows to 50,000 or 100,000 items, and the user is free to attempt to bring the list up in a window in a browser. SharePoint 2010 contains throttling features that prevent such designs from taking down the server farm. However, the user interface will become non-responsive to the user. There are ways to handle lists with very many items. You have to handle this issue intentionally. For more information, see Handling Large Folders and Lists.

SharePoint is built upon SQL Server. Document libraries and SharePoint lists are stored in a database. You see this when you see architectural diagrams of SharePoint installations. The actual SharePoint database is not especially relevant to you as a SharePoint developer. You never directly access this database. You always use the programming interfaces to alter sites, lists, and document libraries. However, you can use that same SQL Server installation to host databases that you consume directly or through Business Connectivity Services. Therefore, it can be a part of the infrastructure that you use to build SharePoint applications.

Microsoft Access services allows you, with limitations, to publish Access databases to SharePoint sites. This opens up interesting possibilities because it enables SharePoint users to share data in a convenient and familiar form.

Client-Side Data Access Technologies

You can programmatically access SharePoint data from client computers. When accessing data from client computers, SharePoint uses networking technologies to send the request to SharePoint Foundation or SharePoint Server and to retrieve the results of the request from SharePoint.

Client-Side Data Access Technologies

Data Access Technology

Description

Business Connectivity Services Client-Side API

Enables online and offline access to data in external content types. You can access this object model from managed add-ins for Microsoft Word, Microsoft Excel, or Microsoft PowerPoint. This is sometimes the most powerful and convenient way to access data that is external to SharePoint in an application.

Web Services

Enables high-performance designs where you host custom WCF services on SharePoint Server. The WCF service can access data and documents through the server object model and allow the client to access the data and documents through the service that you implement.

Excel Services ECMAScript Interface

Enables you to customize, automate, and drive the Excel Web Access Web Part control on a page

Excel Services REST API

Enables you to access workbook parts or elements directly through a URL.

Excel Web Services

Enables applications to calculate, set, and extract values from workbooks, and to refresh external data connections.

ECMAScript Client Object Model

Enables ECMAScript programs to access SharePoint data. This programming interface is a variant of the Managed Client Object Model.

Managed Client Object Model

Enables .NET Framework applications (including stand-alone applications, Silverlight applications, and ASP .NET applications) to access SharePoint data. This framework is similar to the server object model, with additional semantics that allow you to control network traffic to and from the server. It is important to know and use CAML when using the client object model.

OData REST APIs

Provides a RESTful programming interface to lists and list items.

SharePoint Web Services

Provides web service methods that you can use to work remotely with SharePoint Foundation deployments.

Web-Based Distributed Authoring and Versioning (WebDAV)

Consists of a set of methods, headers, and content-types that augment HTTP/1.1 to allow for the creation and management of resource collections. SharePoint support for WebDAV enables you to upload and download documents. WebDAV (RFC 4918) is a working document of the Internet Engineering Task Force (IETF).

Server-Side Data Access Technologies

When writing code that runs directly on the server, regardless of whether you are building a sandboxed solution or server farm solution, you can specify your data queries either in CAML or by using LINQ to SharePoint. CAML returns weakly typed SPListItem objects, whereas LINQ to SharePoint returns strongly-typed objects that are defined in generated code.

Server-Side Data Access Technologies

Data Access Technology

Description

Collaborative Application Markup Language (CAML)

XML-based non-procedural query language

LINQ to SharePoint

Language-Integrated Query

In addition to using CAML or LINQ to SharePoint to query for a set of items, you can use the server object model to iterate through items in a list or library directly. However, you should not do this unless you must process all items in the list and you know how many items are in the list. I often design applications to be data-driven. I design a list to contain some specific data that drives behavior. My design might be so that I always know that there are a certain number of records, and that I have to process all records to drive my application behavior. In this case, I iterate directly through the items without querying by using CAML or LINQ to SharePoint. Note that this same approach does not work when you use the managed client object model. By using the managed client object model, you must supply CAML queries when you are retrieving collections of list items.

Comparing SharePoint Applications to Traditional Rich Client Applications

SharePoint has much in common with an operating system:

  • SharePoint has storage. Document libraries that contain hierarchies of folders are very similar to files in directories in a file system.

  • SharePoint has a programmable user interface.

  • As with an operating system, you can write code that runs at varying levels of privilege. By using SharePoint development, you can write code that must run with farm administrative credentials. You can write code that runs in a sandboxed solution, and you can write JavaScript that runs in the client browser.

  • As with an operating system, you can write services, which provide necessary infrastructure for building SharePoint applications with complex dynamics.

Scale, again, is the biggest difference. Scale drives the programmability characteristics of SharePoint. You can write software that runs with great performance across hundreds of servers that provides a whole enterprise with a consistent experience that increases collaboration and productivity. Dealing with the issues of scalability is the price of admittance when you move from traditional application development to SharePoint development.

One question that is at the top of the minds of all new SharePoint developers is as follows: Just how much do I have to learn? It is interesting to list the languages and platforms that you must know when doing SharePoint development. The first item in the following list is the most important. Beyond that, it depends on what you are doing. It is not necessary to master every technology in the following list. You have to learn only what applies to your current task.

  • Microsoft Visual C#, Microsoft Visual Basic, and Microsoft .NET Framework. You use the .NET Framework in one form or another when you build full-trust applications for the server farm, sandboxed site-collection level solutions, or within Microsoft Silverlight applications. You are a more effective developer if you understand the various LINQ technologies: LINQ to XML, LINQ to Entities, LINQ to SharePoint, and LINQ to SQL.

  • ASP.NET. SharePoint is based on ASP.NET and is hosted in IIS. It uses master pages and application pages, which enable you to standardize page appearance and behavior. It also uses Web Parts pages and Web Parts, which allow users to configure their own pages. You can connect Web Parts to each other so that you can create parent/child data editing capabilities or web mashup applications.

  • HTML/CSS. There are several areas in SharePoint development where it is helpful to write and understand HTML and CSS.

  • ECMAScript and the HTML Document Object Model (DOM).JavaScript is a powerful tool in your tool-chest. You can use the JavaScript client object model or the OData REST API from JavaScript to obtain data and documents from your sites. JQuery and AJAX enable you to add rich interactivity.

  • Silverlight/XAML.Silverlight is a development platform for creating engaging, interactive applications for many screens across the web, desktop, and mobile devices. You can use the managed client object model within the Silverlight code to read and write site data and documents. Microsoft Expression Blend is an effective tool for designing Silverlight Web Parts for SharePoint.

  • XML. You have to know how to code valid XML documents. Regarding developing with XML, my personal recommendation is to learn and master LINQ to XML. It is a more effective way to work with XML. Knowing XSLT can be useful. You can manually write custom XSLT list views, although the main purpose of XSLT list views is to enable you to apply interesting formatting to lists by using Microsoft SharePoint Designer. In other words, XSLT is more often used as the output of tooling instead of as a frequently used developer tool.

To make it easier to talk about SharePoint application development, I define four terms (taken from the SharePoint marketing sites) so that I can discuss them more easily through the rest of this article series.

SharePoint application. Consists of customizations that, together, fill market or customer needs. An application consists of some combination of custom Web Parts, workflows, event handlers, content types, pre-defined lists, or any of the other technologies that you use to enhance SharePoint functionality.

SharePoint capabilities. Key areas of SharePoint functionality. Developers build on these areas of functionality to put together solutions where the whole is more than the sum of the parts. These capabilities include:

SharePoint developer scenarios. Activities that a developer wants to enable for SharePoint users. For example, you may want to enable the user generate several hundred Word documents based on data that is stored in a SharePoint list, generate PDF or XPS files from those documents, and to print them or send them by email messages. To complete this scenario, you use several developer building blocks. One possible implementation uses the Open XML SDK 2.0 for Microsoft Office (see Welcome to the Open XML SDK 2.0 for Microsoft Office) in a Web Part that is implemented as a sandboxed solution. Then use Word services to convert from DOCX to PDF.

SharePoint developer building blocks. Technologies in SharePoint that you use to do your development. Examples are Web Parts, workflows, event handlers, data, and services. When you build an application, you develop by using the building blocks so that the parts all work together.

What are the varieties of applications that SharePoint developers build? This resembles asking the question about the varieties of applications that you can build on an operating system. Developers are creating uses for SharePoint that cover many scenarios. Here are several varieties of applications.

  • Vertical markets. Vendors customize SharePoint for many markets such as real-estate management, resort and entertainment management, or equipment rental.

  • Customized team applications. Sometimes custom applications can help departments of large organizations meet their goals in a more organized and visible manner. A testing organization could build a system to maintain a library of testing instruments and resources.

  • Content publishing systems. These systems can be as simple as a site or site collection that is maintained by a single author. They can be as complex as a technical document generation system that is used by hundreds of writers, developers, and managers such as the Office.com website.

  • Accounting systems. Although Microsoft does not recommend that transactional systems be implemented in SharePoint, bringing data into SharePoint from external accounting systems significantly augments the user experience.

  • Management information systems. Some automated management systems do not have the same transactional requirements as accounting systems. You can maintain lists of customers, products, assets, and resources by using a default SharePoint installation. When extended programmatically, you can provide a very sophisticated experience.

  • Sales and marketing. Many aspects of market and sales systems resemble management information systems. Integration of SharePoint with CRM systems such as Microsoft Dynamics results in powerful hybrid applications. In addition, mobile access to SharePoint sites provides road warriors with tools to interact with data that they need to do their job.

  • Project management. The integration of Microsoft Project Server 2010 with SharePoint Server Enterprise provides project and portfolio management to help organizations prioritize investments, align resources and execute projects. You can develop for Project Server by using its API.

Although many examples of SharePoint applications fit into one of these categories, this list is definitely incomplete. I am certain that someone somewhere has written a game that runs on SharePoint that takes advantage of the collaboration features. I would also guess that someone has built a system that controls computer-automated manufacturing systems using SharePoint lists, event handlers, and more. Vendors serving other vertical markets such as computer-aided design or architecture will invent their own variety of improvements of SharePoint.

The kinds of applications that you can build for SharePoint are almost as varied as the kinds of applications that you can build for the Windows operating system.

You can find much information about SharePoint capabilities from the Microsoft SharePoint 2010 Product Overview. The capabilities include the following:

  • Sites. You can develop and manage all the business websites by using SharePoint 2010. SharePoint 2010 provides a full set of tools that users can use to create any kind of site. SharePoint provides infrastructure that simplifies site management.

  • Communities.SharePoint 2010 provides collaboration tools that anyone can use to share ideas, find people and expertise, and locate business information. By default, SharePoint has many collaboration capabilities, but extending SharePoint in interesting ways can make collaboration proactive instead of reactive. Integrating presence into SharePoint applications by using Microsoft Lync Server 2010 makes it easier for colleagues to contact one another.

  • Content. Publishing organizations come in a variety of forms, from international press organizations to small departments that are responsible for publishing documentation for a product. Organizations can use SharePoint 2010 to manage content and use Word 2010 to edit content. I also divide the content capabilities into the following three subcategories:

    • Publishing Systems. You can use SharePoint Server 2010 to manage a comprehensive publishing system. You may deliver the final product in PDF, XPS, or some other form. Open XML is an especially powerful technology that you can use to build publishing systems. You may have to write a sophisticated transformation to other formats. For example, see Open XML to other forms such as XHtml.

    • Web content management. Whether using SharePoint Server 2010 for Internet Sites to build external-facing websites, or using SharePoint for building wikis and pages inside the firewall, SharePoint lets users build out rich content with a minimum of effort.

    • Enterprise content management. Many organizations have lots of organizational experience, often in the form of documents that are managed by using revisions, approval processes, and metadata.

  • Search. Customizing the search experience can be critical to making information come alive for your users. With SharePoint Server 2010, you can customize to allow the SharePoint search engine to search through custom documents. See SharePoint Enterprise Search in SharePoint Server 2010 for more information. In addition, you can customize search to allow it to search through external content types (ECTs).

  • Insights. A recent survey asked CEOs of very large companies which issues were "top-of-mind" for them. Business intelligence (BI) was mentioned most often as the top priority. Excel Services and the ability to build dashboards are important for building powerful, flexible BI systems. PerformancePoint Services for SharePoint provide another option to deliver customizable BI dashboards.

  • Composites. Create innovative solutions that combine customer user interfaces, data access, business logic, and services. You can document and standardize business processes using workflows.

Many developers determine and document developer scenarios while they design software but before they write functional specifications. Sometimes this process consists of the construction of storyboards that show how the user moves from point A to point B to perform a specific task. Sometimes you may do formal use-case analysis. You may even mock up a scenario by using PowerPoint or a modeling tool. In most development organizations, you write formal specifications that outline and detail the scenarios.

There are even more varieties of scenarios than there are varieties of applications. However, there are several general kinds of scenarios.

  • Scenarios that provide better user interfaces to existing SharePoint data. For instance, you may want to display calculated values that are based on data in several SharePoint lists.

  • Scenarios that pull in external data from external systems. You may be connecting to a modern ERP or CRM system, or you may be connecting to a proprietary database that is implemented on UNIX by using an emulator of ancient hardware architecture.

  • Scenarios around document generation. You may generate hundreds or thousands of Open XML document based on data that is stored in SharePoint lists or external systems, and then convert to PDF or other format .

  • Scenarios around process and procedure. Scenarios can be as simple as a document approval process, or can be as involved as shepherding a RFP through generating a proposal to signing of a contract.

You implement each developer scenario by using one or more SharePoint developer building blocks. Of course, as systems designers, you must determine your own scenarios. Then, your task is to be familiar with the available building blocks that you can use to implement your scenario. In many cases, you have more than one option; each option has different characteristics, and one option might suit more than another for a particular scenario. To a large extent, you probably already know the direction that you need to take. A high-end marketing and design site or a luxury goods website may from the outset plan to use Silverlight wherever necessary, whereas a part-time developer for a team site might be interested in whatever accomplishes the task in the fastest and easiest way.

In this article, I explored how SharePoint development compares and contrasts to other kinds of application development. This includes scalable, loosely-coupled ASP.NET application development, database application development, and traditional rich client development. I listed the varieties of languages and platforms that you need to know, and I discussed some SharePoint developer terms. I explored the capabilities of SharePoint, and discussed how developers extend those capabilities to meet the needs of their users. In the next article, SharePoint Developer Building Blocks: Technologies for Creating SharePoint Applications (Part 2 of 2), I enumerate and discuss all of the building block technologies that you use to build a SharePoint application.

Show:
© 2014 Microsoft