SharePoint 2010 Architectures Overview

Summary:  Learn about the architectures of Microsoft SharePoint Foundation 2010 and Microsoft SharePoint Server 2010, including the platform stack, the Microsoft ASP.NET-IIS integrated request pipeline, the server and client object models, and the execution process system for sandboxed solutions and farm solutions.

Applies to: Business Connectivity Services | Office 2010 | Open XML | SharePoint Designer 2010 | SharePoint Foundation 2010 | SharePoint Online | SharePoint Server 2010 | Visual Studio

Provided by:  Ricky Kirkham, Microsoft Corporation

Contents

• Overview of SharePoint Foundation and SharePoint Server Architectures

• What Is SharePoint?

• SharePoint Development Platform Stack

• SharePoint as an ASP.NET-IIS Application

• SharePoint Configuration Stack

• SharePoint Deployment Models

• Process and Execution Trust Model

• Pages, Parsing, and Safe Mode

• Data Model, Data Management, and Query System

• Services and the Service Application Framework

• Site Definitions and Web Templates

• SharePoint Security

• Server Object Model in SharePoint

• Client Object Models in SharePoint

• Workflows in SharePoint

• Conclusion

• Additional Resources

Overview of SharePoint Foundation and SharePoint Server Architectures

This article provides an overview of the architectures of Microsoft SharePoint Foundation and SharePoint Server, including the platform stack, the Microsoft ASP.NET-IIS integrated request pipeline, the server and client object models, the execution process system for sandboxed solutions and farm solutions, and more.

What Is SharePoint?

The use of the plural term "architectures" in the title is not a mistake. SharePoint has many architectures, partly because "architecture" has many meanings in the context of software development, but also because SharePoint itself is many things—things that in the past would have been distinct applications or platforms.

The following is a selection of some of the most important things that SharePoint is:

• A portal server with delegated administration. SharePoint enables information workers (IW) who have no knowledge of website design or website administration to create, almost instantly, attractive and functioning websites. This relieves information technology (IT) departments from the burden of creating and administering the sites, and it empowers the IWs to create their own sites for teams, blogs, wikis, and other purposes.

• A groupware application kit. SharePoint provides a platform on which IWs can create collaboration solutions that include document libraries and workspaces, workflows, wikis, blogs, and team-oriented lists, such as Events, Announcements, and Tasks. Microsoft SharePoint Workspace (formerly Microsoft Office Groove 2007) provides an offline experience for these collaboration solutions.

• A workflow host. Business processes can be systematized and modeled with workflows that are triggered by associated events; for example, the addition of a document to a document library.

• A content management application. SharePoint Server Enterprise Content Management (ECM) features include document management, records management, and web content management.

• A Business Intelligence (BI) application kit. The Microsoft Business Connectivity Services (BCS) features of SharePoint enable data from non-SharePoint sources, such as a SAP installation or Oracle database, to be accessed (read/write) just as if it were an ordinary SharePoint list.

• The operating system of an intranet. SharePoint can provide for an intranet many of the functions that an operating system provides for a computer, including storing and copying files, hosting services, starting applications, and securing data. (This is not to imply that SharePoint can only be used on an intranet. SharePoint can also host extranet and Internet-facing solutions.)

• A host for services. SharePoint deployments make data available through a client object model, the REST-based Windows Communication Foundation (WCF) Data Services (formerly ADO.NET Data Services), and many out-of-the-box ASMX web services. In addition, the SharePoint Service Application Framework provides a platform that enables developers to build scalable middle-tier services that can provide data or processing resources to other SharePoint features.

• A data store. SharePoint stores data as multicolumn lists in a Microsoft SQL Server database. You can query the data by using LINQ and also using Collaborative Application Markup Language (CAML). The data can be mirrored, backed up, restored, and, depending on the edition of SQL Server being used, you may be able to take snapshots of the data store.

• A data and processing layer for multiple user interfaces (UIs). Besides its native UI of webpages (including special versions for mobile devices), which can contain ECMAScript (JavaScript, JScript), SharePoint also supports access from Microsoft Silverlight applications and the Microsoft SharePoint Workspace client application. With the SharePoint client object model, you can access SharePoint using Windows Presentation Foundation (WPF), Windows Forms, or any other managed code application.

This article discusses the architectures of SharePoint 2010 (with many links to more detailed information) in several senses, including the platform stack on which SharePoint runs, the configuration layers, the client and server object models, the services framework, the HTTP request pipeline, and the worker process system.

SharePoint Development Platform Stack

Figure 1 shows how SharePoint Foundation is built on Microsoft .NET Framework 3.5, ASP.NET, and Internet Information Services (IIS). SharePoint is also built on SQL Server, but you can install SQL Server on a dedicated server that does not have SharePoint installed, which is typical in a production farm. All of these platforms must be running on a 64-bit installation of Windows Server 2008 on the server computers. SharePoint Server, in turn, is built on SharePoint Foundation. There are two editions of SharePoint Server: Standard and Enterprise.

Figure 1. The platform stack for SharePoint development

Perhaps the most noteworthy aspect of the figure is that IIS and ASP.NET are shown as a single platform. This is because SharePoint requires that IIS operate in integrated mode with ASP.NET. Hence, from a SharePoint point of view, they are effectively a single web-hosting application. For more information about this, see the next section, SharePoint as an ASP.NET-IIS Application.

In Figure 1, the smaller boxes that have no fill represent some selected subparts of the platform that contains them, or on which they depend. The two thin, downward-pointing arrows indicate some specific dependencies that are shown only as examples. Many other specific dependencies are not shown in the figure. The thick, left-pointing arrows indicate that the entity on the right side accesses the entity to which the arrow points. For example, BCS accesses external databases.

SharePoint as an ASP.NET-IIS Application

As the High Level Object Model section later in this article shows in more detail, the highest level of organization in a SharePoint deployment, other than the farm itself, is the web application. A web application in SharePoint terminology is closely related to what is called a website in IIS terminology. An IIS website monitors for incoming requests through a particular port, and it checks for a particular host header or IP address, or both. Every SharePoint Foundation web application is hosted in an IIS website that has the same name as the web application. It can be helpful, especially when you are trying to see the relation between SharePoint and IIS from a high and broad perspective, to think of the SharePoint web application and its corresponding IIS website as a single entity.

Nevertheless, the SharePoint web application and the IIS website are not quite the same thing (and they are represented by different classes in the SharePoint object model: the SPWebApplication class and the SPIisWebSite class). For one thing, although there is usually a one-to-one relation between SharePoint web applications and IIS websites, this is not always the case. It is possible to extend a SharePoint web application to multiple IIS websites, although that is not a common design.

Figure 2 shows the IIS websites and application pools on a SharePoint Foundation front-end web server.

Figure 2. IIS Manager on a SharePoint Foundation front-end web server

Just as in any application that is built on the ASP.NET-IIS integrated pipeline, when a front-end web server receives a request from a client for a page or other resource in a SharePoint site, the request is passed through a pipeline of units that process the request. This processing includes authenticating the user, verifying the user’s authorization, building the response, sending the response, and finally, logging the request. For more information about the integrated pipeline, see ASP.NET Integration with IIS 7.

The response to any request is produced by an HTTP handler object. Requests are assigned to one or another HTTP handler object (or to a handler factory class that creates HTTP handler objects) depending on the resource requested and the HTTP verb in the request. The assignment is determined by a stack of configuration files named applicationhost.config, machine.config, and web.config.

The request pipeline also contains HTTP modules. Modules are assemblies that typically contain one or more event handlers or define new events that other modules can handle. An HTTP module can register for one or more events in the life cycle of the request. They are often used to preprocess requests or postprocess the response. The result of a module’s preprocessing is stored in the HttpContext object. For example, the value of the User property is produced by an authentication module.

The processing of a request by HTTP modules and an HTTP handler is governed by an HttpApplication object or an object derived from that class. SharePoint installs a global.asax file in the root of each web application (IIS website) that identifies SPHttpApplication as the class from which an HTTP application object is created.

SharePoint also modifies the pipeline with a custom HTTP module (SPRequestModule), and with changes to the default applicationhost.config file. For detailed information about SharePoint and the integrated pipeline, see Microsoft SharePoint Foundation as an ASP.NET Application.

Figure 3 shows the relations among the classes that handle HTTP requests for a SharePoint front-end web server.

Figure 3. SharePoint customizations of the HTTP request pipeline

In addition to its use of the integrated pipeline, SharePoint uses some other features of IIS and technologies from ASP.NET. Among them are the following:

• Active pages (aspx files): SharePoint pages are ASP.NET pages. SharePoint controls are referenced in the page markup with a registered "SharePoint" prefix. (For more information about the ASP.NET page parsing and compiling system, see Understanding ASP.NET Dynamic Compilation. Note that the standard page parser of ASP.NET compiles a page the first time it is requested unless the page is explicitly set not to be compiled. However, SharePoint pages that have been customized by a SharePoint website owner are stored in the SharePoint content database and are not compiled when they are accessed. SharePoint uses a custom virtual path provider, an internal class named SPVirtualPathProvider that is derived from VirtualPathProvider, to locate a requested page from either the file system or the content database and load it.

• The master page/content page distinction: A SharePoint page is a merger of a master page and a content page. For more information, see ASP.NET Master Pages and Master Pages. (For some SharePoint Server pages, a third foundational page is involved in the merger, a page layout.)

• Web Parts: Web Parts actually originated in an early version of SharePoint. The idea was adopted by ASP.NET. In most SharePoint Web Part development situations, we recommend that you use the ASP.NET version, but in some situations the original SharePoint incarnation is a better choice. The SharePoint UI for mobile devices also uses the ASP.NET concept of registered Web Part adapters. For more information about Web Parts in SharePoint, see Building Block: Web Parts and Building Block: Mobile Pages, Controls, and Adapters.

• User controls (.ascx files): Many SharePoint web controls and Web Parts are built with ASP.NET user controls.

• Web services (.asmx files): Some, but not all, SharePoint web services are implemented as ASP.NET ASMX services. For more information about ASP.NET web services in SharePoint, see ASP.NET Web Services.

• Virtual directories: SharePoint maps several virtual directories in each IIS website to physical folders in the SharePoint root, which is the folder %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\. When you as a developer need to provision SharePoint web applications with images, resource files, and other important files, they are deployed to the appropriate physical folder on the front-end web servers. This process is handled automatically by the solution deployment mechanism of SharePoint. For example, user controls are deployed to %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\ControlTemplates (or a subfolder), which is mapped to the virtual directory _controltemplates.

ASP.NET development experience can be an advantage for a SharePoint developer, but the SharePoint development experience is different from the ASP.NET experience in many ways. Some examples of the differences are as follows:

• SharePoint has its own installation and deployment model. For more information, see SharePoint Deployment Models later in this article.

• SharePoint makes a distinction, unknown in ASP.NET, between application pages and site pages. For more information, see Pages, Parsing, and Safe Mode later in this article.

• SharePoint has its own systems for modifying web.config files. For more information, see Working with Web.config Files.

• The ASP.NET worker process model is modified in SharePoint. See Process and Execution Trust Model later in this article.

For more information about how SharePoint development differs from ASP.NET development, see ASP.NET vs. SharePoint: How Development Differs.

SharePoint Configuration Stack

SharePoint configuration settings exist at several levels. As in all ASP.NET applications that use the integrated pipeline, some fundamental settings are in the machine.config file, the global web.config file, and the applicationhost.config file, which is the IIS configuration store. SharePoint makes no changes in the default versions of the first two files. It does make some changes in the IIS configuration store. For more information about the changes, see Microsoft SharePoint Foundation as an ASP.NET Application.

Each IIS website and, therefore, each SharePoint web application, can have a web.config file in its root folder. This file is substantially customized by SharePoint whenever a SharePoint web application is created. For more information about the changes, see Microsoft SharePoint Foundation as an ASP.NET Application. SharePoint also takes advantage of the fact that web.config files can be applied to specific virtual or physical folders within a web application. For example, SharePoint puts a web.config file in the virtual directory _layouts\mobile (which is mapped to the physical directory %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\LAYOUTS\MOBILE\). Among other things, the web.config file registers a series of filters that control how a page is rendered, based on the capabilities of the mobile device that has requested the page.

All of the configuration files discussed so far must be identical across all front-end web servers. The SharePoint deployment model and its system for programmatically modifying web.config files helps ensure conformity to this rule. For more information, see Building Block: Solutions and Working with Web.config Files.

Farm-wide configuration settings are stored in the SharePoint configuration database on the computer that is running SQL Server in the farm. Configuration for specific types of websites is contained in various kinds of XML files, such as the Onet.xml file, and configuration for specific instances of websites is contained in the content database. In addition, several classes in the SharePoint object model have property bags that can be used to store custom configuration information for specific objects, such as objects that represent websites, Features, alerts, and files.

Finally, context information for specific HTTP requests is contained in the ASP.NET HttpContext object of the request. The SharePoint object model adds an SPContext class that does not inherit HttpContext; however, it gets many of its property values from the current HttpContext object and adds other properties to represent SharePoint-specific context information, such as the current website, list, and list item.

SharePoint Deployment Models

SharePoint solutions are generally not installed by using MSI or ClickOnce technology. (Some exceptions are noted later.) SharePoint has its own installation system. Instead of MSI files, the solutions are packaged in SharePoint solution package files. These are CAB files that have a special extension: .wsp. SharePoint solution package files can contain several kinds of elements, including assemblies, user controls, custom ASP.NET pages, XML configuration files, resource files, images, list definitions, Web Parts, Features, and others. JavaScript files and Silverlight .xap files can also be deployed to the servers by using SharePoint Solution Packages.

There are two kinds of solutions in SharePoint: farm solutions and sandboxed solutions. The differences between them are discussed in this section and in Worker Processes, Farm Solutions, and Sandboxed Solutions later in this article.

Installation of a SharePoint solution package is a multi-step process:

1. Adding: A solution package file is added to one of two stores.

   • Farm solutions are deployed by farm administrators to the farm's solution store, which is in the farm's configuration database. This is done by using either the SharePoint Management Shell or the object model.

   • Sandboxed solutions are deployed to a specific site collection's Solution Gallery by a site collection administrator. The gallery is a special SharePoint list, so it is in the content database. Solution deployment to the gallery is done through the Site Actions UI of the site collection or by using SharePoint Management Shell. For information about the difference between site collections and websites, see Middle Level Object Model later in this article.

   Thus, SharePoint Solution Package files are never installed to the file system of any server, although elements of a farm solution may be installed to the file system when the solution is deployed.

2. Deploying: The solution package is unpacked and its elements are deployed to their appropriate places.

   • For a farm solution, this step also requires a farm administrator and can be done by using either Central Administration or SharePoint Management Shell, or the object model. Some examples of how elements are deployed: user control files (.ascx) are copied to %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\ControlTemplates and assemblies are deployed to the global assembly cache or to a web application's \bin folder. If there are SharePoint Features (defined later) in the farm solution, they are in this step copied to a subfolder of %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\FEATURES.

   • For sandboxed solutions, the deployment step is taken by a site collection administrator. There are limits on what kinds of elements can be in sandboxed solutions and where they can be deployed. For example, nothing can be deployed to the file system of the servers from a sandboxed solution. Features in sandboxed solutions are deployed to the content database.

   It is also in this second step that any Features in the solution are installed in the Feature Gallery of the farm, web application, site collection, or website, depending on the scope of the Feature. (For more information about the relations of farm, web applications, site collections, and websites to one another, see Content Hierarchy later in this article.)

Within a SharePoint solution package, there can be an additional level of encapsulation because a solution can have one or more SharePoint Features. A Feature can be installed at the scope of the farm, the web application, the site collection, or the website. After it is installed, a Feature must be activated by owners of any website within the scope; so Activating becomes a third step of installation for Features. Features can contain content types, controls, custom actions, custom fields, files, workflows, list instances, list templates, event receivers, and document converters; although some of these cannot be included in certain scopes. (Features that are deployed in sandboxed solutions can be scoped only to a site collection or website. For site collection-scoped features in sandboxed solutions, the second and third steps are combined. The Features are activated when the solution is deployed.)

Some SharePoint solutions target one of the client object models, either exclusively or in addition to targeting the server object model: the JavaScript, Silverlight, and Microsoft .NET Framework client object models. (For more information about the client object models, see Client Object Models in SharePoint later in this article.) There is nothing unusual about the installation of the client portion of such solutions.

• The script files that define the JavaScript object model are downloaded to a client computer when a page that references them is opened.

• Similarly, a Silverlight .xap executable is downloaded to the client computer when a page that is hosting it is accessed. The SharePoint assemblies that contain the Silverlight client object model are encased in the .xap file. Those assemblies are Microsoft.SharePoint.Client.Silverlight.dll and Microsoft.SharePoint.Client.Silverlight.Runtime.dll. (It is also possible to cache these assemblies on the front-end web server. For details, see RIA Technologies: Benefits, Tradeoffs, and Considerations.)

• Stand-alone .NET Framework applications (such as a WPF application) that target the SharePoint client object model are installed just as any other client applications: using MSI files or ClickOnce. The assemblies that contain this client object model, Microsoft.SharePoint.Client.dll and Microsoft.SharePoint.Client.Runtime.dll, must be distributed as part of the solution. The redistributable is located at SharePoint Foundation 2010 Client Object Model Redistributable.

For more information about the processing of client-side logic in SharePoint, see Client Application Models in SharePoint 2010.

For more information about the deployment system, see Building Block: Features, Building Block: Solutions, and Packaging and Deployment in SharePoint Foundation.

Process and Execution Trust Model

In the simplest scenarios, the process model of SharePoint is the same as any other ASP.NET application; but in SharePoint, the distinction between farm solutions and sandboxed solutions entails some more complex scenarios.

Worker Processes, Farm Solutions, and Sandboxed Solutions

When an HTTP request is received by a front-end web server, a special driver, HTTP.SYS, detects the request and routes it to the application pool that is handling requests for the targeted IIS website, and thus, the targeted SharePoint web application. Every application pool has an IIS worker process (w3wp.exe) in which the request pipeline for each request is executed. (For more information about the IIS 7.0 worker processes and application pools, see Introduction to IIS 7 Architecture.) On a SharePoint server, the IIS worker process runs in the application pool account, which gives the process read and write permissions to SharePoint resources. On a multiserver farm, the application pool account is a domain user that is not a machine administrator on any server in the farm, but is a member of the WSS_WPG, WSS_ADMIN_WPG, and IIS_USERS groups on each server. (This should be a different account from the farm administration account. The latter is also not a local machine administrator on any farm server. The farm account is a member of all the same groups as the application pool account and also the WSS_RESTRICTED_WPG_V4 and Performance Monitor User groups. An exception is made for the application pool of the Central Administration web application: its application pool account is the farm account. Also, the SharePoint 2010 Timer Service executes in the context of the farm account.) For more information about the accounts needed in a SharePoint farm, see Plan for administrative and service accounts (Office SharePoint Server).

Figure 4 shows how an IIS worker process that is running on a front-end web server processes an HTTP request.

Figure 4. Request processing model for full-trust farm solution

However, unlike a standard ASP.NET application, SharePoint makes a distinction between sandboxed solutions and farm solutions. Farm solutions run in the IIS worker process just like any ASP.NET application. Sandboxed solutions run in a specially restricted execution environment. This is necessary because sandboxed solutions are installed on (and scoped to) SharePoint site collections without the intervention of the IT professionals that are managing the SharePoint farm. To prevent rogue or poorly performing code from slowing or crashing the application pool, SharePoint imposes restrictions on what the code in a sandboxed solution can do. As a crucial part of the implementation of this system, sandboxed solutions must run in a special sandboxed worker process (SPUCWorkerProcess.exe).

When a request attempts to access a sandboxed solution, a SharePoint execution manager that runs in the IIS worker process finds a sandbox worker process (or starts one, if none is running) in which the code of the sandboxed solution will run. In principal, this sandboxed worker process can be started on any server in the farm that is running the SharePoint 2010 User Code Host service (SPUCHostService.exe). (In the UI of the Central Administration application, this is known as the Microsoft SharePoint Foundation Sandboxed Code Service.)

The server that is running the SharePoint 2010 User Code Host service can be, but does not have to be, the front-end web server on which the IIS worker process is running. Which server is used is configurable in the Central Administration application: Administrators can choose to have each sandboxed process run in "local mode," which means that each request for a sandboxed solution is processed on the same front-end web server on which the IIS worker process is running; or they can have the execution manager start each sandboxed process in "remote mode," also known as "affinity mode." In affinity mode, the execution manager looks for a server that is running the SharePoint 2010 User Code Host service and which already has created an application domain inside its SPUCWorkerProcess.exe process for the very same sandboxed solution. (This would be the case if that same sandboxed solution was requested before, possibly by another user on another site collection.) If there is a matching application domain, the request is sent to that same application domain for handling. If none of the servers that are running the SharePoint 2010 User Code Host service already has an application domain for the sandboxed solution, the execution manager assigns the request to the least busy of those servers. The server then creates the needed application domain and processes the request for the sandboxed solution. The application domain stays alive after the request is processed and is reused if there is another request for the same sandboxed solution.

By default, all sandboxed solutions that are handled by a given server run in the same sandbox worker process, but this is configurable through the object model. Each sandboxed solution gets its own application domain within the common process, and this, too, is configurable through the object model. The SharePoint 2010 User Code Host service runs in an account that has the same rights as a typical application pool account. It should be a member of the WSS_WPG, WSS_ADMIN_WPG, and IIS_USERS groups on the server on which the service instance is running.

Code Execution and Access Constraints on Sandboxed Solutions

All code that runs in this sandbox worker process is subject to execution and access constraints. There are two systems of constraints: One applies to all and only calls to any assembly, except Microsoft.SharePoint.dll, whether it is a SharePoint assembly or not. The other applies to all and only calls made to the parts of the SharePoint Foundation object model that are in the assembly Microsoft.SharePoint.dll. The calls that this second system applies to are not just calls in custom SharePoint solutions. Calls to Microsoft.SharePoint.dll from other SharePoint assemblies (that have themselves been called by custom code) such as Microsoft.SharePoint.Linq.dll are also subject to this constraint.

The first system of is imposed by two mechanisms:

1. A highly restrictive code access security (CAS) policy significantly limits what code in the sandboxed worker process can do. This policy is defined in the wss_usercode.config file in %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\CONFIG, and it is referenced in the web.config file in %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\UserCode. Among the restrictions imposed by the CAS policy are the following:

   • Code in the sandbox cannot call unmanaged code.

   • Code in the sandbox cannot call the Microsoft .NET Framework 3.5 reflection APIs.

   • Code in the sandbox can call only the .NET Framework 3.5 assemblies that have the AllowPartiallyTrustedCallersAttribute attribute. This blocks access to about two-thirds of all the .NET Framework 3.5 APIs, including System.Printing, for example.

   Note

   The CAS policy makes an exception for strong-named Microsoft Office assemblies. These are granted full trust.

2. Secondly, the sandboxed worker process has a low-privileged security token.

   • The token denies the process the right to read from or write to the file system.

   • The token denies the process the right to call to the network. Therefore, only resources available on the server that is running the sandboxed worker process may be accessed. An external database, for example, cannot be accessed.

   • The token denies the process the right to write to the registry.

   • The token denies the right to call to any assembly that is not in the general assembly cache, even if it has the AllowPartiallyTrustedCallersAttribute attribute and would otherwise be eligible to be called from the sandboxed worker process.

As noted, a second system of constraints imposes restrictions on what APIs in Microsoft.SharePoint.dll can be directly called by code in the sandboxed worker process, and a call to any forbidden API in the object model results in an exception (which is caught and reported to the user as an error). The implementation of these restrictions is accomplished by a pair of specially restricted versions of the Microsoft.SharePoint.dll assembly, sometimes called shim assemblies, that are located in %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\UserCode\assemblies. One of the two assemblies is loaded by the sandboxed worker process. The other assembly is loaded in a special proxy process (SPUCWorkerProcessProxy.exe) that runs in full trust and that is also managed by the SharePoint 2010 User Code Host. The standard Microsoft.SharePoint.dll assembly is also loaded in this proxy process.

The main job of the two shim assemblies is to filter out forbidden SharePoint classes and members. When the sandboxed solution calls an approved API, the first shim assembly passes it to the second in the proxy process, which in turn passes it to the standard Microsoft.SharePoint.dll. Any returned results are passed back to the original calling code. This cross-process interaction is possible through .NET Framework remoting. A sandboxed worker process and a full-trust proxy process are always started together and paired with one another. If either process crashes, the other is also stopped.

The shim assemblies also have a secondary job. Some SharePoint APIs are available to sandboxed solutions, but only with special restrictions on the parameters that are passed to them. It is the shim assemblies that enforce these input restrictions and ensure that an exception is thrown when there is a violation. The only case of this in SharePoint Foundation 2010 are the SPSite(String) and SPSite(Guid) constructors. These constructors can be called in sandboxed solutions, but only URLs or GUIDs that refer to the site collection in which the sandboxed solution is running can be passed to them.

The following are some of the restrictions on the SharePoint object model that can be accessed:

• The SPWebApplication class cannot be accessed. Among other things, this means that a sandboxed solution cannot access anything outside its hosting site collection.

• Almost all classes in the Microsoft.SharePoint.WebControls namespace cannot be accessed, which means that you are mainly restricted to ASP.NET controls in sandboxed solutions.

For a complete list of APIs in Microsoft.SharePoint.dll that are available to sandboxed solutions, see Microsoft.SharePoint.dll APIs That Are Available from Sandboxed Solutions.

Resource Usage Restrictions on Sandboxed Solutions

Sandboxed solutions are also subject to three kinds of resource usage restrictions that can be organized based on the kind of entity to which the restriction applies and the kind of entity on which the penalty for exceeding the restriction is imposed.

• Per Request with the Request Penalized: There is a hard limit to how long a sandboxed solution can take to finish. By default, this is 30 seconds. If a sandboxed solution exceeds the limit, the request (but not the sandboxed worker process) is terminated. (This limit is configurable, but only through custom code against the object model. The relevant parts of the object model cannot be accessed by sandboxed solutions, so no sandboxed solution can change the limit.)

• Per Request with the Process Penalized: A set of 15 resource limits apply to requests. If a request exceeds one of them, the process (and all the sandboxed solutions that are running in it) is terminated.

• Per Day/Per Site Collection with the Site Collection Penalized: Each site collection is subject to a configurable maximum of daily resource points. These points accumulate based on an algorithm that takes into account the use of resources in the 15 resource categories by the sandboxed solutions that are installed in the site collection. When a site collection exceeds its maximum allowed points, all sandboxed solutions in the site collection are terminated and no more can run for the rest of the day.

SharePoint provides a solution validator framework that can be used to develop custom solution validators, such as a validator that verifies whether a solution is signed with a specific certificate. The validators in a site collection run when a sandboxed solution is activated (that is, deployed, in the terminology used earlier in this article). The activation of any invalid solution is blocked. If a validator is updated or a new validator is added, each activated solution is rechecked by the validators the next time it is executed. Invalid solutions are deactivated. For an introduction to custom validators, see Developing, Deploying, and Monitoring Sandboxed Solutions in SharePoint 2010.

For more information about the sandbox restrictions, see Sandboxed Solutions in SharePoint 2010 and its child topics and the Microsoft patterns and practices guidelines for Sandboxed Solutions.

Figure 5 shows how an HTTP request is handled when it accesses a sandboxed solution.

Figure 5. Request processing model for sandboxed solutions

The SPUCHostService.exe, SPUCWorkerProcess.exe, and SPUCWorkerProcessProxy.exe files are located at %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\UserCode.

Not all SharePoint execution is in an IIS worker process, a sandboxed worker process, or the proxy process. The following are some examples:

• The SharePoint Timer Service (owstimer.exe) runs on all servers and is used to execute prescheduled timer jobs. It runs under the farm account.

• The SharePoint Tracing Service (wsstracing.exe) runs under the local service account.

• The SharePoint Administration Service (wssadmin.exe) runs under the local system account.

Within the category of farm solutions, a further distinction can be made when the requested resource is an .aspx page. If the requested page is what SharePoint calls an application page, the returned page is passed to the regular ASP.NET page parser; but if the requested page is what SharePoint calls a site page, the returned page is routed through a special safe mode parser. The distinction between the two kinds of parsing is best understood in light of the distinction between the two kinds of pages. For more information about both subjects, see Pages, Parsing, and Safe Mode later in this article. (Application pages cannot be included in sandboxed solutions. All pages installed as part of a sandboxed solution use safe mode parsing.)

Assembly Deployment, Execution, and Persistence

Farm solutions divide into two types depending on where their assemblies are deployed and the trust level of the assemblies' execution:

• GAC/Full Trust: The assemblies are deployed to the global assembly cache (GAC) of every front-end web server in the farm and run with full trust. They are callable from any SharePoint web application on the farm.

• Bin/CAS: The assemblies are deployed to the \bin folder (on every front-end web server) of a specific SharePoint web application and their trust level is determined by a CAS policy referenced in the web application's web.config file. They have to be separately deployed to every SharePoint web application that needs to call them.

  Tip

  We recommend that you use the Bin/CAS model only when neither full-trust farm solutions nor sandboxed solutions (nor the hybrid solutions described in Hybrid Solution Techniques) are possible for your solution.

Noncompiled files for both kinds of farm solutions—including, for example, images, user controls, and string resources—are deployed to subfolders of %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\.

Sandboxed solutions are deployed inside a SharePoint solution package (.wsp file) to the Solution Gallery of a specific site collection. Thus, they are deployed and persisted in the site collection's content database. As noted earlier, they do not run in full trust: Instead, they run within a highly restricted CAS policy and can only call a restricted subset of the SharePoint object model. A sandboxed solution can be accessed only in site collections to which it is deployed.

When a sandboxed solution is accessed for the first time, such as when a user navigates to a page that contains a Web Part from a sandboxed solution, any assemblies in the solution are unpacked from the solution package and copied to the file system of the server that is handling the sandbox request. The default location is C:\ProgramData\Microsoft\SharePoint\UCCache, but this is configurable on each server that is running the User Code Host Service. (Recall that the server that handles the sandbox request is not necessarily the front-end web server that is handling the initial HTTP request. The User Code Host Service can be run on back-end application servers in the farm instead.) Because the sandboxed worker process cannot copy anything to the file system, the copying is done by the User Code Host Service.

The assemblies do not stay in the file cache perpetually. When the user session that accessed the assemblies finishes, the assemblies stay in the cache for only a short time, and they may be reloaded from there if another user session accesses them. Eventually, if they are not accessed, they are removed according to a proprietary algorithm that takes into account how busy the server is and how much time has gone by since the assemblies were last accessed. If the sandboxed solution is used after that time, the assemblies are unpacked again and copied to the UCCache.

Hybrid Solution Techniques

The SharePoint solutions architecture includes a technique by which a sandboxed solution can call custom operations that run in full trust. The technique requires that a farm solution be developed that includes one or more classes that derive from SPProxyOperation. Each of these defines an operation that will run in full trust and can be called from sandboxed solutions by using the ExecuteRegisteredProxyOperation method. Specifically, these full-trust proxy operations execute in the same proxy process (SPUCWorkerProcessProxy.exe) that was described earlier in Worker Processes, Farm Solutions, and Sandboxed Solutions. The proxy operations can return data to the sandboxed solution.

Like all farm solutions, the assembly with the proxy operations can be deployed only if it is from a trusted source.

Figure 6 shows how a request that accesses a sandboxed solution is processed when the sandboxed solution makes a call to a full-trust proxy.

Figure 6. Request processing model when a sandboxed solutions calls a full-trust proxy

The preceding description might give the impression that, with the hybrid technique, a farm solution and a sandboxed solution are always developed together by the same development team. In fact, the farm solution may be developed specifically to provide certain operations to any and all sandboxed solutions that need those services, including sandboxed solutions that are developed by other teams. For example, because a sandboxed solution cannot write to the SharePoint Unified Logging Service (ULS) logs, a farm solution that opened proxy logging operations to sandboxed solutions would be very useful.

Another hybrid technique uses client-side code to access the resources that cannot be accessed from a sandboxed solution. For example, a sandboxed solution could include a custom site page with JavaScript that makes calls to the SharePoint JavaScript client object model. Also, a sandboxed solution could include a Web Part that hosts a Silverlight application. The latter application can make calls to the SharePoint Silverlight client object model. For more information about the client-side code in SharePoint, see Client Object Models in SharePoint later in this article.

For more information about hybrid solution techniques, see Hybrid Approaches.

Pages, Parsing, and Safe Mode

As noted earlier in SharePoint as an ASP.NET-IIS Application, SharePoint makes extensive use of the master page/content page distinction. But SharePoint Foundation also divides its ASPX pages along a different axis: It distinguishes between application pages and site pages. Both of these kinds of pages can be mergers of master and content pages and, indeed, every ASCX page that is built into SharePoint, whether it is an application page or a site page, is a combination of a master and content page. Hereafter, in this section, there is no further mention of the master/content page distinction.

Application pages differ from site pages in the following ways:

• Typical purpose: Application pages tend to be function-oriented, especially functionality that is needed by many kinds of websites within a given web application; for example, the standard form for creating a new list item is an application page. Site pages tend to be content-oriented; for example, the list-of-lists page of a standard team site. However, exceptions to both tendencies are possible. Indeed, we currently recommend that third-party developers develop custom Web Parts, which can be added to site pages, to handle their solution's functionality whenever possible, rather than develop custom application pages.

• Customizablility: Site owners (and other users who have appropriate permissions) can customize site pages, but not application pages. Users can also add an entirely new ASCX page to a website's Site Pages gallery, but only web application administrators can install a new application page.

• Class inheritance: Application pages are objects of the LayoutsPageBase class or the UnsecuredLayoutsPageBase class. Site pages are objects of the WikiEditPage class or the WebPartPage class. (All of these classes derive from the ASP.NET Page class.)

• Web Part support: Application pages cannot have Web Part zones or dynamic Web Parts. They can have static Web Parts, but there is little point to using a static Web Part instead of an ordinary control on an application page because end users cannot customize application pages anyway. Some kinds of site pages can have static Web Parts, and other kinds can have Web Part zones with dynamic Web Parts. See later in this section for more information.

• Storage location: Application pages are stored on the file system of the front-end web servers in the _layouts virtual directory of a web application (which maps to the %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\LAYOUTS physical directory) or one of its subdirectories. A site page that has not been customized is stored in other subdirectories of %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE, most typically in %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\SiteTemplates and %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\FEATURES. A site page that has been customized, such as with Microsoft SharePoint Designer, is stored in the content database of the website for which it was customized. New site pages that have been added to the Site Pages gallery are stored just as customized pages.

  Note

  Even an uncustomized site page has an entry in the content database; but whereas the entry of a customized page contains the ASPX markup that constitutes the page, the entry for an uncustomized page contains the path of the .aspx file on the local front-end web server. An uncustomized page can be shared by many websites. If it is, it is referenced in each website's content database, with all the entries pointing to the same physical file. For example, every Team Site has a content database entry pointing to the default Team Site home page. If the home page is customized on any of the sites, it is copied into the page's entry in the content database in its customized form, and the pointer to the uncustomized version is removed.

  Sometimes the uncustomized site page is referred to as a page template and the pages in the content database as page instances. However, remember that if the page is not customized in a particular website, the "instance" of the page is simply the database entry pointing back to the file on the file system.

• Availability: An application page can be accessed from every website in the web application and is therefore shared among all the websites. But a site page is available only to a user of the website where it is deployed. (Uncustomized site pages are shared by multiple websites, as noted earlier, but only websites where they have been specifically provisioned either as part of a Feature or as part of a site definition.)

• Parsing mode: Application pages are parsed in direct mode, which just means that they are parsed by the standard ASP.NET page parser. The first time an application page is requested, it is parsed and compiled and cached in the front-end web server's memory, where it remains until the application domain or the entire IIS host is recycled. With every subsequent request for the page, it is served from the cache, if it is there. Uncustomized site pages are also parsed in direct mode, but customized site pages, and new pages that are added to the Site Pages gallery, are parsed in safe mode. Safe mode processing differs from direct mode (that is, from standard ASP.NET page processing) in the following ways:

  • Only controls (including Web Parts) that are registered as safe in the web application's web.config file are rendered.

  • Inline server-side code is not allowed in safe mode, and an error page is returned to the user if the safe mode parser finds inline code on the requested site page. Inline code includes <script> elements, such as <script runat="server"> [code is here] </script>, and control event handlers, such as <asp:button OnClick="MyButtonHandler()" />. (Code behind that is compiled into a separate assembly is allowed, and so is embedded JavaScript.)

  • The page is not compiled. (And, thus, any compilation directives in the file are ignored.)

  Note

  The terms safe mode parsing, safe mode processing, and safe mode rendering are used interchangeably in SharePoint documentation.

Security and performance are the motivations for the two kinds of pages, especially the difference in processing mode. One of the purposes of SharePoint is to delegate administrative control to ordinary business users instead of requiring the intervention of network administrators and IT professionals. For this reason, users are allowed to add new pages and customize existing pages. If these users were allowed to add any control or Web Part they wanted or add any code blocks they wanted to the pages, there is a great danger that poor performing or malicious code would be added to a page. The reason that pages that use safe mode are not compiled is that there may be thousands of pages on a SharePoint installation. If every one of them was compiled into an assembly and loaded in memory, the performance of the server would be degraded, and only recycling the application domain would remove them. Moreover, recycling the application domain would remove all assemblies, not just the seldom-used pages. There is also a limit on the number of assemblies that can be loaded in an application domain that is imposed by the .NET Framework.

The safe mode parser only creates a control tree for the page, which can be unloaded again from memory immediately after use. On the other hand, pages that are shared, such as application pages and uncustomized site pages should be compiled on first use, because these pages are used a lot and subsequent requests for them can be handled much faster if the pages are cached as assemblies in the server memory. That is why they run in direct mode.

Security Note

It is possible to allow inline code, unsafe controls, or both, on selected customized site pages. You can do both things by adding one or more <PageParserPath> elements to the <SafeMode> element of the <SharePoint> section of the web application's web.config file. You can use the attributes of the <PageParserPath> element to specify a customized site page or set of customized site pages, and to enable inline code and unsafe controls for the designated site pages. However, you should use extreme caution when you make these kinds of changes, because they cancel the security benefits of safe mode. For example, if you allow inline code for all pages that have names on the pattern Contoso*.aspx in the directory /sites/contoso/SitePages, anyone with the right to add pages can create a page that has a name following that pattern and add it to that directory, including a page that contains malicious or poor-performing code. (The directory path is a "virtual path" that points to a set of files in the content database.) Notice that allowing unsafe controls only enables such controls that are added in an editing tool, such as SharePoint Designer. When an end user adds a Web Part to a page in the browser, the Web Part must still be registered as safe even if unsafe controls have been enabled in a <PageParserPath> element. You can turn on compilation for selected customized site pages by using an attribute of the <PageParserPath> element. This might be useful if you have reason to believe that the customized page will be visited often enough to justify compiling it into a DLL cached in memory.

Within the category of site pages, there are additional distinctions. In SharePoint Foundation, there are standard site pages and Web Part site pages. Standard site pages are wiki-enabled pages that allow inline Web Parts. Standard pages are objects of a class that derives from WikiEditPage. Web Parts pages, on the other hand, derive from WebPartPage. They have one or more Web Part zones into which Web Parts can be added, and they have no wiki-editable area.

SharePoint Server 2010 adds a third kind of site page: PublishingPage. (There is also a PageLayout class in SharePoint Server 2010, but this is an extension of the master page/content page system of ASP.NET, not another kind of site page.)

SharePoint also has a set of built-in pages that are designed for mobile devices. They do not use the ASP.NET master page/content page technology, and they are not divided into application pages and site pages. SharePoint mobile pages are all application pages, and they are located in the \_layouts\Mobile folder. There is one respect in which a SharePoint mobile page is more like a customized site page: If the page contains a mobile Web Part adapter, the adapter must be registered as a safe control or it is not rendered.

For more information about page types in SharePoint, see Building Block: Pages and User Interface, ASP.NET vs. SharePoint: Page Development, SharePoint Page Types, and Publishing Programming Model in SharePoint Server 2010 (ECM). For more information about mobile pages, see Building Block: Mobile Pages, Controls, and Adapters and Overview of Mobile Pages and the Redirection System. For more information about safe controls and safe mode, see Microsoft SharePoint Foundation as an ASP.NET Application.

Data Model, Data Management, and Query System

The primary data structure in SharePoint Foundation is the list. Every list belongs to a list type. Similarly, every column in a list has a field type, and every list item has a content type. External data—that is, data from outside the SharePoint content databases—can also be shown and managed in SharePoint.

Lists

Data in SharePoint Foundation is primarily stored as tables much as it is in a relational database, except that the tables are called "lists" in SharePoint lingo. Indeed, the back-end storage of content data for a SharePoint Foundation web application is in one or more SQL Server databases. These are called content databases. (There is also a special configuration database on the computer that is running SQL Server that holds farm configuration data, and a special BDC database that supports Microsoft Business Connectivity Services (BCS). For more information about BCS, see External Lists and the Business Connectivity Service later in this article.) However, there are some differences between relational database tables and SharePoint Foundation lists:

• Data is not queried by SQL. Instead, data is queried in server-side code either by LINQ or by queries formulated in CAML. For more information about server-side data querying, see Building Block: Queries and Views and the topic Querying from Server-side Code, along with its child topics. You can programmatically write data to the lists using either the server object model or the LINQ to SharePoint provider. For more information, see Managing Data with LINQ to SharePoint. From client-side code, data is queried by using either the client object model or WCF Data Services (formerly ADO.NET Data Services). For more information about client-side data querying, see Querying from Client-side Code.

  Warning

  Directly accessing the back-end computer that is running SQL Server by using SQL queries, stored procedures, or any other method is not supported.

• A list can have a column whose possible values are the values of a column on a different list. The lookup column relationship between the lists is somewhat like a foreign key relationship between two relational tables. However, the field on the target list that provides the values is not necessarily the foreign key. All SharePoint Foundation lists have an ID column. This column is, in effect, always the foreign key in any lookup relationship. For more information about lookup relationships in SharePoint Foundation, see Lookups and List Relationships and List Relationships in SharePoint 2010.

• Lists can be joined just as tables can, but with some restrictions. There must be a lookup relation between the lists or they cannot be joined. For more information about list joins, see List Joins and Projections.

• A SharePoint Foundation document library is a special kind of list in which each row includes an attached document, and the other columns are data about the document, such as its author, when it was last edited, and who has it checked out. Picture libraries are similar except that the attached file is an image file. For more information, see Building Block: Lists and Document Libraries.

Every list has a list type, and SharePoint Foundation includes many built-in list types that enable end-users to create the most common kinds of business and team solutions. Among these are Announcements, Tasks, and Calendar. Developers can also create custom list types. For more information about list development, see SharePoint List Data Model.

Content Types and Field Types

A row in a list—that is, a list item—also has a type. These are called content types. Each is basically a set of columns and metadata. The simplest is the built-in Item content type. All other content types are derived from Item. SharePoint Foundation includes many built-in content types, such as Event and Announcement. Developers can create custom content types. For more information about content types, see Building Block: SharePoint 2010 Content Types, Content Types, and SharePoint Columns, Lists, and Content Types.

A column in a list, also known as a "field," also has a type, and it is distinct from the data type of the values that can be stored in the field. A SharePoint Foundation field type includes not only information about the underlying data type, but also information about how the data is formatted and rendered on forms, such as the forms for creating, displaying, and editing specific list items. For example, it is the field type that determines whether a field value is entered as string or from a drop-down list of values. Many field types are built-in to SharePoint Foundation, such as the Modified By field on a document library and the Due Date field on a Task list. Developers can create custom field types. For more information, see Building Block: Columns and Field Types, Custom Field Types, and SharePoint Columns, Lists, and Content Types.

External Lists and the Business Connectivity Service

External data, such as data in an SAP installation or Oracle database, can also be represented as a list on a SharePoint Foundation page and within SharePoint Workspace and Microsoft Office client applications. The Microsoft Business Connectivity Services (BCS) of SharePoint Foundation enables read/write access to this data.

The critical components of BCS are Business Data Connectivity (BDC) service models. Each model is an XML file that describes a type of external data source, such as SAP services. One or more specific instances of the data source, such as a particular SAP database, are defined in the model, including connection and security information about the data source. The business entities in the data source, such as Customer or Order, are represented in the model by external content types, and the model also defines relationships between these entities. Finally, the model defines, for each entity, a set of standard operations, called stereotyped operations, that can be performed on the entity, including create, retrieve, update, and delete operations. For more information about the structure of BDC models, see BDC Model Infrastructure. The models are stored in a dedicated database on the SharePoint farm computer that is running SQL Server, which is distinct from the configuration database and the content databases. This database is called the metadata store.

After a model is added to the metadata store, users can access the external data in a variety of ways. For example, in SharePoint Foundation, users can create a SharePoint list out of an external content type, such as Customer, or add a column of data from the external source to an existing list. SharePoint Server 2010 has some BCS enhancements: It includes some built-in Web Parts for working with external data, and it enables users to search the external data. Also, SharePoint Workspace and Microsoft Office client applications can access and display the external data directly. Finally, with SharePoint Server 2010, users can index and search the external data sources.

A service named Business Data Connectivity service manages the interactions between SharePoint and the external data source. The service runs on an application server in the farm. There is also a BDC Runtime that runs on all front-end web servers. When a SharePoint client application requests external data, the BDC Runtime on the front-end web server requests the metadata that defines the BDC model from a locally cached copy of the BDC metadata store. The runtime then uses the metadata provided to perform data operations directly on the external system. Similarly, on client computers that are running SharePoint Workspace and Microsoft Office client applications, a BDC Client Runtime enables direct client access to the external data by using the BDC model, which is also cached on the client computer (although the SharePoint Server 2010 farm is still needed for long-term persistence of the BDC models). The external data itself can be cached on the client computer to enable an offline experience.

After the BDC service is running and the needed BDC models are registered, end users can create non-code solutions that involve external data. For more complex solutions, the BCS Runtime APIs enable developers to make custom Web Parts or other solutions to interact with the external data. Because a BDC model provides a kind of translation between operations in code and the stereotyped operations of the external data source, the same APIs can be used for all external data sources no matter how different their individual access systems are. BDC solutions are packaged and deployed as farm solutions.

Figure 7 shows some of the major components of BCS and their relationships.

Figure 7. BCS runtime and deployment components

The BDC service is built in conformance with the Service Application Framework of SharePoint Foundation. See also Services and the Service Application Framework and Services Hierarchy later in this article.

For more information about BCS architecture, see BDC Architecture, Mechanics of Using Business Connectivity Services, Business Data Connectivity (BDC) Service, and External Data in SharePoint 2010.

Services and the Service Application Framework

Services on a SharePoint farm can be usefully divided into four groups:

• Windows Services

• Web Services

• IIS Web Services

• Configuration Services

For details and examples of each of these four kinds of services, see Background: Service Entities in Microsoft SharePoint Foundation.

Services in the latter two categories can be created to conform to the Service Application Framework. Many of the services built-in to SharePoint Foundation and SharePoint Server implement the framework. Moreover, the framework lets developers build scalable middle-tier shared services that are hosted in SharePoint Foundation.

A service that conforms to the framework can be split into multiple configured farm-scoped instantiations (CFSIs). Each of these provides the functionality of the service, but each has its own individual permission and provisioning settings. A CFSI is not an actual running process on a particular server. The same CFSI can be running on multiple servers, but it is not the same as the entirely abstract service itself either. Each of the servers on which the CFSI runs has its own actual instance (a running process) of the CFSI. Moreover, more than one CFSI of a given service can be running on the same server or servers. Thus, the framework provides a means for different versions of the same basic service to be available simultaneously. A consuming application on a front-end web server can target a specific CFSI.

Figure 8 shows the services and service instances on a hypothetical 10-server farm. This figure is repeated, with a more detailed explanation of its contents, in Background: Service Entities in Microsoft SharePoint Foundation. For this article, note only that the light translucent rectangles represent services, the darker translucent rectangles represent CFSIs (also known as service applications), and the smaller solid rectangles represent instances of services. The CFSIs are present only for the services that implement the Service Application Framework, which in this example are the Usage Service, the Application Discovery and Load Balancer Service, the Security Token Service, and the BDC Service.

Figure 8. Services and service instances on a typical 10-server farm

Applications that need to consume a particular CFSI of a service do so through proxies. The front-end web server that hosts the application has a proxy to represent the service itself and a second proxy to represent the CFSI that is being targeted. The proxies are not depicted in Figure 8, but Figure 9 shows a single-server SharePoint Foundation farm immediately after installation. Note also the following:

• Web services that implement the Service Application Framework are represented with a dot-bordered box. At initial installation, each has a single CFSI, sometimes called a "service application".

• The service proxies belong to the farm, but each CFSI proxy (also known as a service application proxy) belongs to a web application. The content publishing web application and the Central Administration web application each have their own proxy for the Business Data Catalog CFSI, and they each have their own for the Usage and Health Data CFSI. Neither has a proxy for the Subscription or Application Discovery and Load Balancer CFSIs at initial installation.

Figure 9. Services, CFSIs, service instances, and web applications in a new single server deployment

For more information about services in SharePoint and the Service Application Framework architecture, see Background: Service Entities in Microsoft SharePoint Foundation, Service Application Framework Architecture, and SharePoint Service Application Topologies.

Site Definitions and Web Templates

What makes it possible for ordinary business users to create their SharePoint websites without the intervention of IT professionals is the fact that types of websites in SharePoint are given detailed, stored definitions. Using the SharePoint UI, users can then instantiate a particular website from any of the defined types. There are actually two kinds of definitions of site types: site definition configurations and web templates. A site definition is stored on the file system of the front-end web servers, in a subfolder of %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\SiteTemplates, as a set of XML configuration files and possibly also page files and other supporting files. A web template is stored in a SharePoint Solution Package (.wsp file) in the content database, specifically in the Solution Gallery of a site collection. Either kind of type definition specifies such things as a default home page, various aspects of the default look-and-feel and layout of the site's pages, the types of lists available in sites of the specified type, and other configuration details of the site type.

For more information, see Building Block: Websites and Site Collections and the Site Types: WebTemplates and Site Definitions node of the SharePoint Foundation SDK.

SharePoint Security

The SharePoint security system protects deployments from both errant users and errant code.

User Security

SharePoint Foundation supports security for user access at the website, list, folder, and item levels. Security management is role-based at all levels. The authorization process assumes that the user has already been authenticated, which refers to the process by which the current user is identified. SharePoint Foundation does not implement its own system for authentication or identity management, but instead relies solely on external systems, whether Windows authentication or non-Windows authentication.

Authentication

SharePoint supports several forms of authentication. The default is Windows claims-based authentication. The claims-based identity model for SharePoint is built upon Windows Identity Foundation (WIF). Under this model, the user presents an identity to your SharePoint farm as a set of claims. One claim could be the user's name, another might be an email address. An external identity system is configured to give SharePoint all the information that it needs about the user with each request, along with cryptographic assurance that the identity data comes from a trusted source. Other types of supported authentication include Windows classic authentication and ASP.NET forms-based authentication. For more information about authentication and SharePoint, see the Getting Started with Security and Claims-Based Identity Model and SharePoint Claims-Based Identity nodes of the SharePoint Foundation SDK.

Authorization

Access to websites, lists, folders, and list items is controlled through a role-based membership system by which users, and groups of users, are assigned to roles that authorize their access to SharePoint objects. By default, permissions are inherited in the sense that a user who has certain permissions for an object, such as a list, will have the same permissions for its child objects, such as folders and list items. However, it is possible to break inheritance and assign to users and groups a different set of permissions to a child object. A role definition is a set of rights, such as rights to read, create, or delete. A role assignment associates a user or group with a role definition.

SharePoint supports two kinds of groups: domain groups and SharePoint groups. Domain groups remain outside SharePoint control; users cannot use SharePoint to define, browse, or modify domain group membership. SharePoint groups are scoped to the site-collection level, and they can be used only within the site collection. Domain groups can be used anywhere within the scope of the Active Directory service.

For more information, see Authorization, Users, and Groups.

Code Security

Much of the code security story for SharePoint was already described in earlier sections. In this section, different parts of the story are very briefly pulled together, and some gaps in the story are filled.

Every web application runs in an IIS application pool that processes HTTP requests. The pool itself runs in a user account known as the application pool identity. In a multiserver farm, this is usually a domain user. The application pool identity is the user identity for code that runs in an IIS worker process. However, access to various SharePoint objects is determined by the permissions of the user who has made the request that is being processed. The isolation of web applications into separate application pools ensures that if one of them crashes, the others are not affected.

The IIS worker process calls assemblies that may operate under their own additional restrictions. If the assembly is loaded out of the global assembly cache, it operates in full trust. However, if the assembly is loaded out of the web application's \bin directory, it is subject to the trust limitations that are defined by a CAS policy.

If the request is for a sandboxed solution, the SharePoint execution manager that runs in the IIS worker process spawns a sandboxed worker process. The latter process runs within a highly restrictive CAS policy, is limited to a subset of the SharePoint server object model, and can only access resources within the site collection to which the solution was deployed.

If the request is for a customized site page, all Web Parts on the page must be registered as safe controls and, by default, the page is not returned at all if it contains inline code blocks.

Server Object Model in SharePoint

The server object model in SharePoint is large, and only some of the truly critical classes can be described in this article. These classes can be usefully divided into three hierarchies: the Physical Objects Hierarchy, the Content Hierarchy, and the Services Hierarchy. Each is described briefly in the following subsections with links to more extended discussions. For a synoptic overview of the major classes from all three hierarchies, see Server and Site Architecture: Object Model Overview.

Physical Objects Hierarchy

The Physical Objects Hierarchy includes classes that represent physical entities, such as servers and farms—the two most important.

A SharePoint Foundation farm, and its configuration database, is represented by the SPFarm class. A server farm consists of one or more physical servers. These may include one or more front-end web servers, zero or more application servers, and a computer running SQL Server that may be hosted on a dedicated database server or on one of the other servers in the farm. (If it is a dedicated server, SharePoint is not actually installed on it, although it is still seen as a member of the farm in the farm management UI of the Central Administration application.) A farm may consist of just a single server. If the farm has multiple front-end web servers, they are usually load-balanced. You can use any hardware or software load-balancing solution, including the built-in NLB (Network Load Balancing) in Windows Server 2008. SharePoint does not itself supply the load-balancing.

SPFarm inherits from SPPersistedObject, which means that the object (there is only one) that instantiates the class persists in the configuration database. The three most important child classes of SPFarm are SPServer, SPService, and SPSolution.

A physical server in a SharePoint Foundation farm is represented by the SPServer class. In addition to many inherited members, it has an Address property that holds the IP address of the server and a Role property that identifies the server's role in the farm. It also has a ServerInstances() property that holds references to all the instances of Windows services and web services that are running on the server. SPServer also inherits from SPPersistedObject, so server objects are persisted in the configuration database.

For more information about the Physical Objects Hierarchy in the object model, see Physical Objects Hierarchy of Microsoft SharePoint Foundation and Background: Physical Objects in Microsoft SharePoint Foundation.

Content Hierarchy

The Content Hierarchy includes classes that represent publishable items of data, such as list items. There are also classes that represent nested containers of data, such as lists, content databases, websites, site collections, and web applications.

High Level Object Model

Beneath the farm, the broadest content container is the web application, which is represented by the SPWebApplication class. An SPWebApplication object represents a content-publishing web application in SharePoint Foundation. It contains one or more content databases, which hold the data of one or more site collections. Each such web application is served by at least one (and usually only one) IIS website and typically has its own application pool in IIS. Also, each web application has its own security and authentication settings. The SharePoint Foundation object model provides some hooks into the web application's other life as an IIS object through the IisSettings and ApplicationPool properties. For more information about the relation between SharePoint web applications and IIS, see SharePoint as an ASP.NET-IIS Application earlier in this article.

The SPWebApplication class has a ContentDatabases property that holds all its child content database objects.

Every web application contains one or more content databases. Each of these is represented by the SPContentDatabase class. A content database is a SQL Server database that contains all the data (lists, list items, blog posts and comments, wiki pages, and documents in document libraries) and the customized page files that constitute the site collections that belong to the database. Objects that represent each child site collection are in the Sites property.

Both SPWebApplication and SPContentDatabase inherit from SPPersistedObject, so these objects persist in the farm's configuration database.

For more detailed discussions of web applications, content databases, and their classes, see Content Hierarchy of SharePoint Foundation and Background: Content Entities in Microsoft SharePoint Foundation.

Middle Level Object Model

At the middle level of the content hierarchy are site collections and their subsites. An SPWeb object represents a single website. An SPSite object represents a collection of websites within a SharePoint web application that are grouped together for mainly administrative reasons. (The SPSite class is not a collection in the sense of a class that implements ICollection.)

The SPSite class has a RootWeb property that holds its child top-level website. (Top-level websites were called "root webs" in the first version of SharePoint Foundation, Microsoft SharePoint Team Services.) In turn, the SPWeb object that represents the top-level website has a Webs property that holds all its immediate child subsites (but not the subsites of those subsites). (The AllWebs property returns all the descendent sites and the top-level website.)

Among the SharePoint elements that can be scoped to the site collection level are master pages, Web Parts, themes, lists, content types, and Features. A site collection can also be a unit of backup and restoration. It is also the level at which groups of users are created and assigned default permissions. Site collections are the largest possible scope for a search in SharePoint Foundation, although broader search scopes are possible in SharePoint Server 2010.

The content of a site collection is always included within a single content database.

Websites can be children of other websites, and all websites belong to a site collection. The SPWeb class has dozens of properties and methods for programmatic handling of every aspect of the website, including users, lists, fields, content types, Features, alerts, and much more.

For much more information about site collections and websites, see Building Block: Websites and Site Collections, Content Hierarchy of SharePoint Foundation, and Background: Content Entities in Microsoft SharePoint Foundation.

Low Level Object Model

At the heart of SharePoint are lists and list items, which are represented, respectively, by the SPList and SPListItem classes. The SPList class has members for programmatically adding, deleting, and retrieving list items, and also for managing metadata about the list such, as its content types and fields. The SPListItem class has members for managing the item's fields, the values of its fields, its content type, and any associated workflows. Lists can have folders that give them a hierarchical structure. Folders are represented by the SPFolder class.

Note that, although any given list item has a specific content type, the object model provides only the SPListItem class to represent all list items. Hence, working with list items and their fields is, in many ways, like working in a weakly typed environment. SharePoint includes a tool, SPMetal, that can generate code for an object relational mapping. The generated code defines classes for each content type in a website and, for each field in the content type, it declares a strongly typed property. The tool makes it possible for LINQ queries to be strongly typed. For example, if a developer mistakenly refers to the DueDate field of the Task content type as "DateDue", the compiler catches the mistake if SPMetal is used and the reference is to a property of a Task type object, myTask.DateDue. However, without SPMetal, the reference would have to be to a member of the fields collection of an SPListItem object, myItem["DateDue"]. The mistake would not be caught until run time.

Each field (column) on a list item is represented by an SPField object.

For more information about lists, list items, folders, and fields, see Building Block: Lists and Document Libraries, Building Block: Columns and Field Types, and SharePoint List Data Model.

Services Hierarchy

The Services Hierarchy includes classes that represent web services, Windows services, other types of services, instances of services, and CFSIs of services (see Services and the Service Application Framework earlier in this article).

A SharePoint Foundation service is represented by a class that inherits from SPService. The class provides members that get information about the jobs the service is performing. If a service implements the Service Application Framework, the SPService object has an Applications property that holds all the CFSIs of the service that are running on the farm. A CFSI of a service is represented by an object of a class that inherits from the SPServiceApplication class.

Every SPService object has an Instances property that holds all the instances of the service that are running on various servers in the farm. The instances that host a particular CFSI are held in the ServiceInstances property of the SPServiceApplication object that represents the CFSI. No more than one instance of each CFSI runs on any one server. However, a given service can have multiple CFSIs, and they can run on the same servers. Moreover, a given CFSI (and, thus, a given service) can run on multiple servers, in which case each server has its own actual instance (a running process) of the CFSI. If the service has no CFSIs, it still has instances on each server on which it runs, but there can be no more than one instance of such a service on a given server. Each instance is represented by an object of a class derived from SPServiceInstance.

Applications that consume services that conform to the framework are represented by proxies. Here, also, there is a proxy object for the service as a whole and a proxy for the particular CFSI that is being consumed. The consumer proxy for a service is represented by an SPServiceProxy object and the proxy for the CFSI is represented by an SPServiceApplicationProxy object.

Figure 10 illustrates the relationships among the main classes in the Service Application Framework.

Figure 10. Relation of major classes in the Service Application Framework

The following kinds of objects are persisted in the configuration database because these classes inherit from SPPersistedObject:

• SPService

• SPServiceApplication

• SPServiceInstance

• SPServiceProxy

• SPServiceApplicationProxy

For more information about the services object model, see Services Hierarchy of Microsoft SharePoint Foundation and Background: Service Entities in Microsoft SharePoint Foundation.

Client Object Models in SharePoint

There are three client object models in SharePoint Foundation: one each for Microsoft Silverlight applications, Microsoft .NET Framework applications, and JavaScript. They are almost the same in the APIs that they expose. The *.js files that contain the JavaScript client object model, like all *.js files, are downloaded automatically to the user's computer when a page that references the object model is accessed. (The standard built-in master page for SharePoint references these files.)

The assemblies that contain the Silverlight object model can be downloaded in the .xap file, but another alternative is to download at run time the file %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\LAYOUTS\ClientBin\Microsoft.SharePoint.Client.xap, which encases the Microsoft Silverlight assemblies.

The .NET Framework object model can be called only if the assemblies that contain the object model have been installed to the client computer. You must use the official redistribution package, SharePoint Foundation 2010 Client Object Model Redistributable.

These object models provide a subset of the classes in the server-side microsoft.sharepoint.dll assembly, although many of the class names have been slightly changed (usually by dropping the "SP" at the beginning of the class name). The client object models are implemented as a WCF service (.../_vti_bin/client.svc), but they use web bindings to implement efficient request batching. Commands are serialized into XML and sent to the server in a single HTTP request. For every command, a corresponding server object model call is made, and the server returns a response to the client in compacted JavaScript Object Notation (JSON) format, which the proxy parses. The client APIs provide a familiar, object-oriented interface to the WCF service, so developers are shielded from the details of the service. In particular, the client-side runtime handles all communication between the clients and server.

Microsoft.SharePoint.Client is the core namespace used for the Microsoft .NET Framework managed and Silverlight object models, and SP is the core namespace for the JavaScript object model. Client objects inherit from the ClientObject class (JavaScript: ClientObject).

SharePoint code on a client begins by retrieving a client context object that represents the current request context, and through this context, you can obtain access to client objects at the site-collection level or lower in the SharePoint Foundation hierarchy. Client objects inherit from the ClientObject class (JavaScript: ClientObject), and you can use them to retrieve properties for a specific SharePoint object, to retrieve child objects and their properties, or to retrieve child items from a collection.

After a client object is obtained, a query is defined and loaded. This is followed by a call of the ExecuteQuery() method or the ExecuteQueryAsync(ClientRequestSucceededEventHandler, ClientRequestFailedEventHandler) method (JavaScript: executeQueryAsync) to send the query to the server. The query commands are packaged as XML and processed on the server by the WCF service, Client.svc, which runs the batched commands by calls to the server-side object model. Returned data is sent back to the client by the service as JSON data.

Figure 11 shows the major components and events of the client's interaction with the server when the client object models are used.

Figure 11. Client and server interaction with the client object models

For more information about the client object models, see the following topics:

• Using the SharePoint Foundation 2010 Managed Client Object Model

• Using the SharePoint Foundation Client APIs

• SharePoint 2010 Client Object Model

• SharePoint 2010 Client Object Model Hierarchy and Identity

• Client Context as Central Object

• Client Application Models in SharePoint 2010

Workflows in SharePoint

Like all workflows, SharePoint workflows model and systematize business processes. However, SharePoint workflows are generally oriented around the SharePoint list data model. In most cases, each instance of a SharePoint workflow embodies processes that surround an item in a list or document library. Indeed, the most common kinds of SharePoint workflow are workflows that are associated with a list or with one or more content types. Some SharePoint workflows are manually started, but they are most commonly designed to start automatically in response to some event connected with a list item or library item, such as adding, deleting, or updating an item.

There are also workflows associated with websites rather than lists or content types. These are always started manually (or programmatically), not automatically in response to some event. In addition, they are generally used to systematize processes that transcend particular list items, such as retrieving and setting values of multiple list items across different lists, or performing non-list operations, such as creating and configuring subsites.

Workflows in SharePoint are built on the framework provided by Windows Workflow Foundation (WF). When a workflow is running in SharePoint, the WF runtime engine is hosted in the SharePoint process. The WF runtime engine loads and unloads workflow templates and provides sequencing and persistence for workflows. The persistence services are crucial because they enable workflows to remain active through discontinuous user sessions, through resets of the front-end web server, and even through reboots of the server.

For many of its services, WF enables customization by hosting applications. SharePoint provides custom implementations of the following services for the engine: transaction, persistence, notifications, roles, tracking, and messaging. For example, when a workflow instance reaches a point at which it is waiting for user input, SharePoint unloads that workflow instance from memory and persists its data in the content database. Then, when an appropriate event occurs that requires that the workflow instance start again, such as when a user enters input, SharePoint re-instantiates the workflow instance by using persisted data, so the workflow instance can receive and handle the event as necessary.

A SharePoint workflow type is represented by two entities that work in partnership with each other. First, there is a workflow definition. These can be defined in code and compiled into an assembly, or defined in XOML markup. When the workflow type is defined in XOML, it is called a declarative or "no code" workflow, and it persists uncompiled in the content database until it is called, at which point it is compiled just-in-time for use. The workflow definition, in either compiled or declarative form, specifies the parameters, events (such as activation), and sequence of activities in the workflow. It also defines the branching structures of the workflow and the conditions that determine the paths of execution.

The partner of the workflow definition is the workflow template definition. A workflow template definition is an XML file that contains the information SharePoint requires to instantiate and run the workflow, such as the following:

• Name and description of the workflow

• Class within the workflow assembly to call

• Identity of the workflow assembly

• Location of any custom forms used in this workflow

You can store these XML files in either of two locations. The first is %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\lcid\Workflow. For example, the template definition of the built-in moderation workflow is stored in the moderationworkflow.xml file. The second location is the Workflows list of the root website of a site collection. The built-in Three-state workflow is an example. Regardless of where it is stored, at run time, the workflow template definition is used to create an SPWorkflowTemplate object, which is cached to speed up the creation of workflow instances.

You can associate workflow templates with lists (including document libraries), content types, and websites. For any workflow template that is associated with a list or content type, an item of that list or content type can have an instance of the workflow in progress. Although there can be only one instance of a specific workflow type in progress at any one time for a given item, any item can have multiple workflows in progress, each of a different type.

You can create compiled workflows by using the Microsoft Visual Studio workflow designer. You can also create declarative workflows by using SharePoint Designer. Workflows are installed as Features at the site collection level. But you can also "publish" a workflow directly from SharePoint Designer to a website, in which case there is no Feature.

A workflow type is available for a list, content type, or website only if it is associated with the list, content type, or website. These associations are stored in the content database of each site collection. This association data typically includes whether the workflow is started automatically or by users. If a workflow is added to multiple content types, lists, or websites, it will have one entry for each such association. Likewise, if you add multiple workflows to a specific content type, list, or website, the database contains one entry for each workflow that is added to the content type, list, or website.

For more information about the SharePoint workflow infrastructure, see Introduction to Workflows in SharePoint Foundation.

Conclusion

Even an article as long as this one cannot cover all the SharePoint architectures or say everything that could be said about the architectures that it does cover. It does provide an overview of the most important SharePoint architectures and should enable you to do further research on each of them.

Additional Resources

For more information, see the following resources.

• SharePoint 2010 Developer Center

• Microsoft SharePoint 2010 SDK (online)

• Download: SharePoint 2010 Reference: Software Development Kit

• Download: Microsoft SharePoint Foundation 2010

• Download: Microsoft SharePoint Server 2010

• SharePoint 2010 Development Basics Technical Articles