SharePoint 2010 Development for ASP.NET Developers

  • Article

Summary:

This paper provides ASP.NET developers with helpful insight into SharePoint concepts from an ASP.NET developer’s perspective, making it easier to develop SharePoint applications. As a prerequisite, you should have a basic understanding of the differences between the SharePoint and ASP.NET platforms. These differences are explained well in SharePoint 2010: Developer Platform White Paper, and you should read and understand this paper before continuing.

SharePoint is the fastest growing product in Microsoft’s history and its developer community is growing quickly. In the beginning, SharePoint was simply a collaboration platform, but it has evolved throughout the years to become what it is today; an integrated suite of server features including: content management, enterprise search, business connectivity services, and information management.  With such powerful capabilities, the SharePoint application landscape ranges from simple web part solutions over business collaboration applications to sophisticated applications for integrating Line of Business (LOB) data.

Although SharePoint is considered to be a development platform, it was hard for developers to quickly ramp up on SharePoint development because it required a deep knowledge of SharePoint architecture and infrastructure. Significant changes were made in SharePoint 2010 and Visual Studio 2010 to improve the development experience and some of the key concepts familiar to ASP.NET developers were introduced such as easy deployment and debugging of your SharePoint applications and designers for building different types of SharePoint customizations.

SharePoint developer tooling has been greatly enhanced in Visual Studio 2010 and it includes many different templates for SharePoint projects and project items. Visual Studio provides developers with a familiar development environment and all of its benefits, such as IntelliSense and enhanced debugging. It makes it easier for ASP.NET developers who want to target the SharePoint platform. For more information, see the Glide Path for ASP.NET Developers on MSDN.

SharePoint Overview

There are three versions of SharePoint 2010:

Microsoft SharePoint Foundation 2010: Microsoft SharePoint Foundation is the essential solution for organizations that need a secure, manageable, web-based collaboration platform. It serves as the basis for SharePoint server and offers out of the box elements such as blogs, wikis, team workspaces, and document libraries, providing users with the ideal way to share information and collaborate within a customized website. In addition, it provides services such as Business Data Connectivity services to integrate external data, basic search services and workflow services. 

Microsoft SharePoint Server 2010: SharePoint Server 2010 is a superset of SharePoint Foundation 2010 that adds enterprise content management and enterprise search capabilities as well as support for using InfoPath forms, Access services, social networking features, community features and Dashboards. Although SharePoint Server 2010 is built largely on SharePoint Foundation 2010, it’s a separate product—using it requires purchasing a license.

SharePoint Online: A Microsoft-hosted version of SharePoint technology, this version lets both users and developers use SharePoint’s functionality without installing any on-premises SharePoint software.

SharePoint provides great opportunities for application development at all levels of the software development stack. Being built upon ASP.NET and Windows Server platforms allows SharePoint to expose infrastructure features and inherent points of extensibility to application developers.

SharePoint Developer Tools: SharePoint Designer and Visual Studio 2010

SharePoint Designer and Visual Studio 2010 are two tools that can be used (together or separately) to customize or develop SharePoint solutions.

SharePoint Designer 2010 is the tool of choice for the rapid development of SharePoint applications. Using SharePoint Designer, advanced users and developers alike can rapidly create SharePoint solutions in response to business needs. Advanced users can compose no-code solutions that encompass a variety of common scenarios, from collaborative sites and web publishing to Line-Of-Business data integration, business intelligence solutions, and human workflows, all leveraging the building blocks available in SharePoint in an easy to use environment. In addition, developers can use SharePoint Designer 2010 to get a quick start on SharePoint development projects. As SharePoint Designer works in a “connected mode”, meaning you must be connected to a SharePoint site, all SharePoint artifacts created by SharePoint Designer, such as SharePoint pages and content types, are stored in the content database.

This concept might be new to ASP.NET developers and cause some confusion. To understand why SharePoint stores some artifacts in the database you need to understand the difference between customized and uncustomized files in SharePoint.

Uncustomized Files

When you create a new SharePoint site in a site collection, SharePoint provisions instances of its files into a content database that resides on the file system. The files on the file system are called templates. Alone, they are not very useful because they are not mapped to any specific URL and therefore cannot be directly accessed with a browser. The instances of the files in the content database are merely added to the site. When ASP.NET receives a request for a file, it first locates it in the content database. The entry in the content database tells ASP.NET that the file is actually based on a file on the file system and, therefore, ASP.NET retrieves the source of the file on the file system when it constructs the page.

This is how most files start in a SharePoint site. They are called uncustomized files – files that reside on the file system and not in the content database. The only thing that resides in the content database is an instance of the uncustomized file that registers the file in the site.

One thing to note is that there can be multiple instances of a file in a site collection's content database, or content databases, all referencing the same file. This is how the master page included with SharePoint, v4.master, is implemented. Every SharePoint site starts with an uncustomized instance of the v4.master page. If an environment contained 100 SharePoint sites created with the Blank Site template, and no customizations had been made, there will be 100 uncustomized instances of the v4.master page in the various content databases, yet there would be only a single v4.master page on the file system.

Customized Files

The other type of file is a customized file. A customized file is one in which the source of the file lives exclusively in the site collection's content database. How can a file become customized? If you modify the file in any way through the SharePoint API, the source of the new page is saved in the content database. This includes SharePoint Designer 2010, which uses the SharePoint API via RPC and web service calls to change files in sites. When the file is requested, ASP.NET first finds the file in the content database. The entry in the database tells ASP.NET whether the file is customized or uncustomized. If it is customized, it contains the source of the file which is used by ASP.NET in the page contraction phase.

A file can start as an uncustomized instance and become customized. For example, after a site has been created, someone can open the site in SharePoint Designer 2010 and make a few changes to the v4.master page. When the file is saved, it changes from uncustomized to customized. This system enables SharePoint to store only a single instance of a file template on the file system regardless of how many sites implement it, while at the same time enabling site-by-site customizations of the file.

Andrew Connell has written a great article for Windows SharePoint Services V3 which can be found here: https://msdn.microsoft.com/en-us/library/cc406685.aspx. The concept of customized vs. uncustomized files hasn’t changed in SharePoint Foundation 2010 or SharePoint Server 2010 so the content of that article is still valid.

Visual Studio 2010

Visual Studio 2010 uses a different approach than traditional SharePoint design tools. First of all, Visual Studio targets developers writing code as opposed to SharePoint Designer which targets the Power User or breadth developer. Secondly, Visual Studio works in a “disconnected mode” (Note: Visual Studio 2010  must be installed on the same machine as SharePoint 2010) which means that the project output, such as dlls, files, etc., is stored on disc and packaged into a SharePoint solution package (WSP) file (see Deploying Solutions to SharePoint). This enables full application lifecycle management capabilities, such as source control and daily builds.

Because Visual Studio 2010 offers visual designers, packaging and deployment support, project and item templates, and project and tool extensibility, you might be wondering when to use which tool. In general, use Visual Studio 2010 if you:

  • Are a developer.
  • Need to use Application Lifecycle management for your SharePoint solution.
  • Have more complex requirements for your SharePoint solutions which require code.
  • Visual Studio is your preferred development environment.

Also, there are some SharePoint development tasks which can be accomplished only in Visual Studio. Use Visual Studio if you want to create:

  • Site definitions
  • List definitions
  • Custom web parts
  • Application pages
  • BDC models that use a .NET connector
    • Foreign key-less associations between two external content types
  • Code-based workflows
  • User controls
  • Event receivers
  • Feature event receivers

Although SharePoint Designer and Visual Studio have different target audiences and development approaches, they work well together. For example, SharePoint Designer offers graphical designers for creating SharePoint pages and lists whereas Visual Studio only provides an XML editor for list templates.  Let’s assume you need to build a simple SharePoint library solution containing a web part, a SharePoint list and an event receiver. Because the web part and the event receiver require code, you would develop those in Visual Studio, but what about the list? As mentioned earlier, Visual Studio provides only an XML editor for creating lists. To build a list using the XML editor you need to understand CAML (Collaborative Application Markup Language) which is used by SharePoint to define lists, sites templates and definitions. A good solution to this problem is to take advantage of the List Designer in SharePoint Designer which doesn’t require a knowledge of CAML. Once you have designed the list in SharePoint Designer, you can save the entire SharePoint site which contains the list as a WSP (see Web Templates for more information) and then import the WSP into Visual Studio. (See Importing Items from an Existing SharePoint Site.) Now, you can add the web part and the event receiver to the list from the WSP and deploy all of the contents together in the library solution.

SharePoint Extension points

There are many different ways to customize and extend SharePoint. The following sections highlight a number of these different SharePoint extension points.

SharePoint Sites
Often, it is hard to understand the difference between a SharePoint site and an ASP.NET site. From an ASP.NET developer’s point of view, a web site is a group of related web pages hosted by an HTTP server on the World Wide Web or on an intranet site. The same applies to SharePoint sites. Most web sites have a home page as their starting point. The home page is interconnected with other pages by using hyperlinks. You can use top-level sites and subsites. A top-level site is a web site at the top of the hierarchy in a site collection from which you can manage site collection features. A top-level site can have one or more subsites, and subsites can have other subsites. A subsite is a complete web site stored in a named subdirectory of the top-level web site. Each subsite can have administration, authoring, and browsing permissions that are independent from the top-level web site and other subsites. This strategy divides site content into distinct, separately manageable sites. The entire structure of a top-level web site and all of its subsites is called a web site collection, a set of web sites that have the same owner and share administration settings. Each site collection contains a top-level site, can contain one or more subsites, and may have a shared navigation structure. Site collections can also be seen as information boundaries in SharePoint because content isn’t typically shared between site collections.

This hierarchy enables users to have a main working site for the entire team, plus individual working sites or shared sites for side projects. Top-level web sites and subsites allow different levels of control over the features and settings for sites. The administrator of the Web site controls the ability to create, access, and contribute content to a Web site.

SharePoint Lists and Libraries

A list in SharePoint is a collection of information that is shared with team members. For example, you can create a sign-up sheet for an event, or you can create tasks that you need to track. You can use the lists that come with SharePoint as-is, you can change the lists, delete lists that you don't need, or create new custom lists. SharePoint provides list templates for links, announcements, contacts, events, tasks, and issues. You can also add views to lists. For example, you can create a view on a task list that only shows tasks that are active. When you create your own lists, you can base them on the designs used for the built-in lists, or you can create custom lists from scratch. The following is an example of a task list.

Find more information about working with SharePoint lists from an end user perspective and from a developer perspective.

SharePoint libraries are technically lists for collections of files, such as Word documents, Excel documents, images etc., that are shared with team members on SharePoint. For example, you can create a library of common documents for a project, and team members can use their Web browsers to find the files, read them, and make comments. Users with Microsoft Office can check out and edit the files as if they resided on a local or network drive.

Application Pages and Site Pages

SharePoint Foundation makes distinctions in web pages that are unknown to ASP.NET. To begin with, SharePoint Foundation divides web pages into two broad categories: Application pages and Site pages. There are significant differences in the processes of developing these page types.

Application Pages

Application pages are mostly similar to traditional Microsoft ASP.NET 3.5 Web pages. However, an application page is not directly derived from System.Web.UI.Page. Instead, it is derived from either LayoutsPageBase or UnsecuredLayoutsPageBase which are defined in the Microsoft.SharePoint.WebControls classes.
Application pages reside in the _layouts folder on each web front-end server and can be accessed from any site within SharePoint by using the /_layouts virtual folder (for example, https://Server/SiteCollection/Site/_layouts/MyPage.aspx). Application pages can be accessed from any web site within a SharePoint farm, making them ideal for implementing functionality that needs to be accessed from all web sites. A good example of an application page is the Settings.aspx page which is accessible from all SharePoint sites and allows users to adjust settings on different levels of the SharePoint hierarchy.  

Perhaps the most crucial differences for developers to be aware of is that application pages do not run in SharePoint safe mode, and they can contain inline code. This relaxation of security is possible because application pages can be installed only by a farm administrator.

The following image shows an example of a custom application page.

Site Pages
Site pages tend to be content-oriented, such as welcome pages, home pages, and list view pages. However, a site page can also be highly functional and contain interactive web controls and web parts. End users can create and add site pages to SharePoint Foundation web sites. For this reason, there are significant restrictions on the code that can run in site pages. Specifically, site pages run in safe mode. This means that the only controls rendered on a site page are those registered as safe controls. Strictly speaking, a site page can contain inline code, but such code will work only if the page is compiled. Some pages may be set not to compile by default. One great advantage of using SharePoint Foundation is that its site pages can be customized. If a page contains inline code, however, its code will no longer work. Because of this, we recommend against including inline code in site pages. (“Inline code” refers to embedded code that runs on the server. ECMAScript that runs on the client can be used on any kind of web page.)

There are three kinds of site pages:

Standard Pages: These wiki-enabled pages can include web controls and inline web parts. A standard page derives from the WikiEditPage class, not directly from System.Web.UI.Page. The following image shows a standard site page.

Web Parts Pages: As the name suggests, these are web parts pages with web part zones. A SharePoint Foundation web parts page derives from WebPartPage, not directly from System.Web.UI.Page. The following image shows a custom web part page.

Publishing Pages: Publishing pages are used by SharePoint Web Content management which is available only in SharePoint Server and is not discussed in this paper.

Visual Studio 2010 only supports the development of application pages. Application pages are available as project item templates and can be added to a SharePoint project. You can also create site pages outside of the SharePoint project and use Visual Studio to package and deploy them. For more information, see Creating Master Pages in Visual Studio. SharePoint Designer allows you to create lists and site pages.

Event Receiver

The Microsoft SharePoint Foundation events model relies on managed components called event receivers that respond to SharePoint Foundation objects when specific events take place.
An event receiver is a piece of managed code that responds to SharePoint Foundation events whenever specific triggering actions occur on SharePoint objects. Triggering actions include: adding items, moving items, and checking items in and out. Objects that can raise events are called event hosts and include objects like site collections, sites, lists, workflows, or features.
For example, you can add an event receiver to the item deleting event that prevents users from deleting items that meet a certain criteria. For more information about event receivers, see Introduction to Events in SharePoint Foundation. Visual Studio provides a project template for creating event receivers.

Business Data Connectivity Model

Business Data connectivity models are used to design external content types. For more information, see Working with Data in SharePoint solutions section later in this document.

Workflows

SharePoint workflows are based on the Windows Workflow foundation (WWF) and can be associated at a site or list level.

There are two types of SharePoint workflows: sequential and state machine.

Sequential Workflows: Represents a workflow as a procession of steps that execute in order until the last activity completes. However, sequential workflows are not purely sequential in their execution. Because they can receive external events and include parallel logic flows, the exact order of activity execution can vary.

State Machine workflows: Represents a set of states, transitions, and actions. One state is denoted as the start state, and then, based on an event, a transition can be made to another state. The state machine can have a final state that determines the end of the workflow.

The following table compares workflows between Visual Studio 2010 and Reusable Workflows in SharePoint Designer 2010.

Visual Studio 2010 Reusable Workflows in SharePoint Designer 2010
Can write workflows for SharePoint Foundation and SharePoint Server. Can write workflows for SharePoint Foundation and SharePoint Server.
Code-behind file enables developer to write custom Visual C# or Visual Basic .NET code to express business logic. No code-behind file; workflow rules and XOML files declaratively encapsulates business logic.
Workflow is authored as a template, which can be associated with multiple sites and lists. Creates list workflows, but also has site workflows and workflow templates as well.
The workflow markup file, or markup and code-behind files, is compiled into the workflow assembly. Workflow markup, workflow rules, and supporting file are stored, uncompiled, in a specific document library on the site.
Workflow template must be associated with each list on which it is to be available. Association occurs when the workflow is authored against the specific list.
Can use any forms technology. For example, ASP.NET forms for SharePoint workflows, or InfoPath forms for SharePoint Server workflows. Automatically generates ASP.NET forms, which you can then customize. ASP.NET forms are only generated for SharePoint Foundation. SharePoint Designer auto-generates InfoPath forms for SharePoint Server.
Can include workflow modifications. Workflow modifications are not available.
Can use custom symmetrical InfoPath forms, which enables Office client integration of custom workflow forms. For SharePoint Server, InfoPath forms are the default technology and there’s very tight integration between InfoPath and SPD.
Can author custom activities for inclusion in workflows. Must use activities provided. However, custom activities authored in Visual Studio can also be used.
Packages workflow assembly and workflow definition as a SharePoint feature and deploys it to the site. Automatically handles deployment to specific list.
Can use Initiation and Association forms to gather information from the user when starting the workflow. Can use Initiation and Association forms to gather information from the user when starting the workflow.
Can use custom forms for users to interact with SharePoint tasks. Can use custom forms for users to interact with SharePoint tasks.
Visual Studio debugging and deployment are available. No step-by-step debugging is available.
Can author both sequential and state workflows. Can author only sequential workflows.

SharePoint Infrastructure and ASP.NET Extensibility

SharePoint's infrastructure starts with Windows Server and the .NET Framework and heavily relies on IIS, ASP.NET and SQL Server.  In SharePoint Server 2010, you can configure your web sites to use either classic-mode authentication or claims-based authentication. Consider using classic-mode authentication if you are upgrading from Microsoft Office SharePoint Server (MOSS) 2007 and you have no plans to implement forms-based or SAML token-based authentication. If you ever plan to use forms-based authentication or SAML token-based authentication, claims-based authentication will be a requirement because classic-mode authentication only supports the Windows authentication provider. For more information about SharePoint 2010 authentication, see Plan authentication methods (SharePoint Server 2010).

SharePoint resource authorization can be managed at site collection, site, page, item, and custom action levels; security permissions can also be applied in custom code. Role membership can be defined individually or by using SharePoint or Active Directory groups.

ASP.NET extensibility goes further by allowing developers to add HttpHandlers and HttpModules to allow for the custom handling of requests. Custom request handling can be used to allow for features such as global changes to the SharePoint UI, custom file type processing, custom authentication, and more. For example, an HttpHandler could be created to prevent cross-site scripting errors when processing requests for file types which are stored in an external system. The HttpHandler could accept a request for the custom file type, forward the request to the external system, and then return the results to the requesting browser (Note: credentials delegation may be required). The file which resides on an external server can be requested directly from SharePoint.  The SharePoint API, including site and user context, are available from within the handler.  Because SharePoint is built upon ASP.NET, application developers have the ASP.NET customization options as well as SharePoint context and configuration available.

Another example of extensibility provided by ASP.NET includes creating custom membership providers that allow login credentials to be retrieved from a custom store. Membership providers supplied by SharePoint allow connections to a SQL database, LDAP, or Active Directory. The base MembershipProvider, like many ASP.NET and SharePoint classes, can be extended to enable developers to create a custom implementation. The membership provider is configured in the site web.config file. This enables you, for example, to supply a custom login page, among other functionality.

Claims-based authentication is a significant infrastructure feature new to SharePoint 2010. With claims, user authentication is provided by an "identity system" which may include corporate credential providers such as Active Directory, LDAP, or credentials stored in databases. Credentials authentication may also be provided by web sign-on providers such as LiveID and OpenID. Claims-based authentication makes it easier for additional identity systems to integrate with SharePoint, thus enabling another point of extensibility.

Deploying Solutions to SharePoint

Probably the hardest thing for ASP.NET developers to understand is how solution deployment works in a SharePoint environment. Depending on the SharePoint Project Item, customization and development tool SharePoint solutions are deployed in one of two locations. The first location is in the files system of the web front end servers and is called the “SharePoint root directory,” located in {Local_Drive}:\Program Files\Common Files\Microsoft Shared\web server extensions\14. The second location is the content database of the web application or site collection. For more information, see the section “SharePoint Developer tools: SharePoint Designer and Visual Studio 2010” earlier in this paper.

SharePoint artifacts such as web parts, event receivers, and so forth are packaged into a WSP file which contains one or more features. A feature is a package of SharePoint elements that can be activated for a specific scope, such as the farm, web application, site collection, or site. A feature could provide, for example, a "My Favorite Items" functionality that includes the following elements:

  • A custom list that stores, per user, a list of favorite items, which is created as a single hidden list per workspace when the feature is activated.
  • A custom menu item that is attached to all lists, called "Add to Favorites," which adds an item to the Favorites list.
  • A web part that implements usage and link tracking to display the user's top ten favorites at the top of the page. The feature itself would not contain the actual web part but rather its xml configuration file.

Each element of the feature, by itself, may not be very useful, but when you enable the feature on a site, all of the elements add up to a complete solution.

Features are stored in a subfolder containing a feature definition within the features setup directory ({Local_Drive}:\Program Files\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\FEATURES). The feature subfolder includes a Feature.xml file that defines the base properties of the feature and list elements bound to it, such as XML files containing element manifests and other supporting files. A feature subfolder may contain only a Feature.xml file, or it may contain a Feature.xml file and any number of supporting element files, such as XML files and .aspx, .htm, .xsn, .resx, .dll, and other file types.

When you create a SharePoint project in Visual Studio 2010, Visual Studio automatically creates a feature for the SharePoint artifact and sets its scope accordingly. It also creates a WSP package containing the feature for each project. A WSP package is comparable to an ASP.NET web deployment package because it contains the features, files, and settings for the feature artifact you want to deploy. For example, when you create Business Data Connectivity model, a feature is created containing the model and the project output assembly and the feature scope is set to Farm, which is the expected scope for Business Data Connectivity models. If you add a list instance to the project, Visual Studio creates a new feature for this instance and adds it to the WSP.

Trust Levels of SharePoint Solutions

When you create a SharePoint solution, you must choose a trust level for it.

Farm Solution: A farm solution is a collection of resources that you deploy through the server-side file system in your SharePoint environment. These resources execute within the same process space as the SharePoint application, which means that your code can use the full SharePoint object model and access all the same resources SharePoint does.

When you deploy a farm solution, you can choose from two different execution models: the full trust execution model and the bin folder/code access security (bin/CAS) execution model.

BIN + CAS: The bin/CAS approach is a partial trust execution model which applies only to web parts. When you use the bin/CAS execution model, you deploy your assemblies to the bin directory associated with a SharePoint Web application. The worker process associated with the SharePoint web application loads the assembly from the bin directory. However, the operations your code may perform are restricted by the code access security policies that are applied in the Web.config file to assemblies in the bin directory.

Sandboxed Solution: With sandboxed solutions, WSPs are deployed to a content database. As the name suggests, sandboxed solutions are restricted to accessing data within the same site collection. This also means that you cannot make calls to external databases or consume any kind of services from within an artifact (e.g. web part) that is deployed as a sandboxed solution. The following table shows trust levels supported by SharePoint artifacts created in Visual Studio 2010**.**

Artifact / Trust Level Sandboxed BIN + CAS Full trust
Business Data Connectivity Model No No Yes
Visual Web Part No Yes1 Yes
Web Part Yes Yes Yes
Application Page No Yes Yes
State Machine Workflow No Yes Yes
Sequential Workflow No Yes Yes
Event Receiver Yes Yes Yes
Content Type Yes Yes Yes
Module Yes Yes Yes
Site Definition No Yes Yes
Import Reusable Workflow No Yes Yes
Import SharePoint Solution Page Yes Yes Yes
List Definition from Content Type Yes Yes Yes
List Definition Yes Yes Yes
List Instance Yes Yes Yes
Empty Element Yes Yes Yes
User Control No Yes Yes

For more information about sandboxed solutions, see Developing, Deploying and Monitoring Sandboxed Solutions in SharePoint 2010.

Working with Data in SharePoint Solutions

When talking about data access in SharePoint, you need to differentiate between internal and external data. Internal data means that the data is stored in the SharePoint content database. One example of this type of data is data entered into a SharePoint list or document. External data means data that is not stored in a SharePoint controlled environment, such as HR data stored in SAP HR or some other offsite database.

Internal Data

SharePoint provides object models and services to work with internal data. Following is an overview of the different object models and services as well as recommendations for when to use each one.

**Server OM The server object model is designed for code that is running on the server in the same process as the SharePoint server. The server object model is implemented in Microsoft.Sharepoint.dll.
Client OM**
The client object model is designed for code that is running out of process from the SharePoint server, typically on a different machine.  The client object model is designed to batch up requests and actions so that it can minimize the traffic over the wire from the machine by using the client object model to communicate with the SharePoint server servicing the requests. SharePoint 2010 provides three client object models: the ECMA Script (JavaScript, JScript), .NET managed, and Silverlight client object models each provide a subset of the server object model that is defined in Microsoft.SharePoint.dll, including objects that correspond to major objects at the site-collection level or lower in the SharePoint Foundation hierarchy.**
SPLinq**
The LINQ to SharePoint Provider is defined in the Microsoft.SharePoint.Linq namespace. It translates LINQ queries into CAML queries, making it unnecessary for developers to know how to write CAML queries. LINQ queries can be used in server code. For more information on how to use SPLinq, see this MSDN article.**
Rest Interface**
SharePoint 2010 exposes list data through a REST (REpresentational State Transfer) service which you can consume in your projects. The core concept of any RESTful interface is the resource, and in SharePoint Foundation, lists and items are represented as HTTP resources that can be addressed by remote URLs. The REST interface uses flexible URL conventions for querying data, enabling you to filter, sort and select information from SharePoint Foundation lists directly through the browser. You can also retrieve information from SharePoint Foundation lists through multiple representations, including support for JavaScript Object Notation (JSON), Atom, and AtomPub standards. Standard representations for SharePoint Foundation lists makes list data available to standards-compliant clients, starting from simple Atom feed-readers, to more advanced clients that allow end users to analyze large quantities of data in Microsoft Excel. Advanced features of ADO.NET Data Services are also supported by the SharePoint Foundation REST interface, such as batching multiple operations into a single request, concurrency control through HTTP ETags, paging of large data sets, and access to document streams in SharePoint document libraries. The actual ADO.NET data service that SharePoint provides is called ListData.svc and resides in the _vti_bin directory of the site collection associated with the list with which you want to work. It can be accessed by entering http://myserver/_vti_bin/ListData.svc in the browser. See more information on the SharePoint REST interface here.

External Data

There are two major ways to integrate external data into SharePoint. The first is to use Business Connectivity Services (BCS), which is a set of services and features that provides a way to connect SharePoint solutions to sources of external data and to define external content types that are based on the external data. External content types resemble content types and enable you to present and interact with external data in SharePoint lists (known as external lists), web parts, Microsoft Outlook 2010, Microsoft SharePoint Workspace 2010, and Microsoft Word 2010 clients. External systems that Microsoft BCS can connect to include: SQL Server databases, SAP applications, web services (including Windows Communication Foundation web services), custom applications, and web sites based on SharePoint. By using Microsoft BCS, you can design and build solutions that extend SharePoint collaboration capabilities and the Office user experience to include external business data and the processes that are associated with that data. Both SharePoint Designer and Visual Studio provide tooling for creating BCS solutions. As the BDC designer in Visual Studio shares the same concept as the ADO.NET entity designer and class designer, you’ll find it provides a familiar development experience. The following table shows the major differences in both tools. For more information on SPD vs. Visual Studio see the section “SharePoint Designer vs. Visual Studio” earlier in this paper.

 

Capability / Tool Visual Studio SPD
Connect to Any data source through .Net Assembly WCF, SQL Server, .Net Assembly
Modeling Approach Create and Publish Discover and Configure
Target Scenarios - Author complex models with custom connectivity logic for aggregation, transformation, security, etc.
- Author  custom UX via VSTO customizations
- Author reusable OBPs and web parts		 Author “simple” models with OOB UX (External Lists, Outlook, Groove, InfoPath, Search, simple FK-based associations)
Limitations - Visual designer only for .Net object based models
- Separate UX development and packaging experience		 Data source needs to expose the proper interface “shape”

 

Connection Strings

There may be situations where you need to display data from databases or REST and SOAP services. Accessing data from these data sources requires either connection strings or at least the URL and endpoints of the services to be stored somewhere. Web.config files are an obvious choice but using Web.config files in SharePoint has some caveats. 
The first differentiation we have to make is whether or not we are developing a sandboxed solution. Let’s look at the farm solution first. Farm solutions have access to the file system. Because every site collection is hosted in a web application and every web application has a Web.config file, the Web.config file associated with your farm solution is one place you can store connection strings or other data. The method for doing this is similar to how it is done in ASP.NET development.
The Web.config files of SharePoint web applications are also stored in C:\inetpub\wwwroot\. If you open the folder wwwroot , you’ll see a folder called WSS. The WSS folder has a subfolder called VirtualDirectories which  contains all the SharePoint web applications in the farm. If there are only the default web applications, you should see two folders: one named 80 and one with a different number. The folder 80 is the one we are after as this one represents the web application which is created when running the SharePoint configuration wizard with the default settings. The other folder represents the SharePoint Central Admin site. Note: If there are additional web applications to host other site collections, you must open the folder with the name matching the port number of your web application.
If you open the folder 80 you can find its Web.config file. To add a connection string to the Web.config file, you can simply add a connectionStrings section between the closing (</SharePoint>) and opening ( <system.web>) elements. The code below shows a sample connectionStrings section. The name of the connection string is MyConnectionString and the connectionString attribute stores the actual connection string.

</SharePoint>
  <connectionStrings>
  <add name="MyConnectionString" connectionString="Data
       Source=serverName;Initial 
       Catalog=DemoDB;Persist 
       Security Info=True;User ID=userName;Password=password;"/>
  </connectionStrings >
<system.web>

For readability purposes, you can see the password for the connection string in the sample, but in production environments you should always encrypt passwords before storing them in files. For more information, see MSDN for encrypting connection information at: https://msdn.microsoft.com/en-us/library/ms178372.aspx.
The connection string can be read during runtime, such as in a web part, by using the WebConfigurationManager object available in the System.Web.Configuartion namespace and use its ConnectionStrings collection to access the element containing the connection string. This is similar to how you access connection strings in ASP.NET applications. The following line shows you how to access the connection string in code.

In C#

string connectionString = WebConfigurationManager.
 ConnectionStrings["MyConnectionString"].ConnectionString;

In VB:

Dim connectionString As String = WebConfigurationManager.
 ConnectionStrings("MyConnectionString").ConnectionString

The only problem with this approach is that you need to manually add the connection string element to the connectionStrings section in the Web.config file. This also needs to be done in the Web.config files of the target web applications on the destination servers.
Another approach for adding connection strings to the connectionStrings section is to use the SPWebConfigModification class in a feature event receiver to update the SharePoint Web.config file automatically. Using a feature event receiver has the advantage that the connection strings are added to the Web.config files on the destination servers automatically when the feature is activated (or removed when the feature is deactivated). For more information on feature event receivers see: https://msdn.microsoft.com/en-us/library/ms469501.aspx. The following sample shows code that adds a connection string to the connectionStrings section in the Web.config when the feature is activated.

In C#:

public override void FeatureActivated(SPFeatureReceiverProperties properties)
{
  SPWebService myService = SPWebService.ContentService;
  SPWebConfigModification myModification = 
                      new SPWebConfigModification();
  myModification.Path = "configuration/connectionStrings";
  myModification.Name = "add[@name='MyConnectionString']" +
                 "[@connectionString='DataSource=serverName;" + 
                 "Initial Catalog=DemoDB;" + 
                 "Persist Security Info=True;" +
                 "User ID=userName;Password=password;']";
  myModification.Sequence = 0;
  myModification.Owner = "UniqueOwner";
  myModification.Type = SPWebConfigModification.
                        SPWebConfigModificationType.EnsureChildNode;
  myModification.Value = "<add name='MyConnectionString'" + 
                 "connectionString='DataSource=serverName;" + 
                 "Initial Catalog=DemoDB;" + 
                 "Persist Security Info=True;" + 
                 "User ID=userName;Password=password;'/>";
  myService.WebConfigModifications.Add(myModification);
  myService.Update();
  myService.ApplyWebConfigModifications();
}

In VB:

Public Overrides Sub FeatureActivated(ByVal properties As SPFeatureReceiverProperties)
  Dim myService As SPWebService = SPWebService.ContentService
  Dim myModification As New SPWebConfigModification()
  myModification.Path = "configuration/connectionStrings"
  myModification.Name = "add[@name='MyConnectionString']" + _
                        "[@connectionString='DataSource=serverName;" + _
                        "Initial Catalog=DemoDB;" + _
                        "Persist Security Info=True;" + _
                        "User ID=userName;Password=password;']"
  myModification.Sequence = 0
  myModification.Owner = "UniqueOwner"
  myModification.Type =  SPWebConfigModification.SPWebConfigModificationType.EnsureChildNode
  myModification.Value = "<add name='MyConectionString'" + _
                        "connectionString='DataSrource=serverName;" + _
                        "Initial Catalog=DemoDB;" + _
                        "Persist Security Info=True;" + _
                        "User ID=userName; Password=password;'/>"
  myService.WebConfigModifications.Add(myModification)
  myService.Update()
  myService.ApplyWebConfigModifications()
End Sub

To access the Web.config file and make updates you must get a reference to the web application first. Because the modifications should apply to all web applications in a farm, we create an object of SPWebService rather than accessing the farm instance through SPWebApplication. SPWebService serves as a container for all the web applications in a SharePoint farm. After that we create the SPWebConfigModification object. The Path property sets an XPath expression that is used to locate the node that is being modified or created, which is in our case the connectionStrings section node. The Name property sets the name of the attribute or section node to be modified or created which in this case is the attributes name and connectionString. The Sequence property is set to 0 as we have only one notification. If there were more, we would need to increment the sequence number. The Owner property needs a unique string which we set in our sample to “UniqueOwner.” The modification type is set to EnsureChildNode which basically makes sure that there is a child node. Other values for the modification type are EnsureSection and EnsureAttribute. When creating new section you should use the EnsureChildNode modification type value instead of the EnsureSection. EnsureChildNode allows us to remove the section programmatically if needed. The value property contains the actual value of our connection string. Next, we need add the modification object to the WebConfigModifications collection of the service object and call the Update() method on the service object. The Update() method causes this web service to save its state and to propagate changes to all the machines in the farm. Finally, we call the ApplyWebConfigModifications method on the service object which applies the list of Web.config modifications to all web applications in this web service across the farm. 
Can the same techniques be used for sandboxed solutions? No, they cannot. As mentioned earlier, sandboxed solutions do not have access to the Web.config files. To be honest, there is not even a need to store connection information in sandboxed solution because they prevent us from writing code that references services or accesses databases anyway.

Debugging and troubleshooting SharePoint Solutions

Another big challenge for ASP.NET developers is how to debug and troubleshoot SharePoint applications. Visual Studio 2010 makes the process of debugging SharePoint applications as simple as debugging ASP.NET web applications. However, due to the SharePoint architecture, it is not always possible to find potential issues through debugging.  This section lists useful information to help troubleshoot SharePoint applications.

Debugging

As mentioned earlier, debugging SharePoint applications isn’t much different than debugging ASP.NET web pages and user or web controls. To debug a SharePoint project in Visual Studio simply set a breakpoint in the source editor and press F5. The following figure shows the code editor with a breakpoint set in the Page_Load event of a SharePoint application page.

Pressing F5 instructs the project system to build the assembly, package the feature, and deploy the solution to the SharePoint site. In addition, the debugger is attached and the browser launched to display the SharePoint site. When the browser launches, you're taken to the home page of the site you specified during project creation and you can step through the code with the familiar Visual Studio debugging commands.

Due to the sometimes complex interactions between the Visual Studio process (devenv.exe), the Visual Studio SharePoint host process (vssphost4.exe), SharePoint, and the WCF layer, diagnosing errors that occur while building, deploying, and so forth can be difficult. To help you resolve such errors, you can enable enhanced debugging information. To do this, go to the following registry key in the Windows registry:

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\10.0\SharePointTools]

Change the "EnableDiagnostics" key value from "0" to "1."
Setting this key value to 1 causes stack trace information to appear in the Output window whenever project system errors occur while you are running in Visual Studio. To disable enhanced debugging information, set EnableDiagnostics back to 0.
The image below shows the output window after an error has occurred:

Diagnostics Log files

Sometimes neither debugging nor enabling enhanced diagnostics provides you with enough details why something doesn’t work as expected or fails. In that case, the best option for finding more information to help to narrow down the problem is to search the SharePoint log files. The log files are provided by the Unified Logging Service (ULS). The service writes events to SharePoint’s Trace Logs, which are stored in the file system in {local drive}:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\LOGS. The ULS service can be configured by SharePoint administrators and developers can write to the log files.

Developer Dashboard

Another useful diagnostics tool in SharePoint 2010 is the Developer Dashboard. The Developer Dashboard can help diagnose a class of bugs which are easy to introduce through custom code. A good example is a web part that retrieves a large amount of data in its load event and, thus, makes the page less responsive. Developer Dashboard contains an extensible mechanism for measuring various performance counters at various scopes. Within Developer Dashboard, the following performance counters are used to monitor usage and resource consumption at each stage of the requests.

Per-Thread Counters (these measure actually values for the current request or timer job):

  • Execution time
  • SPRequests Allocated
  • Number of SQL Queries performed
  • The duration of each SQL query
  • The query text of each SQL query
  • URL (or timer job name)
  • Current User
  • Start time

Per-Process Counters

  • Global heap size
  • Total size of all native heaps
  • Number of native heaps
  • Number of active native heaps
  • Number of SharePoint operations in progress

Client Counters

  • Page Size

System Counters (these include any System Performance Counter, these are just examples)

  • Managed memory heap size
  • Hard page faults
  • Processor utilization
  • Available memory on the machine

Note:*** Any values that exceed acceptable ranges will be shown in red.*

Developers can monitor any piece of code by wrapping it in the SPMonitoredScope or even create custom monitors for your own resources through implementing ISPScopedPerformanceMonitor and adding the monitor to the SPMonitoredScope. To use the Developer Dashboard, it must be enabled. There are three modes. The first mode is the default setting which is Off. On mode displays the Developer Dashboard in every page rendered. Finally, OnDemand mode enables Developer Dashboard but hides it until a user manually activates it by selecting the DevDashboard icon.

There are three ways to change the mode of the Developer Dashboard. You can use stsadm, SharePoint 2010 Management Shell, or the SharePoint object model. Stsadm is a tool for command-line administration of SharePoint servers and sites. Stsadm is located at the following path on the drive where SharePoint Products and Technologies is installed: %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\14\bin.

STSADM

‘On’ Mode
STSADM –o setproperty –pn developer-dashboard –pv On
‘Off’ Mode
STSADM –o setproperty –pn developer-dashboard –pv Off
‘OnDemand’ Mode
STSADM –o setproperty –pn developer-dashboard –pv OnDemand
To display the developer dashboard only if one or more counters (acceptable values) are exceeded, there’s a way to do that too by running:

STSADM –o setproperty –pn developer-dashboard –pv expensiveoperationsonly

SharePoint 2010 Management Shell

Add the following script to a notepad file and name DeveloperDashboard.ps1 and run the file inside the SharePoint 2010 Management Shell (All Programms->Microsoft SharePoint 2010 Products). Note the display level is set to OnDemand, you need to change this if you want another mode.

$DevDashboardSettings = [Microsoft.SharePoint.Administration.SPWebService]::ContentService.DeveloperDashboardSettings; 
$DevDashboardSettings.DisplayLevel = 'OnDemand'; 
$DevDashboardSettings.RequiredPermissions = 'EmptyMask'; 
$DevDashboardSettings.TraceEnabled = $true; 
$DevDashboardsettings.Update()

SharePoint Object model

The Developer Dashboard can also be enabled through the SharePoint object model. The code below shows how to enable the dashboard and set its mode to OnDemand.

SPWebService cs = SPWebService.ContentService;
cs.DeveloperDashboardSettings.DisplayLevel = SPDeveloperDashboardLevel.OnDemand;
cs.DeveloperDashboardSettings.Update();

The OnDemand option adds the icon shown below to the SharePoint sites.

Clicking the icon displays the Developer Dashboard at the bottom of the page as shown in the following image.

Building SharePoint Master Pages with Visual Studio

SharePoint master pages are based on ASP.NET master pages, but there are differences when it comes to creating them. All of the available master pages are stored in the SharePoint Master Page Gallery, which is accessed by entering https://localhost/_catalogs/masterpage/Forms/AllItems.aspx in a browser. The default.master file is used mainly for backward compatibility of SharePoint. SharePoint 2010 has a compatibility mode which was added to give SharePoint users time to adapt to the new SharePoint user interface during a migration. The minimal.master file is a master page with the minimal requirements for a SharePoint master page.  V4.master is the primary master page for SharePoint Foundation.

Visual Studio doesn’t provide a template for SharePoint master pages and it doesn’t let you add an ASP.NET master page either. The workaround is to create an ASP.NET master page file outside of a Visual Studio SharePoint project. Once the file is created in Visual Studio, you can add the markup. You can copy the markup code from an existing master page such as the v4.master and customize it to meet your needs. You can also use one of the master page templates provided by the SharePoint development community, such as the SharePoint 2010 Starter Master pages on CodePlex http://startermasterpages.codeplex.com/ and customize it. Those two approaches work great and require only an understanding of ASP.NET to identify what markup code is responsible for which functionality.  As mentioned previously, SharePoint includes a master page, minimal.master, that provides minimal functionality. This can be used as a good starting point for creating your own master page. MSDN has a great list of placeholders and controls used in the v4.master that you can use as reference material when creating a master page from scratch. For more information, see Upgrading an Existing Master Page to the SharePoint Foundation Master Page.

Once you have customized the new master page by adding the markup to it, you can use Visual Studio 2010 to deploy the master page by using a module. Modules can be added to SharePoint projects through the Add New Item dialog. Once the module is added, you need to add the master page as an existing item and change its elements xml file. The following is a sample elements file which deploys the master page file MyMaster to the master page gallery.

Elements.XML file for the Module deploying the master page file

<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="https://schemas.microsoft.com/sharepoint/">
  <Module Name="MasterPage" Url="_catalogs/masterpage">
  <File Path="MasterPage\MyMaster.master"
        Url="CustomMinimal.master" 
        Type="GhostableInLibrary" />
</Module>
</Elements>

Upgrading SharePoint Solutions

In ASP.NET, you use XCopy or Web Deployment Packages to deploy the latest version of ASP.NET files and assemblies to the destination servers. As mentioned previously, SharePoint uses features and WSP packages to deploy artifacts to SharePoint. Let’s paint a picture to better explain the differences in an ASP.NET solution upgrade vs. a SharePoint solution upgrade.

Deploying a simple web part to a SharePoint site

A web part that shows the weather in Seattle is already deployed and used by a SharePoint web site. The web part was packaged and deployed through a WSP. A developer extends the web part to enable users to choose a location and see the weather data based on their selection, and then repackages the web part. One way to update the web part on SharePoint is to deactivate the feature containing the web part on the SharePoint site and then retract the WSP that contains the feature. Then you can redeploy the WSP with the updated web part and reactivate the feature. Visual Studio 2010 does all these steps for you in the development environment. The WSP package created by Visual Studio can be copied to a staging environment and then imported by using the Admin UI, PowerShell, or stsadm.

Extending an existing feature

A developer has built an orientation application for new team members. The application consists of a web part, where team managers can enter the personal information of the new team members, and a couple of workflows that execute based on the information entered. For example, if the new team member is a developer, a workflow is started to automatically order necessary hardware for the developer. The orientation solution is deployed as Version 1. Now, the developer is asked to add a list to the solution to display all of the training courses the new team member must take.  Rather than retracting the entire solution, you can define upgrade actions at a feature level. In this case, the upgrade action adds the training list to the orientation solution at upgrade time. For more information, see Feature Upgrade Overview.

Additional Resources:

Glide Path for ASP.NET Developers on MSDN

SharePoint 2010: Developer Platform White Paper

Understanding and Creating Customized and Uncustomized Files

SharePoint Development with Visual Studio 2010

SharePoint Developer Center on MSDN

Authors:

Boris Scholl – Program Manager, Visual Studio Office Tools

Reza Chitsaz – Lead Program Manager, Visual Studio Office Tools

Contributors:

Saaid Khan – Program Manager, Visual Studio Office Tools

Reviewers:

Kemp Brown – Programming Writer, Visual Studio Developer Guidance