Moving External Data into Microsoft CRM: Lead Generation
Peter Hecke
Microsoft Corporation
September 2004
Applies to:
Microsoft CRM version 1.2
Requires:
Microsoft Business Solutions CRM version 1.2
Microsoft Visual Studio .NET 2003
Summary: This article describes how to collect customer provided information from a public (Internet) Web server and pass the data through a corporate firewall into the Microsoft CRM database. The customer information is inserted into the Microsoft CRM database as a new lead object. (17 printed pages)
Download the sample code for this article: LeadGeneration.exe.
Contents
Introduction
The Big Picture
The Contact Us Form
The Lead Generation Tool
Security Concerns
Additional Information
Introduction
The scenario demonstrated in this article revolves around a Contact Us Web form on a public Web server designed to collect potential lead information from customers. Because the server hosting this form is outside the corporate firewall, it cannot directly access the Microsoft CRM server. The solution discussed in this article uses a small program running as a scheduled task on a server behind the firewall to pull the customer data from the public server. The program uses the Microsoft CRM Software Development Kit (SDK) to create a lead in Microsoft CRM for each contact.
The Big Picture
The sample code provided in this article contains two Microsoft Visual Studio .NET solutions. The first solution is the public ContactUs Web application, which stores the customer data it collects in an XML file named leads.xml on a public Web server outside the corporate firewall. The second solution is the LeadGenerator program, which is designed to be run as a scheduled task on the back end server. The program opens the XML file on the public Web server using file sharing on Microsoft Windows platforms, and then creates a lead in Microsoft CRM for each new lead in the XML file by calling the CRMLead.Create method. The LeadGenerator program also maintains a log file stored on the backend server to keep a record of when it ran and how many leads were created, and to log any problems it might have encountered while processing the XML file.
The whole process is illustrated in Figure 1.
.gif)
Figure 1. The process of collecting lead information from customers.
The Contact Us Form
The Contact Us form contains a fictitious company logo and five TextBox controls: contact first and last name, company name, e-mail address, and phone. Below the TextBox controls is a Submit button. To use the form, enter appropriate values into all TextBox controls and then click the Submit button.
.gif)
Figure 2. The Contact Us Form.
Installing the Solution
Install the Visual Studio. NET solution files by following these steps:
Run the LeadGeneration.exe program. After extracting the files, copy or move the LeadGeneration\ContactUs folder to the C:\Inetpub\wwwroot folder of the target public Web server that is running Internet Information Services (IIS).
Create a virtual directory that points to the ContactUs folder.
- Right-click the ContactUs folder in Windows Explorer and then click Sharing and Security.
- On the Web Sharing tab of the Properties dialog box, select the appropriate Web site from the Share on drop-down list.
- Select Share this folder. Click OK to close the dialog box.
Create a folder named C:\leads where the leads.xml file will be stored.
Open the project in Visual Studio. NET by double-clicking the solution file ContactUs.sln that is in the ContactUs folder.
Note If the Web Access Failed dialog box is displayed, select the option Retry using a different file share path. Type C:\Inetpub\wwwroot\ContactUs into the Location field and click OK.
Build the solution.
Code Analysis
The ContactUs form solution includes two primary ASP.NET files. One file is named default.aspx and contains the HTML code for the Web form. The other file is the C# code-behind page, which is named default.aspx.cs.
The HTML Code
The HTML code in the default.aspx file contains several notable items. The form contains a hidden label named thankYouText. The event handler code for the Submit button uses this label to display either a thank you message, or an error message to the user.
<asp:label id="thankYouText" runat="server" visible="False">Thank you!
</asp:label>
All the form input fields of type asp:textbox have two ASP.NET validator controls associated with them. The asp:requiredfieldvalidator control checks whether the user has entered any value into the associated field. The other validator control is of type asp:regularexpressionvalidator, which uses a regular expression to validate that the user's input is limited to the letters, numbers, and punctuation that is appropriate for that field. The following are some code snippets from the default.aspx file, which highlight the regular expression usage. The HTML comments describe the scope of a valid value for a field. The validationexpression property is set to the regular expression pattern that defines the scope of permitted values.
<asp:textbox id="firstName" runat="server" maxlength="50"></asp:textbox>
<%-- This regular expression validator requires the firstName text box to
contain 1 through 50 characters, which must all be either alphabetical
characters or the apostrophe. --%>
<asp:regularexpressionvalidator id="firstNameValidator" runat="server"
errormessage="This field may contain alphabetical characters only."
validationexpression="^[A-Za-z']{1,50}$" controltovalidate="firstName">
</asp:regularexpressionvalidator>
<asp:textbox id="email" runat="server" maxlength="50"></asp:textbox>
<%-- This regular expression validator requires the e-mail text box to
contain only valid characters and be of the form "a@b.c" --%>
<asp:regularexpressionvalidator id="emailValidator" runat="server"
errormessage="This field may contain only valid e-mail address
characters." validationexpression="^[\w\.\-]+@[a-zA-Z0-9\-]+(\.[a-zA-Z0
-9\-]{1,})*(\.[a-zA-Z]{2,3}){1,2}$" controltovalidate="email">
</asp:regularexpressionvalidator>
<asp:textbox id="phoneNumber" runat="server" maxlength="50"></asp:textbox>
<%-- This regular expression validator requires the phoneNumber text box
to contain only text in the form "(xxx) xxx-xxxx" where each "x" is a
numeric character. --%>
<asp:regularexpressionvalidator id="phoneNumberValidator" runat="server"
errormessage="Phone numbers must be of the form (212) 555-0123."
validationexpression="^\s*\(\d{3}\)\s*\d{3}\s*-\s*\d{4}\s*$"
controltovalidate="phoneNumber"> </asp:regularexpressionvalidator>
For additional information about regular expression usage, refer to the .NET Framework Developer's Guide in the MSDN library.
The Code-Behind Class
The following shows the partial C# code for the ContactUs class. Comments in the code describe the purpose of each code section.
namespace ContactUs
{
public class ContactForm : System.Web.UI.Page
{
// Path to the XML output file where the leads will be stored.
// Note that this file must be stored on a file share so that it
// can be accessed by the backend server which runs the
// LeadGenerator.
private const String LeadsFilePath = @"c:\leads\leads.xml";
private System.Text.UnicodeEncoding encoding = new
UnicodeEncoding();
#region Web Form Designer generated code
private void Page_Load(object sender, System.EventArgs e) { }
// Event Handler for the Submit button. Adds the new lead
// information to the leads XML file.
private void SubmitLead(object sender, System.EventArgs e)
{
// Check whether all the TextBox controls on the page contain
// valid inputs.
if (!Page.IsValid)
{
return;
}
FileStream leadsFile = null;
// Store the closing </leads> root tag as an array of bytes. The
// array will be used to seek back a byte offset from the end of
// the XML file.
Byte[] endTagBytes = this.encoding.GetBytes("</leads>");
try
{
bool exists = File.Exists(ContactForm.LeadsFilePath);
// Open the leads file. If the file does not exist, a new
// one will be created.
leadsFile = File.OpenWrite(ContactForm.LeadsFilePath);
// If the file does not exist, write the XML header
// and opening <leads> root tag.
// The preamble is required so that Internet Explorer can be
// used to view the XML file.
if (!exists)
{
leadsFile.Write(this.encoding.GetPreamble(), 0,
this.encoding.GetPreamble().Length);
Byte[] headerBytes = this.encoding.GetBytes("<?xml
version=\"1.0\" encoding=\"utf-16\"?>\r\n<leads>\r\n");
leadsFile.Write(headerBytes, 0, headerBytes.Length);
}
// If the file does exist, move the cursor to just
// before the closing </leads> root tag so that you can
// append the new lead. The existing </leads> closing
// tag will be overwritten.
else
{
leadsFile.Seek(-endTagBytes.Length, SeekOrigin.End);
}
// Create an XML representation of the new lead using the
// data from the TextBox controls.
StringBuilder lead = new StringBuilder("\t<lead>\r\n");
lead.AppendFormat("\t\t<firstname>{0}</firstname>\r\n",
this.firstName.Text);
lead.AppendFormat("\t\t<lastname>{0}</lastname>\r\n",
this.lastName.Text);
lead.AppendFormat("\t\t<companyname>{0}
</companyname>\r\n",this.companyName.Text);
lead.AppendFormat("\t\t<emailaddress1>{0}
</emailaddress1>\r\n", this.email.Text);
lead.AppendFormat("\t\t<telephone1>{0}</telephone1>\r\n",
this.phoneNumber.Text);
lead.AppendFormat("\t\t<subject>New Lead from the Contact
Us Web form</subject>\r\n");
lead.Append("\t</lead>\r\n");
// Write the new lead to the XML file.
Byte[] leadBytes =
this.encoding.GetBytes(lead.ToString());
leadsFile.Write(leadBytes, 0, leadBytes.Length);
// Write the closing XML tag.
leadsFile.Write(endTagBytes, 0, endTagBytes.Length);
}
catch (Exception ex)
{
this.thankYouText.Text = "There was an error submitting
the form. Please try again.";
}
finally
{
if (leadsFile != null)
{
leadsFile.Flush();
leadsFile.Close();
}
}
// Hide the form from the user.
this.formPanel.Visible = false;
this.formText.Visible = false;
// Display the thankYouText label which contains either the
// default thank you message (defined in default.aspx) or
// error text from a caught exception (see the catch block).
this.thankYouText.Visible = true;
}
}
}
The following is an example of what a lead might look like in the leads.xml file:
<lead>
<firstname>John</firstname>
<lastname>Smith</lastname>
<companyname>Adventure Works</companyname>
<emailaddress1>john@adventure-works.com</emailaddress1>
<telephone1>(425) 555-0100</telephone1>
<subject>New Lead from the Contact Us Web form</subject>
</lead>
The Lead Generation Tool
The LeadGenerator program is a command-line tool designed to be run as a scheduled task. When run, the tool moves the leads.xml file to the current folder on the local computer. Keeping a local copy of the XML file is useful for troubleshooting problems such as if the LeadGenerator was not able to successfully read the original leads.xml file. The tool then inserts each new lead from the XML file into Microsoft CRM. A log file is also written to the current folder. The log file contains a history of the successes and failures of processing the XML file by the lead generation tool.
Installing the Solution
To install the LeadGenerator solution, follow these steps:
Copy or move the folder "LeadGeneration\LeadGenerator" to a folder on a Microsoft CRM server inside your firewall.
Open the project in Visual Studio .NET by double-clicking the solution file LeadGenerator.sln that is in the LeadGenerator folder.
Press Ctrl+Shift+B to build the solution.
Note The project contains references to the two Microsoft CRM DLLs: Microsoft.Crm.Platform.Proxy.dll and Microsoft.Crm.Platform.Types.dll.
If you receive a build error because Visual Studio .NET cannot find these DLLs, you must remove the references to the DLLs in the Solution Explorer and add new references that point to the location where these DLLs are installed on your system. By default, they are located in the C:\inetpub\wwwroot\bin folder.
Command Line Syntax and Testing
The LeadGenerator tool accepts the name of the Microsoft CRM server and the fully-qualified path of the leads.xml file as arguments. Both arguments are required.
**LeadGenerator.exe **crmServerName leadsFilePath
Test the Web form
- Point the Internet Explorer Web browser to the URL of the Web form (https://<servername>/ContactUs/).
- Fill out the Contact Us form and click the Submit button.
- View the C:\leads\leads.xml file in a text editor or Internet Explorer to verify that the file contains the same data entered into the form.
Test the LeadGenerator tool
Run the LeadGenerator from a command prompt window using the command line parameters described previously. For example,
C:\LeadGenerator\bin\LeadGenerator.exe myCrmServer z:\leads\leads.xmlNote In this example, the drive C on the public Web server has been mapped as a network drive (z:) on the back end server so that the LeadGenerator can access the leads.xml file.
Verify that no errors are reported in the log file.
Run the Microsoft CRM application and verify that the new lead has been created in the database.
Scheduling the Tool
You should run the LeadGenerator tool periodically to convert all the contacts collected in the XML file to leads in Microsoft CRM.
To schedule the LeadGenerator tool to run nightly, follow these steps:
In Control Panel, open Scheduled Tasks, and then double-click Add Scheduled Task. The Scheduled Task Wizard is displayed.
Click Next and then Browse, and select C:\LeadGenerator\bin\LeadGenerator.exe. Click Open.
Select the Daily option, and then click Next.
Specify a start time. Select the Every Day option and then click Next.
Enter the logon information of a valid user in Microsoft CRM. This user must have rights to generate leads in the Microsoft CRM system. Click Next.
Select the Open Advanced Properties check box, and then click Finish.
In the Properties dialog box, the path of the LeadGenerator executable is displayed in the Run text box. Append the name of your Microsoft CRM server and the path of the leads.xml file to the contents of the Run text box. For example,
C:\LeadGenerator\bin\LeadGenerator.exe myCrmServer z:\leads\leads.xml
Code Analysis
The following shows the C# source code for the LeadGenerator. Comments in the code describe the purpose of each code section.
// Generator.cs
namespace LeadGenerator
{
using System;
using System.IO;
using System.Net;
using System.Xml;
using System.Text.RegularExpressions;
using Microsoft.Crm.Platform.Proxy;
/// <summary>
/// Class to generate new leads in Microsoft CRM based on an XML file
/// on a remote server
/// </summary>
public class Generator
{
private const Int32 MaxLength = 1000000000;
/// <summary>
/// The main entry point for the application
/// </summary>
/// <param name="args">Parameter 1 is the Microsoft CRM server to
/// use. Parameter 2 is the path of the leads.xml file to
/// process.</param>
[STAThread]
public static void Main(string[] args)
{
// Print usage statement if not called with the correct
// parameters.
if (args.Length != 2)
{
System.Console.WriteLine("usage:\nleadGenerator.exe
[MSCRM server] [path of leads file]");
Environment.Exit(1);
}
Generator g = new Generator();
g.Process(String.Concat("https://", args[0],
"/mscrmservices/"), args[1]);
}
/// <summary>
/// Open the leads.xml file from the remote computer, and insert
/// each new lead listed into Microsoft CRM.
/// </summary>
/// <param name="crmUrl">URL of the MSCRMServices folder on the
/// Microsoft CRM server.</param>
/// <param name="leadsFile"> Path Of the leads file to
/// be processed.</param>
private void Process(String crmUrl, String leadsFile) {
Log log = new Log(String.Concat(Environment.CurrentDirectory,
@"\log.txt"));
string leadsXml = String.Empty;
XmlDocument leads = new XmlDocument();
try
{
// Check that leads file is within the size limit.
FileInfo fileInfo = new FileInfo(leadsFile);
if (fileInfo.Length > Generator.MaxLength)
{
throw new Exception("File larger than maximum allowed
size.");
}
// Read the XML file into an XML document.
StreamReader sr = new StreamReader(leadsFile);
leadsXml = sr.ReadToEnd();
sr.Close();
leads.LoadXml(leadsXml);
// Move the XML file locally, prepending its name with
// today's date.
String localFile =
String.Concat(Environment.CurrentDirectory, @"\",
DateTime.Today.ToString("yyyy-MM-dd"),
"leads.xml");
File.Move(leadsFile, localFile);
}
catch (Exception e)
{
log.Write(e);
log.Close();
System.Environment.Exit(1);
}
if (String.Empty.Equals(leadsXml.Trim()))
{
log.Write("No leads to process.");
}
else
{
// Create Microsoft CRM objects.
BizUser bizUser = new BizUser();
bizUser.Credentials = CredentialCache.DefaultCredentials;
bizUser.Url = String.Concat(crmUrl, "BizUser.srf");
CRMLead crmLead = new CRMLead();
crmLead.Credentials = bizUser.Credentials;
crmLead.Url = String.Concat(crmUrl, "CRMLead.srf");
// Authenticate with Microsoft CRM.
CUserAuth crmUserAuth = null;
try
{
crmUserAuth = bizUser.WhoAmI();
}
catch (Exception e)
{
log.Write(e);
log.Close();
Environment.Exit(1);
}
// Create an XML node containing the owning user for the
// leads.
XmlElement ownerId = leads.CreateElement("ownerid");
ownerId.SetAttribute("type",
Microsoft.Crm.Platform.Types.ObjectType.otSystemUser.ToString());
ownerId.InnerText = crmUserAuth.UserId;
// Insert each new lead into Microsoft CRM.
int successCount = 0;
XmlNodeList leadNodes = leads.SelectNodes("/leads/lead");
foreach (XmlNode lead in leadNodes) {
try
{
lead.AppendChild(ownerId);
crmLead.Create(crmUserAuth, lead.OuterXml);
successCount++;
}
catch (Exception e)
{
log.Write(e);
log.Write(String.Concat("lead that caused the
exception: ", lead.OuterXml));
}
}
log.Write(String.Format("Successfully inserted {0} of {1}
new leads.", successCount, leadNodes.Count));
log.Close();
}
}
}
}
The following shows the C# source code for the Log class. The Log class manages the creation and writing of a textual log file. You can also use the System.Diagnostics.EventLog class to achieve the same purpose.
// Log.cs
namespace LeadGenerator
{
using System;
using System.IO;
/// <summary>
/// Reusable logging class. It uses lazy initialization for opening
/// the log file.
/// </summary>
public class Log
{
private String logPath;
private StreamWriter logStream = null;
/// <summary>
/// Create a new instance.
/// </summary>
/// <param name="logPath">Path of file to log to</param>
public Log(String logPath)
{
this.logPath = logPath;
}
/// <summary>
/// Record an exception in the log file.
/// </summary>
/// <param name="e">Exception to log</param>
public void Write(Exception e)
{
this.Write(e.ToString());
}
/// <summary>
/// Record a message in the log file.
/// </summary>
/// <param name="s">String to log</param>
public void Write(String s)
{
if (this.logStream == null)
{
this.StartLogging();
}
this.logStream.WriteLine(s);
}
/// <summary>
/// Close the underlying log file.
/// </summary>
public void Close()
{
this.Write("--");
this.logStream.Flush();
this.logStream.Close();
this.logStream = null;
}
/// <summary>
/// Open the log file.
/// </summary>
private void StartLogging()
{
this.logStream = File.AppendText(this.logPath);
this.logStream.WriteLine("Logging Started: {0}",
DateTime.Now.ToString("F"));
}
}
}
Note The log file will grow over time and should be periodically purged.
Security Concerns
The file share containing the leads.xml file should deny permission to everyone other than the ASP.NET account and the account used to map the Web server drive to the internal server.
As a security precaution to hinder Denial of Service (DoS) attacks, the leads.xml file will not be processed if it is larger than one megabyte. You can tune this parameter with the MaxLength member variable in the Generator class of the LeadGenerator tool.
The leads.xml file is stored behind the firewall to restrict public access to the lead data.
To hinder DoS attacks, a FileStream object instead of an XmlDocument object is used in the ContactUs class to write leads.xml. The FileStream object does not need to read the whole document into memory, whereas XmlDocument does. Because the LeadGenerator runs on a server behind the firewall, the problem of DoS attacks is reduced.
Additional Information
For more information about customizing the Microsoft CRM application, see the Microsoft CRM Software Development Kit (SDK).
For more information about Windows file sharing, see Help and Support in the Start menu. Search or browse for these topics:
- Share a drive or folder on the network.
- Assign a drive letter to a network computer or folder.