Tutorial: Creating a Type Provider (F#)
The type provider mechanism in F# 3.0 is a significant part of its support for information rich programming. This tutorial explains how to create your own type providers by walking you through the development of several simple type providers to illustrate the basic concepts. For more information about the type provider mechanism in F#, see Type Providers.
F# 3.0 contains several built-in type providers for commonly used Internet and enterprise data services. These type providers give simple and regular access to SQL relational databases and network-based OData and WSDL services. These providers also support the use of F# LINQ queries against these data sources.
Where necessary, you can create custom type providers, or you can reference type providers that others have created. For example, your organization could have a data service that provides a large and growing number of named data sets, each with its own stable data schema. You can create a type provider that reads the schemas and presents the current data sets to the programmer in a strongly typed way.
Before You Start
The type provider mechanism is primarily designed for injecting stable data and service information spaces into the F# programming experience.
This mechanism isn’t designed for injecting information spaces whose schema changes during program execution in ways that are relevant to program logic. Also, the mechanism isn't designed for intra-language meta-programming, even though that domain contains some valid uses. You should use this mechanism only where necessary and where the development of a type provider yields very high value.
You should avoid writing a type provider where a schema isn't available. Likewise, you should avoid writing a type provider where an ordinary (or even an existing) .NET library would suffice.
Before you start, you might ask the following questions:
Do you have a schema for your information source? If so, what’s the mapping into the F# and .NET type system?
Can you use an existing (dynamically typed) API as a starting point for your implementation?
Will you and your organization have enough uses of the type provider to make writing it worthwhile? Would a normal .NET library meet your needs?
How much will your schema change?
Will it change during coding?
Will it change between coding sessions?
Will it change during program execution?
Type providers are best suited to situations where the schema is stable at runtime and during the lifetime of compiled code.
A Simple Type Provider
This sample is Samples.HelloWorldTypeProvider in the SampleProviders\Providers directory of the F# 3.0 Sample Pack on the Codeplex website. The provider makes available a "type space" that contains 100 erased types, as the following code shows by using F# signature syntax and omitting the details for all except Type1. For more information about erased types, see Details About Erased Provided Types later in this topic.
namespace Samples.HelloWorldTypeProvider
type Type1 =
/// This is a static property.
static member StaticProperty : string
/// This constructor takes no arguments.
new : unit -> Type1
/// This constructor takes one argument.
new : data:string -> Type1
/// This is an instance property.
member InstanceProperty : int
/// This is an instance method.
member InstanceMethod : x:int -> char
/// This is an instance property.
nested type NestedType =
/// This is StaticProperty1 on NestedType.
static member StaticProperty1 : string
…
/// This is StaticProperty100 on NestedType.
static member StaticProperty100 : string
type Type2 =
…
…
type Type100 =
…
Note that the set of types and members provided is statically known. This example doesn't leverage the ability of providers to provide types that depend on a schema. The implementation of the type provider is outlined in the following code, and the details are covered in later sections of this topic.
Warning
There may be some small naming differences between this code and the online samples.
namespace Samples.FSharp.HelloWorldTypeProvider
open System
open System.Reflection
open Samples.FSharp.ProvidedTypes
open Microsoft.FSharp.Core.CompilerServices
open Microsoft.FSharp.Quotations
// This type defines the type provider. When compiled to a DLL, it can be added
// as a reference to an F# command-line compilation, script, or project.
[<TypeProvider>]
type SampleTypeProvider(config: TypeProviderConfig) as this =
// Inheriting from this type provides implementations of ITypeProvider
// in terms of the provided types below.
inherit TypeProviderForNamespaces()
let namespaceName = "Samples.HelloWorldTypeProvider"
let thisAssembly = Assembly.GetExecutingAssembly()
// Make one provided type, called TypeN.
let makeOneProvidedType (n:int) =
…
// Now generate 100 types
let types = [ for i in 1 .. 100 -> makeOneProvidedType i ]
// And add them to the namespace
do this.AddNamespace(namespaceName, types)
[<assembly:TypeProviderAssembly>]
do()
To use this provider, open a separate instance of Visual Studio 2012, create an F# script, and then add a reference to the provider from your script by using #r as the following code shows:
#r @".\bin\Debug\Samples.HelloWorldTypeProvider.dll"
let obj1 = Samples.HelloWorldTypeProvider.Type1("some data")
let obj2 = Samples.HelloWorldTypeProvider.Type1("some other data")
obj1.InstanceProperty
obj2.InstanceProperty
[ for index in 0 .. obj1.InstanceProperty-1 -> obj1.InstanceMethod(index) ]
[ for index in 0 .. obj2.InstanceProperty-1 -> obj2.InstanceMethod(index) ]
let data1 = Samples.HelloWorldTypeProvider.Type1.NestedType.StaticProperty35
Then look for the types under the Samples.HelloWorldTypeProvider namespace that the type provider generated.
Before you recompile the provider, make sure that you have closed all instances of Visual Studio and F# Interactive that are using the provider DLL. Otherwise, a build error will occur because the output DLL will be locked.
To debug this provider by using print statements, make a script that exposes a problem with the provider, and then use the following code:
fsc.exe -r:bin\Debug\HelloWorldTypeProvider.dll script.fsx
To debug this provider by using Visual Studio, open the Visual Studio command prompt with administrative credentials, and run the following command:
devenv.exe /debugexe fsc.exe -r:bin\Debug\HelloWorldTypeProvider.dll script.fsx
As an alternative, open Visual Studio, open the Debug menu, choose Debug/Attach to process…, and attach to another devenv process where you’re editing your script. By using this method, you can more easily target particular logic in the type provider by interactively typing expressions into the second instance (with full IntelliSense and other features).
You can disable Just My Code debugging to better identify errors in generated code. For information about how to enable or disable this feature, see How to: Step Into Just My Code. Also, you can also set first-chance exception catching by opening the Debug menu and then choosing Exceptions or by choosing the Ctrl+Alt+E keys to open the Exceptions dialog box. In that dialog box, under Common Language Runtime Exceptions, select the Thrown check box.
Implementation of the Type Provider
This section walks you through the principal sections of the type provider implementation. First, you define the type for the custom type provider itself:
[<TypeProvider>]
type SampleTypeProvider(config: TypeProviderConfig) as this =
This type must be public, and you must mark it with the TypeProvider attribute so that the compiler will recognize the type provider when a separate F# project references the assembly that contains the type. The config parameter is optional, and, if present, contains contextual configuration information for the type provider instance that the F# compiler creates.
Next, you implement the ITypeProvider interface. In this case, you use the TypeProviderForNamespaces type from the ProvidedTypes API as a base type. This helper type can provide a finite collection of eagerly provided namespaces, each of which directly contains a finite number of fixed, eagerly provided types. In this context, the provider eagerly generates types even if they aren't needed or used.
inherit TypeProviderForNamespaces()
Next, define local private values that specify the namespace for the provided types, and find the type provider assembly itself. This assembly is used later as the logical parent type of the erased types that are provided.
let namespaceName = "Samples.HelloWorldTypeProvider"
let thisAssembly = Assembly.GetExecutingAssembly()
Next, create a function to provide each of the types Type1…Type100. This function is explained in more detail later in this topic.
let makeOneProvidedType (n:int) = …
Next, generate the 100 provided types:
let types = [ for i in 1 .. 100 -> makeOneProvidedType i ]
Next, add the types as a provided namespace:
do this.AddNamespace(namespaceName, types)
Finally, add an assembly attribute that indicates that you are creating a type provider DLL:
[<assembly:TypeProviderAssembly>]
do()
Providing One Type And Its Members
The makeOneProvidedType function does the real work of providing one of the types.
let makeOneProvidedType (n:int) =
…
This step explains the implementation of this function. First, create the provided type (for example, Type1, when n = 1, or Type57, when n = 57).
// This is the provided type. It is an erased provided type and, in compiled code,
// will appear as type 'obj'.
let t = ProvidedTypeDefinition(thisAssembly,namespaceName,
"Type" + string n,
baseType = Some typeof<obj>)
You should note the following points:
This provided type is erased. Because you indicate that the base type is obj, instances will appear as values of type obj in compiled code.
When you specify a non-nested type, you must specify the assembly and namespace. For erased types, the assembly should be the type provider assembly itself.
Next, add XML documentation to the type. This documentation is delayed, that is, computed on-demand if the host compiler needs it.
t.AddXmlDocDelayed (fun () -> sprintf "This provided type %s" ("Type" + string n))
Next you add a provided static property to the type:
let staticProp = ProvidedProperty(propertyName = "StaticProperty",
propertyType = typeof<string>,
IsStatic=true,
GetterCode= (fun args -> <@@ "Hello!" @@>))
Getting this property will always evaluate to the string "Hello!". The GetterCode for the property uses an F# quotation, which represents the code that the host compiler generates for getting the property. For more information about quotations, see Code Quotations (F#).
Add XML documentation to the property.
staticProp.AddXmlDocDelayed(fun () -> "This is a static property")
Now attach the provided property to the provided type. You must attach a provided member to one and only one type. Otherwise, the member will never be accessible.
t.AddMember staticProp
Now create a provided constructor that takes no parameters.
let ctor = ProvidedConstructor(parameters = [ ],
InvokeCode= (fun args -> <@@ "The object data" :> obj @@>))
The InvokeCode for the constructor returns an F# quotation, which represents the code that the host compiler generates when the constructor is called. For example, you can use the following constructor:
new Type10()
An instance of the provided type will be created with underlying data "The object data". The quoted code includes a conversion to obj because that type is the erasure of this provided type (as you specified when you declared the provided type).
Add XML documentation to the constructor, and add the provided constructor to the provided type:
ctor.AddXmlDocDelayed(fun () -> "This is a constructor")
t.AddMember ctor
Create a second provided constructor that takes one parameter:
let ctor2 =
ProvidedConstructor(parameters = [ ProvidedParameter("data",typeof<string>) ],
InvokeCode= (fun args -> <@@ (%%(args.[0]) : string) :> obj @@>))
The InvokeCode for the constructor again returns an F# quotation, which represents the code that the host compiler generated for a call to the method. For example, you can use the following constructor:
new Type10("ten")
An instance of the provided type is created with underlying data "ten". You may have already noticed that the InvokeCode function returns a quotation. The input to this function is a list of expressions, one per constructor parameter. In this case, an expression that represents the single parameter value is available in args.[0]. The code for a call to the constructor coerces the return value to the erased type obj. After you add the second provided constructor to the type, you create a provided instance property:
let instanceProp =
ProvidedProperty(propertyName = "InstanceProperty",
propertyType = typeof<int>,
GetterCode= (fun args ->
<@@ ((%%(args.[0]) : obj) :?> string).Length @@>))
instanceProp.AddXmlDocDelayed(fun () -> "This is an instance property")
t.AddMember instanceProp
Getting this property will return the length of the string, which is the representation object. The GetterCode property returns an F# quotation that specifies the code that the host compiler generates to get the property. Like InvokeCode, the GetterCode function returns a quotation. The host compiler calls this function with a list of arguments. In this case, the arguments include just the single expression that represents the instance upon which the getter is being called, which you can access by using args.[0].The implementation of GetterCode then splices into the result quotation at the erased type obj, and a cast is used to satisfy the compiler's mechanism for checking types that the object is a string. The next part of makeOneProvidedType provides an instance method with one parameter.
let instanceMeth =
ProvidedMethod(methodName = "InstanceMethod",
parameters = [ProvidedParameter("x",typeof<int>)],
returnType = typeof<char>,
InvokeCode = (fun args ->
<@@ ((%%(args.[0]) : obj) :?> string).Chars(%%(args.[1]) : int) @@>))
instanceMeth.AddXmlDocDelayed(fun () -> "This is an instance method")
// Add the instance method to the type.
t.AddMember instanceMeth
Finally, create a nested type that contains 100 nested properties. The creation of this nested type and its properties is delayed, that is, computed on-demand.
t.AddMembersDelayed(fun () ->
let nestedType = ProvidedTypeDefinition("NestedType",
Some typeof<obj>
)
nestedType.AddMembersDelayed (fun () ->
let staticPropsInNestedType =
[ for i in 1 .. 100 do
let valueOfTheProperty = "I am string " + string i
let p = ProvidedProperty(propertyName = "StaticProperty" + string i,
propertyType = typeof<string>,
IsStatic=true,
GetterCode= (fun args -> <@@ valueOfTheProperty @@>))
p.AddXmlDocDelayed(fun () ->
sprintf "This is StaticProperty%d on NestedType" i)
yield p ]
staticPropsInNestedType)
[nestedType])
// The result of makeOneProvidedType is the type.
t
Details about Erased Provided Types
The example in this section provides only erased provided types, which are particularly useful in the following situations:
When you are writing a provider for an information space that contains only data and methods.
When you are writing a provider where accurate runtime-type semantics aren't critical for practical use of the information space.
When you are writing a provider for an information space that is so large and interconnected that it isn’t technically feasible to generate real .NET types for the information space.
In this example, each provided type is erased to type obj, and all uses of the type will appear as type obj in compiled code. In fact, the underlying objects in these examples are strings, but the type will appear as Object in .NET compiled code. As with all uses of type erasure, you can use explicit boxing, unboxing, and casting to subvert erased types. In this case, a cast exception that isn’t valid may result when the object is used. A provider runtime can define its own private representation type to help protect against false representations. You can’t define erased types in F# itself. Only provided types may be erased. You must understand the ramifications, both practical and semantic, of using either erased types for your type provider or a provider that provides erased types. An erased type has no real .NET type. Therefore, you cannot do accurate reflection over the type, and you might subvert erased types if you use runtime casts and other techniques that rely on exact runtime type semantics. Subversion of erased types frequently results in type cast exceptions at runtime.
Choosing Representations for Erased Provided Types
For some uses of erased provided types, no representation is required. For example, the erased provided type might contain only static properties and members and no constructors, and no methods or properties would return an instance of the type. If you can reach instances of an erased provided type, you must consider the following questions:
What is the erasure of a provided type?
The erasure of a provided type is how the type appears in compiled .NET code.
The erasure of a provided erased class type is always the first non-erased base type in the inheritance chain of the type.
The erasure of a provided erased interface type is always Object.
What are the representations of a provided type?
- The set of possible objects for an erased provided type are called its representations. In the example in this document, the representations of all the erased provided types Type1..Type100 are always string objects.
All representations of a provided type must be compatible with the erasure of the provided type. (Otherwise, either the F# compiler will give an error for a use of the type provider, or unverifiable .NET code that isn't valid will be generated. A type provider isn’t valid if it returns code that gives a representation that isn't valid.)
You can choose a representation for provided objects by using either of the following approaches, both of which are very common:
If you're simply providing a strongly typed wrapper over an existing .NET type, it often makes sense for your type to erase to that type, use instances of that type as representations, or both. This approach is appropriate when most of the existing methods on that type still make sense when using the strongly typed version.
If you want to create an API that differs significantly from any existing .NET API, it makes sense to create runtime types that will be the type erasure and representations for the provided types.
The example in this document uses strings as representations of provided objects. Frequently, it may be appropriate to use other objects for representations. For example, you may use a dictionary as a property bag:
ProvidedConstructor(parameters = [],
InvokeCode= (fun args -> <@@ (new Dictionary<string,obj>()) :> obj @@>))
As an alternative, you may define a type in your type provider that will be used at runtime to form the representation, along with one or more runtime operations:
type DataObject() =
let data = Dictionary<string,obj>()
member x.RuntimeOperation() = data.Count
Provided members can then construct instances of this object type:
ProvidedConstructor(parameters = [],
InvokeCode= (fun args -> <@@ (new DataObject()) :> obj @@>))
In this case, you may (optionally) use this type as the type erasure by specifying this type as the baseType when constructing the ProvidedTypeDefinition:
ProvidedTypeDefinition(…, baseType = Some typeof<DataObject> )
…
ProvidedConstructor(…, InvokeCode = (fun args -> <@@ new DataObject() @@>), …)
Key Lessons
The previous section explained how to create a simple erasing type provider that provides a range of types, properties, and methods. This section also explained the concept of type erasure, including some of the advantages and disadvantages of providing erased types from a type provider, and discussed representations of erased types.
A Type Provider That Uses Static Parameters
The ability to parameterize type providers by static data enables many interesting scenarios, even in cases when the provider doesn't need to access any local or remote data. In this section, you’ll learn some basic techniques for putting together such a provider.
Type Checked Regex Provider
Imagine that you want to implement a type provider for regular expressions that wraps the .NET Regex libraries in an interface that provides the following compile-time guarantees:
Verifying whether a regular expression is valid.
Providing named properties on matches that are based on any group names in the regular expression.
This section shows you how to use type providers to create a RegExProviderType type that the regular expression pattern parameterizes to provide these benefits. The compiler will report an error if the supplied pattern isn't valid, and the type provider can extract the groups from the pattern so that you can access them by using named properties on matches. When you design a type provider, you should consider how its exposed API should look to end users and how this design will translate to .NET code. The following example shows how to use such an API to get the components of the area code:
type T = RegexTyped< @"(?<AreaCode>^\d{3})-(?<PhoneNumber>\d{3}-\d{4}$)">
let reg = T()
let result = T.IsMatch("425-555-2345")
let r = reg.Match("425-555-2345").Group_AreaCode.Value //r equals "425"
The following example shows how the type provider translates these calls:
let reg = new Regex(@"(?<AreaCode>^\d{3})-(?<PhoneNumber>\d{3}-\d{4}$)")
let result = reg.IsMatch("425-123-2345")
let r = reg.Match("425-123-2345").Groups.["AreaCode"].Value //r equals "425"
Note the following points:
The standard Regex type represents the parameterized RegexTyped type.
The RegexTyped constructor results in a call to the Regex constructor, passing in the static type argument for the pattern.
The results of the Match method are represented by the standard Match type.
Each named group results in a provided property, and accessing the property results in a use of an indexer on a match’s Groups collection.
The following code is the core of the logic to implement such a provider, and this example omits the addition of all members to the provided type. For information about each added member, see the appropriate section later in this topic. For the full code, download the sample from the F# 3.0 Sample Pack on the Codeplex website.
namespace Samples.FSharp.RegexTypeProvider
open System.Reflection
open Microsoft.FSharp.Core.CompilerServices
open Samples.FSharp.ProvidedTypes
open System.Text.RegularExpressions
[<TypeProvider>]
type public CheckedRegexProvider() as this =
inherit TypeProviderForNamespaces()
// Get the assembly and namespace used to house the provided types
let thisAssembly = Assembly.GetExecutingAssembly()
let rootNamespace = "Samples.FSharp.RegexTypeProvider"
let baseTy = typeof<obj>
let staticParams = [ProvidedStaticParameter("pattern", typeof<string>)]
let regexTy = ProvidedTypeDefinition(thisAssembly, rootNamespace, "RegexTyped", Some baseTy)
do regexTy.DefineStaticParameters(
parameters=staticParams,
instantiationFunction=(fun typeName parameterValues ->
match parameterValues with
| [| :? string as pattern|] ->
// Create an instance of the regular expression.
//
// This will fail with System.ArgumentException if the regular expression is not valid.
// The exception will escape the type provider and be reported in client code.
let r = System.Text.RegularExpressions.Regex(pattern)
// Declare the typed regex provided type.
// The type erasure of this type is 'obj', even though the representation will always be a Regex
// This, combined with hiding the object methods, makes the IntelliSense experience simpler.
let ty = ProvidedTypeDefinition(
thisAssembly,
rootNamespace,
typeName,
baseType = Some baseTy)
...
ty
| _ -> failwith "unexpected parameter values"))
do this.AddNamespace(rootNamespace, [regexTy])
[<TypeProviderAssembly>]
do ()
Note the following points:
The type provider takes two static parameters: the pattern, which is mandatory, and the options, which are optional (because a default value is provided).
After the static arguments are supplied, you create an instance of the regular expression. This instance will throw an exception if the Regex is malformed, and this error will be reported to users.
Within the DefineStaticParameters callback, you define the type that will be returned after the arguments are supplied.
This code sets HideObjectMethods to true so that the IntelliSense experience will remain streamlined. This attribute causes the Equals, GetHashCode, Finalize, and GetType members to be suppressed from IntelliSense lists for a provided object.
You use obj as the base type of the method, but you’ll use a Regex object as the runtime representation of this type, as the next example shows.
The call to the Regex constructor throws a ArgumentException when a regular expression isn’t valid. The compiler catches this exception and reports an error message to the user at compile time or in the Visual Studio editor. This exception enables regular expressions to be validated without running an application.
The type defined above isn't useful yet because it doesn’t contain any meaningful methods or properties. First, add a static IsMatch method:
let isMatch = ProvidedMethod(
methodName = "IsMatch",
parameters = [ProvidedParameter("input", typeof<string>)],
returnType = typeof<bool>,
IsStaticMethod = true,
InvokeCode = fun args -> <@@ Regex.IsMatch(%%args.[0], pattern) @@>)
isMatch.AddXmlDoc "Indicates whether the regular expression finds a match in the specified input string."
ty.AddMember isMatch
The previous code defines a method IsMatch, which takes a string as input and returns a bool. The only tricky part is the use of the args argument within the InvokeCode definition. In this example, args is a list of quotations that represents the arguments to this method. If the method is an instance method, the first argument represents the this argument. However, for a static method, the arguments are all just the explicit arguments to the method. Note that the type of the quoted value should match the specified return type (in this case, bool). Also note that this code uses the AddXmlDoc method to make sure that the provided method also has useful documentation, which you can supply through IntelliSense.
Next, add an instance Match method. However, this method should return a value of a provided Match type so that the groups can be accessed in a strongly typed fashion. Thus, you first declare the Match type. Because this type depends on the pattern that was supplied as a static argument, this type must be nested within the parameterized type definition:
let matchTy = ProvidedTypeDefinition(
"MatchType",
baseType = Some baseTy,
HideObjectMethods = true)
ty.AddMember matchTy
You then add one property to the Match type for each group. At runtime, a match is represented as a Match value, so the quotation that defines the property must use the Groups indexed property to get the relevant group.
for group in r.GetGroupNames() do
// Ignore the group named 0, which represents all input.
if group <> "0" then
let prop = ProvidedProperty(
propertyName = group,
propertyType = typeof<Group>,
GetterCode = fun args -> <@@ ((%%args.[0]:obj) :?> Match).Groups.[group] @@>)
prop.AddXmlDoc(sprintf @"Gets the ""%s"" group from this match" group)
matchTy.AddMember prop
Again, note that you’re adding XML documentation to the provided property. Also note that a property can be read if a GetterCode function is provided, and the property can be written if a SetterCode function is provided, so the resulting property is read only.
Now you can create an instance method that returns a value of this Match type:
let matchMethod =
ProvidedMethod(
methodName = "Match",
parameters = [ProvidedParameter("input", typeof<string>)],
returnType = matchTy,
InvokeCode = fun args -> <@@ ((%%args.[0]:obj) :?> Regex).Match(%%args.[1]) :> obj @@>)
matchMeth.AddXmlDoc "Searches the specified input string for the first occurrence of this regular expression"
ty.AddMember matchMeth
Because you are creating an instance method, args.[0] represents the RegexTyped instance on which the method is being called, and args.[1] is the input argument.
Finally, provide a constructor so that instances of the provided type can be created.
let ctor = ProvidedConstructor(
parameters = [],
InvokeCode = fun args -> <@@ Regex(pattern, options) :> obj @@>)
ctor.AddXmlDoc("Initializes a regular expression instance.")
ty.AddMember ctor
The constructor merely erases to the creation of a standard .NET Regex instance, which is again boxed to an object because obj is the erasure of the provided type. With that change, the sample API usage that specified earlier in the topic works as expected. The following code is complete and final:
namespace Samples.FSharp.RegexTypeProvider
open System.Reflection
open Microsoft.FSharp.Core.CompilerServices
open Samples.FSharp.ProvidedTypes
open System.Text.RegularExpressions
[<TypeProvider>]
type public CheckedRegexProvider() as this =
inherit TypeProviderForNamespaces()
// Get the assembly and namespace used to house the provided types.
let thisAssembly = Assembly.GetExecutingAssembly()
let rootNamespace = "Samples.FSharp.RegexTypeProvider"
let baseTy = typeof<obj>
let staticParams = [ProvidedStaticParameter("pattern", typeof<string>)]
let regexTy = ProvidedTypeDefinition(thisAssembly, rootNamespace, "RegexTyped", Some baseTy)
do regexTy.DefineStaticParameters(
parameters=staticParams,
instantiationFunction=(fun typeName parameterValues ->
match parameterValues with
| [| :? string as pattern|] ->
// Create an instance of the regular expression.
let r = System.Text.RegularExpressions.Regex(pattern)
// Declare the typed regex provided type.
let ty = ProvidedTypeDefinition(
thisAssembly,
rootNamespace,
typeName,
baseType = Some baseTy)
ty.AddXmlDoc "A strongly typed interface to the regular expression '%s'"
// Provide strongly typed version of Regex.IsMatch static method.
let isMatch = ProvidedMethod(
methodName = "IsMatch",
parameters = [ProvidedParameter("input", typeof<string>)],
returnType = typeof<bool>,
IsStaticMethod = true,
InvokeCode = fun args -> <@@ Regex.IsMatch(%%args.[0], pattern) @@>)
isMatch.AddXmlDoc "Indicates whether the regular expression finds a match in the specified input string"
ty.AddMember isMatch
// Provided type for matches
// Again, erase to obj even though the representation will always be a Match
let matchTy = ProvidedTypeDefinition(
"MatchType",
baseType = Some baseTy,
HideObjectMethods = true)
// Nest the match type within parameterized Regex type.
ty.AddMember matchTy
// Add group properties to match type
for group in r.GetGroupNames() do
// Ignore the group named 0, which represents all input.
if group <> "0" then
let prop = ProvidedProperty(
propertyName = group,
propertyType = typeof<Group>,
GetterCode = fun args -> <@@ ((%%args.[0]:obj) :?> Match).Groups.[group] @@>)
prop.AddXmlDoc(sprintf @"Gets the ""%s"" group from this match" group)
matchTy.AddMember(prop)
// Provide strongly typed version of Regex.Match instance method.
let matchMeth = ProvidedMethod(
methodName = "Match",
parameters = [ProvidedParameter("input", typeof<string>)],
returnType = matchTy,
InvokeCode = fun args -> <@@ ((%%args.[0]:obj) :?> Regex).Match(%%args.[1]) :> obj @@>)
matchMeth.AddXmlDoc "Searches the specified input string for the first occurence of this regular expression"
ty.AddMember matchMeth
// Declare a constructor.
let ctor = ProvidedConstructor(
parameters = [],
InvokeCode = fun args -> <@@ Regex(pattern) :> obj @@>)
// Add documentation to the constructor.
ctor.AddXmlDoc "Initializes a regular expression instance"
ty.AddMember ctor
ty
| _ -> failwith "unexpected parameter values"))
do this.AddNamespace(rootNamespace, [regexTy])
[<TypeProviderAssembly>]
do ()
Key Lessons
This section explained how to create a type provider that operates on its static parameters. The provider checks the static parameter and provides operations based on its value.
A Type Provider That Is Backed By Local Data
Frequently you might want type providers to present APIs based on not only static parameters but also information from local or remote systems. This section discusses type providers that are based on local data, such as local data files.
Simple CSV File Provider
As a simple example, consider a type provider for accessing scientific data in Comma Separated Value (CSV) format. This section assumes that the CSV files contain a header row followed by floating point data, as the following table illustrates:
Distance (meter) |
Time (second) |
|---|---|
50.0 |
3.7 |
100.0 |
5.2 |
150.0 |
6.4 |
This section shows how to provide a type that you can use to get rows with a Distance property of type float<meter> and a Time property of type float<second>. For simplicity, the following assumptions are made:
Header names are either unit-less or have the form “Name (unit)” and don't contain commas.
Units are all Systeme International (SI) units as the Microsoft.FSharp.Data.UnitSystems.SI.UnitNames Module (F#) module defines.
Units are all simple (for example, meter) rather than compound (for example, meter/second).
All columns contain floating point data.
A more complete provider would loosen these restrictions.
Again the first step is to consider how the API should look. Given an info.csv file with the contents from the previous table (in comma-separated format), users of the provider should be able to write code that resembles the following example:
let info = new MiniCsv<"info.csv">()
for row in info.Data do
let time = row.Time
printfn "%f" (float time)
In this case, the compiler should convert these calls into something like the following example:
let info = new MiniCsvFile("info.csv")
for row in info.Data do
let (time:float) = row.[1]
printfn "%f" (float time)
The optimal translation will require the type provider to define a real CsvFile type in the type provider's assembly. Type providers often rely on a few helper types and methods to wrap important logic. Because measures are erased at runtime, you can use a float[] as the erased type for a row. The compiler will treat different columns as having different measure types. For example, the first column in our example has type float<meter>, and the second has float<second>. However, the erased representation can remain quite simple.
The following code shows the core of the implementation.
// Simple type wrapping CSV data
type CsvFile(filename) =
// Cache the sequence of all data lines (all lines but the first)
let data =
seq { for line in File.ReadAllLines(filename) |> Seq.skip 1 do
yield line.Split(',') |> Array.map float }
|> Seq.cache
member __.Data = data
[<TypeProvider>]
type public MiniCsvProvider(cfg:TypeProviderConfig) as this =
inherit TypeProviderForNamespaces()
// Get the assembly and namespace used to house the provided types.
let asm = System.Reflection.Assembly.GetExecutingAssembly()
let ns = "Samples.FSharp.MiniCsvProvider"
// Create the main provided type.
let csvTy = ProvidedTypeDefinition(asm, ns, "MiniCsv", Some(typeof<obj>))
// Parameterize the type by the file to use as a template.
let filename = ProvidedStaticParameter("filename", typeof<string>)
do csvTy.DefineStaticParameters([filename], fun tyName [| :? string as filename |] ->
// Resolve the filename relative to the resolution folder.
let resolvedFilename = Path.Combine(cfg.ResolutionFolder, filename)
// Get the first line from the file.
let headerLine = File.ReadLines(resolvedFilename) |> Seq.head
// Define a provided type for each row, erasing to a float[].
let rowTy = ProvidedTypeDefinition("Row", Some(typeof<float[]>))
// Extract header names from the file, splitting on commas.
// use Regex matching to get the position in the row at which the field occurs
let headers = Regex.Matches(headerLine, "[^,]+")
// Add one property per CSV field.
for i in 0 .. headers.Count - 1 do
let headerText = headers.[i].Value
// Try to decompose this header into a name and unit.
let fieldName, fieldTy =
let m = Regex.Match(headerText, @"(?<field>.+) \((?<unit>.+)\)")
if m.Success then
let unitName = m.Groups.["unit"].Value
let units = ProvidedMeasureBuilder.Default.SI unitName
m.Groups.["field"].Value, ProvidedMeasureBuilder.Default.AnnotateType(typeof<float>,[units])
else
// no units, just treat it as a normal float
headerText, typeof<float>
let prop = ProvidedProperty(fieldName, fieldTy,
GetterCode = fun [row] -> <@@ (%%row:float[]).[i] @@>)
// Add metadata that defines the property's location in the referenced file.
prop.AddDefinitionLocation(1, headers.[i].Index + 1, filename)
rowTy.AddMember(prop)
// Define the provided type, erasing to CsvFile.
let ty = ProvidedTypeDefinition(asm, ns, tyName, Some(typeof<CsvFile>))
// Add a parameterless constructor that loads the file that was used to define the schema.
let ctor0 = ProvidedConstructor([],
InvokeCode = fun [] -> <@@ CsvFile(resolvedFilename) @@>)
ty.AddMember ctor0
// Add a constructor that takes the file name to load.
let ctor1 = ProvidedConstructor([ProvidedParameter("filename", typeof<string>)],
InvokeCode = fun [filename] -> <@@ CsvFile(%%filename) @@>)
ty.AddMember ctor1
// Add a more strongly typed Data property, which uses the existing property at runtime.
let prop = ProvidedProperty("Data", typedefof<seq<_>>.MakeGenericType(rowTy),
GetterCode = fun [csvFile] -> <@@ (%%csvFile:CsvFile).Data @@>)
ty.AddMember prop
// Add the row type as a nested type.
ty.AddMember rowTy
ty)
// Add the type to the namespace.
do this.AddNamespace(ns, [csvTy])
Note the following points about the implementation:
Overloaded constructors allow either the original file or one that has an identical schema to be read. This pattern is common when you write a type provider for local or remote data sources, and this pattern allows a local file to be used as the template for remote data.
You can use the TypeProviderConfig value that’s passed in to the type provider constructor to resolve relative file names.
You can use the AddDefinitionLocation method to define the location of the provided properties. Therefore, if you use Go To Definition on a provided property, the CSV file will open in Visual Studio.
You can use the ProvidedMeasureBuilder type to look up the SI units and to generate the relevant float<_> types.
Key Lessons
This section explained how to create a type provider for a local data source with a simple schema that's contained in the data source itself.
Going Further
The following sections include suggestions for further study.
A Look at the Compiled Code for Erased Types
To give you some idea of how the use of the type provider corresponds to the code that's emitted, look at the following function by using the HelloWorldTypeProvider that's used earlier in this topic.
let function1 () =
let obj1 = Samples.HelloWorldTypeProvider.Type1("some data")
obj1.InstanceProperty
Here’s an image of the resulting code decompiled by using ildasm.exe:
.class public abstract auto ansi sealed Module1
extends [mscorlib]System.Object
{
.custom instance void [FSharp.Core]Microsoft.FSharp.Core.CompilationMappingAtt
ribute::.ctor(valuetype [FSharp.Core]Microsoft.FSharp.Core.SourceConstructFlags)
= ( 01 00 07 00 00 00 00 00 )
.method public static int32 function1() cil managed
{
// Code size 24 (0x18)
.maxstack 3
.locals init ([0] object obj1)
IL_0000: nop
IL_0001: ldstr "some data"
IL_0006: unbox.any [mscorlib]System.Object
IL_000b: stloc.0
IL_000c: ldloc.0
IL_000d: call !!0 [FSharp.Core_2]Microsoft.FSharp.Core.LanguagePrimit
ives/IntrinsicFunctions::UnboxGeneric<string>(object)
IL_0012: callvirt instance int32 [mscorlib_3]System.String::get_Length()
IL_0017: ret
} // end of method Module1::function1
} // end of class Module1
As the example shows, all mentions of the type Type1 and the InstanceProperty property have been erased, leaving only operations on the runtime types involved.
Design and Naming Conventions for Type Providers
Observe the following conventions when authoring type providers.
Providers for Connectivity Protocols
In general, names of most provider DLLs for data and service connectivity protocols, such as OData or SQL connections, should end in TypeProvider or TypeProviders. For example, use a DLL name that resembles the following string:
Fabrikam.Management.BasicTypeProviders.dllEnsure that your provided types are members of the corresponding namespace, and indicate the connectivity protocol that you implemented:
Fabrikam.Management.BasicTypeProviders.WmiConnection<…> Fabrikam.Management.BasicTypeProviders.DataProtocolConnection<…>Utility Providers for General Coding
For a utility type provider such as that for regular expressions, the type provider may be part of a base library, as the following example shows:
#r "Fabrikam.Core.Text.Utilities.dll"In this case, the provided type would appear at an appropriate point according to normal .NET design conventions:
open Fabrikam.Core.Text.RegexTyped let regex = new RegexTyped<"a+b+a+b+">()Singleton Data Sources
Some type providers connect to a single dedicated data source and provide only data. In this case, you should drop the TypeProvider suffix and use normal conventions for .NET naming:
#r "Fabrikam.Data.Freebase.dll" let data = Fabrikam.Data.Freebase.Astronomy.AsteroidsFor more information, see the GetConnection design convention that's described later in this topic.
Design Patterns for Type Providers
The following sections describe design patterns you can use when authoring type providers.
The GetConnection Design Pattern
Most type providers should be written to use the GetConnection pattern that's used by the type providers in FSharp.Data.TypeProviders.dll, as the following example shows:
#r "Fabrikam.Data.WebDataStore.dll"
type Service = Fabrikam.Data.WebDataStore<…static connection parameters…>
let connection = Service.GetConnection(…dynamic connection parameters…)
let data = connection.Astronomy.Asteroids
Type Providers Backed By Remote Data and Services
Before you create a type provider that's backed by remote data and services, you must consider a range of issues that are inherent in connected programming. These issues include the following considerations:
schema mapping
liveness and invalidation in the presence of schema change
schema caching
asynchronous implementations of data access operations
supporting queries, including LINQ queries
credentials and authentication
This topic doesn't explore these issues further.
Additional Authoring Techniques
When you write your own type providers, you might want to use the following additional techniques.
Creating Types and Members On-Demand
The ProvidedType API has delayed versions of AddMember.
type ProvidedType = member AddMemberDelayed : (unit -> MemberInfo) -> unit member AddMembersDelayed : (unit -> MemberInfo list) -> unitThese versions are used to create on-demand spaces of types.
Providing Array, ByRef, and Pointer types
You make provided members (whose signatures include array types, byref types, and instantiations of generic types) by using the normal MakeArrayType, MakePointerType, and MakeGenericType on any instance of System.Type, including ProvidedTypeDefinitions.
Providing Unit of Measure Annotations
The ProvidedTypes API provides helpers for providing measure annotations. For example, to provide the type float<kg>, use the following code:
let measures = ProvidedMeasureBuilder.Default let kg = measures.SI "kilogram" let m = measures.SI "meter" let float_kg = measures.AnnotateType(typeof<float>,[kg])To provide the type Nullable<decimal<kg/m^2>>, use the following code:
let kgpm2 = measures.Ratio(kg, measures.Square m) let dkgpm2 = measures.AnnotateType(typeof<decimal>,[kgpm2]) let nullableDecimal_kgpm2 = typedefof<System.Nullable<_>>.MakeGenericType [|dkgpm2 |]Accessing Project-Local or Script-Local Resources
Each instance of a type provider can be given a TypeProviderConfig value during construction. This value contains the "resolution folder" for the provider (that is, the project folder for the compilation or the directory that contains a script), the list of referenced assemblies, and other information.
Invalidation
Providers can raise invalidation signals to notify the F# language service that the schema assumptions may have changed. When invalidation occurs, a typecheck is redone if the provider is being hosted in Visual Studio. This signal will be ignored when the provider is hosted in F# Interactive or by the F# Compiler (fsc.exe).
Caching Schema Information
Providers must often cache access to schema information. The cached data should be stored by using a file name that's given as a static parameter or as user data. An example of schema caching is the LocalSchemaFile parameter in the type providers in the FSharp.Data.TypeProviders assembly. In the implementation of these providers, this static parameter directs the type provider to use the schema information in the specified local file instead of accessing the data source over the network. To use cached schema information, you must also set the static parameter ForceUpdate to false. You could use a similar technique to enable online and offline data access.
Backing Assembly
When you compile a .dll or .exe file, the backing .dll file for generated types is statically linked into the resulting assembly. This link is created by copying the Intermediate Language (IL) type definitions and any managed resources from the backing assembly into the final assembly. When you use F# Interactive, the backing .dll file isn't copied and is instead loaded directly into the F# Interactive process.
Exceptions and Diagnostics from Type Providers
All uses of all members from provided types may throw exceptions. In all cases, if a type provider throws an exception, the host compiler attributes the error to a specific type provider.
Type provider exceptions should never result in internal compiler errors.
Type providers can't report warnings.
When a type provider is hosted in the F# compiler, an F# development environment, or F# Interactive, all exceptions from that provider are caught. The Message property is always the error text, and no stack trace appears. If you’re going to throw an exception, you can throw the following examples:
Providing Generated Types
So far, this document has explained how provide erased types. You can also use the type provider mechanism in F# to provide generated types, which are added as real .NET type definitions into the users' program. You must refer to generated provided types by using a type definition.
open Microsoft.FSharp.TypeProviders
type Service = ODataService<" http://services.odata.org/Northwind/Northwind.svc/">
The ProvidedTypes-0.2 helper code that is part of the F# 3.0 release has only limited support for providing generated types. The following statements must be true for a generated type definition:
IsErased must be set to false.
The provider must have an assembly that has an actual backing .NET .dll file with a matching .dll file on disk.
You must also call ConvertToGenerated on a root provided type whose nested types form a closed set of generated types. This call emits the given provided type definition and its nested type definitions into an assembly and adjusts the Assembly property of all provided type definitions to return that assembly. The assembly is emitted only when the Assembly property on the root type is accessed for the first time. The host F# compiler does access this property when it processes a generative type declaration for the type.
Rules and Limitations
When you write type providers, keep the following rules and limitations in mind.
Provided types must be reachable.
All provided types should be reachable from the non-nested types. The non-nested types are given in the call to the TypeProviderForNamespaces constructor or a call to AddNamespace. For example, if the provider provides a type StaticClass.P : T, you must ensure that T is either a non-nested type or nested under one.
For example, some providers have a static class such as DataTypes that contain these T1, T2, T3, ... types. Otherwise, the error says that a reference to type T in assembly A was found, but the type couldn't be found in that assembly. If this error appears, verify that all your subtypes can be reached from the provider types. Note: These T1, T2, T3... types are referred to as the on-the-fly types. Remember to put them in an accessible namespace or a parent type.
Limitations of the Type Provider Mechanism
The type provider mechanism in F# has the following limitations:
The underlying infrastructure for type providers in F# doesn't support provided generic types or provided generic methods.
The mechanism doesn't support nested types with static parameters.
Limitations of the ProvidedTypes Support Code
The ProvidedTypes support code has the following rules and limitations:
Provided properties with indexed getters and setters aren't implemented.
Provided events aren't implemented.
The provided types and information objects should be used only for the type provider mechanism in F#. They aren't more generally usable as System.Type objects.
The constructs that you can use in quotations that define method implementations have several limitations. You can refer to the source code for ProvidedTypes-Version to see which constructs are supported in quotations.
Type providers must generate output assemblies that are .dll files, not .exe files.
Development Tips
You might find the following tips helpful during the development process.
Run Two Instances of Visual Studio. You can develop the type provider in one instance and test the provider in the other because the test IDE will take a lock on the .dll file that prevents the type provider from being rebuilt. Thus, you must close the second instance of Visual Studio while the provider is built in the first instance, and then you must reopen the second instance after the provider is built.
Debug type providers by using invocations of fsc.exe. You can invoke type providers by using the following tools:
fsc.exe (The F# command line compiler)
fsi.exe (The F# Interactive compiler)
devenv.exe (Visual Studio)
You can often debug type providers most easily by using fsc.exe on a test script file (for example, script.fsx). You can launch a debugger from a command prompt.
devenv /debugexe fsc.exe script.fsxYou can use print-to-stdout logging.