Lexical Closure and Query Comprehension
Visual Studio 2008 Technical Articles
Binyam Kelile—Visual Basic Team
Microsoft Corporation
Summary: This article enables you to get an overview of Lexical Closure and the basic concept involved with Query Comprehension in Visual Basic 2008. A previous knowledge of LINQ and Lambda is required to get a better understanding of this article. If you need to read more about LINQ and Lambda click here.
(23 printed pages)
Contents
Introduction
Lexical Closures
Scope
Lifetime
Static Locals
Generics
Lifetime shim
Method Call
MyBase/MyClass
Restrictions
Conclusion
Introduction
Linq is an excellent new capability in VS2008, powerful and flexible at the same time. But like any new technology, it can often seem like a 'black box' in that you know something is going on under the hood to make all this cool stuff work, you just do not know what! The purpose of this article is to push aside some of the mystery and give you an insight into how queries are constructed and passed around.
First, let's set up a standard scenario we are going to work with, a simple query which queries an in-memory collection that determines the students in the class who failed in math (a score of less than 50).
Let's define the schema by using the following class.
Example 1
Class Student
Public name As String
Public mathScore As Integer
End Class
Here is a small subset of data that we will use as our running example:
Example 2
Dim Students() As Student = {New Student With {.name = "Roger", .mathScore = 43}, _
New Student With {.name = "Chris", .mathScore = 37}, _
New Student With {.name = "Sarah", .mathScore = 95}}
Given the above data, we can query for all students to get the students in the class who failed in math.
Example 3
Dim results = From student In Students _
Where student.mathScore < 50 _
Select student
For Each p In results
MsgBox(p.name)
Next
The above LINQ code compiles and runs fine, returning us the names of those students who failed in math. But what exactly is the query above?
Well, all queries are composed of at least three basic concepts: an extension method, lambda expression and closure, which is the main focus of this article. In terms of these basic concepts, here is how the compiler compiles the above query.
The compiler translates the above query comprehension into invocation of method call by splitting the query up into its component parts, pulls the expression embedded in each part of the query, and generates a lambda function that gets passed into method calls of the object being queried. The methods can be instance methods of the object being queried or extension methods that are external to the object which implement the actual execution of the query.
In this case, the translation could be as simple as the following:
Example 4
Dim results = Students.Where(Function(student) student.mathScore < 50). _
Select(Function(student) student)
The above extension method syntax depends on lambda expressions, which represent inline functions that return the result of an expression. The lambda expression is converted to a delegate and passed to Where and Select extension methods that are available on an instance of a particular type or object that implements an interface called IEnumerable(of T).
Now let's discuss the concept of closure (also known as a 'Lexical Closure').
Lexical Closures
A Lexical Closure, often referred to as a closure, is not an immediately visible feature to an end user and is not intended to be used directly from your code. But rather it is a compiler-generated class, which is nested in the containing class or module of the outermost function, and contains the local variables or parameters that need to be referred to by other methods.
Let's see an example that will make this clear:
Example 5
Class ClosureSample
Sub Example()
Dim passMark As Integer = 50
Dim results = From student In Students _
Where student.mathScore < passMark _
Select student
End Sub
Or
Sub Example(ByVal passMark As Integer)
Dim results = From student In Students _
Where student.mathScore < passMark _
Select student
End Sub
End Class
This query retrieves the list of students who failed on math that represented by passMark variable (a score of 50) and when the compiler translates the above query statement into invocation of method call, it first pulls the expression "student.mathScore < passMark" out and generates the following lambda function, which gets passed to the Where extension method that expects a delegate. Then, the Where extension method uses this delegate to filter out the result.
Example 5 — Compiler Generated
Public Function _Lambda$__1(ByVal student As Student) As Boolean
Return (student.mathScore < Me.Local_passMark)
End Function
But the question here will be how we can pass the above lambda function into another method, when the "passMark" variable referred by the above lambda function that is not a local variable or parameter of the lambda function, we call it free variable.
Here is the compiler behind-the-scenes trick, commonly known as variable lifting, that extends the lifetime of the "passMark" variable beyond the local function scope using closure, which represents the entirety of everything we need to run a query that facilitates passing the lambda function into another method.
Let's see what the compiler basically does when it needs to extend the lifetime of a local variable beyond the local function scope. Generate the following closure class, which has a field "passMark" and a lambda expression that gets passed to Where extension method and creates a new instance of a closure class inside the Sub Example, and then redirects all variable references into the "local_passMark" of the closure class.
Example 5 — Compiler Generated
Friend NotInheritable Class ClosureSample
Friend Class _Closure$__1Public Sub New(ByVal other As _Closure$__1)
If (Not other Is Nothing) Then
Me.local_passMark = other.local_passMark
End If
End Sub
Public Function _Lambda$__1(ByVal student As Student) As Boolean
Return (student.mathScore < Me.local_passMark)
End Function
Public local_passMark As Integer
End Class
Private Shared Function _Lambda$__2(ByVal student As Student) As Student
Return student
End Function
Sub Example()
Dim closureVariable_A_8 As New _Closure$__1
closureVariable_A_8.local_passMark = 50
Dim results As IEnumerable(Of Student) = Students.Where(Of
Student)(New Func(Of Student, Boolean)(AddressOf
closureVariable_A_8._Lambda$__1)).Select(Of Student,
Student)(New Func(Of Student, Student)(AddressOf _Lambda$__2))
End Sub
End Class
As I mentioned in the closure definition, the generated closure class lives outside of the method nested in the containing class of the outermost function. Therefore, the Where extension methods created by the above query statement can legally access the local variables captured in the closure and passed into.
Therefore, closure encapsulates all of the data needed for the query to run. This is where the closure concept comes from: the enclosure of everything necessary for the query to run.
Scope
Closure is structured in Visual Basic based on the concept of scope, which decides where the variables can be used in a program. And for every scope where lifted variables declared, the compiler generates a new "closure" class associated with that scope and lifts the variable into the same closure and all references to the variables are redirected to that closure.
Consider the following query that returns list of students and their letter grade based on their score:
Example 6
Module Module1
Sub Example(ByVal scoreRange As Integer)
Select Case scoreRange
Case 80 To 90
Dim gradeB As String = "B"
Dim results = From student In Students _
Where student.mathScore >= 80 _
And student.mathScore <= 90 _
Select New With _
{Key .name = student.name, .finalscore = gradeB}
Case 95 To 100
Dim gradeA As String = "A"
Dim results = From student In Students _
Where student.mathScore >= 95 _
And student.mathScore <= 100 _
Select New With _
{Key .name = student.name, .finalscore = gradeA}
End Select
End Sub
Module
In Example 6, the compiler will generate two closure classes for each "Case" block. Let's see what the generated code looks like:
Example 6 — Compiler Generated
Friend NotInheritable Class Module1
Private Shared Function _Lambda$__1(ByVal student As Student) As Boolean
Return ((student.mathScore >= 80) And (student.mathScore <= 90))
End Function
Private Shared Function _Lambda$__3(ByVal student As Student) As Boolean
Return ((student.mathScore >= &H5F) And (student.mathScore <= 100))
End Function
Public Shared Sub Example(ByVal scoreRange As Integer)
End Sub
Friend Class _Closure$__1
Public Sub New(ByVal other As _Closure$__1)
If (Not other Is Nothing) Then
Me.local_gradeB = other.local_gradeB
End If
End Sub
Public Function _Lambda$__2(ByVal student As Student) As VB$AnonymousType_0(Of String, String)
Return New VB$AnonymousType_0(Of String, String)(student.name, Me.local_gradeB)
End Function
Public local_gradeB As String
End Class
Friend Class _Closure$__2
Public Sub New(ByVal other As _Closure$__2)
If (Not other Is Nothing) Then
Me.local_gradeA = other.local_gradeA
End If
End Sub
Public Function _Lambda$__4(ByVal student As Student) As VB$AnonymousType_0(Of String, String)
Return New VB$AnonymousType_0(Of String, String)(student.name, Me.local_gradeA)
End Function
Public local_gradeA As String
End Class
End Class
As you can see in the compiler generated code, _Closure$__1 and _Closure$__2 are created.
Notice that _Lambda$__1 and _Lambda$__3 are generated to get passed to Where extension method outside of the closure class, because they don't refer any lifted local. Therefore, there is no need to generate a closure class if the lambda expression doesn't refer any lifted local.
Then a closure instance will be created at the beginning of each "Case" block. Here is what the query with the closure instance looks like:
Example 6 — Compiler Generated
Public Shared Sub Example(ByVal scoreRange As Integer)
Select Case scoreRange
Case 80 to 90
Dim Closure_ClosureVariable_14_C As New _Closure$__1
Closure_ClosureVariable_14_C.local_gradeB = "B"
Dim results…
Exit Select
Case 95 to 100
Dim Closure_ClosureVariable_1B_C As New _Closure$__2
Closure_ClosureVariable_1B_C.local_gradeA = "A"
Dim results...
Exit Select
End Select
End Sub
Let's modify the previous example to take a look at the case of nested block or variable lifted from different levels of enclosing blocks, where the outermost and innermost blocks declare lifted variable.
Example 7
Sub Example(ByVal score As Integer)
Select Case score
Case 90 To 100
Dim gradeB As String = "A"
If score >= 95 Then
Dim type As String = " With Honor"
Dim results = From student In Students _
Where student.mathScore = score _
Select New With _
{Key .name = student.name, _
.finalscore = gradeB, .type = type}
End If
End Select
End Sub
From the above example "If" block is nested within "Select...Case" block and "Select...Case" block itself is nested inside Example (...) method block and, the variables declared in each block will have local scope. In this case, the compiler generates a nested closures class. Here is how the compile code looks like:
Example 7 — Compiler Generated
Friend Class _Closure$__1
Public Sub New(ByVal other As _Closure$__1)
If (Not other Is Nothing) Then
Me.local_score = other.local_score
End If
End Sub
Public local_score As Integer
Friend Class _Closure$__2
Public Sub New(ByVal other As _Closure$__2)
If (Not other Is Nothing) Then
Me.local_gradeB = other.local_gradeB
End If
End Sub
Public local_gradeB As String
Friend Class _Closure$__3
Public Sub New(ByVal other As _Closure$__3)
If (Not other Is Nothing) Then
Me.local_type = other.local_type
End If
End Sub
Public Function _Lambda$__1(ByVal student As Student) As Boolean
Return (student.mathScore = Me.closureVariable_10_8.local_score)
End Function
Public Function _Lambda$__2(ByVal student As Student) As VB$AnonymousType_0(Of String, String, String)
Return New VB$AnonymousType_0(Of String, String, String)(student.name, Me.closureVariable_11_C.local_gradeB, Me.local_type)
End Function
Public local_type As String
Public closureVariable_10_8 As _Closure$__1
Public closureVariable_11_C As _Closure$__2
End Class
End Class
End Class
If you closely look at the above compiler generated code, you can see that the "_Closure$__3" class; "_Lambda$__1" and "_Lambda$__2" functions are trying to gain access to the "_Closure$__1" and "_Closure$__2" local variables called "local_score" and "local_gradeB" respectively. Notice that for deep nested blocks nesting closure will continue recursively.
Lifetime
The lifetime of local variables in Visual Basic determines how long the storage for that variable exists in memory that is equal to the lifetime of the function where it is declared. But, in Visual Basic 9.0, closures allow the lifetime of a local variable to be extended beyond the lifetime of the function.
Imagine I want a list of students who got above 70 in math exam. Using query expression I would write like this:
Example 8
Delegate Function Func(ByVal score As Integer) As IEnumerable(Of String)
Function Example(ByVal score As Integer) As IEnumerable(Of String)
Dim results = From student In Students _
Where student.mathScore > score Select student.name
Return results
End Function
Dim temp As Func = New Func(AddressOf Example)
In the preceding code "score" is added to the closure referenced by the query expression inside Example function. Creating a delegate in the next line to this function keeps it alive as long as the delegate is alive and hence extends the lifetime of "score" beyond the normal lifetime of Example function.
Static Locals
A static local is a special type of local variable in Visual Basic, which allows a value to be retained from one call of a function to other. This can be thought of as a global variable as its value remains in memory for the lifetime of the program. The CLR does not support this, but a compiler does this with just a simple compiler trick by creating a shared class-level variable to maintain the value of the static local variable. As a result the variable does not need to be lifted into the closure. Consider the following example:
Example 9
Class StaticSample
Sub Example()
Static score As Integer = 97
Dim results = From student In Students _
Where student.mathScore >= score _
Select student.name
End Sub
End Class
In the preceding example compiler doesn't generate closure, but it does generate a lambda, that gets passed into the Select and Where extension methods. The generated lambda will be inside the outermost function class and the compiler generated code looks like the following:
Example 9 — Compiler Generated
Public Class StaticSample
Private Function _Lambda$__1(ByVal student As Student) As Boolean
Return (student.mathScore >= Me.score)
End Function
Private Shared Function _Lambda$__2(ByVal student As Student) As String
Return student.name
End Function
Public Sub Example()
Dim results As IEnumerable(Of String) = Students.Where(Of Student)(New Func(Of Student, Boolean)(AddressOf Me._Lambda$__1)).Select(Of Student, String)(New Func(Of Student, String)(AddressOf sample._Lambda$__2))
End Sub
Private score As Integer
End Class
As you can see, the compiler created a shared class level variable to maintain the value of "score". Therefore, the compiler didn't generate a closure class instead it did generate "_Lambda__1" and "_Lambda$__2" that gets passed into Where and Select extension methods.
Generics
VB.NET compiler does the same thing for generic locals as what it does for non-generics. Consider the following example:
Example 10
Sub Example(Of T As {Student, New})(ByVal arg As T)
Dim studentObject = arg
Dim results = From student In Students _
Where studentObject.name = student.name _
End Sub
Here, the compiler will generate a generic closure class, where a generic local gets lifted. The compiler generated closure class looks like the following:
Example 10 — Compiler Generated
Friend Class _Closure$__1(Of $CLS0 As { Student, New })
Public Function _Lambda$__1(ByVal student As Student) As Boolean
Return (Me.local_studentObject.name = student.name)
End Function
Public local_studentObject As $CLS0
End Class
As you can see in the code, the compiler generated a generic closure class called _Closure$__1, including the type constraints "Student" and "New".
Note that for every generic lifted variable, the compiler-generated closure class will have generic parameters equal to the captured variables and these type parameters will include any constraints that might be present on the generic local declaration.
Lifetime Shim
In Visual Basic and, of course, in most programming languages, the lifetime of a local variable is not the same as its scope, which is consistent with the way CLR works. You can simply observe this behavior in Visual Basic, that affects two parts of the language, i.e. declared but uninitialized variables and GoTo, with the following example:
Example 11
Sub Example()
For i = 0 To 2
Dim score As Integer
Console.WriteLine(score)
score += 2
Next
End Sub
The above code will print out: 0, 2, and 4.
You may think that the above statement would print out all 0's. Assuming that the lifetime of a local variable is the same as the scope of the local variable and you may expect that when the code reaches the end of the "For ... Next" block, both the scope and the lifetime of the local variable "score" ends. However, the lifetime of a local variable "score" is not the same as its scope instead it is always that of the entire function, even if the local variable "score" scope is block-level. Since "score" is not explicitly initialized, it would be created and initialized to 0 only once. Therefore, this implies that even if a block is entered more than once, the value of a local variable "score" will be retained from the previous iteration.
But if we include initialization when we declare local variable "score", the above statement would print out all 0's:
Example 12
Sub Example()
For i = 0 To 2
Dim score As Integer =0
Console.WriteLine(score)
score += 2
Next
Sub
The above code will print out: 0, 0, and 0.
But, what would happen if there is a query statement in between? Consider the following example that returns a number of students who scored 0 and 100.
Example 13
Sub Example()
For i = 0 To 1
Dim score As Integer
Dim results = From student In Students _
Where student.mathScore = score _
Select student.mathScore
Console.WriteLine(score)
score += 100
Next
End Sub
Since Closures are structured in Visual Basic based on the concept of scope/blocks, that would capture lifted variables called "score" declared in that block, there will no longer be a single instance of those variables per function, and as the result, every iteration will cause a new instance of the closure class to be created and the output would be all 0's.
But to fix this issue here is what the compiler basically does, when it enters a new scope containing a lifted variable, the compiler will check to see if an instance of closure already exists; if so, the compiler will create a new instance of closure and reset the value of the variable from the previous closure.
Note that the compiler only does the above check if it detects a loop or a GoTo in the function where the closure is generated.
Example 14
Sub Example()
For i = 0 To 1
Dim closureVariable_F_8 As _Closure$__1
closureVariable_F_8 = New _Closure$__1(closureVariable_F_8)
Dim score As Integer
Dim results = From student In Students _
Where student.mathScore = score _
Select student.mathScore
Console.WriteLine(score)
score += 100
Next
End Sub
And here is what the closure constructor looks like:
Example 15
Public Sub New(ByVal other As _Closure$__1)
If (Not other Is Nothing) Then
Me.local_score = other.local_score
End If
End Sub
So, the preceding examples show you what the compiler basically does to get around the issue.
Method Call
Method call is one of the possible things to do inside query comprehension. The following is a simple example of calling shared method.
Example 16
Class SharedExample
Shared Function GetGrade(ByVal score As Integer) As String
Select Case score
Case 90 To 100
Return "A"
Case 70 To 89
Return "B"
Case Else
Return "C"
End Select
End Function
End Class
Sub Example()
Dim results = From student In Students _
Select New With {Key .name = student.name, _
.grade = SharedExample.GetGrade(student.mathScore)}
End Sub
Method declared inside a Module or Shared method doesn't need an object instance like a regular method, but rather it is called directly. Therefore, there is no need to create a closure.
But, for instance method, the compiler lifts "Me" into closure as what it does for local variable lifting.
Example 17
Class InstanceExample
Function GetGrade(ByVal score As Integer) As String
Select Case score
Case 90 To 100
Return "A"
Case 70 To 89
Return "B"
Case Else
Return "C"
End Select
End Function
Sub Example(ByVal courseName As String)
Dim results = From student In Students _
Select New With {Key .name = student.name, _
.courseName = courseName, _
.grade = GetGrade(student.mathScore)}
End Sub
End Class
In the above case, the compiler generates closure and lifts "Me" and "courseName" into that closure. And here is what the compiler generated closure code looks like.
Example 17 — Compiler Generated
Public Class InstanceExample
Friend Class _Closure$__1
Public Sub New(ByVal other As _Closure$__1)
If (Not other Is Nothing) Then
Me.$VB$Me = other.$VB$Me
Me.local_courseName = other.local_courseName
End If
End Sub
Public Function _Lambda$__1(ByVal student As Student) As VB$AnonymousType_0(Of String, String, String)
Return New VB$AnonymousType_0(Of String, String, String)(student.name, Me.local_courseName, Me.$VB$Me.GetGrade(student.mathScore))
End Function
Public local_courseName As String
Public $VB$Me As InstanceExample
End Class
End Class
There are cases, of course, where the compiler is smart enough to see through your use and determine in either to generate closure or not, even if you call instance method inside Query Comprehension. For example, consider the following Query Comprehension inside "InstanceExample" class:
Example 18
Class InstanceExample
Function GetGrade(ByVal score As Integer) As String
Select Case score
Case 90 To 100
Return "A"
Case 70 To 89
Return "B"
Case Else
Return "C"
End Select
End Function
Sub Example()
Dim results = From student In Students _
Select New With {Key .name = student.name, _
.grade = GetGrade(student.mathScore)}
End Sub
End Class
In the above case the compiler doesn't generate closures, because the above query expression doesn't refer to any local variables inside Example method that need to be referred to by the compiler-generated lambda expression that can be passed as an argument for the "Select" extension method. Therefore, instead of emitting another wrapper class called closure, the compiler places the generated lambda expression inside an "InstanceExample" class. The following compiled code shows what the compiler basically does for the above query comprehension:
Example 18 — Compiler Generated
Public Class InstanceExample
Private Function _Lambda$__1(ByVal student As Student) As AnonymousType_0(Of String, String)
Return New AnonymousType_0(Of String, String)(student.name, Me.GetGrade(student.mathScore))
End Function
Public Sub Example()
Dim results As IEnumerable(Of AnonymousType_0(Of String, String)) =
Students.Select(Of Student, AnonymousType_0(Of String,
String))(New Func(Of Student, AnonymousType_0(Of String,
String))(AddressOf Me._Lambda$__1))
FakePre-93912ee065d948d19ea12591b424edb3-439730546dd0403bba49130b3f15de85FakePre-44853dd37fd14fcaabe2105b4114cf42-58672e4232ec447e8885519a4f273fd9
Therefore, as you can see, in the previous code example, there are cases where the compiler performs code optimization.
MyBase/MyClass
MyBase or MyClass are the other possible things to do inside Query Comprehension and will be captured into a closure. Let's see an example that uses MyBase to call to the base class method inside a query expression from a function that overrides this method.
Example 19
Module Module1
Public Class Base
Public Overridable Function Example(ByVal score As Integer) As String
Select Case score
Case 90 To 100
Return "A"
Case 70 To 89
Return "B"
Case Else
Return "C"
End Select
End Function
End Class
Class Derived
Inherits Base
Public Overrides Function Example(ByVal score As Integer) As String
Return Nothing
End Function
Public Function Sample(ByVal courseNumber As String) As String
Dim results = From student In Students _
Select New With {Key .name = student.name, _
.course = courseNumber, _
.grade = MyBase.Example(student.mathScore)}
End Function
End Class
End Module
Here is what the compiler does behind the scenes to support "MyBase". It generated a method and called mybase inside that method, and then it generated closure and lambda that gets passed to the Select extension method which uses this method and lifts "Me" and "courseNumber" into that closure. The compiler-generated code look like the following:
Example 19 — Compiler Generated
Friend NotInheritable Class Module1
Public Class Base
Public Overridable Function Example(ByVal score As Integer) As String
Dim t_i0 As Integer = score
If (IIf(((t_i0 >= 90) AndAlso (t_i0 <= 100)), 1, 0) <> 0) Then
Return "A"
End If
If (IIf(((t_i0 >= 70) AndAlso (t_i0 <= &H59)), 1, 0) <> 0) Then
Return "B"
End If
Return "C"
End Function
End Class
Public Class Derived
Inherits Base
Public Function Example_MyBase(ByVal p0 As Integer) As String
Return MyBase.Example(p0)
End Function
Public Overrides Function Example(ByVal score As Integer) As String
Return Nothing
End Function
Public Function Sample(ByVal courseNumber As String) As String
Dim Sample As String
Dim closureVariable_23_C As New _Closure$__1
closureVariable_23_C.$VB$Me = Me
closureVariable_23_C.courseNumber = courseNumber
Dim results As IEnumerable(Of VB$AnonymousType_0(Of String, String,
String)) = Module1.Students.Select(Of Student, VB$AnonymousType_0(Of String,
String, String))(New Func(Of Student, VB$AnonymousType_0(Of String,
String, String))(AddressOf closureVariable_23_C._Lambda$__1))
Return Sample
End Function
Friend Class _Closure$__1
Public Function _Lambda$__1(ByVal student As Student) As
VB$AnonymousType_0(Of String, String, String)
Return New VB$AnonymousType_0(Of String, String, String)(student.name,
Me.$VB$Local_courseNumber, Me.$VB$Me.Example_MyBase(student.mathScore))
End Function
Public courseNumber As String
Public $VB$Me As Derived
End Class
End Class
Let's see another example for the case of "MyClass" that allows you to call an overridable method of the current instance inside a query expression:
Example 20
Module Module1
Public Class Base
Public Overridable Function Example(ByVal score As Integer) As String
Select Case score
Case 90 To 100
Return "A"
Case 70 To 89
Return "B"
Case Else
Return "C"
End Select
End Function
Public Sub Sample(ByVal courseNumber As String)
Dim results = From student In Students _
Select New With {Key .name = student.name, _
.course = courseNumber, _
.grade = MyClass.Example(student.mathScore)}
End Sub
End Class
Class Derived
Inherits Base
Public Overrides Function Example(ByVal score As Integer) As String
Return Nothing
End Function
End Class
End Module
In the same way as for "MyBase", the compiler generates a method and calls "MyClass" inside that method, and then generates closure and lambda that gets passed to Select extension method which uses this method and lifts "Me" and "courseNumber" into that closure. Below is the compiler generated code:
Example 20 — Compiler Generated
Friend NotInheritable Class Module1
Public Class Base
Public Function Example_MyClass(ByVal p0 As Integer) As String
Return Me.Example(p0)
End Function
Public Overridable Function Example(ByVal score As Integer) As String
Dim t_i0 As Integer = score
If (IIf(((t_i0 >= 90) AndAlso (t_i0 <= 100)), 1, 0) <> 0) Then
Return "A"
End If
If (IIf(((t_i0 >= 70) AndAlso (t_i0 <= &H59)), 1, 0) <> 0) Then
Return "B"
End If
Return "C"
End Function
Public Sub Sample(ByVal courseNumber As String)
Dim closureVariable_35_C As New _Closure$__1
closureVariable_35_C.courseNumber = courseNumber
closureVariable_35_C.$VB$Me = Me
Dim results As IEnumerable(Of VB$AnonymousType_0(Of String, String,
String)) = Module1.Students.Select(Of Student, VB$AnonymousType_0(Of
String, String, String))(New Func(Of Student, VB$AnonymousType_0(Of String,
String, String))(AddressOf closureVariable_35_C._Lambda$__1))
End Sub
Friend Class _Closure$__1
Public Function _Lambda$__1(ByVal student As Student) As
VB$AnonymousType_0(Of String, String, String)
Return New VB$AnonymousType_0(Of String, String, String)(student.name,
Me.courseNumber, Me.$VB$Me.Example_MyClass(student.mathScore))
End Function
Public courseNumber As String
Public $VB$Me As Base
End Class
End Class
Public Class Derived
Inherits Base
Public Overrides Function Example(ByVal score As Integer) As String
Return Nothing
End Function
End Class
End Class
The preceding examples are to show you what the compiler basically does to use "MyBase" or "MyClass" inside Query Comprehension.
Restrictions
There are some restrictions you should be aware of regarding lifting variables.
ByRef Parameters
Any reference to a ByRef parameter inside query comprehensions will cause a compile time error. Because, a ByRef parameter allows the value of the argument to be changed in the calling routine. Therefore, it is not legal to arbitrarily extend the lifetime for a ByRef variable.
Example 21
Sub Example(ByRef score As Integer)
Dim results = From student In Students _
Where student.mathScore < score Order By student.name _
Ascending Select student.name
End Sub
For the above query comprehension, compiler generates the following error:
"error BC36533: 'ByRef' parameter 'arg' cannot be used in a query expression."
"Me" and Structures
Query Comprehension inside a structure will not be allowed to lift "Me", because closure will extend the lifetime of the lifted variable that includes "Me". But, Closure will hold value by reference and also structure is created on the stack. So, it's not possible to lift "Me" of the structure by reference and extend its lifetime in that manner.
Example 22
Structure Struct
Private age As Integer
Public Sub Example()
Dim results = From student In Students Select _
New With {.age = age, .name = student.name}
End Sub
End Structure
Therefore, the above query statement will cause a compile-time error:
"BC36535: Instance members and 'Me' cannot be used within query expressions in structures."
GoTo
Compiler doesn't allow to GoTo into scope that contains closure.
Example 23
Sub Example()
GoTo lable1
While True
Dim score As Integer = 90
lable1:
Dim results = Aggregate student In Students _
Where student.mathScore > score Into Count()
Exit While
End While
End Sub
In my earlier example, in the Lifetime shim section, there is an extra work the compiler has to do here. The closure has to be initialized before the local variable "score" is accessed. Therefore, allowing this will make it difficult for the compiler to maintain the previous task. As a result, the above query statement will cause a compile-time error:
"error BC36597: 'GoTo lable1' is not valid because 'lable1' is inside a scope that defines a variable that is used in a lambda or query expression."
Restricted Types
There are some types in the CLR that are restricted types based on where they can be placed and how they can be used, and this placement restriction prevents them from being declared or used as a member field in a class as such. It is not possible to lift this variable. Trying to do so will result in a compile-time error.
Example 24
Sub Example()
Dim arg As New ArgIterator
Dim results = From student In Students _
Where arg.GetRemainingCount > 0 Select student.name
End Sub
For the above query expression, compiler generates the following error:
"error BC36598: Instance of restricted type 'System.ArgIterator' cannot be used in a query expression."
Conclusion
Query Comprehension provides a concise and compositional expression of query with the basic query capabilities that includes different pre-defined query operator keyword such as projection, selection, cross-product, grouping and sorting, which enables you to write SQL like queries directly in your VB code with better IntelliSense experience in the IDE. And query Comprehension is supported by other new features such as Type Inference, Extension Method, Lambda Expression, Anonymous Types and Lexical Closure. Lexical Closure is the main purpose of this article, which is not an immediately visible feature to you and also is not intended to be used directly from your code, but rather it is under the hood compile feature. But knowing and understanding what the compiler do behind the scene will help you to write simple, correct and fast query statement directly from your VB.NET code.
Author
Binyam Kelile is a software development engineer with the Visual Basic test team at Microsoft.