Basic Instincts
Programming I/O with Streams in Visual Basic .NET
Ted Pattison
Contents
What is a Stream?
Programming with Streams
Text-Based I/O Using a Reader and Writer
Stream-Based Programming with XML
Using Streams with Network-Based Programming
Wrap-Up
If you are just now migrating from Visual Basic® 6.0 to Visual Basic .NET, be prepared for a change, because the way you program file I/O is very different. The good news is that an idiosyncratic, proprietary approach has been replaced by an elegant and reusable one that has become an industry standard.
I/O programming changes in both Visual Basic .NET and in the Microsoft® .NET Framework represent far more than just new syntax and method names. There's a complete paradigm shift because everything is based on the concept of a stream.
In this column, I will introduce stream-based programming and the .NET Framework classes associated with it. Along the way, I will walk through several practical scenarios. You'll see that streams create a consistent I/O programming style regardless of the source or destination of your data.
What is a Stream?
A stream is an object that represents a generic sequence of bytes. Any type of data can be stored as a sequence of bytes, so the details of writing and reading data can be abstracted. For example, you can write a generic piece of code that writes an XML document to a stream. Then you can use that code in an application to write the XML document to several different types of streams, such as an in-memory buffer, a file, or even a network connection.
While a stream object works at the byte level, it's fortunate that programmers aren't typically required to write and read data on a byte-by-byte basis. Imagine how hard it would be to take the data associated with a high-level document, such as an invoice, and transform it into a byte array. While that sounds like a challenging task, it would be even more difficult to write the code to reassemble the invoice from the byte array when it's time to present this data to the user.
The .NET Framework Class Library includes several complementary pairs of reader and writer objects that improve the usefulness and ease of use of streams. Figure 1 illustrates the typical relationship between writer objects, stream objects, and reader objects.
Figure 1** Reader and Writer Objects **
To write data into a stream in an efficient manner, create a writer object and initialize it so it is associated with a target stream object. A writer object exposes high-level methods (such as WriteLine) that allow you to think in terms of high-level data structures (such as strings) instead of concerning yourself with the nitty-gritty details of actually writing bytes. Behind the scenes, the writer object does the work of crunching the data into a byte array and writing it into the target stream.
When it's time to retrieve the data, you simply call high-level methods on the reader object (for example, ReadLine). The reader object responds by fetching bytes from the stream and reassembling the data back into its original form.
Before looking at any code, let's discuss where stream objects come from. The .NET Framework Class Library contains a Stream class in the System.IO namespace. The Stream class itself is an abstract class (a MustInherit class), meaning it cannot be directly instantiated to create an object. Its sole purpose is to serve as a base class for other Stream-derived classes. Actual stream objects must be instantiated from creatable classes that inherit either directly or indirectly from the Stream class.
A key point is that the Stream class has been designed to provide a common implementation and a standard programming contract for every stream object. The .NET Framework provides several concrete implementations of the Stream class, a subset of which are shown in Figure 2. As you can see, each of these Stream classes inherits from the Stream class and extends it to provide a unique implementation of a stream.
Figure 2** Inheriting from the Stream Class **
For example, you can use the MemoryStream class to create stream objects to store and retrieve data from an in-memory buffer. You can use the FileStream class to create stream objects to read or write data in a file on the local file system. You can also use stream objects created from one of the network stream classes to read or write the body of a network message.
Programming with Streams
Let's start with an example of programming against a stream in the raw without using a reader or a writer. While this style of programming doesn't yield high levels of productivity, this simple example will begin to build your understanding of stream-based programming from the ground up.
Examine the code in Figure 3. It creates and initializes a MemoryStream object with a capacity of four bytes. The code then writes four bytes of data into the MemoryStream, one byte at a time. You can imagine how programming at the byte level would become increasingly tedious and inefficient as the complexity of the data structure increases.
Figure 3 Memory Stream Object
'*** create memory stream object Dim stream1 As Stream = New MemoryStream(4) '*** write data to stream stream1.WriteByte(2) stream1.WriteByte(4) stream1.WriteByte(6) stream1.WriteByte(8) '*** clear stream buffer (if it has one) stream1.Flush() '*** read data from stream stream1.Position = 0 Dim result As Integer result = stream1.ReadByte() Do While result <> -1 Console.WriteLine(result) result = stream1.ReadByte() Loop '*** close stream as soon as work is complete stream1.Close()
Next, note the call to the Flush method of the Stream class. This is a method used with buffered streams (not all streams are buffered) to proactively push any data from the buffer to its destination or to clear the buffer.
Now, examine the code that reads data from the stream. You should notice how the code sets the Position property back to 0 before moving through the array of bytes. As you can see, some stream classes, such as MemoryStream, support cursor-like navigation where your code can move through the byte array, one byte at a time, and extract the data using the ReadByte method.
Finally, notice that the Close method is called as soon as the application is finished using the stream object. Keep in mind that many types of stream objects open expensive resources. You should get in the habit of closing a stream as soon as you are finished using it. For example, the call to the Close method on a FileStream object is what releases the OS-level lock on the file and allows the file to be accessed by other stream objects and by other applications.
Text-Based I/O Using a Reader and Writer
Now that you are familiar with programming a stream in the raw, it's time to introduce a writer object and a reader object to make things easier when you are programming text-based I/O. In particular, you can write text to a stream and read it back using a pair of classes that exist in the System.IO namespace named StreamWriter and StreamReader.
The StreamWriter and StreamReader classes inherit from two other abstract base classes: TextWriter and TextReader. The class hierarchy has been designed like this so you can write generic I/O code using TextWriter and TextReader that can write and read text to and from either a stream object or a StringBuilder object. If you wanted to work with text inside a StringBuilder object, you would then use the StringWriter class and the StringReader class instead of the StreamWriter class and StreamReader class, respectively.
The code in Figure 4 shows an example of a generic method named WriteSpecialMessage that accepts a single parameter of type TextWriter. This code demonstrates the polymorphic abilities of the TextWriter base class, because the WriteSpecialMessage method can be used to write text to either a stream object or a StringBuilder instance.
Figure 4 WriteSpecialMessage
Imports System Imports System.IO Imports System.Text Class MyApp '*** can be used to write to Stream or StringBuilder objects Shared Sub WriteSpecialMessage(ByVal writer As TextWriter) writer.WriteLine("Hello") writer.WriteLine("Goodbye") End Sub Shared Sub Main() '*** write text message to Stream Dim stream1 As New MemoryStream() Dim writer1 As New StreamWriter(stream1) WriteSpecialMessage(writer1) writer1.Close '*** write text message to StringBuilder Dim builder1 As New StringBuilder() Dim writer2 As New StringWriter(builder1) WriteSpecialMessage(writer2) writer2.Close End Sub End Class
Now let's move ahead to a typical scenario in which you need to create a new text file on the local file system. You can start by using the FileStream class, which provides support for general file I/O. The following code shows an example of how to create and initialize a FileStream object using a constructor that accepts three parameters—a file path, a file mode, and a file access setting:
Dim fs As New FileStream("MyData.txt", _ FileMode.Create, _ FileAccess.Write)
This code creates a new file in the current directory and opens it up for write access. Note that the System.IO namespace contains two utility classes, named File and Path, that make it easier to open files, find and search through directories, generate file paths, and determine whether a file already exists.
Once you have opened the file by creating and initializing a FileStream object, you can create a new StreamWriter object to write text into the file. When you create a StreamWriter object, you associate it with a stream by passing a reference to the target stream object in the StreamWriter constructor. Figure 5 shows how to use a StreamWriter object together with a FileStream object. Note that the call to the Flush method on a StreamWriter object is required to push the data into the destination stream and a call to the Close method on the FileStream object closes the file and releases the OS-level lock.
Figure 5 StreamWriter Object and FileStream Object
Dim fs As New FileStream("MyData.txt", FileMode.Create, FileAccess.Write) '*** now create and initialize a writer object Dim writer As New StreamWriter(fs) '*** now use the writer object writer.WriteLine("Hello") '*** call Flush on writer when writing is done writer.Flush() '*** call Close on stream to save and close file fs.Close()
Reading text is very similar and also fairly easy. You create another FileStream object, except this time, you initialize it for read-only access. Instead of creating a StreamWriter object, you create a complementary StreamReader object. Figure 6 uses a StreamReader object to read text from the file line by line.
Figure 6 Reading Line by Line with StreamReader
Dim fs As New FileStream("MyData.txt", FileMode.Open, FileAccess.Read) Dim reader As New StreamReader(fs) '*** read text out of file one line at a time Do Until reader.Peek() = -1 Console.WriteLine(reader.ReadLine) Loop '*** make sure to close stream when work is complete fs.Close()
If you don't want to read text from files on a line-by-line basis, you can retrieve the entire contents of a text file as a single string value using a StreamReader object's ReadToEnd method.
As mentioned previously, the File class provides some useful shared methods to simplify access to files. Two of these methods, OpenRead and OpenWrite, make it easy to open files as I've been doing, but with less code. For example, instead of using the following code to open MyData.txt for reading:
New FileStream("MyData.txt", FileMode.Open, FileAccess.Read)
I can use the equivalent:
File.OpenRead("MyData.txt")
Stream-Based Programming with XML
Now let's perform I/O with something more complex: an XML document. As you will see, the programming techniques revolve around a similar set of objects. Imagine you are required to read an XML document that conforms to the following structure:
<?xml version="1.0" encoding="utf-8" ?> <Dancers> <Dancer>Fritz</Dancer> <Dancer>Keith</Dancer> <Dancer>Ted</Dancer> <Dancer>Aaron</Dancer> </Dancers>
The fastest way to read this data from its XML format is to use a stream-based technique based on the XmlTextReader class defined in the System.Xml namespace. When you create a new XmlTextReader object, you can explicitly initialize it with a FileStream object and a StreamReader object:
'*** create and initialize stream object Dim stream As New FileStream("MyData.xml", _ FileMode.Open, _ FileAccess.Read) '*** create and initialize StreamReader using stream Dim reader1 As New StreamReader(stream) '*** create and initialize XmlTextReader using StreamReader Dim reader2 As New XmlTextReader(reader1)
Of course, there is an easier way to accomplish the same goal. You can create and initialize an XmlTextReader object using a string pointing to the target file path. When you do this, the XmlTextReader object automatically creates and manages a FileStream object and a StreamReader object behind the scenes:
'*** this code does exactly what the previous code does. Dim reader As New XmlTextReader("MyData.xml")
Once you have initialized an XmlTextReader object with a specific XML file, you can begin to access the data inside using a stream-based approach. You can move forward node by node to read the contents of the XML document, but you cannot move backwards. The code that is shown in Figure 7 reads data from an XML file using this particular approach.
Figure 7 Using XmlTextReader
Dim reader As New XmlTextReader("MyData.xml") '*** determine how XmlTextReader deals with whitespace. reader.WhitespaceHandling = WhitespaceHandling.None '*** move through XML content node by node reader.MoveToContent() Do While reader.Read() Select Case reader.NodeType Case XmlNodeType.Element Console.Write(reader.Name & ": ") Case XmlNodeType.Text Console.WriteLine(reader.Value) End Select Loop '*** close reader when done to release resources reader.Close()
Using Streams with Network-Based Programming
So far you have seen how streams allow you to read and write into memory buffers as well as files. Now I would like to move on to other scenarios in which you are required to write and read the body of network messages. My goal here is to demonstrate how stream-based programming for the network is consistent with what you have already seen in this column.
Imagine you are writing a .NET-based client app that needs to retrieve data from a Web server using HTTP. For client-side HTTP programming, the .NET Framework provides classes in the System.Net namespace that let you submit HTTP requests to a Web server using operations such as GET and POST.
Using this approach, you can use a WebRequest object to transmit GET and POST operations to a Web server. Once you have initialized the WebRequest object, you call a method named GetResponse to send the HTTP request across the network to the Web server. The GetResponse method doesn't return until a response is received from the Web server. When the GetResponse method does complete, it returns a reference to a WebResponse object, which allows the client application to retrieve data sent by the Web server by providing stream-based access to the body of the HTTP response message.
Now take a look at the code in Figure 8, which sends an HTTP request using a GET operation and then retrieves text from the body of the HTTP response message returned from the Web server. As you can see, you create a WebRequest object using a shared factory method named Create. Once the object has been created, you can determine the type of HTTP operation by assigning a string value, such as GET or POST, to the Method property. You will also need to add security code at this point for authenticating the client application, unless you can make the assumption that all requests to the Web server can be made using anonymous access.
Figure 8 Using a GET
Dim req As WebRequest req = WebRequest.Create("https://LocalHost/TestApp/CustomerList.aspx") req.Method = "GET" '*** submit synchronous HTTP request to Web server Dim rsp As WebResponse = req.GetResponse '*** WebResponse provides stream-based access to body Dim str As Stream = rsp.GetResponseStream Dim reader As New StreamReader(str) Console.WriteLine(reader.ReadToEnd()) reader.Close() rsp.Close()
When you call GetResponse, the WebRequest object responds by submitting an HTTP request message to the Web server. GetResponse is a synchronous method that blocks for the duration of the trip to the Web server and back. When the HTTP response message is returned from the Web server, the GetResponse method completes and returns a WebResponse object to its caller.
When you call GetResponseStream, the WebResponse object returns a reference to a stream object that allows you to access the body of the incoming HTTP response message. This, in turn, lets you use stream-based techniques, such as the ones that use a reader object to work with text and XML. A high-level view of how all these pieces fit together is shown in Figure 9.
Figure 9** Stream-Based Access to the HTTP Response **
Before I conclude, I'd like to shift focus to the server-side code written to handle an incoming HTTP request in an ASP.NET page. While the classes you will use in this scenario have been written by the ASP.NET team, they offer a consistent programming approach because they too are based on streams.
When an ASP.NET page is processed in response to an incoming HTTP request, the body of the HTTP request message is exposed through the InputStream property of the ASP.NET HttpRequest object. Likewise, you can write into the body of the HTTP response message that will be sent back to the client by using the OutputStream property of the ASP.NET HttpResponse object. Figure 10 is an ASP.NET page that uses a stream-based technique involving the OutputStream property and a StreamWriter object to send a few lines of text back to the client.
Figure 10 Using OutputStream and StreamWriter
<% @Page Language="vb" %> <% @Import namespace="System.IO" %> <script runat=server> Sub Page_Load(ByVal sender As Object, ByVal args As EventArgs) WriteCustomerList(Response.OutputStream) End Sub Sub WriteCustomerList (ByVal str As Stream) Dim writer As StreamWriter = New StreamWriter(str) writer.WriteLine("Nancy Davolio") writer.WriteLine("Andrew Fuller") writer.WriteLine("Ted Pattison") writer.Flush() End Sub </script>
As you can see, the generic method WriteCustomerList can be used to write text into any type of stream. You can take this example even further and develop ASP.NET pages and custom HttpHandler objects that use the OutputStream property to write other types of data, such as XML documents and binary data, into the body of the HTTP response message.
Wrap-Up
This month I took an introductory look into programming I/O in the .NET Framework with Visual Basic .NET. As you have seen, all I/O work is built on top of the concept of a stream. Streams are valuable because they abstract away the details of where your data is going to and coming from. You can code generic routines that can be used to write and read binary data, text, and XML data to and from memory buffers, files, and network messages.
You have also seen how you can complement streams by using a pair of writer and reader objects, which simplify stream-based I/O when you are working with more complicated kinds of data structures. You should now be ready to augment your I/O programming repertoire with streams.
Send your questions and comments for Ted to instinct@microsoft.com.
Ted Pattison is an author and trainer who delivers hands-on training through PluralSight, provides consulting through Ted Pattison Group, and is the author of several books.