Background Copying

 

Duncan Mackenzie
Microsoft Developer Network

February 28, 2003

Summary: Duncan Mackenzie describes how to build a tool that uses the Background Intelligent Transfer Service features of Microsoft Windows XP to download large files over slow or intermittent links. (15 printed pages)

Applies to:
   Microsoft® Visual Basic® .NET
   Microsoft® Windows® Server 2003
   Microsoft® Windows® XP

Download the source code for this article

Hotels, Modems, and Coffee Shops

I'm sitting in a hotel room in San Francisco working on a project that can't wait until I get back to my office, and all I can think about is my reliance on connectivity. I, and a bunch of other happy developers, am here because of the VSLive/VBits conference. Trouble is, I didn't bring everything I need for my work, so now I desperately need a network connection. Sure, I have dial-up access in the hotel room, but though quick enough for e-mail, it's anything but practical for large file transfers.

Sound like a familiar situation? Well, it happens to me all the time, and usually it's a pain. This time, however, I decided I'm not going to be beaten by mere, piddling circumstances. I am, after all, a developer. Not only that, I'm a developer looking for a good time (developing that is). I'm going to solve this problem—and turn the solution into this month's article. (Which I have just decided is about using the Background Intelligent Transfer Service features of Microsoft® Windows® XP to download large files over slow or intermittent links.)

Okay. So let's say there is a file that I need and I can't wait until I'm back in Redmond. For instance, let's say it is the demo for Impossible Creatures (which is a whopping 285 MB), and that I really, really need it for some serious research. As I said, my dial-up access is slower than a dead snail, but there is wireless in a nearby coffee shop that should be considerably faster. Even so, as much as I like coffee, I couldn't possibly sit there sipping caffè mochas while the entire file downloads—not if I ever want to sleep again. Not only that—what if the wireless connection dropped for a moment, or I decided to actually attend a session at the conference (which would require leaving the coffee shop and probably shutting down my machine)? What would happen to my mission-critical download then?

Well, that's precisely the problem the Background Intelligent Transfer Service (BITS) was built to solve. Well, okay, maybe not for this precise problem, but BITS was created to download large files across intermittent and/or slow network connections, and even allow the machine to be shut down without losing the download. I had worked with the BITS library before, and I even wrote a .NET wrapper for it (in an earlier article on MSDN), but I had never used it for ad hoc downloads. What I needed now was a system that would allow me to add "jobs," which is the BITS term for a set of files to be downloaded from a Web server, whenever I find something on the Web that I want to pull down. A console application would be enough to get my game demo download started, but I'd prefer to extend it into a system capable of more general use.

Note   For more information on BITS, check out the detailed reference material available on MSDN.

The first thing I needed to do was to compile a list of the features that would make this utility useful to people besides me:

  1. Drag-and-drop feature for dragging links to start a background copy.
  2. Display progress of copies.
  3. Cancel jobs.
  4. Set default receive folder.
  5. Create a job by manually entering a URL and a destination.

To save time, I grabbed the "Browse for Folder" component from Microsoft's support Web site (part of Microsoft Knowledge Base article 306285), which helped me with requirement 4 from the list above. For the rest, the wrapper provided all of the functionality I needed for working with BITS, so I was left to focus on the user interface. I needed to display the list of jobs currently in progress, provide support for dragging links from the Web browser, and allow the user to configure their desired download location.

The DataGrid Is Your Friend

This is one of the few applications I have created that does not involve a database, but I still used data binding to build my interface. With Microsoft® Windows® Forms applications, you can bind to arrays and collections in addition to databases, so all I really needed was a collection of background copy jobs in progress. The BITS wrapper already provides a collection that contains the current set of background copy jobs, but I decided to create new classes that wrapped the raw BITS information into a display-friendly format.

I created a class to encapsulate a single BITS "job" and a strongly typed collection class to hold a list of all my jobs. Then I bound a data grid to that collection. I wanted to display the copy progress graphically, and although I probably could have found a progress component on the Web, I went ahead and built one of those from scratch. I still had to add a couple of buttons, a main menu, and a context menu, but the grid is the heart of this user interface (see Figure 1).

Figure 1. The copy utility attempts to look professional, no matter what it is downloading

Once I add drag-and-drop support, and the ability to set a default save location for downloaded files, the application will essentially be ready to use.

Dragging and Dropping All Over the Place

The feature that really makes this utility usable is the ability to drag links in from your browser. Without this, you would have to copy and paste your links or type them in manually, and neither is a very enjoyable user experience. In general, URLs are not designed for use by humans; whenever possible, do not make people type them into your application(s).

Implementing a link drag-and-drop feature in your own application takes only a few steps. First, set the AllowDrop property of the target control (the control that will accept dragged objects) to True. Next, you will need to add code in the DragDrop event for that control, which will check whether the dragged object is of the desired type, and process the link (in this case, adding it as a BITS job).


'm_Options is my user settings class, discussed later...
Private Sub dgJobs_DragDrop(ByVal sender As Object, _
        ByVal e As DragEventArgs) _
    Handles dgJobs.DragDrop

    Try
        If e.Data.GetDataPresent(DataFormats.Text, _
                True) Then
            e.Effect = DragDropEffects.Link
            'should contain the URL if a link is 
            'dragged in from IE
            Dim sURL As String = _
                e.Data.GetData(DataFormats.Text, True)
            Dim myURI As New Uri(sURL)
            Dim fileName As String = _
                Path.GetFileName(myURI.LocalPath)
            Dim localPath As String = _
                Path.Combine( _
                        Me.m_Options.defaultSaveLocation, _
                        fileName)

            Dim newJob As BITS.Job
            newJob = Me.m_BITS.CreateJob(fileName)
            newJob.AddFile(localPath, sURL)
            newJob.ResumeJob()

            Dim myJob As New BITSJob(m_BITS, newJob.ID)
            myJob.Refresh()
            m_Jobs.Add(myJob)

            CType(Me.BindingContext(dgJobs.DataSource), _
                CurrencyManager).Refresh()
        Else
            e.Effect = DragDropEffects.None
        End If
    Catch ex As Exception
        e.Effect = DragDropEffects.None
        MsgBox(ex.ToString)
    End Try
End Sub

In addition to DragDrop, I also handle the DragEnter event where I check the type of data being dragged and indicate whether I am able to accept it. Setting the DragEventArgs.Effect property will change the cursor displayed during the drag operation.

Private Sub dgJobs_DragEnter(ByVal sender As Object, _
        ByVal e As DragEventArgs) _
    Handles dgJobs.DragEnter

    Try
        If e.Data.GetDataPresent(DataFormats.Text, True) Then
            e.Effect = DragDropEffects.Link
        Else
            e.Effect = DragDropEffects.None
        End If
    Catch ex As Exception
        MsgBox(ex.ToString)
    End Try
End Sub

With the drag-and-drop feature available, I expect most jobs will be created using this technique, but it does not cover all situations. To cover occasions where you do not have a direct link but you can discover the appropriate URL, I added a dialog and associated menu option for manually creating jobs (see Figure 2).

Figure 2. A dialog allows jobs to be created manually.

This dialog is a good example of building and using a standard dialog in Microsoft® Visual Basic® .NET. You create an instance of the Form, set up its properties, display it modally, and then retrieve its properties once it has been closed.


'm_Options is my user settings class, discussed later...
Private Sub addNew_Click(ByVal sender As Object, _
        ByVal e As System.EventArgs) _
        Handles addNew.Click
    Dim newJobDialog As New addNewJob()
    newJobDialog.Options = Me.m_Options
    Dim newJobResult As DialogResult

    newJobResult = newJobDialog.ShowDialog

    If newJobResult = DialogResult.OK Then

        Dim sURL As String = _
            newJobDialog.SourceURL

        Dim myURI As New Uri(sURL)
        Dim localPath As String = newJobDialog.Target
        Dim fileName As String = _
            Path.GetFileName(localPath)

        Dim newJob As BITS.Job
        newJob = Me.m_BITS.CreateJob(fileName)
        newJob.AddFile(localPath, sURL)
        newJob.ResumeJob()

        Dim myJob As New BITSJob(m_BITS, newJob.ID)
        myJob.Refresh()
        m_Jobs.Add(myJob)

        CType(Me.BindingContext(dgJobs.DataSource), _
            CurrencyManager).Refresh()

    End If
End Sub

In addition to retrieving and providing information to a dialog, the New Job form also illustrates the use of regular expressions to validate the source URL and the use of the SaveFileDialog component to allow the user to pick a location for the downloaded file.


'Validate Source URL
Private Function ValidURL( _
    ByVal sourceURL As String) As Boolean
    Dim urlValidator As String = _
    "http:\/\/[\w]+(.[\w]+)([\w\-\.," & _
        "@?^=%&:/~\+#]*[\w\-\@?^=%&/~\+#])?"
    Dim r As New Regex( _
        urlValidator)
    Return r.IsMatch(sourceURL.Trim)
End Function

'Pick Save Location
'm_Options is my user settings class, discussed later...
Private Sub browseForFile_Click( _
        ByVal sender As System.Object, _
        ByVal e As System.EventArgs) _
    Handles browseForFile.Click

    'Pull filename out of Source URL
    Dim fileName As String
    If ValidURL(Me.Source.Text) Then
        Dim source As New Uri(Me.Source.Text)
        'get file name
        fileName = Path.GetFileName(source.LocalPath)
    Else
        fileName = ""
    End If

    'saveDestinationAs is a SaveFileDialog control on Form
    Me.saveDestinationAs.FileName = fileName
    If Not m_Options Is Nothing Then
        Me.saveDestinationAs.InitialDirectory = _
            m_Options.defaultSaveLocation
    End If

    If Me.saveDestinationAs.ShowDialog() _
            = DialogResult.OK Then
        Me.Destination.Text = saveDestinationAs.FileName
    End If
End Sub

Remembering User Settings

This application only has one user-configurable option—the default directory for downloads—but the concepts illustrated in this sample could apply equally well to a more complicated set of configuration options. By using a class to hold my user settings, I can take advantage of the built-in serialization features of Microsoft® .NET and save/restore the entire Options class with a few simple lines of code. Serialization is how I will save my settings, but where should I save them? There are two main locations recommended for saving user-specific configuration: the user's application data folder and isolated storage. For this example, I decided to use Isolated Storage, which allows me to create files without any concern about where those files will be located or whether they will conflict with the setting files for another program. Using the classes available in the System.IO.IsolatedStorage namespace combined with the System.Runtime.Serialization classes, I can save and restore my serialized settings specifying only a file name of "settings.xml". There is a bit more code in my listing than you would require for just saving and retrieving a settings object, but I decided to pull the default download directory of Microsoft® Internet Explorer out of the registry for use as a default.


Imports System.IO
Imports System.IO.IsolatedStorage
Imports Microsoft.Win32

Public Class UserSettings

    Private Shared Function _
        GetIEDefaultSaveLocation() As String
        'When you download a file from IE, 
        'it defaults to the last location 
        'you downloaded a file to, this code 
        'just pulls that location out of the 
        'registry. If the setting isn't available,
        'the user's My Documents folder is used
        'as a default.
        Dim saveDir As String
        Try
            Dim IEMain As RegistryKey
            IEMain = Registry.CurrentUser.OpenSubKey _
                ("Software\Microsoft\Internet Explorer\Main", _
                 False)

            saveDir = IEMain.GetValue("Save Directory", _
                                    Nothing)
            If saveDir Is Nothing Then
                'default to user's My Documents folder
                saveDir = Environment.GetFolderPath( _
                    Environment.SpecialFolder.Personal)
            End If
        Catch ex As Exception
            MsgBox(ex.ToString)
            saveDir = Directory.GetDirectoryRoot( _
                Application.StartupPath)
        End Try
        Return saveDir
    End Function

    Public Shared Function GetSettings() _
        As Options
        Try
            Dim m_Options As Options
            Dim settingsPath As String = "settings.xml"
            Dim isf As IsolatedStorageFile
            isf = IsolatedStorageFile.GetUserStoreForAssembly

            If isf.GetFileNames(settingsPath).Length > 0 Then
                Dim myXMLSerializer As New _
                    Xml.Serialization.XmlSerializer( _
                    GetType(Options))
                m_Options = CType( _
                    myXMLSerializer.Deserialize( _
                    New IsolatedStorageFileStream _
                    (settingsPath, IO.FileMode.Open, _
                        IO.FileAccess.Read)), Options)
            Else
                m_Options = New Options()
                m_Options.defaultSaveLocation = _
                    GetIEDefaultSaveLocation()
            End If
            Debug.WriteLine(m_Options.defaultSaveLocation)
            Return m_Options
        Catch ex As System.Exception
            MsgBox(ex.ToString)
            Return Nothing
        End Try
    End Function

    Public Shared Sub SaveSettings( _
        ByVal currentSettings As Options)
        Try
            Dim isf As IsolatedStorageFile
            isf = IsolatedStorageFile.GetUserStoreForAssembly

            Dim settingsPath As String = "settings.xml"
            Dim myXMLSerializer As _
                New Xml.Serialization.XmlSerializer( _
                GetType(Options))
            myXMLSerializer.Serialize( _
                New IsolatedStorageFileStream( _
                    settingsPath, IO.FileMode.Create, _
                    IO.FileAccess.ReadWrite), _
                currentSettings)
        Catch ex As System.Exception
            MsgBox(ex.ToString)
        End Try
    End Sub
End Class

<Serializable()> Public Class Options
    Dim m_defaultSaveLocation As String

    Public Property defaultSaveLocation() As String
        Get
            Return m_defaultSaveLocation
        End Get
        Set(ByVal Value As String)
            Try
                If Path.IsPathRooted(Value) AndAlso _
                    Not Path.HasExtension(Value) Then
                    If Not Directory.Exists(Value) Then
                        Directory.CreateDirectory(Value)
                    End If
                    m_defaultSaveLocation = Value
                End If
            Catch ex As Exception
                MsgBox(ex.ToString)
            End Try
        End Set
    End Property
End Class

By calling UserSettings.GetSettings when my application loads and UserSettings.SaveSettings when it exits, I can ensure that user preferences are correctly saved and restored. To actually change the settings of my utility, I could have used an Options dialog, but with only a single available setting ("Default Save Location") it seemed easy enough just to provide a "Set Default Download Location" menu item. In the click event for the menu item, I used the Browse for Folder component to allow the user to select an existing folder or to create and select a new folder.


Private Sub mnuSetDefaultLocation_Click( _
        ByVal sender As System.Object, _
        ByVal e As System.EventArgs) _
    Handles mnuSetDefaultLocation.Click

    If defaultSaveLocation.ShowDialog() = _
        DialogResult.OK Then
        m_Options.defaultSaveLocation = _
            defaultSaveLocation.DirectoryPath
    End If
End Sub

If I had been writing this application using the 1.1 version of the .NET Framework, then I could have used the FolderBrowserDialog class directly from the Windows Forms namespace, but I am building this sample using 1.0. Fortunately, the freely available component from Microsoft's support site works just fine, and since the price is right, I am quite happy to use it.

All Work and No Play?

The nature of BITS makes it well suited to many work scenarios, because it is useful anytime you need to transfer files over an intermittent network connection. Accordingly, this means that this utility is potentially a work tool. Unfortunately, this work connection can't really be avoided in this case. So if you happen to use the program more for downloading files over your company's VPN connection than for pulling down the latest game demos, that's okay; I won't tell anyone. For more ideas on how BITS can be used in work situations, check out this article from MSDN Magazine, which shows an application that updates itself through background downloading of new files. Of course, the most widely used example of the usefulness of BITS is in the Windows Update features of Windows XP, where BITS can be used to download patches, drivers, and software updates completely in the background.

Using the Copy Utility

The finished background copy utility includes a few more features than have been discussed in this article, but you can download it and dig through the code at your leisure. Although you can download the complete code for this little program, there is no user manual available, so let me give you a few tips about using this utility to download files:

  • Files must be accessible through a Web server; BITS depends on the ability of Web servers to return a specific range of bytes from a file.
  • Files must be static, due to the same need for a range of bytes. Dynamic content, such as an Active Server Page, does not support the Range header and cannot be downloaded using this method.
  • Files cannot require authentication to download. BITS 1.5 does provide the ability to supply credentials for a download, or to use NTLM authentication, but the wrapper I wrote is based only on features in version 1.0 of the BITS library.
  • Downloads will continue even if the utility is closed, even if the machine is restarted, and even if the network is disconnected (jobs will continue when a network connection becomes available again). BITS is handling the file copy, this utility is just adding jobs to the queue to be downloaded.
  • Fully transferred files have to be acknowledged to complete the download and make them available at the specified local path. The timer on this utility will acknowledge any fully transferred jobs, thereby completing the download, but this will only happen when the utility is running.
  • BITS is pre-installed on Microsoft Windows XP and Microsoft® Windows® Server 2003, but it will also be available for Microsoft Windows 2000 through a redistributable from the SDK Update site.

Coding Challenge

At the end of some of my Coding4Fun columns, I will have a little coding challenge—something for you to work on if you are interested. For this article, the challenge is to build your own application using the Background Intelligent Transfer Service (BITS) in some way. As I mentioned above, BITS lends itself to quite a few different work and personal scenarios, so I am sure you will be able to come up with interesting applications. Managed code is preferred (Visual Basic .NET, C#, J#, or Managed C++ please), but an unmanaged component that exposes a COM interface would also be good. Just post whatever you produce to GotDotNet and send me an e-mail message (at duncanma@microsoft.com) with an explanation of what you have done and why you feel it is interesting. You can send me your ideas whenever you like, but please just send me links to code samples, not the samples themselves (my inbox thanks you in advance).

In response to my Playing with Music Files column, I received quite a few comments and links to very cool sample applications. I appreciate the feedback, and I'll post links to a few of these samples in my next column. If you are looking for more digital-music–related development information right now, I would suggest looking into the managed code samples in the Windows Media Player SDK.

Have your own ideas for hobbyist content? Let me know at duncanma@microsoft.com, and happy coding!

 

Coding4Fun

Duncan Mackenzie is the Microsoft Visual Basic .NET Content Strategist for MSDN during the day and a dedicated coder late at night. It has been suggested that he wouldn't be able to do any work at all without his Earl Grey tea, but let's hope we never have to find out. For more on Duncan, see his profile on GotDotNet.