Questions and Answers at the XPE Corral
March 18, 2002
Here we are, another month, and another Microsoft® Windows® XP Embedded column for you. I know some of you were really looking forward to an in-depth exposé into the inner workings of component and configuration scripting. Well, I'm sorry to have to put this off yet another month, but I have to—we will not be discussing component scripting this month.
You see, the past month has been a busy one for me. About the time of the publication of my last column, I had the privilege of attending the Embedded Systems Fair in Nuremberg, Germany (a "herzlichen Guten Tag" to my German readers). I've also been fairly active monitoring and responding to issues you've posted in the newsgroups. I've even gotten feedback from customers from other channels here at Microsoft. Through all this customer feedback and interaction, I've noticed an interesting trend:
Hardly anyone is authoring components, and the people who are aren't worried about component scripting.
So, I've decided to dedicate this article to answering some of your burning newsgroup questions once and for all. If nothing else, this article should show that I am writing these as we go along—they weren't written months ago and just dragged up for publication each month.
One quick note: I want everyone who attended the Embedded Systems Fair in Nuremberg to know what a great time I had there. It was a pleasure to meet so many of our customers, and get to know the projects you're working on (and the headaches you're suffering with them; I hope I was able to help alleviate some of those). I had lots of fun in Nuremberg, both during and after the ESF, and have even planned a future return trip. For all those who made my stay challenging, productive, and very enjoyable (including those of you who kindly corrected my German), Danke schön.
Burning Question #1: Enhanced Write Filter
Well, unless you've been living under a rock somewhere (or at least not active in the Windows XP Embedded newsgroup), you know that Enhanced Write Filter (EWF for short) on compact flash cards is a hot issue. I got a lot of questions about this feature at the ESF as well. Let's divide this into two separate questions—how to setup EWF, and how to make it work on compact flash. But first:
What Is Enhanced Write Filter?
For those of you familiar with Write Filter under Windows NT Embedded 4.0, forget what you know. While the functionality and names are similar between Write Filter and Enhanced Write Filter, you'll get confused if you think of EWF in the same way as you did the Windows NT Embedded 4.0 Write Filter.
To understand EWF, I need to introduce the concept of an overlay. An overlay logically sits on top of the protected volume; all writes and reads to the protected volume go to the overlay first. The volume underneath is not written to, but may be read from. The overlay passes reads through if the desired information is not in the overlay.
So, where can the overlay live? In Windows NT Embedded 4.0, Write Filter cached everything in RAM—it ate as much memory as it needed, and when you ran out of memory, the system stopped. EWF, however, allows the overlay to live either in RAM or on a second volume; for instance, on a small hard disk partition. What are the main differences (other than where they are stored)?
RAM overlays are temporary. They are created when the machine boots, and are lost when the machine is restarted. They are very fast, as RAM access is much quicker than disk access. However, the size of the RAM overlay is restricted by the size of the non-paged memory pool, which varies from machine to machine.
Disk overlays are preserved between boots, so static information can be saved in the overlay. The information can also be committed from the overlay to the underlying media. Disk overlays are slower than RAM overlays, as they are replacing disk writes with other disk writes. The size of the overlay is set by the size of the EWF Volume rather than specified in Target Designer.
Both types of overlays share some commonality—they both disallow writes to the protected media when active. Both can be either enabled or disabled dynamically to allow updates to the protected media to occur. You can also have more than one overlay on each volume, and each overlay can be independently controlled in the runtime.
Why would you use EWF? Well, if you want to boot from an El Torito CD-ROM, you've got no choice—you have to use it to provide for normal operation of the system. (A normal Windows XP system does a number of registry writes simply booting up.) You would also use it when you want to protect your boot volume from unwanted writing, increase system robustness, or protect your boot volume from possible power outages. And in cases like we'll discuss below, compact flash devices benefit greatly from EWF. This is because compact flash deteriorates rapidly under heavy writing. EWF protects the flash volume from all writes, and extends the life of the flash device.
How to Configure Enhanced Write Filter
In order to use EWF, you need to add the "Enhanced Write Filter" component to your configuration. Make sure you re-check dependencies after adding it—it requires the EWFLDR component rather than NTLDR. Once you've added the component to your configuration, click Settings under it to configure it.
The top half of the Details pane shows settings that are in effect for all overlays. These include the number of protected volumes, the number of overlay levels, and the size of the EWF volume, which will hold the overlays.
The bottom half of the Details pane shows settings in effect for each particular protected volume. You can start with EWF enabled or disabled on the given volume, and enable lazy write mode (which means the data will be written in the background, rather than at the time of the write). Disk number, partition number, and disk type are very important—we need to know which volume to protect! You can also specify whether you want the overlay to be RAM- or disk-based for this volume. You can further specify the optimization of EWF here. (More details on these options are available in the System Design Guide for Windows XP Embedded.)
What About EWF on Compact Flash Devices?
Here's the main question people have: They configure EWF for their hard drives, and it works flawlessly. They then reconfigure for their flash devices, and EWF fails to load properly. There are a few reasons why this will happen.
With the exception of El Torito volumes, EWF will expect to find some sort of disk-based volume to store information in between boots for both RAM and disk-based overlays. In the case of a RAM overlay, this EWF volume can be as small as 32 KB—but it has to be there. The EWF volume is used to store enable/disable information between boots and other configuration information. There can only be one EWF volume present in a system; if EWF finds one during First Boot Agent (FBA) processing, it deletes the EWF volume and creates a new one. This EWF volume is a type 0x45 partition, and is essential to proper operation of EWF. You should leave enough raw volume space on your boot media to accommodate this EWF volume.
In addition to the above paragraph, remember that while you may have enough raw space on your volume for the EWF partition, you may not have a free entry in the partition table for us to create the EWF volume partition. There has to be a free partition entry in either the primary or extended partition table. The limit on partition table entries per volume is four in the primary partition table, and four in the extended partition table.
The second thing to remember is that EWF protects fixed disks only. You cannot enable EWF on removable media. Most compact flash devices are marked as removable devices, although some are not. EWF cannot handle the insertion and removal events that removable devices normally generate—not on the volumes it is protecting, nor on its EWF volume. If the system sees your compact flash as removable, EWF will too, and cannot load. This is a hardware issue that we cannot get around. If you find EWF isn't loading, check to make sure your compact flash device is not marked as removable.
A corollary to the above statement is that your compact flash device has to be configured as a normal IDE non-removable disk. Proprietary flash drivers aren't going to work here. Remember that the EWF driver is an upper driver on the storage volume stack. If your driver doesn't live in this stack, EWF won't be able to use it.
Burning Question #2: Problems Installing the Database
There were quite a few questions dealing with installation problems on the database machine. There are a couple of things to remember here.
Our installation process is fairly dependent on Windows Scripting Host 5.5. If you're putting the database on an older Windows 2000 machine, you should make sure you've updated Windows Scripting Host to version 5.5 before attempting the install.
One other big problem is installing our tools on a localized version of Windows 2000 or Windows XP (in other words, non-U.S. English builds). The problem here is that the installation routine looks for a group called "Everyone" when it's assigning share permissions to the repository folders. On some localized builds, the Everyone group has a different name (like "Jeder" in German builds). Our installation routine therefore can't find a group named Everyone, and fails. The solution to this has been posted a few times in the newsgroup: Create a temporary, local group named "Everyone" before doing the database install. When the install is complete, add permissions for your localized Everyone group to the shares that were created, using the same permissions as the install gave to the temporary Everyone group. You can then delete the Everyone group you created.
Burning Question #3: System Cloning and FBAReseal
One of the problems with deploying a single built Windows XP Embedded image onto hundreds of devices is that everything is the same—and that includes the System Identifier, or SID. However, one of the things you really don't want to be the same on hundreds of unique devices is the SID. That's because under the hood, Windows XP identifies each machine by the SID. Imagine hundreds of machines on the same network with the same SID; each of them thinks they're unique, but they're all the same.
It's important that each unique device have a unique SID, and that's where system cloning comes into play. By cloning your system, you can maintain everything that should be the same between unique devices, but change the SID for each device. To clone your configuration on different devices, add the System Cloning component to your configuration. You deploy this image to a master device, which you boot through the FBA phase. After FBA completes, it will reseal the device, and prompt you with a dialog box. At this point, you can copy the master image to your other devices.
But what if you have configuration to do on the runtime after FBA finishes? Easy enough. In the System Cloning component, there is an extended property called cmiResealPhase. Set the value of this property to 0. This effectively stops FBA from running the reseal operation. After FBA completes, perform whatever configuration you need to, then execute \Windows\FBA\Reseal.EXE to finish the reseal operation manually. Your master image is now ready for copying.
Burning Question #4: BOOTPREP Problems
There have been a few BOOTPREP problems listed in the newsgroup as well. What is BOOTPREP? It's a utility we provide to put an NT boot sector on your media. This is what allows you to boot Windows XP Embedded on your device. Here's the long and short of how to get through BOOTPREP issues.
BOOTPREP is a DOS-based utility—and this means booting the system to DOS, not running it in a DOS window under Windows 2000 or Windows XP. It works on FAT16 and FAT32 volumes only (it's unnecessary for NTFS volumes). You need to FDISK and FORMAT the volume using the normal DOS-based tools first, and then run BOOTPREP. Documentation for how to run BOOTPREP can be found in the System Design Guide (BootPrep), or in the \Program Files\Windows Embedded\utilities folder along with the BOOTPREP.EXE tool.
Some customers have had problems running BOOTPREP on FAT32 volumes. We haven’t been able to track down a solid repro of this issue. If you find BOOTPREP is having problems working with FAT32 volumes, switch to FAT16.
Why isn't BOOTPREP necessary for NTFS volumes? Because all NTFS volumes have a boot sector on them; it's part of the NTFS formatting process.
Burning Question #5: Static IP Addresses
Our last issue doesn't really have an answer to it, but rather an explanation. There has been a lot of talk about setting static IP addresses for images in Target Designer. That capability doesn't exist, and here's why:
The System Cloning component, as mentioned above, resets the SID of a device, but not the IP address. If we allowed static IP addresses to be set in Target Designer, every image cloned from that configuration would have the same IP. Rather than create a problem by allowing runtimes to be created (and possibly cloned) with static IP's, we decided to not allow it.
Well, then, how about an automated way of setting IP's on specific devices (maybe as part of the System Cloning component)? That's a good idea in theory, but let's think this through. How would any automated tool on the embedded device know which IP to set? That information could be provided manually, but if it's a manual operation, why automate the process in the first place? So we want an automated way to get the proper IP to set on the device, and the only place to get it is through some sort of network service. However, since there's no usable IP on the device at this point, there's no way to query a network service for the next available IP. Even if there were a usable IP on the machine to get an IP address through, how is that different from what DHCP does? That's what DHCP does—it sets the IP address on machines in an automated fashion. In short, we've described the problem of automating IP address assignment as being solved by DHCP.
There's really no good general solution to setting a static IP on a set of devices. Specific device-dependent solutions are up to the individual OEM to produce and implement. There are a number of possibilities that have been discussed in the newsgroup, including setting IP addresses in Control Panel, using Network Scripting Host, and programmatically using the Windows XP SDK. It's up the individual OEMs to find and develop the solution that works for them. Look to general Windows XP support for help with these solutions.
Hopefully, I've been able to answer some of your more burning questions—and if I couldn't answer them, I hope that I could at least tell you why I couldn't.
Keep your questions coming! Log on to your favorite news server, and find the microsoft.public.windowsxp.embedded newsgroup. If your news server doesn't have a subscription, you can always hit msnews.microsoft.com, or go to Windows Embedded Newsgroups for a Web-based viewer. And while you're there, be sure to check out Tips and Tricks, soon to be overflowing with tips from the Windows XP Embedded team.
Next month, I promise, scripting your components will be discussed. Until then, if you've got component scripting questions, post them to the newsgroup above.