Customizing the Windows CE Build System Using DIRS, SOURCES, and CECs

Mike Hall
Microsoft Corporation

Steve Maillet
Entelechy Consulting

January 3, 2002

Last month we provided an overview of the typical steps used to build an image of Microsoft® Windows® CE .NET, and took a look at how the operating system is built under the covers. The build system is extensible at a number of levels, providing a means for adding your own drivers, technologies, and applications to a platform. These can either be in pre-build binary form, or in source-code form.

In this article, we will take a look at how to customize the build system through "catalog feature files" (previously known as CE component files, or simply CEC files—we will call these CEC files for the rest of the article) and DIRS and SOURCES.

DIRS and SOURCES

We touched on DIRS in last month's article; let's review how and when these are used.

A typical Board Support Package (BSP) contains multiple folders, most of which will be 'touched' during the platform build process. BSPs are normally located in the C:\WINCE400\PLATFORM\ folder, assuming Platform Builder is installed into the default folders.

Let's examine the folder structure of the EMULATOR platform, which is located in C:\WINCE400\PLATFORM\EMULATOR. Below is shown a directory listing. You will notice a number of interesting looking folders—Drivers, GWE, KERNEL, and so on. Each of these folders may contain source files that need to be assembled, compiled, and linked during a platform build process. How does the build system know which files/folders to build?

C:\WINCE400\PLATFORM\EMULATOR>dir
Volume in drive C has no label.
Volume Serial Number is 109F-4316

Directory of C:\WINCE400\PLATFORM\EMULATOR

12/13/2001 04:22 PM <DIR> .
12/13/2001 04:22 PM <DIR> ..
12/02/2001 02:29 AM 393,091 Build.dat
12/02/2001 02:30 AM 131,526 Build.log
12/02/2001 02:30 AM 1,620 Build.wrn
12/13/2001 04:22 PM <DIR> CESYSGEN
12/10/2001 01:00 AM 142 dirs
12/13/2001 04:23 PM <DIR> DRIVERS
12/10/2001 01:00 AM 546 emulator.bat
12/13/2001 04:22 PM <DIR> FILES
12/13/2001 04:22 PM <DIR> GWE
12/13/2001 04:22 PM <DIR> INC
12/13/2001 04:22 PM <DIR> KERNEL
12/02/2001 02:28 AM 13 LCDTERM.bif
11/07/2001 02:50 AM <DIR> lib
12/10/2001 01:00 AM 103 sources.cmn
12/02/2001 02:28 AM 1,784 sources.gen
11/07/2001 02:50 AM <DIR> target
8 File(s) 528,825 bytes

Notice the dirs file; let's examine its contents. (Note that I've stripped comments out of the file to make is easier to read.)

DIRS=        \
    gwe      \
    kernel   \
    drivers

So, this is how we determine the folders that need to be built for the Emulator Platform. We will build gwe, then kernel, then drivers. Now let's take a look into one of the folders: Drivers. (Note that the same build mechanism is used in the other emulator folders, and in fact in other BSP folders.)

C:\WINCE400\PLATFORM\EMULATOR\DRIVERS>dir
Volume in drive C has no label.
Volume Serial Number is 109F-4316

Directory of C:\WINCE400\PLATFORM\EMULATOR\DRIVERS

12/13/2001 04:23 PM <DIR> .
12/13/2001 04:23 PM <DIR> ..
12/10/2001 01:00 AM 576 dirs
12/13/2001 04:23 PM <DIR> DISPLAY
12/13/2001 04:23 PM <DIR> DMA
12/13/2001 04:23 PM <DIR> KBDMOUSE
12/13/2001 04:23 PM <DIR> NET
12/13/2001 04:22 PM <DIR> WAVEDEV
1 File(s) 576 bytes

Again, we have a number of folders, including Display, Keyboard/Mouse, and so on. We also have another "dirs" file. Let's examine its contents.

!if 0
Copyright (c) Microsoft Corporation.  All rights reserved.
!endif
DIRS= dma \
      display  \ 
      kbdmouse \
      net      \
      wavedev  \

We can see that we again have a list of folders that will be included in the build process. To add your own driver to the list, you would simply append the name of your driver folder into the DIRS file, like so:

!if 0
Copyright (c) Microsoft Corporation.  All rights reserved.
!endif
DIRS= dma \
      display  \ 
      kbdmouse \
      net      \
      wavedev  \
   MyDriver \

Let's now examine the display folder; this again contains a DIRS file that lists a single folder, VGAFLAT. The C:\WINCE400\PLATFORM\EMULATOR\DRIVERS\DISPLAY\VGAFLAT folder doesn't contain a DIRS file, but does contain two files that are also used in the build process. These are SOURCES and MAKEFILE, which we will examine shortly.

That's as hard as it gets for DIRS files; DIRS files simply contain a list of folders that will be included in the build process. Each of these folders may contain a DIRS file, which points at additional folders to include in the build process.

SOURCES

Using DIRS files to map out the folders included in the build process is all well and good, but at some point we actually need to build something. This is where SOURCES and MAKEFILE are used. The SOURCES file contains information that determines how folder contents will be built, and what the output will be from the build process (EXE, DLL, LIB).

Build invokes make/nmake under the hood, so a makefile is needed. In the same directory as a SOURCES file, there is a makefile that #includes makefile.def (the "master" makefile). The purpose of that file is essentially to set environment variables, but it also handles precompiled headers, cleans old build targets, gets include paths right, and sets the correct library list for linking a particular component. It also contains the compiler and linker invocation command lines.

MAKEFILE is deceptively simple. Here are the contents of the file:

!INCLUDE $(_MAKEENVROOT)\makefile.def

The _MAKEENVROOT macro (when expanded) equates to:

_MAKEENVROOT=C:\WINCE400\public\common\oak\misc

**Hint   **To expand a macro, start Platform Builder and open a recent workspace. Select Build and then select Open Build Release Directory. This will launch a command window. Type "set _makeenvroot <return>". This will display the contents of the _makeenvroot environment variable (if set). Typing "set <return>" will list out all currently configured environment variables.

Therefore, the actual system-wide makefile is:

C:\WINCE400\public\common\oak\misc\makefile.def

The build process uses macro definitions in the DIRS and SOURCES files to build the source code for a platform. These macro definitions use the following syntax:

Macroname= value

You can include white space to the left of the macro name and to the right of the equals sign. There can be no white space between the macro name and the equal sign. If the value has a backslash (\) as the last character on a line, the macro definition continues to the next line. Given these rules, the following macro definitions are equivalent.

SOURCES= MyProj1.c MyProj1.h
SOURCES= MyProj1.c \
  MyProj1.h
SOURCES= \
  MyProj1.c \
  MyProj1.h

Macro names are not case sensitive. Both Nmake.exe and Build.exe expand macro definitions. For example, Build.exe tries to expand the BBB macro in the following line:

AAA=  $(BBB)\Myproj

If BBB is not defined as a macro or environment variable, Build.exe defines the AAA macro as \Myproj.

A number of macros are commonly used in SOURCES files. Take a look at the following SOURCES example from the Emulator/Display/VGAFLAT folder. We will then examine some of the common macros. (A number of other macros can be used in SOURCES files. Take a look at the Platform Builder online help for a complete listing.)

!if 0
Copyright (c) Microsoft Corporation.  All rights reserved.
!endif
!IF 0


Module Name:

    sources.

Abstract:

    This file specifies the target component being built and the list of
    sources files needed to build that component. It also specifies optional
    compiler switches and libraries that are unique for the component being
    built.


!ENDIF

!if "$(_TGTCPUFAMILY)" != "x86"
SKIPBUILD=1
!endif

TARGETNAME=DDI_EMUL
TARGETTYPE=DYNLINK

# uncomment the next line to build a DDraw enabled version of this driver
DDI_DD_ENABLE=1

CLEARTYPE=1
TARGETLIBS= \
    $(_COMMONSDKROOT)\lib\$(_CPUINDPATH)\coredll.lib \
    $(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\emul.lib \
    $(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\ceddk.lib \
    $(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\gpe.lib \
!IFDEF CLEARTYPE
    $(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\ctblt.lib \
!ENDIF
!IFDEF DDI_DD_ENABLE    
    $(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\dxdrvguid.lib \
!ENDIF    

CDEFINES=$(CDEFINES) -DDDI -DVGA_DEBUG_MSGS -DCLEARTYPE

!IFDEF DDI_DD_ENABLE
CDEFINES=$(CDEFINES) -DDD_ENABLE
!ENDIF

PRECOMPILED_INCLUDE=precomp.h
PRECOMPILED_PCH=precomp.pch
PRECOMPILED_CXX=1

INCLUDES= \
    ..\ddgpe\inc; \
    ..\ddgpe\ddgpe; \
    $(_PUBLICROOT)\common\oak\csp\i486\inc;

!IFDEF DDI_DD_ENABLE    
SOURCELIBS= \
    $(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\ddgpe.lib \
!ENDIF    

SOURCES=               \
   gpeflat.cpp    \
        surf.cpp       \
        halcaps.cpp    \
        haldd.cpp      \
        halpalette.cpp \
        halsurf.cpp

We've examined DIRS and SOURCES—two files used by the build system to generate the DLLs and EXEs for the final operating system image used in a device. It may be useful to "wrap up" a series of folders, or a single SOURCES file, into a component that lives in the catalog, and which therefore can be selected and included in your platform. That's what CEC files are all about.

To build the Sources or DIRS file, you can add it directly into the workspace by right clicking on the workspace icon at the root of the tree in the Feature View, and then select Insert User Feature. The insert dialog will appear. You will need to select the file type for DIRS and SOURCES to add it.

Figure 1. Insert User Feature

However, if more then one developer will use the code, it's better to create a CEC file to allow developers to add it into a build with a simple click from the catalog. CEC files are a bit more complicated in version 4.0, as there is more functionality provided in the IDE. The good news is that Windows CE .NET includes a CEC file editor. (Which coincidentally has many features and ideas included in a tool published by one of the authors of these articles… Hmmmm, wonder where they got those ideas!) We'll stay away from the syntax, as it isn't much different from previous versions (it's documented, and there is a GUI editor for it). Instead we'll focus on the concepts of what it's for and how to apply the various elements properly. You'll see how CEC files are useful for much more than just building some code; they are actually the major control point for the build in the IDE.

Now, a little bit about GUIDS…

Globally Unique Identifiers (GUIDS) are 128-bit values represented in text in the form {XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}, where each X is a hexadecimal digit. The CEC editor will generate GUIDS automatically for you, and you can also generate new ones for a component. The GUIDGEN tool is also useful for those editing the CEC file as text.

The following UML Diagram describes the elements of a CEC file.

(Click thumbnail to view larger image.)

Figure 2. UML diagram showing the elements of a CEC file

Don't worry if you aren't familiar with the UML notation. (Though learning it at some point is strongly recommended!)

The CEC File has one CECInfo block. This block assigns a unique ID for the CEC file, and specifies a name and description for the file, as well as the version of the CEC file structure. The original CEC file structure from Platform Builder V2.12 did not include this block. Version 3.0 added this block primarily to include the version number. For Windows CE .NET, the version number is 4.00.

The ComponentType block is directly related to a Feature in the catalog or feature view. It is represented in the IDE by the collection of three cylinders (2 green on blue) icon. The component type is much like a class in object-oriented programming. It is an abstraction only, and requires an implementation before it is useful.

The ComponentType block may contain one or more implementations. Each implementation provides information specific to a particular feature item you want to include. By way of example, let's look at the display drivers component type.

The Display Component Type has several implementations for various types of display cards. You can have implementations that are CPU-specific as well.

It is important to know that every ComponentType has a GUID.

BSP files no longer exist in Windows CE .NET. That functionality is now in the CEC file for the BSP. There is a special component type called, informatively enough, "BSPs," with a GUID of {3CABD6DA-B577-4A87-9490-9432E80DC2F1}. To add your own BSP into the catalog, you must include an implementation of this type in the CEC file. In your BSP implementation, you will include a DefaultDrivers attribute that specifies the drivers (by GUID) that the IDE will include in any new platforms created from the wizard based on your BSP. If your BSP supports multiple drivers of a particular type (such as a few variations on a display driver in the CEPC BSP), then you can specify the display component type GUID, and the end user will then resolve to a specific implementation before performing a build.

The BIB Info block is a great addition to CEC files in Windows CE .NET. This element allows you to specify BIB file settings to be automatically included whenever the user selects the feature from the catalog. If you simply used a DIRS/SOURCES file, then the user would still need to add the BIB entries manually.

Implementations contain one or more build methods, each associated with a particular step in the build process.

Some of the step names are actually redundant, but provide consistency and compatibility with previous versions of CEC Files. (PreMakeImg and MakeImg are both run before Makeimg.exe. Although it's not clear if the rules for actions (see below) are different for each at the time of this writing. We'll check into that on the final release, and let you know what's up.)

Build Methods contain a set of actions to perform during the appropriate stage of the build process, as described in the following table:

The actions have a set of rules associated with them. Each build step has restrictions on which actions are allowed, as follows:

There are a few attributes of the various elements of a CEC file that provide important information to the IDE. Understanding these will help you appreciate the various ways in which the IDE tries to make your life easier.

1. The correct component type GUID is important. In Platform Builder 3.0, the component type GUID didn't matter much. In Platform Builder 4.0, if you're creating a standard driver, you must use the component type GUID associated with that driver.

2. The IDE uses the required CE modules (RequiredCEModules) attribute of the ComponentType to identify an item as a driver. The driver updater code requires this to figure out when to add the driver. These will be strings either from the CE_MODULES environment variable, or from the COREDLL_COMPONENTS environment variable. Display drivers, for example, need "display" and "device." If you're creating a standard driver type, you don't need to worry about this, because it's already been done for you.

3. The only way the IDE can know whether to include or exclude a given driver is to know which environment variable(s) will exclude it. This is specified by the ExcludeWhenSet attribute. For a display driver, for example, the exclusion variable is BSP_NODISPLAY. If you look at C:\Wince400\Platform\CEPC\cepc.bat, you'll see such things as BSP_NOTOUCH=1, BSP_NOUSB=1, and so on. That's why you won't get a USB driver for your CEPC, even if you've got USB support in your operating system. The IDE can and will override this variable, but only if you tell it what it is. As in number 2 above, if you use a standard driver type, you don't need to worry about this, because it's already been done for you.

4. The per-implementation variables (Variable) will generally be two or three variables required for this driver to be included in the build. The first will be BSP_NO exclusion variable associated with this driver (set to NULL, of course). The second will be the BSP_ inclusion variable associated with this driver (set to 1). The third will be the sysgen variable that would be needed if we are to sysgen and build just this driver. For the ATI display driver, for example, we set the following:

      BSP_NODISPLAY=
      BSP_DISPLAY_ATI=1
      PB_REQUIRES_SYSGEN=ddi_ati
   

   That last variable is a special environment variable to notify Platform Builder that when building this particular driver, Platform Builder should sysgen it first. The value is the command-line value passed to the sysgen process. This is only used if you right-click on the driver and select Build Selected Features. The sysgen will happen as part of the CESYSGEN phase normally. This can be useful for drivers, such as a display or Unified Audio driver, to sysgen the driver to remove Microsoft® DirectX® support if it's not included in the operating system.

5. DefaultDrivers specifies the default drivers to be selected. Vendors used to use the .bsp file to tell the IDE which drivers to include for a given configuration. Much of that work is replaced by the driver updater, which is capable of figuring out the type of driver to add. Which specific implementation of each type to add, though, is left to the BSP vendor to specify in the CEC file. That's where the new DefaultDrivers field comes into play. You need to add a DefaultDrivers attribute for each type of driver that the BSP supports; this provides the implementation GUIDs that will tell Platform Builder what driver to select.

6. BSPPlatformDir identifies the BSP platform directory in which the BSP source is located. This field should be used for each BSP, and for each driver implementation whose source is also located in that directory. In the case of the BSP, it tells the IDE where to go to sysgen and build the BSP. In the case of the drivers, it lets the IDE know that it should place those drivers under the BSP node, rather than in the common Device Drivers folder in the catalog. If you use the BSPPlatformDir field on your native drivers, you won't need to use the Children or OptionalChildren field; the IDE will place the drivers in the correct place in the catalog and in the FeatureView (under the BSP).

7. MaxResolvedImpsAllowed identifies the maximum number of implementations this driver supports. This tells Platform Builder whether to enforce exclusivity on this driver type. At this point, the IDE only supports two values for this field: 1 and not 1. (The CEC files provided by Microsoft all use 999.) If you tell the IDE that only one driver of this type can be in your platform at a time, and you subsequently try to drag a duplicate driver over, it will ask you if you wish to swap the current driver with the new driver. If you tell it that the driver type supports more than one implementation (for example, multiple serial drivers), it will let you keep adding as many drivers as you wish. As before, if you use a standard driver type, you won't need to set this field—it's already done for you.

8. CPUSizeMultiplier is the size multiplier associated with a given BSP. The operating system team is trying to provide estimates on how large the image will be for the features selected. For a number of reasons, these are only estimates of the final impact on your image (generally, worst case estimates). Many of these items are CPU-dependent, so that they are bigger or smaller depending on the compiler/linker for that CPU. The default for this field is 1.0, based on the size of the item in a CEPC. The most common settings are 2.0 for MIPS, and 1.5 for ARM CPUs.

9. ImplSize is the size field associated with a given BSP or driver. This allows you to tell the IDE approximately how many bytes this BSP or driver will be in the final image. If the item is a driver, the size field should be set based on the size of the image in a CEPC (x86).

Anyone creating a BSP should keep these points in mind. It is especially important for anyone delivering a BSP for third parties as part of a reference design, COTS solution, or software Dev Board. By providing your customers with a good clean BSP, with as much configurability isolated to CEC files as possible, you simplify and shorten the development process for your customers (read as, "save them money!"). That always makes for happy customers.

Vendors providing drivers for Windows CE should also use CEC files for your drivers. It is an easy task to export a driver feature from the catalog (and any related files), and generate an installer for it. Microsoft has added this as a feature of the catalog, so that it will be easy to create installations for various features in the catalog from your existing ones. I would seriously question the quality or stability of any driver from a third party that does not provide this, since it is so easy for them to do, and it saves the end user so much time.

Get Embedded

Mike Hall is a Product Manager in the Microsoft Embedded and Appliance Platform Group (EAPG). Mike has been working with Windows CE since 1996—in developer support, Embedded System Engineering, and the Embedded product group. When not at the office, Mike can be found with his family, working on Skunk projects, or riding a Honda ST1100.

Steve Maillet is the Founder and Senior Consultant for Entelechy Consulting. Steve has provided training and has developed Windows CE solutions for clients since 1997, when CE was first introduced. Steve is a frequent contributor to the Microsoft Windows CE development newsgroups. When he's not at his computer burning up the keys, Steve can be found jumping out of airplanes at the nearest drop-zone.