Build Management Basics Using Microsoft Windows CE .NET with Microsoft Platform Builder 4.0
Jay Loney, Windows Embedded MVP
Andy Jackson and the Build Team
Microsoft® Windows® CE .NET with Microsoft Platform Builder version 4.0
Summary: Windows Embedded MVP, Jay Loney, and others from BSQUARE provide an overview of Microsoft Windows CE .NET build management for the Windows Embedded Developer Community. (7 printed pages)
Author's note This content is for informational purposes and is provided only as a convenience. The entire risk of use or results in connection with the use of this article remains with the user.
One thing every company should implement is some scheme for verifying a team's image build regularly in a clean environment. This document provides overviews of Microsoft® Windows® CE .NET build management. It begins with the purpose of a nightly build. The nightly build procedure is then described. This document also talks about the requirements that the build system places on the Microsoft Visual SourceSafe® (VSS) tree structure of a platform.
Throughout this document we will make some assumptions. First, that you will be using VSS as your source code store. Next, the build is occurring at the command prompt. In Windows CE .NET terminology, the term platform refers to the image (nk.bin) being built. This document also assumes that the product is a board support package (BSP) provided in source code form. The word kit refers to the final product. Kitting refers to the packaging of the source code or binaries into an installable form.
The Purpose of a Nightly Build
The main intent of the nightly build is to create a centralized point from which to coordinate the efforts of developers and testers. The development of software is heavily team oriented, and as a company grows and expands, it becomes more important that the company be able to effectively coordinate and leverage its engineering assets. Having a centralized build which can be dropped to test to verify results can bypass many of the pitfalls involved in synchronizing teams of individuals and groups of teams.
Common problems, such as multiple code bases, version conflicts, missing check-ins, and inconsistent build environments, are avoided or exposed by using a common nightly build. With all platforms being built on clean machines, the nightly build can quickly bring to light most problems. For example, there may be an environment variable that a developer may have forgotten to explicitly set or had set erroneously. The nightly build synchronizes the work of multiple developers, reveals breaks that recent check-ins have introduced, and acts as a sanity check for missing/unchecked-in files. Coupled with a backup scheme, the nightly build also provides an easy way to reproduce drops from the past, and to examine the build environments/tools that were used to build them.
In the grander view of things, the nightly build can be viewed simply as one component of the overall production cycle. Since it can be owned by the same group that is responsible for production, platforms can transition from a development/testing phase to a Release Candidate/acceptance phase and, finally, to production, smoothly and seamlessly.
Build System Capabilities
A build system should be fully automated, and standardize many common tasks.
- Automated nightly build, seven nights a week
- The ability to pull source from VSS, UNC Shares, Local Drives
- Can build platforms using a variety of tools
- Platform Builder (any version)
Clean tree for every build
Can build with any number of QFE's installed together or individually
Ability to change and modify the tree for custom platforms
- Microsoft eMbedded Visual C++® (eVC++)
- Run custom scripts via windows scripting host
- Platform Builder (any version)
- Daily drop for each platform
- Automated setup creation
- Install Shield
- Standard or custom setups
- Daily build email
Filtering of errors in the log
Notification of drop locations
- Notification of build expiration
- Notification of platform specific errors
- Daily build email
- Backed up nightly (the past seven days worth of drops are readily available)
- The ability to re-release existing platforms with new documentation and/or a new setup
- Production/Release Management
- RC promotion cycle
- RTM promotion
- Archival of released platforms
A nightly build process is separated into several stages, with many of them being maintenance related. Since the maintenance of the system lies outside of the scope of this document, details on these stages will not be included. The stages involved in the building of an individual platform are as follows:
- Start clean
- Get the source code
- Get pre-built third party components (optional)
- Place the source code
- Get the documentation (optional)
- Create the source kit setup (optional)
- Build the platform
- Assemble the binary kit (optional)
- Create the binary kit setup (optional)
- Copy the results to the drop server
The first thing that must be done in any build environment is to guarantee that the tree is clean and does not contain any unknown modifications. This might represent QFEs, environment variables stored in batch files, or other source or binary file modifications. One way to guarantee this is to copy over a clean tree before every build. This is a brute force approach and is recommended. Another is to windiff trees for differences and revert to clean. A final mechanism is to simply know and revert every branch your particular build touches. This last approach is the fastest, but is also the most dangerous as it does not take into account unforeseen changes.
Get the source code
During this initial stage, everything that will be used to build the platform is pulled into a working repository. These components may be source code, 3rd party applications, prebuilt binaries, and so on. The main purpose at this point is to gather everything in one place, and create a position to work from. Source can be pulled from any location: multiple VSS servers, network shares, or local drives. However, the locations that all these bits are pulled from must be official repositories such as VSS servers. Knowing where the bits come from and being able to control the sources is crucial to being able to reproduce any build.
Get pre-built third party components (optional)
The first build stage of the nightly build is to build all third party binaries (if any) that will be used by the platform. The source for these bits are copied into the Platform Builder tree, built, and then saved off. The tree is then cleaned up for the primary build. Having third party source built separately from the actual build helps prevent those sources from accidentally making it into any released product, thus violating any source agreements. It also allows them to be built in their own build environment. A text file called a BOM file, or Bill of Materials, is used by the build system to determine which sources within the repository need to be built by the pre-build process. See the section BOM files for more information about the Bill of Materials files.
Place the source code
After the pre-build stage, the source code is inserted into the Platform Builder build tree from the files in the source repository and any pre-built binaries. The source code is basically the tree structure that contains all of the components necessary to build a particular platform. This structure is arranged in the exact way that it needs to be in _WINCEROOT. In the case of a BSP, these files are normally all stored under %_WINCEROOT%\Platform\%_TGTPlat%. A BOM file describes all the components that need to be placed into the Platform Builder tree, and where each component is to be placed. This BOM file can then be used in creating the installation routine for any BSPs or drivers you are distributing. As a rule, no builds should be done on source code that is shared out from a developer's workstation. All code should be checked into some type of shared source server.
Get the documentation (optional)
After the main source is pulled and arranged, documentation is pulled in from a documentation database. Documentation is kept separate from the source in order to allow for easier re-packaging of new documents with an existing product. This is the key feature that allows for parallel development of code and documentation.
Create the source kit setup (optional)
If you are creating a source code product for release, you should create the installation before running the build. This is because of the files that are created and inserted into the tree by the build during image creation. After the source kit is assembled, and the documentation grabbed, the source kit setup is generated. The setup includes shortcuts to documentation files, shortcuts to build windows or Platform Builder workspace files, and uninstall options, in addition to the contents of the kit. This is an optional step, in that it only occurs if a drop of the source kit is desired.
There is a new alternative to this phase of build management. Platform Builder includes two tools that are used to export a BSP from the IDE. They are the BSP Wizard and the Export Wizard. The BSP Wizard is used to create BSP structure and import the BSPs CEC file definitions into the IDE. The Export Wizard can then be used to package an individual driver or an entire BSP into MSI file format. A precursor to this step involves populating the catalog with the appropriate CEC files to build all the drivers from the IDE. In Windows CE .NET, however, no clean method for build management exists. Build Managers are still relegated to the command prompt for now. This can cause problems if your engineers are creating their platforms using the integrated development environment (IDE) methodology of CEC files vs. the command prompt methodology of DIRS, SOURCES, and batch files.
Build the platform
Once the source kit tree is constructed, it is then overlaid (copied) into the wince tree and built. For most platform builds, wince.bat is run along with any setenv.bat files in order to setup the environment. Afterwards, blddemo.bat is called, which will build the platform. The final step of blddemo is to call makeimg, which creates the image assuming there are no errors. If there are errors, it is your responsibility to capture those errors and email them to the engineering team for analysis. Depending on the specific requirements of a platform, wince.bat and/or blddemo.bat may be replaced by custom build batch files. If any images are needed, they are created during this stage. For eVC++ type builds, the main workspace file is passed to the respective tool and built.
In order to give developers more flexibility in their build environments, the build system makes use of a developer owned setenv.bat (as mentioned above) to initialize the build environment. This setenv.bat file is also automatically included in the setup via build window shortcuts. This is to allow customers to easily recreate the environments that a platform was built in and quickly launch builds. The setenv.bat file needs to be checked into VSS in order to be automatically pulled from VSS and utilized by the build system.
Assemble the binary kit (optional)
If you are assembling a binary kit, it is very similar to assembling the source kit. It too uses a BOM file, but instead of pulling pieces from the source repository, they are pulled straight from the WINCE tree. They can be pulled either from the %_FLATRELEASEDIR% or the Target directory of your BSP. This way, the resulting binaries from a build may be included in the kit along with any source that will be delivered.
Create the binary kit setup (optional)
Once the build is completed, if you are creating a binary release of your product, you can create the install. Just as in creating the source kit setup, the binary kit setup is also optional. This process is exactly the same as the process for creating a source kit setup, except that it packages the binary kit tree instead of the source kit tree.
Copy the results to the drop server
The final step is to copy the drop, the source repository, the source kit tree, and binary kit tree to the drop server. The drop is the only item that will be accessible by everyone, with the source files and kits being secured for archival and rekitting purposes.
In order to create a source kit or a binary kit, BOM files are used. BOM stands for Bill of Materials. A BOM file specifies what sources/binaries need to be included in a kit. This file is used by the automated build system to pick and choose the different files/directories that will go into a kit. In comparison, BOM files are to kitting as BIB files are to the build of NK.bin. They specify which files are to actually be included in the product installation.
The build system begins each build by first pulling all the relevant sources for a platform into a repository (VSS, static bits, third party source, binaries, and so forth). The BOM file then specifies which files/directories out of this repository are to be used in the nightly build, and also the tree structure in which these files/directories are to be arranged. Blddem.bat is used to build the platform. If the kit to be delivered is to be a source kit, the tree structure described by the BOM file is then packaged and copied to the drop site.
If built binaries are needed as well as source, this needs to be included in the BOM as well. Also, there may be prebuilt binary files that are included in the same fashion.
Ideally, the BOM file should be owned/managed by development, since the developers are the ones who should know best what needs to be included in a kit in order for it to work. However, program managers need to also be involved in the creation of the BOM files, since these files do describe what will eventually be delivered to customers. In this respect, the program managers need to verify that restricted materials (3rd party source, internal IP, and so on) are not being included in the deliverables.
For the latest Windows CE .NET Developer information:
- Developer information, visit http://msdn.microsoft.com/embedded
- Product information visit: http://msdn.microsoft.com/embedded/ce.net
- Community Tips and Tricks, visit: http://www.microsoft.com/windows/embedded/community
The online documentation and context-sensitive Help included with Windows CE .NET also provide comprehensive background information and instructions for using Windows CE .NET.
To access the online documentation
- Open Platform Builder 4.0.
- Select Contents on the Help menu to view the documentation.
Note Install the latest Windows CE .NET Help File updates from the Windows Embedded Download Center on MSDN.
Microsoft Windows Embedded Developer Center
You can always find the latest developer information including news, product documentation, technical articles, columns, How To's, Downloads, Code Samples, Fixes, and Updates for Windows CE .NET on the Windows Embedded Developer Center on MSDN.
Join, Share, and Develop in the Windows Embedded Community
Visit the Windows Embedded Community for resources that help you stay current on platform technologies including Tips and Tricks, chats, newsgroups, featured downloads, and more. You can also share your development expertise, gain from valuable peer-to-peer support relationships, and find answers to many Windows CE .NET related questions.