Installing an Integrated or Isolated Shell Application

  • Article

You must perform the following steps to install an integrated or isolated shell application.

  • Prepare your solution.

  • Create a Windows Installer (MSI) package for your application.

  • Create a Setup bootstrapper.

All of the example code in this document comes from the Isolated Shell Deployment Sample on the MSDN Code Gallery Web site. The sample shows the results of performing each of these steps.

Prerequisites

To perform the procedures described in this topic, you must have the following tools installed on your system.

Preparing Your Solution

By default, the Shell Templates build to a VSIX package, but this is intended for debugging purposes only. VSIX does not support deployment to other systems. We recommend that you deploy Shell applications in MSI packages to allow for registry access and for restarts during installation. To prepare your application for MSI deployment, perform the following steps.

To prepare a shell application for MSI deployment

  1. Edit each .vsixmanifest file in your solution.

    In the Identifier element, add an InstalledByMSI element and a SystemComponent element, and set their values to true.

    This prevents the VSIX installer from trying to install your components, and prevents the user from uninstalling them in Extension Manager.

  2. If your application includes project templates or item templates,

    • In the project properties, edit the build tasks to output the template to a compressed file.

    • Or, use the Export Template wizard to create the compressed file, add it to the solution as a solution item, and remove the project.

    If you created your templates by using the templates for creating templates, you can skip this step.

  3. For each project that contains a VSIX manifest, edit the build tasks to output the content to the location that your MSI will install from. Include the VSIX manifest in the build output, but do not build a .vsix file.

Creating an MSI for Your Shell

To build your MSI package, we recommend that you use the Windows Installer XML Toolset because it gives greater flexibility than using a standard Setup project.

Set following elements in your Product.wxs file:

  • Detection Blocks

  • Layout of Shell components

  • Custom Actions

Then, create Registry entries, both in the .reg file for your solution, and in ApplicationRegistry.wxs.

Detection Blocks

A detection block consists of a Property element that specifies a pre-requisite to detect, and a Condition element that specifies a message to return if the prerequisite is not present on the system. For example, your Shell application will require the Microsoft Visual Studio Shell redistributable, and the detection block will resemble the following markup.

<Property Id="ISOSHELLSFX">
  <RegistrySearch Id="IsoShellSfx" Root="HKLM" Key="Software\Microsoft\VisualStudio\$(var.ShellVersion)\Setup\IsoShell\$(var.ProductLanguage)" Name="ProductDir" Type="raw" />
</Property>

<Condition Message="This application requires $(var.ShellName).  Please install $(var.ShellName) then run this installer again.">
  <![CDATA[Installed OR ISOSHELLSFX]]>
</Condition>

The example block above assumes an isolated Shell application. For an integrated Shell application, the Id values would be set to "IsoShellSfx" or "ISOSHELLSFX", and the registry key would point to \IntShell\$(var.ProductLanguage) instead of \IsoShell\$(var.ProductLanguage).

Layout of Shell Components

You must add elements to identify the target directory structure and components to install.

To set layout of Shell components

  1. Create a hierarchy of Directory elements to represent all of the directories to create on the file system on the target computer, as shown in the following example.

        <Directory Id="TARGETDIR" Name="SourceDir">
      <Directory Id="ProgramFilesFolder">
        <Directory Id="CompanyDirectory" Name="$(var.CompanyName)">
          <Directory Id="INSTALLDIR" Name="$(var.FullProductName)">
            <Directory Id="ExtensionsFolder" Name="Extensions" />
            <Directory Id="Folder1033" Name="1033" />
          </Directory>
        </Directory>
      </Directory>
      <Directory Id="ProgramMenuFolder">
        <Directory Id="ApplicationProgramsFolder" Name="$(var.FullProductName)"/>
      </Directory>
    </Directory>

    These directories are referred to by Id when files that have to be installed are specified.

  2. Next, identify the components that are required for the Shell and your Shell application, as shown in the following example.

    Note

    Some elements may refer to definitions in other .wxs files.

        <Feature Id="ProductFeature" Title="$(var.ShortProductName)Shell" Level="1">
      <ComponentGroupRef Id="ApplicationGroup" />
      <ComponentGroupRef Id="HelpAboutPackage" />
      <ComponentRef Id="GeneralProfile" />
      <ComponentGroupRef Id="EditorAdornment"/>      
      <ComponentGroupRef Id="SlideShowDesignerGroup"/>

      <!-- Note: The following ComponentGroupRef is required to pull in generated authoring from project references. -->
      <ComponentGroupRef Id="Product.Generated" />
    </Feature>

    1. The ComponentRef element refers to an additional .wxs file that identifies files that are required by the current component. For example, GeneralProfile has the following definition in HelpAbout.wxs.

      <Fragment Id="FragmentProfiles">
  <DirectoryRef Id="INSTALLDIR">
    <Directory Id="ProfilesFolder" Name="Profiles">
      <Component Id='GeneralProfile' Guid='*'>
        <File Id='GeneralProfile' Name='General.vssettings' DiskId='1' Source='$(var.BuildOutputDir)Profiles\General.vssettings' KeyPath='yes' />
      </Component>
    </Directory>
  </DirectoryRef>
</Fragment>

      The DirectoryRef element says where these files go on the user computer. The directory element says that it will be installed into a sub-directory, and each File element represents a file that is built or exists as part of the solution and tells where it can be found when the MSI file is created.

    2. The ComponentGroupRef element refers to a group of other components (or components and component groups). For instance, the ApplicationGroup ComponentGroupRef is defined as follows in Application.wxs.

          <ComponentGroup Id="ApplicationGroup">
      <ComponentGroupRef Id="DebuggerProxy" />
      <ComponentRef Id="MasterPkgDef" />
      <ComponentRef Id="SplashResource" />
      <ComponentRef Id="IconResource" />
      <ComponentRef Id="WinPrfResource" />
      <ComponentRef Id="AppExe" />
      <ComponentRef Id="AppConfig" />
      <ComponentRef Id="AppPkgDef" />
      <ComponentRef Id="AppPkgDefUndef" />
      <ComponentRef Id="$(var.ShortProductName)UI1033" />
      <ComponentRef Id="ApplicationShortcut"/>
      <ComponentRef Id="ApplicationRegistry"/>
    </ComponentGroup>

    Note

    Required Dependencies for Isolated Shell Applications are: DebuggerProxy, MasterPkgDef, Resources (especially the .winprf file), Application, and PkgDefs. Integrated shell has all of these already installed.

Custom Actions (Integrated Shell)

As part of the layout of Shell components, there may be Extensions and Templates to add to the user computer. In the case of Isolated shell, nothing extra has to be done because it is a separate product. However, in the case of Integrated Shell, the user may already be a Visual Studio user and that instance of Visual Studio will not be aware of added extensions or templates until you tell it to check for new templates and extensions. This is handled by a custom action.

You can ensure that your extensions load by specifying custom actions in product.wxi, just after the detection block, as shown in the following example.

    <InstallExecuteSequence>
      <Custom Action="SetInstallDir" Before="CostFinalize" />
      <Custom Action="SetDevEnv" Before="CostFinalize" />
      <Custom Action="RunSlashSetup" After="InstallFinalize">NOT Installed</Custom>
    </InstallExecuteSequence>

    <CustomAction Id="SetInstallDir" Property="INSTALLDIR" Value="[INTSHELLSFX]$(var.ExtensionsFolder)" />
    <CustomAction Id="SetDevEnv" Property="DEVENV" Value="[INTSHELLSFX]$(var.IdeFolder)\devenv.exe" />
    <CustomAction Id="RunSlashSetup" Property="DEVENV" ExeCommand="/setup" Return="ignore" />

The SetInstallDir action tells the system where to find your extensions and the SetDevEnv action tells it where to find devenv.exe. The RunSlashSetup action runs the devenv /setup command, which reloads all installed extensions.

Registry Entries

The Isolated Shell project template includes a ProjectName.reg file for registry keys to merge on installation. These registry entries must be part of the MSI for both installation and cleanup purposes. You must also create matching registry blocks in ApplicationRegistry.wxs.

To integrate registry entries into the MSI

  1. In the Shell Customization folder, open ProjectName.reg.

  2. Replace all instances of the $RootFolder$ token with the path of the target installation directory.

  3. Add any additional registry entries that are required to run your application.

  4. Open ApplicationRegistry.wxs.

  5. For each registry entry in ProjectName.reg, add a corresponding registry block, as shown in the following examples.

    ProjectName.reg

    ApplicationRegisty.wxs

    [HKEY_CLASSES_ROOT\CLSID\{bb431796-a179-4df7-b65d-c0df6bda7cc6}]

    @="PhotoStudio DTE Object"

    <RegistryKey Id='DteClsidRegKey' Root='HKCR' Key='$(var.DteClsidRegKey)' Action='createAndRemoveOnUninstall'>

    <RegistryValue Type='string' Name='@' Value='$(var.ShortProductName) DTE Object' />

    </RegistryKey>

    [HKEY_CLASSES_ROOT\CLSID\{bb431796-a179-4df7-b65d-c0df6bda7cc6}\LocalServer32]

    @="$RootFolder$\PhotoStudio.exe"

    <RegistryKey Id='DteLocSrv32RegKey' Root='HKCR' Key='$(var.DteClsidRegKey)\LocalServer32' Action='createAndRemoveOnUninstall'>

    <RegistryValue Type='string' Name='@' Value='[INSTALLDIR]$(var.ShortProductName).exe' />

    </RegistryKey>

    Var.DteClsidRegKey in this example resolves to the registry key in the top row. Var.ShortProductName resolves to PhotoStudio.

Creating a Setup Bootstrapper

After your MSI is complete, it will only install if all the prerequisites are installed first. To ease the end user experience, create a Setup program that handles the gathering and installing of all prerequisites before it installs your application. To ensure a successful installation, perform these actions:

  • Enforce Administrator installation.

  • Detect whether the Visual Studio Shell is installed.

  • Run the shell installer.

  • Report progress while the shell is installing.

  • Handle restart requests.

  • Run your MSI.

Enforcing Administrator Installation

This is required to enable the Setup program to access required directories such as \Program Files\.

To enforce administrator installation

  1. Right-click the Setup project and then click Properties.

  2. Under Configuration Properties/Linker/Manifest File, set UAC Execution Level to requireAdministrator.

    This puts the required attribute in the embedded manifest file to always run the program as administrator.

Detecting Shell Installation

To determine whether the Visual Studio Shell has to be installed, first determine whether it is already installed.

Do this in your code by checking the registry value of HKLM\Software\Microsoft\VisualStudio\10.0\Setup\ShellType\LCID\ProductDir\, where ShellType is either "IsoShell" or "IntShell", and LCID is the target culture code (1033 for US English.) This is the same value that is read by the Shell detection block in Product.wxs.

HKLM\Software\Microsoft\VisualStudio\10.0\Setup\(ShellType)\(LCID)\ProductDir\ contains the location where the Visual Studio Shell was installed, and you can also check for files there.

For an example about how to detect a Shell installation, see the GetProductDirFromReg() function of Utilities.cpp in the Isolated Shell Deployment Sample.

If the Visual Studio Shell is not installed on the computer, then you have to add it to your list of components to install. For an example, see the ComponentsPage::OnInitDialog() function of ComponentsPage.cpp in the Isolated Shell Deployment Sample.

Running the Shell Installer

Running the shell just requires calling the Visual Studio Shell redistributable by using the correct command-line arguments. At a minimum, you have to use the command-line arguments "/norestart /q" and watch for the return code to determine what should be done next. The following example runs the Isolated Shell redistributable.

dwResult = ExecCmd("VsIsoShell.exe /norestart /q", TRUE);

Reporting Progress

If you want to report the progress of your installation, get a named pipe ready to receive progress messages, modify the SFX launch, and then decipher the callback information.

Setting Up the Named Pipe and Callback Function

The following example uses a CSetupWatcher instance to create the pipe and set the listener to the SetupUIHandler function.

CSetupWatcher* pSetupWatcher = new CSetupWatcher(setupWatcherPipe);
pSetupWatcher->Connect();
pSetupWatcher->ReceiveMessages(SetupUIHandler, NULL);

Modifying the Shell SFX Launch

After the pipe is established, you have to tell the SFX to send progress information to that pipe by using the /progress argument. After /progress, include the name of the pipe that it should send the progress information to.

    fullCmdLind.Format(cmdLine, setupWatcherPipe);
    shellInstallCmd.Format(_T("\"%s\" %s"), sfx, fullCmdLind);

In the sample, cmdLine resolves to "/norestart /q /progress %s", and setupWatcherPipe is the same string that is used to create the named pipe. Therefore, the final launch of the Shell SFX is

VsIsoShell.exe /norestart /q /progress ShellSfxInstallWatcher

Deciphering Callback Information

The callback function should expect three arguments, as follows.

int CALLBACK SetupUIHandler(void* pvContext, UINT iMessageType, MSIHANDLE hrecMsg)

When iMessageType is of type INSTALLMESSAGE_SUITEPROGRESS, then hrecMsg contains the information about how it is progressing. Using the standard MsiQuery functions yields something that resembles the following example.

    if (iMessageType == INSTALLMESSAGE_SUITEPROGRESS)
    {
        switch(MsiRecordGetInteger(hrecMsg, 1))
        {
        case 1: //Initializing
            percentComplete = MsiRecordGetInteger(hrecMsg, 2);
            break;
        case 2: //Downloading
            /*
            Field 2: Integer indicating the number of KB downloaded so far.
            Field 3: Integer indicating the total number of KB to be downloaded.
            Field 4: Integer indicating the current transfer rate in KB/s.
            */
            downloadedKb = MsiRecordGetInteger(hrecMsg, 2);
            totalKb = MsiRecordGetInteger(hrecMsg, 3);
            downloadRate = MsiRecordGetInteger(hrecMsg, 4);
            percentComplete = (int) 100.0f * downloadedKb / totalKb;            
            break;
        case 3: //Installing
            break;
        case 4: //Rollback
            break;
        };

Add logic to display this information to your users.

On most operating systems, the Visual Studio Shell installation will require a restart during installation. This can be determined by the return code of the call to ExecCmd().

Return value

Meaning

ERROR_SUCCESS

Installation completed, you may now install your application

ERROR_SUCCESS_REBOOT_REQUIRED

Installation completed, you may install your application after the computer has been restarted.

3015

Installation is in progress, a computer restart is required to continue the installation.

Handling Restarts

When you executed the shell installer by using the /norestart argument, you specified that it would not restart the computer or ask for the computer to be restarted. Now you have to ensure that your installer continues the Setup after restart.

To handle restarts correctly, make sure that only one Setup program is set to resume, and that the resume process will be handled correctly.

If either the ERROR_SUCCESS_REBOOT_REQUIRED or 3015 is returned, your code should restart the computer before it continues with the installation.

To handle restarts, perform these actions:

  • Set the registry to resume Setup when Windows starts.

  • Perform a double restart of the bootstrapper.

  • Delete the shell installer ResumeData key.

  • Restart Windows.

  • Reset the start path of the MSI launch.

Setting the Registry to Resume Setup When Windows Starts

The HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce\ registry key executes at system startup, with administrator privileges, and then is erased. There is a similar key in HKEY_CURRENT_USER, but it runs as normal user and is not appropriate for installations. You can resume installation by putting a string value in the RunOnce key that calls your installer, but we recommend that you call the installer by using a "/restart" or similar parameter to inform the application that it is resuming instead of starting from the beginning. You can also include parameters to indicate where you are in the installation process, which is especially useful in installations that may require multiple restarts.

The following example shows a RunOnce value for resuming an installation.

"c:\MyAppInstaller.exe /restart /SomeOtherDataFlag"

Initiating Double Restart of Bootstrapper

If the Setup is used directly from RunOnce, then the desktop does not have a chance to load all the way. To make the full user interface available, you must create a new execution of Setup and end the RunOnce instance.

You have to re-execute the Setup program so that it obtains the correct privileges, and you have to give it enough information to know where you left off, as shown in the following example.

if (_cmdLineInfo.IsRestart())
{
    TCHAR path[MAX_PATH]={0};
    GetModuleFileName(NULL, path, MAX_PATH * sizeof(TCHAR));    
    ShellExecute( NULL, _T( "open" ), path, _T("/install"), 0, SW_SHOWNORMAL );
}

Deleting the Shell Installer ResumeData Key

The shell installer sets the HKLM\Software\Microsoft\VisualStudio\10.0\Setup\ResumeData registry key with data to resume Setup after restart. Because your application, not the shell installer, is resuming, delete that registry key, as shown in the following example.

CString resumeSetupPath(MAKEINTRESOURCE("SOFTWARE\\Microsoft\\VisualStudio\\10.0\\Setup\\ResumeData"));
RegDeleteKey(HKEY_LOCAL_MACHINE, resumeSetupPath);

Restarting of Windows

After you set the required registry keys, you can restart Windows. The following example invokes the restart commands for different Windows operating systems.

OSVERSIONINFO ov;
ov.dwOSVersionInfoSize = sizeof(ov);

if ( GetVersionEx(&ov) )
{
    if(ov.dwPlatformId == VER_PLATFORM_WIN32_NT)
    {
        //NT platform
        HANDLE htoken ;
        //Ask for the SE_SHUTDOWN_NAME token as this is needed by the thread calling for a system shutdown.
        if (OpenProcessToken(GetCurrentProcess(), TOKEN_ADJUST_PRIVILEGES | TOKEN_QUERY, &htoken))
        {
            LUID luid ;
            LookupPrivilegeValue(NULL, SE_SHUTDOWN_NAME, &luid) ;
            TOKEN_PRIVILEGES    privs ;
            privs.Privileges[0].Luid = luid ;
            privs.PrivilegeCount = 1 ;
            privs.Privileges[0].Attributes = SE_PRIVILEGE_ENABLED ;
            AdjustTokenPrivileges(htoken, FALSE, &privs, 0, (PTOKEN_PRIVILEGES) NULL, 0) ;
        } 

        //on Whistler and future NT use InitiateSystemShutdownEx to avoid unexpected restart message box
        try
        {            
            if ( (ov.dwMajorVersion > 5) || ( (ov.dwMajorVersion == 5) && (ov.dwMinorVersion  > 0) ))
            {
                bExitWindows = InitiateSystemShutdownEx(0, _T(""), 0, TRUE, TRUE, REASON_PLANNED_FLAG);
            }
            else
            {
#pragma prefast(suppress:380,"ignore warning about legacy api")
                bExitWindows = InitiateSystemShutdown(0, _T(""), 0, TRUE, TRUE);
            }
        }
        catch(...)
        {
            //advapi32.dll call not available! Will not restart!
        }
    }
    else
    {
        //9x system - use ExitWindowsEx
#pragma prefast(suppress:380,"ignore warning about legacy api")
        return ExitWindowsEx(EWX_REBOOT, 0);
    }
}

Resetting Start Path of MSI Launch

Before restart, the current directory is the location of your Setup program, but after restart, the location becomes the system32 directory. Your Setup program should reset the current directory before each MSI call, as shown in the following example.

CString GetSetupPath()
{
    TCHAR file[MAX_PATH];
    GetModuleFileName(NULL, file, MAX_PATH * sizeof(TCHAR));
    CString path(file);
    int fpos = path.ReverseFind('\\');
    if (fpos != -1)
    {
        path = path.Left(fpos + 1);
    }

    return path;
}

Running the Application MSI

After the Visual Studio Shell installer returns ERROR_SUCCESS, you can run the MSI for your application. Because your Setup program is providing the user interface, start your MSI in quiet mode (/q), and with logging (/L), as shown in the following example.

TCHAR temp[MAX_PATH];
GetTempPath(MAX_PATH, temp);

CString boutiqueInstallCmd, msi, log;
CString cmdLine(MAKEINTRESOURCE("msiexec /q /I %s /L*vx %s REBOOT=ReallySuppress"));
CString name(MAKEINTRESOURCE("PhotoStudioIntShell.msi"));
log.Format(_T("\"%s%s.log\""), temp, name);
msi.Format(_T("\"%s%s\""), GetSetupPath(), name);
boutiqueInstallCmd.Format(cmdLine, msi, log);

//TODO: You can use MSI API to gather and present install progress feedback from your MSI.
dwResult = ExecCmd(boutiqueInstallCmd, FALSE);

See Also

Tasks

Walkthrough: Creating a Basic Isolated Shell Application

Concepts

Integrated Shell and Isolated Shell

Deploying an Isolated Shell-Based Application on a 64-Bit Operating System