Export (0) Print
Expand All
Information
The topic you requested is included in another documentation set. For convenience, it's displayed below. Choose Switch to see the topic in its original location.

Windows App Certification Kit tests for Windows Phone

Applies to Windows Phone only

The Windows App Certification Kit contains a number of tests that can help ensure that your Windows Phone 8.1 app is ready to be published on the Store.

Test namePossible test resultsCertification requirements testedOS applicability

Package compliance test

Pass

Fail

App manifest compliance

Windows Phone 8.1

Application count

Package size check

Bundle manifest validation

Windows security features test

Pass

Fail

Binary analyzer

Windows Phone 8.1

Banned file analyzer

Private code signing

Supported APIs test

Pass

Fail

Supported APIs

Windows Phone 8.1

App manifest resources test

Pass

Fail

App resources validation

Windows Phone 8.1

Branding validation

Debug configuration test

Pass

Fail

Debug configuration

Windows Phone 8.1

File encoding test

Pass

Fail

UTF-8 file encoding

Windows Phone 8.1

App capabilities test

Pass

Fail

Special use capabilities

Windows Phone 8.1

Windows Runtime metadata validation

Pass

Fail

ExclusiveTo attribute test

Windows Phone 8.1

Type location test

Type name case-sensitivity test

Type name correctness test

General metadata correctness test

Properties test

Package Sanity check

Pass

Fail

Platform appropriate files

Windows Phone 8.1

Package Validation

Resource usage test

Pass

Fail

WinJS background task test

Windows Phone 8.1

 

Tests for Windows Phone 8.1 app certification

Package compliance test

Tests the contents of the package to ensure the contents are valid.

App manifest compliance test

Tests the contents of the app manifest to ensure the contents are valid.

Background

Apps must have a correctly formatted app manifest.

Test details

Examines the app manifest to verify the contents are correct as described in the Package metadata requirements.

  • File extensions and protocols

    Your app can declare the file extensions that it wants to associate with. Used improperly, an app can declare a large number of file extensions, most of which it may not even use, resulting in a bad user experience. This test will add a check to limit the number of file extensions that an app can associate with.

  • Framework Dependency rule

    This test enforces the requirement that Windows Phone 8.1 apps take appropriate dependencies on the Windows Runtime. If there is an inappropriate dependency, this test will fail.

    FrameworkOS MinRuns on Windows Phone 8.1

    Microsoft.VCLibs.120.00.Phone

    6.3.1

    Yes

     

    If there is a mismatch between the OS version the app applies to and the framework dependencies made, the test will fail. The test will also fail if the app refers to any preview versions of the framework .dlls.

Corrective action

Review the app manifest against the requirements described in the Package metadata requirements.

Windows Security features test

Background

Changing the default Windows security protections can put customers at increased risk.

Test details

Tests the app's security by running the BinScope Binary Analyzer.

The BinScope Binary Analyzer tests examine the app's binary files to check for coding and building practices that make the app less vulnerable to attack or to being used as an attack vector.

The BinScope Binary Analyzer tests check for the correct use of the following security-related features.

BinScope Binary Analyzer tests

The BinScope Binary Analyzer tests examine the app's binary files to check for coding and building practices that make the app less vulnerable to attack or to being used as an attack vector.

The BinScope Binary Analyzer tests check for the correct use of these security-related features:

AllowPartiallyTrustedCallersAttribute

Windows App Certification Kit error message: APTCACheck Test failed

The AllowPartiallyTrustedCallersAttribute (APTCA) attribute enables access to fully trusted code from partially trusted code in signed assemblies. When you apply the APTCA attribute to an assembly, partially trusted callers can access that assembly for the life of the assembly, which can compromise security.

What to do if your app fails this test

Don't use the APTCA attribute on strong named assemblies unless your project requires it and the risks are well understood. In cases where it's required, make sure that all APIs are protected with appropriate code access security demands. APTCA has no effect when the assembly is a part of a Windows Store app.

Remarks

This test is performed only on managed code (C#, .NET, etc.).

/SafeSEH Exception Handling Protection

Windows App Certification Kit error message: SafeSEHCheck Test failed

An exception handler runs when the app encounters an exceptional condition, such as a divide-by-zero error. Because the address of the exception handler is stored on the stack when a function is called, it could be vulnerable to a buffer overflow attacker if some malicious software were to overwrite the stack.

What to do if your app fails this test

Enable the /SAFESEH option in the linker command when you build your app. This option is on by default in the Release configurations of Visual Studio. Verify this option is enabled in the build instructions for all executable modules in your app.

Remarks

The test is not performed on 64-bit binaries or ARM chipset binaries because they don't store exception handler addresses on the stack.

Data Execution Prevention

Windows App Certification Kit error message: NXCheck Test failed

This test verifies that an app doesn't run code that is stored in a data segment.

What to do if your app fails this test

Enable the /NXCOMPAT option in the linker command when you build your app. This option is on by default in linker versions that support Data Execution Prevention (DEP).

Remarks

We recommend that you test your apps on a DEP-capable CPU and fix any failures you find that result from DEP.

Address Space Layout Randomization

Windows App Certification Kit error message: DBCheck Test failed

Address Space Layout Randomization (ASLR) loads executable images into unpredictable locations in memory, which makes it harder for malicious software that expects a program to be loaded at a certain virtual address to operate predictably. Your app and all components that your app uses must support ASLR.

What to do if your app fails this test

Enable the /DYNAMICBASE option in the linker command when you build your app. Verify that all modules that your app uses also use this linker option.

Remarks

Normally, ASLR doesn't affect performance. But in some scenarios there is a slight performance improvement on 32-bit systems. It is possible that performance could degrade in a highly congested system that have many images loaded in many different memory locations.

This test is performed on only apps written in managed code, such as by using C# or .NET Framework.

Read/Write Shared PE Section

Windows App Certification Kit error message: SharedSectionsCheck Test failed.

Binary files with writable sections that are marked as shared are a security threat. Don't build apps with shared writable sections unless necessary. Use CreateFileMapping or MapViewOfFile to create a properly secured shared memory object.

What to do if your app fails this test

Remove any shared sections from the app and create shared memory objects by calling CreateFileMapping or MapViewOfFile with the proper security attributes and then rebuild your app.

Remarks

This test is performed only on apps written in unmanaged languages, such as by using C or C++.

AppContainerCheck

Windows App Certification Kit error message: AppContainerCheck Test failed.

The AppContainerCheck verifies that the appcontainer bit in the portable executable (PE) header of an executable binary is set. Windows Phone 8.1 apps must have the appcontainer bit set on all .exe files and all unmanaged DLLs to execute properly.

What to do if your app fails this test

If a native executable file fails the test, make sure that you used the latest compiler and linker to build the file and that you use the /appcontainer flag on the linker.

If a managed executable fails the test, make sure that you used the latest compiler and linker, such as Microsoft Visual Studio, to build the Windows Phone 8.1 app.

Remarks

This test is performed on all .exe files and on unmanaged DLLs.

ExecutableImportsCheck

Windows App Certification Kit error message: ExecutableImportsCheck Test failed.

A portable executable (PE) image fails this test if its import table has been placed in an executable code section. This can occur if you enabled .rdata merging for the PE image by setting the /merge flag of the Visual C++ linker as /merge:.rdata=.text.

What to do if your app fails this test

Don't merge the import table into an executable code section. Make sure that the /merge flag of the Visual C++ linker is not set to merge the “.rdata” section into a code section.

Remarks

This test is performed on all binary code except purely managed assemblies.

WXCheck

Windows App Certification Kit error message: WXCheck Test failed.

The check helps to ensure that a binary does not have any pages that are mapped as writable and executable. This can occur if the binary has a writable and executable section or if the binary’s SectionAlignment is less than PAGE_SIZE.

What to do if your app fails this test

Make sure that the binary does not have a writeable or executable section and that the binary's SectionAlignment value is at least equal to its PAGE_SIZE.

Remarks

This test is performed on all .exe files and on native, unmanaged DLLs.

An executable may have a writable and executable section if it has been built with Edit and Continue enabled (/ZI). Disabling Edit and Continue will cause the invalid section to not be present.

PAGE_SIZE is the default SectionAlignment for executables.

Banned File Analyzer

Windows App Certification Kit error message: Banned File Check test failed.

Windows Phone 8.1 apps should not contain certain files. These files have a newer version available that provides important security, reliability or other improvements. Microsoft blocks these files in the Windows App Certification Kit to ensure that all developers use a current version.

Test details

The Banned File Check in the Windows App Certification Kit currently checks for the following files:

  • Bing.Maps.JavaScript\js\veapicore.js

    This check commonly fails when an app is using a "Release Preview" version of the file instead of the latest official release.

    The latest version of the Windows App Certification Kit provided with the Windows 8.1 SDK has added a check for pre-release versions of the Microsoft Advertising SDK. If pre-release versions of the following files are found during the banned file test phase, you’ll need to update to the released version of the Microsoft Advertising SDK:

    • Microsoft.Advertising.Mobile.Display.winmd
    • Microsoft.Advertising.Mobile.UI.winmd
    • Microsoft.Advertising.Mobile.UI.Design.dl

Corrective actions

To correct this, use the latest version of the Bing Maps SDK for Windows Store Apps.

To correct errors with the Microsoft Advertising SDK, use the latest version of the Microsoft Advertising SDK.

Private Code Signing

Tests for the existence of private code signing binaries within the app package.

Background

Private code signing files should be kept private as they may be used for malicious purposes in the event they are compromised.

Test details

Tests for files within the app package that have an extension of .pfx or.snk that would indicate that private signing keys were included.

Corrective actions

Remove any private code signing keys (e.g. .pfx and .snk files) from the package.

Application count test

Tests the contents of the app package to ensure that only one app is present.

Background

Windows Phone 8.1 app packages can only contain a single app when submitted to the Store.

Corrective actions

Ensure that the app manifest defines only one app per package.

Package size check

Tests the size of the app package to ensure it does not exceed 4GB.

Background

Windows Phone 8.1 app packages are restricted to a size of 4GB for submission to the Store. Note that this restriction applies to the package, not to expanded contents.

Corrective actions

Reduce the size of the app package to ensure that the releasing version is within the allowed 4GB size limitation.

Bundle manifest validation

Tests to ensure that there is only one main app package in a bundle and that the architecture is either ARM or Neutral.

Background

Windows Phone 8.1 app bundles are expected to contain a single main app package that targets either an ARM or Neutral architecture. There can be up to 512 packages in the bundle.

Corrective actions

Update the app package to ensure that there is only a single main app package in the bundle, and that the appropriate architecture is defined. You may need to remove x86 and/or x64 architecture packages from the bundle. If the bundle contains more than 512 packages, you will need to reduce the number of packages to meet the requirement.

Supported API test

Supported APIs

Tests the app for the use of any non-compliant APIs.

Background

Apps must use the APIs for Windows Phone 8.1 apps to be certified for the Store. This test also identifies situations where a managed binary takes a dependency on a function outside of the approved Windows Phone 8.1 profile.

Test details

  • Verifies that each binary within the app package doesn't have a dependency on an unsupported API for Windows Phone 8.1 app development by checking the import address table of the binary.

  • Verifies that each managed binary within the app package doesn't have a dependency on a function outside of the approved Windows Phone 8.1 profile.

Corrective actions

Make sure that the app was compiled as a release build and not a debug build.

Note  The debug build of an app will fail this test even if the app uses only APIs for Windows Phone 8.1 apps.

Review the error messages to identify the APIs used by the app that are not considered an API for Windows Phone 8.1 apps.

App Manifest resources test

App resource validation

The app may not install if the strings or images declared in the app manifest are incorrect. The app will not install with these errors, or your app's logo and other images may not display correctly.

Test details

Inspects the resources defined in the app manifest to make sure that they are both present and valid.

Corrective actions

Use the following table as guidance.

Error MessageComments

The image {imageName} defines both Scale and TargetSize qualifiers. You can define only one qualifier at a time.

You can customize images for different resolutions.

In the actual message, {imageName} contains the name of the image with the error.

Make sure that each image defines either Scale or TargetSize as the qualifier. For more info, see How to name resources using qualifiers.

The image {imageName} failed to meet the size restrictions.

Ensure that all the app images adhere to the proper size restrictions.

In the actual message, {imageName} contains the name of the image with the error.

For more info about app images and sizes, see App images.

The image {imageName} is missing from the package.

A required image is missing.

In the actual message, {imageName} contains the name of the image that is missing.

For more info about app images and sizes, see App images.

The image {imageName} is not a valid image file.

Ensure that all the app images adhere to the proper file format type restrictions.

In the actual message, {imageName} contains the name of the image that is not valid.

For more info about app images and sizes, see App images.

The image "BadgeLogo" has an ABGR value {value} at position (x,y) that is not valid. The pixel must be white (##FFFFFF) or transparent (00######)

The badge logo is an image that appears next to the badge notification to identify the app on the lock screen. This image must be monochromatic (it can contain only white and transparent pixels).

In the actual message, {value} contains the color value in the image that is not valid.

For more info about app images and sizes, see App images.

The image "BadgeLogo" has an ABGR value {value} at position (x,y) that is not valid for a high-contrast white image. The pixel must be (##2A2A2A) or darker, or transparent (00######).

The badge logo is an image that appears next to the badge notification to identify the app on the lock screen. Because the badge logo appears on a white background when in high-contrast white, it must be a dark version of the standard badge logo. In high-contrast white, the badge logo can only contain pixels that are darker than (##2A2A2A) or transparent.

In the actual message, {value} contains the color value in the image that is not valid.

For more info about app images and sizes, see App images.

The image must define at least one varient without a TargetSize qualifier. It must define a Scale qualifier or leave Scale and TargetSize unspecified, which then defaults to Scale-100.

For more info, see Guidelines for scaling to pixel density and How to name resources using qualifiers.

The package is missing a "resources.pri" file.

If you have content that can be localized in your app manifest, make sure that the app package includes a valid resources.pri file.

The "resource.pri" file must contain a resource map with a name that matches the package name {packageFullName}

You can get this error if the manifest changed and the name of the resource map in the resources.pri no longer matches the package name in the manifest.

In the actual message, {packageFullName} contains the package name that resources.pri must contain.

To fix this, you need to rebuild resources.pri. The easiest way to do this is by rebuilding the app package.

The "resources.pri" file must not have AutoMerge enabled.

MakePRI.exe supports an option called AutoMerge. The default setting for this option is Off.

When enabled, AutoMerge enables the app language pack resources into a single resources.pri at runtime.

The string {string} failed the max length restriction of {number} characters.

Refer to the Package Metadata Requirements.

In the actual message, {string} is replaced by the string with the error and {number} contains the maximum length.

The string {string} must not have leading/trailing whitespace.

The schema for the elements in the app manifest don't allow leading or trailing white space characters.

In the actual message, {string} is replaced by the string with the error.

Make sure that none of the localized values in the manifest fields in resource.pri have leading or trailing white space characters.

The string must be non-empty (greater than zero in length).

For more info, see Package Metadata Requirements.

There is no default resource specified in the "resources.pri" file.

For more info, see Defining app resources.

There is no resource value specified in the "resources.pri" file.

Make sure that the app manifest has valid resources defined in resources.pri.

For more info, see Localizing the package manifest.

 

See App package resource validation for more info.

Branding validation

Windows Phone 8.1 apps are expected to be complete and fully functional. Apps using the default images (from templates or SDK samples) present a poor user experience and cannot be easily identified in the Store catalog.

Test details

The test will validate if the images used by the app are not default images from either the SDK samples or Visual Studio templates.

Corrective actions

Replace default images with something distinct and representative of your app.

Debug configuration test

Debug configuration

Test the app to make sure that it is not a debug build.

Background

To be certified for the Store, apps must not compile for debug and they must not reference debug versions of an executable file. Additionally, you must build your code as optimized to pass this test.

Test details

Test the app to make sure that it is not a debug build and is not linked to any debug framework.

Corrective actions

  • Build the app as a release build before you submit it to the Store.
  • Make sure that you have the correct version of the .NET Framework installed.
  • Make sure the app isn't linking to debug versions of a framework and that it is with a release version. If the app contains Microsoft .NET components, make sure that you install the correct version of the .NET Framework.

File encoding test

UTF-8 file encoding

Background

HTML, CSS, and JavaScript files must be encoded in UTF-8 form with a corresponding byte-order mark (BOM) to benefit from bytecode caching and avoid certain runtime error conditions.

Test details

Test the contents of app packages to make sure that they use the correct file encoding.

Corrective Action

Open the affected file and select Save As from the File menu in Visual Studio. Select the drop-down control next to the Save button and select Save with Encoding. From the Advanced save options dialog, choose the Unicode (UTF-8 with signature) option and click OK.

App Capabilities test

Special use capabilities

Background

Special use capabilities are intended for very specific scenarios. Only company accounts are allowed to use these capabilities.

Test Details

Validate if the app is declaring any of the below capabilities:

  • EnterpriseAuthentication
  • SharedUserCertificates
  • DocumentsLibrary

If any of these capabilities are declared, the test will display a warning to the user.

Corrective Actions

Consider removing the special use capability if your app doesn't require it. Additionally, use of these capabilities are subject to additional on-boarding policy review.

Windows Runtime metadata validation

Background

Ensures that the Windows Runtime components that ship in an app conform to the Windows Runtime type system.

Test Details

Verifies that the .winmd files in the package conform to Windows Runtime rules.

Corrective Actions

  • ExclusiveTo attribute test: Ensure that Windows Runtime classes don't implement interfaces that are marked as ExclusiveTo another class.
  • Type location test: Ensure that the metadata for all Windows Runtime types is located in the winmd file that has the longest namespace-matching name in the app package.
  • Type name case-sensitivity test: Ensure that all Windows Runtime types have unique, case-insensitive names within your app package. Also ensure that no Windows Runtime type name is also used as a namespace name within your app package.
  • Type name correctness test: Ensure there are no Windows Runtime types in the global namespace or in the Windows top-level namespace.
  • General metadata correctness test: Ensure that the compiler you are using to generate your Windows Runtime types is up to date with the Windows Runtime specifications.
  • Properties test: ensure that all properties on a Windows Runtime class have a get method (set methods are optional). Ensure that the type of the get method return value matches the type of the set method input parameter, for all properties on Windows Runtime types.

Package Sanity tests

Platform appropriate files test

Apps that install mixed binaries may crash or not run correctly depending upon the user's processor architecture.

Background

This test validates the binaries in an app package for architecture conflicts. An app package should not include binaries that can't be used on the processor architecture specified in the manifest. Including unsupported binaries can lead to your app crashing or an unnecessary increase in the app package size.

Test Details

Validates that each file's "bitness" in the PE header is appropriate when cross-referenced with the app package processor architecture declaration

Corrective Action

Follow these guidelines to ensure that your app package only contains files supported by the architecture specified in the app manifest:

  • If the Target Processor Architecture for your app is Neutral processor Type, the app package cannot contain x86, x64, or ARM binary or image type files.

  • If the Target Processor Architecture for your app is ARM processor type, the app package must only contain ARM binary or image type files. If the package contains x64 or x86 binary or image type files, it will fail the test.

Package validation

This test verifies that the app packages are compliant with Windows Phone 8.1 rules for XAP packages.

Background

This test validates a number of elements that exist within the compressed XAP package to ensure compliance with Windows Phone 8.1.

Test details

For a list of specific error messages that you may encounter, refer to Understanding app submission errors.

Corrective action

Review and address any errors you may encounter from this test.

Note  While this test only applies to XAP based apps, the final report will still reflect this test for non-XAP based apps with a pass result.

Resource Usage test

WinJS Background Task test

WinJS background task test ensures that JavaScript apps have the proper close statements so apps don’t consume battery.

Background

Apps that have JavaScript background tasks need to call Close() as the last statement in their background task. Apps that do not do this could keep the system from returning to connected standby mode and result in draining the battery.

Test Details

If the app does not have a background task file specified in the manifest, the test will pass. Otherwise the test will parse the JavaScript background task file that is specified in the app package, and look for a Close() statement. If found, the test will pass; otherwise the test will fail.

Corrective Action

Update the background JavaScript code to call Close() correctly.

 

 

Show:
© 2014 Microsoft