Beta testing your app and in-app products

Applies to Windows Phone

No matter how carefully you test your app, there’s nothing like the real-world test of having other people use it. Before you put your app in the Windows Phone Store, consider releasing a beta version.

TipTip:

To learn about beta testing for Windows Store apps and universal Windows apps, see Beta test your app.

To run a beta test of your Windows Phone Store app, you submit your app to Dev Center as a beta. You’ll get a link to your app’s download page in the Windows Phone Store, and your beta testers can use that link to access the app. The beta app won't be discoverable by others through search or browsing in the Windows Phone Store.

This topic describes the benefits and limitations of beta publication. Depending on your needs, you may also want to consider Targeted app distribution as a real-world testing alternative.

  • Like beta publication: targeted app distribution is free, the app is not discoverable through search or by browsing the public Store, testers download the app through a restricted URL, the app becomes available within two hours of submission, and the app never expires.

  • Unlike beta publication: targeted app distribution lets you set a price for the app and in-app purchases, the app has all the functionality of a publically published app, you don’t need to republish the app to make it public, and an unlimited number of testers can access the app by visiting its URL.

For more info, see Targeted app distribution.

When you beta test an app, you let a set of people try out the app before it’s released to the general public. Your beta testers may discover issues that you’ve overlooked, such as misspellings, confusing app flow, or errors that could cause the app to crash. You’ll then have a chance to fix those problems before you release the app to the public, resulting in a more polished final product.

NoteNote:

To make your app available for public release after the beta, you’ll need to complete a separate app submission with a new app name. You can’t convert a beta app into a public app.

Here are some things to keep in mind about offering a beta version of your app through Dev Center:

  • Your beta app can support up to 10,000 testers.

  • Your beta will go through an automated certification process, and will be available in a few hours.

  • Beta apps must be free. You cannot charge a price for your beta app.

  • You shouldn’t associate a Windows Phone beta app with a published Windows Store app.

  • Beta distribution is free and doesn’t count against your free app submission limits.

  • You can update your beta at any time.

  • You can add or remove beta testers as you see fit. You don’t need to re-upload your app packages to update your list of testers.

  • Beta testers will need to have a Microsoft account in order to access the Windows Phone Store and download your app. You’ll need to know the email address associated with each tester’s Microsoft account in order to add them to your beta tester list. To create a new Microsoft account, testers can go to Microsoft account setup.

  • Beta testers must have a retail Windows Phone device in order to use the beta app, but the device doesn’t need to be unlocked or registered.

  • Beta testers don't have to be registered developers.

  • You can’t revoke access to the beta app after a tester downloads it. Once they have downloaded the beta app, they can continue to use it.

  • You will need to determine how you’d like to collect feedback. Consider providing an email or website link in the beta app so that testers can quickly provide comments.

  • The app name and package identity name that you use for your beta app must be different from the name you’ll use for the production (non-beta) version.

Submitting a beta app is very similar to submitting an app for general publication through the Windows Phone Store Dashboard. Follow these steps to get your beta certified and out to your testers:

  1. Sign in to Dev Center and go to Submit app. You must complete the App info and Upload and describe your app package(s) sections, just as you would for any app submission. Make sure that your beta app doesn’t use the same package identity name as a public Windows Phone app (or that you plan to use for your final, public app).

  2. In the App info section, click More Options, and click Beta.

  3. Add one or more testers by entering email addresses to the beta app’s access control list (ACL), using the text box in the More Options>Beta section. The ACL controls who can download and install your beta app. You can update this list later if needed.

    NoteNote:

    When you add email addresses to the ACL, make sure they’re separated by a semicolon.

  4. Complete the App info and Upload and describe your package(s) sections, then submit the beta app. After your beta is validated and published, you’ll receive an email that includes:

    • A link to your app in the Windows Phone Store.

      NoteNote:

      Your beta app won’t appear in Store search results. Anyone who has access to the beta app link can see the beta app’s Store listing info, but only testers on your beta’s ACL will be able to download it.

    • The email addresses of the testers you added to the ACL during app submission.

      NoteNote:

      You can change the list of beta testers at any time by submitting an update.

  5. Send the link to your beta app to everyone on the ACL. This link is stored in the app’s Detail view in Dev Center.

  6. When testers get the link, they can open it on their Windows Phone and download your beta app from the Store.

You can submit as many apps as you’d like for beta testing. Submissions are completely independent, so you’ll need to maintain the list(s) of testers yourself and provide your testers’ email addresses for each beta app submission.

While testers are using your beta app, you can monitor their feedback and make app updates.

Make sure you’ve let your testers know how they should provide feedback (via a web link in the app, sending email, or however you choose).

You can also review reports for your beta app in Dev Center just as you would for any app. You can check your download numbers to see how many beta testers have downloaded the app, although you won’t be able to see which testers have downloaded it. You can also see ratings and reviews left by your testers. For more about available reports, see Watch your progress.

You can update the beta app at any time by replacing the existing app package. To do so, go to your beta app in Dev Center and click Update app. Replace the existing app package with your new one and click Save. Then review and submit your app. The update should be available shortly, and you’ll receive an email with the beta app’s link in it when it’s available. Your testers will also be notified that an update is available.

NoteNote:

The link to download your app will stay the same, whether you’ve submitted updates or not.

You can test whether in-app products work properly by including them in your beta app.

NoteNote:

You can only use in-app products in Windows Phone 8 and Windows Phone 8.1 apps.

When you submit in-app products for a beta app, keep the following points in mind:

  • In-app products submitted for beta apps will only exist in that beta version.

  • An in-app product inherits the ACL from the app it’s associated with, and will only be available to testers on that list.

  • Your app’s beta testers don’t pay to test your in-app products.

At some point you’ll want to stop your beta test and release the production version of your app. Remember that the production version cannot have the same package identity as the beta version.

Beta apps don’t expire, so you have the option to keep your beta running in parallel with the real app for continued feedback from the beta community. However, you can choose to unpublish or delete your beta app at any time. If you do this, no new users can download the beta app.

NoteNote:

If you wish to restrict the usage of the beta app after your beta testing phase is over, you must build support for that in your app. Otherwise, beta testers that already have your app installed will be able to continue using it, even after you unpublish or delete the app.

Your beta testers will not automatically get “upgraded” to your production version. They’ll have to download it from the Windows Phone Store like any other customer in order to use it.

If you’re having issues with beta distribution and need help, contact us through Dev Center support.

Show:
© 2014 Microsoft