Beta Testing Your Application

1. What is beta testing?

Beta testing is a term used to describe the process of giving your application to a set of users to try before you release it to the general public. The goal of beta testing is to place your application in the hands of real users outside of your own engineering team to discover any flaws or issues from the user's perspective that you would not want to have in your final, released version of the application. The list of users that are given the application is typically restricted.

2. What are the advantages of beta testing my application?

• You have the opportunity to get your application into the hands of users prior to releasing it to the general public.

• Users can install, test your application, and send feedback to you during this beta testing period.

• Your beta testers can discover issues with your application that you may have overlooked, such as typographical errors, confusing application flow, and even crashes.

• Using the feedback you get from these users, you can fix problems before it is released to the general public.

• The more issues you fix that solve real user problems, the higher the quality of your application when you release it to the general public.

• Having a higher-quality application when you release to the general public will increase customer satisfaction.

• These users, who are early adopters of your application, will generate excitement about your application.

3. How can Microsoft help me beta test my application?

We provide a Beta Distribution option during application submission on App Hub that allows you to distribute your application through Windows Phone Marketplace to your test users before you are ready to submit it for certification and release to the public.

4. What are the main advantages of submitting my application for Beta Distribution?

• Submitting your application for beta distribution is a lightweight procedure in comparison to a full application submission.

• Your application will go through an automated certification process, making the time to publication much less than a full application submission.

• Submitting your application for beta distribution is free.

• Submitting your application for beta distribution does not count against your submission credits.

• You put your application into the hands of testers prior to releasing the application to the general public.

• You have complete control over who can participate in the beta testing of your application.

• Users download and install your application using the standard Windows Phone Marketplace procedures.

• Your application can be tested on retail devices. Your users do not have to be registered developers.

• Your beta application will automatically expire 90 days after it is published

5. What are the main steps for submitting my application for Beta Distribution?

1. Submit your application by using the App Hub application submission process. Select the Private Beta Test submission option. The first page of the submission process, which shows the Private Beta Test option, is shown in the following image.

    

   

2. During the submission process, you can define a set of beta testers by adding their Windows Live ID email addresses to your beta tester list. This list of users forms the access control list (ACL) for your beta distribution, and only those users can download and install the application. The following screen shot shows the part of the submission process that you use to add beta testers.

    

   

   Note:
   When adding email addresses to the beta tester list on the preceding screen shot, make sure they are separated by a semicolon.

3. After your application is validated and published in Windows Phone Marketplace, you will receive an email from the Windows Phone team that has a link to your application. The email will include the expiration date of your beta period, which is 90 days after the date you submit your application. The email also will contain the list of the email addresses of people you added to the beta tester ACL when you submitted your application. Note that you can modify the list of beta testers at any time. For more information, see question 14.

4. Send the link you received in the preceding email to all users that you have added to your beta tester list.

5. When a beta tester receives this link, they can tap it on their Windows Phone and download your application from the Windows Phone Marketplace.

6. After the application has been successfully installed, each beta tester will run your application and test it for any issues. During the beta testing period, you should have an open dialog with your testers and encourage them to send you feedback.

7. The license for your application will expire 90 days after the successful publication of your application through the beta distribution program. Your beta testers will no longer be able to run your application. At this point, when they attempt to start your application, they will be given the option to uninstall the application or send you feedback through the Windows Phone Marketplace client application.

8. During the 90-day beta-testing period for your application, you can update the list of users in your beta tester list.

6. How do I submit my application for Beta distribution?

To submit your application for beta testing, go to the Application Submission page on App Hub and select the Private Beta Test submission option. You will be guided through the rest of the process, where, among other things, you will specify a list of beta testers to test your application. For a complete guide on submitting your application, see App Submission Walkthough. For answers to frequently asked questions about application submission, see Application Submission.

7. What certification criteria does my application need to meet during the submission process?

• Your application should not violate the Windows Phone Marketplace Terms of Use. In the event Microsoft discovers a violation of the Windows Phone Marketplace Terms of Use, the developer account will be suspended and the beta application will be revoked.

• Your application will go through an automated certification process, making the time to publication much less than that for a full application submission.

8. How long does it take for my application to be published?

After you complete the beta application submission process, your application will go through an automated certification process and will be marked as published in a few hours. However, your testers will not be able to download the app until the download link is live on Windows Phone Marketplace. This can take up to 24 hours from the time you receive the email that confirms that your application has been published. Wait 24 hours before notifying your testers that the app is available to download.

Note:

To track when your testers will be able to see the app and download it from Windows Phone Marketplace, click your app’s deep link on the details section of the app’s page on App Hub. This link has the following format: http://windowsphone.com/s?appId=[App ID] [App ID] is the App ID of your app. When your app is available to download, this link will display the app’s page on Windows Phone Marketplace.

9. How much does it cost to submit an application for Beta distribution?

Submitting an application for beta distribution is free. It does not count against your allowed number of application submissions.

10. When does my beta testing period expire?

The beta testing period of your applications expires 90 days after your beta application has been successfully submitted to App Hub. When this happens, your application will no longer run on your beta participants’ phone and they will be given the option to send you feedback and uninstall the application.

11. What happens when my beta testing period expires?

The license for your application will expire 90 days after your beta application has been successfully submitted to App Hub. Your beta testers will no longer be able to run your application. At this point, when they attempt to start your application, they will be given the option to uninstall the application or send you feedback through the Windows Phone Marketplace client application.

12. What is a beta tester?

A beta tester is someone you have added to your beta tester list that you want to install and test your application. You have complete control over who is added to your list of testers. Ideally, these will be people who will try out your application and give you actionable feedback. The only requirements for a beta tester are that they must have a Windows Phone and they must have a valid Windows Live ID (email address). You will use this email address to add the tester to your beta tester list, and only people on this list will be able to download and install your application beta.

13. Is there a limit to the number of beta testers I can add to my beta tester list?

Yes. You can provide up to 100 email addresses (separated by semicolons) to invite your friends and acquaintances to test your beta application.

14. Can I modify my list of beta testers that are allowed to download and install my application?

Yes. You can modify your list of beta testers any time during the 90-day beta period. To do this, click Update testers on the lifecycle tab of the app’s page in your App Hub dashboard.

15. Do beta testers need to have a Windows Phone?

Yes. Each beta tester must have a Windows Phone to install and test your application.

16. Do beta testers need to be registered developers?

No. Anyone with a Windows Phone can participate in your application beta testing program.

17. How does a beta tester get my application to test?

After your application has been successfully submitted to through App Hub, you will be sent an email. The email will contain a link to your application on the Windows Phone Marketplace. Send this link to each participant you have listed in your beta tester list. Participants will need to sign into their Windows Phone using the supplied Windows live ID to download and install your application using the link that you sent to them.

18. Do my beta testers need to have a phone that has been unlocked for development?

No. Distribution of your application through the Beta Distribution program enables testing by any user from their personal device, without the need to unlock the phone.

19. Does the deployment of my beta application onto a beta tester’s phone count against the developer application limit?

No. Beta applications are installed onto a device as a retail application and do not count against the developer application limit.

20. Can beta applications be run on retail devices?

Yes. The beta application goes through the standard application packaging and processing process to ensure the application will run on a retail device.

21. How do I get feedback from my beta participants?

At the end of the beta testing period, the beta tester will be prompted to give you feedback through the standard Windows Phone Marketplace review client application on their device. We also recommend that you engage with your beta testing community during the beta-testing period to get as much feedback as possible from them.

22. Can I promote my application to a full application automatically?

No. To make your application available for public release, you must resubmit the application using the Application Submission procedure on App Hub.

23. Can I update my application during the beta distribution period?

No. You must resubmit your updated application for a new beta-testing period.

24. Can I terminate my beta program before the 90-day period is over?

No. All users who were sent the link to your beta application and have installed it can continue to use the application for 90 days following the successful submission of the application through the Windows Phone Marketplace.

25. How many applications can I submit for beta distribution?

You can submit as many applications as you like for beta testing. Each submission is completely independent, and the list of beta testers for each beta-testing program must be specified, and maintained, by you.

26. Can I test the trial functionality of my application using the beta program?

No. All applications submitted for beta distribution are made available as free, fully licensed, time-limited versions of your application. The license for the application expires 90 days from the date the application was published through the beta distribution program.

27. Can beta applications be priced?

No. All applications submitted for beta distribution are made available as free, fully licensed, time-limited versions of your application. The license for the application expires 90 days from the date the application was published through the beta distribution program.

28. Is my beta application discoverable by browsing or searching marketplace?

No.  Only people who click on the link you received when the application was published will be able to access the details page of the application.

29. Will anyone who has the link be able to browse to the details page of the application in the Windows Phone Marketplace?

Yes. Anyone with access to the link will be able to see the application metadata and description in the Windows Phone Marketplace. If this is a concern for you, we recommend that you obscure the name and description of your application when you submit the application for beta distribution.

30. Can anyone who is not on my beta tester list acquire my app?

No. Only people on your beta tester list can acquire your application and install it.

31. Can I stop my beta program?

No. Your beta testers can use your application for 90 days from the date your application was first published. Although you can delete your beta application from the Windows Phone Marketplace through App Hub, you cannot revoke its use from your beta testers’ phones before the beta-test period of 90 days expires.

32. Can I delete my beta application from App Hub?

You can remove your beta application from the Windows Phone Marketplace through App Hub after the 90-day expiration period has ended.

33. Where can I find my application’s link to send to my users?

When your application has been successfully submitted, we will send you an email containing a link to your application. For more information about the submission process and an example of the email that you will be sent, see What are the main steps in submitting my application for Beta Distribution? Alternatively, you can find the link to your application on the details section of the application page on App Hub. Copy this link and send it to all users who you want to install and use your application. An example of part of the details section for an application showing the Deep link is shown in the following figure.

34. What other application distribution options are available to me?

For a list of the application distribution options available, see Application Distribution Options.

35. How do I get help if I have an issue submitting my app?

The App Hub forum provides answers to common questions and help for known issues. When signed in, you can post a thread if you need answers that are not already provided.

For further App Hub assistance, those that have App Hub memberships can use the Support e-Form in the Windows Phone Dashboard to submit a ticket to the Windows Phone Marketplace Support Team. Also, check out the support resources in App Hub.

Note:
The Support e-Form is available only for assistance with submitting apps and games for Windows Phone Marketplace.