Modern standby Wi-Fi-connected scenarios

The goal of Wi-Fi-connected testing is to identify activities that should not be happening during modern standby and resolve them. The activities might be a result of network activities, real-time notifications, or unexpected hardware interrupts.

After the system can reliably enter and exit modern standby and can achieve good results in modern standby with airplane mode on, you can validate modern standby system behaviors with Wi-Fi connected.

To prepare for WiFi-connected testing, the following two prerequisites must be met:

• The system has a reliable Wi-Fi connection. The best way to assess this is to test the Wi-Fi connection during active use to make sure that the connection is stable and signal/speed is reasonably good.
• The Wi-Fi device is compatible with the NDIS 6.3 feature to support wake on LAN (WoL) patterns, protocol offloads, and D0 packet coalescing. These capabilities are required so that the SoC can enter low-power states while the Wi-Fi device maintains connectivity.

The list of Wi-Fi-connected scenarios includes connectivity to communications apps and downloads from the Internet. These scenarios should be tested serially to focus on issues that might fix similar issues in other scenarios. The Windows Hardware Certification Kit (HCK) includes tests that help validate basic NDIS 6.3 functionality for modern standby. There are individual tests for each type of network adapter, including Wi-Fi, CDMA MBB, GSM MBB, and wired LAN. Please see the test details information for each test at Device.Network Testing.

We recommend that each network adapter in the system pass the following tests before you proceed to system-level connectivity testing for modern standby:

• WLAN Connected Standby End to End - Basic
• Win.MBN.CDMA.TestCSConnectivity
• Win.MBN.GSM.TestCSConnectivity
• LAN CS Test - IPv4 Basic

Modern standby push mail

Testing the push mail scenario ensures that the system can receive emails via the Mail app while the system is in modern standby. This allows users to receive the most up-to-date information immediately after the system resumes.

We recommend that you create a Microsoft account for testing purposes. You can use this account to set up the system to receive emails through the Mail app. Be sure that the system is running on AC power, then put it into modern standby. While the system is in modern standby, send an email to the account a few times. Wake up the system from modern standby and verify that all the emails are received. If Mail is configured to be on the lock screen, you should also see the badge updates occur just as the screen is powered on.

• Test area: Push mail during modern standby.
• Purpose: Ensure that push mail is functional during modern standby.
• System configuration:
  • Factory image is installed on the system.
  • All drivers are loaded in Device Manager.
  • Factory Microsoft Store apps are installed.
  • System is running on AC power.
  • System is connected to a Wi-Fi access point (AP) that has Internet connectivity.
  • Mail app is configured with a test Microsoft account.
  • In Background Apps, Mail is set to On.
  • In Battery Usage:
    • Mail is set to Allow the app to run background tasks.
    • Mail is not set to Let Windows decide... or Reduce the work....
  • Email operation is first validated with the display on.

The Windows Hardware Certification Kit (HLK) includes modern standby tests that exercise entering and exiting modern standby on a system. Running the following tests will further validate the readiness of the system:

• Modern standby basic verification test on AC-power
• Modern standby basic verification test on DC-power

Modern standby instant messaging and video chat

NOTE: Skype no longer supports this functionality starting with version 8.61.

Instant-message and video-chat notifications are important Wi-Fi-connected scenarios. The goal of testing this scenario is to ensure users can receive instant messages and video calls in real-time while the system is in modern standby.

You can test the instant messaging scenario with Skype by putting the system into modern standby and sending an instant message to the test Microsoft account. You should observe that the system powers on the display when an instant message is received and shows an instant message toast notification on the lock screen.

After you verify that instant message notifications are working, try to make a video call and an audio conference call. When the system is in modern standby, make a video call to the Skype test account. You should observe that the system powers on the display and shows a ringing toast notification at the upper right hand corner. If you do not pick up the call and let it finish ringing, the system will immediately go back into modern standby.

When the system wakes from modern standby, make sure that all notifications are received as expected. When an instant message or a call is missed, the icon on the lock screen will show the number of missed messages and calls.

The instant messaging and video chat scenarios are enumerated in the following table. All modern standby systems should be tested for expected behavior for each scenario that is listed in the table.

• Test area: Instant messaging (IM) and video chat during modern standby.
• Purpose: Ensure that real-time incoming functionality is operational during modern standby.
• System configuration:
  • Factory image installed on the system.
  • All drivers loaded in Device Manager.
  • Factory Microsoft Store apps are installed.
  • System is running on battery power.
  • System is connected to a Wi-Fi AP with Internet connectivity.
  • Skype app configured with test Microsoft account.
  • Skype IM and video chat are validated to work with the screen turned on (outside of modern standby).
  • Skype has been configured to run in the background on the Battery Settings page

Modern standby background download

Background download through a Wi-Fi connection is an important modern standby scenario to test. This scenario includes downloads from Windows Update for critical updates, and downloads of music and movies while the system stays in modern standby.

The goal of testing background download is to make sure that download is done in the background and does not continuously block the system from entering the lowest power state during modern standby. Note that background download is allowed to continuously block the system from using the lowest power state when the system is running on AC power. Testing background download on battery is required.

To test background downloads, it is best to test using a large file. You should also ensure the system is running on battery (not connected to AC) to get the correct behavior since systems on AC have special download policies to allow downloads to continue uninterrupted.

One way to test background download is use the inbox Video app to kick off a large download (over 3 gigabytes), and then put the system into modern standby. Windows permits the download to run through an activator in the Resiliency phase. Please see Prepare software for Modern Sleep for a conceptual overview of this software flow.

The key observation in the test is to verify that the large download does not prevent the system from entering Modern Sleep.

On resume from modern standby, you can run a SleepStudy report to view the time when the system was in Modern Sleep and, more specifically, DRIPS, to determine whether the background download was blocking the system from entering low power. For the best testing scenaio, the modern standby session should extend for at least one hour.

• Test area: Background file download during modern standby.
• Purpose: Ensure that a background download is performed at low-power operation during a modern standby session.
• System configuration:
  • Factory image installed on the system.
  • All drivers loaded in Device Manager.
  • Factory Microsoft Store apps are installed.
  • System is running on battery power.
  • System is configured with a test Microsoft account for inbox Video app access.

Modern standby roaming connectivity

The most basic roaming connectivity scenario involves functional validation of network list offload (NLO). NLO is the capability of the Wi-Fi device to automatically roam between previously associated Wi-Fi networks, without the involvement of Windows running on the SoC.

NLO starts with Windows tracking the list of previously associated Wi-Fi networks. When the system transitions into modern standby, Windows communicates the list of previously associated Wi-Fi networks to the Wi-Fi device.

If connectivity to the associated Wi-Fi network is lost, the Wi-Fi device periodically searches for one of the previously associated Wi-Fi networks while the device operates in a very low power mode. The periodic scans for a previous network are performed without waking the SoC so that the SoC can stay in a low-power mode. When the Wi-Fi device detects a previously used network, it connects to the network and wakes the SoC. Windows completes the connection, including obtaining an IP address and re-establishing system connections, including Windows Notification Services (WNS).

Start testing Wi-Fi roaming by setting up two Wi-Fi networks (A and B) with public Internet connections. Each Wi-Fi network should have a different SSID and be sufficiently distant so that the system cannot see both networks at the same time. Manually move the system near each Wi-Fi access point and connect to each network for a few minutes with the screen on and validate that Internet connectivity is functional. After the system has been demonstrated to connect to each Wi-Fi network reliably, you can start testing Wi-Fi roaming in modern standby.

Modern standby Wi-Fi roaming

The most basic roaming connectivity scenario involves functional validation of network list offload (NLO). NLO is the capability of the Wi-Fi device to automatically roam between previously associated Wi-Fi networks, without the involvement of Windows running on the SoC.

NLO starts with Windows tracking the list of previously associated Wi-Fi networks. When the system transitions into modern standby, Windows communicates the list of previously associated Wi-Fi networks to the Wi-Fi device.

If connectivity to the associated Wi-Fi network is lost, the Wi-Fi device periodically searches for one of the previously associated Wi-Fi networks while the device operates in a very low power mode. The periodic scans for a previous network are performed without waking the SoC so that the SoC can stay in a low-power mode. When the Wi-Fi device detects a previously used network, it connects to the network and wakes the SoC. Windows completes the connection, including obtaining an IP address and re-establishing system connections, including Windows Notification Services (WNS).

Start testing Wi-Fi roaming by setting up two Wi-Fi networks (A and B) with public Internet connections. Each Wi-Fi network should have a different SSID and be sufficiently distant so that the system cannot see both networks at the same time. Manually move the system near each Wi-Fi access point and connect to each network for a few minutes with the screen on and validate that Internet connectivity is functional. After the system has been demonstrated to connect to each Wi-Fi network reliably, you can start testing Wi-Fi roaming in modern standby.

• Test area:Wi-Fi roaming operation during modern standby.
• Purpose:Ensure that the system will automatically connect to previously used Wi-Fi networks during modern standby as the user moves the system between home, work, and coffee shops.
• System configuration:
  • Factory image is installed on the system.
  • All drivers are loaded in Device Manager.
  • Factory Microsoft Store apps are installed.
  • System is running on battery power.
  • Wi-Fi connectivity is validated to work by connecting to the Internet while the screen is turned on.

The test engineer for Wi-Fi roaming should build a close working relationship with the Wi-Fi vendor's support staff. Errors in this test should first be discussed with the Wi-Fi firmware developer.

A common problem is that the Wi-Fi device generates spurious SoC wake events because of false-positive NLO network detection events. This condition appears in SleepStudy as extra Wi-Fi device active time—typically, much more active time than any other PDC activator (such as BI, Windows Update, or WNS).

Modern standby Wi-Fi to MBB failover

During modern standby, if Wi-Fi connectivity is lost, Windows automatically transitions to an available MBB connection. This behavior enables the system to remain constantly connected during modern standby. However, Windows will favor the cheaper, faster, and (typically) more power-efficient Wi-Fi connection over a cellular-based connection.

MBB and cellular connectivity are highly influenced by signal strength and proximity to a cellular tower. Make sure that the system under test has good cellular network connectivity before you test this scenario.

You should fully validate basic Wi-Fi roaming before you try to validate failover from Wi-Fi to MBB. The same Wi-Fi NLO technology that enables basic Wi-Fi roaming is critical to the operation of Wi-Fi to MBB failover.

The system under test should be configured to have an active cellular data plan and SIM card. The test setup should include a Wi-Fi access point with public Internet access. We recommend that you configure the system under test with a Microsoft account, and set up an additional device that is configured to initiate email and Skype calls.

Start the Wi-Fi to MBB failover testing by confirming that the system is connected to the Wi-Fi access point and has MBB access. You should validate that both network types are functional with the screen on. To do this, connect to Wi-Fi with the MBB radio off, and then connect to MBB with the Wi-Fi radio off. For each network connection, you should connect to the Internet and browse the web to verify IP connectivity.

Next, put the system in modern standby with both Wi-Fi and MBB radios turned on and the system connected to the Wi-Fi network. Use a Wi-Fi connection to test push mail and Skype during modern standby. While the system under test remains in modern standby, power down the Wi-Fi access point and wait a few minutes for the system to connect to the cellular network. After a few minutes, re-test push mail and Skype calls to validate that Windows automatically connected to the cellular network and re-established connectivity.

• Test area:Wi-Fi to MBB transitions during modern standby.
• Purpose:Ensure that the system automatically connects to Wi-Fi or MBB during modern standby.
• System configuration:
  • Factory image installed on the system.
  • All drivers loaded in Device Manager.
  • Factory Microsoft Store apps are installed.
  • System is running on battery power.
  • Wi-Fi and MBB are validated to function correctly with the screen on and manually configured connections to each network.