Hot-Add Memory Support in Windows Server

Updated: October 28, 2010

On x86-based and x64-based platforms platforms, some Windows Server editions support hot-add memory, which allows ranges of physical memory to be added to a running operating system without requiring a system reboot.

This article describes how system manufacturers can take advantage of hot-add memory support in the following Windows Server editions:

• Windows Server 2008 R2, Enterprise Edition and Datacenter Edition

• Windows Server 2008, Enterprise Edition and Datacenter Edition

• Windows Server 2003, Enterprise Edition and Datacenter Edition

On This Page

  Introduction
  Hardware Support for Hot-Add Memory
  Operating System Support for Hot-Add Memory
  Call to Action
  Related Information

Introduction

Hot-add memory minimizes a common source of server downtime: Requiring the server to be powered off and subsequently rebooted during memory upgrades. Hot-add memory allows ranges of physical memory to be added to a running operating system without requiring the system to be shut down. This allows an administrator to add memory to a running system to address existing or future needs for memory without incurring downtime for the system or applications.

Starting with Windows Server 2003 Service Pack 1, Enterprise Edition and Datacenter Edition supports hot-add memory on both x64-based and Intel Itanium-based 64-bit platforms.

Windows Server 2003, Enterprise Edition and Datacenter Edition support hot-add memory on x86-based ACPI platforms only.

 Top of page

Hardware Support for Hot-Add Memory

To support hot-add memory in hardware, system manufacturers must:

• Implement a mechanism for adding physical memory regions to an operating system instance without requiring a system power operation.

• Design the BIOS to describe hot-add memory accurately to the operating system.

Hardware Mechanisms for Adding Physical Memory Regions

Hot-add memory can be made available to the operating system by:

• Installing additional memory modules. The memory controller must support the ability to physically install memory modules without disrupting simultaneous access to existing memory, including any required power boundary management.

• Changing the hardware-defined partition that the operating system runs in to include additional memory ranges. The server must support hardware-enforced partitioning. The operating system runs on a subset of the hardware resources contained in the server. The administrator can add memory ranges to such a partition by changing the hardware partition boundaries. Such partition boundary changes are often managed from a dedicated service processor included in the server product.

The operating system does not distinguish between memory modules that are physically added to a machine that is designed to support that operation, and memory that materializes within an operating system instance through some combination of service processor and management console technology.

Describing Hot-Add Memory to the Operating System

The BIOS must describe hot-add memory accurately to the operating system by providing the following information:

• Provide a Static Resource Affinity Table (SRAT) to indicate where memory might potentially be hot-added

• Define memory device objects in the ACPI 2.0 namespace to describe hot-add memory ranges

• Report physical memory that is present on the system through the E820 mechanism

Note: To support hibernate/resume successfully, the BIOS must perform the POST configuration of PCI devices on resume from hibernate as it did in the previous boot, even though there will be additional physical memory in the machine.

Providing an SRAT

The Static Resource Affinity Table (SRAT) indicates where memory might potentially be hot-added (or hot-removed). At a minimum, the SRAT must contain memory affinity structures that define which ranges of memory can be hot plugged. If the system does not have NUMA properties, the proximity ID of the all affinity structures should be set to 0.

This information will be used to determine the size of data structures used by the operating system and to determine whether the loader should put the machine into PAE mode in anticipation of future hot-added memory with physical addresses above 4 GB.

For information about the SRAT, see Static Resource Affinity Table. This white paper also contains an example showing how memory that can be hot plugged is described using the SRAT.

Defining Memory Device Objects

Starting with Windows Server 2003, Enterprise Edition and Datacenter Edition, the operating system supports the concept of describing memory as a device, as defined in section 10.12 of the ACPI 2.0 specification. The ACPI 2.0 memory device object represents ranges of physical memory. Representing memory as a device allows the operating system to use existing Plug and Play and ACPI mechanisms to detect specific memory ranges that have been added to the operating system instance.

The ACPI 2.0 specification is available at
http://www.acpi.info/

Hot-add memory ranges must be described through ACPI 2.0 memory device objects. If memory ranges can be hot-added independently, they should be described by separate memory device objects. It is not necessary to support the full ACPI 2.0 namespace definition in order to define ACPI 2.0 memory device objects.

ACPI memory device objects have the same semantics as other ACPI objects with the following exceptions:

• Memory ranges described in the _CRS and _PRS objects represent physical memory that they make available to the operating system.

• The memory descriptors provided in the _CRS and _PRS methods must be less than 4GB in length, even if they are described by a QwordMemory descriptor.

• If a memory device object must describe more than 4GB of memory, the object must contain multiple ranges of less than 4GB that describe the hot-added memory.

The following example ASL code fragment defines an ACPI 2.0 memory device object that describes two memory regions:

• A 512MB region starting at 0x10000000 and ending at 0x2fffffff

• A 256MB region starting at 0x30000000 and ending at 0x3fffffff

Scope(\_SB){



    Device(MEM0) {



        Name(_HID, EISAID("PNP0C80"))



        Name(_CRS, ResourceTemplate() {



              QwordMemory(ResourceConsumer,



              ,



              MinFixed,



              MaxFixed,



              Cacheable,



              ReadWrite,



              0xfffffff,



              0x10000000,



              0x2fffffff,	



              0,



              ,,)



              QwordMemory(ResourceConsumer,



              ,



              MinFixed,



              MaxFixed,



              Cacheable,



              ReadWrite,



              0xfffffff,



              0x30000000,



              0x3fffffff,



              0,



              ,,)



        }



   }



}

Reporting Physical Memory at POST

The BIOS must use the E820 mechanism to report any memory present in the system at POST, even if that memory is also described by an ACPI memory device object. This ensures platform compatibility with earlier operating systems, including Windows 2000. Memory described by the E820 mechanism can overlap memory described by an ACPI memory device object.

The BIOS is not required to dynamically update the E820 information to reflect the hot-added memory. The operating system accesses E820 information only during bootstrap, so there is no benefit to having E820 calls to reflect memory that is hot-added after boot.

 Top of page

Operating System Support for Hot-Add Memory

Operating system support for hot-add memory consists of the following:

• Support for the memory device object as described in Section 10.12 of the ACPI 2.0 specification and in this article.

• A system-supplied kernel-mode driver for ACPI 2.0 memory device objects that describes the memory ranges contained in the objects to the Memory Manager.

• Changes to the Memory Manager to support dynamic addition of memory to a running system.

• Use of the Static Resource Affinity Table (SRAT) to determine the amount and ranges of hot-add memory and to determine whether to enable PAE on x86-based platforms when hot-add memory has addresses over 4GB.

When memory is hot added to the system, the following steps occur:

1. The ACPI BIOS generates an ACPI Notify(1) (device check) to the memory device object that describes the hot-added memory.

2. The operating system re-evaluates the _STA method on the memory device object to determine if it now claims that the memory is present.

3. If the memory is present, the Plug and Play subsystem loads the system-supplied driver for memory device objects.

4. This memory device object driver informs the Memory Manager about the new memory ranges.

5. The Memory Manager makes the new memory available to the operating system, drivers, and applications.

 Top of page

Call to Action

To support hot-add memory in hardware on x86-based and x64-based systems, manufacturers should:

• Implement a mechanism for adding physical memory regions to an operating system instance without requiring operating system downtime or system power operation.

• Design the BIOS to describe hot-add memory accurately to the operating system. In particular:

  • Provide a Static Resource Affinity Table (SRAT) to indicate where memory might potentially be hot-added.

  • Define memory device objects in the ACPI 2.0 namespace to describe hot-add memory ranges.

  • Report available physical memory through the E820 mechanism.

  • Perform the POST on resume from hibernate as it did in the previous boot, even though there is now more physical memory in the machine.

To ensure that hot -add memory hardware interoperates well with the supported editions of Windows Server, the Windows Development labs need an opportunity to test this feature on your platforms. Please contact Microsoft and provide test hardware as soon as possible so we can work together to ensure that hot add memory works well on your platform. For information about submitting hardware to Windows Development labs, write to wintrack@microsoft.com.

 Top of page

Related Information

Static Resource Affinity Table

ACPI 2.0 Specification at
http://www.acpi.info/

 Top of page