PC Storage Directions

Updated: December 4, 2001

User demand for PC storage capacity continues to grow, especially for large audio, video, image, and multimedia files. This capacity demand affects hard disks and removable media such as CDs and DVDs.

PC storage technologies are evolving to meet user demands for increasingly greater capacity and more flexible capabilities.

The Microsoft Windows XP operating system delivers new support for storage devices, including performance optimizations, support for the largest hard disks and for storing data on optical disks, plus the ability to boot the PC from new kinds of devices on new connection technologies.

On This Page

  Consumer Storage Options
  Issues for PC Storage
  Windows XP and Storage
  Windows and DVD
  Windows Logo Program Requirements
  Guidelines for Manufacturers
  Industry Standards and Activity
  Industry Standards
  Terms

Consumer Storage Options

This article focuses on storage options for consumer PCs in general, and for PCs that will run Windows XP Home Edition or Windows XP Professional.

Common PC storage options support a variety of applications and capacities:

• Hard disk for PC data storage. Current disk size limits of 130 GB will be surpassed with the upcoming availability of devices supporting new ATA/ATAPI-6 48-bit logical block addressing (LBA) technology.

  This article focuses on ATA as the connector for the host controller on client PCs, with some discussion of IEEE 1394 storage.

• CD optical devices for PC data and audio storage. These are typically used for software installation and audio recording and playback. Current size limits are 650 MB of data or 74 minutes of audio.

  CD-RW optical devices are becoming increasingly popular with users because of their ability to hold much more data than floppy disks or other removable storage media.

• DVD for video playback, and storage of audio, video, and PC data. These discs hold 4.7 GB to 17 GB of data, or more than a full-length feature movie.

  Users are increasingly choosing DVD drives as a PC option because DVD was designed for multimedia applications such as video playback.

  This article describes new support and background information about various DVD formats. However, it does not discuss DVD device design and the graphics subsystem support required for a PC that will support a good video playback experience.

• Flash memory devices. PC Card and other miniature form factors have been increasingly popular with mobile PC users.

  There is a growth of consumer interest in digital cameras, for which Microsoft has provided new easy transfer, editing, printing, and Web publishing support in Windows XP. We expect to see a growing consumer interest in being able to transfer images from the new proprietary miniature storage devices to the PC.

  This article does not describe this type of storage. Microsoft expects to see USB implemented as the popular interconnect for PC devices that can read such media.

 Top of page

Issues for PC Storage

Ease of Use Issues. New PC storage devices can support a variety of user applications. Both Windows developers and device designers have to address issues such as the following:

• Removable media drives and new Windows boot capabilities. BIOS issues, unique removable media identifiers, dynamic configuration changes on hot-pluggable buses such as USB or IEEE 1394all create technical implementation problems, and make it necessary that system designers thoroughly understand Windows built-in support. In addition, ease-of-use issues must be addressed to both protect data and allow flexibility for the user.

• Floppy drive replacement. The limited storage capacity of floppy disks compared to other removable media has made the floppy drive a relic of the past. Cost-effective, easy-to-use, read/write optical media devices can solve the problem of writing large amounts of data to a single disk. However, new issues of compatibility arise, if the resulting disc cannot play in other CD or DVD drives.

New Boot Models. Windows XP delivers boot support for a variety of new media and connection options. Giving users full flexibility for using different boot storage devices requires correct, standards-based firmware support for:

• Boot order/drive selection in the BIOS

• Boot support for ATAPI, CD, DVD, and IEEE 1394 devices

Performance Improvements. In particular, to realize the fast system startup/fast resume goals for Windows XPespecially in home scenarios where quiet operation and instant access are importantimprovements in hard disk performance are required. This also requires BIOS refinements and support for bus mastering.

Compatibility and Installed Base. Most PCs shipped today include a high-capacity removable media drive, typically a CD-ROM or CD-RW drive. Lower cost has been a primary reason, to date, for the popularity of CD technology over DVD and other removable media. However, users expect continued support from PCs and from Windows for the full range of current storage options.

One crucial element of this support is selection of the file system for devices. For compatibility and performance reasons, Microsoft believes that this comes down to these decisions for devices that will share data with PCs:

• NTFS for the hard disk file system

• FAT or FAT32 for removable media, including flash-type memory

• CDFS (ISO-9660) for CD-R or CD-RW media

 Top of page

Windows XP and Storage

New and enhanced support in Windows XP focuses on these critical areas:

• Improvements for system boot and resume processes, including functions that reduce and optimize disk reads.

• Native support for CD-R/RW through a new Imaging Mastering API (IMAPI), to make copying files to CD as easy as it is to copy files to a floppy disk.

• New support for DVD-RAM.

• Enhanced support for multi-function MMC-compliant drives.

• Solid support for ATA/ATAPI controllers and devices, including support for 100-MHz ATA hard disks.

• Support for system manufacturers to select cycle times, to enable faster Ultra DMA (UDMA) support.

• Support for removable media identification.

  Based on proposals to the T13 and T10/MMC working groups, this support enables both software- and user-based content protection schemes.

Windows XP also includes support for IEEE 1394-connected hard disks, as was implemented in Windows Millennium Edition (Windows Me). Driver support for IEEE 1394 hard disks is based on the Serial Bus Protocol-2 (SBP-2) specification.

Planned Additions. Support will be implemented in Windows XP for the new ATA/ATAPI-6 48-bit LBA technology when this technology reaches market. This technology is intended to overcome current 130-GB size limitations for ATA drives. Microsoft also plans to expand IMAPI to include DVD technologies.

Storage Device Drivers in the Windows DDK

http://www.microsoft.com/whdc/devtools/ddk/default.mspx

Boot Support for Windows XP

Under Windows XP, it is possible to boot the system from hard drives connected to ATA, SCSI, Fibre Channel, and the IEEE 1394 bus. (Note, however, that IEEE 1394 support does not include support for dynamic disks.)

It is also possible to boot and install the operating system from CD or DVD devices connected to ATAPI, SCSI, USB, and IEEE 1394 buses, assuming that the device and the BIOS comply with the El Torito specification.

Installation of Windows XP from the retail CD product is limited to device classes supported during the first portion of setup and for which the system BIOS includes Int13 support. PCs that supply CD class drive support by way of PCMCIA or CardBus connections must use "recovery" CD or DVD images to reinstall the factory operating system image.

Note, however, that only the Microsoft storage stacks for ATA and SCSI are guaranteed to support hibernation and crash dump. There is no support for these capabilities for storage connected by way of USB or IEEE 1394.

NTFS for Hard Disks

NTFS is the preferred file system for systems running Windows XP because of the recoverability, scalability, functionality, and enhanced performance benefits that NTFS provides for users.

Microsoft encourages system manufacturers to provide NTFS volumes on all systems where Windows XP is preinstalled in order to gain benefits related to recoverability, scalability, functionality, and performance on large volumes.

Recoverability. NTFS is a journaling file system. As NTFS makes changes to the on-disk format, it writes a log of the changes being made, similar to the technique used by a database to capture transactions for rollback. This journaling feature is important when a system crashes, loses power, or is reset unexpectedly.

In the event of a system reset or crash, NTFS is able to quickly return the disk to a consistent state without running Chkdsk. This robustness yields a better user experience, ensuring fewer reasons for users to make support calls.

In contrast, during a system reset or crash, the FAT file system must always run a Chkdsk process before it mounts the volume. This recovers the FAT file system, but provides a poor user experience because FAT Chkdsk can take a long time to complete, and it can result in support calls.

Scalability. NTFS is designed to scale to very large disk sizes, making it possible to partition large disks with a single volume. The theoretical practical maximum size for an NTFS volume is 2 terabytes. In the next few years, hard disk sizes are expected to be 64 GB to 256 GB in size, even on consumer systems. In addition:

• NTFS can support large volume sizes with no real constraints on cluster or disk size (see Size Constraints for File Systems table). By comparison, the FAT file system has constraints that affect customer scenarios.

• NTFS supports much larger file sizes than FAT32, making it more suitable for large data files such as streaming media. For example, the 4 GB file size limit on a FAT volume is sufficient for only 18 minutes of downloaded video from a digital-video camcorder, and a FAT volume cannot contain a full-length MPEG-encoded movie.

NTFS functionality. NTFS offers advanced features that are either unavailable or difficult to implement on FAT32. Some of these include:

• Security and access control on private files

• Distributed link tracking (Windows shell shortcuts will track files even if they are moved)

• File compression and file encryption

• Snapshot Backup (enables backup of open files)

For information about the advanced features of NTFS, see "Enterprise Class Storage in Windows 2000" at http://www.microsoft.com/ windows2000/ techinfo/ howitworks/ fileandprint/ storage.asp

Notice that applications sometimes offer additional or improved functionality on NTFS volumes. For example, when burning audio CDs, Windows Media™ Player 8 uses hard links, so that it uses disk space twice as efficiently on NTFS compared to FAT file systems.

NTFS performance with large volumes. NTFS performs well when reading, writing, and mounting large volume sizes.

FAT32 performance is reduced for volumes larger than 32 GB in two areas:

• Boot time is increased due to time required to read all of the FAT structure to calculate the amount of free space.

• Read/write performance is affected because the FAT file system must determine the free space on the disk through the small views of the massive FAT structure.

High-performance file system configurations for end-users. Compared to Windows 2000, NTFS has gained numerous performance improvements in Windows XP, plus improved disk defragmentation. Microsoft will continue to optimize NTFS performance.

Cluster size - for all file systems: Windows XP, Windows 2000, Windows NT:

• A power of 2 between 512 bytes and 65536 bytes, inclusive Windows 95/98, Windows Me:

• A power of 2 between 512 bytes and 32768 bytes, inclusive

CD-MRW and Windows XP

In addition to support for CD media creation with IMAPI (described later), Microsoft expects to provide support for the Mt. Rainier CD-MRW physical format, once drives that implement this technology reach the market.

Together, the Mt. Rainier standards and the IMAPI model will make CD-MRW the best replacement for the floppy drive. Support for CD-MRW drives may be included in Windows XP and all future versions of Windows after these drives reach market, depending on market penetration.

IMAPI and Windows XP

IMAPI allows the user to write files to a CD-R/RW drive by repeatedly dragging-and-dropping files until the disc is full.

IMAPI allows an application to "stage" and "burn" a simple audio or data image onto a 74-minute CD on a set of CD-R or CD-R/W devices. IMAPI will provide an interface between the Windows shell and Windows Media Player.

After the user has selected all the files to be burned onto a disc in one session, the next act is to click the Write to CD button to name the disc and start the mastering/writing process. The user can add more files later, but there is a ~50 MB overhead associated with each additional session.

From the Windows Media Player interface, discs can be created that can be read in most consumer players. This requires that the discs must be written in a single session; otherwise, the disc cannot be read by consumer players.

Microsoft developed IMAPI in partnership with Roxio, a subsidiary of Adaptec, Inc. (http://www.roxio.com )

IMAPI and Third-Party Vendors. IMAPI support in Windows means that OEMs can offer basic CD recording support without having to include a third-party application. The built-in support makes CD burning an easy, good experience for their users.

The IMAPI interface will be published shortly after Windows XP Beta 2 in the Microsoft Platform SDK. IMAPI is an open interface that supports third parties adding their own user interface.

IMAPI SDK

http://www.microsoft.com/ msdownload/ platformsdk/ setuplauncher.asp

To take advantage of IMAPI, vendors must ensure that their firmware passes the new MMC compliance test included in the Windows XP HCT from WHQL. This test checks for:

GET_CONFIGURATION
GET_EVENT_STATUS_NOTIFICATION

Details about this test are provided in the Windows HCT 10.0.

Windows XP HCT and Test Specifications

http://www.microsoft.com/whdc/whql/default.mspx

 Top of page

Windows and DVD

Microsoft focuses its DVD technology support on compliance with industry standards and the formats that have received high market acceptance by OEMs. Windows XP and future versions will deliver industry-standard methods to support multiple DVD formats.

New DVD support in Windows XP includes support for DVD-RAM (read/write), in addition to support for DVD-Video and DVD-ROM, the read-only versions supported in earlier versions of Windows.

DVD formats
Several different formats are currently defined for DVD.

DVD-RAM support in Windows XP uses the FAT32 file system for read/write and allows use of the Universal Disk Format (UDF) file system for read-only. DVD-RAM also supports multisession recording or logical block addressing.

Microsoft delivers many built-in capabilities to support DVD playback of video, including DirectX video acceleration, DirectShow filters, and new Video Mixer Rendering support in Windows XP. These capabilities are not discussed in this article.

DVD Support in Other Windows Versions. Microsoft Windows 2000, Windows Me, and Windows 98 Second Edition support the following capabilities for DVD drives:

• DVD movie playback

• DVD as a read-only data storage device (DVD-ROM)

• DVD-ROM class driver and UDF read-only file system for data storage

• Streaming class driver

• System boot from DVD drives that comply with the El Torito specification

• Microsoft DirectShow and DirectDraw hardware acceleration support, with Video Port Extensions (VPE) for DVD playback

 Top of page

Windows Logo Program Requirements

Most standards and design guidelines for storage and controllers are well defined in the industry. The Windows Logo requirements reflect those standards as well as incremental advances made in the industry and at Microsoft. For Windows XP, new requirements include:

• Controller and devices must support bus mastering.

  UDMA/DMA Programmed I/O (PIO) support is allowed only for flash memory devices.

  To tolerate UDMA, ATAPI devices must support the termination scheme defined in ATA/ATAPI-5.

• Hard drives must support ATA/ATAPI-5, minimum. Support for ATA/ATAPI-6 will be a requirement for 48-bit LBA and ATA Removable Media Serial numbers.

• CD and DVD drives must support the ANSI T10 MMC-2 standard plus features and commands intended to support the built-in features of Windows XP.

Windows Logo Program Requirements

http://www.microsoft.com/whdc/winlogo/default.mspx

 Top of page

Guidelines for Manufacturers

For device manufacturers:

• Do not invent a proprietary file system. Functional and useful file systems exist todayconsider FAT32.

• Send Microsoft your prototypes and currently shipping hardware for compatibility testing in the Windows development labs. Work with us in early phases to identify issues and ensure optimal design.

• Follow industry standards and test with the HCTs from WHQL to ensure compatibility with Windows.

• For USB devices, always populate the device serial and model numbers, even though a specification may define such identifiers as optional.

• For writable optical devices, implement IMAPI support for the device and any utilities you create.

• Use built-in Windows driver support and follow the Windows DDK when building miniport drivers for storage.

Windows DDK http://www.microsoft.com/whdc/devtools/ddk/default.mspx

For system manufacturers:

• Choose compatible devices, based on the Windows Logo Program.

• Choose devices whose performance supports the goals for fast startup/fast resume. Follow the guidelines and test with the tools: http://www.microsoft.com/whdc/hwdev/ platform/ performance/ fastboot/default.mspx

• Format hard disks as NTFS in your factory preinstallation process. See the OPK for tools and details.

• Select IMAPI-based solutions for read/write optical devices, to ensure best experience for end users and lower support costs.

• Do not implement multiple logical unit numbers (LUNs) optical drives. See: http://www.microsoft.com/whdc/hwdev/tech/stream/dvd/multiLUN.mspx

• Work with your BIOS vendor to ensure that the system firmware adequately addresses issues for multiple types of boot storage devices, fast-resume performance, and so on.

• Send Microsoft your prototypes and shipping hardware, for compatibility testing in the Windows development labs. In particular, make sure that Microsoft is testing any new type of hardware or solution that you are testing.

Resources from Microsoft http://www.microsoft.com/whdc/device/storage/default.mspx

dvd@microsoft.com

 Top of page

Industry Standards and Activity

The ANSI NCITS T10 and T13 working groups are the primary standards bodies for PC storage devices. The T10 group publishes the MMC standards for CD and DVD drives, and specifications for the SCSI architecture. The T13 group publishes standards for ATA and ATAPI interfaces.

ANSI T10 Working Group

http://www.t10.org 

The Mt. Rainier Group, a joint effort of Microsoft, Compaq, Philips, and Sony, produced the CD-MRW protocol, which has been submitted to the T10/MMC.

Mt. Rainier Group

http://www.mt-rainier.org 

 Top of page

Industry Standards

DVD Specifications for Rewritable Disc

DVD-RAM Specification

http://global.ihs.com/ 

SFF 8090 ("Mt. Fuji 5" specification), submitted to the SCSI T10 MMC group at revision 0.9

Available in MMC-3 review draft:

ftp://ftp.t10.org/t10/drafts/mmc3/mmc3r03.pdf 

Optical Storage Technology Association (OSTA) MultiRead Specification for Universal Disk Format Specification, V.1.5, 2.0

http://www.osta.org/ 

ECMA Standards ECMA-267 (DVD-ROM), ECMA-274 (DVD+RW), and ECMA-273 (DVD-RAM)

http://www.ecma.ch/ 

ATA and ATAPI draft standards and other working documents

http://global.ihs.com 

ATAPI Removable Rewriteable Media Devices (SFF-8070i)

http://www.isdaman.com/alsos/hardware/hdc/INF-8070.PDF

ATAPI Removable Media Device BIOS Specification (ARMD) Compaq-Intel-Phoenix BIOS Boot Specification, V.1.01

El ToritoBootable CD-ROM Format Specification, V.1.0

http://download.intel.com/support/motherboards/desktop/sb/specscdrom.pdf

Mt. Rainier Command set (NCITS T10 MMC) in MMC-3

ftp://ftp.t10.org/t10/drafts/mmc3/mmc3r03.pdf 

NCITS ANSI T10 Multi-Media Command Set-2 (MMC-2)

NCITS Serial Bus Protocol-2 (SBP-2) transport protocols [ANSI NCITS 325-1998]

http://global.ihs.com/ 

 Top of page

Terms

ATA

AT Attachment. A compatible register set, and an 80 or 40-pin connector and its associated signals. Previously known as IDE.

ATAPI

ATA Packet Interface. A hardware and software specification that documents the interface between a host computer and CD-ROM drives using the ATA bus.

DMA

Direct Memory Access. A means of data transfer between device and host memory without host processor intervention.

MMC

Multimedia Command Set, as administered by the NCITS ANSI T10 sub-committee.

PIO

Programmed Input/Output

 Top of page