This topic covers the various considerations for application control of file buffering, also known as unbuffered file input/output (I/O). File buffering is usually handled by the system behind the scenes and is considered part of file caching within the Windows operating system unless otherwise specified. Although the terms caching and buffering are sometimes used interchangeably, this topic uses the term buffering specifically in the context of explaining how to interact with data that is not being cached (buffered) by the system, where it is otherwise largely out of the direct control of user-mode applications.
When opening or creating a file with the CreateFile function, the FILE_FLAG_NO_BUFFERING flag can be specified to disable system caching of data being read from or written to the file. Although this gives complete and direct control over data I/O buffering, in the case of files and similar devices there are data alignment requirements that must be considered.
Note This alignment information applies to I/O on devices such as files that support seeking and the concept of file position pointers (or offsets). For nonseeking devices, such as named pipes or communications devices, turning off buffering may not require any particular alignment. Any limitations or efficiencies that may be gained by alignment in that case are dependent on the underlying technology.
In a simple example, the application would open a file for write access with the FILE_FLAG_NO_BUFFERING flag and then perform a call to the
WriteFile function using a data buffer defined within the application. This local buffer is, in these circumstances, effectively the only file buffer that exists for this operation. Because of physical disk layout, file system storage layout, and system-level file pointer position tracking, this write operation will fail unless the locally-defined data buffers meet certain alignment criteria, discussed in the following section.
Note Discussion of caching does not consider any hardware caching on the physical disk itself, which is not guaranteed to be within the direct control of the system in any case. This has no effect on the requirements specified in this topic.
For more information on how FILE_FLAG_NO_BUFFERING interacts with other cache-related flags, see CreateFile.
Alignment and File Access Requirements
As previously discussed, an application must meet certain requirements when working with files opened with FILE_FLAG_NO_BUFFERING. The following specifics apply:
- File access sizes, including the optional file offset in the OVERLAPPED structure, if specified, must be for a number of bytes that is an integer multiple of the volume sector size. For
example, if the sector size is 512 bytes, an application can request reads and writes of 512, 1,024, 1,536,
or 2,048 bytes, but not of 335, 981, or 7,171 bytes.
- File access buffer addresses for read and write operations should be sector-aligned, which means aligned on
addresses in memory that are integer multiples of the volume sector size. Depending on the disk, this
requirement may not be enforced.
An application can determine a volume sector size by calling the
GetDiskFreeSpace function.
Because buffer addresses for read and write operations must be sector-aligned, the application must have direct control of how these buffers are allocated. One way to sector-align buffers is to use the
VirtualAlloc function to allocate the buffers. Consider the following:
- VirtualAlloc allocates memory that is aligned on addresses that are integer multiples of the system's page size. Page size is 4,096 bytes on x64 and x86 or 8,192 bytes for the Intel Itanium processor family. For additional information, see the GetSystemInfo function.
- Sector size is typically 512 to 4,096 bytes for direct-access storage devices (hard drives) and 2,048 bytes for CD-ROMs.
- Both page and sector sizes are powers of 2.
Therefore, in most situations, page-aligned memory will also be sector-aligned, because the case where the sector size is larger than the page size is rare.
Another way to obtain manually-aligned memory buffers is to use the _aligned_malloc function from the C Run-Time library.
For an example of how to manually control buffer alignment, see the C++ language code example in the Example Code section of WriteFile.
Send comments about this topic to Microsoft
Build date: 10/22/2009