Memory-mapped files

Article
12/14/2022

A memory-mapped file contains the contents of a file in virtual memory. This mapping between a file and memory space enables an application, including multiple processes, to modify the file by reading and writing directly to the memory. You can use managed code to access memory-mapped files in the same way that native Windows functions access memory-mapped files, as described in Managing Memory-Mapped Files.

There are two types of memory-mapped files:

Persisted memory-mapped files

Persisted files are memory-mapped files that are associated with a source file on a disk. When the last process has finished working with the file, the data is saved to the source file on the disk. These memory-mapped files are suitable for working with extremely large source files.
Non-persisted memory-mapped files

Non-persisted files are memory-mapped files that are not associated with a file on a disk. When the last process has finished working with the file, the data is lost and the file is reclaimed by garbage collection. These files are suitable for creating shared memory for inter-process communications (IPC).

Processes, Views, and Managing Memory

Memory-mapped files can be shared across multiple processes. Processes can map to the same memory-mapped file by using a common name that is assigned by the process that created the file.

To work with a memory-mapped file, you must create a view of the entire memory-mapped file or a part of it. You can also create multiple views to the same part of the memory-mapped file, thereby creating concurrent memory. For two views to remain concurrent, they have to be created from the same memory-mapped file.

Multiple views may also be necessary if the file is greater than the size of the application's logical memory space available for memory mapping (2 GB on a 32-bit computer).

There are two types of views: stream access view and random access view. Use stream access views for sequential access to a file; this is recommended for non-persisted files and IPC. Random access views are preferred for working with persisted files.

Memory-mapped files are accessed through the operating system's memory manager, so the file is automatically partitioned into a number of pages and accessed as needed. You do not have to handle the memory management yourself.

The following illustration shows how multiple processes can have multiple and overlapping views to the same memory-mapped file at the same time.

The following image shows multiple and overlapped views to a memory-mapped file:

Screenshot that shows views to a memory-mapped file.

Programming with Memory-Mapped Files

The following table provides a guide for using memory-mapped file objects and their members.

Task	Methods or properties to use
To obtain a MemoryMappedFile object that represents a persisted memory-mapped file from a file on disk.	MemoryMappedFile.CreateFromFile method.
To obtain a MemoryMappedFile object that represents a non-persisted memory-mapped file (not associated with a file on disk).	MemoryMappedFile.CreateNew method. - or - MemoryMappedFile.CreateOrOpen method.
To obtain a MemoryMappedFile object of an existing memory-mapped file (either persisted or non-persisted).	MemoryMappedFile.OpenExisting method.
To obtain a UnmanagedMemoryStream object for a sequentially accessed view to the memory-mapped file.	MemoryMappedFile.CreateViewStream method.
To obtain a UnmanagedMemoryAccessor object for a random access view to a memory-mapped file.	MemoryMappedFile.CreateViewAccessor method.
To obtain a SafeMemoryMappedViewHandle object to use with unmanaged code.	MemoryMappedFile.SafeMemoryMappedFileHandle property. - or - MemoryMappedViewAccessor.SafeMemoryMappedViewHandle property. - or - MemoryMappedViewStream.SafeMemoryMappedViewHandle property.
To delay allocating memory until a view is created (non-persisted files only). (To determine the current system page size, use the Environment.SystemPageSize property.)	CreateNew method with the MemoryMappedFileOptions.DelayAllocatePages value. - or - CreateOrOpen methods that have a MemoryMappedFileOptions enumeration as a parameter.

Security

You can apply access rights when you create a memory-mapped file, by using the following methods that take a MemoryMappedFileAccess enumeration as a parameter:

You can specify access rights for opening an existing memory-mapped file by using the OpenExisting methods that take an MemoryMappedFileRights as a parameter.

In addition, you can include a MemoryMappedFileSecurity object that contains predefined access rules.

To apply new or changed access rules to a memory-mapped file, use the SetAccessControl method. To retrieve access or audit rules from an existing file, use the GetAccessControl method.

Examples

Persisted Memory-Mapped Files

The CreateFromFile methods create a memory-mapped file from an existing file on disk.

The following example creates a memory-mapped view of a part of an extremely large file and manipulates a portion of it.

using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Runtime.InteropServices;

class Program
{
    static void Main(string[] args)
    {
        long offset = 0x10000000; // 256 megabytes
        long length = 0x20000000; // 512 megabytes

        // Create the memory-mapped file.
        using (var mmf = MemoryMappedFile.CreateFromFile(@"c:\ExtremelyLargeImage.data", FileMode.Open,"ImgA"))
        {
            // Create a random access view, from the 256th megabyte (the offset)
            // to the 768th megabyte (the offset plus length).
            using (var accessor = mmf.CreateViewAccessor(offset, length))
            {
                int colorSize = Marshal.SizeOf(typeof(MyColor));
                MyColor color;

                // Make changes to the view.
                for (long i = 0; i < length; i += colorSize)
                {
                    accessor.Read(i, out color);
                    color.Brighten(10);
                    accessor.Write(i, ref color);
                }
            }
        }
    }
}

public struct MyColor
{
    public short Red;
    public short Green;
    public short Blue;
    public short Alpha;

    // Make the view brighter.
    public void Brighten(short value)
    {
        Red = (short)Math.Min(short.MaxValue, (int)Red + value);
        Green = (short)Math.Min(short.MaxValue, (int)Green + value);
        Blue = (short)Math.Min(short.MaxValue, (int)Blue + value);
        Alpha = (short)Math.Min(short.MaxValue, (int)Alpha + value);
    }
}

Imports System.IO
Imports System.IO.MemoryMappedFiles
Imports System.Runtime.InteropServices

Class Program

    Sub Main()
        Dim offset As Long = &H10000000 ' 256 megabytes
        Dim length As Long = &H20000000 ' 512 megabytes

        ' Create the memory-mapped file.
        Using mmf = MemoryMappedFile.CreateFromFile("c:\ExtremelyLargeImage.data", FileMode.Open, "ImgA")
            ' Create a random access view, from the 256th megabyte (the offset)
            ' to the 768th megabyte (the offset plus length).
            Using accessor = mmf.CreateViewAccessor(offset, length)
                Dim colorSize As Integer = Marshal.SizeOf(GetType(MyColor))
                Dim color As MyColor
                Dim i As Long = 0

                ' Make changes to the view.
                Do While (i < length)
                    accessor.Read(i, color)
                    color.Brighten(10)
                    accessor.Write(i, color)
                    i += colorSize
                Loop
            End Using
        End Using
    End Sub
End Class

Public Structure MyColor
    Public Red As Short
    Public Green As Short
    Public Blue As Short
    Public Alpha As Short

    ' Make the view brighter.
    Public Sub Brighten(ByVal value As Short)
        Red = CType(Math.Min(Short.MaxValue, (CType(Red, Integer) + value)), Short)
        Green = CType(Math.Min(Short.MaxValue, (CType(Green, Integer) + value)), Short)
        Blue = CType(Math.Min(Short.MaxValue, (CType(Blue, Integer) + value)), Short)
        Alpha = CType(Math.Min(Short.MaxValue, (CType(Alpha, Integer) + value)), Short)
    End Sub
End Structure

The following example opens the same memory-mapped file for another process.

using System;
using System.IO.MemoryMappedFiles;
using System.Runtime.InteropServices;

class Program
{
    static void Main(string[] args)
    {
        // Assumes another process has created the memory-mapped file.
        using (var mmf = MemoryMappedFile.OpenExisting("ImgA"))
        {
            using (var accessor = mmf.CreateViewAccessor(4000000, 2000000))
            {
                int colorSize = Marshal.SizeOf(typeof(MyColor));
                MyColor color;

                // Make changes to the view.
                for (long i = 0; i < 1500000; i += colorSize)
                {
                    accessor.Read(i, out color);
                    color.Brighten(20);
                    accessor.Write(i, ref color);
                }
            }
        }
    }
}

public struct MyColor
{
    public short Red;
    public short Green;
    public short Blue;
    public short Alpha;

    // Make the view brigher.
    public void Brighten(short value)
    {
        Red = (short)Math.Min(short.MaxValue, (int)Red + value);
        Green = (short)Math.Min(short.MaxValue, (int)Green + value);
        Blue = (short)Math.Min(short.MaxValue, (int)Blue + value);
        Alpha = (short)Math.Min(short.MaxValue, (int)Alpha + value);
    }
}

Imports System.IO.MemoryMappedFiles
Imports System.Runtime.InteropServices

Class Program
    Public Shared Sub Main(ByVal args As String())
        ' Assumes another process has created the memory-mapped file.
        Using mmf = MemoryMappedFile.OpenExisting("ImgA")
            Using accessor = mmf.CreateViewAccessor(4000000, 2000000)
                Dim colorSize As Integer = Marshal.SizeOf(GetType(MyColor))
                Dim color As MyColor

                ' Make changes to the view.
                Dim i As Long = 0
                While i < 1500000
                    accessor.Read(i, color)
                    color.Brighten(30)
                    accessor.Write(i, color)
                    i += colorSize
                End While
            End Using
        End Using
    End Sub
End Class

Public Structure MyColor
    Public Red As Short
    Public Green As Short
    Public Blue As Short
    Public Alpha As Short

    ' Make the view brigher.
    Public Sub Brighten(ByVal value As Short)
        Red = CShort(Math.Min(Short.MaxValue, CInt(Red) + value))
        Green = CShort(Math.Min(Short.MaxValue, CInt(Green) + value))
        Blue = CShort(Math.Min(Short.MaxValue, CInt(Blue) + value))
        Alpha = CShort(Math.Min(Short.MaxValue, CInt(Alpha) + value))
    End Sub
End Structure

Non-Persisted Memory-Mapped Files

The CreateNew and CreateOrOpen methods create a memory-mapped file that is not mapped to an existing file on disk.

The following example consists of three separate processes (console applications) that write Boolean values to a memory-mapped file. The following sequence of actions occur:

Process A creates the memory-mapped file and writes a value to it.
Process B opens the memory-mapped file and writes a value to it.
Process C opens the memory-mapped file and writes a value to it.
Process A reads and displays the values from the memory-mapped file.
After Process A is finished with the memory-mapped file, the file is immediately reclaimed by garbage collection.

To run this example, do the following:

Compile the applications and open three Command Prompt windows.
In the first Command Prompt window, run Process A.
In the second Command Prompt window, run Process B.
Return to Process A and press ENTER.
In the third Command Prompt window, run Process C.
Return to Process A and press ENTER.

The output of Process A is as follows:

Start Process B and press ENTER to continue.  
Start Process C and press ENTER to continue.  
Process A says: True  
Process B says: False  
Process C says: True

Process A

using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Threading;

class Program
{
    // Process A:
    static void Main(string[] args)
    {
        using (MemoryMappedFile mmf = MemoryMappedFile.CreateNew("testmap", 10000))
        {
            bool mutexCreated;
            Mutex mutex = new Mutex(true, "testmapmutex", out mutexCreated);
            using (MemoryMappedViewStream stream = mmf.CreateViewStream())
            {
                BinaryWriter writer = new BinaryWriter(stream);
                writer.Write(1);
            }
            mutex.ReleaseMutex();

            Console.WriteLine("Start Process B and press ENTER to continue.");
            Console.ReadLine();

            Console.WriteLine("Start Process C and press ENTER to continue.");
            Console.ReadLine();

            mutex.WaitOne();
            using (MemoryMappedViewStream stream = mmf.CreateViewStream())
            {
                BinaryReader reader = new BinaryReader(stream);
                Console.WriteLine("Process A says: {0}", reader.ReadBoolean());
                Console.WriteLine("Process B says: {0}", reader.ReadBoolean());
                Console.WriteLine("Process C says: {0}", reader.ReadBoolean());
            }
            mutex.ReleaseMutex();
        }
    }
}

Imports System.IO
Imports System.IO.MemoryMappedFiles
Imports System.Threading

Module Module1

    ' Process A:
    Sub Main()
        Using mmf As MemoryMappedFile = MemoryMappedFile.CreateNew("testmap", 10000)
            Dim mutexCreated As Boolean
            Dim mTex As Mutex = New Mutex(True, "testmapmutex", mutexCreated)
            Using Stream As MemoryMappedViewStream = mmf.CreateViewStream()
                Dim writer As BinaryWriter = New BinaryWriter(Stream)
                writer.Write(1)
            End Using
            mTex.ReleaseMutex()
            Console.WriteLine("Start Process B and press ENTER to continue.")
            Console.ReadLine()

            Console.WriteLine("Start Process C and press ENTER to continue.")
            Console.ReadLine()

            mTex.WaitOne()
            Using Stream As MemoryMappedViewStream = mmf.CreateViewStream()
                Dim reader As BinaryReader = New BinaryReader(Stream)
                Console.WriteLine("Process A says: {0}", reader.ReadBoolean())
                Console.WriteLine("Process B says: {0}", reader.ReadBoolean())
                Console.WriteLine("Process C says: {0}", reader.ReadBoolean())
            End Using
            mTex.ReleaseMutex()

        End Using

    End Sub

End Module

Process B

using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Threading;

class Program
{
    // Process B:
    static void Main(string[] args)
    {
        try
        {
            using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting("testmap"))
            {

                Mutex mutex = Mutex.OpenExisting("testmapmutex");
                mutex.WaitOne();

                using (MemoryMappedViewStream stream = mmf.CreateViewStream(1, 0))
                {
                    BinaryWriter writer = new BinaryWriter(stream);
                    writer.Write(0);
                }
                mutex.ReleaseMutex();
            }
        }
        catch (FileNotFoundException)
        {
            Console.WriteLine("Memory-mapped file does not exist. Run Process A first.");
        }
    }
}

Imports System.IO
Imports System.IO.MemoryMappedFiles
Imports System.Threading

Module Module1
    ' Process B:
    Sub Main()
        Try
            Using mmf As MemoryMappedFile = MemoryMappedFile.OpenExisting("testmap")
                Dim mTex As Mutex = Mutex.OpenExisting("testmapmutex")
                mTex.WaitOne()
                Using Stream As MemoryMappedViewStream = mmf.CreateViewStream(1, 0)
                    Dim writer As BinaryWriter = New BinaryWriter(Stream)
                    writer.Write(0)
                End Using
                mTex.ReleaseMutex()
            End Using
        Catch noFile As FileNotFoundException
            Console.WriteLine("Memory-mapped file does not exist. Run Process A first." & vbCrLf & noFile.Message)
        End Try

    End Sub

End Module

Process C

using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Threading;

class Program
{
    // Process C:
    static void Main(string[] args)
    {
        try
        {
            using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting("testmap"))
            {

                Mutex mutex = Mutex.OpenExisting("testmapmutex");
                mutex.WaitOne();

                using (MemoryMappedViewStream stream = mmf.CreateViewStream(2, 0))
                {
                    BinaryWriter writer = new BinaryWriter(stream);
                    writer.Write(1);
                }
                mutex.ReleaseMutex();
            }
        }
        catch (FileNotFoundException)
        {
            Console.WriteLine("Memory-mapped file does not exist. Run Process A first, then B.");
        }
    }
}

Imports System.IO
Imports System.IO.MemoryMappedFiles
Imports System.Threading

Module Module1
    ' Process C:
    Sub Main()
        Try
            Using mmf As MemoryMappedFile = MemoryMappedFile.OpenExisting("testmap")
                Dim mTex As Mutex = Mutex.OpenExisting("testmapmutex")
                mTex.WaitOne()
                Using Stream As MemoryMappedViewStream = mmf.CreateViewStream(2, 0)
                    Dim writer As BinaryWriter = New BinaryWriter(Stream)
                    writer.Write(1)
                End Using
                mTex.ReleaseMutex()
            End Using
        Catch noFile As FileNotFoundException
            Console.WriteLine("Memory-mapped file does not exist. Run Process A first, then B." & vbCrLf & noFile.Message)
        End Try

    End Sub

End Module