Note: If you only need to open the temporary file once, then as mentioned by Catalin Pop Sever, it is better to pass the option FileOptions.DeleteOnClose when opening the file.

It is a common pattern to create a temporary file using Path.GetTempFileName, and wrap it in a try...finally block to ensure that it gets cleaned up:

C#:

string temporaryFile = null;

try
{
    temporaryFile = Path.GetTempFileName();

    // Use temporary file.
}
finally
{
    if (temporaryFile != null)
    {
        try
        {
            File.Delete(temporaryFile);
        }
        catch (IOException)
        {
        }
        catch (UnauthorizedAccessException)
        {
        }    
    }
} 

Writing the above code over and over can become both tiresome and error prone. So instead, extract it into own class:

using System;
using System.IO;

namespace System.IO
{
    public class TemporaryFile : IDisposable
    {
        private bool _IsDisposed;
        private bool _Keep;
        private string _Path;

	public TemporaryFile() : this(false)
        {
        }

        public TemporaryFile(bool shortLived)
        {
            _Path = CreateTemporaryFile(shortLived);
        }

        public string Path
        {
            get { return _Path; }
        }

        public bool Keep
        {
            get { return _Keep; }
            set { _Keep = value; }
        }

        ~TemporaryFile()
        {
            Dispose(false);
        }

        public void Dispose()
        {
            Dispose(false);
            GC.SuppressFinalize(this);
        }

	protected virtual void Dispose(bool disposing)
        {
            if (!_IsDisposed)
            {
                _IsDisposed = true;

                if (!_Keep)
                {
                    TryDelete();   
                }
            }
        }

        private void TryDelete()
        {
            try
            {
                File.Delete(_Path);
            }
            catch (IOException)
            {
            }
            catch (UnauthorizedAccessException)
            {
            }
        }

        public static string CreateTemporaryFile(bool shortLived)
        {
            string temporaryFile = System.IO.Path.GetTempFileName();

            if (shortLived)
            { 
                // Set the temporary attribute, meaning the file will live in memory and will not be written to disk 
                File.SetAttributes(temporaryFile, File.GetAttributes(temporaryFile) | FileAttributes.Temporary);
            }

            return temporaryFile;
        }
    }
}

Using the new class is easy, just type the following:

using (TemporaryFile temporaryFile = new TemporaryFile())
{
    // Use temporary file
}

If you decide, after constructing a TemporaryFile, that you want to prevent it from being deleted, simply set the TemporaryFile.Keep property to true:

using (TemporaryFile temporaryFile = new TemporaryFile()) 
{ 
    temporaryFile.Keep = true; 
}

A good programming pattern is to open temporary files with FileOptions.DeleteOnClose

// Note that File.Create() also overrides the file if it exists

using (FileStream fs = File.Create(Path.GetTempFileName(), Int16.MaxValue, FileOptions.DeleteOnClose)) {

	// Use temp file

}

// The file will be deleted here