UnicodeEncoding Class

Encoding is the process of transforming a set of Unicode characters into a sequence of bytes. Decoding is the process of transforming a sequence of encoded bytes into a set of Unicode characters.

The Unicode Standard assigns a code point (a number) to each character in every supported script. A Unicode Transformation Format (UTF) is a way to encode that code point. The Unicode Standard uses the following UTFs:

• UTF-8, which represents each code point as a sequence of one to four bytes.

• UTF-16, which represents each code point as a sequence of one to two 16-bit integers.

• UTF-32, which represents each code point as a 32-bit integer.

Note
The UTF-7 encoding supports certain protocols for which it is required, most often e-mail or newsgroup protocols. Since UTF-7 is not particularly secure or robust, it should generally not be used. UTF-8 should normally be preferred to UTF-7.

For more information about the UTFs and other encodings supported by System.Text, see Character Encoding in the .NET Framework and Using Unicode Encoding.

The GetByteCount method determines how many bytes result in encoding a set of Unicode characters, and the GetBytes method performs the actual encoding.

Likewise, the GetCharCount method determines how many characters result in decoding a sequence of bytes, and the GetChars and GetString methods perform the actual decoding.

UnicodeEncoding corresponds to the Windows code pages 1200 (little endian byte order) and 1201 (big endian byte order).

The encoder can use the big endian byte order (most significant byte first) or the little endian byte order (least significant byte first). For example, the Latin Capital Letter A (code point U+0041) is serialized as follows (in hexadecimal):

• Big endian byte order: 00 00 00 41

• Little endian byte order: 41 00 00 00

It is generally more efficient to store Unicode characters using the native byte order. For example, it is better to use the little endian byte order on little endian platforms, such as Intel computers.

Optionally, the UnicodeEncoding object provides a preamble, which is an array of bytes that can be prefixed to the sequence of bytes resulting from the encoding process. If the preamble contains a byte order mark (BOM), it helps the decoder determine the byte order and the transformation format or UTF. The GetPreamble method retrieves an array of bytes that can include the BOM.

Note

To enable error detection and to make the class instance more secure, the application should use the UnicodeEncoding constructor that takes a throwOnInvalidBytes parameter, and set that parameter to true. With error detection, a method that detects an invalid sequence of characters or bytes throws a ArgumentException. Without error detection, no exception is thrown, and the invalid sequence is generally ignored.

The following example demonstrates how to encode a string of Unicode characters into a byte array, using UnicodeEncoding. The byte array is decoded into a string to demonstrate that there is no loss of data.

using System;
using System.Text;

class UnicodeEncodingExample {
    public static void Main() {
        // The encoding.
        UnicodeEncoding unicode = new UnicodeEncoding();

        // Create a string that contains Unicode characters.
        String unicodeString =
            "This Unicode string contains two characters " +
            "with codes outside the traditional ASCII code range, " +
            "Pi (\u03a0) and Sigma (\u03a3).";
        Console.WriteLine("Original string:");
        Console.WriteLine(unicodeString);

        // Encode the string.
        Byte[] encodedBytes = unicode.GetBytes(unicodeString);
        Console.WriteLine();
        Console.WriteLine("Encoded bytes:");
        foreach (Byte b in encodedBytes) {
            Console.Write("[{0}]", b);
        }
        Console.WriteLine();

        // Decode bytes back to string.
        // Notice Pi and Sigma characters are still present.
        String decodedString = unicode.GetString(encodedBytes);
        Console.WriteLine();
        Console.WriteLine("Decoded bytes:");
        Console.WriteLine(decodedString);
    }
}

.NET Framework

.NET Framework Client Profile

Portable Class Library

Windows 7, Windows Vista SP1 or later, Windows XP SP3, Windows XP SP2 x64 Edition, Windows Server 2008 (Server Core not supported), Windows Server 2008 R2 (Server Core supported with SP1 or later), Windows Server 2003 SP2

Reference

Other Resources