0 out of 1 rated this helpful - Rate this topic

CryptMsgOpenToDecode function

Applies to: desktop apps only

The CryptMsgOpenToDecode function opens a cryptographic message for decoding and returns a handle of the opened message. The message remains open until the CryptMsgClose function is called.

Important changes that affect the handling of enveloped messages have been made to CryptoAPI to support Secure/Multipurpose Internet Mail Extensions (S/MIME) email interoperability. For details, see the Remarks section of CryptMsgOpenToEncode.

Syntax

Copy

HCRYPTMSG WINAPI CryptMsgOpenToDecode(
  __in      DWORD dwMsgEncodingType,
  __in      DWORD dwFlags,
  __in      DWORD dwMsgType,
  __in      HCRYPTPROV_LEGACY hCryptProv,
  __in      PCERT_INFO pRecipientInfo,
  __in_opt  PCMSG_STREAM_INFO pStreamInfo
);

Parameters

dwMsgEncodingType [in]

Specifies the encoding type used. It is always acceptable to specify both the certificate and message encoding types by combining them with a bitwise-OR operation as shown in the following example:

X509_ASN_ENCODING | PKCS_7_ASN_ENCODING

Currently defined encoding types are:

X509_ASN_ENCODING
PKCS_7_ASN_ENCODING

dwFlags [in]

This parameter can be one of the following flags.

Value	Meaning
CMSG_DETACHED_FLAG	Indicates that the message to be decoded is detached. If this flag is not set, the message is not detached.
CMSG_CRYPT_RELEASE_CONTEXT_FLAG	If set, the hCryptProv passed to this function is released on the final CryptMsgUpdate. The handle is not released if the function fails.

dwMsgType [in]

Specifies the type of message to decode. In most cases, the message type is determined from the message header and zero is passed for this parameter. In some cases, notably with Internet Explorer 3.0, messages do not have headers and the type of message to be decoded must be supplied in this function call. If the header is missing and zero is passed for this parameter, the function fails.

This parameter can be one of the following predefined message types.

Value	Meaning
CMSG_DATA	The message is encoded data.
CMSG_ENVELOPED	The message is an enveloped message.
CMSG_HASHED	The message is a hashed message.
CMSG_SIGNED	The message is a signed message.
CMSG_SIGNED_AND_ENVELOPED	The message is a signed and enveloped message.

hCryptProv [in]

This parameter is not used and should be set to NULL.

Windows Server 2003, Windows XP, and Windows 2000: Specifies a handle for the cryptographic provider to use for hashing the message. For signed messages, hCryptProv is used for signature verification.
This parameter's data type is HCRYPTPROV.

Unless there is a strong reason for passing in a specific cryptographic provider in hCryptProv, set this parameter to NULL. Passing in NULL causes the default RSA or DSS provider to be acquired before performing hash, signature verification, or recipient encryption operations.

pRecipientInfo [in]

This parameter is reserved for future use and must be NULL.

pStreamInfo [in, optional]

When streaming is not being used, this parameter must be set to NULL.

Note Streaming is not used with CMSG_HASHED messages. When dealing with hashed data, this parameter must be set to NULL.

When streaming is being used, the pStreamInfo parameter is a pointer to a CMSG_STREAM_INFO structure that contains a pointer to a callback to be called when CryptMsgUpdate is executed or when CryptMsgControl is executed when decoding a streamed enveloped message.

For a signed message, the callback is passed a block of the decoded bytes from the inner content of the message.

For an enveloped message, after each call to CryptMsgUpdate, you must check to determine whether the CMSG_ENVELOPE_ALGORITHM_PARAM property is available by calling the CryptMsgGetParam function. CryptMsgGetParam will fail and GetLastError will return CRYPT_E_STREAM_MSG_NOT_READY until CryptMsgUpdate has processed enough of the message to make the CMSG_ENVELOPE_ALGORITHM_PARAM property available. When the CMSG_ENVELOPE_ALGORITHM_PARAM property is available, you can iterate through the recipients, retrieving a CERT_INFO structure for each recipient by using the CryptMsgGetParam function to retrieve the CMSG_RECIPIENT_INFO_PARAM property. To prevent a denial of service attack from an enveloped message that has an artificially large header block, you must keep track of the amount of memory that has been passed to the CryptMsgUpdate function during this process. If the amount of data exceeds an application defined limit before the CMSG_ENVELOPE_ALGORITHM_PARAM property is available, you must stop processing the message and call the CryptMsgClose function to cause the operating system to release any memory that has been allocated for the message. A suggested limit is the maximum allowable size of a message. For example, if the maximum message size is 10 MB, the limit for this test should be 10 MB.

The CERT_INFO structure is used to find a matching certificate in a previously opened certificate store by using the CertGetSubjectCertificateFromStore function. When the correct certificate is found, the CertGetCertificateContextProperty function with a CERT_KEY_PROV_INFO_PROP_ID parameter is called to retrieve a CRYPT_KEY_PROV_INFO structure. The structure contains the information necessary to acquire the recipient's private key by calling CryptAcquireContext, using the pwszContainerName, pwszProvName, dwProvType, and dwFlags members of the CRYPT_KEY_PROV_INFO structure. The hCryptProv acquired and the dwKeySpec member of the CRYPT_KEY_PROV_INFO structure are passed to the CryptMsgControl structure as a member of the CMSG_CTRL_DECRYPT_PARA structure to permit the start of the decryption of the inner content. The streaming code will then perform the decryption as the data is input. The resulting blocks of plaintext are passed to the callback function specified by the pfnStreamOutput member of the CMSG_STREAM_INFO structure to handle the output of the decrypted message.

Note Streamed decoding of an enveloped message queues the ciphertext in memory until CryptMsgControl is called to start the decryption. The application must initiate decrypting in a timely manner so that the data can be saved to disk or routed elsewhere before the accumulated ciphertext becomes too large and the system runs out of memory.

In the case of a signed message enclosed in an enveloped message, the plaintext output from the streaming decode of the enveloped message can be fed into another streaming decode to process the signed message.

Return value

If the function succeeds, the function returns the handle of the opened message.

If the function fails, it returns NULL. For extended error information, call GetLastError.

The following table lists the error codes most commonly returned by the GetLastError function.

Value	Description
E_INVALIDARG	One or more arguments are not valid.
E_OUTOFMEMORY	A memory allocation failure occurred.

Examples

For examples that use this function, see Example C Program: Signing, Encoding, Decoding, and Verifying a Message, Alternate Code for Encoding an Enveloped Message, Example C Program: Encoding an Enveloped, Signed Message, and Example C Program: Encoding and Decoding a Hashed Message.

Requirements

Minimum supported client	Windows 2000 Professional
Minimum supported server	Windows 2000 Server
Header	Wincrypt.h
Library	Crypt32.lib
DLL	Crypt32.dll

The sequence of API calls described here does not work! You have to obtain an handle to a crypto provider for the private key and not to a public one as described here. That to say you must call CryptAcquireCertificatePrivateKey() when you have the information relative to one of the recipient certs.

Here is the complete code, hope this can help someone out there!

#include <stdio.h>

#include <Windows.h>
#include <WinCrypt.h>

#define BUFSIZE                             1024
#define MY_ENCODING_TYPE         PKCS_7_ASN_ENCODING | X509_ASN_ENCODING

static BOOL WINAPI CmsgStreamOutputCallback(IN const void *pvArg, IN BYTE *pbData, IN DWORD cbData, IN BOOL fFinal);

int main(int argc, char *argv[]) {
    printf("Reading %s\n", argv[1]);

    HANDLE hFile= CreateFile(argv[1], GENERIC_READ, 0, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);

    if ( hFile == INVALID_HANDLE_VALUE ) {
        printf("Cannot open file 0x0%X\n", GetLastError());
        exit(1);
    }

    HCERTSTORE store= CertOpenSystemStore(NULL, "MY");
    if ( !store ) {
        printf("Cannot open store 0x0%X\n", GetLastError());
        exit(1);
    }

    CMSG_STREAM_INFO StreamInfo;
    StreamInfo.cbContent= CMSG_INDEFINITE_LENGTH;
    StreamInfo.pfnStreamOutput = CmsgStreamOutputCallback;
    StreamInfo.pvArg = NULL;

    HCRYPTMSG hMsg = CryptMsgOpenToDecode(MY_ENCODING_TYPE, 0, 0,    NULL, NULL, &StreamInfo);
    if ( !hMsg ) {
        printf("Cannot open message 0x0%X\n", GetLastError());
        exit(1);
    }

    BOOL isFinal = FALSE;
    CMSG_CTRL_DECRYPT_PARA decryptPara = {sizeof(CMSG_CTRL_DECRYPT_PARA)};

    do {
        DWORD readed = 0;
        BYTE buf[BUFSIZE];

        if ( !ReadFile(hFile, buf, BUFSIZE, &readed, NULL) ) {
            printf("Cannot read file 0x0%X\n", GetLastError());
            exit(1);
        };

        printf("Read %u\n", readed);
        if( readed == 0) isFinal = TRUE;

        if ( !CryptMsgUpdate(hMsg, buf, readed, isFinal) ) {
                printf("Cannot update message 0x0%X\n", GetLastError());
                exit(1);
        }

        if( isFinal ) {
            DWORD size = 0;
            CryptMsgGetParam(hMsg, CMSG_ENVELOPE_ALGORITHM_PARAM,    0, NULL, &size);
            PCRYPT_ALGORITHM_IDENTIFIER pai = (PCRYPT_ALGORITHM_IDENTIFIER)new BYTE[size];
            if ( !CryptMsgGetParam(hMsg, CMSG_ENVELOPE_ALGORITHM_PARAM, 0, pai, &size) ) {
                printf("Cannot get algorithm 0x0%X\n", GetLastError());
                exit(1);
            };

            size = 0;
            CryptMsgGetParam(hMsg, CMSG_RECIPIENT_INFO_PARAM,    0, NULL, &size);
            PCERT_INFO pci = (PCERT_INFO)new BYTE[size];
            if ( !CryptMsgGetParam(hMsg, CMSG_RECIPIENT_INFO_PARAM,    0, pci, &size) ) {
                printf("Cannot get recipient 0x0%X\n", GetLastError());
                exit(1);
            }

            PCCERT_CONTEXT pContext = CertGetSubjectCertificateFromStore(store,    MY_ENCODING_TYPE, pci);
            if ( !pContext ) {
                printf("Recipient not found 0x0%X\n", GetLastError());
                exit(1);
            }

            BOOL flagHandle= FALSE;
            if ( !CryptAcquireCertificatePrivateKey(pContext, 0, NULL, &decryptPara.hCryptProv, &decryptPara.dwKeySpec, &flagHandle) ) {
                printf("Private key not acquired 0x0%X\n", GetLastError());
                exit(1);
            } else {
                printf("handle %i\n", flagHandle);
            };
            decryptPara.dwRecipientIndex = 0;

//            size = 0;
//            CertGetCertificateContextProperty(pContext,    CERT_KEY_PROV_INFO_PROP_ID, NULL, &size);
//            PCRYPT_KEY_PROV_INFO pki = (PCRYPT_KEY_PROV_INFO)new BYTE[size];
//            if ( !CertGetCertificateContextProperty(pContext,    CERT_KEY_PROV_INFO_PROP_ID, pki, &size) ) {
///                printf("Cannot get key info 0x0%X\n", GetLastError());
//                exit(1);
//            };

//            printf("Container: %S\n", pki->pwszContainerName);
//            printf("Provider : %S\n", pki->pwszProvName);
//            if ( !CryptAcquireContextW(&decryptPara.hCryptProv, pki->pwszContainerName,    pki->pwszProvName, pki->dwProvType, 0) ) {
//                printf("Cannot acquire provider 0x0%X\n", GetLastError());
//                exit(1);
//            };

            if(pContext) CertFreeCertificateContext(pContext);
//            delete [] pki;
            delete [] pai;
            delete [] pci;
        }
    } while(!isFinal);

    if ( !CryptMsgControl(hMsg, 0, CMSG_CTRL_DECRYPT,    &decryptPara) ) {
        printf("Cannot decypher 0x0%X\n", GetLastError());
        exit(1);
    };

    CryptMsgClose(hMsg);
}

static BOOL WINAPI CmsgStreamOutputCallback(IN const void *pvArg, IN BYTE *pbData, IN DWORD cbData, IN BOOL fFinal) {
    printf("Write data, %i\n", cbData);
    return TRUE;
}

History

3/4/2011
N.Gaggi

Decoding an enveloped message using streaming

The algorithm to decode such a message appears clear, but probably an example will help more. I have written a c++ function for this algorithm but the call to CryptMsgControl() that starts message decyphering fails with a NTE_BAD_TYPE error, probably i am missing something really important.

History

3/2/2011
N.Gaggi

C# syntax

[DllImport("crypt32.dll", CharSet=CharSet.Auto, SetLastError=true)]
internal static extern IntPtr CryptMsgOpenToDecode([In] uint dwMsgEncodingType, [In] uint dwFlags, [In] uint dwMsgType, [In] IntPtr hCryptProv, [In] IntPtr pRecipientInfo, [In] IntPtr pStreamInfo);

History

5/6/2009
dmex

vb.net syntax

<DllImport("crypt32.dll", CharSet:=CharSet.Auto, SetLastError:=True)> _
Public Shared Function CryptMsgOpenToDecode(<[In]> ByVal dwMsgEncodingType As UInt32, <[In]> ByVal dwFlags As UInt32, <[In]> ByVal dwMsgType As UInt32, <[In]> ByVal hCryptProv As IntPtr, <[In]> ByVal pRecipientInfo As IntPtr, <[In]> ByVal pStreamInfo As IntPtr) As IntPtr
End Function

History

5/6/2009
dmex

CryptMsgOpenToDecode function

Syntax

Parameters

Return value

Examples

Requirements

See also