Dynamic Format Changes in DirectShow

Microsoft Windows Digital Media Division

Updated October 2003

Summary: This article discusses how Microsoft DirectShow filters can change formats after they connect. Scenarios include reconnecting pins while the graph is stopped in order to renegotiate the format, and switching formats while the graph is streaming data. This article is intended for developers planning to write their own DirectShow filter. (10 printed pages)

Contents

• Introduction
• Changing Formats When the Graph Is Stopped
• Changing Formats When the Graph Is Active
• Conclusion

Introduction

When two Microsoft® DirectShow® filters connect, they agree on a media type that describes the format of the data that the upstream filter will deliver. In most cases, the media type is fixed for the duration of the connection.

However, DirectShow does offer limited support for filters to change the media type. When a filter switches media types, it is called a dynamic format change. If you are writing a DirectShow filter, you should be aware of the mechanisms for dynamic format changes. Even if your filter does not support such changes, your filter should respond correctly if another filter requests a new format.

DirectShow defines several distinct mechanisms for dynamic format changes, depending on the state of filter graph and the type of change that is required. This article discusses the mechanisms for each scenario in these sections:

• Changing Formats When the Graph Is Stopped — Discusses how to change the format when the graph is stopped in order to renegotiate the format.
• Changing Formats When the Graph Is Active — Discusses how to change the format when the graph is running or paused.

This article assumes that you are familiar with the DirectShow architecture and with DirectShow filter development. For more information, see DirectShow Architecture for Filter Developers.

Changing Formats When the Graph Is Stopped

When the graph is stopped, a filter can change media types by renegotiating the connection with its peer, as follows:

1. The filter calls IPin::QueryAccept on the other filter's pin, and specifies the new media type (Figure 1A).

2. If QueryAccept returns S_OK, the filter calls IFilterGraph2::ReconnectEx to reconnect the pins (Figure 1B).

   

   Figure 1. Changing formats when the graph is stopped

Example Scenarios

• Tee filters. A tee filter is a filter that splits an input stream into multiple outputs without changing the data in the stream. A tee filter can accept a range of media types, but the types must match across all pin connections. Therefore, when the input pin connects, the filter might need to renegotiate any existing connections on the output pins, and vice versa. For an example, see the InfTee filter sample in the DirectShow Software Development Kit (SDK).

• Trans–in–place filters. A trans–in–place filter is a transform filter that modifies the input data in the original buffer instead of copying the data to a separate output buffer. A trans–in–place filter must use the same allocator for both the upstream and the downstream connections. The first pin to connect (input or output) negotiates an allocator in the usual way. When the other pin connects, however, the first allocator might not be acceptable. In that case, the second pin chooses a different allocator, and the first pin reconnects using the new allocator (Figure 2). For an example implementation, see the CTransInPlaceFilter class.

  

  Figure 2. Renegotiating and Allocator

Implementation Notes

Streaming must not start while a filter is reconnecting a pin. In the previous two examples, the reconnection occurs while another pin is connecting, so the filter graph remains stopped throughout the connection process. If you call the ReconnectEx method during pin connection on the same thread, it will be safe. In other situations, the filter might need to set an internal flag that disallows streaming. Also, be sure not to call ReconnectEx unless QueryAccept returns S_OK.

Changing Formats When the Graph Is Active

When the graph is active (running or paused), it is harder to change formats. There are three basic mechanisms, each with limited applications:

• QueryAccept (Downstream) is used when an output pin proposes a format change to its downstream peer, but only if the new format does not require a larger buffer.
• QueryAccept (Upstream) is used when an input pin proposes a format change to its upstream peer. The new format can be the same size, or it can be larger.
• ReceiveConnection is used when an output pin proposes a format change to its downstream peer, and the new format requires a larger buffer.

QueryAccept (Downstream)

In this scenario, an output pin proposes a new format to its downstream peer. The new format must not require a larger buffer size. The output pin does the following:

1. Calls IPin::QueryAccept or IPinConnection::DynamicQueryAccept on the downstream pin, to verify whether the other pin can accept the new media type (Figure 3A).

2. If the return value from step 1 is S_OK, the pin attaches the media type to the next sample. To do this, first it calls IMemAllocator::GetBuffer to obtain the sample (Figure 3B). Then it calls IMediaSample::SetMediaType to attach the media type to that sample (Figure 3C). By attaching the media type to the sample, the filter indicates that the format has changed, starting with that sample.

3. The pin delivers the sample (Figure 3D).

4. When the downstream filter receives the sample, it calls IMediaSample::GetMediaType to retrieve the type.

   

   Figure 3. QueryAccept (Downstream)

All pins support the IPin::QueryAccept method. However, the semantics of this method are somewhat ambiguous, because an S_OK return value does not always guarantee that you can change the format while the graph is active. Some filters might return S_OK but reject the change if the graph is active. Therefore, Microsoft DirectX® 8.0 introduced the IPinConnection::DynamicQueryAccept method for input pins. This method explicitly defines the return value S_OK to mean the pin can change formats while active. If an input pin supports the IPinConnection interface, you should call DynamicQueryAccept rather than QueryAccept.

Example Scenarios

In most cases, this mechanism does not allow for drastic changes to the format, such as changing the bit depth of video. One example where it can be used is a video decoder switching palettes. The basic format details stay the same, such as the image dimensions and the bit depth, but the new media type has a new set of palette entries.

Implementation Note

In the DirectShow base classes, CBasePin::QueryAccept calls the pure virtual function CBasePin::CheckMediaType, which is also the method that gets called when the pin first connects. This means the default behavior for dynamic format changes is identical to the behavior for the initial connection. That may or may not be correct, depending on your filter.

For transform filters, the input pin's CheckMediaType method should alway check whether the output pin is connected. If it is, the filter must verify that the proposed input format is compatible with the current output format. As long as your implementation of CheckMediaType behaves like this, the default implementation of CBasePin::QueryAccept is probably adequate. (If not, override QueryAccept to perform any additional checks that are needed.)

If you are using CTransformFilter, this class has the following behavior:

• CheckMediaType calls CTransformFilter::CheckInputType
• If the filter's output pin is connected, CheckMediaType calls CTransformFilter::CheckTransform.

Use CheckTransform to verify whether the input format can be transformed into the current output format.

If you are using CTransInPlaceFilter class, this class has the following behavior:

• CheckMediaType calls CTransformFilter::CheckInputType.

• If the filter's output pin is connected, CheckMediaType calls QueryAccept on the downstream filter.

  

  Figure 4. Base-Class Implementations of QueryAccept

The CBaseInputPin::Receive method checks for a media type on the incoming sample. If there is one, the method calls CheckMediaType. However, it does not update the pin's m_mt member, which holds the current media type. When your filter processes the sample, you should check the sample for a media type. If there is a new type, you will probably need to store it, either by calling SetMediaType on your pin or by setting the value of m_mt directly. On the other hand, the CVideoTransformFilter class, which is designed for video transform filters, stores the media type when it changes. For details, see the source code for CVideoTransformFilter::Receive in the DirectShow base–class library.

In some cases, you might simply pass the QueryAccept call downstream, then attach the media type to the output sample and let the downstream filter handle the format change.

QueryAccept (Upstream)

This scenario is the opposite of the QueryAccept (Downstream) scenario. Here, the input pin proposes a format change to its upstream peer.

This scenario is somewhat more complicated than the previous one. The downstream filter attaches a media type to the sample that the upstream filter will obtain in its next call to IMemAllocator::GetBuffer. In order to do this, however, the downstream filter must provide a custom allocator for the connection. This allocator must implement a private method that the downstream filter can use to set the media type on the next sample.

In this scenario, the following steps occur:

1. The downstream filter checks whether the pin connection uses the filter's custom allocator. If the upstream filter owns the allocator, the downstream filter cannot change the format.

2. The downstream filter calls QueryAccept on the upstream output pin (Figure 4A).

3. If QueryAccept returns S_OK, the downstream filter calls the private method on its allocator in order to set the media type. Within this private method, the allocator calls IMediaSample::SetMediaType on the next available sample (Figure 4B).

4. The upstream filter calls GetBuffer to get a new sample (Figure 4C), and IMediaSample::GetMediaType to get the media type (Figure 4D).

5. When the upstream filter delivers the sample, it should leave the media type attached to that sample. That way, the downstream filter can confirm that the media type has changed (Figure 4E).

   

   Figure 5. QueryAccept (Upstream)

If the upstream filter accepts the format change, it must also be able to switch back to the original media type.

Example Scenarios

The main examples of this kind of format change involve the DirectShow video renderers.

• The original DirectShow video renderer filter can switch between RGB and YUV types during streaming. When the filter connects, it requires an RGB format that matches the current display settings. This guarantees that it can fall back on the Graphics Device Interface (GDI) if it needs to. After streaming begins, if Microsoft DirectDraw® is available, the video renderer requests a format change to a YUV type. Later, it might switch back to RGB if it loses the DirectDraw surface for any reason.
• The new Video Mixing Renderer (VMR) filter will connect with any format that is supported by the graphics hardware, including YUV types. However, the graphics hardware might change the stride of the underlying DirectDraw surface in order to optimize performance. The VMR filter uses QueryAccept to report the new stride, which is specified in the biWidth member of the BITMAPINFOHEADER structure. The source and target rectangles in the VIDEOINFOHEADER or VIDEOINFOHEADER2 structure identify the region where the video should be decoded.

Implementation Note

It is unlikely that you will write a filter that needs to request upstream format changes, since this is mainly a feature of video renderers. However, if you write a video transform filter or a video decoder, your filter must respond correctly to requests from the video renderer.

A trans-in-place filter that sits between the video renderer and the decoder should pass all QueryAccept calls upstream. Store the new format information when it arrives.

A copy-transform filter (that is, a non-–trans-–in-–place filter) should implement one of the following behaviors:

• Pass format changes upstream and store the new format information when it arrives. Your filter must use a custom allocator so that it can attach the format to the upstream sample.
• Perform the format conversion inside the filter. This is probably easier than passing the format change upstream. However, it might be less efficient than letting the decoder filter decode into the correct format.
• As a last resort, simply reject the format change. For more information, refer to the source code for the CTransInPlaceOutputPin::CheckMediaType method in the DirectShow base-class library. Rejecting a format change can reduce performance, however, because it prevents the video renderer from using the most efficient format.

On a related note, you should be aware that in–place video transformations can seriously impact rendering performance. In–place transforms require read, modify, and write operations on the buffer, and if the memory resides on a graphics card, read operations are significantly slower. Even a copy transform can cause unintended read operations, so you should do performance testing if you write a video transform.

The following pseudocode, derived from CTransformFilter, shows how you might implement a copy–transform filter that can switch between YUV and RGB output types. This example assumes that the filter does the conversion itself, rather than passing the format change upstream.

HRESULT CMyTransform::CheckInputType(const CMediaType *pmt)
{
    if (pmt is a YUV type that you support) {
        return S_OK;
    }
    else {
        return VFW_E_TYPE_NOT_ACCEPTED;
    }
}

HRESULT CMyTransform::CheckTransform(
    const CMediaType *mtIn, const CMediaType *mtOut)
{
    if (mtOut is a YUV or RGB type that you support)
    {
        if ((mtIn has the same video dimensions as mtOut) &&
            (you support the mtIn-to-mtOut transform))
        {
            return S_OK;
        }
    }
    // otherwise
    return VFW_E_TYPE_NOT_ACCEPTED;
}

// GetMediaType: Return a preferred output type.
HRESULT CMyTransform::GetMediaType(int iPosition, CMediaType *pMediaType)
{
    if (iPosition < 0) {
        return E_INVALIDARG;
    }
    switch (iPosition)
    {
    case 0:
        Copy the input type (YUV) to pMediaType
        return S_OK;
    case 1:
        Construct an RGB type that matches the input type.
        return S_OK;
    default:
        return VFW_S_NO_MORE_ITEMS;
    }
}

// SetMediaType: Override from CTransformFilter. 
HRESULT CMyTransform::SetMediaType(
    PIN_DIRECTION direction, const CMediaType *pmt)
{
    // Capture this information...
    if (direction == PINDIR_OUTPUT)
    {
       m_bYuv = (pmt->subtype == MEDIASUBTYPE_UYVY);
    }
    return S_OK;
}

HRESULT CMyTransform::Transform(
    IMediaSample *pSource, IMediaSample *pDest)
{
    // Look for format changes from downstream.
    CMediaType *pMT = NULL;
    HRESULT hr = pDest->GetMediaType((AM_MEDIA_TYPE**)&pMT);
    if (hr == S_OK)
    {
        hr = m_pOutput->CheckMediaType(pMT);
        if(FAILED(hr))
        {
            DeleteMediaType(pMT);
            return E_FAIL;
        }
        // Notify our own output pin about the new type.
        m_pOutput->SetMediaType(pMT);
        DeleteMediaType(pMT);
    }
    // Process the buffers
    if (m_bYuv) {
        return ProcessFrameYUV(pSource, pDest);
    }
    else {
        return ProcessFrameRGB(pSource, pDest);
    }
}

ReceiveConnection

In this scenario, an output pin proposes a format change to its downstream peer, and the new format requires a larger buffer.

The QueryAccept mechanism described previously does not allow for the upstream filter to specify a larger buffer size. Therefore, if the new format requires larger buffers, the output pin should do the following:

1. Call IPin::ReceiveConnection on the downstream input pin.
2. If ReceiveConnection succeeds, call IMemInputPin::NotifyAllocator on the input pin.

In addition, the output pin may need to call IMemAllocator::SetAllocatorProperties and then decommit and recommit the allocator in order to change buffer sizes. Make sure to deliver all pending samples in the old format before changing the buffer size.

Example Scenarios

Some MPEG–2 decoders use this mechanism to switch between MPEG–1 and MPEG–2 output or if the video size changes.

Conclusion

To learn more about DirectShow, see HREF="https://go.microsoft.com/fwlink/?LinkId=20721" TARGET="_blank"DirectShow in the DirectX SDK documentation.