What Is a Stencil Buffer?
The outcome of that stencil test determines if the color value of the pixel is written to the render target and if the depth value of that pixel is written to the depth buffer.
For example, if some objects are rendered to a scene while the reference stencil is 0, and the stencil buffer has been cleared to 1, the stencil buffer will contain a cut-out pattern of zeros where those objects were rendered. If the reference value was then set to 1, and the StencilFunction was set to CompareFunction.LessEqual, the only pixels that could be rendered to the scene are those that are not in the same location where the stencil value was set to 0. This is a basic way to use the stencil buffer to create an area that is off-limits to the current rendering pass.
The stencil buffer can be used in more sophisticated ways. It is possible to specify StencilOperations that go beyond replace or discard, to increment or decrement the stencil buffer after each successful stencil test. This can be combined with the StencilMask value to ensure that the stencil test only operates on a part of the stencil buffer.
To use the stencil buffer, the DepthFormat must reserve some bits for the stencil buffer. The DepthFormat.Depth24Stencil8 depth format allows 8 bits for a stencil buffer - combined with the RenderState.StencilMask Property, this can provide eight different stencil buffers for the programmer. The Depth24Stencil4 depth format uses 4 bits for the stencil buffer, while the DepthFormat.Depth15Stencil1 format only allows one bit. The stencil buffer can be cleared seperately by passing ClearOptions.StencilBuffer to the GraphicsDevice.Clear method.
You can create your own depth-stencil buffer using the DepthStencilBuffer class. You may need to create your own depth-stencil buffer when you create a custom render target