Though not physically accurate, casting shadows directly under characters and objects can help players get a better read of depth in a platformer.
One way to cast shadows this way is to use shadow volumes. A geometric volume is used to represent an area within which we want things to appear as if they were in shadow, hence the name. For a character, its shadow volume could be a cylinder that extends down from the base of the character.
The method in which we render shadow volumes entails several render passes for the shadow volume geometry after rendering the scene as usual.
It relies on a key insight. Consider what would happen if we rendered our shadow volumes, first just their back faces and then their front faces. The areas where the back faces fail to render due to failing the depth test, but where the front faces would render due to passing the depth test, will correspond exactly to the areas where we’d want to render the scene in shadow.
To mark these areas without actually having to visibly render the shadow volume geometry we can utilize the stencil buffer. This is buffer which stores an integer value per fragment.
We first render the back faces and wherever they fail the depth test we’ll decrement the stencil buffer by one. We then render the front faces and wherever they fail the depth test we’ll increment the stencil value by one. Once we’re done, anywhere the stencil buffer is not zero corresponds to the areas we would like to render darker: they represent areas where the back face failed the depth test but the front face didn’t. This method, aptly named Depth Fail, works even when we render multiple shadow volumes.
The last step is to use where the stencil buffer is non-zero to render one last pass for our shadow regions. A simple way to make things darker is to use a multiply blend mode and render black with some alpha. But what geometry do we render to the color buffer? One option is to use a fullscreen tri that sits just in front of the camera. Another option is to render the back faces of the shadow volumes but with depth check turned off (always pass). With either option we reset the stencil value to zero on fragment write, clearing the stencil buffer for future passes.
Shadow volumes have the nice property that overlapping shadow regions render as one continuous shape. If we want visibly distinct regions we also have the option of rendering shadow volumes in stages.
For instance, if we wanted character shadows to render as visibly separate of other shadows, we could split our shadow volume rendering into two stages. In the first stage we render only the shadow volume passes for our characters. In the second stage we render the shadow volume passes for the rest of our objects. The result is darker shadows wherever there is overlap between the two stages.