Brick spall when they are too soft and porous. This permits the brick to absorb and retain too much water which then can freeze and expand breaking away the brick surface.
While this is merely a general rule and not an absolute, there are two typical times when brick spalling occurs. The first is when brick have been reclaimed and re-layed as "used brick". The second is a manufactured brick that was simply substandard.
The interesting thing about the first reason is that early American masons actually did understand brick density and vulnerability issues. They culled and set aside the soft brick for the interior withes of solid masonry construction. Unfortunately, masons in the 60's and 70's weren't as thoughtful and installed them as exterior brick. It didn't take long for them to fail.
Occasionally an inspector may notice spalled brick on the side of an old row house and assume this is an exception. But, typically, upon closer examination it will become obvious that even these exposed soft brick were not originally exterior brick. Another building which has been torn down once adjoined this building. The silhouette of the previous building will be apparent (outlined) through a change in brick appearance and quality of workmanship.
The second common circumstance under which an inspector will see spalled brick is in houses built in the 60's and 70's. Some brick manufacturers during that period of time were using inferior materials. When that fact became apparent, the ASTM devised absorption standards that some pits and manufacturers could not meet. They were forced to close down.
For these reasons, an inspector should not expect to see much brick spalling in homes built in the recent past or future.
As a side note, on occasion we may see brick that have come apart near the top of a chimney, but this is not typically spalling. When the crown of top joints fail on a chimney and permit water to enter and freeze in the cores bricks can literally spit apart.