That will occur if/when the metal cover is not grounded. Remember, the tester is shunting 10-15 ma to ground when the test button is pressed. If the ground, i.e. on the cover, j-box, conduit, etc is not present downstream then your body will provide a path (assuming you are somewhat grounded) and a mild shock will occur.