Drill holes with a masonry bit through the mortar surrounding the brick you wish to remove. Place the holes as close as possible to loosen the mortar's grip on the brick and to keep any actions made to the brick that you're removing from affecting adjacent bricks.
Cut through the mortar, connecting the drilled holes, with a cold chisel. Place the tip of the chisel across adjacent holes on the top joint of the brick and strike the head of the chisel with a hammer gently several times to break through the mortar. Clear the top joint as completely as possible, followed by the sides and then the joint at the bottom of the brick. Clear away as much mortar as possible with the chisel to loosen the brick in place.
Scrape the mortar debris from the joints with the chisel point.
Pull the loosened brick from the fireplace face.
Clear out any remaining mortar debris with a wire brush.