Clean both the brick and the brick chip with soap and water. Use a power washer to clean the brick within the structure if it's exceptionally dirty. Allow the brick to dry overnight, as it is porous.
Run a dry cloth over the exposed brick to remove any dust that may have accumulated overnight.
Coat both the area where the chip came off and the chip itself with construction adhesive. Use a glue roller to ensure even coating.
Press the brick chip into its niche in the brick, fitting it into place. Hold it in for at least 30 seconds until a bond forms and the pieces stick. Wipe away any oozing glue with a damp rag.