Examine the underside of your staircase with a flashlight. If there is a finished ceiling applied to the underside of the staircase you will have to remove it to reinforce the stairs. Cut away the drywall with a sharp utility knife. Note the joint between the underside of the tread and the top edge of the riser. If there is a gap here, it is probably the source of your problem.
Measure the width of your stairs between the inside faces of the stringers. The stringers are the diagonal pieces of wood that support the treads and risers.
Cut pieces of 3/4 inch-by-3/4 inch strips of wood to the measurement that you made in Step 2.
Drill pilot holes every six inches along the strips of wood. Turn the strips of wood 90 degrees and drill more pilot holes that are perpendicular to the first pilot holes.
Spread a thin line of glue along two adjoining surfaces of a strip of wood.
Place the strip of wood into the joint created by the underside of the tread and the top of the riser.
Drive 1 1/2-inch-long screws through the pilot holes and into the underside of the tread and the back of the riser, permanently affixing the strip of wood to the back of the stair.