Use a vacuum cleaner to suck out all of the loose debris inside the holes.
Cut the tip off the nozzle of a tube of caulk with a utility knife and load it into a caulking gun.
Set the nozzle in a hole and start squeezing the trigger until the hole is filled. Don't allow the nozzle to seat completely inside the hole, or an air-tight seal will form. This won't allow the caulk to enter the hole, because the air in the hole needs to be able to exit the hole for the caulk to flow in. Hold the nozzle to the side of the hole to accomplish this.
Smooth the caulk flush with the surface of the siding using a putty knife. Wipe off excess caulk with a moist painter's rag. Allow the caulk to dry for 24 hours. It will shrink during this time. Add more caulk, smooth it out and allow it to dry. Repeat this process until the level of the caulk in the hole is the same as the surface area of the siding around the hole.