Hold a 1/2-inch wood chisel's sharp edge at a 40-degree angle against the rotten wood. Gently tap the chisel's handle with a hammer to push the chisel through the wood. Continue to remove the rotten wood until the majority of it is gone.
Drill holes into the damaged area, using a 1/4-inch drill bit. Drill three to four holes, depending on the damaged area's size. Place one hole near the right-hand side, one hole near the left-hand side and one or two holes in the center. Evenly space the holes.
Pour 1/4 cup of wood consolidant over the damaged area. Wait for the wood to absorb it. Pour another 1/4 cup of consolidant on the damaged area. Repeat this step until the wood no longer absorbs the consolidant. Absorb the excess consolidant with a rag.
Mix epoxy wood filler according to the manufacturer's directions. Push the filler into the damaged area. Use your fingers to press it tight against the damaged area's surface.
Dip a small putty knife into some solvent. Use the putty knife to smooth the filler's surface and mold it to match the surrounding windowsill. Wait for the filler to harden according to the manufacturer's directions.
Use 120-grit sandpaper to smooth the filler. Wipe away dust particles, using a piece of damp paper towel. Use a paintbrush to apply primer to the sanded area. Wait for the primer to dry, and paint the area a color to match the surrounding windowsill.