Prune the bush down to as close to the ground as you can get. Dispose of the trimmings by renting a wood chipper for the day or making a pile of the brush and letting it dry for later burning. The small twigs will make excellent kindling if you have a fireplace or woodstove.
Dig around the base of the plant, prying up the roots.
Cut through the roots with your lopping shears to loosen, then pull them up by hand.
Leave the roots that are very deep in the ground alone. Eventually they will die and rot away. Forsythia roots do not usually regenerate.
Use a pry bar or landscape bar to pry up the roots around larger stumps. As you pry up thicker roots, cut through them with your shears or with mattock. In some cases a bow saw might be necessary to get through really thick branches and roots.
Clean all of the stems and twigs away from the site when finished. Forsythia grows from cuttings and those left behind could start to take root.