Cut the caragana down to the ground. If your caragana is large, like a tree rather than a shrub, you may want to hire a professional. Smaller, more manageable shrubs can be cut down carefully. Use lopping shears to trim off the taller branches and a saw to cut off the thicker stems and trunks.
Remove the stumps and roots, if possible. A spade may be insufficient to remove a large stump and root system. If so, you will need a backhoe to dig into the soil and lift out the stumps and main roots. You can also rent a stump grinder, per manufacturer's instructions. The leftover wood chips make good mulching material.
Paint glyphosate directly onto the stumps and any other remaining stems within five minutes of cutting. This is an alternative to removing or grinding the stumps. Spray caragana growth with glyphosate whenever it sprouts. The chemicals kill the new growth and its roots. In time and with persistence, the entire caragana plant will die.