Shrubs are easier to remove if the branches are cut back so you have easier access to the stump and roots. Use hand pruners for wood that is less than 1/2-inch thick; but, for most overgrown shrubs, you need to use loppers, a hand saw, and even a chainsaw. Start by topping the shrub and reducing the height. Then remove the lateral branches. The most important area is the lower part of the shrub. You should have at least 1 foot of clearance for easier working.
If you are removing just one smaller shrub, you can dig it out with a shovel and use a root saw to pry out the roots. Larger shrubs, or an entire series of shrubs, will need some other method. A stump grinder may be rented at an equipment supply store. You can also use chemicals that will kill the stump and hasten its decomposition. Many of these can contaminate the soil, however, so be careful where you use them. Never use a rope and a truck to pull out a stump. It can snap or rebound and seriously injure someone. Also consider any infrastructures that may be destabilized by volatile removal.
Most trees and shrubs have the majority of their roots in the top 3 feet of soil. The finer feeder roots are in the top 12 inches as a rule. This means digging out a pruning shrub can be achieved with some deep digging. Very few shrubs have a taproot, which is good, as they can extend many feet into the earth. If you encounter a taproot, you can cut it off and dab glyphosate on the open surface of the root to kill the remainder buried underground. If you choose not to do this, you may find the shrub will return after removal.
Really mature shrubs that have been established for years can be grown into other plants or become so entrenched that removal may be beyond your manual abilities. You can use a jack to give you the extra force necessary for really hard-to-remove trunks. Dig around the trunk and remove as many roots as you can. Make a scaffold over the tree root out of a thick beam and attach a chain to the trunk and center of the beam. At one end of the beam push the tire jack underneath. Gradually raise the jack, cutting exposed roots as they surface. It will take some time, but the trunk and most of the root system will come out.