If the tree was small and fairly young, the cheapest way to remove the stump is to simply pull it by hand. This method, also known as grubbing, is dependent upon your own strength versus the strength of the tree stump. It may be necessary to dig around the stump, making a trench that allows you to cut away at the roots and/or gain leverage with a tool.
As cheap as pulling by hand -- but hardly as quick -- is allowing the stump to simply rot away. Nature's own organisms, including fungi, will slowly but surely cause the stump to rot away. This method can take years, so the cheapness has to be weighed against the desired quickness.
While this method requires an actual purchase, using a chemical to destroy the stump is still cheaper than calling a removal professional. This method may not be very quick, because the chemical may have to be used more than once before the stump is dissolved enough to be pulled out of the ground by hand. With a few holes drilled into the stump, chemicals for stump removal actually cause the stump to rot away, but more quickly than natural rotting.
Another cheap method of stump removal is to burn the stump out of the ground by loading kindling on top of the stump and burning the kindling until the stump also burns away.
Be careful if you are pulling the stump free, or burning or grinding the stump out, because the roots of the tree may have grown around power, water or phone lines. If there is any uncertainty, consult the utility company to obtain the location of the utility lines.