Cut down the tree with a handsaw or chainsaw if it is full-grown. Ensure that there is plenty of room for the tree to fall by moving items out of the way. Cut straight into the tree about half way on the side that you want the tree to fall, then cut an angled cut below the first, aiming it in toward the first cut to form a wedge.
Make a third cut on the back of the tree from the first two. With the wedge on the front, the tree has a lot of tension as it tries to lean in on this cut area. By cutting the back side, you release the tension and the tree will fall toward the wedge.
Dig up the tree with a shovel. If you are dealing with newly formed banana grass, you could use a spade instead of shovel. The roots are shallow and are found within the first 12 or so inches of soil. Get all of the roots, or the bananas will grow back.