Rigid foam is an effective material for cutting out cold basement drafts. It is particularly suitable for use in older basements because it is resistant to damp and moisture, elements that are often present in such basements. You can cut rigid foam with a sharp utility knife to fit tightly between the floor joists and pressed against the interior surface of the foundation walls. Doing this carefully and minimizing any gaps will greatly reduce the amount of cold air that enters through the basement walls. The most important part of the wall to insulate is the part that is above ground level on the outside.
Expanding foam is an insulating product that you can buy in a can with an attached nozzle. The advantage of foam is that you can place the nozzle in hard-to-access nooks and crannies and spray in a bit of foam, which then expands and cuts off any infiltrating air. This product is ideal for the uneven surfaces and poor insulating values of an old stone foundation wall. You should use expanding foam only for cracks and drafts rather than large areas, because it costs a lot per square foot compared to a product like rigid foam panels.
Building a complete framed wall on the inside of the foundation wall in the basement allows you to insulate the entire surface and isolate the interior of the basement from the foundation wall. You can insulate between the studs of the wall with fiberglass batts or with rigid foam sheets, then finish the wall with water resistant drywall or with wood paneling. Unless you are planning to use the basement as a living space, this amount of work is probably not justified. For keeping the living area above the basement warmer, it's more important to prevent drafts from hitting the underside of the floor.
The interior of the foundation wall can be insulated without the need to build an entire wall by applying furring strips to the surface of the wall and installing sheets of rigid foam between the furring strips. If the inside surface of the wall is rough and uneven -- a common condition for stone walls -- this will present problems for attaching furring strips. You may have to shim them extensively to make them consistent enough to install rigid foam.