Burn wood in a fireplace if your home has one on the second floor. The master bedroom is usually upstairs and sometimes has a fireplace included. Fireplaces are not overly efficient but they are effective at heating the room they are in and they are inexpensive if you have access to firewood. Make sure your chimney is clean and the flue works properly if you haven't used the fireplace in a while. Master bedrooms sometimes utilize gas fireplaces, which don't require firewood at all.
Place a kerosene heater in the main area of your house where you spend the most time. Kerosene heaters are efficient and effectively warm a room, but they should not be left on unsupervised, so only use it when someone is present. Install a carbon monoxide detector in the room where the heater is to alert you if the heater starts burning improperly. Check your local ordinances regarding open-flame heater usage and follow all heater instructions for safe operation.
Plug in electric space heaters in the second story bedrooms. These heaters will raise your electric bill but if you don't have a furnace, you don't have the cost of one in your bill as it is. The space heaters are usually small and they help heat the individual rooms of the second floor that aren't getting the warmth from the kerosene heater.