Older water heaters have less internal insulation than modern water heaters, and are prone to losing heat quickly. The only way to keep the water hot inside is to keep the heating elements active, regardless of whether they are gas fired or electric. Because the water heater is not insulated, so much heat is lost through the sides of the tank that it is more efficient to keep the water heater on at all times.
Modern insulated water heaters, or those with insulating blankets, hold the heat in far longer than non-insulated types. Because the water remains hot longer, turning the heating elements off is a more efficient way to heat the water inside. However, allowing the water inside to go completely cold before heating it up again defeats the purpose and is less efficient. However, if all of the hot water is used up during these cycles, and the water heater is allowed to sit for a few days of non-use, it is more efficient to let the water go cold and heat the water only when it is needed.
The lower the temperature setting of a water heater, the less energy is consumed to heat the water, and the more efficient the water heating cycle becomes. In any water heating scenario, the water heater becomes more efficient if the water only needs to be heated to a lower temperature. Turning the water heater off for a period of time, then reheating the water inside, uses less energy if the water is only heated to 100 degrees Fahrenheit as opposed to heating it to 130 degrees.
Tankless water heaters are designed to be left on all the time and heat the water as it is needed for use. These are the most efficient water heaters on the market, and give a substantial energy savings over tank-style water heaters. By design, they heat water only when hot water is called for, and once the need for hot water is done, the heater shuts off automatically. There is no water that needs to remain hot, therefore, no wasted energy to heat water that is not being used.