Build terraces into the steepest areas of the hill to help control erosion. Employ bricks, railroad ties or stacked stones to do this. Place pea gravel or soil on the inside area of the terrace to allow water to escape. Create other terraces below and above the first one to make a tiered effect.
Plant plants along the terraces that will help hold the soil and prevent erosion.
Plant groundcover along the steep sections of the hill. Choose hardy groundcovers such as ivy or creeping juniper. As the groundcover takes root, it will help prevent the soil from eroding. This is a good option if the hill is too steep for walking on it. It will also mean that you won't have to worry about mowing the area.
Don't place plants or a flower bed at the base of the hill. Water will naturally gather there after rainfall, causing the plants to flood and likely become overwatered.