Use a pencil eraser. While tedious, this is the most common method for removing non-grease stains from wallpaper. It's also the least likely to cause damage to your wallpaper. Gently rub the eraser over the water stain. You'll only be able to remove surface dirt, but it may make the stain less visible.
Mix a solution of mild dishwashing liquid and water, and gently wash the area of the water stain. Again, this won't remove the wood tannins that cause the water stain, but it can remove other dirt and grime so the water stain isn't as noticeable.
Finally, try a bleach and water solution that's 1/3 bleach and 2/3 water. Carefully sponge the solution onto the area with the water stain. Don't use too much--just enough to dampen the area, but not enough to drip down the wall. Give it 5 minutes to work, then rinse the area with water. Repeat as necessary.