Look for screws on the face of the hanger. If you cannot see the screws, look at the top or bottom of either end; the screw holes are sometimes hidden. Take the screws out to remove the hanger. You may need to lift up on the hanger to remove it. If there are no screws, it is attached in an alternate way.
Check the towel hanger for evidence of glue. If the hanger is glued onto the wall, take a putty knife or a thin metal knife and attempt to break the glue on both ends. If tapping the glue with the knife doesn’t break it, use a hammer on the knife to provide more force. After you have chipped away the glue, gently pry the towel hanger away from the wall on both sides. Do not try to pry away one side only; loosen one side, then proceed to the other side.
Tap the bottom of the hanger with a force that lifts it up, if it is neither screwed into nor glued onto the wall. Some hangers rest on a bracket. Removing the hanger from the bracket requires you to exert a small amount of force on the bottom of the hanger to dislodge it from the bracket.