Remove anything stored in the ceramic soap holder.
Place plastic sheeting on the bathtub or shower to protect it from any chips during the replacement process.
Cover the tiles immediately surrounding the soap holder with duct or painter's tape. This will protect them against scratches. Don't cover up any of the grout spaces around the soap holder.
Use a utility knife to cut away any caulk around the edges of the soap holder.
Cut away the grout around the edges of the soap holder with a grout saw. You can also use a rotary cutting tool if you have one. Take your time and work your way around all of the edges of the holder.
Insert the edge of a chisel in the grout line so it's pointed toward the soap holder. Gently tap on the chisel to separate the soap holder from the wall. Work your way around the soap holder with the chisel from different sides, if needed, until the soap holder comes off the wall.
Use the chisel to chip away at any remaining adhesive on the wall.
Apply ceramic tile adhesive to the back of the new soap holder. Press this into place on the wall. Continue holding the holder in place for about five minutes. Use duct or painter's tape to secure the holder in place on the wall.
Remove the tape when the ceramic adhesive has dried according to the manufacturer's instructions.
Insert a tube of silicone caulk into a caulk gun. Trim an angle off the end of the caulk nozzle with a pair of scissors. Work your way around the soap holder with the caulk gun, sealing the edges of the holder against the wall.