Pull up the existing carpet. Unless only a very small section of wood has rotted, it is better to completely remove the entire carpet segment covering up the wood you suspect of being rotted. If the carpet smells moldy, it should be disposed of. If it is not moldy, you could potentially reuse the carpet. This is a better option when the wood rot isn't linked to leakage seeping through the carpet, but rather from water coming from underneath the floor or from down a wall.
Remove any underlay that was installed under the original carpet.
Mark out the area with a pencil where the wood is rotted. If the majority of a plywood sheet is rotted, use a pry bar to pull it out completely. If only a portion of it is rotted, use a carpenter's square to mark out a square cutting area around the rotted portion of the floor. Cut the square out of the sheet using a skill-saw. Use a mallet and wood chisel to break the board out near walls if you can't reach it with your saw.
Cut a piece of plywood the same thickness as the original subfloor to fit in the hole. Nail it in place. Nail an entire new sheet of plywood onto the floor joists if you opted to remove an entire sheet instead.