Measure the diameter of the hole you drilled. If it is 1 inch in diameter or less, fill it with a piece of a dowel. Buy the dowel at any hardware or home improvement store. If the hole is larger than 1 inch in diameter, use a hole saw to cut the hole larger, then use the same hole saw to cut a plug that you can fit into the hole in the door. A hole saw is a circular blade that fits onto a drill bit for cutting round holes.
Cut a length of dowel that is 1/2 inch longer than the thickness of the closet door.
Spread glue on the sides of the dowel, and tap it through the hole in the door with a hammer. Allow the glue to dry for at least an hour.
Cut the ends of the dowel using a handsaw so it is flush with the face of the door. Sand the ends smooth with fine sandpaper.
Re-drill the hole for the knob through the center of the dowel, being careful to make it the right size.