Cut all flower heads with sharp scissors and bag them if the weeds have formed seed heads. This is to keep the seeds from dispersing as you are digging out the weed.
Use a hoe to dig out the weed carefully. Be sure to dig out all of its roots as well.
Look for small rosettes and fallen seeds in the cleared area. Remove all of these to prevent the germination of seeds and the appearance of new plants.
Use an herbicide if the milk thistle is too widespread to remove by hand or if you are unable to dig out the entire plant. Herbicides containing dicamba are effective for control of milk thistle, according to the Oregon State University Extension site. During spring, spray the herbicide on newly germinated weeds that are still in the rosette or seedling stage for best control. Spray again in early summer and then in fall to cover any later-germinating seeds. Use the herbicide on warm days for better effect of the chemical.
Do not mow the treated area until the weeds are completely dead to minimize chances of seed dispersal. Dig out from the area all dead weeds and their roots and dispose of them, along with any rosettes or seeds.