Hornworms and budworms are caterpillars that bore through the buds of petunia plants, causing the buds to die before they open. Small brown holes in unopened buds and flowers with a "chewed" appearance are telltale signs of an infestation, which will be most apparent in late summer. Picking the worms off individually is the best option for small gardens, but pyrethrin or Bacillus thuringiensis (Bt)-based pesticides are effective for large areas or tough populations.
Examine your petunia plants one by one to find individual worms. They are green, and easily blend with foliage, but can sometimes take on the color of the buds they are eating. Use a flashlight and a magnifying glass if necessary. Don't be afraid to break open damaged buds--they sometimes hide inside, and the bud won't bloom anyway. They hide near the base of the plant during the day, but they come out to eat at dusk, making them easier to find.
Pick each worm off by hand, and drop them into a container of soapy water to kill them. Dump the water down a drain to dispose of it--dumping it on the ground gives surviving worms a chance to re-infest your garden.
Apply a pyrethrin or Bt-based pesticide to each individual plant. Bt-based pesticides are more effective if the worms appear to be eating leaf portions, but pyrethrins are more effective if they seem to be sticking to the buds. Apply the pesticide according to package directions.
Transfer each petunia plant to a pot with fresh potting soil for overwintering. Using soil from the garden may allow the worms to migrate inside with the plant, where they can completely destroy it over the course of the winter.
Rototill your petunia bed after you remove plants for overwintering, and again before you replant in the spring. This crushes the worm pupa that hide in the soil and destroys the environment the adult worms live in, leaving them less likely to survive long enough to infect your petunias.