Brush aphids off the herbs into a plastic box. This is particularly effective if you only have a few plants, perhaps on a windowsill. Crush the aphids, feed them to fish or drown them afterward.
Mix a couple of squirts of dishwashing liquid with water in a plant mister. Spray the herbs with the mixture, focusing on areas where the aphids are crowded. Look at new growth and the undersides of leaves. Avoid spraying beneficial insects, such as ladybugs, with the mixture.
Hose large patches of herbs. Have your thumb partially over the end of the hose to produce a strong jet and aim at the aphids. This damages them and knocks them off. Some will drown during the process and the rest will have problems getting back onto the plants. They’ll also make easy pickings for predators, including birds, while on the ground.
Develop a natural pest control team in your garden. Encourage ladybugs with shelters or by growing thick vegetation. Rosemary, an herb itself, provides an excellent ladybug shelter and makes a good addition to your herb garden. A small garden pond encourages frogs, which eat aphids when small and move onto larger pests as they grow.