The front door color should provide contrast with the brick. Hopefully, your entire home is made out of bricks that are the same color -- if only part of your home is made out of yellow brick, the other colors on the exterior of the home may clash with the yellow brick and make the home difficult to look at. Black often works well for doors to contrast with the yellow exterior of the home.
The shutters and the front door should be the same color. This creates a unified contrasted look to the yellow brick. If the shutters are not the same color as the door, it creates the same problem as painting the brick more than one color. Viewers' eyes won't know how to react to too many colors on the house and will tend to look away.
Some people prefer not to have any contrast between the brick and the doors, shutters and trim. You don't want your front door to be yellow, as it will blend in too much with the brick and make it difficult for visitors to find the door. However, if you prefer a monochromatic color scheme, try making your front door beige, cream-colored or white. These colors blend well with yellow brick and provide limited contrast.
If you plan to have a garden, you should consider how the colors in your garden contrast with your yellow brick house and your door. Yellow and purple are complementary colors; thus, any kind of lavender plants will go well with your yellow brick house. Consider a black or white door for your home so that the door won't clash with the color of your garden or the color of the brick in your home.