According to the National Zoological Park, some hummingbirds, such as the rufous hummingbirds, use California as breeding grounds. When it becomes cold, they leave to their wintering grounds in the south.
According to San Fransisco State University, some Allen's hummingbirds stay in California for the winter. Birdscope reports that Anna's hummingbirds also don't migrate south in the winter.
Because some non-migrating hummingbirds remain in California for the winter, you can enjoy watching the birds year round. You can choose not to take down the feeders at all.