Determine where the gravel will go once you pull it up. Call local contractors and masonry centers. You may find a resource who will take it from you. If not, call the local dump and see where you are allowed to dump it.
Inventory what tools and resources you have at your disposal and calculate a budget. Access to a tractor and a large pick-up truck makes the job a lot easier. If you don't have access to these items and can't afford to rent them, your job will take a lot longer with a rake, shovel and trash cans.
Divide the area up into sections to work in. Clearing one section at a time reduces the amount of gravel that moves from one area to the next while you are trying to pick it up. This is especially important if you are doing the job without larger tools. Shovel large sections, dumping it into the trash can. Rake up residual gravel into small piles and shovel. A tractor makes life a lot easier by picking up large amounts and dumping them into the back of a truck.