Turn on the faucet that's connected to the frozen pipe. If you're unsure of which faucet this is, turn on all the faucets. A faucet that doesn't run water or that emits very little water is connected to a frozen pipe.
Plug an extension cord into an exterior outlet. Plug a hair dryer into the extension cord and bring it under your crawl space.
Turn the hair dryer on and aim it at the frozen pipe to thaw the water. Alternatively, wrap an electric heating pad around the pipe. If you have neither of these items, wrap the pipe in hot towels. You'll have to unwrap the towels frequently to reheat them. Continue applying heat until all water flows freely from all of your faucets.
Insert pipe insulation sleeves over all the pipes in your crawl space. Exposed pipes in a pier and beam foundation can quickly refreeze if they lack insulation. An insulation sleeve for pipes has a slit running down its length. Fit the pipe into this opening.