Stone dust is particles of stones that have been reduced to a dust form. While sand is an aggregate that is grains of sand that vary in size depending on the quality of the sand, stone dust is just as the name suggests, which means it is a powder versus a rough type of aggregate. This means it compacts far tighter than sand.
The type of soil you are dealing with directly affects whether stone dust is right for your paver needs. Because stone dust is a powder in comparison to sand, it does not drain in the same manner. Since it does not drain as easily it naturally retains moisture, which can turn the powder into a clay-like substance over time if the soil around it is consistantly wet.
Similar to how moist soil will eventually affect the stone dust and turn it into a clay-like substance, heavy rainfall will do the same thing. If you live in a section of the country where heavy rainfall is a regular occurrence, stone dust may not be the right base layer for your pavers because it lacks the draining capabilities of sand.
The freeze/thaw cycle is the single most important factor in determining whether stone dust can be used for your pavers. If you live in an area where the temperature is constant, stone dust can be used regardless if it rains or moisture is present in the soil. However, because water expands when it freezes, you do not want to use stone dust if you live in an area where there is a freeze/thaw cycle in the spring and winter.