Concrete is formed into many shapes and sizes of masonry units. Standard cement blocks are made 8-by-8-by-12 inches, as well as 8-by-8-by-8 inches, and they come with and without cells, or openings. Cement blocks are especially effective in a border and they can simply be set end-to-end without using mortar. Flat pavers can be turned on their side and set partially in the ground to serve as a border. Some masonry units are specifically designed for borders. They may have scallops on the top and may even have grooves and tongues where they can be interlocked. You can also buy molds and make your own border components.
Wood that is not treated to withstand moisture and insects does not fare well in contact with the ground. Some species--like cedar and redwood--will take ground contact well, but their cost and natural beauty makes them better suited for building applications. Landscape timbers come in various sizes but the most common is 3-by-3 inches. Using just one as a shallow border works well. Or stack them to handle border areas where some dirt will need to be retained. Install uprights to fasten them to, or hold them together with screws or nails.
Just about every place has stones that are natural to the area, and they can be used to add a very natural look to any garden border. Use the thick ones--6 inches or thicker. Stacking these one or two high will give you plenty of height for most borders. Or vary the height by having the stones in a single layer where the plants behind the border are short, and stacked up where the plants are taller. When placing stones consider their colors and markings so you can create a natural looking border.
Metal suppliers sell plate metal that is ¼ inch thick and 4 or 6 inches wide. These usually come in 20-foot sections; they bend into curves and they are strong and long lasting. You will need to drive in stakes where two ends butt together and wherever there are corners. In most cases, if they are set slightly into the ground the plates will be largely self supporting. Thinner versions of these are sold at home improvement stores with all the accessories to install them.