If you are looking for a bit of peace and tranquility for your small space, do not forget to add the element of running water. Whether you purchase a tiny pond or table-top/wall fountain, you can bring nature to you, while muffling annoying exterior sounds. With a pond, you can even add a water lily or water hyacinth, thus utilizing the pond as another planting area.
Growing up instead of out helps free up space on the ground for other plantings. Plant a climbing rose or jasmine on a trellis, then just below it plant creeping phlox or thyme. Not only do you bring the scents of flowers to nose-level, you visually heighten your garden, making it seem larger.
An arbor not only provides shade, but once wisteria, honeysuckle or another heavy-flowering climber has grown over it, it also helps to bring the eye upward and you can use the area beneath the arbor for shade-loving plants, such as bleeding hearts or columbine, or for a small table and chair set.
Along the lines of bringing the eye upward and utilizing precious space in a small yard, try using hanging or wall baskets. Place a black-eyed Susan or fuchsia in these baskets, as these plants tend to grow beautifully over and around the baskets in which they are placed. Wall planters use the wall space on your porch, deck or exterior wall, so you can fill unused space with flowering plants.
Probably the most important aspect to small landscaping is to add a place where you can enjoy the beauty you have created. Look for a small table and chair set which nests inside one another when not in use. You can even combine seating with plants by purchasing a bench that features a pot on either side that you can fill with flowers. Even a small garden bistro set will work well in a small space; and when the chairs and table are not in use, you can place potted plants on them.