Cut synthetic paper into a 4 1/2-by-6-inch rectangle. Fold a 1/2 inch crease on each side. Pinch the corners and fold each inward. This will form the floating base of the lantern.
Glue two of the toothpicks together side-by-side. Allow glue to dry completely. Repeat with remaining toothpicks.
Create four tissue paper designs, one for each side of the lantern. Each design should be as tall as the toothpicks and measure 3 inches in length. Glue one toothpick pair to the edges of each tissue paper design. Make sure the toothpicks are glued to the backside of the design. Allow glue to dry completely.
Apply a small strip of glue to the toothpick pairs on the back of each design. Arrange the pieces into a standing box and secure by pressing the edges together. Allow glue to dry completely.
Place a lit tea candle in the center of the synthetic paper float. Place the completed paper lantern over the tea candle. Gently set the lantern on the water's surface.