Lay a pressure-treated 2-by-4 inch board on the floor. Push the 2-by-4 tightly into the corner where the floor meets the rigid foam insulation. Check that the board is straight enough not to leave any large gap between itself and the insulation.
Flip the 2-by-4 inch board upside down. Run a continuous bead of construction adhesive down the board's length. Flip it back over and press it into the corner. Butt a second pressure-treated 2-by-4 inch board end against the first board's end. Glue this board to the floor as well. Continue until you have boards running the entire length of the wall.
Drill angled pilot holes through the top of the 2-by-4 inch boards and into the concrete wall with a hammer drill and 1/8-inch masonry bit. Space the holes 16 inches apart and penetrate the concrete by 1 1/2 inches.
Pound 3-inch fluted nails through each pilot hole and into the concrete wall with a 3-pound sledgehammer.
Measure the height from the bottom plate (the pressure-treated 2-by-4 inch board you just installed) to the ceiling joists. Transfer this measurement to a furring strip. Cut the strip to the appropriate length with a saw.
Run a bead of construction adhesive down the furring strip's length. Hold the furring strip against the foam insulation in the wall's left corner. Use a level to check that the furring strip is plumb. Drill 1/8-inch pilot holes through the furring strip, insulation and 1 1/2 inches into the concrete wall. Space the holes 16 inches apart. Hammer a 3-inch fluted nail through the pilot hole.
Measure 16 inches down the wall from the furring strip's right edge. Place a mark on the insulation. Measure the height from the bottom plate to the ceiling joist in the marked area. Cut a second furring strip to the appropriate length with the saw.
Run a bead of construction adhesive down the second furring strip. Hold its right edge against the mark. Use a level to check that the furring strip is plumb. Secure it to the insulation and concrete wall the same way you secured the first furring strip.
Repeat Steps 7 and 8 until you reach the wall's right-hand corner. Mount a furring strip to the concrete wall in the corner, even if this means the space between the furring strips will be less than 16 inches.