The process that modulates the frequency, rate or extent of cell proliferation in the dorsal spinal cord.