Any process that modulates the frequency, rate or extent of CD8-positive, alpha-beta T cell proliferation.