Any process that modulates the frequency, rate or extent of cell proliferation involved in outflow tract morphogenesis.