Any process that modulates the frequency, rate or extent of the physical partitioning and separation of a cell into daughter cells.