The multiplication or reproduction of cells, resulting in the expansion of a cell population in the midbrain.