Any process that modulates brood size. Brood size is the number of progeny that survive embryogenesis and are cared for at one time.