Any process that modulates the frequency, rate or extent of extracellular vesicular exosome assembly.