Any process that modulates the frequency, rate or extent of adhesion between pre- and post-synaptic membranes.