Any process that modulates the frequency, rate or extent of high voltage-gated calcium channel activity.
