Any process that modulates the frequency, rate or extent of telomerase RNA reverse transcriptase activity.