Any process that modulates the frequency, rate or extent of transcription from RNA polymerase V promoter.