Any process that stops, prevents or reduces the frequency, rate or extent of semaphorin-plexin signaling pathway.