Any process that modulates the frequency, rate or extent of cell chemotaxis to fibroblast growth factor.