Any process that results in a change in the behaviour of an organism as a result of a formalin pain stimulus.