Any process involved in the maintenance of an internal steady state of ornithine within an organism or cell.