The process in which a relatively unspecialized cell acquires specialized features of a tongue muscle cell.