Definitions of on the contrary:

• adverb:   contrary to expectations

  Example: "On the contrary, he went out with his friends"