Ivo is much closer to Ivor/Ivan than Yvonne to me and they're definitely usually pronounced with an "eye" sound in English.
Just FYI, Ivo is not a variant of Ivan or Ivor. They are unrelated names. Ivo is a name in its own right but variants include Yves and Ives (as in St. Ives), and the feminine versions are Yvonne/Ivona, Iva and Yvette. Traditionally it was pronounced Ee-vo (vo rhyming with so).