For cons supporting school vouchers?

Wouldn't the rich children or parents in private school object to the poor children on vouchers being in the same school? I know I'm being a bit cynical, but wouldn't that happen sometimes?