I found you can knock it out with a punch. Once I twisted it out with a pair of dikes. The mechanical advantage of having your seal carrier attached to 400 pounds of engine is what makes prying much better suited to the quick on-the-car method than on the bench. Like you, I prefer to remove the carrier and have good access to clean the seal contact surface. My guess is just perhaps some of the failures you read about, where the seals, etc. are blamed, are really just due to poor lighting, poor cleaning, and well, just poor installation. Just my, uh, hydrodynamically opposed opinion...

rear main seal install
--
Art Benstein near Baltimore

Remember what Cardinal Cooke once said:
"A man would do nothing if he waited until such time as no one would find fault with what he has done". - originally John Henry Cardinal Newman