Depending on circumstance, it could be easier. I think more people would understand if they took him out in the 9th if his pitching count is getting in the "high terrority" or he's throwing high level stress pitches. My overall point is that baseball has gotten so "smart" people can't even comprehend why baseball fans would be upset by this. That's when there's disconnect. Baseball, like all sports, needs a balance between being "smart and entertaining", because this is an entertainment business first and foremost over anything else. Not even allowing him to attempt to get through it is going to piss people off, because that's a potential historic moment that everyone who loves the sport would have loved to see.
Not saying let him get the perfect game no matter what. There's an obligation to do right by your players, to try to win and to entertain the public. It seems like the last part is what people in baseball care the least about now.