I think anything has the potential to be funny... but the overuse is something can kill any comedic routine. The first time I saw the old dude I thought it was funny. Not laugh out loud funny, but eh. Same as the greased up deaf guy and the black weatherguy thats really to the point. But that all gets real old, real quick.
I have volume 1-4 of Family Guy on DVD (I used to work at EB Games and we sold them there so each one was like 3 bucks or something...) But I don't watch the new episodes at all. Its trash. Stale. Formulaic trash.
Anything can be funny. Technically.
Arrested Development does consistant incest humor which is absolutely funny. So, pedophillia could be made funny theoretically, same as rape or anything else... it would just require talent I don't think Family Guy's writing staff has.