It can be boiled down to a number of things:
First and foremost: the level of perversion that is common place in the medium and tolerated. Characters casually committing sexual assault on women by touching them against their consent which is played for laughs(see 7 deadly sins or literally any scene involving master Roshi and a female in the original dragon ball)
Or the justifications for pedophilic behavior and depictions of children or child like characters in sexual situations (mushoku tensei, the 'reverse grooming' in welcome to the NHK and once again--7 deadly sins, and also made in abyss.)
2nd:
The fans. The fans are some of the most cringe people you will ever meet in your life. They gatekeep their interest with a manic level of aggression, and they emulate the mannerisms of the characters on their favorite shows. It's extremely embarrassing to watch a teenager over the age of 13 or a grown ass man saying "you're so mean uwu~" in person. It makes me wish the neutralizer from men in black was real so I could un-experience that shyt.
And of course, many of the fans are racist.
3. Much of the tropes in anime are predictable and it gets tiring seeing the same thing over and over again. There's literally a Wikipedia article for "truck-kun" the majority of popular anime also is centered around teenagers, which makes people look at adults who engage with it the same way they look at adults who engage with Harry potter or other young adult fiction. People assume you're childish.
And I say this as an anime enjoyer. I understand completely why people don't fukk with it.
