RZA said the same as OP before.
Other members in the group have too, over the years. He said that after they went platinum, he started to realize that most Wu fans weren't black. He's said a few times that when they first started, at the shows, the whole crowd would be black. But now, when he looks out there, he sees mostly white people. Ghost said that he thinks "his people" moved on from Wu when white people got put up on them, lol.
And it's true. Back when OB4CL dropped, Wu was still very much held mad high in the community. Once they went global, and hit the suburbs, there was definitely a shift in support from the street. White folks worship Wu. RZA said when he started seeing that, he saw less support from the day one black fans.