The perception of Malcolm even in the black community should be re-examined. Towards the end of his life, his views dramatically started to change. The Manning Marble book is good, but people don't want to hear the truth so they say it's inaccurate. Malcolm basically worshiped Elijah and finally came around to it that he was being played. Instead of taking one little section of his life to define the man, look at the whole picture.
If you want to talk about the "HARDEST" person of that period, it probably was Adam Clayton Powell JR. He was outspoken and didn't give crap what others thought, but still was getting legislation passed. It's stated he passed the most legislation of any member in the history of congress.
Booker T Washington has a somewhat "soft" reputation. However, he was getting money from the richest people of his era to help him build schools and programs. That takes a skill that many will just call c00ning. He was still funding the equality movement under the table, but knew how to play the poker game to get what HE wanted accomplished. It's better to learn to play the game than to sit on the couch and complain.