The damage has already been done as far as legitimizing Bannon's views and what he represents. Barely anyone outside of a fringe group of basement dwellers knew what 'alt right' was before the election, and now it's been normalized and propped up onto the national stage. That might actually turn out to be more damaging than any influence he has over policy decisions.
The good thing is he's hitching his ideology to an administration that will likely either crash and burn or at the very least leave office as disgraces to the country. That won't be a good look for his 'movement' but i'm sure this cockroach will be one of the first to jump ship if shyt starts to go south.