I almost worked there, I know the history pretty well.
Started by Henrry Brooks, he was a food salesman, then he sold his shop and opened up a haberdashery/tailor spot in Manhattan. When he died, his sons changed the name and expanded the clothing store. Pretty soon it was known for quality (a quality it had until the 60s/70s).
Presidents, statesman, and bankers wore the clothing, it wasn't cheap.