Mike Brown was a problem but I think the roster they gave Brown was worse. The front office signed a 6'1" tweener guard who couldn't play defense or pass off the dribble and had no experience in big moments and that was supposed to be LeBron's top sidekick in a title run? Remember when Kobe was demanding more help in LA and they got Pau, imagine how he would have reacted if they got him Mo Williams instead.
Their roster was like a grab-bag of basketball skill elements with no rhyme or reason to how they would fit together.
And then there was also just bad luck. The Cavs actually could have handled the Lakers because they just happened to match up well - the Lakers had no beast offensive presence inside to take advantage of Cleveland's skinny big men(Bynum was a 20min defensive presence at that point with little offensive development), Varejao and Iglauskas matched up fairly well with Pau, neither Fisher nor Farmar were going to burn Mo's weak defense, and LeBron outplayed Kobe more often than not.
On the other hand the Cavs had a horrible roster against Orlando. Dwight could just manhandle Z inside, Varejao wasn't nearly fast enough to chase Rashard out to the 3pt line, Turkoglu was a good 7" taller than Delonte West, and Mo got his ass burned by Skip to My Lou all series long. Mike Brown's attempt to have LeBron guard Pietrus and then just help on everyone didn't work for shyt, but what would have worked? He probably should have had LeBron stick on Turk all game with Delonte on Pietrus and then had LeBron double off of Turk, but that would have only been slightly more effective. It was just not a very skilled or balanced roster.
