Protecting the rim/paint could be a problem down the road, especially against a team like the Raptors in a series.
While they don't have another star like Kyrie, they have more than enough self-scorers and microwave players that can get theirs at will (Love, Hood and Clarkson) that will go a long way to making up the difference. The thing here is, if you have your best defender on LeBron, who's guarding 6' 8" Hood on the perimeter? Who's guarding Love down low or on the perimeter? Who's guarding Clarkson? Who's guarding Korver, Hill, Green? The Cavs now have A LOT of versatility on offense, as well as youth/athleticism to keep the heat turned up high, and by the law of averages will eventually expose a weak point in a team's defense - there's just far too much scoring talent not to.
Most of their issues on defense came down to effort (LeBron being the most guilty), and players who were either limited or over-the-hill in that area - IT, Rose, and Wade - those players have now gone. And LeBron and all the others there will only naturally give more effort on that end because of the moves made, and because they'll feed off the energy all the young guys bring to the team, even guys who were on the team like Cedi. Even if they won't be notably better, structurally, on defense, they'll be a whole lot better just because they'll have more youth, size and give more effort.
Hill's still a damn good defender too, so at least they'll have a better initial point of resistance in the PnR, as opposed to when IT was there.