He lost Mozgov a few times. But when your first line of defence gets broken down so quickly and easily, most big men are going to have trouble recovering to cover up that penetration, especially with a guard as quick who handles the ball as well as Kyrie Irving. I'm not excusing Valanciunas' defence, but I don't think this particular game was a glaring example of his defensive woes. He made some mistakes, but I don't feel his defence was as bad as you're making it out to be.
As a center, part of being the last line of defence is you have to stop the ball on penetration. The problem with Valanciunas is he gets caught in no mans land defensively because he doesn't commit to either guarding the rim or stopping the penetration. That, and his tendency to not recover back to the rim after helping on penetration are his biggest issues.