I was reading this post about why Stannis was so pissed at Robert for giving him Dragonstone when Storms End was the ancestral seat of House Baratheon, and I personally always saw it as a punishment for allowing Viserys and Daenery's to escape from Dragonstone after Ned Stark broke the siege, but I just read a pretty interest quote from GRRM about it...
That was actually kind of dope of Robert even though Stannis always saw it as some kind of disrespect, pretty much means Robert always saw that Stannis should be king if he should die with no heir. Cersei claims that it was a slight but she doesnt know what she's talking about ever so there's that.