It happens. I'm living proof. Ask my old lover I dealt with for over 5 years who loved me, but wouldn't be with me because I didn't fit into his image.
About how I went away to college, lost all the weight, came back and he started stalking me when I moved on. How all his friends clowned him, because of how bad he treated me and now I wouldbt give him the time of day because I knew better and has options. How he Felt entitled for me to marry him, because be dealt with me when I was big. (Even though he didn't treat me as good as he treated other women messed with, and never made me a priority) how he called me 60 times in one day, and parked in front of my house threatening to kill himself if I didn't come out and talk to him. How he admitted he had been p*ssy whipped and scared to live without me.
I had to actually threaten him with a restraining order to get him to stop. I experienced this with someone else from my past when I lost weight as well.
It's not likely, but it happens.