People forget that often times men will find "exotic" women (meaning, those that are not the norm) more attractive. Thus, in predominantly Black cultures, it will be women who have that different skintone, light-skinned or white. And in predominantly white cultures, it will actually be tanned women and light-skinned Blacks (and Arabs in Europe). There's a reason why Latinas and Arab women are tops in erotic stereotypes among white people.
So you end up with light-skin Black women being seen as more attractive by a whole lot of men, from different groups. Plus Latinas and Arabs (in some Euro countries, it's a high fetish here in France).
But you have to indeed add social construct. They work hand in hand.