If you look up the Genesis Apocrypha, which is a manuscript found in the Dead Sea Scrolls which has been carbon dated to about the 3rd-2nd Century BCE, if you read between the lines in the section about Noah's birth, There does seem to be an insinuation linking pale skin to the Nephalim/Rephaim.
The Nephalim/Rephaim were the elite Military ruling class amongst various Bronze age Canaanite/Levantine dwelling tribes and nations according to the Torah during the time when Israel would've been coming out of Egypt and settling the Levant.
The Nephalim/Rephaim/Canaanites are also heavily associated with the Mittani (Hurrian) empire, who were a Caucasus/Russian Steppe derived confederation who colonized the Levant during the mid-bronze age and absorbed and amalgamated various Canaanite and Babylonian/Amorite tribes according to a 2022 academic research paper called "The Historicity of the Rephaim" and a book called "Military History of Ancient Israel" (This event is also written about in Genesis 9:27).
The Canaanites that the Israelites that came out of Egypt in the late bronze age would've encountered were likely of the Caucasus/Hurrian/Mittani variety. I.E. modern "Middle Easterners"/Levantines, who have Caucasus derived ancestry. As opposed to indigenous Natufian Levantine/Mesopotamian/Arabian/Shemitic ancestry.