Mosley moved straight up to welterweight in 99 in order to pursue a fight with De La Hoya. Can't really blame him there as De La Hoya was the big money fight, not Floyd, but that small window between 98-99 was the only time they could have fought until around 06-07 (and by that point Mosley was past his prime anyway).
A Trinidad fight was never a real possibility as he had moved all the way up to middleweight by 01 while Floyd was still at super feather.
Mayweather cleaned out 130 and beat the main man at 135 in Castillo. The only name fighter you'd have a case of Floyd not fighting at either weight was Joel Casamayor, who was too busy going back and forth with Corrales (at 130 three or four years after he fought Floyd). I really hope no one brings up that sparring video with Paul Spadafora as evidence that Mayweather ducked him.
Floyd moved up to 140 to pursue a fight with Kostya Tszyu but Tszyu got his ass beat by Ricky Hatton before Mayweather could get to him. Then he moved up to 147 to face Zab Judah and Judah in his tune up fight ended up getting embarrased by no name Baldomir.
Now here's where Margarito came in the picture. Arum and Top Rank started this campaign that Margarito was the most feared fighter at welterweight in order to make him a big name, and people fell for the bait by assuming Floyd was afraid of him when he chose to face Judah and Baldomir instead. If Floyd did fight and beat Margarito at the time though no one would give him credit because Margarito's resume was actually very thin. People would instead say that Floyd didn't go after the lineal champion in Baldomir after Baldomir beat Judah.
No one brings up Margarito literally being forced to fight Paul Williams after he and Top Rank tried to go straight for a fight with Cotto. Williams and his team had to threaten legal action in order for Margarito to fight him, so who really was the most feared fighter at welterweight?
06-07 was the only time for Mayweather to realistically face De La Hoya as that was when both were in the same weight range (147-154). Then Hatton was spewing off at the mouth saying Mayweather was afraid of him, so Floyd ducked Hatton's head into a ring post with a check hook to shut his ass up. Hatton literally fell into a depression afterward.
Cotto was being groomed as the superfight for Floyd and would've been after 07, but Cotto of course got wrecked by Margarito. Margarito then got exposed as a cheater before getting knocked the fukk out himself by an old ass Mosley.
Pacquiao wasn't even a possibility until around 08, and as another poster has said Mayweather wanted the fight with Pacquiao in 09 but Manny refused to do a drug test.
I fail to see who else Floyd was supposed to fight from his generation. He eventually did face most of the guys he supposedly ducked as quickly as he could, and he really had nothing else to prove after 07. The only thing that kept him going was the possiblity of a fight with Pacquiao. Guys like Maidana, Canelo and GGG are a different generation of fighters that Floyd didn't have to mess with at all. Hell Canelo's 13 years younger as well as bigger and stronger yet Floyd still took him on and beat him.
Look no fighter on earth has a resume that can't be nitpicked. There's always someone else that a fighter should have supposedly faced, or fought at a certain time. Floyd is no exception, but I believe he made the most out of his boxing career given the time and circumstances.