Exactly. The multiverse ish doesn't work either, because mutants always exist in every universe. The latent X-Gene has always existed, and will still have to always exist...most mutants don't awaken their mutant powers until puberty. Apocalypse, Magneto, Xavier, Mystique, Logan already were mutants in the comics before the X-Men made their first appearance...even if you say Logan/Mystique/Apocy were retcons since they came later, Xavier & Magneto were still mutants for decades when the X-Men were children. They could do stuff like "going in and out of the Hex" awakened Monica's latent X-gene.
Since Iron Man 1, they've introduced Ant-Man & Wasp, Goliath, Sorcerer Supreme/Ancient One, Cap, Asgard, Black Panther, Captain Marvel, Skrulls, Winter Soldier, Inhumans, and all of the powered beings that happened during the Agent Carter Series, and they're going to put in the Super Solider Serum Experiments in Falcon & WS - all of having been around before Iron Man...the "In the 8 years since Mr. Stark announced himself as Iron Man, the number of known enhanced persons has grown exponentially..." If you want to count Netflix Marvel - you have Jessica Jones, The Hand, Iron Fist.
It seems the people that want the Multiverse have a problem with the X-Men just sitting around during the events of the MCU. Well, one most important thing is that the X-Men don't give AF about most of the Avenger issues...just like the Avengers don't give AF about theirs. If anything the "don't know/never cared" works best for future tension. Hell, honestly, the Battle of NY would be the only one of note. Infinity War were specific targets...only 1 happened publicly (Time stone), and it was over fairly quickly. Endgame was off in ones "secure" location, and you can easily retcon some random X-Men in the battle (i.e. Battle of NY).. If the mutant numbers are low, the handful of adult X-Men are not about possibly lower their numbers by participating in a battle...they simply were protecting their home/the children in case any came close. Even Quentin Beck turned out to be a liar.
When has the Multiverse ever worked in the comics? They really only use it for the Spidey. The alternate timelines people (Kang, Future-Franklin, Summers Clan, Bishop, OML, Dark Beast, AoA folk) all are used in the 616. The Ultimate line flopped hard...it was never intended to be a Multiverse. They didn't even use Ultimate Fury, they created their own...and they ended up folding over the only ones that anyone cared about (Miles & Maker).
Endangered Species/Five Lights story makes way more sense for the X-Men than multiverse world full of only mutants. In Endangered Species "there aren't enough of us around to do anything other than protect our own, to ensure we thrive."