They can do X-Men without them crossing the Avengers world. And that's how I would want it. The Netflix shows, Agents of SHIELD, The Inhumans, Dr Strange, and the MCU movies themselves, show that there are other people around with abilities, people that have been around. However, with the MCU, since fans seemingly want them incorporated, the best story to do would probably be 5 Lights, using the 2nd Snap. Mutants have been around, dealing with the usual, X-Men are out there, but such an afterthought, but since it was less than 200 (known) in the world, the snap gives an exponentially increasing amount, causing even more concern and uproar.
But I would also say that they must keep Ryan Reynolds. They can probably keep the New Mutants cast, if the movie ever gets released. But there's no way they sit on the X-Men for 5 years. They don't even have much on slate except GotG3, Widow and Eternals. Dr Strange 2, Shang Chi and BP2 are so far away from development, they can easily put the X-Men out there. And Fantastic Four just doesn't seem to work with today's crowd, but I do think they'll go with that first.