Yeah I mean like when they figured out time travel they should have went to stop Loki from taking the space stone
Yeah but they was not supposed to also disturb the flow of time, Loki is paying the [price for taking the space stone from the timeline in the movie...