I think you can for sure have that connection, over dinner, I know I have, but what you said very accurately summed up something I think about a lot, to have a really close connection you have to drop some of the pretenses, some of the time constraints, the kind of scheduled dinner/date, and just be with each other, when there's no check to pay, or meal to finish, or closing time. You open up time. Forgive me if I sound too metaphysical on the time part, I just read Saving Time.
and to expound, I think sometimes people intentionally avoid that for social reasons (coming over = automatic sex) and because you lose the kind of performative nature of some dates, and get way closer, you also have to both let down your guard, and a lot of people aren't ok with that. Even relaxed and fun dates, can have a scheduled/ritualistic aspect to them, but just sitting in a place where no one has commitments for at least a few hours is different.
People very much like to talk around things, and it's hard to say "lets just be alone together", rather than "Come over for a drink", or "lets watch a movie", we tend to talk in code.