People are saying fighting over bytches you don't know. How about fighting over a man puttin his hands on you? I don't care how stupid the argument was about if someone puts their hands on me they will receive mine
this is how nikkas get in trouble.
OP said he ain't much of a fighter. Based on that ALONE, he was right to back down after that shove. Better to take a lowercase l by walking away than taking a capital L and getting half-killed for trying to start a fight from that shove.
And what if that other nikka had a gun/knife/other weapon on him? Y'all gotta realize most of these overly aggressive nikkas out here are too bytchmade to fight fair. The other nikka shoved the OP first so he obviously felt confident for some reason, and that reason was most likely either a weapon, having his boys nearby ready to jump in, or knowing that he can whoop some ass on his own (OP didn't mention that the guy looked drunk or looked big/tough; those are the only other possible reasons for the guy being so confident)
A simple shyt like a shove is not enough to fight someone. That ain't causin' no injury, and it sounds like the OP only fell afterwards because he was drunk. You can walk away from that shyt provided that the other nikka doesn't rush up on you afterwards - which doesn't sound like what the guy in this scenario did. Now if the other nikka went up at him after the shove, or he had punched him to begin with, then the OP should have tried to fight back.
Now the OP needs to learn to fight/defend himself properly. Don't mind the "you 36 in a karate class" crew; better late than never.