That was when the commercial capabilities of music was still being discovered, and most people we can references to likely got albums. I agree with you because most current (rap) artists are not capable of making captivating albums.
We also have to acknowledge the unique quality of hip-hop where performer authenticity and perspective is more important, via lyrics, than other genre from back then. That lends to having more to say, not less.
I don't mind short songs but as someone who has made songs, I can tell that this wave isn't about those songs being well-constructed or brief for the sake of storytelling. It generally feels lazy.