It fails because Woodbury doesn't matter.
In the comics, Woodbury wasn't a town. I mean, it was literally, but in the comics Woodbury was just an alliteration for The Governor. Woodbury was that town where the Governor lived and it was NEVER focused on. Woodbury was just the enemy and we never had to deal with any of its stupid residents, save for a few cronies here and there. We saw the Woodbury army attacking the prison twice, we saw the zombie arena, but that was it. There was practically no Woodbury in the comics and that worked, because you always got the feeling the Prison was fighting this mythical enemy. The group never knew what they were up against, they didn't know how big Woodbury was, the people in it. Robert Kirkman keeping Woodbury away in the dark made things so much more exciting, because the audience didn't know what the prison was up against. It was just like, "well, Woodbury could be 10 people, or it could be 100 people, or 1,000 people." We never knew what Woodbury was hiding, and that made it all the more exciting and intriguing.
And it also gave Kirkman a chance to finally TRULY develop ALL the characters and bring them to their true selves.
Woodbury doesn't work on the show because the show has tried to humanize Woodbury and I'm sorry, I don't care about Woodbury's humanization. I don't care about Milton, Karen, Haley, Tom (or whatever the asthma kid's name was). These people are just wasting my time. I feel like the writers just use them as filler. Don't give me Milton if you're not going to develop him and he's just gonna say some lines here and there.
It's kind of ironic that the ONLY Comic Woodbury resident that actually mattered and became important to the prison, Alice, was never introduced to the show. Having Alice would be SO MUCH better than having to deal with the constant stream of Woodbury residents that no one gives a shyt about.
Woodbury feels like a cop out, like Mazzara just didn't have enough material for the prison, so he needed to introduce this other universe to fill in the gaps. I can just picture the writer's room:
Writer 1: Oh, shyt, guys. Why is Merle at the prison? Why would the prison accept Merle? I feel like we should explore this.
Writer 2: We still have 18 minutes to cover.
Mazzara: Hmmm.... Well, Merle needs to be at the prison because of reasons.
Writer 1: .... Glen, it just doesn't make any sense that--
Mazzara: Just cut to a Woodbury resident.
Writer 3: What?
Writer 2: Who?
Mazzara: That one... with the dark hair.
Writer 1: ... Who are we talking about?
Writer 3: You mean Penny?
Writer 4: Dude, Penny's a walker.
Writer 2: Glen, we killed Haley and we can't give Martinez any humanity because character depth is just not our thing, remember?
Mazzara: Hmm. Okay, easy. Just give Karen some lines.
Writer 1: ...What?
Writer 2: Who the fukk is Karen?
Writer 3: Don't you mean Haley?
Writer 1: No, we just killed Haley.
Writer 4: Who the fukk is Haley? We have a character named Haley?
Mazzara: Just give some lines to Karen.
Writer 2: Glen... who's Karen?
Mazzara: She's in Woodbury, okay? She's just like... a girl in Woodbury. Do I have to do all the thinking?
Writer 1: ...
Writer 2: ...
Writer 3: ...
Writer 4: ...Glen, can you just explain what it is you want us to do?
Mazzara: LOL look guys. I got a rubber pencil going on here.