The Decomposition of Rotten Tomatoes:The most overrated metric in movies is erratic, reductive, and easily hacked — and yet has Hollywood in its grip.

[bnew]

The Decomposition of Rotten Tomatoes

The most overrated metric in entertainment is erratic, reductive, and easily hacked — and yet has Hollywood in its grip.

www.vulture.com

SCREEN TIME SEPT. 6, 2023

The Decomposition of Rotten Tomatoes​

The most overrated metric in movies is erratic, reductive, and easily hacked — and yet has Hollywood in its grip.​

By Lane Brown With reporting by Luke Winkie

Photo: Bobby Doherty

This article was featured in One Great Story, New York’s reading recommendation newsletter. Sign up here to get it nightly.

In 2018, a movie-publicity company called Bunker 15 took on a new project: Ophelia, a feminist retelling of Hamlet starring Daisy Ridley. Critics who had seen early screenings had published 13 reviews, seven of them negative, which translated to a score of 46 percent on the all-important aggregation site Rotten Tomatoes — a disappointing outcome for a film with prestige aspirations and no domestic distributor.

But just because the “Tomatometer” says a title is “rotten” — scoring below 60 percent — it doesn’t need to stay that way. Bunker 15 went to work. While most film-PR companies aim to get the attention of critics from top publications, Bunker 15 takes a more bottom-up approach, recruiting obscure, often self-published critics who are nevertheless part of the pool tracked by Rotten Tomatoes. In another break from standard practice, several critics say, Bunker 15 pays them $50 or more for each review. (These payments are not typically disclosed, and Rotten Tomatoes says it prohibits “reviewing based on a financial incentive.”)

In October of that year, an employee of the company emailed a prospective reviewer about Ophelia: “It’s a Sundance film and the feeling is that it’s been treated a bit harshly by some critics (I’m sure sky-high expectations were the culprit) so the teams involved feel like it would benefit from more input from different critics.”
“More input from different critics” is not very subtle code, and the prospective critic wrote back to ask what would happen if he hated the film. The Bunker 15 employee replied that of course journalists are free to write whatever they like but that “super nice ones (and there are more critics like this than I expected)” often agreed not to publish bad reviews on their usual websites but to instead quarantine them on “a smaller blog that RT never sees. I think it’s a very cool thing to do.” If done right, the trick would help ensure that Rotten Tomatoes logged positive reviews but not negative ones.

Between October 2018 and January 2019, Rotten Tomatoes added eight reviews to Ophelia’s score. Seven were favorable, and most came from critics who have reviewed at least one other Bunker 15 movie. The writer of a negative review says that Bunker 15 lobbied them to change it; if the critic wanted to “give it a (barely) overall positive then I do know the editors at Rotten Tomatoes and can get it switched,” a Bunker 15 employee wrote. I also discovered another negative review of Ophelia from this period that was not counted by Rotten Tomatoes, by a writer whose positive reviews of other Bunker 15 films have been recorded by the aggregator. Ophelia climbed the Tomatometer to 62 percent, flipping from rotten to “fresh.” The next month, the distributor IFC Films announced that it had acquired Ophelia for release in the U.S.
Ophelia’s production company, Covert Media, didn’t return requests for comment. Bunker 15’s founder, Daniel Harlow, says, “Wow, you are really reaching there,” and disagrees with the suggestion that his company buys reviews to skew Rotten Tomatoes: “We have thousands of writers in our distribution list. A small handful have set up a specific system where filmmakers can sponsor or pay to have them review a film.” Noted.

The Ophelia affair is a useful microcosm for understanding how Rotten Tomatoes, which turned 25 in August, has come to function. The site was conceived in the early days of the web as a Hot or Not for movies. Now, it can make or break them — with implications for how films are perceived, released, marketed, and possibly even green-lit. The Tomatometer may be the most important metric in entertainment, yet it’s also erratic, reductive, and easily hacked.
“The studios didn’t invent Rotten Tomatoes, and most of them don’t like it,” says the filmmaker Paul Schrader. “But the system is broken. Audiences are dumber. Normal people don’t go through reviews like they used to. Rotten Tomatoes is something the studios can game. So they do.”

In a recent interview, Quentin Tarantino, whose next film is reportedly called The Movie Critic, admitted that he no longer reads critics’ work. “Today, I don’t know anyone,” he said (in a translation of his remarks, first published in French). “I’m told, ‘Manohla Dargis, she’s excellent.’ But when I ask what are the three movies she loved and the three she hated in the last few years, no one can answer me. Because they don’t care!”

This is probably because Rotten Tomatoes — with help from Yelp, Goodreads, and countless other review aggregators — has desensitized us to the opinions of individual critics. Once upon a time, Gene Siskel and Roger Ebert turned the no-budget documentary Hoop Dreams into a phenomenon using only their thumbs. But critical power like that has been replaced by the collective voice of the masses. A third of U.S. adults say they check Rotten Tomatoes before going to the multiplex, and while movie ads used to tout the blurbage of Jeffrey Lyons and Peter Travers, now they’re more likely to boast that a film has been “Certified Fresh.”

 

[bnew]

To filmmakers across the taste spectrum, Rotten Tomatoes is a scourge. Martin Scorsese says it reduces the director “to a content manufacturer and the viewer to an unadventurous consumer.” Brett Ratner has called it “the destruction of our business.” But insiders acknowledge that it has become a crucial arbiter. Publicists say their jobs revolve around the site. “In the last ten years,” says one, “it’s become much more important as so many of the most trusted critics have retired without replacements.” Studios are so scared of what the Tomatometer might say that some work with a company called Screen Engine/ASI, which attempts to forecast scores. (“According to the studios, the predictions are very close,” says another publicist. I’ll refer to these informers, who asked for anonymity to speak candidly, as Publicists Nos. 1 and 2.) An indie-distribution executive says, “I put in our original business plan that we should not do films that score less than 80. Rotten Tomatoes is the only public stamp of approval that says, ‘This is of immense quality, and all critics agree.’”

But despite Rotten Tomatoes’ reputed importance, it’s worth a reminder: Its math stinks. Scores are calculated by classifying each review as either positive or negative and then dividing the number of positives by the total. That’s the whole formula. Every review carries the same weight whether it runs in a major newspaper or a Substack with a dozen subscribers.

If a review straddles positive and negative, too bad. “I read some reviews of my own films where the writer might say that he doesn’t think that I pull something off, but, boy, is it interesting in the way that I don’t pull it off,” says Schrader, a former critic. “To me, that’s a good review, but it would count as negative on Rotten Tomatoes.”

There’s also no accounting for enthusiasm — no attempt to distinguish between extremely and slightly positive (or negative) reviews. That means a film can score a perfect 100 with just passing grades. “In the old days, if an independent film got all three-star reviews, that was like the kiss of death,” says Publicist No. 2. “But with Rotten Tomatoes, if you get all three-star reviews, it’s fantastic.”

Another problem — and where the trickery often begins — is that Rotten Tomatoes scores are posted after a movie receives only a handful of reviews, sometimes as few as five, even if those reviews may be an unrepresentative sample. This is sort of like a cable-news network declaring an Election Night winner after a single county reports its results. But studios see it as a feature, since, with a little elbow grease, they can sometimes fool people into believing a movie is better than it is.

Here’s how. When a studio is prepping the release of a new title, it will screen the film for critics in advance. It’s a film publicist’s job to organize these screenings and invite the writers they think will respond most positively. Then that publicist will set the movie’s review embargo in part so that its initial Tomatometer score is as high as possible at the moment when it can have maximal benefits for word of mouth and early ticket sales.

Granted, that is not rocket science or even particularly new. But the strategy can be surprisingly effective on tentpole releases, for which studios can leverage the growing universe of fan-run websites, whose critics are generally more admiring of comic-book movies than those who write for mainstream outlets. (No offense to comicbookmovie.com.) For example, in February, the Tomatometer score for Ant-Man and the Wasp: Quantumania debuted at 79 percent based on its first batch of reviews. Days later, after more critics had weighed in, its rating sank into the 40s. But the gambit may have worked. Quantumania had the best opening weekend of any movie in the Ant-Man series, at $106 million. In its second weekend, with its rottenness more firmly established, the film’s grosses slid 69 percent, the steepest drop-off in Marvel history.

In studios’ defense, Rotten Tomatoes’ hastiness in computing its scores has made it practically necessary to cork one’s bat. In a strategic blunder in May, Disney held the first screening of Indiana Jones and the Dial of Destiny at Cannes, the world’s snootiest film festival, from which the first 12 reviews begot an initial score of 33 percent. “What they should’ve done,” says Publicist No. 1, “was have simultaneous screenings in the States for critics who might’ve been more friendly.” A month and a half later, Dial of Destiny bombed at the box office even though friendly critics eventually lifted its rating to 69 percent. “They had a low Rotten Tomatoes score just sitting out there for six weeks before release, and that was deadly,” says a third publicist.

For smaller movies, the opposite is more common at film festivals, where critics tend to get swept up in the glamour (or maybe just the jet lag) and give kinder reviews than their peers back home. “It happens all the time,” says the indie-distribution exec. “A movie will come out of a festival with a 90-plus -Rotten Tomatoes score and then, boom, when it hits the marketplace, it goes down to 60 percent.” At the Venice Film Festival last September, critics raved about The Whale with Brendan Fraser and Netflix’s Marilyn Monroe biopic, Blonde, sending the two films’ Tomatometer scores to 84 and 86 percent, respectively. Later, back on dry land, sanity prevailed as other critics downgraded those ratings to 64 and 42.

Naturally, studios have learned to exploit this dynamic. Publicist No. 1 recalls working on a 2022 title that premiered to acclaim at a festival a few months before its release: “I wanted to screen it more widely, but the movie had a 100 and the studio didn’t want to damage that because they wanted to use the ‘100 percent’ graphic in their marketing. I said, ‘Why don’t we get a couple more reviews?,’ and they were like, ‘We just want the 100.’ ” The film won an Oscar.

All of this would be one thing if Rotten Tomatoes were merely an innocent relic from Web 1.0 being preyed upon by Hollywood sharks. But the site has come a long way from its founding, in 1998, by UC Berkeley grads, one of whom wanted a place to catalogue reviews of Jackie Chan movies. Rotten Tomatoes outlasted the dot-com bubble and was passed from one buyer to another, most recently in 2016. That year, Warner Bros. sold most of it to Fandango, which shares a parent company with Universal Pictures. If it sounds like a conflict of interest for a movie-review aggregator to be owned by two companies that make movies and another that sells tickets to them, it probably is.

 

[bnew]

Before the acquisition, Fandango had its own five-star rating scale on its app and website under which it was almost impossible for a movie to receive fewer than three stars. Since then, even the ostensibly well-intentioned changes it has made to Rotten Tomatoes have seemed to produce score-boosting side effects.

Rotten Tomatoes allows users to rate movies alongside critics, and three years after the Fandango deal, it changed the way these “audience scores” were calculated. Misogynist trolls had hijacked the platform, coordinating to tank women-led movies like Captain Marvel before they opened. As a fix, for users’ reviews to count, they would need to verify that they bought tickets — which they could do most easily by purchasing them via Fandango. Under the new rules, audience scores for tentpole movies have often gotten an early lift since most of the first-weekend crowds are diehards who buy tickets in advance. (In June, ads for The Flash bragged about an audience score of 95 percent — “as of 6/14/23,” which was the Wednesday that showtimes began in international markets such as Belgium and Finland but two days before the film’s U.S. release. Today, that score is 83.)

A bigger change came in 2018 when Rotten Tomatoes loosened the restrictions on whose reviews could be indexed. Once, the site had required its contributors to write for publications with substantial web traffic or print circulations. Now, more freelance and self-publishing critics have been allowed to join along with some who review movies via YouTube or podcasts.

The move has been widely characterized as a response to long-standing complaints over a lack of gender and racial diversity on the site and in criticism at large. A 2017 study found that 82 percent of Rotten Tomatoes’ reviews of the highest-grossing movies of that year had been written by white critics and 78 percent by men. With its more relaxed criteria, Rotten Tomatoes gave the “critical conversation a hard push in the direction of inclusion,” declared the New York Times.

Rotten Tomatoes says that more than 1,000 new critics have become “Tomatometer-approved” since 2018, bringing the site’s total to about 3,500. Of those new members, the company says, 50 percent are women and 24 percent are people of color. (Rotten Tomatoes also says that with individuals who identify as LGBTQ+ or say they have a disability factored in, 66 percent of the new critics come from underrepresented groups.) Every bit helps, of course, and I wouldn’t presume to argue with a company whose whole business is calculating percentages. But I might quibble that adding 500 women and another 500 men, three-quarters of them white, to an already overwhelmingly male and white group of around 2,500 does not seem like it would radically alter the imbalances that precipitated the original criticism.

But the change helped with another issue. In 2017, a string of bad movies including Baywatch (Tomatometer score: 17 percent) and Pirates of the Caribbean: Dead Men Tell No Tales (30 percent) flopped in theaters. Studios blamed Rotten Tomatoes. “The critic aggregation site increasingly is slowing down the potential business of popcorn movies,” reported Deadline. “Many of those in the industry severely question how Rotten Tomatoes computes its ratings, and the fact that these scores run on Fandango (which owns RT) is an even bigger problem.”

Could the allegedly more inclusive Rotten Tomatoes have simply expanded its ranks in hopes that the new critics would be nicer to the IP-driven event movies that Hollywood now mostly depends on? Intentional or not, this appears to be what happened. According to a study by Global News, in 2016, the average Tomatometer score for all wide releases was in the rotten low 50s. By 2021, that average had climbed to a fresh 60 percent.

The benefits have not been universally distributed. Some whom I spoke with complained that Rotten Tomatoes’ larger pool has been tougher on art-house movies. Publicist No. 2 worked on an indie director’s recent drama “that got rave reviews from all the highbrow critics, including a great Times review. And yet it was their lowest Rotten Tomatoes score ever. The movies that need high scores most are often more challenging and may not appeal to the whole gamut of Rotten Tomatoes reviewers.”

Maybe that indie director should’ve hired Bunker 15. Rotten Tomatoes’ new membership rules might have enabled the publicity company’s M.O. by providing a wider supply of critics receptive to its pitch, which seems to have become more explicit over time. (“I would like to know if you don’t post negative reviews on Rotten Tomatoes,” a Bunker 15 employee wrote to one critic in August 2022.)

 

[bnew]

Bunker 15’s main business appears to be small films released to VOD with little other promotion; it often helps them meet the five-review threshold required to receive a Tomatometer score. The company’s website mentions micro-indies such as Cold November, Tulsa, and Busman’s Holiday, which have only a smattering of reviews each. But Bunker 15 has worked on medium-size titles, too. According to critics who have transacted with the company, these include 2022’s Wildflower with Kiernan Shipka and Alexandra Daddario, 2023’s Burt Reynolds: The Last Interview, and Bruce Willis’s Gasoline Alley, whose 2022 release was overshadowed by news that Willis had been diagnosed with aphasia and may not have been aware he was still making movies. (I found negative reviews of several 2023 movies, including one of the above, on a Bunker 15–affiliated site, where, unlike their author’s other reviews, they were apparently hidden from Rotten Tomatoes.)

After I asked Rotten Tomatoes about Bunker 15, it delisted a number of the company’s movies from its website and sent a warning to writers who reviewed them. In a statement, Rotten Tomatoes wrote, “We take the integrity of our scores seriously and do not tolerate any attempts to manipulate them. We have a dedicated team who monitors our platforms regularly and thoroughly investigates and resolves any suspicious activity.”

And yet manipulation still happens. The question might be, Is it making a difference where it counts? Attempts to evince a relationship between movies’ Tomatometer scores and their financial success have yielded conflicting results. A 2017 study by the director of USC’s Data & Analytics Project concluded that “Rotten Tomatoes scores have never played a very big role in driving box office performance, either positively or negatively.” In 2020, an investigation by the Ringer found that Tomatometer scores do correlate with box-office returns, especially for comedies and horror films, but the authors admit that the pandemic may have scrambled moviegoing habits in ways that data may not fully account for yet.

What this suggests is that viewers may have developed their own formulas for choosing movies, in which Tomatometer scores are just one important variable. “If there was a new film by, I don’t know, Klaus Von Boringstein,” says Schrader, “and he had a three-hour drama about a housewife in the Middle Ages, do you think people would go see it because it had a 90 percent on Rotten Tomatoes? No. But if it were a movie about a serial killer in the wilds of Alaska and it had a 50 percent? They might check that out.” Maybe they’d have better luck if they read the reviews.

 

[Tflasha]

Marvel used it as warfare on DC films

 

[AnonymityX1000]

Marvel used it as warfare on DC films

Why didn't DC films strike back? They are attached to a multi-billion corporation to.

 

[re'up]

One of my friends who works in the industry was telling me a few years ago how everyone in the industry hated RT

and I was like why? I use it. and he says how do you use it?

uhhh go on after the movie and read all the reviews

and he says yeah that's not what fukking anyone does