Borderlands review bomb triggers Steam’s “off topic” fix [Updated]

[Update 10:30 p.m.: In a statement sent to Ars Technica just now, Valve’s Doug Lombardi says that “after a review of current review activity on Borderlands titles, the decision was made to tag the franchise for off-topic reviews on Steam, effective immediately. As a result, user reviews submitted while the titles are tagged will not count towards the games’ Review Scores. User reviews written during the tagged period will still be accessible and users can choose to include these reviews in the Review Score by changing their preferences.”

Steam reviews for Borderlands franchise games since April 2 are now shown in a histogram view with a large asterisk and the warning “Period(s) of off-topic review activity detected. Excluded from the Review Score (by default).” The “Recent Reviews “summaries for those games have also flipped from “Mixed” to “Very” or “Mostly Positive” as a result of the move.]

Original Story (12:21 p.m.)

Last month, Valve announced a new policy intended to curb the problem of “off-topic review bombs” on Steam, wherein a horde of players post negative reviews of a game for reasons other than the game’s actual content. This month, that policy is so far failing to correct for some clear off-topic review bombs on Gearbox’s Borderlands games.

In the six months leading up to March 31, Borderlands 2 received a total of 412 negative reviews on Steam—a paltry 4.5 percent of all the reviews posted in that period. In the few days since April 1, by contrast, the game has received a flood of 2,156 negative reviews, or nearly 62 percent of reviews during that time. You can see similar recent review patterns on the Steam pages for the original Borderlands, The Pre-Sequel, and various series DLC packages as well.

Dive into the text of those recent negative reviews, and you’ll find practically no one criticizing the games themselves. Instead, you’ll find hundreds of people expressing their displeasure with Gearbox’s recently announced decision to
release the upcoming Borderlands 3 exclusively on the Epic Games Store for a six-month period.

One characteristic reviewer with over 300 hours spent on Borderlands 2 writes that the game is “one of the few looter shooters that actually keeps me engaged.” But the reviewer then justifies their “thumbs down” rating “[because of] Epic Games exclusivity [for Borderlands 3], please don’t play into Epic’s game.” Plenty of other reviews use copy/pasted ASCII art of double middle-fingers to express their general distaste for Epic and Gearbox’s decision.

This would seem to be a textbook case of the kind of off-topic review bombing Valve is trying to combat. The company defined that kind of review bomb
back in 2017 as “issue[s] players are concerned about… outside the game itself.” Valve
restated that definition in March as a situation “where players post a large number of reviews in a short period of time, aimed at lowering the Review Score of a game [and] where the focus of those reviews is on a topic that we consider unrelated to the likelihood that future purchasers will be happy if they buy the game.”

This isn’t the first time Steam users have taken aim at companies that are making the move to Epic exclusivity. Games in 4A Games’ Metro series faced similar off-topic review bombs in the months after sequel Metro Exodus migrated away from Steam. Plenty of other Steam titles have faced review bombs for issues totally unrelated to the games’ content itself. But those games’ review bombs predate Valve’s announcement of a potential fix for the problem on March 15, making Borderlands the new system’s first real test.

We’re waiting

Valve says it has an automated tool that “identifies any anomalous review activity on all games on Steam in as close to real-time as possible.” But the final decision to remove such potential review bombs from the aggregate review score is up to “a team of people at Valve” who investigate the machine-identified anomalies.

If a review bomb is manually identified by Valve, the developer is notified and “the reviews within that time period will then be removed from the Review Score calculation,” Valve wrote last month. While such reviews will continue to exist and be visible, they will no longer factor into the top-line aggregate summaries (i.e. “Mostly positive”).

As of this writing, the apparent Borderlands review bombing has been going on for over 48 hours in a distinct pattern that should have been quickly identifiable. So far, though, there has been no apparent action by Valve. If there’s pending action in the works, Valve has failed to identify it in response to multiple unanswered requests for comment from Ars Technica.

(It’s also possible that Valve doesn’t see reviews focused on Borderlands 3‘s move to Epic Games Store as “off topic” to previous Borderlands games on Steam. But that would raise other questions.)

In the grand scheme of things, this isn’t a huge problem for Gearbox (which has sold over 41 million copies of Borderlands 2 alone) or Steam users themselves. Yes, the prominent “Recent Reviews” metrics for Borderlands games on Steam have been flipped from “Very” or “Mostly Positive” to “Mixed” or worse because of these review bombs. But those anomalies are still just a drop in the bucket compared to the tens of thousands of positive reviews the games have received over the years, as listed in their “All Reviews” summaries.

This will almost certainly end up being the kind of “temporary distortion of the Review Score” Valve called out in 2017, forgotten by all but obsessive fans. That said, this is also the very kind of temporary problem Valve said it was now equipped to quickly identify and combat with a combination of new policies and tools. Borderlands‘ Steam situation is the first high-profile test of that new system and, just over two days in, it’s a test Valve is failing.