Does Reddit Have An Astroturfing Problem? What The Data Suggests

TL;DR

Buying for a business?Offer from Amazon

Get business pricing on monitors, keyboards and dev gear

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

An analysis of 51,129 comments across six knife subreddits found that the 5% most brand-focused accounts wrote 11.3% of brand mentions in buying threads, compared with 7.9% in a randomization test. One chef’s-knife brand had a larger concentration, but the author found no evidence establishing that any company paid for recommendations.

Peter Vijeh’s analysis of 51,129 comments from six knife subreddits found that the most brand-focused 5% of frequent commenters contributed 11.3% of brand mentions in buying threads, versus 7.9% in a randomization test. The result shows that recommendations for some knives are concentrated among a small group of accounts; it does not establish that those accounts were paid to post.

Vijeh, who runs the knife-collector site New Knife Day, used software to identify brand mentions and flag posts that appeared to ask what knife to buy. The corpus covered r/knives, r/knifeclub, r/chefknives, r/japaneseknives, r/FixedBladeEdc and r/KnifeSteels. It contained 6,675 posts and 51,129 comments after Vijeh refreshed comment histories for 3,607 posts older than 48 hours. That update mattered because early collection had captured too few replies in buying threads to make the comparison informative.

The author counted 987 accounts with at least 10 comments in the corpus. He ranked them by how often they named brands, with additional weight for repeatedly naming the same brand, and examined the top 5%: 49 accounts. To estimate a chance baseline, he reassigned author identities at random across brand mentions 1,000 times while keeping each mention’s thread and brand intact. The 49 accounts contributed 11.3% of buying-thread mentions, compared with an average expected share of 7.9% under that procedure.

The pattern varied by subreddit and brand. In r/chefknives, the group accounted for 14.9% of buying-thread mentions against 7.7% expected; in r/knifeclub, 12.4% against 7.3%. For one coded chef’s-knife brand, it accounted for 31.2% against 8.0%. Vijeh said that brand also has a large, vocal fan base. The brand identities were withheld, and he cautioned that the per-brand results involve relatively few accounts.

How Concentrated Advice Shapes Buying

People often add “Reddit” to a product search to find discussion from users rather than sales pages. That makes recommendation patterns relevant to shoppers: a small number of frequent voices can account for a sizable share of advice about a particular brand, even when readers encounter many separate comments.

The findings give readers a measurable signal to examine, but not a verdict about intent. Concentration can arise from enthusiastic customers, employees participating on their own, or paid promotion. Vijeh’s analysis supports the narrower conclusion that some accounts recommend certain brands more heavily than a random assignment of authors would predict. It cannot determine whether those recommendations influenced purchases or reflected a coordinated campaign.

That distinction matters for interpreting online advice. A pattern that resembles what paid promotion might produce can also result from ordinary loyalty. Treating the percentage as proof of fraud would go beyond the evidence; ignoring the concentration would also miss a potentially useful clue for consumers evaluating recommendations.

How the Reddit Sample Was Built

Vijeh said he began the project because he uses Reddit discussions when researching chef’s knives and wanted to test whether recommendations in the communities he follows showed measurable concentration. New Knife Day collects knife-related Reddit comments and tracks named brands, models and steels. The author disclosed that he runs the site and that its subject overlaps with the brands counted in the analysis.

To define buying threads, the analysis used a regular expression matching phrases such as “should I buy,” “recommend,” “under $” and “best knife.” It was not a human-verified classifier. The brand detector was a stock GLiNER model; Vijeh said it can miss brands or incorrectly identify model names, and its results have not been checked against a hand-labeled sample of this corpus.

The report also cited online services advertising paid Reddit comments and aged accounts as evidence that a market for such services exists. Those pages do not connect any vendor to the accounts in the knife data. Vijeh described the test as an attempt to measure the concentration such a campaign might leave behind, rather than an investigation that identifies clients or paid posters.

“Public Reddit data can show that recommendations are concentrated, and cannot show why.”

— Peter Vijeh, report author

What the Account Histories Show

Vijeh examined the Reddit histories of 23 accounts associated with brands showing concentration and compared them with 23 other frequent commenters. Eight histories in the first group and seven in the comparison group were hidden, suspended or deleted. Among the readable accounts, the concentration group had a median age of 4.5 years, matching the comparison group. Its comments were spread across 66 subreddits, compared with 47 for the comparison accounts, and just 3% were in the six knife communities, versus 10% for the comparison group.

The author reported that the focused accounts named several knife brands across their histories, and store links were rare in both groups. He said none of roughly two dozen account-feature comparisons showed a difference larger than could often occur when splitting 31 readable accounts into two random groups. Those observations do not prove the accounts were unpaid: aged accounts can be used in campaigns, and eight unavailable histories in the focus group leave a gap.

The estimates also depend on automated labels and a keyword-based definition of buying threads, neither of which Vijeh had validated against hand-reviewed examples. The per-brand figures involve small groups, and testing several brands and communities raises the possibility that some high results occur by chance. The sample covers six knife subreddits, not Reddit as a whole; the report does not establish how common paid promotion is across the platform.

Checks That Could Refine the Findings

Vijeh said the code, anonymized data and charts are available in a public repository, while usernames, comment text, account histories and the key linking coded brands to real names are not published. The next useful checks would be validating the brand detector and buying-thread labels against manually reviewed samples, then repeating the analysis on a larger or different set of communities.

For shoppers, the author suggested checking whether an unfamiliar recommender has discussed other brands, while noting that this cannot identify a paid post made through a credible, aged account. The report does not name any company as a client of Reddit-comment vendors. Whether any of the concentrated recommendations were sponsored remains unresolved.

Key Questions

Did the analysis prove that brands paid Reddit users?

No. It found concentrated recommendations, but did not identify a payment, advertiser or relationship between a vendor and any account in the sample.

What did the 11.3% figure measure?

It was the share of brand mentions in buying threads written by the 49 most brand-focused accounts among 987 authors with at least 10 comments. The comparison was 7.9% under 1,000 random reassignments of authors to mentions.

Why was one brand’s result higher?

For one coded chef’s-knife brand, the 49-account group contributed 31.2% of buying-thread mentions, compared with 8.0% expected in the randomization test. The brand was not named, and its strong fan base could also help explain the concentration.

Can the findings be generalized to all of Reddit?

No. The sample covers six knife-related subreddits, and the report notes that its automated labels and buying-thread definition have not been validated against manually reviewed data.

Source: hn