Scientific papers are often years in the making. For the researchers responsible, the moment of publication is the culmination of an awful lot of work, and cause for both celebration and relief.
But it’s also the opportunity, if only for a little while, to be a celebrity.
In December 2013, Professor Robert Nemiroff of Michigan Technological University found out what that feels like while driving to see relatives for Christmas. He wrote about it for The Winnower.
“I received an unexpected call from a major US news outlet: NBC. I called them back from a rest stop off Interstate 80,” he said. “The editor had seen our manuscript and wanted to interview me.”
“So I gave the interview. And soon after, another. The emails poured in. Now almost every time we stopped, I returned someone’s email or phone request.”
What makes Prof. Nemiroff’s story odd is that his paper – an investigation into whether time travellers have left any traces of themselves on the internet – hadn’t even been published. Well, not in traditional sense of a journal anyway.
He’d put the manuscript on arXiv, an open access service that hosts many papers before they are published.
“Hundreds of manuscripts appear on arXiv daily and it’s understood that many of them are not finished works,” wrote Prof. Nemiroff. “I was surprised that they even knew about the manuscript on arXiv.”
Not only did journalists know about it, so did an awful lot of other people. As of December 2014, the paper had been shared 2,262 times on Twitter – enough to make it the 8th most popular scientific publication of 2014.
“The most unexpected email I received was from a movie producer who wanted to buy the movie rights,” Prof. Nemiroff said. “Neither the producer nor I had ever heard of a scientific manuscript being optioned.”
The huge interest in Prof. Nemiroff’s paper was clearly out of the ordinary. “I have never had a published paper become so popular, not even papers accompanied by a press release,” he said.
This paper had neither press release nor traditional publication, so why did it make such a splash? What is it about some articles that makes them so much more popular than others?
Hunting for answers in altmetrics
The traditional measure of a scientific article’s impact is its citations – the number of times other scientists have referred to it. But citations take time to build up, and say little about an article’s impact outside of the scientific community.
For that, you need altmetrics: alternative measures of a paper’s impact, like the number of news stories written about it, the number of tweets, or the number of posts on Facebook and Google+.
The aptly named London-based company Altmetric records all this information. If you’ve viewed a scientific paper online in the last few years, you might well have crossed paths with their colourful ‘doughnut’ symbols that show an article’s impact online.
Altmetric produce an annual list of the 100 most popular papers in a given year, and by delving into the data underlying the most popular articles of 2014 we can begin to get an idea of what it takes to make a splash with a scientific paper.
Make it medical
The top 100 articles in 2014 were dominated by those related to medicine and human health. 44 per cent of the articles were related to the Medical and Health Sciences; the next largest group, Biological Sciences, makes up another 20 per cent.
So if you want to cause a sensation, it’s a safer bet to study disease than it is to unravel black holes. Perhaps it’s simply harder for ordinary people to relate to, but physicists seem to have a more difficulty selling their papers to the public.
Go big or be more open
When it comes to scientific journals, Nature is the daddy. Every scientist knows it, and a paper in there pretty much guarantees good coverage in global news media and a decent amount of buzz on social media. Not everything can be published in Nature though, and for everything else the data points another path to internet fame: publish in an open access journal.
Open access publication allows people to view the original article in full without having to pay.
If we look at the 10 journals that pop-up most in the top 100, we can see that 60 per cent require a subscription to view more than just an overview.
This doesn’t appear to have any effect on their coverage in professional news media, likely because outlets either have access or – whisper it – just read the press releases.
Where open access publication does seem to make a difference is on social media. Freely available articles punch above their weight on Twitter, getting many more shares than their coverage in mainstream news would suggest.
On Reddit, the trend is even more pronounced. Even the mighty Nature can’t hold on to top spot, falling to fourth behind three open access journals.
It sounds so obvious: if you can read an article for free then it’s more likely to be shared on social media.
And as we’re about to find out, being shared on social media is pretty important.
Forget news coverage – it’s all about Twitter
When calculating a paper’s score to put inside those colourful doughnuts of theirs, Altmetric gives news articles much greater weight than tweets.
One news story is worth eight common-or-garden tweets. If Stephen Fry or Justin Bieber happens to tweet about a scientific paper though, their tweets count double because they’ll reach more people.
The reasoning is that a news article on a respectable website has a bigger impact than your average tweet. Sounds plausible.
Despite this weighting though, an article’s ranking in the top 100 is much more tightly related to the number of tweets it gets than it is to its coverage in traditional media.
The number of tweets about a paper seems to have a far greater bearing on its ranking within the top 100. Any scientist who wants to make the most of their fifteen minutes of fame really needs to crack Twitter if they want to mix it with the big boys.
You might assume that getting plenty of news articles and blog posts written about a paper would be a sure-fire way to do drive a conversation on social media. But the surprising truth is that a large number of news articles does not translate into more social media discussion about the original paper very well at all.
The relationship between posts on social media and news coverage amongst the 100 most popular papers is pretty weak, and in some cases the disparity is enormous.
Take the paper “Does Nursing Assistant Certification Increase Nursing Student’s Confidence Level of Basic Nursing Care When Entering a Nursing Program?” as an extreme example.
It makes the suggestion that a particular training course for nurses gives them more confidence about doing their job.
Hardly stuff to set pulses racing, and journalists and bloggers everywhere agreed – not a single article relating to the paper was published (until now I suppose). Nor was it shared even once on Facebook, Google+ or Sina Weibo, Twitter’s Chinese equivalent.
Yet somehow, this paper was tweeted 2,021 times. That’s enough to make it the 17th most popular paper of the year.
“We’re not entirely sure why this particular article received so much Twitter attention yet so little news coverage,” said Altmetric’s Fran Davies.
“It may be that the institution did not release a press release to go with the article, or that it was talked about in other sources which we don’t – or can’t – track.”
Whatever the reason – and a glance at a lot of identical tweets suggests someone may have been playing the system somewhat – the nursing paper isn’t alone in vastly outperforming expectations on social media.
While papers like “An Earth-sized planet in the habitable zone of a cool star”, which was covered in 172 news and blog posts, flounder on social media (it was tweeted only 150 times), others find it a natural home.
“The survival time of chocolates on hospital wards: covert observational study”, published in the Christmas edition of the British Medical Journal (BMJ) was by far and away the most successful scientific paper on Facebook in 2014, amassing 1,617 posts despite mentions in only seven news articles.
“We were as surprised as anyone about the popularity of our paper, but it goes to show that people enjoy good quality research when it is something that they can relate to,” said author Dr Parag Gajendragadkar.
Dr Gajendragadkar speculated that the fact his article did not benefit from a press release, and therefore received minimal press coverage, may actually have helped it on social media.
“Maybe because it didn’t have great press coverage people shared it more,” he said. “It was more heavily shared than the ‘James Bond alcoholism’ article in the same Christmas edition of the BMJ, even though that got more media coverage.”
“Anyway,” he said, “it proves that chocolate consumption is a universal pleasure!”
Which brings us neatly to the final point.
You don’t have to cure cancer
I’m sure if someone did find a conclusive cure for cancer then that would be quite big news, but as Dr Gajendragadkar’s chocolate stake-out and Prof. Nemiroff’s hunt for time travellers show, it’s not necessarily scientific importance that gets a paper noticed.
That’s not to say it never matters. There are cases where it is the potential importance of the finding that catches the eye.
The paper “Artificial sweeteners induce glucose intolerance by altering the gut microbiota?” for example, published in Nature and occupying 3rd spot on the list, is a serious paper that suggests artificial sweeteners could have harmful effects.
“People were greatly surprised by the findings,” said Professor Eran Segal, one of the paper’s many authors. “Artificial sweeteners are advertised in the official guidelines of the American Diabetes Association as being beneficial for weight loss and blood glucose control.”
Just one place higher on the list (but in many ways at the other end of the spectrum) is the most popular paper on Twitter in 2014: “Variation in Melanism and Female Preference in Proximate but Ecologically Distinct Environments”.
The paper, which looks at sexual preference and skin pigmentation in fish, was tweeted 6,669 times, but covered in only 15 news and blog posts.
Unlike the mysterious nursing paper though, the reason this one took on a life of its own on social media is clear. In an error that has now been corrected, the authors accidentally included the note “should we cite the crappy Gabor paper here?” in the final published version.
— Dave Harris (@davidjayharris) November 10, 2014
The path from hardworking scientist to global fame is not a straightforward one, but one thing is clear: social media plays a big part, independent of news coverage.
Sorry scientists, but it looks like if you want to make a splash with your paper, you’re not going to be able to rely on journalists to do it for you.If you have any desire to look through the data used in this post yourself, be my guest. Let me know if you do anything fun with it.