Click here for Part One of Where to Find Scientific Research Papers (and How to Know if They’re Legit).
Yesterday, I wrote about predatory journals. I suppose I shouldn’t have been surprised to learn that there are shitty people in the research world who get off on exploiting academics and undermining science, but I was. Blame my mother. She raised me in a world where all people, on some level, are good. I never quite bought it, but I also didn’t learn to look at everything and everyone with skepticism. I tended to assume that people were just doing the best they can. They may be severely annoying in the process, but ultimately it was all with good intention.
The internet has shattered that illusion. People are fucked.
And so the burden falls on to the individual to see through the bullshit. Historically, we’re not great at that, but when it comes to sussing out whether or not a research paper is legitimate, there are a few quick and easy ways to verify your science.
Check the Citations
Google Scholar is one of my favorite ways to source research, but because Google Scholar is a search engine and not a curated database, articles published in known predatory journals may pop up in your search results.
The quickest way to determine if the article is legit is to check the “Cited by” number at the bottom of the search. If an article has multiple citations, it means other researchers are referring to the research in their own articles, which indicates legitimacy. It’s rare that articles are cited hundreds of thousands of times like Eugene Paykel’s excellent study in the photo above. (Paykel’s study is the research equivalent of a New York Times bestselling book.) According to my smarty-pants academic boyfriend Justin, even mid-single digits is enough to assume the research isn’t bunk.
While citations are a great place to start, they benefit from time in the system. Paykel’s article has been around since 1976, which means it has nearly half a century of research built upon it. New research won’t come with shiny citations, so you need to look at the journal it’s published in to see if it’s legitimate.
Academic journals are ranked for impact and quality. Think of it like the college system. Harvard isn’t the same as Iowa State, but that doesn’t mean that Iowa State isn’t capable of producing damn good citizens (and we all know question marks who graduated from top tier universities.) The top journals produce great work, but there is still plenty of meaningful work to be found in smaller journals.
Find journal rankings by googling the name of the journal and the word “ranking.” The Scimago Journal & Country Rank (SJR) should be the first result, and that will take you to a list with the journal in question buried somewhere in there. The rank is determined by the H-Index, the details of which I don’t entirely understand. The H-Index is determined by the number of publications and citations, and higher H-Index indicated a higher ranking. However, the H-Index is not standardized across subject areas, so you can’t cross-compare.
For our purposes, the H-Index doesn’ matter too much. In Justin’s words, “A low ranking isn’t necessarily a problem. No ranking is a problem.”
Crosscheck Beall’s List
If the journal article doesn’t appear on the SJR, your predatory journal spidey sense should go off. Cross-reference the journal against Beall’s List, an archive of predatory journals created by librarian Jeffrey Beall. The sheer number of journals listed on Beall’s List is astounding, and it’s easy to see how naive readers could be duped.