Transparent and thorough communication of data has the potential to streamline major scientific advances. For Dr. Maryam Zaringhalam, open science practices like these would have transformed her Ph.D. thesis. “While I was at Rockefeller, I was scooped five times,” she said.
As a student in Nina Papavasiliou’s lab, Zaringhalam aimed to develop a method to map the RNA modification pseudouridine throughout the transcriptome. However, she was stopped cold by the simultaneous publication of several similar methods. Had her field been more communicative and forthright about work in progress, she could have redirected her time and energy to other pursuits.
Zaringhalam pivoted her focus to a comparative analysis on these techniques and encountered another frustrating development: the methods were difficult, if not impossible, to reproduce.
Due to the problems she encountered in her field, Zaringhalam developed passions for both transparent science communication and ways to improve reproducibility. She published her Methods paper “Pseudouridylation meets next-generation sequencing” in September 2016 and graduated the following spring with an offer in hand to become an American Association for the Advancement of Science (AAAS) science policy fellowship.
Though many biologists may support the concept of open science, including open access publishing, many are also apprehensive. This contradiction is a product of our academic culture, which tends to assess the worth of data by the journal it is published in, rather than evaluating robustness on a case-by-case basis. In an increasingly competitive field, publishing in Cell, Nature, and Science has become the expectation at elite institutions. Scientists at all career stages want to change these policies and practices, but fighting the system is risky for trainees and untenured professors. Established heads of lab and scientists working in science communication and policy, however, can leverage their influence and security to promote a move towards open science.
As an AAAS Fellow, Zaringhalam specialized in open science and data science policy in biomedical research. “Policy is really exciting for me because I can keep learning, which is the reason why I, and a lot of people, wanted to become scientists,” she said. “I see how that learning applies to how research is ultimately done, in academia and beyond.”
A big part of Zaringhalam’s work focuses on reproducibility and equipping scientists with tools to generate reproducible research. Using electronic laboratory notebooks like Jupyter and version-control software like GitHub, for example, facilitates easy access to data and pipelines necessary to reproduce and repurpose data.
Zaringhalam recently led a workshop in which scientists were asked to reproduce data from several papers, which proved difficult. The group then discussed ideas such as introducing reproducibility as a criterion in the peer review process. “We had some nice discussions and ideas coming out about what needs to change within our culture to create a research environment that’s more collaborative rather than competitive,” Zaringhalam said.
Zaringhalam recently transitioned into a new role as a Data Science and Open Science Specialist at the National Library of Medicine, where she will continue to tackle the reproducibility problem. “This is the first time I’ve had a job that wasn’t a fellowship,” Zaringhalam said. “I will have a lot more opportunity to be thinking long-term about what kind of presence and impact I can have.”
Scientists tend to focus their efforts on communicating positive, exciting results because it is difficult to publish negative results in high-impact journals. Zaringhalam argues that this culture impedes progress. “The publishing space is very competitive, and people don’t necessarily want to read about the things that didn’t go right even though there’s a lot of value in that,” she said.
“We do have this responsibility to show what we’ve done, whether it’s positive, negative, or non-confirmatory,” Zaringhalam said. Cleaning up the data to make it sharable and reusable allows it to be repurposed. Moreover, if an experiment does not work, that is good for the next person to know. “There’s some work to be done to think about how we can change that culture and how we can see negative and non-confirmatory results as being useful,” Zaringhalam said.
Recent developments to ameliorate these issues include open access journals and pre-prints, which allow researchers to publish primary research manuscripts without being subjected to an extensive review process that favors high-impact results.
“You have to have these results published where researchers are already looking if you want them to encounter them,” Zaringhalam said. BioRxiv, a pre-print server, has become increasingly popular, with over 1 million papers downloaded as pre-prints every month, many to later be published in peer-reviewed journals. ASAPbio, headed by biochemist Dr. Jessica K. Polka, is another organization that encourages pre-prints in biology and calls for the publication of peer review to make the publication process more transparent and accountable.
Dr. Harold Varmus, co-recipient of the 1989 Nobel Prize in Physiology or Medicine for his work on retroviral oncogenes, has been influential in shaping science policy and promoting open science. He has served as the Director of the National Institutes of Health and President and CEO of Memorial Sloan Kettering Cancer Center, and currently heads a lab as the Lewis Thomas University Professor of Medicine at Weill Cornell. Varmus co-founded Public Library of Science (PLoS), and headed several successful efforts to make papers publicly available. “Science should be shared,” he said.
Varmus, Zaringhalam, and many others share the belief that the pressure to publish in prestigious journals undermines the accessibility of science. “In the biomedical world, we are not very open,” Varmus said. “We all work hard, but our values have been distorted.”
Still, a rigorous peer review process is important. “If you believe that peer review does something, you can’t be satisfied by only the preprint form,” Varmus said. “Work needs to be subjected to stiff statistical analysis and validation that people have been lax about.” The goal is to judge data more on its robustness and reproducibility, and less on bold and flashy claims.
Although most scientists may agree with this in principle, trainees and untenured investigators hesitate to sacrifice prestige and potential career advancement. Refraining from the opportunity to publish in elite journals might not result in an impact big enough to be worth the risk. It may be that real change needs to come from people with career and financial security. “The government and other funders have the real power here,” Varmus said.
For trainees and junior faculty, Zaringhalam recommends using electronic laboratory notebooks to record protocols; she also emphasizes the importance of designing clear presentations. These practices gear lab culture towards reproducibility and collaboration. “Even if it’s on a small scale, it still matters,” she said. “Science is something that fundamentally builds on itself.”