STAR TREC takes on reproducibility, standardization, and better science

An initiative at Frederick National Laboratory for Cancer Research (FNL) is confronting what’s been dubbed the scientific “reproducibility crisis.” 

The effort, called STAR TREC (the Standards References and Training Research Committee), seeks to help the scientific community address reproducibility issues and aims to spur dialogue about better standards in science.

Several articles and studies in recent years have indicated that 50% or more of published studies can’t be reproduced. A scientist should be able to conduct the same experiment using the same steps as a colleague and both arrive at the same result. Reproducibility is required for independently validating scientific findings. 

Poorly documented methods and inconsistent standards have been cited as major culprits. Better standards, training, and documentation can ameliorate the problem, said FNL Chief Science Officer Leonard Freedman, Ph.D.

STAR TREC is a touchstone where scientists can find tools, explore standardization topics, and discover launching points for their own solutions. This isn’t a standard-setting organization and instead focuses on elevating resources for researchers.  

The effort’s categories span seven disciplines, ranging from data science to serology, each including resources that let researchers integrate standardization and thorough methods into their work. 

“It’s important to serve the research community and provide resources so that people who aren't as connected to the standards world have at least a first place to go to find those resources if they're trying to learn best practices,” Freedman said. “Rather than just sort of waving the flag, I think it’s important to provide tangible resources to the community.”

 

Digital standards hub 

The STAR TREC web pages currently house white papers and standard operating procedures related to foundational processes in the seven areas. There are also recordings of FNL’s High Fidelity Seminar Series, which focuses on expanding the discussion about reproducibility, standards, and research quality. 

“If you start from the basics and you get the right tools in your hands, it might take a little bit longer because you have to validate your tools, but at the end, the outcomes should be better,” said Simona Colantonio, Ph.D., director of FNL’s Antibody Characterization Laboratory.  

Colantonio and her colleagues are among the scientists who founded STAR TREC to produce, validate, and share these tools with the community. 

Part of the effort is designed to encourage practices that add value to research. With FNL producing the tools, the hope is this will ease the burden on others and save time and effort. 

“Seeing all of these young investigators try and set up core facilities and suffer through all the same problems that we did is really very inefficient, right?” said Dom Esposito, Ph.D., a STAR TREC co-founder and the director of FNL’s Protein Expression Laboratory. “It makes no sense to be doing that. It would make much more sense to have them be able to go to a place like STAR TREC, find protocols, and contact people.”  

Growing the available resources 

Members involved with the initiative say the tools currently available through STAR TREC are only the beginning. Scientists at FNL plan to expand the repertoire with additional processes, training materials, and opportunities to engage in conversations about reproducibility. They also see room for STAR TREC to propose solutions to standardization challenges. 

“There's just an immense amount of information we can impart to the to the community, and to me, that's part of the goal that a national lab should have,” Esposito said. 

STAR TREC is spearheaded by FNL’s Science and Technology Group, headed by Freedman. Each of the seven disciplines is overseen by one or more senior-level scientists who are leaders in their respective fields. 

“It's remarkable how many areas we've covered within the membership of STAR TREC, which makes me optimistic about the initiative,” Freedman said. 

STAR TREC grew from grassroots conversations between FNL scientists who wanted to help groups that might be struggling with standardization and reproducibility. Freedman, a long-time proponent of reproducibility, first became aware of this interest at FNL when he visited to give a lecture on standardization years before joining FNL as chief science officer. 

“They were already believers. I was preaching to the choir,” he recalled. 

The golden opportunity manifested when Freedman joined FNL in 2018. After a handful of conversations, he marshaled like-minded scientists into a working group, and the initiative took off. 

Now, STAR TREC is a characteristic part of FNL’s mission to serve as a national resource and perform and promote better science. 

“If we trust the science, we can address the problems better,” Colantonio said. “Better science will allow us to help … more people that are suffering.”