[ad_1]
Scientific research is the foundation of progress, creating innovations such as new treatments for melanoma and providing insights into actions that can guide policy in response to events like the COVID-19 pandemic. This potential for real-world impact is best realized when research is rigorous, reliable, and requires external confirmation. However, the evidence is too often unreproducible and unreliable, making it difficult for policy makers, practitioners, researchers, and the public to It suggests that we are not fully exploiting the potential of science to improve outcomes.
To build on existing federal efforts to support scientific rigor and integrity, funding agencies will encourage researchers to engage in trust-enhancing practices that are currently underrepresented in the scientific enterprise. new programs should be researched and piloted.
detail
Federal science agencies ensure the rigor and reproducibility of scientific research to accelerate discovery and innovation, inform evidence-based policymaking and decision-making, and foster public trust in science. I have been working on this for many years. In the past decade alone, policymakers have produced three National Academies reports, a Government Accountability Office (GAO) study, and a National Science and Technology Council (NSTC) report examining these and related issues. I have been asked to do so. Unfortunately, flawed, unreliable, and potentially fraudulent research continues to impact the scientific enterprise.
The U.S. government and scientific community have high hopes that open science practices, such as sharing research code and data, pre-registering research protocols, and supporting independent replication efforts, will help ensure the rigor and reproducibility of scientific research. I am becoming more and more aware of what I have. In response, many scientific institutions in the United States have launched efforts to encourage these practices in recent decades. Perhaps the best-known example is the creation of clinicaltrials.gov and the requirement for preregistration of publicly and privately funded clinical trials (in 2000 and 2007, respectively), which in some cases This has led to fewer clinical trials reporting meaningful results.
Recent federal actions have focused on promoting the sharing of research data and materials and supporting open science-related education. These efforts seek to build a consensus field given the diversity of the scientific ecosystem and the resulting difficulties in setting appropriate and generalizable standards for methodological rigor. However, further steps are warranted. A number of important practices that could strengthen government efforts to increase the rigor and reproducibility of scientific practices include pre-registration of confirmatory studies and the identification of impactful and decision-relevant discoveries. Reproductions are still far too rare. A key challenge is that incentives to engage in these practices are weak. Researchers perceive that they are expensive or undervalued given the rewards for experts generated by current funding and promotion systems, and new “discoveries” often cannot be replicated. We encourage exploratory exploration. Without structural changes to these incentives, their uptake is likely to remain limited.
Recommendations
To maximize government investments in education and infrastructure for open science, federal funding agencies launch pilot initiatives to encourage researchers to pursue transparency, rigor, and public interest-oriented practices We recommend encouraging and rewarding. Such efforts increase the quality and effectiveness of federally funded research at relatively low cost, promote alignment of priorities and incentive structures with other scientific stakeholders, and help scientists and scientists benefit society. It could help us better deliver on the promise of research that we bring to the table. Specifically, NIH and NSF should:
Establish a specialized office to begin addressing rigor and reproducibility
- We use the National Institute of Neurological Disorders and Stroke’s Office of Research and Quality (ORQ) as a model. A similar ORQ would encourage the adoption of under-incentivized practices, both through internal initiatives and external funding programs.
- To ensure that programs are aligned with the priorities of a single specialty, offices should be established within individual NIH institutes and within each NSF division.
Incorporate transparent and reliable evaluation of research methods into learning assignments
- Include questions to better understand existing practices.For example, “How often and effectively?” [agency]“Funded researchers across disciplines engage in open science practices such as pre-registration, null result publication, and external replication, but these practices have implications for future funding and research output. Is that how they are related?”
- Include questions to inform new policies and initiatives. “What steps are possible? [agency] What efforts are needed to encourage broader uptake of open science practices? And which funding programs, application questions, criteria, evaluation models, etc. are most effective?”
- To answer these questions, we will seek feedback from applicants, reviewers, and program directors, and partner with external “Science of Management” researchers to design rigorous prospective and retrospective studies. Use the information gained to develop new processes to encourage and reward open science practices in funded research.
Extend support for third-party replication
- Allocate a consistent percentage of funding to support independent replication of important discoveries using mechanisms other than grants, such as awards, cooperative agreements, and contracts. The high value placed on scientific novelty inhibits such research that could provide valuable information for treatment, policy, regulatory approval, or future scientific investigation. . Use a combination of agency prioritization and public requests for information to identify topics where additional corroborating or contradictory evidence could have important societal and/or scientific benefits is needed.
- NSF collaborates with independent third parties, such as replication laboratories, to improve the rigor of research and to inform future funding. A pilot study should be conducted to assess the usefulness of commissioning a replication study.
To learn more about the importance of making science open and read the rest of the published memo, visit the Open Science Policy Sprint landing page.
[ad_2]
Source link
