Medical and scientific advances are predicated on new knowledge that is robust and reliable and that serves as a solid foundation on which further advances can be built. In biomedical research, we are in the midst of a revolution with the generation of new data and scientific publications at a previously unprecedented rate. However, unfortunately, there is compelling evidence that the majority of these discoveries will not stand the test of time. To a large extent, this reproducibility crisis in basic and preclinical research may be as a result of failure to adhere to good scientific practice and the desperation to publish or perish. This is a multifaceted, multistakeholder problem. No single party is solely responsible, and no single solution will suffice. Here we review the reproducibility problems in basic and preclinical biomedical research, highlight some of the complexities, and discuss potential solutions that may help improve research quality and reproducibility. © 2015 American Heart Association, Inc.

In the last 10 years, many canonical findings in the social sciences appear unreliable. This so-called “replication crisis” has spurred calls for open science practices, which aim to increase the reproducibility, replicability, and generalizability of findings. Communication research is subject to many of the same challenges that have caused low replicability in other fields. As a result, we propose an agenda for adopting open science practices in Communication, which includes the following seven suggestions: (1) publish materials, data, and code; (2) preregister studies and submit registered reports; (3) conduct replications; (4) collaborate; (5) foster open science skills; (6) implement Transparency and Openness Promotion Guidelines; and (7) incentivize open science practices. Although in our agenda we focus mostly on quantitative research, we also reflect on open science practices relevant to qualitative research. We conclude by discussing potential objections and concerns associated with open science practices.

The current debate about how to improve the quality of psychological science revolves, almost exclusively, around the subordinate level of statistical significance testing. In contrast, research design and strict theorizing, which are superordinate to statistics in the methods hierarchy, are sorely neglected. The present article is devoted to the key role assigned to manipulation checks (MCs) for scientific quality control. MCs not only afford a critical test of the premises of hypothesis testing but also (a) prompt clever research design and validity control, (b) carry over to refined theorizing, and (c) have important implications for other facets of methodology, such as replication science. On the basis of an analysis of the reality of MCs reported in current issues of the, we propose a future methodology for the post-<.05 era that replaces scrutiny in significance testing with refined validity control and diagnostic research designs.

The verity of results about a psychological construct hinges on the validity of its measurement, making construct validation a fundamental methodology to the scientific process. We reviewed a representative sample of articles published in the Journal of Personality and Social Psychology for construct validity evidence. We report that latent variable measurement, in which responses to items are used to represent a construct, is pervasive in social and personality research. However, the field does not appear to be engaged in best practices for ongoing construct validation. We found that validity evidence of existing and author-developed scales was lacking, with coefficient α often being the only psychometric evidence reported. We provide a discussion of why the construct validation framework is important for social and personality researchers and recommendations for improving practice.

Drawing on interviews with peer-review panelists from five multidisciplinary fellowship competitions, this paper analyzes one of the main criteria used to evaluate scholarship in the humanities and the social sciences: originality. Whereas the literature in the sociology of science focuses on the natural sciences and defines originality as the production of new findings and new theories, we show that in the context of fellowship competitions, peer reviewers in the social sciences and humanities define originality much more broadly: as using a new approach, theory, method, or data; studying a new topic; doing research in an understudied area; or producing new findings. Whereas the literature has not considered disciplinary variation in the definition of originality, we identified significant differences. Humanists and historians clearly privilege originality in approach, and humanists also emphasize originality in the data used. Social scientists most often mention originality in method, but they also appreciate a more diverse range of types of originality. Whereas the literature tends to equate originality with substantive innovation and to consider the personal attributes of the researcher as irrelevant to the evaluation process, we show that panelists often view the originality of a proposal as an indication of the researcher's moral character, especially of his/her authenticity and integrity. These contributions constitute a new approach to the study of peer review and originality that focuses on the meaning of criteria of evaluation and their distribution across clusters of disciplines.

In the past few decades, there have been efforts to transform qualitative inquiry, drawing on resources from both imaginative literature and art. There have long been tensions within social science that have encouraged the use of these resources, but the recent “literary turn” is more radical. The assumptions underpinning it are examined, and it is argued that what is most important is the purpose for which modes of expression are employed and how well they serve this. The problem with the literary turn is that it frequently involves substitution of the purposes of art or politics for those of social science.

Pierre Bourdieu’s “epistemic reflexivity” is the cornerstone of his intellectual enterprise, underpinning his claims to provide distinctive and scientific knowledge of the social world. This article considers what this notion offers for research and how it needs to be developed further to underpin progress in social science. Many reflexive research practices are sociological, individualistic, and narcissistic, and the article contrasts this to Bourdieu’s conception of reflexivity as epistemological, collective, and objective. The author then illustrates how, despite Bourdieu’s intentions, this conception when enacted tends toward the very pitfalls it is intended to avoid. Building on a developing conceptualization of the relations of knowledge, the author identifies this problem as intrinsic to Bourdieu’s framework, showing how it bypasses the significance of knowledge structures and so provides the social but not the epistemological conditions for social scientific knowledge. Bourdieu’s reflexivity objectifies objectification but needs development to help achieve objective knowledge. The article concludes by introducing the notion of “epistemic capital” as a first step toward developing a properly epistemic reflexivity and so realizing the potential of Bourdieu’s enterprise.

Several meta-analytical attempts to reproduce results of empirical research have failed in recent years, prompting scholars and news media to diagnose a “replication crisis” and voice concerns about science losing public credibility. Others, in contrast, hoped replication efforts could improve public confidence in science. Yet nationally representative evidence backing these concerns or hopes is scarce. We provide such evidence, conducting a secondary analysis of the German “Science Barometer” (“Wissenschaftsbarometer”) survey. We find that most Germans are not aware of the “replication crisis.” In addition, most interpret replication efforts as indicative of scientific quality control and science’s self-correcting nature. However, supporters of the populist right-wing party AfD tend to believe that the “crisis” shows one cannot trust science, perhaps using it as an argument to discredit science. But for the majority of Germans, hopes about reputational benefits of the “replication crisis” for science seem more justified than concerns about detrimental effects.

During the 1980s and 1990s, Hans J Eysenck conducted a programme of research into the causes, prevention and treatment of fatal diseases in collaboration with one of his protégés, Ronald Grossarth-Maticek. This led to what must be the most astonishing series of findings ever published in the peer-reviewed scientific literature with effect sizes that have never otherwise been encounterered in biomedical research. This article outlines just some of these reported findings and signposts readers to extremely serious scientific and ethical criticisms that were published almost three decades ago. Confidential internal documents that have become available as a result of litigation against tobacco companies provide additional insights into this work. It is suggested that this research programme has led to one of the worst scientific scandals of all time. A call is made for a long overdue formal inquiry.

National Institutes of Health principal investigators reported their perceptions of the ethical defensibility, prevalence in their field, and their personal willingness to engage in questionable research practices (QRPs). Using ethical defensibility ratings, an exploratory factor analysis yielded a two-factor solution: behaviors considered unambiguously ethically indefensible and behaviors whose ethical defensibility was more ambiguous. In addition, increasing perceptions that QRPs affect science predicted reduced acceptability of QRPs, whereas increasing beliefs that QRPs are normative or necessary for career success predicted increased acceptability of QRPs. Perceptions that QRPs are risky were unrelated to QRP acceptability but predicted reduced extramural funding (i.e., researchers’ lifetime extramural grants and total funding secured). These results identify risk (i.e., beliefs that QRPs are normative to stay competitive in one’s field) and protective factors (i.e., beliefs that QRPs have a significant negative impact on society) related to QRP endorsement that could inform educational interventions for training research scientists.

This article is motivated by the ‘reproducibility crisis’ that is being discussed intensely in fields such as Psychology or Biology but is also becoming increasingly relevant to Artificial Intelligence, Natural Language Processing and Digital Humanities, not least in the context of Open Science. Using the phrase ‘repetitive research’ as an umbrella term for a range of practices from replication to follow-up research, and with the objective to provide clarity and help establish best practices in this area, this article focuses on two issues: First, the conceptual space of repetitive research is described across five key dimensions, namely those of the research question or hypothesis, the dataset, the method of analysis, the team, and the results or conclusions. Second, building on this new description of the conceptual space and on earlier terminological work, a specific set of terms for recurring scenarios of repetitive research is proposed. For each scenario, its position in the conceptual space is defined, its typical purpose and added value in the research process are discussed, the requirements for enabling it are described, and illustrative examples from the domain of Computational Literary Studies are provided. The key contribution of this article, therefore, is a proposal for a transparent terminology underpinned by a systematic model of the conceptual space of repetitive research.