When open data meets publish-or-perish

In an era defined by open data and increasingly automated analysis, the demand for critical thinking in academic research has become not merely important but essential. A recent Nature news article illustrates a growing concern: an influx of biomedical studies based on open-access data sets is flooding academic journals. These papers, often repetitive in methodology and thin in conceptual depth, rely heavily on large-scale data sets such as the UK Biobank or the US National Health and Nutrition Examination Survey (NHANES). Between 2021 and 2024, this trend has accelerated well beyond what we might expect from natural growth in access or interest. The problem is not simply one of volume. It is the gradual erosion of thoughtful scholarship.

One study exemplifies the issue with almost absurd clarity: a paper claiming that drinking semi-skimmed milk could reduce the risk of depression. At first glance, this might appear a benign finding, perhaps even amusing. But these are not fringe blog posts; they are peer-reviewed publications, indexed in scientific databases, available for citation and potentially influential in shaping public discourse. The concern arises when such claims are drawn from surface-level correlations, absent of theoretical grounding or critical methodological scrutiny. What enters the academic record as “knowledge” can, under these conditions, amount to little more than noise.

• Eight ways your university can make research culture more open
• (Re)learning critical reading in the age of GenAI
• Spotlight guide: What you need to know about academic publishing today 

To understand how such research gains traction, we must return to the foundations of critical thinking. Critical thinking is not reducible to skepticism or contrarianism. Rather, it is the capacity to analyse, synthesise and evaluate information with intellectual discipline. For researchers, this means continually asking not only what is known but how it is known. It entails clarifying assumptions, interrogating methodological choices and acknowledging limitations. These are not ornamental gestures of academic humility; they are foundational to the integrity of our research.

The epistemological insights of philosopher Karl Popper remain instructive here. Science does not advance through accumulation of supporting evidence but through falsification. Knowledge progresses by subjecting hypotheses to the risk of being proven wrong. Yet, in the current climate, many publications are characterised more by confirmatory associations rather than by rigorous attempts at disproof. The logic of scientific discovery is displaced by the logic of statistically significant results, regardless of their interpretive or theoretical value.

Carl Sagan’s famous “baloney detection kit” emphasised similar principles: consistency of logic, proportionality of evidence and a willingness to revise beliefs in light of new findings. These tools are not academic curiosities. They are essential defences against superficiality in an age where information is abundant and easily manipulated. When researchers ignore them, or fail to teach them, we risk mistaking statistical output for epistemic progress.

This is not to argue against the value of open data. Quite the contrary: the democratisation of data has been one of science’s most promising developments in recent decades. It invites transparency, supports replication and lowers barriers to entry for researchers across institutions and geographies. 

But access alone does not constitute insight. Without critical thinking, open data could encourage a kind of academic convenience where ease of access might be mistaken for depth of analysis. In such an environment, statistically significant findings can be generated in abundance. But significance, in the statistical sense, is not synonymous with relevance or plausibility.

At every stage of the research process, critical thinking acts as a compass. It informs how questions are formulated, how methods are chosen and how findings are interpreted. It resists the temptation to let the data set dictate the question and calls researchers back to the primacy of theory. It urges caution against overconfident claims and reminds us that the goal is understanding, not mere output. It asks not only whether the p-value crosses a threshold but whether the conclusion makes intellectual and practical sense.

The academic culture in which we work has sharpened the urgency of this need. The pressure to publish, and to meet expectations for citations, impact factors and numerical productivity, can create incentives that reward speed and volume over substance. When academic success becomes entangled with metrics rather than meaning, the temptation to produce fast-track findings from accessible data sets grows. Automated tools and data pipelines become not just enablers but accelerators of a system drifting away from critical enquiry.

To their credit, some journals have recognised the danger and have implemented safeguards. Authors may now be asked to disclose how frequently data sets have been reused or how multiple comparisons were managed to avoid false positives. These are necessary steps but they remain procedural. The more profound transformation must occur within our academic culture itself. We must move away from treating research as a mechanical exercise in result generation and reaffirm it as an intellectual endeavour defined by critical thinking.

Educators and supervisors in higher education have a distinct responsibility here. Our task is not simply to teach statistical techniques or train students to navigate publication systems. It is to cultivate a mindset that values doubt, reflection and rigour. Teaching critical thinking is not an abstract goal. It is a daily task embedded in how we frame assignments, how we discuss findings and how we model scholarly judgement. In a world overflowing with data, critical thinking is not merely a virtue; it is the very foundation of trustworthy research.

Timo Lorenz is a work and organisational psychologist and junior professor at MSB Medical School Berlin, Germany.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.