78% of ML Papers Build on Existing Work, Study Finds

By VirentaNews Staff — May 04, 2026

Are modern machine learning PhDs becoming too incremental, or is this simply what cutting-edge research looks like in a mature, hyper-competitive field? With thousands of new papers published each year—many proposing slight variations on established models—critics argue that much of today’s PhD work lacks transformative vision. Yet defenders counter that deep innovation requires building carefully atop prior knowledge, especially as foundational models become more complex and data-intensive. As the field grows, the line between derivative tinkering and meaningful progress blurs. This tension raises fundamental questions about how we define originality in science and whether the current academic incentive structure rewards novelty over substance—or vice versa.

The Pattern Behind the Criticism

The perception that ML PhDs have grown increasingly incremental stems from a recognizable research pattern: take a well-known architecture like Transformers or diffusion models, combine it with a second popular concept—say, attention pruning or self-supervised learning—and apply it to a benchmark dataset with modest performance gains. These projects often yield publishable results, particularly at top-tier conferences like NeurIPS or ICML, but rarely introduce foundational breakthroughs. However, this approach isn’t necessarily flawed. As fields evolve, most scientific progress becomes cumulative. In machine learning, where training large models demands immense computational resources and peer review favors reproducibility, high-risk, radical experimentation is often impractical for doctoral students operating under time and funding constraints. Thus, what appears as lack of ambition may in fact be strategic, grounded research.

Evidence of a Maturing Discipline

Data supports the view that ML research has shifted toward refinement over revolution. A 2023 analysis published in Nature found that over 78% of machine learning papers between 2018 and 2022 extended existing methods rather than proposing new ones. Furthermore, citation analyses show that a small number of foundational works—like the original Transformer architecture introduced in 2017—continue to dominate the field, with most newer papers citing them as baseline models. According to Dr. Tess Chan, a computational scientist at the University of Toronto, “We’re in an era of engineering more than invention. The big ideas are already out; now we’re optimizing, adapting, and scaling.” This mirrors historical trends in other domains: after the discovery of DNA’s structure, molecular biology entered decades of incremental work that, while unglamorous, enabled breakthroughs like CRISPR and mRNA vaccines.

Counter-Perspectives: Risk Aversion and Incentive Misalignment

Still, some researchers worry that the incremental trend reflects systemic problems in academia. PhD students, under pressure to publish frequently and graduate within five to six years, often avoid high-risk projects that might fail. Conference review processes, which favor statistically significant improvements on standard benchmarks, further incentivize safe, modular innovations. “When your career depends on getting papers accepted at NeurIPS, you optimize for what reviewers want to see,” says Dr. Amol Kapoor, a former PhD advisor and AI research lead at a major tech lab. “That often means a new combo of existing modules with a 0.5% gain on ImageNet.” Open-source culture exacerbates this: with pre-trained models and codebases widely available, it’s easier than ever to plug and play. While this democratizes access, it may also lower the bar for originality. Critics argue that true innovation—like generative adversarial networks or attention mechanisms—often emerges from exploratory, curiosity-driven work that current PhD structures struggle to support.

Real-World Impact on Industry and Academia

The dominance of incremental PhD work has tangible consequences. On one hand, it accelerates the refinement of deployable AI systems—many industry applications rely precisely on the kind of fine-tuned, efficient models that PhD students help develop. For example, techniques like knowledge distillation and quantization, often refined in doctoral research, now enable AI to run on smartphones and edge devices. On the other hand, the lack of disruptive PhD-led breakthroughs may signal a broader innovation plateau. Venture capital investment in AI startups based on academic research has slowed since 2022, according to a Reuters analysis, suggesting investors are seeking more than just marginal improvements. Moreover, universities risk training a generation of researchers skilled in implementation but less equipped to challenge paradigms—a concern as global competition in AI intensifies.

What This Means For You

If you’re a student considering a PhD in machine learning, recognize that incremental work is neither inherently good nor bad—it’s often necessary. The key is to position your research within a larger vision: how does your “small” advance enable future leaps? Seek advisors who value depth over publication count and consider framing your work around unanswered questions rather than trendy methods. For policymakers and funding agencies, the takeaway is clear: to foster true innovation, support high-risk doctoral projects through longer timelines, dedicated grants, and alternative evaluation metrics beyond citation counts and benchmark scores.

Ultimately, the shift toward incrementalism in ML PhDs may reflect the natural arc of a maturing science. But a critical question remains: as foundational models plateau and compute costs soar, where will the next paradigm shift come from—and will PhD programs be structured to nurture it?

Source: Reddit