Introduction

A recent study has found that software projects adopting Agile practices are 268 percent more likely to fail than those that do not.

This review explores the study’s conclusions and its implications for product management practices.

The Study

A study conducted by Scottish consultancy Engprax in May 2024 surveyed 600 UK and US software engineers (250 in the UK and 350 in the US) and found that software development projects that adopt Agile Manifesto practices are 268% more likely to fail than those that do the opposite.

The study also finds that software development projects that had clear requirements before starting development were 97% more likely to succeed.

The study’s apparent conclusion is that avoiding Agile practices and employing requirements management greatly increases the chances of project success.

268% Higher Failure Rates for Agile Software Projects, Study Finds

Caveats

The Engprax study repeatedly refers to Agile Requirements, Project Requirements, or Requirements but doesn’t explain what they are and how they differ.

Given the endless debates and conflicting interpretations, it is plausible that some of the survey’s respondents conflate Agile, Agile Manifesto, and Agile Practices with Scrum. Therefore, the survey’s results perhaps reflect upon Scrum, not Agile.

The Engprax consultancy discloses that it has a vested interest in offering an alternative to Agile engineering practices, which, for some, could demonstrate bias and taint the study results.

The Engprax study refers to project failure but does not explain what constitutes project failure or what criteria are used to determine it.

Lightweight Software Development (rebranded as Agile in 2001) is suited for developing certain digital software products (e.g., websites and mobile applications) but not for creating standard, professional, and enterprise grades of commercial software products.

Lightweight Software Development may have been applied to projects it is not appropriate for, resulting in an extraordinary number of failures.

However, despite the study’s inconsistencies, the survey’s data strongly indicates that software development projects struggle to successfully apply Agile and Scrum.

Management Backdrop

Company executives are concerned with smooth and streamlined operations, and they don’t care which software development method is being used.

The Engprax study cites that 81% of business decision-makers in the UK and 89% in the USA are concerned about software projects not being delivered on time.

The study also states that 65% of software projects adopting Agile requirements engineering practices fail to be delivered on time, budget and quality.

The study’s purported overwhelming number of failed Agile-led software development projects is concerning to executives and puts them at odds with Agile/Scrum.

The study also highlights the billions of dollars lost due to failed Agile-led software development projects.

This could be yet another reason for the influx of “Agile is Dead” articles on social media that offer executives alternatives to Agile.

Requirements Management

The Engprax study unequivocally recommends preparing requirements before the software’s coding begins.

It is unclear which type or type of requirements the survey refers to.

The Blackblot PMTK® Methodology recognizes three types of requirements:

1. Market Requirements—These describe the market problem in the product planning stage. Product Managers create market requirements in a Market Requirements Document (MRD).

2. Product Requirements — These describe the solution in the product definition stage. Product Architects create market requirements in a Product Requirements Document (PRD).

3. Technical Requirements — These describe the solution’s technical implementation in the product development stage. Engineers prepare technical requirements in technical documents, collectively named Technical Specifications (Tech. Specs.).

Over the last year, there have been calls to question Agile and advocate for robust requirements processes that are agnostic of any software development method. The Engprax study is a recent example.

Summary

The product development domain is at a crossroads.

In the meantime, while engineers sort out their domain, product managers should focus on their strategic role.

If the Engprax study is prophetic, product managers should also become proficient in preparing market and product requirements.

Written by Gabriel Steinhardt