Why federal government IT projects are doomed to failure

Yogi Schulz has more than 40 years of information technology experience in various industries, including the energy industry; his specialties include IT strategy, web strategy and systems project management. His new book, co-authored by Jocelyn Schulz Lapointe, is “A Project Sponsor’s Warp-Speed Guide: Improving Project Performance.”

Federal government information technology (IT) projects as they’re executed now are doomed to failure. These projects unnecessarily kill careers and cost taxpayers billions while delivering low to modest value.

These IT projects try to satisfy the demands of too many stakeholders. That’s impossible. These sky-high expectations lead to the following:

• Significant delays compared to the original schedule.
• Massive cost overruns compared to the initial budget.
• Reduced scope compared to the original description.
• Disappointing quality compared to a reasonable standard.

Let’s explore how governments arrive at these horrible outcomes and how to position IT projects for success. No one sets out to under-perform on value and quality or over-shoot on cost and schedule so egregiously.

Massive groupthink

Groupthink includes as many civil servants as possible to discuss goals, requirements and scope.

On the positive side, it ensures:

• The opinions of every stakeholder, large and small, essential and not, are considered.
• Project risks are exhaustively examined.
• Every requirement, critical or unimportant, functional and non-functional, is documented in detail.

On the negative side, it ensures:

• Accountability is impossible because no one is in charge. No one can be censured or fired for incompetence or poor performance.
• Prioritization of requirements receives little or no attention, leading to projects with enormous scope and complexity.
• The extensive stakeholder consultation and indecision drive up the IT project cost and extend the schedule substantially.

The groupthink problem afflicted the development of the ArriveCan app incredibly. The government needed the software quickly during a pandemic, where the word “quickly” is already a problem. The requirements were vague and changed from day to day due to groupthink and changing responses to the pandemic. It is in the vendor’s financial interest to play along and do nothing to speed up the process.

Groupthink also hampered the Phoenix payroll project, which received initial federal funding in 2009. The payroll system was launched in February 2016 to more than 34 government departments. Since the government pays every civil servant, they all saw themselves as stakeholders. Everyone wanted the shortcomings of the old system fixed and presented a long wish list of new features. No one held the power or exhibited the leadership to prioritize features.

The solution to minimize groupthink is to tackle large IT projects as a series of smaller projects. Focus on delivering the functionality that end-users will use the most first. Don’t plan projects to last longer than a fiscal year. Don’t promise to provide everything that’s been discussed. Once the system developed by the first project is in production, start the second project.

Distracting blame game

Government organizations are much quicker to pass out blame and ruin people’s careers than to hand out praise for work well done.

This culture creates extreme risk aversion, leading to excessive risk assessments and protracted decision-making that can easily add an order of magnitude to IT project costs and many years to the project timeline.

This blame problem is evident in the ArriveCan story. At the government operations committee investigating what occurred, the Canada Border Services Agency’s then vice-president and chief information officer – the person that one could reasonably believe would be responsible for awarding the contract to develop ArriveCan – claimed he did not make the decision. He was later contradicted by an assistant deputy minister who testified to the committee. More surprisingly, he was subsequently promoted to chief technology officer for the federal government.

Since it’s impossible to tell if the committee is looking to blame someone for the fiasco or is interested in improving government processes, the best strategy for bureaucrats is to avoid accountability.

The solution to minimize the blame game is to keep projects and project teams really small, meaning under 10 people in most cases. That leads to successful completion. There’s no one to blame for failure because there’s only success.

Excessive requirements

Governments always want to make everyone happy so no one can criticize what’s happening. This impossible goal leads to huge requirements lists to cover every eventuality, no matter how rare or inconsequential. That leads to high costs, long schedules and huge software that’s expensive to develop and impossible to test thoroughly.

Many Canadians criticized the ArriveCan app for the complexity of its user interface and the amount of information it asked for. Such criticism indicates a determination to incorporate excessive requirements into an app. In the case of ArriveCan, there were likely concerns about the risk of illegal entry into Canada. I'm sympathetic when civil servants aim for unattainable perfection because of the risk of a few illegals being discovered later and blown out of proportion in the media.

This problem also afflicted the Phoenix payroll project. The requirements were complex due to multiple arcane agreements between the federal government and its various employee groups. Worse, no one in management knew that federal payroll analysts couldn’t pay civil servants accurately and had not done so for years due to the limitations of the old system. The successful vendor for the new system identified thousands of requirement gaps and turned those into reasonable change orders that the government was obliged to pay for.

Seven years after the Phoenix pay system was launched, government records showed there were still 209,000 unresolved pay transactions at the end of April 2023, according to a news report. As of September 2023, the federal government had distributed nearly 4.3 million compensation hours to federal workers affected by the Phoenix pay system woes.

The solution to minimize the adverse impact of excessive requirements is to let the system designers on the project team propose which functionality will be used the most, will provide the most benefits and should be developed first. The designers also should prioritize functionality that must be in place first to support other requirements later.

This approach sidesteps lobbying by various stakeholders to have their needs met first. It also ensures the earliest possible availability of practical functionality to maintain political and financial commitment to the project.

Costly and arduous procurement complexity

Due to the blame game and massive requirements, government procurement processes have become incredibly arduous. This complexity means only the largest, most expensive and most well-financed vendors are eligible for work because only they can afford to participate in this costly process. Also, civil servants see such vendors as the lowest risk to themselves.

The procurement problem manifests itself in overly complex government RFPs and vendor RFP response evaluations for acquiring IT services. Sometimes, these processes have dragged out for years before any business is awarded.

In the ArriveCan mess, civil servants had to circumvent procurement processes to hire a vendor quickly to respond to an active pandemic. Schedule pressure caused civil servants to swing from ridiculously excessive procurement governance to almost no procurement governance at all.

All the problem-inducing factors I’ve listed in this article existed in the ArriveCan project. Government provided the vendor with carte blanche to bill what we now see, with the benefit of hindsight, as unreasonable sums. GCStrategies, a two-person IT staffing firm based in Ottawa, received more than $11 million to work on ArriveCan – more than any other private contractor, witnesses told the government operations committee.

The Phoenix payroll project experienced an arduous procurement process and awarded the business to IBM, which we all know is a large, capable, well-financed vendor. The problem was an inadequate understanding of requirements and an avalanche of historical data errors that needed correction. Click here for my detailed analysis of the Phoenix payroll project.

The solution is to simplify the procurement process dramatically. Such simplification requires acknowledging that the current process fails in its goal to contain risk. It only adds cost and schedule to IT projects.

Instead, contract for much smaller projects with a more straightforward RFP process. In this scenario, the cost of failure will also be much smaller.

There’s a saying that the first 80 per cent of the requirements consume 80 per cent of the IT project budget. The remaining 20 per cent of the requirements consume the other 80 per cent of the budget.

If federal government IT projects can manage to achieve this goal, it will be a win-win for civil servants and Canadian taxpayers.

R$