Image credit: Irving Shipyard

POLICY PERSPECTIVE

by Ian Mack
CGAI Fellow
May 2021

DOWNLOAD PDF

Table of Contents

• Introduction
• Definition of Terms
• Oversight of NSS and CSC
• Enhancing the Value of Governance Oversight
• Enhancing the Value of OAG and PBO Oversight
• Why It Matters
• End Notes
• About the Author
• Canadian Global Affairs Institute

Introduction

Earlier this year, the offices of the Parliamentary Budget Officer (PBO)1 and the Auditor-General (OAG)2 developed and published two balanced and nuanced reports.

The first alleges 11 per cent growth in the project cost of the Canadian Surface Combatant (CSC) project to $77.3 billion since they last reported in 2019. This compares with the Department of National Defence’s (DND) estimates of $26 billion when announced in 2008 and then updated in 2017 to $55 billion-$60 billion. In the recent report, comparison estimates were also generated for fleets of 15 FREMM and 15 Constellation frigates of admitted different capabilities at $71 billion and $27.5 billion respectively. In response, DND expressed confidence in their estimate, presumably because of the multiphase design work now underway and the contracting team’s inability to generate a quality cost estimate and schedule until it is complete. Therefore, it is not surprising that the government is sticking to its earlier estimate; in the absence of greater costing fidelity, it seems better not to develop a new budget target that will also be proven wrong eventually.  

The OAG focused more broadly on the lack of timely progress/delivery of shipbuilding projects under the National Shipbuilding Strategy (NSS) as a result of repeated schedule estimate shortfalls. As well, the OAG found that the tools for assessing, mitigating and monitoring schedule risks were weak, particularly in terms of actionable and detailed mitigation planning. These observations led to recommendations to implement mechanisms to obtain reliable schedules with tracking against milestones to achieve timely deliveries, and improved risk management with emphasis on more effective mitigation.

These reports generated a narrative in the media from various entities which runs the gamut from “all is well” to “the sky is falling”. A sense of anticipation had been building for some time before the release of these reports. The pundits, academics and organizations supporting the status quo published documents highlighting the complexities and benefits of the NSS and the rationale for the CSC project in terms of its suitability in meeting the Royal Canadian Navy’s (RCN) unique and justified requirements.3, 4 Some argued that the latest evidence confirmed the inappropriateness of the NSS, called into question the competence of those in government who have in the past and were now executing the CSC project and even suggested the inappropriateness of acquiring warships domestically.5, 6

Interestingly, the reaction to these reports was somewhat muted soon after they were published. If the related public dialogue is indeed diminishing, there could be many reasons – the exceptionally fair reports that the PBO and OAG generated, or perhaps the early and persistent public sector educational documents generated before (warning of what could be expected) and soon after the reports were published (highlighting nuanced qualifications included by the PBO and OAG). As an aside, one result of the negative media reports will likely be the fuelling of doubt among those entering the workforce as to the viability of a career as a marine engineer or as a shipbuilding blue-collar professional. This is concerning, as the availability of such skilled individuals in Canada has dried up based on the NSS demand, leaving shipyards and maritime industries throughout Canada worried.  

I was involved in the NSS and the CSC project from 2007 to 2017. Therefore, my observations are dated and may be OBE. I was a plank owner in the NSS and involved in its promotion and implementation throughout that decade. I also was responsible in 2008 for initiating the reinvigoration of the project strategy to acquire a combatant fleet for Canada (what is now the CSC project) to replace the four-ship Tribal-class destroyers and the 12-ship City-class frigates in one procurement, as well as assisting in the project’s development throughout the following decade. As such, I am in the camp of supporting these two initiatives and somewhat accountable for their outcomes.

Nevertheless, when reading the PBO and OAG reports, I must admit that the government’s routine oversight of both initiatives may have fallen short of the mark. This paper looks at potential opportunities to enhance the value added of NSS oversight.

TOP OF PAGE

Definition of Terms

It is useful to adopt some definitions before continuing:

• Project oversight – Many documents point out that the two words from which oversight originates indicate that this word means “seeing from above” in an organization. Such statements are soon followed by the caution that oversight should not meddle in the management of activities being overseen. I have selected a definition for oversight developed from the website Simplicable: the process of assuring the quality of project management and delivery, designed to reduce project risk and to improve outcomes by implementing lessons learned and best practices.
• Project governance – Interestingly, the term “governance” is from a Greek word meaning “to steer”, but the term has clearly evolved. This is perhaps one of the poorest researched and developed terms relating to projects as the focus is generally on corporations and societies. The Project Management Institute defines it as an oversight function aligned with the organization’s governance model. The Association for Project Management seems to define this term as including those portions of organizational governance concerned with the project management “infrastructure” put in place by organizations under which projects must function and the mechanisms by which compliance will be assured. Other definitions suggest that governance focuses on meaningful project ownership, efficiency, effectiveness, compliance, risks and issues, communications and clarity in terms of accountability and understanding the business sector involved. As a subset of organizational or corporate governance, all of the standard elements of good corporate governance also apply.
• Project audit – This is generally defined as the review and monitoring of public sector organization project plans and outcomes to ensure that they are achieving expected results, represent good value for money and comply with applicable policies, laws, regulations and ethical standards.

Using these definitions, one can take stock.

It is also useful to remember that oversight is different for each phase of the project life cycle. Leading up to and during project definition, the risks to timely and effective progress are significantly generated within the government, admittedly influenced by industry. The project office works hard to minimize delays and the inflation-related loss of buying power which occur when project approval to enter definition is tardy. Once in project implementation, new risks emerge as selected private sector contractors attempt to meet the terms of their contract on time and within budget, not only for their own work but throughout extensive supply chains and in a very uncertain global marketplace. In this phase, the best the project office can do is track progress and work in collaboration with the contracted team to mitigate significant risks.   

TOP OF PAGE

Oversight of NSS and CSC

Oversight is significant in terms of defence procurement, in Canada as in most Western democracies, and even more so for NSS. Generally speaking, during my tenure there were many tiers of oversight:

• In terms of project management infrastructure, the project management and contracting policies were established primarily by Treasury Board (TB) and each of the involved core departments (DND, Public Services and Procurement Canada (PSPC) and Innovation, Science and Economic Development Canada (ISEDC)) interpreted such policies in directives with supporting processes and programs (e.g., DND used a small project management support organization and launched a project management competency development program).
• The organizational governance involved in project oversight was tiered:
  • At the highest level, it involved the government of the day in terms of ministers and the three central agencies (the Privy Council Office (PCO), TB and Finance), with a special focus on the project approval gates to enter definition and implementation.
  • For ongoing senior governance, deputy ministers (DMs) and assistant deputy ministers (ADMs) each operated separate councils including the three core departments (DND, PSPC and ISEDC) and the three central agencies (Privy Council Office, TB and Finance), typically meeting monthly to receive status reports and address issues arising. These were chaired by PSPC and supported by a secretariat.
  • The Major Capital Projects Interdepartmental Oversight Committee (MCPIOC) met monthly at the ADM levels of the three core departments along with the three central agencies. DND chaired it as an informal information-sharing forum relating to the ongoing action to address emerging risks and issues.
  • Within the core departments, there were also governance bodies and review processes – these being very extensive in DND.
  • At the project execution level, internal governance was a shared responsibility across the core departments and led by the responsible ADMs and their directors-general (DGs) who guided the project managers, contract managers and industrial and technological benefits managers.
  • Specific to NSS and shipbuilding projects such as CSC, joint governance with the shipyards occurred at four levels: twice annually between the DM’s council and each shipyard president, bi-monthly at the ADM/president of shipyard level, bi-monthly at the  DG/VPs of the shipyard level and more frequently at the project managers’ level.
• In terms of audit (or audit-like) activities, the OAG and PBO operated at the government level and core departments employed their own audit services.

As a DG involved in shipbuilding projects and the NSS for many years, I can attest to the gruelling pace for those involved and the complexity of this oversight ecosystem, requiring monthly travel to one or the other coastal shipyards and attendance at many governance meetings when in Ottawa. It also included continual changes to committee members with most new participants lacking domain knowledge of shipbuilding and an understanding of the context and challenges of NSS and the shipbuilding projects. In terms of routine governance, four tiers dealt with all major defence procurements such that the time to truly understand, explore and address important decision items was extremely limited. An external senior advisor was hired under contract to advise the deputy ministers on NSS matters, though an unfortunate potential conflict of interest precluded his access, and thus ability, to offer advice on the CSC project.  

I could offer many perspectives on the strengths and weaknesses of the governance oversight activity that was in place during my tenure. However, I left the business some four years ago and it would be inappropriate to assume that nothing has changed. Therefore, I choose not to do so. However, I will offer four perspectives.

The first relates to an example of what I saw as excellent oversight in the early years following the release of the Canada First Defence Strategy (CFDS). Understanding the challenges facing DND project managers, TB officials reached out to assist us in any way they could. As one example, TB supported a “swarming” methodology whereby days would be spent with all stakeholders at the table until all questions had been answered, the TB submission was complete and appropriate policy waivers agreed upon. This was effective and efficient oversight of routine projects at work.

The second is that senior levels of governance unwittingly created unachievable expectations. The desire to announce immature project cost estimates and milestones too often set projects up for failure from day one. This was clearly evident when CFDS was released in 2018 with a budget of $26 billion for the CSC project.

My third point is that the dozens of officials involved in this extensive governance arrangement could not be expected to understand the complex business of shipbuilding. There was no comprehensive orientation program and only a handful asked for and received a three-day program including visits to each NSS shipyard and extensive formal and informal briefs along the way.

My final point is that project management infrastructure had atrophied and not kept up with the emerging practices related to complex projects.  Lacking experience, project office members could learn about such things as risk-treated and integrated master cost and schedule systems built on detailed work breakdown structures and tracked with mature earned-value management techniques – the backbone of oversight during ship construction. However, without extensive experience, they lacked the judgment to provide appropriate planning assumptions or confirm that industry’s assessments were realistic, thereby jeopardizing the quality of cost and schedule predictions. As we all know, such capabilities once lost take generations to rebuild. Were it not for the fierce dedication of many and the experience gained during the successfully implemented Halifax-class modernization, I believe the CSC project could still be dead in the water.

TOP OF PAGE

Enhancing the Value of Governance Oversight

An assessment of what good governance might look like is perhaps useful. However, books are written on the subject which can better address the matter than I could. Nevertheless, many of the attributes are obvious from the earlier definition of the foci of importance to project governance bodies. I would offer but a handful of additional elements of good project management governance.

A few adages come to mind. One should always try to apply the KISS principle; too many cooks spoil the broth (in this case, oversight and governance); and Tom Peters’ quote: “The simple act of paying positive attention to people [in this case, those charged with developing and executing of complex shipbuilding projects] has a great deal to do with productivity.”

Building on the four perspectives mentioned in the previous section, I believe that project governance is most effective when it is populated with servant leaders – those who “put the needs of the employees first and help people develop and perform as highly as possible”.7 This is particularly important with complex projects that require very different mindsets and tools than do the more common transactional initiatives.

In the particular case of NSS, shipbuilding requires advanced approaches to leading, planning, estimating and risk/issue treatment – all disciplines that are only effective if advanced to the point of specialization. Those involved in governance need to appreciate the differences, establish new standards and ensure that the officials who are assigned project execution responsibilities have access to the necessary tools and skilled experts to apply them.

Clearly, those involved need to be educated in the business of complex project management practices and knowledgeable in the unique context and subject matter of shipbuilding. Both of these are achievable  – beyond comprehensive orientation and the use of external expert advisors, organizations such as the International Centre for Complex Project Management (ICCPM) can be engaged to educate those involved in governing.

Good governance needs to ensure they have ground truth to properly judge the information being provided. This can only be achieved by walking the shipyards and talking in detail to yard and project staff – much as CEO’s often do by spending a week on-site annually for major complex engineering projects.

And as is recommended broadly for governance bodies, periodic independent performance reviews enable incremental improvement towards efficiency and overall effectiveness.

In closing this section, it would be inappropriate not to comment on two related government initiatives that are likely stalled as a result of the pandemic: the government-directed study of options and implementation of a defence procurement agency which could reduce the complexity and improve accountability within project governance, and the TB-led development of a project management scheme to up-skill project managers government-wide to better navigate complex major capital projects. Both of these initiatives could contribute significantly to NSS governance.

TOP OF PAGE

Enhancing the Value of OAG and PBO Oversight

As already mentioned, the two recent PBO and OAG reports were insightful, balanced, nuanced and delivered in accordance with their mandates. These organizations are essential elements in the oversight business today. Yet their reports seem not to speak to the root causes of the dramatic variances in cost estimates for CSC or the reasons why all shipbuilding project schedules have repeatedly failed to be better estimates. I worry that their work is not contributing to lessons learned and remedial actions, one of the key responsibilities of oversight. The PBO and OAG could provide greater value if their mandates were tweaked to explore and report on root causes along with potential practical solutions, rather than symptoms.

In the case of the PBO mandate, I offer two suggested amendments which I do not see based on the legislation – though they may actually be part of their practices. The first is that the office should not have to accept the detailed scope of analysis requested by parliamentary committees or members. In the case of the recent CSC audit, the PBO was directed to provide cost comparisons with specific international shipbuilding projects, one for the United States Navy (USN) and one for the Royal Navy (RN). As the Parliamentary Budget Officer admitted, his team lacked the expertise to conduct a capabilities comparison. Without such an apples-to-apples assessment, such comparisons lack completeness and are easily misleading. I suggest that the PBO’s mandate should be refined to enable the unilateral rejection of requests beyond PBO capabilities. Were this the case before starting work on the latest CSC report, they likely would not have included cost estimates for the Constellation class, which is a much less capable warship that would not meet the RCN’s needs.

Second, I propose that the PBO’s mandate include exploration of material differences between their independent estimates and those of the department executing the project under review. This could lead to improvements in the cost-estimation capabilities government-wide, but it seems that the PBO mandate is interpreted to exclude – or prohibit – such normative work. Again referring to the latest PBO report, an explanation of the major differences in their cost estimates for the CSC project with those of DND would have assisted all stakeholders to better understand the variances. Were this done, it could have triggered a DND response of greater value than stating confidence in their now-dated cost estimate. For example, they might have identified their intentions to update the DND cost estimate going forward and at times when more fidelity was likely – for example, once design work is complete and a contract for construction of the first flight of ships is in place, and subsequently after delivery and acceptance of the first ship. But if nothing else, it could better explain DND’s current position on their cost estimate and also assist observers to understand how to interpret future PBO top-down model-generated estimates.

In the case of the OAG, the office’s mandate states that “Performance audits do not question the merits of government policies … they examine the government’s management practices, controls, and reporting systems based on its own public administration policies and on best practices.” This again can shortchange taxpayers because it may fail to identify root causes of deficiencies identified during audits and virtually guarantee repeated issue identification in follow-on audits. For example in the most recent NSS audit, concerns are raised with the application of risk registers – the latter being the risk-management process related to the relevant TB policy. In the case of CSC risk management, the TB guidance in 2017 was in my view insufficiently robust for use in complex projects. The OAG does not appear to have observed upon this. 

Permit me a brief amplification. Risk management – something I (and others) prefer to call “risk and the related potential issue treatment” – is typically at the root of many shortfalls when things go awry in most endeavours and thus is a routine aspect of OAG audits. In the past decade, the study of complex projects has led to emerging practices, including much more advanced and rigorous risk-treatment systems which interrelate and “dollarize” identified risks, work to mitigate by reducing the resultant delays of realized risks and the resulting issues, and employ protocols to respond to the potentially devastating unknown-unknown or black swan events. However, this is but one example where TB policy should require the approach to complex project standards – again, I recommend contact with ICCPM.

TOP OF PAGE

Why It Matters

We need effective oversight. When the budgets are so large and address the defence of the nation as we face growing evidence of developing existential threats from grey zone, ongoing conflict and great power rivalry, scrutiny is essential to best ensure that the Canadian Armed Forces (CAF) are ready to defend our nation, continent and values against future threats, as one element contributing to our citizens’ security.

I believe Canada needs effective oversight on major defence acquisitions even more than other Western democracies. We live in a nation where defence is a commonly undervalued business line of government. When the procurement of new capabilities is complex and extremely expensive, missteps must be avoided so as not to jeopardize such projects. Warship acquisitions are even more difficult for Canadians to understand and support because the RCN operates far from our shores and out of sight – negative media attention can more easily and detrimentally impact naval shipbuilding activities than it would armoured vehicle and fighter jet projects.

In such an environment, one should not be surprised when even the most balanced and nuanced PBO and OAG reports generate negative reactions among observers. For the same reason, such negative reactions can have lasting consequences to the timely replacement of critical military equipment systems. Such were the outcomes of the cancellation of the New Shipborne Aircraft and Next Generation Fighter projects – projects cancelled for what many observers saw as valid reasons but which have dramatically delayed the acquisition of new capabilities while new procurement processes were implemented.

There can be no question that effective governance bodies are needed to oversee continual incremental improvement in the shipbuilding business. Furthermore, the essential work of the OAG and PBO should deliver maximum value in terms of avoiding repeated missteps. Along these lines, the concepts of a defence procurement agency, an academy to train project managers (and key staff) to execute major capital projects across the government, and transparency based on communications with integrity are also important initiatives – the latter worth a paper in its own right.

We can do better. We owe it to the nation and to the sailors we rely on to defend us with unlimited liability.

TOP OF PAGE

End Notes

1 Parliamentary Budget Office, “The Cost of Canada's Surface Combatants: 2021 Update and Options Analysis.”

2 Independent Auditor’s Report, “REPORT 2 – National Shipbuilding Strategy,” Office of the Auditor General of Canada.

3 Ian Mack, “Launching the Canadian Surface Combatant Project,” Vanguard, vol. 28, no. 1, February/March 2021.

4 Naval Association of Canada, “The National Shipbuilding Strategy and the Canadian Surface Combatant,” February 4, 2021.

5 Scott Gilmore, “It’s Time to Ban the Buying of Made-in-Canada Warships,” Maclean’s, February 25,  2021.

6 Dan Middlemiss, “CSC-NSS Delays, Cost Overruns and the Elephant in the Room,” Canadian Naval Review, March 3, 2021.

7 Sen Sendjaya and James Sarros, “Servant Leadership: Its Origin, Development, and Application in Organizations,” Journal of Leadership & Organizational Studies, September 2002. 

TOP OF PAGE

About the Author

After a 38 year career with the Royal Canadian Navy, Ian Mack (Rear-Admiral Retired) served for a decade (2007-2017) as the Director-General in the Department of National Defence responsible for the conception, shaping and support of the launch and subsequent implementation of the National Shipbuilding Strategy, and for guiding the DND project managers for the Arctic Offshore Patrol Ships, the Joint Support Ships and the Canadian Surface Combatants. He also had responsibility for four vehicle projects for the Canadian Army until 2015. Since leaving the government, he has offered his shipbuilding and project management perspectives internationally. Ian is a longstanding Fellow of the International Centre for Complex Project Management. He also is allied with Strategic Relationships Solutions Inc. He is married to Alex, and has three grown children. With few accommodations for impaired mobility, he remains active.

TOP OF PAGE

Canadian Global Affairs Institute

The Canadian Global Affairs Institute focuses on the entire range of Canada’s international relations in all its forms including (in partnership with the University of Calgary’s School of Public Policy), trade investment and international capacity building. Successor to the Canadian Defence and Foreign Affairs Institute (CDFAI, which was established in 2001), the Institute works to inform Canadians about the importance of having a respected and influential voice in those parts of the globe where Canada has significant interests due to trade and investment, origins of Canada’s population, geographic security (and especially security of North America in conjunction with the United States), social development, or the peace and freedom of allied nations. The Institute aims to demonstrate to Canadians the importance of comprehensive foreign, defence and trade policies which both express our values and represent our interests.

The Institute was created to bridge the gap between what Canadians need to know about Canadian international activities and what they do know. Historically Canadians have tended to look abroad out of a search for markets because Canada depends heavily on foreign trade. In the modern post-Cold War world, however, global security and stability have become the bedrocks of global commerce and the free movement of people, goods and ideas across international boundaries. Canada has striven to open the world since the 1930s and was a driving factor behind the adoption of the main structures which underpin globalization such as the International Monetary Fund, the World Bank, the World Trade Organization and emerging free trade networks connecting dozens of international economies. The Canadian Global Affairs Institute recognizes Canada’s contribution to a globalized world and aims to inform Canadians about Canada’s role in that process and the connection between globalization and security.

In all its activities the Institute is a charitable, non-partisan, non-advocacy organization that provides a platform for a variety of viewpoints. It is supported financially by the contributions of individuals, foundations, and corporations. Conclusions or opinions expressed in Institute publications and programs are those of the author(s) and do not necessarily reflect the views of Institute staff, fellows, directors, advisors or any individuals or organizations that provide financial support to, or collaborate with, the Institute.

TOP OF PAGE