Adaptive Management

Conventional project management methodologies, such as the critical-path method and the traditional "waterfall" method, are suitable for projects with limited uncertainties, variables, and timeframes. The focus on predictive planning, and corrective actions as a project progressed, was a generally well-understood and intuitive approach: use proven processes and tools, define all requirements early-on, develop a step-by-step plan, and follow the plan while measuring progress with previously selected metrics. Simple, right?

Unfortunately, this approach may be too simple for modern project-based organizations. No longer will document and artifact-intensive planning, which can become overloaded with unnecessary features and requirements, dictate the course and final deliverable of a project. To be successful, contemporary project managers must transition from a focus on planning to a focus on execution. This includes the ability to adapt to changing circumstances and changing requirements, and requires constant collaboration with clients and stakeholders.

The Agile Manifesto provides an excellent starting point for contemporary project managers: "Individuals and interactions over processes and tools," "Customer collaboration over contract negotiation," and "Responding to change over following a plan" are three broad tenets aimed at delivering higher-quality results to the mutual satisfaction of clients and the provider. The iterative approach taken by Agile management, and other methodologies such as SCRUM, lead to earlier value realization, reduced conflict, and provide the ability for project managers to quickly adapt the project plan when new developments call for change.

Let's face it - requirements churn is a fact of life in most projects, and it's not necessarily a negative. Our goal, and the customers ultimate requirement, is the best possible solution to their need - this is probably not going to be the solution that most closely conforms to the initial needs analysis or project scope document. Our approach, based on SCRUM's empirical approach, is to accept a problem cannot always be fully understood or defined. So we must maximize the team's ability to respond to changing requirements. This entails a short-term work cycle based on an overarching project strategy and high-level project plan.

 

The plan is divided into multiple phases or iterations, with a more detailed plan created for the next phase (similar to the daily SCRUM meeting that answers these questions: What did you do yesterday? What will you do today? What obstacles are in your way?). Future phases can be changed based on the outcomes of the completed phases, or to meet any new or changing requirements. This allows us to refine plans and estimates as-needed, and to deal with schedule/requirements issues as soon as they are identified.

This adaptive approach provides much more strategic flexibility to the project team and the client. Decisions can be made at the end of each iteration based on the outcomes of previous decisions; when this is combined with a much more collaborative process, the result is a better deliverable that is much more likely to meet actual customer requirements (as opposed to the comprehensive, up-front list of requirements typically found in conventional project artifacts).

Contingency Planning

Nothing ever goes completely according to plan, so no matter the size or scope of your project, having some type of informal or formal contingency fund/plan in place is going to be a necessity (and will help protect your relationship with the client).  At one company I worked for, we had some advantages when it came to contingency planning: namely, the majority of our projects were similar in nature, typically revolving around infratstructure improvements or application deployments.  The key variation among these projects was usually restricted to the starting point, that is, their existing technical situation.

Our projects were usually phased, so phase I would be the work to standardize and stabilize what they had; to implement best-practices and remove disparities. Once we had them at a level we were comfortable with, subsequent phases had very little cost variance, because we were building on a solid foundation that we put in place. This allowed us to show senior managers and key stakeholders that a contingency was necessary for the first phase, as we were transitioning from how they currently operated to how we were going to work together, but which could be drastically reduced once the situation was under our control. This was intuitive for them, as everyone expected some bumps in the beginning, so we experienced very little resistance.

We also benefited from our estimating process. We firmly believed in expectations management as a key to customer satisfaction. From the very beginning of a project, when we were in the evaluation and selection phases, our estimates consisted of value ranges based on best-, expected- and worst-case scenarios. This approach accounted for the inherent uncertainty in a project, but still gave the client detailed enough information to make a decision. Using ranges also gave the client options as the project progressed, allowing for some items to be added, changed, or removed without affecting the overall budget range. While this approach didn’t utilize a contingency fund per se', the concept still recognized the uncertainty of the future and allowed for changes to project cost without forcing the project to go over-budget. Because this approach forced the customer to see that we expect some variation, and gave them control over minor changes to scope with regards to cost, they were usually willing to allow for the range, which acted as an informal contingency fund.

Sometimes, however, something more formal was required.  As mentioned above, many of our projects were similar in nature. However, sometimes we would come up against larger or more complex projects that fell outside of our detailed experience, and required a greater allowance for uncertainty. Again, we would present estimate ranges, but we would also plan on a formal contingency. To overcome resistance, we conducted a thorough analysis of what they were asking for, and would detail our primary concerns, as related to uncertainty, to the client to make them understand why we were putting $X.00 aside as a contingency. We also made it understood that as the project progressed, if these contingencies came to fruition, we would discuss how to proceed with them before applying the funds. This way they maintained some element of control, and were more comfortable with the process.

When everything was said and done, the mix of formal and informal contingency planning we utilized, based on strong communication and expectations management, helped us maintain client satisfaction and our own profitability.

What drives cost overruns?

I’ve recently heard some very compelling points with regards to the systemic nature of projects. Perhaps the most poignant concept is the realization that the sum of project overruns is often greater than its parts. I know that, in hindsight, it is difficult to explain how any one change, or the uncertainty in a single arena, could have led to the final outcome. Rather, it’s the impact of the cause of the overrun, multiplied through feedback mechanisms, and by the resulting negative effect on areas of the project that may not be directly linked to the initial difficulty.

The key villains that drive cost escalation are the project organization and the clients themselves. Of course, external forces such as the changes in the regulatory environment or advances in technology can have an impact. However, even though the consequences, in terms of increased costs, may be more severe from external changes, the likelihood of occurrence is usually low (although the longer the duration of the project, the more critical this area becomes), so I wouldn’t consider it a “key” villain.

Systemicity is really an embedded reality for projects. In and of itself, the systemic structure of a project doesn’t drive cost escalation – it simply magnifies it and helps to create “vicious” or “virtuous” circles. So although it’s worth discussion and is a cause of the sum being greater than the parts, it’s not really a key villain when discussing cost overruns. Neither is schedule acceleration, because it is a reaction to project trouble, not a direct cause.

The factors that cause cost overruns, and contribute to large-scale cost escalations, typically include the planning (or control) estimate, customer interference, and customer or contractor created changes to the project plan. These cause the initial difficulties, and act as a root cause for cost escalation. These stages are usually foreseeable, and are controllable, by the two key villains I mentioned above – the project organization and the customer organization.

The real causes of cost escalation during these stages are usually the customer’s failure to provide thorough information during the planning stages, or their inability to “help” the contractor execute the project. Or it may be the contractor’s failure to follow proper configuration management techniques, or ensure a “meeting of the minds” on key project specifications. Failure by both organizations to manage these types of problems throughout the project will lead to cost escalation for each arena, and give rise to the systemic and acceleration stages that will make the team look back and wonder what happened.

An Analysis of Organizational Structure

For this post, I analyzed a regional consulting firm that has been having some growing pains. This analysis allowed me to see that the structure of the organization was significantly contributing to their problems, and would have to be changed if they hoped to improve their PM processes and operational efficiency.

The graphic allows us to see that while each member of the firm's set (functional areas) is integrated, overall communication throughout the organization is lacking. The concentration of authority and customer contact across the two major hubs (lead technician and business consultant) also presents problems in terms of resource allocation - first, by potentially overextending each hub as business grows (creating bottlenecks); and second, by creating separate factions that may not operate in a unified fashion.

This structure does not facilitate collaboration among the various groups: because of the “stovepipe” reporting structure, and the independent mindset among the functional areas (i.e. marketing operates one way with consulting clients, while the helpdesk uses their own procedures with services clients). With the functional departments empowered to manage and complete isolated projects within their “stovepipe,” there are relatively few issues for small clients with specific needs (which is why this was not a serious issue in the past). However, as the client base shifts, the efforts of more than one department will be required to develop and implement comprehensive programs – efforts that cannot be effectively managed with the existing structure.

From a PM standpoint, as this shift occurs, the functional webs converging on the two main hubs limit overall cooperation throughout the organization and with other key stakeholders. With communication and decision-making authority lying outside of the project structure, because there is no true project manager, the project team and functional managers are removed from the needs analysis, and no one maintains accountability for aggregate team and project performance. When looking specifically at the project planning process and scope definition, this structure would limit the authority and decision-making capability of a project manager. It also serves to erect barriers to teamwork, and results in disparate objectives among “competing” functional areas.

This need for better internal communication, and the division of business and technical personnel, coupled with the lack of a single project manager for large projects, makes it harder to garner scope agreement and create an all-inclusive project plan internally; even before solutions are presented to clients. With no internal consensus on what the scope or plan of action will be, it is impossible to approach the customer as a unified organization with full-spectrum expertise.

Since the firm has moved towards larger and more complex projects, their lack of sophistication and failure to create scope definitions and management plans based on key stakeholder consensus has been very difficult to overcome. As we can see, their organizational structure contributes significantly to this problem.

The "Theory" of Project Management

Generally speaking, a theory is a set of related ideas, principles and techniques that apply to a particular subject. Theories are typically used to explain a set of observations, and can provide us with an expectation of what should happen, barring unforeseen circumstances. However, the underlying theory of project management is somewhat implicit in nature. In other words, projects don't play themselves out systematically like mathematics or logic. What works in Project A will not necessarily succeed in Project B. The transitive property of mathematics is rarely observed in projects.

For that reason, some discussion of the conceptualizations behind the theory is warranted before discussing the fundamental considerations. The benefits of using a theory, in the context of conducting a project, are that the practical actions derived from the proper implementation of the theory can help us achieve our goals. I would start by establishing three levels of goals for any project: first, there is the general goal of getting the intended deliverable completed; second, there are technical (or internal) goals, such as cost minimization and adhering to a schedule, that means the project has been done right; and third, there are customer satisfaction (or external) goals, such as quality and functional utility, that means the right project was done right. By applying project management theory, we can (hopefully) dramatically improve the likelihood of reaching the third, and ultimate, goal.

To begin, project management is about managing work. Turner (1993), claims that work can be managed by decomposing total work into activities and tasks – that’s the basic concept. As the PMBOK Guide shows, these activities and tasks are the unit of analysis in the core processes of project management, such as scope management, time management, and cost management. Morris (1994) supports this description by outlining a project management approach based on four steps: first, what needs to be done; second, who is going to do what; third, when actions are to be performed; and fourth, how much is required to be spent in total, how much has been spent so far, and how much still has to be spent. Central to this sequence is the Work Breakdown Structure (WBS), which is a key deliverable of the project planning process, and is crucial to scope management and project control.

The PMBOK Guide provides us with a second concept in project management, by laying the groundwork for planning progression. Ten core processes, including scope planning, scope definition, activity definition, resource planning, activity sequencing, estimating, and project plan development, have been identified to guide the planning process. The outputs from these processes are the project plans, which form the inputs to the execution processes. The implication here is that if you effectively complete these steps, you’ll have the necessary inputs for successful execution, which will lead to a successful project.

These two primary concepts – the project as a series of tasks and activities, and the core planning methodologies as guiding principles – can help us understand a general theory of project management, which is thus: by undertaking a series of sequential, yet interrelated, tasks to define, plan, prepare and execute a project, you will be able to effectively achieve technical success and customer acceptance.

Although this theory sounds relatively simple, we often find that organizations have trouble putting the concepts into practice. Some of the key reasons are a combination of limited resources, lack of understanding, and the perception that there are no immediate financial benefits to justify the time spent on project management (i.e. it’s a cost-center). More specifically, young organizations view formal project management as overly bureaucratic, and as a hindrance to getting business done. Yet, these organizations are the ones who can most benefit from the application of theories that improve structure and accountability – which are direct results of the two key concepts discussed above.

Additionally, the theory of project management helps organizations gain efficiencies through standardization and streamlining. An organization that understands their own business, by analyzing and consistently following the core processes, can lower costs and enhance service levels. These core processes can be documented, allowing the organization to work on ways to improve the work-flow and customer experience at each stage. This feedback loop creates a virtuous cycle: documenting the steps to produce a deliverable will lead to more consistent application of the processes, which leads to more standardization, then to continuous improvement, and then back to documenting the improved steps.

Furthermore, by increasing consistency, deviation rates will decrease and you will find improvements in predictability – lending credence to the theory suggested above that by breaking a project down into tasks and following the core planning methodologies, we can create a plausible expectation of the outcome. This application of project theory also provides more accurate measures of project progress, which enhances our ability to meet project goals.
By ignoring the fundamentals of project management theory, especially during the planning stages, we find that customer requirements are poorly defined, and the process of clarifying and changing requirements leads to disruption. This is one of the key reasons that projects fail – incomplete or inaccurate scope definition. The constant disruptions cause actual progress to drift from the plan, which becomes too burdensome to regularly update (adding to the perception that project management is overly bureaucratic).

Without an updated plan to work from, informal management becomes prevalent. This causes work to be rushed, which in turn causes tasks to be commenced without all inputs or prerequisites, leading to low efficiency, task interruption, and increased variability (degrading our ability to predict an outcome – hence the importance of the theory). Likewise, controlling by means of a performance baseline that is no longer based on actual status becomes ineffective, or simply counterproductive, and results in the customer needs not being met. When we fail to meet customer needs, it’s impossible for us to achieve project success.

Managing Expectations

Selecting and beginning a project is always complicated - no matter how well you plan, something will (probably) go wrong. Sometimes, and I'm guilty of this myself, it's easy to get caught up in how great things will be at the end of the project, or once things get up and running. Managing expectations, especially preparing for the worst in the beginning (for me during the transition period) is important.

To start with, I've learned to help keep expectations realistic, it's important to have a thorough conceptualization process that uses tools like decision-making trees to help the customer understand that costs associated with feasibility and requirements studies aren’t necessarily part of the project scope and budget. This work needs to be done before the project to determine what a realistic scope will be, and if it’s even possible to solve the problem while creating value. This should be done in a separate selection process - and I know it sounds intuitive, but I'm surprised by how many examples I've seen where this up-front work somehow gets thrown into the planning stage of the project.

The customer also has to understand that there may be times were a no-go decision is made with “sunk” costs that can’t be recouped. These types of expenses need to be budgeted at the business-level, so that they don’t impair project performance. This basic form of expectations management helps ensure the project team isn’t overloading the initial stages of the project at the expense of execution, and helps the customer focus on the real business need and appropriate solution before the projects are initiated – so we know the deliverable will be acceptable.

Finally, and this is from personal experience, sometimes the transition from where they are to where they'll be (i.e. what you're going to accomplish with the project) is going to have some rough patches. As I mentioned previously, it's easy to get caught up talking about how great things will be when you're done. But to launch into a project without discussing the potential problems (which could be part of a formal risk management plan) is setting the expectation for your client that nothing will go wrong. So when it does, they are completely unprepared and the relationship can quickly turn adversarial.

By managing your stakeholders up-front, and focusing on potential problems and risks, you create realistic expectations and prepare them for the worst. If the project still makes sense, even if some things go wrong, then it's probably something worth doing. Now that you've sort of "under-promised," and the client is not expecting miracles or nothing but blue skies, you've set a solid foundation for the project to progress. By communicating and remaining practical from the start, you also build a rapport with the clients that will be invaluable in the future should problems occur. You'll have their confidence, because they'll know you're working with them, and that they can trust you.

The Project Lifecycle and Project Strategy

For a project strategy course I'm taking, we read an interesting case study concerning the traditional project lifecycle graphic (some form of Initiation/Planning/Execution/Termination). Authored by Daniel R. Heiser and Bin Jiang, The Eye Diagram is an expanded model that not only shows the amount of effort and/or budget required for each stage, but also looks at the project as part of a larger system, and highlights the changing set of requirements that each stage creates. Here's a few thoughts on where the traditional model falls short, and some of the benefits that an Eye Diagram-type approach offers the PM.

As I mentioned, the traditional project lifecycle model shows the amount of effort required for each phase, and how those phases overlap. This type of illustration, along with other charts such as percentage of work completed and cumulative budget spent graphs, tend to show the project lifecycle as a function of time or resources. However, these charts are really more of a project performance measure, or a simple planning tool, than they are a strategic model for project management. Two key things are absent from these traditional models – first, the environment that the project is being conducted in isn’t taken into account; and second, the changing perspective that PM’s must have to effectively manage each stage of a project is ignored. Each of these factors is critical to implementing a successful project strategy, and effectively managing the project through each stage of the lifecycle. For this post, I’m going to concentrate on the first issue and how it relates to project strategy.

Relying strictly on the types of charts mentioned above can cause the vacuum effect – where the PM ignores the project supra-system (i.e. the organization, the competitive market, the economy). Because the traditional graphics focus on time, resources and costs, they lock PM’s into a world that considers only project constraints and internal forces. It can fool inexperienced project managers into thinking that operational concerns and time/cost/schedule trade-offs are the keys to success, when in reality they are only supporting elements and performance measures. In fact, no project operates in a vacuum, and the real keys to success aren’t even identified in these models.

To be successful, a project needs to have a strategy that is aligned with the overarching corporate strategy. This strategy also needs to have certain elements to it, such as a business need or an opportunity that is driving the project, and a focus on environmental factors as well as internal ones. It needs to be “client-centric”, with a focus on delivering a satisfactory product to the client, but still must balance the requirements of all stakeholders to maximize value. The strategy also needs to be fluid – that is, it must be regularly evaluated and updated as the project situation changes.

If you go back to the traditional lifecycle, you’ll see that none of these topics are illustrated or dealt with. As I pointed out before, the traditional lifecycle is focused on constraints and performance measurement. A project manager needs to go beyond this model, identify external forces, and focus on the flow of each stage in relation to those forces. The Eye Diagram is one useful tool to do just that, and it also helps the PM focus their efforts as they move through each stage of the lifecycle.

Ultimately, for value realization and stakeholder satisfaction, PM’s require a methodology to help them create and execute the strategic plan; something that does more than simply account for resource usage or percentage of project completed. It should identify and account for the strategic issues that will affect eventual project success. Things such as the external environment, the parent organization and even critical success factors should be identified and illustrated, right from the start. As the project progresses through the lifecycle, the focus of the PM’s strategic efforts needs to shift, so they can better understand what factors will impact decisions and therefore require the most attention.