This post is not about the success and failure of IT projects. Success and failure are subjective terms subject to combinations of the metrics associated with system performance, cost, schedule, value and customer, user and stakeholder satisfaction. For example, a project may be late, cost more than estimated, and can still be considered a success if the customer and users feel the system performance provides them with value and they are satisfied with the overall results. Likewise, a system can be delivered on time and within cost but not provide the value and satisfaction the customer, user and stakeholders are looking for and can be considered a failure.

 

For this reason, this post is only going to address the individual results in system performance, cost and schedule and how well the estimates that established the expectation of what the results would be, actually meet these expectations when the actual results are obtained. So setting the judgment of overall failure or success of IT projects to the side, there are certainly enough instances of the IT system not meeting the complete needs of the customer, users and stakeholders, and enough instances of IT projects not meeting their cost or their delivery commitments to conclude there are problems with IT projects that need to be better addressed.

 

Additionally, there are two sides to IT System and Service Development – The technical side, the technical solution involving the system design and development, and the project management side, the management of the design and development activities of the technical solution. This post is only addressing the project management side, but this is not to discount the importance of an optimal technical solution and how a poor IT solution can affect the project management. However, by properly managing an IT project, even the less than desirable IT solution can be optimally developed from the perspective of cost, schedule and system performance. In other words, cost and schedule can always be minimized and ability and quality maximized within the limits the specific technical solution will allow.

 

There are many people working on improving the technical solution side of the problem and I refer you to them for advice on that side of the problem. In this post I will only be addressing the project management side of the problem and those project management solutions which need to be employed to assure the project is the best the project can be by eliminating and/or minimizing project management problems.

 

This post is therefore meant to enlighten executives, IT project managers, IT architects and enterprise architects in better understanding the devil in the details of the management of IT system and service development for the express purpose of transforming investments in IT Development from being expenses to excellent returns on investment increasing income and profits for the business.

 

Managing The Project Management Problem

 

On the project management side of the problem there is this belief that a person can obtain a PMP certification and is then prepared to manage any type of project including IT projects. The problem is the PMP is a generic certification teaching the generics of project management and the devil is always in the details of the specifics of the project. Without the KNOWLEDGE AND EXPERIENCE in managing the specifics of IT, a PMP certification alone is not sufficient to manage an IT project.

 

IT projects are business projects requiring a holistic approach and understanding of the business which IT project managers seldom have. IT business projects are often large and complex projects, requiring the development of technology which requires 1) an understanding of the BUSINESS USE of information and data and 2) the ENGINEERING design and development of the system which processes the data and information. Unless the PM understands the details of the business and the details of system engineering, the PM will not be able to manage the project details and the project will perform poorly.

 

Managing The Complexity & Technology Problem

 

Managing the problems with IT projects requires an understanding of the contributors to the problems. Many IT project managers fail to understand the contributing factors and what these factors lead to and therefore are in a state of denial concerning the problem and simply write it off to IT being complicated, that this is just the way IT is, and they just have to accept the way IT is at face-value, never attempting to deal with the details.

 

There are many contributing factors to the problem. IT systems are increasing in size, complexity and inter-connectivity. They depend on increasing quantities of more complex software to enable their complex functionality to solve more complex , more interconnected, farther-reaching, and harder to solve problems, along with demands to quickly get them into operation at the same time they are getting increasingly more complex to build, test and deploy. To sum it up, the problem is getting more wicked due to rapidly changing technology and rapidly increasing complexity which are then compounded by the lack of good leadership and project management to address the problem correctly.

 

The IT perspective of this is IT is just too complicated and too large to plan and estimate cost, schedule and risk or to get it right the first time and anyone that disagrees with this just doesn't understand IT. It may be true it is large, but it is not complicated, it is only complex, and the rest of the argument is false, because what is true about it is the lack of proper engineering and project management, especially in the initial phases of the project when systems engineering, management analysis and planning, quality assurance and risk and opportunity management are most critical. Instead of doing all the correct things when the job is large and complex and demands it, they just take the easier path of allowing a half-baked plan to throw a half-baked design together, calling it an iteration or an epic, and rush to deliver it to the customer to be tested in the operational environment. Then when the customers and users yell it is not working or is not what they want, they try again and again and again forever and day after to get it right, blaming the customer for not knowing what they want, followed by blaming creep in requirements, project scope, and the associated rework caused by the customer and the creep.

 

This is why IT organizations are on average currently spending 80% of their budgets maintaining their IT systems and quickly running out of funding for new developments. And "maintaining" implies that the system was fully operational to begin with, which is seldom the case, and the lack of full functionality hidden behind a development methodology called "agile" based on delivering incremental functionality, as the reasoning is it is always better to deliver a fraction of what is wanted, than to deliver nothing at all. At least if the customer gets something, it keeps them busy and temporarily out of the development people’s lives.

This is not to imply there is anything wrong with the use of agile development methodologies, but rather there are many things wrong with the way agile development is executed. For example, the use of agile design methodologies does not preclude the need for good requirements and processes properly managed by good leaders.

 

Managing the problems with IT projects requires properly addressing the increasing complexity and the rapidly changing technology and not taking a simple approach to a complex problem by calling it complicated and using that as an excuse to not do the upfront system engineering and management analysis and planning which the complexity demands including the early application of quality assurance and the proper application of risk and opportunity management which the rapidly changing technology demands.

 

Managing The Functional Silo Problem

 

Being that IT business projects involve many organizations throughout the business, IT organizations suffer the consequences of the functional silo problem and understanding how to manage the problem. To begin to manage it, it must be recognized the functional silo problem is human nature. It is the way we are wired. We will always gravitate to an environment where we feel comfortable and secure.

 

People can be pulled out from their silos but as soon as you let go they will return to them, some faster than others, but they all return eventually. If you knock down the walls, they will build them back up. To use a cliché from the television sitcom “Cheers”, “we all want to go where everybody knows your name. And they're always glad you came. You wanna be where you can see our troubles are all the same.” The silo problem needs to be treated as adaptable behavior to seek, find, build, defend and maintain an environment of comfort, security, commonality and community.

 

When an organization is new it is small, it is one, and regardless of the different types of people within it, regardless of any level of dysfunction, there is a feeling of commonality within the community and this provides the comfort and security. As the organization grows it divides into smaller organizations each having its own environment of comfort, security, commonality and community. Likewise, as organizations shrink, they consolidate, first having two communities within one organization, but in time the division in factions disappear, commonality increases, comfort and security increases, and the two communities become one again. And this is no different with families and friendships. It’s all just our nature. Just the way we are. We just have to deal with it and manage it correctly. We can’t change it, we can only correctly account for it!

 

The functional silo problem is a true paradox, and the solution cannot be to breakdown the silos as such an effort is fruitless and unsustainable. The solution must allow the silos to exist, to try and take advantage of the environment of comfort, security, commonality and community they provide, and make that situation work for the enterprise as a whole.

 

The solution to the functional silo problem is to develop an integration methodology that is applied across the organizations (the functional silos), especially when the projects are very large and complex. The integration methodology needs to align tasks within the flow of work in the enterprise to ensure the right person is doing the right job at the right time, and to provide that person an understanding of the dependencies that person has on others and others have on them, along with a solid expectation of what the maturity level of their work products are at any specific point in their lifecycle. This method of increasing communication and understanding is the most effective mechanism for horizontally integrating capabilities and eliminating the silo problem without eliminating the silos.

 

Managing The Nature of IT Problem

 

The "nature" of IT compounds the functional silo problem, as there is the information side of IT (the provision and business use of information and data) and the technology side of IT (the engineering and development of the systems and related products/applications) and depending on the maturity of the organization, the problem changes, but does not get any better. If anything, the more the problem tends to be solved, the more the problem tends to get worse, which is usually the result when the problem is wicked and has a paradox in it, and why a business supported enterprise architecture practice can significantly reduce this anomaly. That is, provided a real enterprise architect is the leading practitioner and not an IT architect calling themselves an EA.

 

In a small organization there tends to be an information and technology function (i.e., a CIO or Director of IT in terms of an organization and role/responsibility) that usually has strength on the information and data side but a weakness on the technology side (lack of engineering and proper development of the system). 

In larger organizations there tends to be a split of role/responsibility with an information function (CIO/IT Director) and a technology function (CTO/Engineering Director) that then surfaces the functional silo problem. 

The functional silo problem then tends to be solved by giving the CIO/IT Director all the responsibility which then creates a lack of authority problem for the CTO/Engineering Director, worsening the functional silo problem, but this situation usually winds up being accepted at the expense of the users and customers of the IT (who have no choice other than to live with the problem), often making this situation the sustainable state of the problem and the state most enterprises and their stakeholders are stuck with. 

The correct solution is to divide the responsibilities to place the authority in the right organizations or in other words where the associated functionality is supported by the correct skills, abilities, knowledge and experience and just solve the silo problem. But since very few people know how to solve the silo problem, but they certainly know how to protect their turf, the correct solution is seldom applied and the functional silo problem is fostered. 

So the problem will continue until leadership wants to accept the correct solution and set aside the fear of losing their kingdoms. Good leaders do the best thing for the enterprise, not the wrong thing to protect their best interests.

 

Managing The Nature of an Enterprise Problem

 

The question is why is IT such a love-hate relationship, something we can’t live with and can’t live without? And don’t think the answer is to simply employ the right leaders to do the right things as the problem is much more complex than that. There’s yet another part to managing the problems with IT projects. This is the part of the problem which lives within the land of enterprise architecture and why it takes the involvement of a qualified enterprise architect to help the project manager manage the problem with IT project management as it requires a holistic perspective of the enterprise, the problem and the solution, and this part of the problem is as wicked a problem as wicked problems get.

 

This final part of the problem with managing IT projects is caused by the nature of an enterprise. An enterprise is a system of systems (SOS). This means the enterprise is a system, and an enterprise has systems, and the systems in the enterprise have systems, and these systems may have more systems and so on. When you begin looking at the natures of the enterprise as boxes of people problems, inside boxes of IT problems, inside a boxful of enterprise problems, this is what I refer to as a wickedness of wickednesses (WOW), or in other words, what can be considered the “life within” Complex Adaptive Socio-Technical Systems (CASS).

 

CASS shows itself the best in the typical problem of paying off technical debt within the IT business network, an extremely wicked problem suffered by many an enterprise, as solving the problem requires changes in the best interest of some organizations of the enterprises at the expense of not being in the best interests of other organizations of the enterprise. Therefore, solving the technical debt problem and the IT problem requires a very good understanding of an enterprise and how things really work and evolve within it.

 

I could spend a lot of effort on detailing the differences in the 5 system characteristics (autonomy, belonging, connectivity, diversity, and emergence) that distinguish a SOS from a system, but for the sake of this discussion, I will “boil it down” to a simpler summary that is all an executive or project manager needs to know.

 

In a system, the architect of the system has ownership (responsibility and authority) over the elements of the system. The system architect can specify requirements for the system, allocate them down through the elements, and although needs to be “negotiable” in settling issues with the allocations, ultimately has the authority to demand requirements be met and the authority to accept or reject a non-conformance.

 

In a SOS, the architect of the SOS may have some level of responsibility over the elements of the SOS, but has no level of authority over them. Because of the “natural” lack of authority, this is where the functionality of the IT development environment begins to fall apart and where WOW blankets the enterprise architect, the IT architect, and the business architect in performing their responsibilities.

 

Now to pull this all together in what this means in the IT world, comes the devil in the details, the really difficult part of the problem an executive or project manager must take the time and effort to understand in order to solve the problem.

 

The enterprise architect has no authority over the IT system and along with the ITA and BA having no authority over the people developing and using the system, the EA, ITA and BA are all in a very poor position to meet their responsibilities over the Enterprise IT system network. It is this lack of authority by the people that have the responsibility which leads to why IT is unable to meet its commitments to its customers, users and stakeholders.

 

It is not because any one person or leader purposely keeps IT from meeting its commitments, but rather, it is the “nature of the enterprise” itself to keep IT from meeting its commitments, a wickedness of wicknesses of the highest order.

 

The solution therefore lies in gaining the ability at many levels of the governance hierarchy to achieve responsibility without authority and why the typical downward flow of authority through the governance hierarchy that solves other problems in the enterprise is not sufficient at solving the IT problem. IT must be successful in being influential to meet its commitments, it simply does not have the authority it needs to do it. IT does not have the authority it needs to command it. IT can only exert influence and the influence must be accepted for IT to meet its commitments. IT must be an influencer of the business, not a manager of the business. IT must exert real leadership, not forceful management to meet its commitments.

 

 

I’m sure by now many are as confused as can be, understand maybe a fraction of what was read, and are suffering from a headache that can only be compared with the problems in managing IT projects.

 

Well what did you expect, a simple solution to a very complex problem? Unfortunately no simple solution exists but the good news is the solution does exist, and as soon the problems are faced by making sure the right talent is in the right places, doing the right things at the right time, it is possible to solve what is thought to be impossible to solve.

 

Instead of continuing to accept the cost and the agony of handling IT the way you do, instead of continuing to make the problem worse every time you try and fix it, instead of using up your entire budget fixing what you've done instead of moving forward to something new and better, please take the time to read this article again. Until you do the right thing about managing IT project problems, the enterprise will continue to suffer the consequences until a competitor wins your customers. It is only by understanding the complexity, will the investment in IT development pay for itself through increased income and profit benefiting the business, instead of an expense dragging it down into a loss of money and reputation.

 

In summary:

• Executive leadership and IT project managers must properly lead and foster cooperation, collaboration, and healthy business habits.

• IT projects must begin with upfront systems engineering, management analysis and planning, quality assurance and risk and opportunity management.

• Projects must utilize qualified, knowledgeable and experienced project managers supported by a qualified, knowledgeable and experienced enterprise architect.

• The enterprise and the projects and the supporting operations within, must be integrated by:

  Integrating the architectures of the enterprise

  Integrating the governance of the enterprise

  Integrating the flow of work in the enterprise

 

Successful Strategic Transformation Initiatives can happen. Once a truly holistic and thorough approach is taken, which always does all the best things, with all the best people, at all the best times, in all the best interests of the enterprise as a whole, architecture initiatives, governance and the flow of work will be integrated across the enterprise, and the enterprise will then be integrated. When the enterprise is integrated, the operations of the enterprise will operate correctly and produce the correct things, for the correct cost, in the correct amount of time. When the correct things are produced for the correct cost within the correct time the customers, users and stakeholders experience the value expected and are satisfied with what they got, for the cost they paid, in the time it took to get it.

Managing the Headache

Managing IT Project Problems