Problem types in organisations
Introduction
Our work with organisations typically starts by clarifying the needs required by asking about the desired outcomes, the problems, the people involved, the situation surrounding the task such as the history, priority and resources, and how typically this type of challenge would be addressed – what process or system might be used. We call this approach Task Appraisal.
There are two recurring challenges that we find early on during these Task Appraisals:
A wide variety of uses and meanings attributed to words such as strategy, leadership, vision, teams, creativity, problem solving, innovation and change which causes social and economic issues such as confusion, misunderstandings, inefficiencies and ultimately increased costs.
A lack of clarity in expressing the desired state and understanding the situation, challenges and problems combined with high desire to take action quickly – in short a desire to make progress on potentially the wrong problem.
The terminology issue
‘The test of a first-rate intelligence is the ability to hold two opposing ideas in mind at the same time.’
– F. Scott Fitzgerald
Variations in language are often a block to people working together because over time, different disciplines and industries diverge in their use and meaning of the same terms. In the fields we work in – strategy, leadership, teams, creativity, problem solving, innovation and change, this is often a very early challenge that we need to address. Looking at some of the concepts and our approach to finding common ground on which to move forward productively, it is often a case of holding more than one (and often opposing) viewpoint in mind at one time. We sometimes refer to this as the need to be ‘ambidextrous’ or ‘Janusian’ after the Roman God Janus, who had to look in opposite directions at the same time.
For example, when we ask people to generate words to describe ‘creativity’, they often respond with words such as: new, divergent, exciting, wacky, artists, advertising, music, art and unique. When we ask what words they associate with ‘problem solving’, they say: analysis, useful, focusing, overcoming difficulties. Most people acknowledge the tension between these two concepts – the newness and aspirational aspects of creativity contrasted with the more ‘here and now’, useful and reactive notions of problem solving.
Our Janusian approach has been to try and link this tension between newness and usefulness. Newness for its own sake is not necessarily useful, but put the two together and a powerful approach to many (most) situations becomes possible.
We accept that there probably are some differences between these terms, although we suspect many are socially situational. For example, more technical professions such as engineering tend to use the term ‘problem solving’ whilst in advertising and the media the preferred term is ‘creativity’. ‘Change’ is often used by human resources, whilst government and industry tend to use the term ‘innovation’. However, from a practical perspective we see the terms ‘change’ and ‘innovation’ as embracing the concepts of ‘creativity’ and ‘problem solving’ as they all closely relate to thinking style and they all involve the elements of newness and usefulness – at least to the creator.
TENSION BETWEEN THE CURRENT STATE AND DESIRED OBJECTIVE
Ultimately, our approach can be conceived as trying to balance moving away from a current perceived reality towards a desired future objective or vision. If there is a tension between what is and what we want, we will seek resolution. For example, if we are hungry (tension) we usually resolve the tension by eating. Tension is always composed of at least two elements in which there is a contrast, discrepancy, gap, lack, need or problem.
For example, in the case of hunger, there is a discrepancy between the amount of food the body wants and the actual amount of food the body has. In the beginning of the tension-resolution cycle, the difference is pronounced. By eating, the difference lessens, and finally the actual and desired amount of food becomes the same – this is the point of resolution at which the original discrepancy is eliminated.
Managing problems
‘We fail more often because we solve the wrong problem than because we get the wrong solution to the right problem.’
– Russell Ackoff
Detecting the problem is as important as finding the answer – indeed, some argue that defining the problem is more important. Certainly, we agree that how we view the problem determines how and if we will solve it. Based on concepts spanning over 30 years, it is possible to organise problems into four types: tame problems, messy problems, wicked problems and wicked messes. Problems being the gap in achieving the desired goal, objective or purpose – a situation, condition or issue that is unresolved. In a broad sense, a problem exists when an individual becomes aware of a difference between what actually is and what is desired.
Why are these distinctions useful? To address these four problem types in further detail, the remainder of this article references Dr David Hancock (2004, 2010), with permission.
Strategies for solving tame problems differ qualitatively from strategies appropriate for messes. Messes are puzzles – rather than solving them, we resolve their complexities. In turn, solving and resolving both tame problems and messes differ qualitatively from strategies for dissolving the barriers to consensus implicit in wicked problems.
A more compelling reason is not that solving the wrong problems fails to solve the right problems, but that by solving the wrong problems, we unwittingly undermine what it takes for us to solve the right problems. The danger is not that we fail to build bridges across the right rivers, rather that we destroy the materials needed to build bridges across those rivers.
The four types of problem
Tame problems
Do not confuse tame with simple, we solve tame problems through analytical methods, breaking things down into parts, fixing components and assessing the probability of known sequences of failures leading to an accident. We organise ourselves to solve tame problems through specialisation. Culturally, tame problems enjoy consensus: everybody pretty well agrees why something needs to be done and the right way to go about doing it.
Linear ‘tame’ problem solving process:
Gather data > Analyse data > Formulate solution > Implement solution
There are countless examples of tame problems, solving them has been the great forte of science for several hundred years. Due in large part to such successes, tame problems remain the ideal for many social scientists as well as managers and administrators.
Messy problems
We are increasingly faced with problems of organised complexity, clusters of interrelated or interdependent problems, or systems of problems. Ackoff labelled this system a ‘mess’ – problems, which cannot be solved in relative isolation from one another form ‘messes’.
We sort out messes through systems methods, through focusing on processes and through interdisciplinary approaches. Rather than simply breaking things down into parts and fixing components, we examine patterns of interactions among parts. We organise ourselves to sort out messes through such things as cross-functional groups, redundancy, and so-called ‘learning organisations’ (Senge 1990).
Many examples illustrate the concept of messes. Simply building more motorways doesn’t solve vehicle congestion – a risk in mistaking a mess for a tame problem is that it becomes even more difficult to deal with the evolving mess. Karl Weick (1979) noted that “Managerial problems persist because managers continue to believe that there are such things as unilateral causation, independent and dependent variables, origins and terminations.”
In his book Normal Accidents (1999) Charles Perrow details some of the problems inherent in messes. Firstly, ‘interactive complexity’ is the degree to which we cannot foresee all the ways things can go wrong. This may be because there are simply too many interactions to keep track of. More likely, it is because our various theories are simply not up to the task of modelling socio-technical interactions and ‘coupling’ – the degree to which we cannot stop an impending disaster once it starts. This may be because we don’t have enough time, because it is physically impossible, or because we don’t know how. The greater the degree of interactive complexity, the less our capacity to prevent surprises and the greater the degree of coupling, the less our capacity to cure surprises after they occur. Therefore, the greater the degree of interactive complexity and coupling, the greater the likelihood that a system is an accident waiting to happen – what Perrow terms a ‘normal accident’.
In such systems, ‘operator errors’ merely serve as triggers. Strategies for dealing with messes are therefore quite different from those appropriate for tame problems. Strategies logically follow from the ways problems are conceptualised. Thus, increasing our capacity to prevent unanticipated interactions among components entails simplifying systems (‘KISS’); increasing our capacity to cure them entails de-coupling major components (e.g. building in longer response times).
FOUR PROBLEM TYPES
Wicked problems
In short, strategies for dealing with messes are fine as long as most of us share an overriding social theory or overriding social ethic. If we don’t, we face ‘wickedness’.
Wicked problems are referred to as ‘divergent’ as opposed to ‘convergent’. A convergent problem promises a solution – the more it is studied, the more various answers sooner or later converge. Tame problems are convergent by definition. Messes are convergent if we agree on what overlaps, on appropriate strategies, and on the kind of ‘climate’ we wish to maintain. A divergent problem does not promise a solution. The more it is studied, the more people of integrity and intellect inevitably come to different solutions.
As with messes, there are very real dangers in solving the wrong problem. Mistaking or misrepresenting wicked problems for messes, let alone tame problems, almost inevitably leads one to conclude that those with different answers lack integrity, intellect, or both. The great danger is that such conclusions undermine trust, and trust is a fundamental strategy for collectively coping with wicked problems. If wicked problems are becoming more common in our modern era, and there is compelling evidence that they are, we face a strategic choice:
We can continue to misrepresent them as messes or tame problems, hoping they will not degenerate
Or we can acknowledge wicked problems for what they are and try to stabilise them as ‘conditions’.
This is not going to be easy because wicked problems offend our sense of logic and our common beliefs even more than messes – in our modern times, it is difficult to accept that a problem has no solution. Given that many people care about or have something at stake in how the problem is resolved, the process of solving a wicked problem is fundamentally social and solving a wicked problem is fundamentally a social process. Most wicked problems involve lots of stakeholders.
Wicked messes
Much research has been done on behavioural complexity and dynamic complexity – what confuses real decision-making is that the two co-exist and interact in what we call ‘wicked messes’ (Senge & Roth 1996). The fact that problems cannot be solved in isolation from one another makes it even more difficult to deal with people's differing assumptions and values: people who think differently must learn about and create a common reality, one which none of them initially understand adequately.
Systems of interlinked problems interact with the misunderstandings, divergent assumptions, and polarised beliefs of different groups of people. Improving communication and trust among different camps is not enough – people are still likely to focus on symptoms rather than deeper causes and pursue low-leverage changes. Conversely, even if deeper understanding of the systemic forces at play is achieved, such understanding will be viewed with suspicion by the different, competing interests and mental models.
Solutions to wicked messes
The main thrust to the resolution of these types of problems is stakeholder participation and ‘satisficing’ (Simon, 1956). This is because resolving wicked problems produces no single optimum solution, but has many alternative satisfactory solutions. Therefore the production of that solution must be ‘boxed’ or ‘bounded’ either by time or financial constraints to avoid it going on ad infinitum.
How do we propose to deal with wicked messes? Wicked problems are synonymous with what Geertz (1973) terms “a loss of orientation” or what Rittel and Webber term the absence of an “overriding social theory or an overriding social ethic.” Thus, wicked messes are evidenced by the ideological controversies that result when the boundaries of messes expand to include socio-political and moral-spiritual issues. So-called empirical studies and the social sciences are necessarily shot through with implicit and explicit value assumptions and ideological considerations.
‘Wickedness’ occurs when people confer immutability on value assumptions and ideological considerations. Thus, the strategic issue is whether we choose to:
Allow wicked messes to degenerate into chaos
Stabilise them as ‘conditions’, or more radically, try to dissolve them together.
Management AND Leadership
It is key to draw a distinction between management and leadership. Kotter (1990) states that leadership is different from management and that we need both.
Management is about coping with complexity, its practices and procedures are largely a response to the emergence of large organisations and complex projects.
Leadership, by contrast, is about coping with change.
These different functions (coping with complexity and coping with change) shape the characteristic activities of management and leadership. Each system of action involves deciding what needs to be done, creating networks of people and relationships that can accomplish an agenda, and then trying to ensure that those people actually do the job. But each accomplishes these three tasks in different ways. Therefore, how might we choose when we require the skills of leadership or management?
APPROACHING THE FOUR PROBLEM TYPES – MODEL 1
If we now revisit the ‘Four problem types’ graphic (above) and look at the characteristics required to address these issues, we can see that those required to achieve change are predominantly behavioural-based, requiring needs and emotions to be satisfied. This shows a need for a degree of influence rather than power to ensure their resolution, based on the development of trust and relationships, and indicating a high degree of behavioural complexity. This allows us to segment the matrix into two where leadership qualities dominate solutions in the upper half and management qualities the lower – as shown in the diagram ‘Approaching the four problem types – Model 1’.
The correlation between risk and problem types
In the past there has been a considerable difference of opinion between the use and legitimacy of qualitative and quantitative risk assessment (Oldfield 2001). We propose the following hypothesis to why this may have occurred.
If we accept the preceding classification of types of problem and then analyse how risk analysis may be applied to help us with each type of problem, we can see that as we move further to the right of the matrix there appears to be an increased reliance on the use of systems thinking with the emphasis on the interconnectivity of events. As we move further towards the top of the matrix, there seems to be an increased reliance on social science type solutions (where people do not necessarily behave as ‘soft systems’ but their behaviour is dominated by belief systems based on emotions and feelings). If we then apply our two types of risk assessment (quantitative and qualitative), and overlay them on the matrix we produce the diagram shown below (Approaching the four problem types – Model 2).
APPROACHING THE FOUR PROBLEM TYPES – MODEL 2
We can observe that for the bottom sectors where scientific-based solutions are dominant, quantitative risk data is reliable in the assessment of risk. Our assumption here is that there is an optimum solution and therefore an underlying number that we will converge towards. Thus by the generation of more and more data we will move closer to the underlying ‘truth’.
However, as we move up towards the behavioural sector there is no underlying single solution – it will be dependent upon the feelings and ideologies of the people concerned in the assessment. By the use of quantitative data to inform facilitated workshops, qualitative results will ‘reveal’ those underlying beliefs which will lead to ‘satisfactory’ solutions. Therefore, it is critically important to involve all stakeholders who can influence the success of the project and to attempt to limit your boundaries to the case in hand – otherwise the group will grow beyond the limits of achieving a workable solution.
Resolving wicked messes
Based on experience we suggest there are predominantly three ways of dealing with wicked messes, based upon the distribution of social power within the group.
Social power
Cartwright (1959) states that “power is inevitably a part of the accepted phenomena of social psychology.” However, the task of documenting what power actually is, or how we define it, is made difficult because of the ambiguity concerning its boundaries. For ease we will use the bases of social power as illustrated by French & Raven (1959), who define power in terms of influence and influence in terms of “psychological change”, which is defined as the “alteration of the state of some system over time”. Social influence and power in this case is limited to the change effected by an individual (P) over another individual or group of individuals (O).
The basis of power
By the basis of power we mean the relationship between O and P, which is the source of that power. French maintains that there are five bases for O’s power:
Reward power – O’s perception that P has the ability to mediate rewards for him
Coercive power – O’s perception that P has the ability to mediate punishments for him
Legitimate power – The perception by O that P has legitimate right to prescribe behaviour for him
Referent power – O’s identification with P
Expert power – The perception by O that P has some special knowledge or expertise.
For all types of power the stronger the basis of power the greater the power. Some negative aspects are produced by coercive power in that it results in an increased resistance towards the individual exerting that power, and conversely reward power produces a reduction in that individual’s resistance. This can be offset by the fact that the more legitimate the coercive power the less it tends to produce resistance. Reward power will also increase referent power over time as it increases the attraction of O towards P. Whilst most of these powers are broad ranging, expert power is limited to specific areas and use of expert power outside of these areas can undermine and reduce the effect of that power.
Identification of stakeholders
Freeman’s (1984) definition of what constitutes a stakeholder is “any group or individual who can affect or is affected by the achievement of an organisation’s objectives”. This was further subdivided into classes based on (Mitchell 1997):
The stakeholder’s power to influence the firm
The legitimacy of the stakeholder’s relationship with that firm
The urgency of the stakeholder’s claim on the firm.
This can be distilled into stakeholder ‘salience’ to explain how managers prioritise their stakeholder relationships. Where salience is defined as “the importance the individual attaches to supporting or opposing the objective relative to all other decisions with which the individual is concerned,” (Coplin 1976). Therefore, salience is considered to be high where all three of the stakeholder attributes are perceived by individuals to be present.
Strategies for dealing with wicked messes
If power is dispersed among a small number of stakeholders then authoritative strategies can be used. These are taming strategies – they diminish the level of conflict inherent in wicked messes by putting problem-solving into the hands of a few stakeholders who have the authority to provide a solution, the so-called ‘experts’. The advantages are that reducing the stakeholders reduces the complexity. The disadvantages are that what is achieved may please none of the stakeholders involved and ‘experts’ can be wrong.
If power is widely dispersed but cooperation appears impossible in the relevant timescale then a process of muddling through is usually applied. This is where the project deals with problems as they arise on the basis of ‘whoever shouts loudest’ or makes the ‘strongest argument’. The project meanders through the process until it ‘arrives’ at a solution. Advantages are typically short-lived and the disadvantages are that the problem is never ‘fully’ resolved as the solution is constantly changing. Many political problems fall into this category.
If power is widely dispersed and time and consensus is not prohibitive then collaborative strategies can be used. These have at their core a win-win solution and provide ‘satisfactory’ solutions (Simon 1956). Alliances, partnerships and joint ventures are all variations on the collaborative theme. Advantages are numerous and have been catalogued in many other papers. Disadvantages are that they take a great deal of effort to manage and achieve synergy – dialogue can turn into debate and debate into procrastination with little to show for it and at the same time transaction costs can increase whilst the dialogue progresses.
Collaborative strategies are the most difficult to implement but the most rewarding. One way of achieving alignment of stakeholders is through the use of ‘customer experience’ i.e. what type of experience are you seeking for the end users. This focus can instil in the stakeholders a ‘motivating proposition’ for the project. Mapping stakeholders and looking for ‘common ground’ gives substance to what is acceptable, but more importantly what is unacceptable.
Conclusions
The first step in solving problems is to recognise the type you are facing. To help achieve this we have classified problems via a matrix with four main areas: tame problems, messy problems, wicked problems and wicked messes. We have also used this matrix to try to understand the behaviours required to deliver a successful project using management or leadership. We have attempted to overlay the concepts of risk management to use the right risk management techniques with the type of problem. We have put forward some coping strategies for dealing with wicked messes and the use of customer experience for alignment of stakeholders. Many situations are complex both behaviourally and systemically – these situations need to be identified and recognised and then for suitable solutions to be progressed using the correct tools. The solution to wicked messes lies in the people, by the use of discussion and forums for exploring the problems and understanding of the limitations of the solution. Accurate reporting of ‘near misses’ and the use of workshops to identify causes can aid this process. However, for this to be achieved requires an open culture, where an attitude of ‘no-blame’ exists. It also requires the pursuit of designs that limit system coupling and complexity where possible.
References
Ackoff, R.L, (1970) A Concept of Corporate Planning. New York: John Wiley & Sons.
Ackoff, R.L, (1974) Redesigning the Future: A Systems Approach to Societal Problems, John Wiley & Sons, New York.
Cartwright, D, (1959) Power: A neglected Variable in Social Psychology, Studies in Social Power, University of Michigan, Ann Arbor.
Coplin, W. & O’Leary, M. (1976), Political Analysis Through the Prince System. Croton-on-Hudson, NY, USA: Policy Studies Associates.
Degrace, P & Stahl, L.H, (1998) Wicked Problems, Righteous Solutions: A Catalogue of Modern Engineering Paradigms, New Jersey, USA: Yourdon Press/Prentice Hall PTR/Sun Microsystems Press.
Freeman, R.E. (1984) Strategic Management: A Stakeholder Approach, Boston, Pitman.
French, J.R.P. Jnr & Raven, B. (1959) The Basis of Social Power, Studies in Social Power, University of Michigan, Ann Arbor.
Geertz, C. (1973) The Interpretation of Cultures, Basic Books, New York.
Hancock, D.J. (2000) Risk – The Talking Cure, MBA thesis, University of Bath.
Hancock, D.J. (2004) "What new skills must project managers master to deliver successful project outcomes for the future?", 2004 IEEE International Engineering Management Conference (IEEE Cat. No. 04CH37574), pp. 944.
Hancock, D.J. (2010) Tame, Messy and Wicked Risk Leadership. Gower Publishing.
Isaksen, Dorval, Treffinger. (2011) Creative Approaches to Problem Solving. 3rd Edition. Sage Publications.
King, J.B. (1992) “Learning to solve the right problems: The case for nuclear power in America”, Journal of Business Ethics, 12 (2), pp.105-116.
Kotter, J.P. (1990 June) “What Leaders Really Do, Harvard Business Review.
Mitchell R, Agle, B. & Wood, D. (1997) “Toward a Theory of Stakeholder Identification and Salience: Defining the Principle of Who and What really Counts”, Academy of management Review, Vol 22, No.4, pp. 853-886.
Oldfield, A. (2001) Stakeholder Analysis as Decision Support for Project Risk Management, Surrey European Management School, Surrey University, UK.
Perrow, C. (1999) Normal Accidents: Living With High-Risk Technologies, Basic Books, New York.
Rittel, H. & Webber M. (1973) “Dilemmas in a General Theory of Planning”, Policy Sciences, Vol. 4, pp 155-169, Elsevier Scientific Publishing Company, Inc., Amsterdam.
Roth, G.L. & Senge P.M. (1996) “From theory to practice: research territory, processes and structure at an organizational learning centre”, Journal of Organizational Change Management, Vol 9 No.1, pp 92-106.
Senge, P.M. (1990) The Fifth Discipline: The Art and Practice of The Learning Organization, Doubleday, New York.
Simon, H.A. (1956) “Dynamic programming under uncertainly with a quadratic criterion function”, Econometrica, 24, pp. 19-33.
Weick, K. (1979) The Social Psychology of Organizing, Random House, New York.