A FAROUT Way to Manage CI Analysis
By Craig S. Fleisher, WILFRID LAURIER UNIVERSITY, and
Babette Bensoussan, THE MINDSHIFTS GROUP
Despite abundant data, most enterprises suffer intelligence deficiencies. A system for rating analytic tools could set things right.
Having trouble wading through the varied — and often puzzling — business, competitive, and strategic analytical models at your disposal? Ever tried one only to discover that the data couldn’t be “massaged” in ways to use it properly? Or perhaps you’ve undertaken a much-discussed method of analysis only to find it provided no real conclusions relating to your company’s future.
This article, as you now realise, is about the “ugly duckling” component of the competitive intelligence process — data analysis. We call it “ugly” because relatively few CI people write about it, not many want to talk about it, and even fewer claim to be expert at it. If you don’t believe us, compare the commercial availability and visibility of data analysis versus data collection services. Data collection methods are available to most takers, and data collection agencies are plentiful. For data analysis, true expertise is much harder to find. Based on their relative visibility, you would be hard-pressed to realise these two ducklings came from the same duck parents!
Why has analysis gotten the bad rap? There are a number of reasons why analysis is not among the most popular topics of discussion at the competitive intelligence dinner table:
ANALYSIS IS HARD TO DO FOR MOST PEOPLE. As in nature, people tend to prefer the path of least resistance when it comes to putting forth effort. In today’s turbo-charged digital world, it’s far easier to collect a lot of data than it is to figure out what to do with it.
FEW PEOPLE HAVE PUBLICLY RECOGNISED OR ESTABLISHED ANALYSIS EXPERTISE. Even those who are expert at analysis may not necessarily be able to teach others how they do it. Analysis skills can be developed over time as one grows in experience and knowledge, but analysis expertise requires a degree of tacit skill or inherent creativity that is “born and not made.”
THERE ARE FEW FRAMEWORKS FOR UNDERSTANDING HOW THE ANALYSIS COMPONENT CAN BE MANAGED AS PART OF THE CI PROCESS. Few individuals can thoughtfully explain how analysis can be successfully managed according to the “three E’s” of efficiency, effectiveness, and efficacy.
It’s our view, reflected in the findings of several large-scale CI surveys, that data collection is managed far more successfully than data analysis. In our experiences, we see a number of prevalent symptoms that suggest why analysis is not managed properly:
1. TOOL RUT — Like the man with a hammer who thinks everything looks like a nail, people keep using the same tools over and over again. This runs counter to the principle that to address the complexity of this ever-changing world, the CI analyst must look at numerous models to provide value.
2. B-SCHOOL RECIPE — Many charged with doing analysis come out of MBA programs where they have been offered tried-and-true recipes from instructors with financial and management accounting backgrounds. But competitive analysis is as different from accounting analysis as strategy is from cost accounting. This may help explain why few accountants lead CI functions and vice versa.
3. RATIO BLINDER — Most business people conduct analysis based on historical data and financial ratios. This can at best only provide a comparison and tell the analyst the size of the gap (the “what”) between two organisations on a particular data point or data set. It does not help the analyst explain the reasons why the gap exists or how to close it.
4. CONVENIENCE SHOPPING — Individuals frequently do analysis on the basis of the data they happen to have, as opposed to the data they should have. Because analysts have certain data at their disposal, they use the analytical technique that suits the data rather than focussing the analysis on the client’s question and/or the intelligence actually required. This is especially true when accountants are asked to do analysis; they provide outputs that only reflect financial manipulations.
Before we proceed further, it may be helpful to define “analysis” in a competitive intelligence context. We view competitive intelligence analysis as the multifaceted means by which information is interpreted to produce insightful findings (i.e., intelligence) or recommendations for organisational action. Defined as such, we do not view analysis as being solely in the domain of either pure art or science, but as requiring to some degree both of these. We also view analysis as being both a process (i.e., the multifaceted means we defined) and a product (i.e., the interpretation output). Lastly, our definition suggests that analysis must pass the “so what?” test in order to usefully aid decision-making and action-taking.
ANALYSIS MUST PASS THE “SO WHAT?” TEST IN ORDER TO USEFULLY AID DECISION MAKING AND ACTION–TAKING.
Analysis is important from a decision-making sense because it accomplishes a number of important tasks, such as providing:
• Connections among numerous and often disparate data.
This is often analogized as making a picture out of a multitude of jigsaw puzzle pieces.
• More time for decision-making.
Analysis allows decision-makers more confidence to proceed with decisions and lessens the perceived uncertainty, and often the risk, underlying these choices.
• A reduction in the number of input variables to consider.
One of the keys to removing decision paralysis is to focus on those critical few variables that underlie the key aspects of decisions.
• Context that relates data to the enterprise’s situation.
Analysis is vital in helping decision-makers understand how phenomena in the broader environment relate to their company’s mission, objectives, and strategy.
Working hypotheses about relationships between events and trends and organizational actions.
We’d like to propose an easy-to-use framework to help analysts rate the best methods or techniques for managing their particular analytical challenges. This rating system is based on the premise that for analytical output to be intelligent, and therefore valuable to business decision-makers, it needs to meet a number of common characteristics. The output needs to be:
Hence, we call it the FAROUT Rating System.* Failure to meet all these criteria to a satisfactory level will result in the analytical product being of little value to business decision-makers.
Let’s briefly describe each of the six elements below:
The past can be a dangerous predictor of the future. This is especially important if the present and/or the future have little in common with the past, as is increasingly the case in the business marketplace today. Intelligence must be prospective-oriented, looking both deeply and broadly at an indeterminate and uncertain future, and willing to take risks by being both predictive and inventive. Foresight is not gained by looking in the rear-view mirror or by using data pointing toward the past. The better analytical methods for CI will be future, as opposed to historically, oriented. They will be predictive in addition to being descriptive and explanatory.
The analyst should develop analytical products that aim for high levels of accuracy and credibility. Accuracy requires the analyst to give careful consideration to both the natures of the inputs being analyzed and the actual application of the analytical techniques themselves. Analysts need to consider the GIGO (i.e., “garbage in garbage out”) problem whereby poor inputs frequently produce poor outputs. High levels of accuracy are difficult to attain under a number of common conditions, such as when the data underlying the analysis:
〉Has come from only one source;
〉 Has not been cross-validated against both hard and soft information;
〉 Needs to be converted from sources in ways that it was not originally designed for; and
〉Comes from sources filled with high levels of bias in the first place.
Although achieving perfect accuracy is desirable in theory, it is often less desirable in practice and requires that the analyst make trade-offs against other conceptual and pragmatic considerations, including the other five FAROUT elements. In particular, some authors have suggested that accuracy or precision may often be less important than understanding or perspective.
ACCURACY OR PRECISION MAY BE LESS IMPORTANT THAN UNDERSTANDING OR PERSPECTIVE.
In order to do analysis, data needs to come from sources that not only cost less than the resultant output is worth, but also should not take too long to gather, rendering it stale by the time the decision actually needs to be made. When data being used in analysis comes from primary sources (i.e., most “human intelligence”, it potentially lowers the level of analytical accuracy, and it requires greater skill to elicit what is actually required from the primary sources. Nevertheless, many secondary databases may give great accuracy and timeliness but little in the way of a future orientation, despite their high price.
This relates to the presence of biases held by the analyst and/or company. Too many good analyses are clouded by cognitive or social biases, from “prior-hypothesis bias” to “groupthink.” To minimize the potentially destructive nature of these common biases, the data should be viewed and analyzed using a rational and systematic approach. In other words, successful analysis minimizes the destructive potential of analytical and decision oriented biases. This criterion also requires the analyst not only to avoid selectively choosing facts to provide support for pre-ordained conclusions but also to comprehensively consider all available data that might bear on the decision.
Almost by definition, valuable outputs must meet the critical intelligence needs of a decision-maker in a particular decision-making context. Valuable analytical output must be appropriate to the decision-makers responsibilities and organizational context. The key for the analyst is to develop outputs that are “need to know” and not “nice to know” and that meet or surpass the client’s critical intelligence needs. This criterion also suggests that the analytical outputs and process must be clearly communicated in a language that can be easily explained and understood by the recipient.
THE KEY FOR THE ANALYST IS TO DEVELOP OUTPUTS THAT ARE “NEED TO KNOW” AND NOT “NICE TO KNOW” AND THAT MEET OR SURPASS THE CLIENT‘S CRITICAL INTELLIGENCE NEEDS.
How long it takes the analyst to undertake the analysis will either hinder or assist the company’s use of intelligence. Much business information or competitive data has a limited shelf life and loses its value the longer it remains excluded from decisions underlying action. Certain methods of analysis may provide the intelligence required, but take far too long to develop. On the other hand, other methods of analysis may require little time but do not deliver the required features of objectivity, accuracy, utility, and resource efficiencies. Valuable analysis will provide decision-makers enough time to allow the company to implement the course of action recommended by the analysis.
The FAROUT Rating System
The table below demonstrates how the FAROUT system works on several methods of analysis. Depending on the circumstances of their assignment, an analyst would choose the most appropriate tool as measured against the six FAROUT criteria. For example, an analyst should not select Scenario Analysis for a quick, cheap, short-term project.
Mix and Max(imize)
Managing the analysis of business and competitive data is a difficult task, and we are not aware of any “10-minute analyst” books or software that can replace a good balance of both science and creativity. We do know that it is highly unlikely that good data analysis will be based on just one analytical method or tool. Rather, a combination of several techniques will be required.
Each analytical method has unique limitations, and these limitations multiply when placed in specific business contexts. Using the FAROUT system will enable the analyst to mix the appropriate analytical tools in order to maximize the intelligence value. The best analysts recognize and are sensitive to the limitations associated with any particular analytical technique. The sensitized analyst can address these issues throughout the whole of the competitive intelligence process to overcome the recognized limits.
To sum up: our objective in offering the FAROUT framework is to assist analysts. to ensure high intelligence value. If their analysis delivers on all six characteristics, analysts and decision-makers can be confident that the intelligence will pass the “so what?” test and make a difference.
You can’t be expected to know the entire competitive landscape well enough to correctly call ALL the shots. Within today’s complex, chaotic, and globally competitive environment – think VUCA world – the pressing need for sense-making, strategic thinking and improved...
The most critical strategic issue for any business is its competitiveness. No one would disagree with this yet few companies really spend time and effort to deeply understand and manage their future competitiveness. I am always amazed at how many organisations and...
Since writing our book Strategic and Competitive Analysis – Methods and Techniques for Analyzing Business Competition, we have often been asked to identify the most useful or insightful models for CI. Each model has its own strengths and weaknesses: selecting them...
Most people do some form of analysis on a regular basis throughout their lives - when they buy a house, a car, or go on a holiday, for example. Every business also does some form of analysis, whether it is looking at productivity improvements, benchmarking, cost...