Are your IT decisions based on anecdotal fallacies?
“The anecdotal fallacy uses a personal experience or an isolated example instead of a sound argument.”
Would you ride in a plane with a pilot who was "flying blind"? Consider that for a moment. Every modern aircraft is fitted with technologies that monitor vital data and alert the flight crew whenever an indicator deviates from predetermined limits. Wind direction, air speed, air pressure, altitude and amount of fuel left are all monitored by these systems. Small changes in these indicators, if not addressed in a proactive and timely manner, could lead to major issues for a flight.
The same can be said for your IT decisions. When it comes to making IT decisions, are you “flying blind? From where do you get the information and data on which you base your IT decisions? While anyone can agree that IT decisions should be based on the needs of the business, how should this information actually be collected?
“Are you sure that the requests of those managers and staff that make themselves heard are more important than the hundreds of other issues that you have in your backlog?”
While intuition has its place, too often decision-makers rely on their “gut feelings” when making important IT decisions. These “gut feelings” are often based on information gathered from meetings with the top business management and from day-to-day interactions with managers and staff.
An important drawback of relying too much on anecdotal input from the top business management for IT decisions is that it often is not possible for them to know the full scope of the challenges faced by all business leaders and users throughout the organization.
At the same time, information gathered through informal discussions with managers and staff often lead to just listening to “those who scream the loudest”. Are you sure that the requests of those managers and staff that make themselves heard are more important than the hundreds of other issues that you have in your backlog?
As companies increasingly digitalize their business in order to meet their customers and partners’ needs for more flexibility, individualized products and prices, IT is taking a greater and greater share of the company’s overall resources. It therefore becomes increasingly critical to success that the whole organization is heard before making important IT decisions.
So how can bias in information gathering be avoided?
Create a structured process to regularly listen to your IT stakeholders. This should be designed to discover otherwise unvoiced concerns, assess the most important needs, help you detect problems early and uncover future opportunities.
Take into account all stakeholder groups. This will help you get a more comprehensive view of the situation and consequently a much sounder basis for decisions than if only a limited set of stakeholders would be included.
Tailor questions to different stakeholder groups. Different stakeholders have different knowledge and interests. Business leaders will for example be more interested in and have more insight into strategic issues while regular users would be more interested in issues that affect their day-to-day work.
Consider using an external provider. In-house surveys are a good start to unleash the power of a structured inclusion of stakeholders to make your corporate IT more competitive and agile. However, one downside is that you will not be able to compare your results to other similar companies and organizations. You cannot really know if a result is good or bad unless you put it in a context. Another benefit of using an experienced provider is that you get access to a survey design that has been developed and evolved from the input of many different companies and organizations. You get the confidence of using a tried and trusted method. Also, respondents who can be sure that they can answer anonymously are more likely to give honest feedback.
Make sure results can be evaluated correctly. When interpreting results, it’s important to understand what constitutes an anomaly. If a result is good or bad can really only be decided if compared to other organizations.
Make sure that issues can be prioritized. Any survey with stakeholders will reveal a multitude of issues and problems. Your method must be able to also identify how those should be prioritized.
Do follow-ups. It is recommended that you do follow-up surveys at least once a year. This is to make sure your actions result in the expected improvements and that you can identify new areas that need attention. The goal is to achieve a continuously improving organization.