Introduction to Competitive Intelligence

In today’s information age, digital technologies have revolutionized all our life aspects, from personal life to learning and ending with how we do work. One of the most significant impacts is on the business sector. 

Nowadays, organizations have access to large repositories of data about their customers, markets, industries, and themselves! Having the correct tools to analyze and interpret this data became vital to achieving your business’s success and longevity. There are advantages of using this data in the organizational context.

For instance, keeping track of competitor’s 

• products, 
• services,
• patents,
• marketing campaigns, 
• strategies, 

and everything they use to attract and retain customers become crucial for the success of any organization to achieve the following two central goals:

1. Better decision making
2. Increasing revenue while decreasing the risk

In this article, we will define the term Competitive Intelligence (CI), mention its primary sources, discuss its goals and challenges, check whether CI is a legal practice or not and explain why automation is necessary to run CI.

However, before we begin, let us see the difference between CI and other intelligence types.

Market vs Business vs Competitive Intelligence  

There are different forms of organization-scoped intelligence that are often mixed by many people. The most well-known three types of intelligence used in the business world today are:

1. Market Intelligence (MI): This is about gathering and analyzing data about the markets where the organization is located or wants to enter. The results of MI are different market metrics, such as market segmentation, possible opportunities, trends, competitors in general, and other metrics that help the organization to have a holistic view of the market’s state as a whole. MI is often provided by market research institutes, such as GfK.
2. Business Intelligence (BI): This is a set of programs, processes, and technologies used to analyze internal business data and transform it into actionable insights. For example, BI tools are used to analyze business data and display the findings in organized reports, which help management to improve productivity and gain intelligence about the organization’s current state based on analyzed data. There are well-known solutions available, such as Tableau, Microsoft Power BI, or IBM Cognos Analytics.
3. Competitive Intelligence (CI): It is the process of gathering and analyzing information from various sources about your competitors in a legal way. The collected information can help an enterprise to improve its strategy and make better-informed decisions. CI does not apply to one industry type, as any organization can adapt it to its own needs to improve its critical decision-making process. For instance, telecommunications, big stores, pharmaceuticals, automotive, and IT companies all benefit from CI. Especially, bigger groups tend to employ their own CI analysts or to acquire services provided by specialized consultants.

In this article, we focus only on the aspects of Competitive Intelligence and the next-generation version of it.

Is Competitive Intelligence legal?

CI is considered a legal practice because it involves gathering information from open sources (e.g., government database, public records, social media posts, …).

Many people still think that CI is similar to corporate espionage; however, this is not accurate. Corporate espionage tries to collect information about the competitor using illegal or unethical ways.

In our past, we published various articles on Open-Source Intelligence (OSINT) and how it can be used to analyze organizations in a legal way.

What are the goals of CI?

Having a competitive intelligence capability helps businesses achieve the following objectives:

1. Understand what competitors are going to do before they do. For example, if your competitor’s CEO is talking about the benefits of using new technology, consider this as an alarm. Your organization should prepare a counterpunch to avoid losing the race. 
2. Learn from competitor’s successful strategies to adopt them in your organization. For example, a competitor may launch promotional camping addressing something needed by the customer that you have missed before.
3. Understand your competitor’s strengths and weaknesses. For example, by monitoring what customers say about your competitors on social media sites, such as compliments or complaints about their products/service. Your organization can use this knowledge to tailor your advertisements and marketing efforts.    
4. Handle threats against your business more efficiently. When you have a deep understanding of your competitors’ work and strategies, you can predict their future actions that may negatively impact your market share and thus work to avoid it before it poses a threat.
5. Improve your product strategy. CI helps you understand your market needs, customer preferences (e.g., through social media intelligence) and tailor your marketing campaigns accordingly.
6. Update your pricing and product packaging strategy. For example, when your competitor decreases their price for the same product or service you are promoting, then you have to respond fast and see if you can adjust your price offering to remain competitive.

A List of Data Sources for CI

CI data can be aggregated from a variety of sources. In the following, you will get a list of pretty obvious data sources:

1. Competitor’s website: This should be the first place to begin your research. Of high interest are the news pages indicating new contracts, products, services, and many more.
2. Annual reports: These reports often contain information on suppliers, trade partnerships, strategies, … In most cases, you will find those online. 
3. Social media: Company accounts on services, such as Facebook, Twitter, or LinkedIn, are often used to announce new products or explain features in more detail.
4. Premium databases/services: These paid online services offer a vast array of information about businesses. Examples of such services include CapitalIQ, InsideView Insights, ZoomInfo, or Factiva.
5. Patent databases: Google patents can reveal some information about products under development by your competitors and their innovation strategy.
6. Job posting websites: Those websites may reveal important information about competitors, such as the type of technology used for their products/services, new locations, and many more.
7. Newsletters and Trade Magazines: This might be information either provided by competitors themselves or by professional websites. Usually, you can read it in the form of emails or RSS feeds.

All mentioned data sources contain pre-processed and carefully selected information. From our point of view, this is only scraping the surface! 

What about all the information stored in the Deep Web, such as:

• Foreign and topic-specific blogs
• Local and foreign-language newspapers
• Documents published on conference web sites
• Information published on social media by non-company accounts
• …

This is where it starts getting interesting as you can learn some real insights and create a substantial benefit of running Competitive Intelligence. This is what we call Competitive Intelligence 2.0 and what requires a high level of automation and Artificial Intelligence.

Challenges with Competitive Intelligence

The benefits of Competitive Intelligence can be recognized clearly; however, it has some challenges especially when it comes to analyzing the Deep Web:

1. Knowing all the data sources applicable for Competitive Intelligence and understanding how to gather intelligence from those and fuse it with other puzzle pieces requires Open-Source Intelligence experienced analysts which are hard to find. How many analysts are really familiar with all the OSINT tools?
2. The massive volume of data retrieved from all the data sources makes analyzing it an overwhelming task. Without using the proper tools to filter and analyze collected data it is almost impossible.
3. Dealing with duplicate and repetitive information is a really inefficient process to begin with and shouts out for further automation and machine support.
4. Interpreting information in foreign languages retrieved from local newspapers or social media is a daunting task and time-consuming. Either it requires native speakers or machine translations.
5. Dealing with situations that change every day requires an almost real-time Competitive Intelligence process. Getting information in time requires a high level of automation.
6. Forwarding intelligence to the right stakeholders is a challenge for each analyst. A new patent might be of interest to the R&D department, but not for the campaign manager. Spamming your audience with all information is a bad strategy.
7. Staying compliant with GDPR and your own ethical ruleset is mandatory for anyone interested in Competitive Intelligence. Especially, when it comes to social media it happens really fast that you have to deal with personal data.

Without having a tool for automation and assistance, it seems like an impossible task! This is where we can assist you with our Data Fusion Platform which has been designed to solve these challenges.

Key Findings

• Competitive Intelligence provides the needed information for businesses to make intelligence-driven decisions. 
• The massive volume of data and the difficulty of interpreting, organizing, and visualizing search results remain significant challenges. 
• Leveraging automated solutions is the answer to most Competitive Intelligence gathering challenges.