Analysts have tough jobs and are often exposed to considerable time pressure. The results of their analyses must be professionally researched to be convincing and reliable. Sometimes their queries result in answers of enormous complexity, yet they must be reported in a comprehensible and concise form. In addition, OSINT analysts often have to deal with hidden data sources the discovery of which being a time-consuming job requiring a considerable degree of insider knowledge.
We accepted the challenge and collected feedback indicating that analysts are tired of spending hours going through search engines manually just to find irrelevant and poor-quality results. Many analysts are frustrated and stressed by the sheer number of confusing data and browser tabs they have to handle.
What if there was a more efficient solution to their problem by using Federated Search?
In this article we’d like to familiarize the analyst with the approach of using Federated Search to solve his/her search problems, corroborating its advantages by presenting figures from R&D. Eventually, we will reveal Traversals methodology to increase even more the value already added by using a conventional Federated Search.
Federated Search is the technology used to simultaneously search multiple data sources, finding the most relevant and pertinent information in the shortest possible period of time. Instead of manually sifting a score of data sources e.g. Google, Facebook, Yahoo, or others, manually, Federated Search provides one single unified search interface
Federated search is the opposite of having dozens of unorganized browser tabs.
Efficient OSINT analysis requires Federated Search including the following data sources:
Especially at the beginning of an analysis task one is happy about the broad coverage. Maybe you are lucky and find the jackpot in a data source where you would not have searched or which you did not know before.
Once you enter the keywords in the federated search, it will query all connected data sources in parallel. Your one search is converted in the background into multiple searches against the different data sources.
The advantages of a federated search are already obvious.
Especially from executives we often get the question where the added value is compared to Google.
“Why is it better than using Google alone and why should we pay for it if Google is for free?”
The following numbers show how important a federated search is for your costs and quality.
Susan Feldman explains in her IDC White Paper the cost of not finding information:
CMSWire’s research signifies the importance of a federated search. According to their research:
Even a classic federated search that only unifies data sources makes your analysis work more efficiently and increases the quality of your reports.
The entire experience of our developers and advisory board is used to create an advantage for you. Traversals’ Federated Search is heavily supported by AI algorithms.
Starting the research process, the analyst very frequently runs into language problems, since the information searched or retrieved or both very often are in an unfamiliar foreign language. To overcome that language barrier – as demonstrated in our Yemen Covid-19 analysis (….) – it is crucial for the search system to be able to handle as many foreign languages as possible.
Traversals’ Federated Search has integrated a powerful machine translation that allows you to search and understand multilingual content. This feature made it possible for our analysts to look for Arabic Covid-19 news in Yemen without speaking the language.
Getting back hundreds of results from dozens of data sources does not make your analysis job easier. Each data source has its own way of ranking the results, e.g. Twitter sorts by time and Google by generic relevancy. An inconsistent result ranking, however, blurs the analysis process and considerably reduces search and retrieval efficiency.
A lot of AI-based Natural Language Processing modules are running in the background to enrich search results with information that is not provided by the data sources, such as text sentiment or text entities. This extra information is used to estimate a ranking for your context. The analyst can thus focus on the top places of the result list and will not suffer a disruption of the analysis flow.
Especially when it comes to foreign language information, analyses become very time-consuming and thus expensive. In order to not miss any important information, results must be translated into the language you understand. This is a manual process done with Google Translate.
It takes approximately 10 seconds to process a single foreign language result. For 100 hits, it, therefore, takes at least 15 minutes. With more data sources and larger result lists, concentration decreases whereas analysis overhead (e.g. managing multiple browser tabs) increases. The average processing time can thus quickly double or even triple.
Our Federated Search includes a machine translation and other powerful text processing and optimizations automating most of the manual work. This results in time savings of up to 90%. You will be up to 10 times faster by using our Federated Search and improve the reliability and quality of your results.
Especially in the OSINT area new data sources, such as new search engines or data services, are secretly published weekly. It will cost you a lot of free time to read all OSINT blogs just to keep an overview and to think about the integration into your analysis workflow.
The modular structure of our Federated Search allows efficient and transparent integration of new data sources. Our engineers and advisors are constantly looking for new data sources to be integrated in our system, while the analyst can fully devote his/her attention to his genuine job.
Our Federated Search is part of various research projects. Together with smart researchers, we discuss new concepts and new support measures to make the search even more efficient and powerful.
As you can see, the Federated Search is a crucial element, and more than important for enterprise businesses, large corporations, and governmental organizations:
Our Federated Search is one of the driving modules in our solutions like Strategic Intelligence, Medical Intelligence or Competitor Intelligence. Start using our Federated Search to benefit from all the advantages.
Copyright © 2023, Traversals Analytics and Intelligence GmbH. All Rights Reserved.