Under pressure from civil society seeking more democratic transparency of surveillance operations in the aftermath of the Snowden revelations, lawmakers and governments in most NATO- and other democratic countries will, in the near future, amend their national legal provisions to improve the democratic oversight, control and supervision of data collection, monitoring and surveillance activities by their law enforcement, secret intelligence and security services. However, national oversight authorities seem to increasingly face multiple organizational and technical problems in implementing these regulations.
On the other side, law enforcement, intelligence and security agencies find themselves confronted with even more operational, technical and bureaucratic challenges to meet the required compliance regulations imposed by their regulatory legal bodies. Meanwhile, facing new challenges in the terrorist, criminal and military fields, some practitioners and service chiefs are said to have already issued some warnings to their government and legal authorities that some of the planned legal restrictions may sensibly reduce their OSINT/SIGINT collection efficiency, preventing them from fulfilling vital missions.
In addition, the growing technological complexity and ubiquitous use of IT-based surveillance tools and technology by state and non-state intelligence and security actors operating in the wide realm of surveillance will continuously lead to even bigger volumes of data to be controlled, and, by the same token, augment the possibilities for data abuse and the circumvention of legal requirements.
Considering that all data is information, but not all information is intelligence, governments, parliaments, oversight authorities, secret intelligence and security agencies will have to find viable political, organizational, practical and technical solutions
Lawmakers, governments, oversight bodies, intelligence and security agencies will have to make considerable efforts to find legal and organizational solutions to balance the citizen’s legitimate right to privacy and the justified state’s need for secrecy and security. At first glance, meeting that challenge seems to be contradictory and something like squaring the circle.
To establish national authorities responsible for the control and oversight of surveillance activities by law enforcement, secret intelligence and security agencies, governments in various states have adopted different approaches. The fundamental statutory question is whether
Generally, in democratic or “hybrid” (autocratic democracies) regimes, secret intelligence and security agencies are held accountable for their activities mostly by their national government (or ruling political party in hybrid regimes) but also by their parliament, particularly when it comes to the appropriation of budgets, and by civil society as a whole. However, accountability can only be claimed if scrutinous oversight procedures are established, providing, if it were, proof and evidence of lawlessness, organizational or operational abuse, mismanagement, budget and expenditure excess, etc. Moreover, weak intelligence oversight will automatically result in, and intensify activities of, independent investigative journalists and whistleblowers who, in the past, uncovered all sorts of deficiencies resulting in breathtaking intelligence scandals.
Obviously, given the impenetrable “fence of secrecy” surrounding these agencies and their particular deontology and culture of secrecy, effective and informed control and oversight are impossible without the collaboration of those being controlled. That dilemma can only be mitigated by providing oversight bodies – external and internal – with a clear legal statute, competent experts, sufficient budget and powerful technology.
One important precondition for oversight scrutiny and efficiency is the political will to fully implement constitutional law and to elaborate clear-cut rules regulating the way, type and volume of data oversight bodies can access. Ideally, all data collected, stored and disseminated by the agencies should be made available to oversight bodies whenever need be. In the face of mushrooming data volumes, unprecedented complexity of technology, as well as intricate political and operational relations between national and international intelligence actors, only a timely, direct and comprehensive access to critical data may allow for an on-time and effective democratic oversight.
While direct data access is less problematic for internal oversight entities, external oversight bodies, however, are still having considerable problems justifying direct access to sensitive raw data being collected, shared, and stored by the agencies. Confidentiality and security concerns on the part of intelligence practitioners and still existing legal gaps are characterizing the actual debate. In addition, in most countries concerned, there is considerable doubt on the part of the executive branch as to the reliability and security of the technical tools in use by oversight bodies, guaranteeing the trusted and secure handling of secret data.
Another general weakness consists in the interdependence between powerful or monopolistic industrial technology providers and intelligence actors and agencies. As a matter of fact, with open or tacit support of the scientific research community, all agencies must and will always try to acquire or trigger the development of the most sophisticated surveillance technology. Therefore, as a logical consequence, oversight bodies must be authorized to extend their review power to include corporate oversight as early as government contracts with corporate technology providers are being conceived, designed and negotiated. That approach would prevent agencies from falling into a dependency trap, making sure the resulting contract is compliant with all pertinent legal privacy regulations and includes provisions for built-in oversight even before it is signed and implemented, thus saving a lot of taxpayers’ money.
We at Traversals are convinced that there are several possibilities to facilitate and support efficient and innovative solutions to the most pressing oversight challenges by using our Intelligence Oversight Tool Kit. Traversals’ approach in developing and using intelligence oversight tools:
Traversals’ Oversight Tool Kit includes software tools for all activities taking place in the oversight relationship structure shown above to include the following types of functionalities:
The tracing tool will allow for the tracking and tracing of all data access and handling processes. With the agencies having every data set being protocolled, registered and accounted for, oversight bodies could have ad-hoc or, in some cases, even permanent online access to data (log files, registers and protocols). Thus, they are able to track and trace back every critical data set to its originator, author, releaser, destinator or modifier, and include other metadata. If required, content data, named entities, and other linguistic and numerical data can also be used as tracers.
The high number of different data sources, the constantly changing landscape of data sources and the unbelievable amount of data in the data sources to be checked require a solution that can cope with the requirements.
Our Federated Search was designed and conceived with exactly these requirements in mind. It allows the integration of different data sources, such as SQL, NoSQL or web services, transparent to the user. The Federated Search in combination with the Enterprise Knowledge Graph allows the tracing of singular data elements.
There are several types of filters required for oversight purposes: Internal and External Oversight Filters.
Internal oversight entities verify the data minimization effect of the filters implemented by their agency at the collection level, following pertinent compliance rules. To ensure ad-hoc and random control audits and compliance checks to verify the efficiency of the filters implemented, direct and unhampered electronic access to all stored and disseminated pre- and post-collection data must be possible. Since the internal oversight bodies are an integral part of, and working inside, the organization, security aspects shouldn’t be an obstacle. Subject to verification are e.g. privacy data of persons or groups especially protected by law, such as lawyers, clerics, doctors, etc. Internal oversight entities should report directly to the agency director and coordinate with external oversight bodies.
The identification of personal data within collected metadata and the content plays also an essential role in the context of GDPR compliance. Especially in this area, we have been able to gain a lot of experience with our OSINT Intelligence Platform. These insights can be applied to the Oversight Tool Kit as well. The identification and GDPR-compliant handling of personal data is a key element of the Federated Search and can be used to estimate a compliance score relevant for the oversight agents.
Following pertinent legal provisions, external oversight bodies design and apply their own filters as they verify the data minimization of the agency. They need to have direct access to post-collection data that is stored inside the agency and disseminated to third parties according to pertinent policy regulations. They should be able to compare their results with those delivered by the internal filters, thus verifying the accuracy of the filters used by internal oversight entities.
Oversight entities must be equipped with powerful search tools. Since their mission is to verify whether the agencies are compliant with the legally prescribed privacy and transparency rules, the search tools must allow for search in all types of files, metadata, databases, data storage facilities, registries, protocols and log files, etc.
All Big Data criteria (Volume, Velocity, Variety, Veracity and Value) are fulfilled when it comes to intelligence oversight. Often the complexity is projected purely on the data storage. The question of how to search in such data is also demanding. In the past 12 months, our solutions relied on the extremely flexible query language GraphQL, which powers our Federated Search. GraphQL allows graph filtering, geo-filtering, fuzzy searches, etc.
The function of assessment tools is to identify and present the deficiencies, violations and other rule-breaking activities incurred by the agencies or internal oversight entities. They include anomaly pattern analysis, deletion monitoring, intrusion and data access abuse, data dissemination violations to external third parties, suspicious data query and search behavior, etc.
Part of the our Oversight Tool Kit is an event-triggered processing engine which includes a large set of various processing modules. This list can be extended with the latest state-of-the-art technology coming from open research and development.
Oversight entities must be equipped with secure communications, and state-of-the-art presentation and reporting software to ensure a swift and transparent dissemination of their results obtained from their audits to their respective authorities and customers, such as parliament, government institutions and the judiciary.
Our Oversight Tool Kit includes a property based access control that can be adapted to the already existing classification system of intelligence agencies. Users see only what they are allowed and cleared to see. In addition to that, all identified personal data is encrypted with a purpose-specific 256-bit AES key. Access to all data is tracked in an immutable audit log. These facts ensure multi-dimensional secure communication between data storage and the Oversight Tool Kit. All data to be checked is visualized in various analysis views, such as graphs, maps or dashboards.
Intelligence oversight is one of the main pillars of intelligence transparency, a necessary element of “making intelligence information publicly available in a manner that enhances public understanding of intelligence activities while continuing to protect information when disclosure would harm national security.” (ODNI, Principles of Intelligence Transparency – Implementation Plan, 2015).
Due to its vast experience with intelligence software applications, our Intelligence Oversight Tools will provide all types of oversight entities with tailor-made software tools to cope with all major challenges resulting from their difficult task.
Copyright © 2022, Traversals Analytics and Intelligence GmbH. All Rights Reserved.