From Traffic Analysis to Artificial Intelligence

Introduction

In this paper, we present the artificial intelligence (AI) transformation currently taking place at the Data Analytics Center of Unit 8200 (Israeli SIGINT National Unit). This transformation is most apparent in the shift of one of the largest data organizations of the largest intelligence unit in Israel to an organization that operates using human-machine teams (HMTs). We shall analyze the opening conditions, challenges, technologies, and mainly the perceptual shifts that propelled a change of such magnitude.

The Data Analytics Center is a long-standing organization, which is central to the Israeli Intelligence and Security Community and possesses expert knowledge that renders it relevant wherever intelligence bears on reality – from providing daily alerts in terms of the IDF’s ongoing security operations, through safeguarding large-scale operations, to supporting strategic decision making in the IDF and the Security Cabinet.

We live in an era where machine learning (ML) capabilities are changing the world around us. We drive our cars differently, and they are even close to driving themselves. We listen to music differently, use medical services differently, shop differently, and even think differently. These changes have met the Data Analytics Center, which has been coping with a variety of intelligence missions that are more complex than ever before, as well as with a non-linear growth of data in recent years. Our constant interaction with breakthrough technological developments in computational capabilities, machine learning (ML) algorithms, and natural language processing (NLP) technologies have taught us time and again that there is potential for disruptive innovation.

Such innovation includes the potential for a dramatic quantum leap in terms of the quality and precision of meeting information and operational needs. This transformation, combined with other components, will enable the data-processing capability necessary for the actualization of the IDF’s offensive capability and lethality in its concept of victory.

In the following pages, we shall describe the steps involved in the transformation of the Center, which transferred the bulk of work from manual human labor to work based on human-machine interaction. On a par with previous transformations, this time, too, involves changes in perception, organization, and technology. Only the combined change in all three components should enable us to harness the IDF’s advanced digital capabilities for the benefit of its operational needs.

The Main Characteristics of the Center

The Data Analytics Center is found at the heart of the intelligence cycle. Its personnel are the first to deal with the raw intelligence material, and, especially when the data involved is of a foreign language, the Center is in charge of pinpointing the relevant pieces of information and processing them into a finalized intelligence product.

To accomplish this task, several new positions have been established at the Center over the years:

• 1. Audio linguists - Perhaps the first and most famous analysts in the Center. They have existed ever since the days of intercepting the British and the Jordanian communication networks prior to the establishment of the State of Israel. The profession has evolved and changed dramatically over the years, but the focus has remained on proficiency in the spoken language and expertise in audio materials.
• 2.Text analysts - Once the textual medium became more significant as a medium for data transmission, a parallel position was formed for the processing of textual data.
• 3.Transcribers and translators - Products requiring a high level of reliability, i.e. transcription and translation at the level of the single word, as well as linguistically complex data, were handled by these expert office-holders.
• 4.Data Analysts - Being the last one to join the Center as a distinct position, the Data Analyst specializes in data mining and analysis of trends and anomalies in the data flow.
• 5.Traffic Analysts - Traffic analysis officers and non-commissioned officers; run the intelligence production “factory” and are in charge of making products accessible to a wide range of consumers.

The Starting Point

Three of the Center’s characteristics are what enable it to make an extraordinary leap toward AI-oriented change:

Data - The fact that the Center’s personnel were the first to process data over the years now allows us to dominate the most valuable resource in the digital age: high-quality labeled data.

DS technologies rely on three main components: algorithms, computing power, and relevant data. The most substantial competitive advantage of different organizations often lies in their data. The great power of Google’s search engine relies on the fact that millions of users a day teach it what a good result actually is, merely by performing searches and choosing results. Similarly, the power of the Center as an AI organization relies on a large and expert workforce that handles a huge variety of pieces of information daily.

Human Resources - Many organizations trying to undergo an AI transformation fail because the organization’s personnel are accustomed to a certain reality and expertise that they have cultivated over the years. Naturally, it is difficult for them to come to terms with the degree of trust they need to place in a machine performing processes they used to oversee themselves. We have a significant advantage in this respect: Our organization is comprised of a high percentage of young and creative people, accustomed to human-machine interaction from their private lives, and given their high turnover rate, the organization is a lot more willing to accept and embrace ideas that other organizations may perceive as more revolutionary and disruptive.

Digital Environment - A third advantage lies in the fact that we are part of a unit whose name is synonymous with innovation and technological progress. We work very closely with our parallel technological centers who fully participate in this process of change.

Will the News Come from Data Science (DS)?

In recent years, we have witnessed the “explosion” of the world of machine Learning (ML): the creation of new algorithms, AI-based commercial products, “victories” of machines in board games and computer games (AlphaGo, AlphaStar), and even the rise of algorithms for predicting three-dimensional structures of proteins (AlphaFold).

Nowadays, recommendation systems and data-driven decision-making systems have entered the business world and begun dramatically impacting the way decisions are made in any organization that has access to data and computation resources. In addition, human language technologies (HLTs) have led to breakthroughs in recent years in the ability to comprehend human texts, transcribe audio clips (STT - Speech to Text), and recognize texts in images (OCR - Optical Character Recognition).

As the years went by, these technologies have progressed, up to a point where a critical threshold was exceeded in terms of capabilities. At this point, it was understood that the way intelligence is produced can be revolutionized using technologies from the world of ML.

To us, the sky was the limit. On the face of it, it was already possible back then to outline operational processes in which machines could automatically detect entities, analyze suspicious patterns of behavior, transcribe and translate their conversations and texts, and identify the sections and details relevant to the IDF’s information needs. We believed that we were close to overcoming the barriers that had limited us all these years and that we would start a new era of intelligence processing that would be unparalleled to anything seen before.

Not So Fast

The naive idea that technology that will be developed and applied indiscriminately to our problems will bring about the change, is tantamount to the thought that only a good racing car is enough to win the race.

This understanding stemmed from an inspection of the development of AI products in the business world. With the technological advancement in research and academia, there came the disillusionment of organizations that have gained several years of experience with ML-based products in the real world. Reality has taught us that it is not trivial to create products addressing a variety of “out-of-the-lab” scenarios, on a large scale, that would also stand the test of time.

The prevailing conclusion is that, generally speaking, it is impossible to take the recently published algorithm “off the shelf” and apply it to a business problem as is. There are countless obstacles that need to be recognized and overcome and every step needs to be taken responsibly and accurately.

The challenge of creating relevant products only intensifies when it comes to a high-risk decision-making environment. Unlike in the realms of AdTech and Gaming, wrong decisions in the realm of intelligence may cost lives.

So How Do You Do It?

First, the most fundamental understanding we have reached so far is that we do not want to replace our excellent people – whom we have been saying for years are the secret of our success – with machines. Neither the young and brilliant minds who enlist in the IDF with a sense of mission and a desire to contribute to the security of the country, nor the knowledgeable experts whose expertise cannot be found anywhere else. We do not wish to replace the people; we wish to give them the tools that will empower them and allow them to do what only they are able to do. In other words, to get the most out of them. This point, along with the huge potential, poses a risk that must be managed carefully so that the language skills and deep understanding of the enemy and his digital expressions, which we have accumulated over the years, does not decrease.

We build HMTs that make it possible to increase productivity (streamlining) and take a quantum leap in the quality and accuracy of our answers to information needs. The test of AI transformation therefore does not end with these two indicators but also involves the ability to produce new value for our consumers.

Efficiency - Our people invest a great deal of time every day in tasks that machines can accomplish easily and quickly, tasks which our people provide no added value to: Listening to an audio segment in a foreign language requires a lot more time and input in training than reading the same information in a translated version to Hebrew. Even if the machine does not reach 100% success, any action the machine manages to perform saves the time of the person sitting next to it and allows them to more efficiently retrieve information, compile different data points, and better existing ML engines for target identification.

Effectiveness - intelligence work is often tantamount to putting an endless jigsaw puzzle together. To date, our personnel have been divided according to their expertise in the data format (audio, text, geographical signatures), such that each person has seen and specialized in only a few pieces of the entire puzzle. Our perceptual and technological advancement allows us to divide our people by task rather than by data type. Everyone sees more pieces of the jigsaw puzzle, and the very act of working in a multidisciplinary configuration enables us to find better solutions to the problems we face.

New value - There are phenomena that a human find very difficult to recognize, but, when combined with the right tools, can become more distinguishable. Using algorithms to detect anomalies and identify trends in data over time can solve problems that we have not been able to address so far. The new value is also found in the ability to provide consumers with interactive data products and thus put an end to an era in which consumers received only written products.

The new value goes beyond its contribution to the regular information needs of the Israeli Military Intelligence and the operational needs of the military. In case of a war, the described transformation will enable the processing and analysis of the data needed for the IDF’s concept of victory. Thus, this increase in data compiling and analysis speed, combined with the ability to quickly and effectively inject the result with proper context to the actual operation, will be a “game changer” when it comes to strengthening the IDF’s offensive capability and lethality in the battlefield. The algorithms, with the addition of Data Science Domain Experts (DSDX), will in turn enable quick communication between intelligence and operations, such that the ability to expose the enemy will be based on a robust and sturdy method. This value, which must be realized as part of the concept of the IDF’s digital supremacy, is integrated into target processing and multidimensional maneuvering. For example, forces in the battlefield will be able to receive real-time data processed at the appropriate resolution needed for immediate decision-making, in a systematic manner adapted to the operational systems and visualization required. And all of this will take place within seconds with no dependencies on human processes of production.

Considering the above understandings, we embarked on a joint venture with the ISNU’s technological development centers, whose purpose is to give rise to a new ecosystem that uses advanced analytics and data science tools. We identified 3 key vectors for change:

1. Designing and implementing HLTs for the benefit of empowering and gradually replacing human functions of information retrieval, as well as data processing using state-of-the-art transcription and translation engines.
2. Using advanced analytics and establishing designated ad hoc teams for processing, organizing and rendering accessible a wide variety of types of data for the benefit of data analysis and machine learning.
3. Creating an arsenal of ML based products for the benefit of the Center’s core tasks.

Each of these vectors has dictated the design of a new value network that includes defining current processes, new positions, and redefining the Center's outputs

Vector #1: Designing and Implementing HLTs at the Center’s Core

Recent years have witnessed a breakthrough in the development of Human Language Technologies (HLTs). Using innovative algorithms developed by the best universities and companies in the world, machines know how to perform a variety of tasks that until recently were only reserved to humans: Understanding natural language, performing tasks using digital personal assistants and simultaneous transcription while shooting videos.

The main question we have been facing is: Why not replace all our human linguistic functions with algorithms?

The audio linguists will be replaced with Speech to Text elements, the text analysts will be replaced with Optical Character Recognition algorithms, the language experts will be replaced with machine translation, and all translated materials will be aggregated in the same place, accessible to all.

Makes sense, doesn’t it?

The technological development we are witnessing is indeed impressive, but it is limited in its ability to meet our needs in two main respects:

First challenge - Data in the Context of the Operational Need

Machine translation that is available to all via Google Translate enables the conversion of texts from French to English in an overall satisfactory way. Similarly, it is also possible to talk to Amazon’s digital assistant in English and ask her to book movie tickets. However, the languages ​​and scenarios that plague the security system of a small country in the Middle East are not necessarily the same business problems bothering global technology giants.

Second, in order to achieve a satisfactory level of performance in face of the raw material we are dealing with, we were required to bring to the table enough high-quality labeled data that would enable machines to learn how to handle those materials. As stated above, this is one of our advantages in kicking off the transformation, and thousands of hours of human labor have been invested in this data labeling mission. Without our talented people, we would have surely failed.

Even the most sophisticated machine is worthless if there is no one to provide it with quality data to learn from.

Choosing to invest manpower in a task like data labeling requires making tough decisions – investing today in the fruits of tomorrow. We did this with the understanding that this manual work plays an equally important part in this change as all of the shiniest algorithms. Here too, the fact that we and the algorithmic development body are part of the same unit enabled us to collect and label the right data correctly, which is much more difficult when the processes are distinct from each other.

Due to the complexity and dynamics of the data, the labeling craft cannot be completed in one fell swoop, so we made the necessary adjustments and the technological and methodological developments that made the data-labeling task part of the daily job of the office-holders, in order to continue to improve constantly.

Second Challenge - It Doesn’t All Come Down to Mechanics

An amateur cook who carefully follows a chef’s recipe will not be able to prepare food at the level of a fine-dining restaurant, even if s/he uses top-quality raw materials and follows the instructions carefully. In the same way, a person reading a transcript of an audio clip recorded between two people from another country will not understand the conversation as a person who was present in the room during that conversation would understand it. Even if the transcript and translation are perfect, there are cultural and religious aspects, or non-verbal information such as intonation, which dramatically affect one’s ability to understand the essence of the conversation.

As a result, we do not replace people with algorithms, but rather, rearrange our office-holders in accordance with technology: The new value network consists primarily of analysts using HLTs – those who specialize in the worlds of information retrieval and data analytics.

We empower our fine linguists to become language experts in order for them to provide answers where the machine falls short. Language experts have, among other things, the responsibility of training the machine.

Vector #2: Using Advanced Analytics and Establishing Designated Teams for the Processing, Organizing, and Rendering Accessible of Wide Varieties of Data

Our production “factories” consist of multidisciplinary teams, such that each office-holder can provide their added value concerning the information needs of their team. In previous years, the intelligence teams were not capable of naturally handling a variety of new materials, and the bulk of the work focused on extracting specific information details from a selected few data types. This work system allowed for specialization and “boutique” answers for scenarios but did not necessarily provide an optimal answer for large-scale information needs.

Nowadays, in an era where millions of different types of data points flow into our systems every day, we must use the appropriate analytical tools and visualizations to detect patterns in the data flow.

As part of the process of change, we have progressed from observing specific geographical instances of a particular entity to the creation and analysis of heatmaps of a group of entities. From manual analysis of specific relationships within a group of interest, to using algorithms from the world of social network analysis (SNA), meant to cluster the data into “communities” and identify patterns that emerge from the analysis.

Instead of manually going over hundreds of data points dealing with a certain intelligence target and providing a basic diagnosis, we began to focus our work on data points and features relevant to the dates in which anomalies in the target’s behavior patterns were detected using time-series anomaly detection tools.

Even within the “classic” textual materials, we have begun assimilating modern methods from the world of NLP to extract insights and patterns from collections of reports, which do not require manually going over all of them. Methods such as topic classification and clustering, named entity recognition, sentiment analysis, etc.

To make these tools accessible to all users, in-depth work has been done to identify the teams’ most common and significant work processes (referred to as “typicals”) and to create specific tools for each of these processes. The tools are “wrapped” in a convenient way for the users. All they must do is input their data and run the tool. Behind the scenes, the correct “wiring” and advanced logic were determined by experts, based on a variety of data types relevant to the information need.

To ensure the correct use of the tools and the deep assimilation of the concepts described above, a training program was developed for all positions in the research teams dealing with data analysis. In addition, to ensure that each investigation team includes an expert in the field of data analysis, it was decided to create a new position in the Center – the Data Analyst.

The Data Analysts are predestined and trained from their very first day to become expert data analysts. They acquire advance tools and methodologies from the world of data analysis (DA) in a wide variety of fields: big-data inquiry, data processing and preparation, data visualization, social network analysis (SNA), geo-spatial analysis, etc.

To support all the processes described above, “data departments” have been set up at the Center. These are assigned to data curation: The teams detect the variety of databases relevant to the information needs, process them, diagnose them, and render them accessible in the most convenient way for our consumers.

Support tools have been created for the data curators, allowing them to perform each step in the process independently (Self Service), with the infrastructural support of data engineers from the fellow technological development centers.

This way, they can independently process and model different types of data on a huge scale using analytics and tools built by experts.

Today, following the addition of these contents in all of the training programs at the Center, the establishment of data curation departments and the integration of the Data Analyst in the investigation team; we utilize a wider range of materials and detect valuable hidden patterns every day, via big-data analysis tools and visualizations.

Vector #3: Using Machine Learning Algorithms

An important detail that many overlook is that even when using advanced analytics on a huge amount of data, the analyst is constantly forced to make decisions: From which databases information should be retrieved, how the query should be conditioned, which processing and calculations are needed, how to filter the list of results, etc.

Many of these decisions are made based on intuition and “rules of thumb”, which often guarantees suboptimal results. Alongside the paradigm of explicit data analysis, another paradigm has been developed, which has grown dramatically in recent years: Information analysis using human-machine interaction.

In the human-machine paradigm, the user replaces the definition of their rule set by providing examples and interactive feedback with the use of a machine.

For example, instead of setting rule-based conditions set to retrieve objects related to a terrorist cell, the user feeds samples of known objects belonging to terrorist cells and receives back recommendations for objects that behaved in a similar way. When receiving the recommendations, the user can give feedback to the machine, thus improving the results iteratively.

The difference between defining explicit rules manually and learning the rules based on examples is huge. In the former situation, the specialist is required to re-create a set of rules for each problem and manually maintain them over time. In a machine learning (ML) scenario, given that a system is tailored for the task, simple users can perform the operation “with a single click” by providing relevant examples for the machine to learn from and getting back a result that is expected to be more precise than most experts’ manual work. This paradigm allows us to scale up and provide a better solution to a variety of scenarios.

When we turned to identify the areas that could benefit the most from products of this type, many dozens of seemingly different scenarios came up, in terms of which the machine could be used to improve human functioning.

Following an in-depth examination, we discovered something very interesting: All the scenarios and desired projects could naturally be classified into 7 clusters. The same scenarios were basically different examples of specific instances of those 7 “archetypes” of business processes.

Despite the differences between the various scenarios, they are more alike than different. We consequently decided to focus our efforts on building generic solutions that would serve a wide range of scenarios relevant to the same information needs. Accordingly, it was necessary to map the main business processes thoroughly and locate key nodes where ML components would allow for real change. This approach is fundamentally different from the approach arguing that it is necessary to create for each specific scenario its very own tailored project or DS product.

The above decision also stemmed from the understanding that there is great difficulty in creating and maintaining relevant DS projects over time. Based on this understanding, we decided to map the reasons for this difficulty and create the conditions that would render these projects successful in the near future and for years to come.

While examining insights from the business world, in particular materials from consulting firm McKinsey & Co., we noted that it has been repeatedly argued that a key component in the success of an ML-based project is the involvement of a domain expert as part of the project team. In light of this understanding, we created a new position at the Center: The Data Science Domain Expert (DSDX).

The Data Science Domain Expert is responsible for defining the business problem in terms of an optimization problem in the ML world, determining the key performance indicators (KPIs), targeting the collection of the labeled data, doing error analysis, and designing the requirements for the desired product.

The Data Science Domain Experts are intimately integrated with the DS teams at the technology development centers, and work to produce solutions that are accurate methodologically, technologically, and intelligence-wise. They come from the Unit’s production bases and undergo specialized training in the field of ML in a manner tailored for office-holders who have no degree in the exact sciences.

As part of their training, the Data Science Domain Experts learn how to intelligently characterize DS products – from defining the need to deploying and implementing the products in the systems. They learn how to map out the Intelligence work process from beginning to end, at all levels, and to detect the components that can and should be replaced with machine learning models. In addition, they learn how to produce a comfortable human-machine interaction in a way that is explainable to the user, in order for them to make responsible decisions.

The decision to generate a specific population aimed at designing and producing DS products, gives rise to a sort of role pyramid, such that only a few of the office-holders are involved in building the products themselves, while the rest learn how to use those products responsibly and accurately in relation to various intelligence missions.

The Transformation: A Summary

Alongside the revolution described here, there is a great challenge in evolution that produces change. This evolution serves as both a threat and an opportunity for the entire process. Thus, we are engaged in the conversion of existing positions and the leveraging of experts for a new value network that integrates humans and machines, all while developing brand new positions and methods. Accordingly, we use tools from the world of Leading Strategic Change in order to deal with opposition to the process, e.g. analysts who for years have developed their expertise to become the best in their field, as well as to establish confidence of the users in the machine. Considering the size of our Center’s departments, their complexity, and the variety of information needs we are facing, our Center faces a single test: bottom-up change. This perception has set a number of key practices:

• Institutionalizing and empowering the functions involved in the development of the forces (that deal with DS and product design) in each department. Their role is to assimilate the change and make sure it reaches all personnel at the Center, and to enable a dynamic process involving new ideas and data products, which is meant to empower office-holders and consumers.
• Encouraging a new operational culture, emphasizing an approach that allows for mistakes (an approach that coheres with the professional meticulousness that characterizes the work of the Center).
• And, of course, encouraging a new corporate culture, for example, by holding ceremonies to recognizing our soldiers achievements, which is part of the Center’s tradition of excellence – shifting the weight to projects that have proven new value; identifying change agents, and, in particular, celebrating small victories that begin to impact the entire Unit.

Changing the Nature of the Center

Beyond the changes in the various technologies, products, and positions, to truly symbolize the change, the Center’s mission has also been adapted to its modern character:

“The Center processes and produces information based on expertise in language and in digital signature research. The Center designs, develops, and operates advanced data analytics capabilities, directs cyber efforts to meet intelligence needs, and establishes an operational partnership with the headquarters of operations."

The new mission embodies a fundamental change at the very cores of the Center:

1. From a Production Center to a Data Analytics Center.
2. From a “factory” based on human capabilities to a “factory” based on human-machine interactions.
3. From a linguistic community to an analytical-technological community.

Looking Forward

In this paper, we presented the fundamental milestones and insights we have learned along the way as part of the AI transformation of the Data Analytics Center. For us, this is hardly a minor improvement in the work of the Center, but rather a dramatic change – a non-linear leap in efficiency, effectiveness, and our ability to meet complex intelligence needs.

The transformation’s touchstone, like any other process encountered by the Data Analytics Center, is the way in which the research and operational partners benefit from it. Merely data does not change reality, it must be processed, conclusions and insights must be drawn from it, and these in turn must facilitate actions and decision making. The change described enables us to identify and create new value in the Israeli Intelligence and Security Community: data-based interactive products, rendering self-service information accessible, providing data-labeling services, and more. The new services and the increase in the volume and pace of existing products provided by the Center require new skills on the part of the research and operational analysts, in order for them to produce outputs based on the compilation of a great deal of data and accomplish their diverse tasks. This issue constitutes a challenge for the coming years. This has huge potential for the Community, but also runs a significant risk that requires high compatibility between the Center and its consumers.

Finally, this move has a dramatic effect upon the contribution of the Center to the Israeli industry. Even today, hundreds of soldiers are discharged from service in the Center every year, and are then appointed to key positions as data analysts and experts in the design and development of DS-based products for the benefit of decision making in organizations and companies in the Israeli industry. This further contributes to the AI revolution in the State of Israel.

The process is not yet complete, and it is likely to require many years of investment and informed decision making. Having said that, we feel that we have reached a high level of maturity in understanding the guiding principles underlying the process and the steps needed for its implementation. We thus believe that any organization which has data, resources, and a desire for change can draw inspiration from the process described in this paper, as well as from the insights we have discovered along the way.