Author: Daniela Zarate – Data Engineer
It is possible to calculate image similarity for public security with Artificial Intelligence. Vector embeddings allow for the representation of images, audio recordings, videos, among others, in vectors and quantify the similarity between two objects through the distance between the vectors.
An artificial neural network is a common tool for training a model to calculate vector representations of images (vector embeddings). Once the images are translated into vectors, the similarity between vectors must be calculated. Algorithms like KNN use distance metrics between vectors to determine similarity. The smaller the distance value, the more similar the images will be.
The use cases of image similarity for public security with artificial intelligence are countless; however, in this text, the field of public security will be explored in depth, more specifically in the fight against the disappearance of people, a recurring crime worldwide.
How can we calculate how similar one image is to another?
Through a deep learning model, images are translated into a vector of decimal numbers between 0 and 1; this process is called vector embedding. The resulting vectors are usually high-dimensional. Images can be embedded using models such as convolutional neural networks (CNN).
Illustration 1: Vector embedding of the image of a cat.
Convolutional neural networks (CNN) offer solutions for advanced image detection and classification. Unlike a conventional neural network, the convolutional neural network increases the accuracy of the results, by changing the size, rotation, or position of the image, the model will still detect it correctly.
The inputs of the neural network are characteristic; in the case of an image, each pixel is placed in a neuron of the input layer. If you work with a black and white image, each pixel has a value which ranges from 0 for black to 255 for white. Therefore, convolutional neural networks seek to enhance a process that receives the pixels of an image, extracts the features, and classifies them with regular layers. As we provide the model with examples during training, it will discover which features are worth identifying.
Figure 2: Raw images represented as pixels with a convolutional neural network.1
To implement the algorithm that allows us to obtain vector embeddings, it is possible to use a pre-trained model such as TensorFlow Hub, an open repository and a library for reusable machine learning.
Illustration 3: Implementation of the pre-trained model – Taken from https://towardsdatascience.com/image-similarity-with-deep-learning-c17d83068f59 2
Illustration 4: Embedding – Taken from https://towardsdatascience.com/image-similarity-with-deep-learning-c17d83068f59 2
Once the vector of decimal numbers between 0 and 1 that represents the image is obtained, distance metrics such as the cosine distance are used with the SciPy or Sklearn library. Cosine similarity measures the similarity between two vectors by calculating the cosine angle between them. The smaller the distance value, the more similar the images will be. The cosine distance between and is defined as the scalar product of the two vectors divided by the product of the magnitudes of the two vectors.
Illustration 5: Calculation of the distances between vectors
The following image shows the result of the distances between the vectors that represent pet images, using the pre-trained model of TensorFlow Hub.
Illustration 6: Distances between the vectors of pet images
On the other hand, the following image shows the result of the distances between the vectors that represent the images of people and a dog. You can see that person 2 and person 2_1 are the same person, so a short distance is plotted between those two vectors. It also detects a similarity between person 2 and person 3.
Illustration 7: Distances between the vectors of images of people
Use of image similarity with AI in public safety
The application of image similarity in public security represents a significant advancement in leveraging artificial intelligence (AI) to address critical societal issues, particularly in the fight against human trafficking.
The use of image similarity tools, such as the ones developed by Marinus Analytics, exemplifies how AI can offer innovative solutions to enhance the effectiveness of these agencies.
Marinus Analytics, established in 2014 by a team of women entrepreneurs, specialises in the development of AI-based tools to aid law enforcement and humanitarian agencies. Their flagship software, Traffic Jam, is at the forefront of this technology’s application in public security. Traffic Jam is designed to analyse vast amounts of online data, including photographs from internet advertisements related to sexual services, many of which are posted by or on behalf of human trafficking victims.
One of the critical features of Traffic Jam is its FaceSearch capability, which employs advanced image similarity algorithms to match photographs of missing persons or suspected victims with images found online. This capability has proven to be invaluable for law enforcement agencies in expediting the identification and rescue of human trafficking victims.
By allowing detectives to upload a photo of a missing person and search through online advertisements for high-confidence matches, Traffic Jam has dramatically reduced the time it takes to locate and recover victims.
This tool has been instrumental in reuniting families and dismantling trafficking networks, significantly cutting down the investigation time from years to just a few months.
The efficacy of Traffic Jam is highlighted by numerous success stories where law enforcement officials have been able to swiftly identify and rescue trafficking victims based on matches found by the software. In one instance, a detective uploaded a photo received from a victim’s family and, within seconds, found a high-confidence match in an online advertisement. By analysing the location and time associated with the ad, the detective confirmed the identity of the missing person and facilitated a reunion with their family in a matter of days.
It not only aids in the rapid identification and recovery of human trafficking victims but also supports the building of legal cases against traffickers. Currently, Traffic Jam is utilised by security agencies across the United States, Canada, and the United Kingdom, showcasing its global impact on the fight against organised crime and human trafficking.
Organisations like the National Center for Missing & Exploited Children (NCMEC) have also embraced these innovations to enhance operations. NCMEC, a critical player in the fight against child exploitation, receives millions of tips annually about grave crimes against children, including rape, abuse, and trafficking.
A significant portion of these tips—approximately 22,000 per year—relate to missing children. Leveraging AI, NCMEC has optimised the process of preparing photographs for public dissemination and law enforcement use, which is crucial in the search for missing children. This AI application facilitates the enhancement of images by removing irrelevant backgrounds, updating clothing to reflect current appearances, and editing other individuals from the photographs to focus on the missing child.
This innovative use of AI extends beyond just the search and recovery phase. Once a child has been found and reunited with their family, the daunting task of prosecuting the perpetrators begins. Here, AI demonstrates its transformative potential in streamlining legal processes. The accumulation of evidence in cases involving child abductors or abusers is often overwhelming, encompassing a vast array of documents, communications, and digital footprints that need to be meticulously analysed to build a compelling legal case.
Traditional methods required legal teams to spend countless hours in the discovery phase, sifting through emails and attached documents to gather pertinent information. Now, AI significantly reduces this labour-intensive process, enabling quicker, more efficient preparation for trial.
Conclusion
In conclusion, while this article primarily focuses on the application of images and convolutional neural networks (CNNs) with pre-trained models for image processing, it’s important to note that the concept of vector embeddings extends far beyond just visual data. Vector embeddings can be effectively utilised across a variety of data types, including videos, text, and more, leveraging an array of models and methodologies tailored to specific needs. Furthermore, numerous techniques exist for calculating vector distances, each offering unique advantages depending on the specific goals at hand. Ultimately, the key to unlocking the full potential of Artificial Intelligence in this context lies in carefully selecting the model and method that align most closely with your objectives, ensuring optimal outcomes in terms of accuracy and efficiency.
The synergy between AI tools like Traffic Jam and the initiatives led by NCMEC illustrates a broader trend in the use of technology to combat public security challenges. By automating and enhancing tasks that once required extensive manual effort, AI is not only expediting the rescue and recovery of victims but also accelerating the judicial process against those responsible for such heinous crimes. This holistic approach, from identification and recovery to legal prosecution, underscores the essential role of AI in fortifying efforts against child exploitation and trafficking, promising a more effective and timely response to these urgent issues.
References
- Engineer, R.T. (s/f). What are vector embeddings? Pineapple. Retrieved on March 19, 2023, from https://www.pinecone.io/learn/vector-embeddings/
- Heyer, S. (2022, April 12). How to implement image similarity using deep learning. Towards Data Science. https://towardsdatascience.com/image-similarity-with-deep-learning-c17d83068f59
- (Kumon), T.Y. (2020, March 28). Similarity search and similar image search in elasticsearch. Medium. https://medium.com/@kumon/similarity-search-and-similar-image-search-in-elasticsearch-14552a8a8dea
- (Kumon), T.Y. (2020, April 14). How to perform a practical similarity search with elasticsearch. Medium. https://medium.com/@kumon/how-to-realize-similarity-search-with-elasticsearch-3dd5641b9adb
- Larsen, K. (2018, August 9). Marinus Analytics fights human trafficking using Amazon Rekognition.Amazon.com. Retrieved on March 10, 2023, from https://aws.amazon.com/es/blogs/machine-learning/marinus-analytics-fights-human-trafficking-using-amazon-rekognition/
- (S/f-b). Amazon.com. Retrieved on March 9, 2023, from https://aws.amazon.com/es/rekognition/the-facts-on-facial-recognition-with-artificial-intelligence/
- About —. (s/f). Marinus Analytics. Retrieved on March 5, 2023, from https://www.marinusanalytics.com/about
- Tech, R. (2021, August 16). Convolutional Neural Networks -Advanced image classification with AI/ML (CNN). Youtube. https://www.youtube.com/watch?v=4sWhhQwHqug
- https://www.fastcompany.com/91025986/19-ceos-died-on-the-job-leaders-work-life-balance
Daniela Zarate – Data Engineer
EQUINOX
what’s ai?
Discover what is AI and how it will become revolutonary in the industry
