WO2023129055A1

WO2023129055A1 - Reliable in-camera anonymization method for machine learning/deep learning

Info

Publication number: WO2023129055A1
Application number: PCT/TR2022/051615
Authority: WO
Inventors: Mesut GOZUTOK; Engin FIRAT
Original assignee: Havelsan Hava Elektronik San. Ve Tic. A.S.
Priority date: 2021-12-28
Filing date: 2022-12-27
Publication date: 2023-07-06

Abstract

The invention relates to a method for performing anonymization after receiving raw data in a camera and anonymizing the transmitted data when the data is transferred from the camera.

Description

DESCRIPTION

RELIABLE IN-CAMERA ANONYMIZATION METHOD FOR MACHINE LEARNING/DEEP LEARNING

Technical Field

Background

Machine learning and deep learning methods, which are frequently used in autonomous systems and related artificial intelligence applications, require large amounts of image data. This data can be collected from places where there is vehicle and human traffic in general, while information containing the privacy of other people or assets belonging to these people may be included in the collected data. In cases where it is necessary to develop artificial intelligence models based on this data, it may be necessary to share the data with third parties. This situation causes violations in terms of personal data protection law due to the presence of personal privacy elements in the collected data.

In the United States patent document numbered US2020097767, which is in the state of the art, systems, and methods for synthesizing and/or modifying features in images to limit recognition by classifier algorithms are mentioned. This method performs anonymization by overlaying noise or generating synthetic data on photographs that are imperceptible to the human eye, preventing the detection of faces by standard face detection algorithms. There is no visual anonymization, only digital anonymization so that algorithms cannot find the faces.

In the United States patent document numbered US2020089995, which is in the state of the art, computerized face detection is mentioned. The degradation of computerized face detection includes receiving a source image including a representation of a face and calculating a perturbation for the source image. The perturbation is specific to the source image and is configured for a target face detector. A distorted image is then created by adding distortion to the source image and then the distorted image can be output instead of the source image. With this method, faces are found and the corresponding image regions are blurred. Anonymization is achieved both visually and digitally. No method is presented for license plates. Images or videos obtained with this method cannot be used for training machine learning algorithms afterward.

In the International patent document numbered W003049035, which is in the state of the art, an image processing system for automatic face or skin blurring is described. In the developed system, all faces or skin or specific faces can be blurred. This method uses face recognition and face-tracking algorithms to blur faces for all faces except one identified face. The face-tracking algorithm performs blurring on consecutive frames. In this way, the method continues to work correctly when the face detection algorithm cannot find faces. No solution is provided for license plates used on vehicles.

In the United States patent document numbered US2019147185, which is in the state of the art, a method for providing privacy protection in a received image, comprising obtaining image coordinates associated with one or more target persons in the received image. It anonymizes faces by superimposing a calculated noise signal on the input image in a way that face detection algorithms cannot. Similar to the aforementioned patent, the object is to prevent private information from being extracted from photos of people that can be found online by processing them by some algorithms. There is no visual anonymization. No solution is provided for license plates.

Therefore, there is a need to develop the method of the invention.

Objects and Brief Description of the Invention

The object of the present invention is to provide a method for performing the anonymization process after the raw data is received in the camera and to ensure that when data is transferred from the camera, the transferred data is anonymized. Another object of the present invention is the development of a method for identifying faces and license plates in the image taken from the camera and anonymizing them within the camera.

Nowadays, violation of people’s privacy during outdoor shooting is a major issue. The developed method enables privacy-protected photo and video shooting during camera shooting with two different modes.

The developed method can be implemented on a video camera/photo camera as well as on devices such as tablets and smartphones.

When the system is running in safe mode, license plates and human faces are anonymized. Instead of anonymization by blurring/pixelization or anonymization by adding noise that is invisible to the human eye, the method is based on imitating the objects with machine learning methods.

Detailed Description of the Invention

The invention is a reliable in-camera anonymization method for the machine learning/deep learning in safe mode, and it comprises the following operation steps;

- falling of the light captured from a particular scene on the sensor in the camera by being refracted through optical systems such as lenses,

- generating the analog data related to the light falling on the sensor by capturing the light with the sensor, converting the analog data to digital by an analog-to-digital converter integrated, obtaining a digital image from digital data that can be processed by computer vision algorithms, evaluating the region of interest (ROI) of faces and license plates on the image as privacy elements in the image by detecting with binary classifiers artificial intelligence algorithms, estimating the objects found by regression and in the case of the face, calculating the pose information with the help of the reference 3D model, the attributes that can be found with artificial intelligence models from the face frame, anonymizing license plates and faces synthetically as a result of the competition between two equivalent networks, generator, and discriminator, with the Generative Adversarial Networks method by using this pose information of the objects whose pose information is calculated, embedding anonymized new faces generated using machine learning-based methods into the original image by replacing them with Rol, compressing the video, deleting the raw data, sending the compressed video over the network.

In one embodiment of the invention, a reliable in-camera method for the machine learning/deep learning in standard mode is developed, and it comprises the following operation steps;

- generating the analog data related to the light falling on the sensor by capturing the light with the sensor, converting the analog data to digital by an analog-to-digital converter integrated, obtaining a digital image from digital data that can be processed by computer vision algorithms, compressing the video, sending the compressed video over the network.

The mode indicator offers two different modes for two different methods. The main object is to inform the outside world which mode the designed camera is in. This way, people who are being photographed or videotaped will be able to see whether their photo or video was taken in a privacy-protected or conventional way. This indicator can give a light warning, or other types of warnings can be considered to convey this information.

In safe operating mode, this indicator may emit green light. In standard operating mode, it may emit red light. These colors are chosen as representatives, it is possible to give warnings in different colors to show the difference between both modes. The difference between safe operation mode and standard operation mode can also be indicated by the flashing of the lights at different frequencies.

Claims

1. The invention is a reliable in-camera anonymization method for the machine leaming/deep learning in safe mode, characterized in that it comprises the following operation steps;

- generating the analog data related to the light falling on the sensor by capturing the light with the sensor, converting the analog data to digital by an analog-to-digital converter integrated, obtaining a digital image from digital data that can be processed by computer vision algorithms, evaluating the region of interest (ROI) of faces and license plates on the image as privacy elements in the image by detecting with binary classifiers artificial intelligence algorithms, estimating the objects found by regression and in the case of the face, calculating the pose information with the help of the reference 3D model, the attributes that can be found with artificial intelligence models from the face frame, anonymizing license plates and faces synthetically as a result of the competition between two equivalent networks, generator, and discriminator, with the Generative Adversarial Networks method using this pose information of the objects whose pose information is calculated, embedding anonymized new faces generated using machine learning-based methods into the original image by replacing them with Rol, compressing the video, deleting the raw data, sending the compressed video over the network.

2. A method according to claim 1, characterized in that it comprises the step of providing a warning with a light indicating that the device is operating in a safe mode during the application of the method.

6 The invention is a reliable in-camera method for the machine leaming/deep learning in standard mode, characterized in that it comprises the following operation steps;

- generating the analog data related to the light falling on the sensor by capturing the light with the sensor, converting the analog data to digital by an analog-to-digital converter integrated, obtaining a digital image from digital data that can be processed by computer vision algorithms, compressing the video, sending the compressed video over the network. A method according to claim 3, characterized in that it comprises the step of providing a warning with a light indicating that the device is operating in standard mode during the application of the method.

7