CN112115811A

CN112115811A - Image processing method and device based on privacy protection and electronic equipment

Info

Publication number: CN112115811A
Application number: CN202010893701.8A
Authority: CN
Inventors: 曹佳炯; 丁菁汀; 李亮
Original assignee: Alipay Hangzhou Information Technology Co Ltd
Current assignee: Alipay Hangzhou Information Technology Co Ltd
Priority date: 2020-08-31
Filing date: 2020-08-31
Publication date: 2020-12-22

Abstract

The embodiment of the specification discloses an image processing method and device based on privacy protection and an electronic device. The method comprises the following steps: the terminal equipment collects a biological characteristic image; the terminal equipment performs convolution processing on the biological characteristic image by using a convolution kernel to obtain a desensitized image; the terminal equipment sends the desensitization image to a server; the server receives the desensitized image; the server constructs a recovery matrix according to the convolution kernel; and the server recovers the desensitization image by using a recovery matrix to obtain the biological characteristic image. The embodiment of the specification can protect the privacy information in the biological characteristic image.

Description

Image processing method and device based on privacy protection and electronic equipment

Technical Field

The embodiment of the specification relates to the technical field of computers, in particular to an image processing method and device based on privacy protection and an electronic device.

Background

At present, biometric technology is widely applied to various fields, and is one of the main means for authenticating the identity of a user. In the related art, a terminal device may acquire a biometric image, and may transmit the biometric image to a server. The server may receive the biometric image; the user identity may be authenticated using the biometric image.

In the above processing, there is a possibility that the private information in the biometric image is leaked.

Disclosure of Invention

The embodiment of the specification provides an image processing method and device based on privacy protection and an electronic device, so as to protect privacy information in a biological characteristic image. The technical scheme of the embodiment of the specification is as follows.

In a first aspect of embodiments of the present specification, there is provided an image processing method based on privacy protection, including: the terminal equipment collects a biological characteristic image; the terminal equipment performs convolution processing on the biological characteristic image by using a convolution kernel to obtain a desensitized image; the terminal equipment sends the desensitization image to a server; the server receives the desensitized image; the server constructs a recovery matrix according to the convolution kernel; and the server recovers the desensitization image by using a recovery matrix to obtain the biological characteristic image.

In a second aspect of embodiments of the present specification, there is provided an image processing method based on privacy protection, including: collecting a biological characteristic image; carrying out convolution processing on the biological characteristic image by utilizing a convolution kernel to obtain a desensitized image; and sending the desensitization image to a server, wherein the server can recover the biological characteristic image from the desensitization image.

In a third aspect of the embodiments of the present specification, there is provided an image processing method based on privacy protection, including: obtaining a convolution kernel of a biological characteristic image, wherein the biological characteristic image comprises privacy information; carrying out convolution processing on the biological characteristic image by utilizing a convolution core to obtain a desensitized image after protecting the private information; and sending the desensitization image to a server, wherein the server can recover the biological characteristic image from the desensitization image.

In a fourth aspect of embodiments of the present specification, there is provided an image processing method based on privacy protection, including: obtaining a desensitization image, wherein the desensitization image is obtained by performing convolution processing on a biological characteristic image by a convolution kernel; constructing a recovery matrix according to the convolution kernel; and restoring the desensitized image by using a restoration matrix to obtain the biological characteristic image.

In a fifth aspect of embodiments of the present specification, there is provided an image processing method based on privacy protection, including: obtaining a desensitization image, wherein the desensitization image is used for protecting privacy information in a biological characteristic image; determining recovery parameters for the desensitized image; and recovering the desensitized image by utilizing recovery parameters to obtain a biological characteristic image containing private information.

A sixth aspect of embodiments of the present specification provides an image processing apparatus based on privacy protection, including: the acquisition unit is used for acquiring a biological characteristic image; the processing unit is used for carrying out convolution processing on the biological characteristic image by utilizing a convolution kernel to obtain a desensitized image; and the sending unit is used for sending the desensitization image to a server, and the server can recover the biological characteristic image from the desensitization image.

A seventh aspect of the embodiments of the present specification provides an image processing apparatus based on privacy protection, including: the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a convolution kernel of a biological characteristic image, and the biological characteristic image contains privacy information; the processing unit is used for carrying out convolution processing on the biological characteristic image by utilizing a convolution core to obtain a desensitized image after privacy information is protected; and the sending unit is used for sending the desensitization image to a server, and the server can recover the biological characteristic image from the desensitization image.

An eighth aspect of embodiments of the present specification provides an image processing apparatus based on privacy protection, including: the acquiring unit is used for acquiring a desensitization image, and the desensitization image is obtained by performing convolution processing on a biological characteristic image by a convolution kernel; the construction unit is used for constructing a recovery matrix according to the convolution kernel; and the recovery unit is used for recovering the desensitized image by using the recovery matrix to obtain the biological characteristic image.

A ninth aspect of embodiments of the present specification provides an image processing apparatus based on privacy protection, including: the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a desensitization image which is used for protecting privacy information in a biological characteristic image; a determination unit configured to determine a recovery parameter of the desensitized image; and the recovery unit is used for recovering the desensitized image by utilizing the recovery parameters to obtain the biological characteristic image containing the private information.

In a tenth aspect of embodiments of the present specification, there is provided an electronic apparatus including: at least one processor; a memory storing program instructions configured to be suitable for execution by the at least one processor, the program instructions comprising instructions for performing the method of the second or third aspect.

According to the technical scheme provided by the embodiment of the specification, the terminal equipment can perform convolution processing on the biological characteristic image by utilizing a convolution kernel to obtain the desensitized image. The server may construct a recovery matrix using the convolution kernel; the desensitization image may be restored using a restoration matrix to obtain the biometric image. In this way, the privacy information in the biometric image can be protected by the convolution process. On the other hand, the desensitized image may be restored such that the biometric image may be restored from the desensitized image when needed, so that the biometric image may be used further in subsequent processes.

Drawings

In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, the drawings in the following description are only some embodiments described in the present specification, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a schematic flow chart illustrating an image processing method based on privacy protection in an embodiment of the present disclosure;

FIG. 2 is a flowchart illustrating an image processing method based on privacy protection in an embodiment of the present disclosure;

FIG. 3 is a flowchart illustrating an image processing method based on privacy protection according to an embodiment of the present disclosure;

FIG. 4 is a flowchart illustrating an image processing method based on privacy protection in an embodiment of the present disclosure;

FIG. 5 is a flowchart illustrating an image processing method based on privacy protection in an embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of an image processing apparatus based on privacy protection in an embodiment of the present specification;

FIG. 7 is a schematic structural diagram of an image processing apparatus based on privacy protection in an embodiment of the present specification;

FIG. 8 is a schematic structural diagram of an image processing apparatus based on privacy protection in an embodiment of the present specification;

FIG. 9 is a schematic structural diagram of an image processing apparatus based on privacy protection in an embodiment of the present specification;

fig. 10 is a schematic structural diagram of an electronic device in an embodiment of the present specification;

FIG. 11 is a diagram illustrating the determination of convolution regions in a channel image according to an embodiment of the present disclosure;

FIG. 12 is a diagram illustrating a convolution process performed on a biometric image according to an embodiment of the present disclosure;

FIG. 13 is a schematic diagram of a restoration process for a desensitized image according to an embodiment of the present disclosure;

FIG. 14 is a diagram illustrating the construction of a transformation matrix from convolution kernels in accordance with an embodiment of the present disclosure.

Detailed Description

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step should fall within the scope of protection of the present specification.

The embodiment of the specification provides an image processing system.

In some embodiments, the image processing system may include a terminal device. The terminal devices include, but are not limited to, smart phones, tablet electronic devices, portable computers, Personal Computers (PCs), face recognition devices (e.g., face swipes), fingerprint recognition devices (e.g., fingerprint machines), and the like. The terminal device may include a sensor. The sensor may include a camera, a fingerprint sensor, or the like. The terminal device can control the sensor to acquire the biological characteristic image. The biometric image may be a face image, an iris image, a fingerprint image, a palm print image, or the like. The biometric image may include private information. The private information may include face information, iris information, sclera information, fingerprint information, palm print information, heartbeat information, pulse information, chromosome information, tooth bite marks, and the like.

In some embodiments, the image processing system may further include a server. The server may be a single server, a server cluster composed of a plurality of servers, or a server deployed in the cloud.

In some embodiments, a convolution kernel may be agreed in advance between the terminal device and the server. Specifically, a convolution kernel may be agreed in advance between the terminal device and the server. Or, a plurality of convolution kernels and the use sequence of the plurality of convolution kernels in the convolution process may be agreed in advance between the terminal device and the server.

In some embodiments, the terminal device may generate a convolution kernel; the convolution kernel may be sent to the server. The server may receive the convolution kernel. Alternatively, the server may also generate a plurality of convolution kernels; the plurality of convolution kernels and an order of use of the plurality of convolution kernels in a convolution process may be sent to the server. The server may receive the plurality of convolution kernels and an order of use of the plurality of convolution kernels in a convolution process.

In some embodiments, the terminal device may perform convolution processing on the acquired biometric image by using a convolution kernel to obtain a desensitized image; the desensitized image may be sent to a server. The server may receive the desensitized image; a recovery matrix may be constructed from the convolution kernel; the desensitization image may be restored using a restoration matrix to obtain the biometric image. The biometric image may be used by the server in subsequent processes. For example, the biometric image may be used for identification by the server.

In this way, the privacy information in the biometric image can be protected by the convolution process. For example, the terminal device may transmit a desensitized image to the server instead of transmitting a biometric image, so that privacy leakage due to illegal theft of the transmitted image can be avoided. For another example, the server may store desensitized images instead of biometric images, thereby being able to avoid privacy disclosure due to stored images being illegally stolen. On the other hand, desensitized images may be restored. Thus, the server can recover the biological characteristic image from the desensitized image when needed, so that the biological characteristic image can be used by the server in subsequent processes.

An image processing method based on privacy protection according to an embodiment of the present specification is described below with reference to fig. 1, fig. 12, fig. 13, and fig. 14. The image processing method based on privacy protection may include the following steps.

Step S101: the terminal equipment collects the biological characteristic image.

In some embodiments, the biometric image may be a face image, an iris image, a fingerprint image, a palm print image, or the like. The biometric image may include private information. The private information may include face information, iris information, sclera information, fingerprint information, palm print information, heartbeat information, pulse information, chromosome information, tooth bite marks, and the like.

In some embodiments, the biometric image may include at least one color channel. For example, the biometric image may be captured by an IR camera (infrared camera), and the biometric image may include a color channel. As another example, the color space of the biometric image may include a YUV color space, a YCbCr color space, an RGB color space, an HSL color space, and the like. The YUV color space may include 3 color channels, Y, U and V, with color channel Y representing luminance and color channel U and V representing chrominance. The YCbCr color space may include 3 color channels Y, Cb and Cr, with color channel Y representing luminance, color channel Cb representing blue chrominance, and color channel Cr representing red chrominance. The RGB color space may include R, G and B3 color channels, color channel R representing red, color channel G representing green, and color channel B representing blue. The HSL color space may include H, S and L3 color channels, where color channel H represents hue, color channel S represents saturation, and color channel L represents brightness.

In the biometric image, the pixel value of each pixel point may include at least one channel value, and each channel value corresponds to one color channel. For example, the color space of the biometric image may be an RGB color space, and the biometric image may include R, G and 3 color channels, such as B. In the biometric image, the pixel value of each pixel point may include a channel value of an R color channel, a channel value of a G color channel, and a channel value of a B color channel.

The biometric image may include at least one channel image, and each channel image may correspond to a color channel. In the channel image, each pixel point may include a channel value. For example, the color space of the biometric image may be an RGB color space, and the biometric image may include R, G and 3 color channels, such as B. The biometric image may include a channel image of an R color channel, a channel image of a G color channel, and a channel image of a B color channel. In the channel image of the R color channel, each pixel point may include a channel value of the R color channel. In the channel image of the G color channel, each pixel point may include a channel value of the G color channel. In the channel image of the B color channel, each pixel point may include a channel value of the B color channel.

In some embodiments, the terminal device may include a sensor. The sensor may include a camera, a fingerprint sensor, or the like. The terminal device can control the sensor to acquire the biological characteristic image. For example, the terminal device may be a smartphone. The smartphone may include a camera. The smart phone can provide a human-computer interaction interface. And a user can trigger a human face image acquisition instruction on the human-computer interaction interface. After receiving the face image acquisition instruction, the smart phone can control the camera to photograph the face of the user to obtain the face image.

Step S103: and the terminal equipment performs convolution processing on the biological characteristic image by using a convolution kernel to obtain a desensitized image.

In some embodiments, the convolution kernel may be a matrix. The convolution kernel may be a square matrix or other type of matrix. For example, the convolution kernel may be a3 × 3 square matrix, or a 4 × 4 square matrix, etc. The convolution kernel may include a plurality of data elements. The plurality of data elements may be random numbers. In particular, the plurality of data elements may be completely random. Alternatively, the plurality of data elements may also be random numbers subject to a particular mathematical distribution. The specific mathematical distribution may include a normal distribution, a uniform distribution, an exponential distribution, and the like. Alternatively, the sum of the plurality of data elements may also be a specific value, for example, may be 1.

The convolution kernel may be pre-agreed. Alternatively, the convolution kernel may be generated by the terminal device after the biometric image is acquired. The terminal device may generate the convolution kernel using a function that obeys a particular mathematical distribution. For example, the terminal device may generate the convolution kernel using a normal distribution function. Specifically, for example, the convolution kernel may be a3 × 3 square matrix. The terminal device may generate 9 random numbers; the 9 random numbers can be respectively input into a normal distribution function to obtain 9 values; the 9 values may be taken as 9 data elements in the convolution kernel. Of course, the terminal device may also generate the convolution kernel in other manners. For example, the convolution kernel may be a3 × 3 square matrix. The terminal device may generate 9 random numbers directly as 9 data elements in the convolution kernel.

In some embodiments, the desensitized image is used to protect private information in the biometric image.

In practical application, aiming at each channel image of the biological characteristic image, channel values of pixel points in the channel image can form a channel matrix; the terminal device may perform convolution operation on the convolution kernel and the channel matrix to obtain a channel image after convolution processing. The terminal device can generate a desensitized image by using the channel image after convolution processing.

Please refer to fig. 11. The terminal device may determine a plurality of convolution regions in the channel image. Specifically, the terminal device may construct a sliding window; the sliding window can slide on the channel image from the upper left corner of the channel image according to the horizontal step length and the longitudinal step length and from left to right in the horizontal direction and from top to bottom in the vertical direction; for each sliding, the sliding window may cover an area on the channel image, and the covered area may be used as a convolution area. Wherein the lateral side length of the sliding window may be equal to the number of columns of the convolution kernel. The sliding window may have a longitudinal side length equal to the number of rows of the convolution kernel. The size of the transverse step length and the size of the longitudinal step length can be flexibly set according to requirements. For example, the size of the horizontal step may be 1 pixel point, and the size of the vertical step may be 1 pixel point, so that the size of the desensitized image is equal to the size of the biometric image. As another example, the size of the lateral step can be equal to the lateral side length of the sliding window and the size of the longitudinal step can be equal to the longitudinal side length of the sliding window, such that the size of the desensitized image is less than the size of the biometric image.

The channel values of the pixels in the convolution region may form a sub-channel matrix. The terminal device may perform convolution operation on the convolution kernel and the subchannel matrix; the convolution operation result can be treated as convolution processingThe channel value of the pixel point in the channel image. For example, the convolution kernel may be a3 × 3 square matrix

An image coordinate system can be established by taking the upper left corner of the channel image as an origin, the horizontal rightward direction as the positive direction of the X axis and the vertical downward direction as the positive direction of the Y axis. The subchannel matrix of a certain convolution region in the channel image can be represented as

And f (x-1, y-1) ═ 100 represents the channel value of the pixel point with the coordinate value of (x-1, y-1). And f (x, y-1) ═ 100 represents the channel value of the pixel with the coordinate value of (x, y-1). And f (x +1, y-1) ═ 100 represents the channel value of the pixel point with the coordinate value of (x +1, y-1). And f (x-1, y) ═ 100 represents the channel value of the pixel with the coordinate value of (x-1, y). And f (x, y) ═ 200 represents the channel value of the pixel with the coordinate value of (x, y). And f (x +1, y) ═ 100 represents the channel value of the pixel with the coordinate value of (x +1, y). And f (x-1, y +1) ═ 100 represents the channel value of the pixel point with the coordinate value of (x-1, y + 1). If f (x, y +1) ═ 100, the channel value of the pixel whose coordinate value is (x, y +1) is represented. If f (x +1, y +1) ═ 100 represents the channel value of the pixel whose coordinate value is (x +1, y + 1). The terminal device can use convolution kernel

And subchannel matrix

Carrying out convolution operation; the convolution operation result can be obtained

0×100+0.01×120+0.02×120+0.4×200+0.3×200+0.05×150+0.02×250+0×250+0.2×200＝34.1

And the channel value is used as the channel value of the pixel point with the coordinate value of (x, y) in the channel image after convolution processing.

The biometric image may comprise a channel image. The terminal device can obtain a channel image after convolution processing through convolution processing. The terminal device may use the convolved channel image as a desensitized image. Alternatively, the biometric image may include a plurality of channel images. The terminal device may obtain a plurality of convolution-processed channel images by convolution processing. The terminal equipment can perform channel combination on the channel images after the convolution processing to obtain a desensitized image. For example, the biometric image may include a channel image of an R color channel, a channel image of a G color channel, and a channel image of a B color channel. The terminal device may obtain a convolution-processed channel image of an R color channel, a convolution-processed channel image of a G color channel, and a convolution-processed channel image of a B color channel. The terminal device can perform channel combination on the channel image after convolution processing of the R color channel, the channel image after convolution processing of the G color channel and the channel image after convolution processing of the B color channel to obtain a desensitized image.

The terminal device can directly perform convolution processing on the biological characteristic image. Or, in order to facilitate convolution operation, the terminal device may further expand the boundary of the biometric image; the biometric image after the boundary expansion can be subjected to convolution processing. The biometric image after the expansion of the boundary may include an expanded region. In the extended area, the pixel value of the pixel point may be a fixed value. For example, in the extended area, the channel values of the pixels may all be 0. So that the expanded area may be black. For another example, in the extended area, the channel values of the pixels may all be 255. So that the extended area may be white.

In some embodiments, the number of convolution kernels may be 1. The terminal device can perform convolution processing on the biological characteristic image by using a convolution kernel to obtain a desensitized image. Alternatively, the number of the convolution kernels may be plural. The terminal device can perform multilayer convolution processing on the biological characteristic image by using a plurality of convolution kernels to obtain a desensitized image.

In the case where the number of convolution kernels is plural, the terminal device may iteratively perform the following steps with respect to the image to be processed, with the biometric image as the image to be processed, until unused convolution kernels are no longer included in the plural convolution kernels.

Step S1031: selecting an unused convolution kernel from the plurality of convolution kernels;

step S1032: and carrying out convolution processing on the image to be processed by utilizing the selected convolution kernel to obtain a new image to be processed.

After the end of the iteration, the terminal device can treat the current image to be processed as a desensitized image. The terminal device may randomly select an unused convolution kernel from the plurality of convolution kernels; alternatively, the unused convolution kernels may be selected from the plurality of convolution kernels in a particular order. The particular order may be an order of use of the plurality of convolution kernels in a convolution process. The process of performing convolution processing on the image to be processed by using the convolution kernel can be seen in the previous embodiment.

For example, the biometric image may be X. The convolution kernels may include a1, a2, a 3. The terminal device may perform convolution processing on the biometric image X by using a convolution kernel a1 to obtain a processing result X a 1; the processing result X a1 may be convolved with a convolution kernel a2 to obtain a processing result (X a1) a 2; the processing result (X × a1) × a2 may be subjected to convolution processing using a convolution kernel a3, resulting in a desensitized image Y ((X × a1) × a2) × a 3.

In some embodiments, the terminal device may identify a privacy zone in the biometric image; the privacy zone may be convolved with a convolution kernel to obtain a desensitized image. The privacy zone may be a zone in which the private information is located. The terminal device may employ an algorithm to identify a privacy zone in the biometric image. For example, the terminal device may employ an image segmentation algorithm to identify privacy zones. Alternatively, the terminal device may also identify the privacy zone using a machine learning model. The machine learning model may include a neural network model, a support vector machine model, or the like.

The procedure of performing convolution processing on the privacy area by using the convolution kernel can be referred to the previous embodiment.

On the one hand, the terminal equipment can only carry out convolution processing on the privacy area, and the operation amount is reduced. In another aspect, the desensitized image includes other regions (e.g., background regions) of the biometric image other than the privacy region, such that the desensitized image may directly include non-privacy information (e.g., background information of the biometric image).

Step S105: the terminal device sends the desensitized image to the server.

Step S107: the server receives the desensitized image.

In some embodiments, the terminal device may send the desensitized image to the server. The server may receive a desensitized image.

In some embodiments, the terminal device may identify a privacy zone in the biometric image; the privacy zone may be convolved with a convolution kernel to obtain a desensitized image. In this way the terminal device can also send location information to the server. The server may also receive location information. The location information is used to represent the location of the privacy zone in the biometric image. The location information may include coordinate values. For example, the privacy zone may be a rectangular zone. The position information may include coordinate values of a vertex at an upper left corner and coordinate values of a vertex at a lower right corner of the rectangular region.

In some embodiments, the terminal device may also send the convolution kernel to the server. The server may also receive a convolution kernel.

The terminal device may perform convolution processing on the biometric image using a convolution kernel. The terminal device can then send a convolution kernel to the server. The server may receive a convolution kernel. Alternatively, the terminal device may further perform multi-layer convolution processing on the biometric image using a plurality of convolution kernels. Thus, the terminal device can send the plurality of convolution kernels and the use sequence of the plurality of convolution kernels in the convolution process to the server. The server may receive the plurality of convolution kernels and an order of use of the plurality of convolution kernels in a convolution process.

The terminal device can directly send the plurality of convolution kernels and the use sequence of the plurality of convolution kernels in the convolution process to the server. The server may receive a plurality of convolution kernels and an order of use of the plurality of convolution kernels in a convolution process. Alternatively, the terminal device may transmit a plurality of convolution kernels arranged in a specific order to the server. The server may receive a plurality of convolution kernels arranged in a particular order. The specific order may be an order of use of the plurality of convolution kernels in the convolution process.

In some embodiments, the server may perform subsequent steps S109-S111 directly after receiving the desensitized image. Alternatively, the server may also store the desensitized image (or correspondingly store desensitized image and location information) after receiving the desensitized image. The server may read the stored desensitization image (or read the stored desensitization image and location information) when needed; the subsequent steps S109-S111 may be performed.

Step S109: and the server constructs a recovery matrix according to the convolution kernel.

In some embodiments, the convolution kernel may be pre-agreed; or, the information may be sent by the terminal device.

In some embodiments, the convolution operation may be converted to a multiplication operation of a matrix. Specifically, the convolution operation of the convolution kernel and the channel matrix may be converted into a multiplication operation of the transform matrix and the channel matrix. In this way, if the inverse matrix of the transform matrix is used as the restoration matrix, the channel matrix can be restored from the convolution operation result of the convolution kernel and the channel matrix by using the restoration matrix.

Based on the above considerations, the server may construct a transformation matrix from the convolution kernel and the dimensions of the desensitized image; an inverse of the transformation matrix may be calculated as a recovery matrix. The transformation matrix has the following characteristics: the convolution operation result of the convolution kernel and a certain matrix is equal to the multiplication operation result of the transformation matrix and the matrix.

The server can determine the number of rows and columns of a transformation matrix according to the size of the desensitized image; the data elements in the transformation matrix may be determined from the data elements in the convolution kernel. In practical application, the server may determine the data elements in the transformation matrix from the data elements in the convolution kernel by means of splicing.

For example, the convolution kernel a may be an n × n square matrix. The desensitized image may have a lateral edge length W and a longitudinal edge length H. The server may construct the transformation matrix a. The transformation matrix a may be a square matrix of m × m, where m is W × H. The ith row and jth column data element A in the transformation matrix A_i,j＝a_{imodn，jmodn}，a_{imodn，jmodn}Denotes the i mod n row and j mod n column data elements in the convolution kernel a, and mod denotes the remainder operation. The transformation matrix a has the following characteristics: a ═ C ═ dot (a, C). a × C represents the convolution operation result of the convolution kernel a and the matrix C, and dot (a, C) represents the multiplication operation result of the transformation matrix a and the matrix C. The server may compute an inverse matrix a of the transformation matrix a^-1As a recovery matrix.

In some embodiments, the number of recovery matrices may be equal to the number of convolution kernels. Specifically, the number of the convolution kernels may be one or more, and the number of the recovery matrices may also be one or more.

For each convolution kernel, the server may construct a transformation matrix based on the convolution kernel and the size of the desensitized image; the inverse of the transform matrix may be computed as the recovery matrix corresponding to the convolution kernel. For example, the desensitized image Y may be obtained by subjecting the biometric image X to a multi-layer convolution process by convolution kernels a1, a2, a 3. Y ═ a (X × a1) × a2) × a 3. The server may construct a restoration matrix A1 based on the size of the convolution kernel a1 and the desensitized image Y^-1(ii) a The restoration matrix A2 may be constructed from the size of the convolution kernel a2 and the desensitized image Y^-1(ii) a The restoration matrix A3 may be constructed from the size of the convolution kernel A3 and the desensitized image Y^-1。

Step S111: and the server recovers the desensitization image by using a recovery matrix to obtain the biological characteristic image.

In some embodiments, the desensitized image may include at least one channel image, and each channel image may correspond to a color channel. For each channel image of the desensitization image, channel values of pixel points in the channel image can form a channel matrix; the server may multiply the restoration matrix and the channel matrix to obtain a restored channel image. The server may generate the biometric image using the restored channel image.

The desensitized image may include a channel image, and the server may obtain a restored channel image by restoring processing. The server may use the restored channel image as a desensitized image. Alternatively, the desensitization image may include a plurality of channel images, and the server may obtain a plurality of recovery-processed channel maps by recovery processing. The server can perform channel merging on the multiple restored channel images to obtain a desensitized image.

In some embodiments, the number of recovery matrices may be 1. The server can recover the desensitized image by using a recovery matrix to obtain the biological characteristic image. Alternatively, the number of the recovery matrices may be plural. The server can perform multi-layer restoration processing on the desensitized image by using a plurality of restoration matrixes to obtain the biological characteristic image.

In the case where the number of the restoration matrices is plural, the server may iteratively perform the following steps for the image to be processed, with the desensitized image being the image to be processed, until unused restoration matrices are no longer included in the plural restoration matrices.

Step S1111: selecting an unused recovery matrix from the plurality of recovery matrices;

step S1112: and restoring the image to be processed by using the selected restoration matrix to obtain a new image to be processed.

After the iteration is finished, the server can take the current image to be processed as the biological characteristic image.

Wherein the server may select an unused recovery matrix from the plurality of recovery matrices in a particular order. The particular order may be an order of use of the plurality of convolution kernels in a convolution process. The specific order may be pre-agreed; or may be transmitted by the terminal device. The process of performing the restoration process on the image to be processed by using the restoration matrix can be seen in the previous embodiment.

For example, the desensitized image Y may be obtained by subjecting the biometric image X to a multi-layer convolution process by convolution kernels a1, a2, a 3. Y ═ a (X × a1) × a2) × a 3. The server may construct a restoration matrix A1 based on the size of the convolution kernel a1 and the desensitized image Y^-1(ii) a The restoration matrix A2 may be constructed from the size of the convolution kernel a2 and the desensitized image Y^-1(ii) a The restoration matrix A3 may be constructed from the size of the convolution kernel A3 and the desensitized image Y^-1. The server may utilize a recovery matrix A1^-1The desensitized image Y is subjected to restoration processing to obtain a processing result dot (Y, A1)^-1) (ii) a The recovery matrix a2 may be utilized^-1For the processing result dot (Y, A1)^-1) Recovery processing is carried out to obtain a processing result dot (dot (Y, A1)^-1),A2^-1) (ii) a The recovery matrix a3 may be utilized^-1To the processing result

dot(dot(Y,A1^-1),A2^-1) Performing recovery processing to obtain biological characteristic image

X＝dot(dot(dot(Y,A1^-1),A2^-1),A3^-1)。

In some embodiments, the server may also receive location information representing a location of the privacy zone in the biometric image. The server may perform restoration processing on the region represented by the position information in the desensitized image using a restoration matrix to obtain a biometric image. The procedure of performing the restoration process on the region represented by the position information in the desensitized image using the restoration matrix can be seen in the previous embodiment.

In the image processing method according to the embodiment of the present specification, the terminal device may perform convolution processing on the biological characteristic image by using a convolution kernel to obtain the desensitized image. The server may construct a recovery matrix using the convolution kernel; the desensitization image may be restored using a restoration matrix to obtain the biometric image. In this way, the privacy information in the biometric image can be protected by the convolution process. On the other hand, the desensitized image may be restored such that the biometric image may be restored from the desensitized image when needed, so that the biometric image may be used further in subsequent processes.

An example of a scenario of an embodiment of the present specification is described below.

The terminal device may be a smartphone. The smartphone may include a camera. The smart phone can provide a human-computer interaction interface. And a user can trigger a human face image acquisition instruction on the human-computer interaction interface. For example, a user may trigger a facial image acquisition instruction by clicking a button on the human-computer interaction interface. After receiving the face image acquisition instruction, the smart phone can control the camera to photograph the face of the user to obtain a face image X.

The smartphone may generate 3 convolution kernels a1, a2, a 3. The convolution kernels a1, a2, a3 may all be 3 × 3 square matrices. The terminal device may perform convolution processing on the biometric image X by using a convolution kernel a1 to obtain a processing result X a 1; the convolution kernel a2 may be used to continue the convolution processing on the processing result X a1, resulting in a processing result (X a1) a 2; the convolution kernel a3 may be used to continue the convolution process on the processing result (X × a1) × a2, resulting in desensitized image Y ((X × a1) × a2) × a 3.

The smartphone may send the server the order of use of convolution kernels a1, a2, a3, desensitized image Y, and convolution kernels a1, a2, a 3. The server may receive the order of use of the convolution kernels a1, a2, a3, desensitized image Y, and convolution kernels a1, a2, a 3. The server may construct a restoration matrix A1 based on the size of the convolution kernel a1 and the desensitized image Y^-1(ii) a The restoration matrix A2 may be constructed from the size of the convolution kernel a2 and the desensitized image Y^-1(ii) a The restoration matrix A3 may be constructed from the size of the convolution kernel A3 and the desensitized image Y^-1. The server may utilize a recovery matrix A1^-1The desensitized image Y is subjected to restoration processing to obtain a processing result dot (Y, A1)^-1) (ii) a The recovery matrix a2 may be utilized^-1For the processing result dot (Y, A1)^-1) Recovery processing is carried out to obtain a processing result dot (dot (Y, A1)^-1),A2^-1) (ii) a The recovery matrix a3 may be utilized^-1For the processing result dot (dot (Y, A1)^-1),A2^-1) The restoration process is performed to obtain a biometric image X ═ dot (dot (Y, a1)^-1),A2^-1),A3^-1)。

The above embodiments may be implemented separately as an image processing method based on privacy protection on the terminal device side and an image processing method based on privacy protection on the server side. An image processing method based on privacy protection on the terminal device side and an image processing method based on privacy protection on the server side in the embodiment of the present specification will be described below with reference to fig. 2, fig. 3, fig. 4, and fig. 5.

Please refer to fig. 2. The present specification provides another embodiment of an image processing method based on privacy protection. The execution subject of the image processing method based on privacy protection may be a terminal device, and specifically may include the following steps.

Step S21: a biometric image is acquired.

Step S23: and carrying out convolution processing on the biological characteristic image by utilizing a convolution kernel to obtain a desensitized image.

Step S25: and sending the desensitization image to a server, wherein the server can recover the biological characteristic image from the desensitization image.

Please refer to fig. 3. The present specification provides another embodiment of an image processing method based on privacy protection. The execution subject of the image processing method based on privacy protection may be a terminal device, and specifically may include the following steps.

Step S31: obtaining a convolution kernel of a biometric image, wherein the biometric image comprises privacy information.

In some embodiments, the terminal device may obtain a convolution kernel; alternatively, multiple convolution kernels may also be obtained.

In some embodiments, the terminal device and the server may agree on a convolution kernel. The terminal device may obtain a contracted convolution kernel. Alternatively, the terminal device may also generate a convolution kernel.

Step S33: and carrying out convolution processing on the biological characteristic image by utilizing a convolution core to obtain a desensitized image after protecting the private information.

Step S35: and sending the desensitization image to a server, wherein the server can recover the biological characteristic image from the desensitization image.

Please refer to fig. 4. The present specification provides another embodiment of an image processing method based on privacy protection. The execution subject of the image processing method based on privacy protection may be a server, and specifically may include the following steps.

Step S41: and acquiring a desensitization image, wherein the desensitization image is obtained by performing convolution processing on the biological characteristic image by a convolution kernel.

Step S43: and constructing a recovery matrix according to the convolution kernel.

Step S45: and restoring the desensitized image by using a restoration matrix to obtain the biological characteristic image.

Please refer to fig. 5. The present specification provides another embodiment of an image processing method based on privacy protection. The execution subject of the image processing method based on privacy protection may be a server, and specifically may include the following steps.

Step S51: obtaining a desensitization image, wherein the desensitization image is used for protecting privacy information in the biological characteristic image.

Step S53: determining recovery parameters for the desensitized image.

In some embodiments, the server may obtain one or more convolution kernels; one or more recovery matrices may be constructed from one or more convolution kernels. Specifically, the server may obtain one or more agreed convolution kernels; alternatively, one or more convolution kernels sent by the terminal device may also be received.

Step S55: and recovering the desensitized image by utilizing recovery parameters to obtain a biological characteristic image containing private information.

Please refer to fig. 6. The embodiment of the present specification further provides an image processing apparatus based on privacy protection, which may be disposed in a terminal device, and specifically may include the following units.

An acquisition unit 61 for acquiring a biometric image;

a processing unit 63, configured to perform convolution processing on the biological characteristic image by using a convolution kernel to obtain a desensitized image;

a sending unit 65, configured to send the desensitized image to a server, where the server can recover the biometric image from the desensitized image.

Please refer to fig. 7. The embodiment of the present specification further provides an image processing apparatus based on privacy protection, which may be disposed in a terminal device, and specifically may include the following units.

An obtaining unit 71, configured to obtain a convolution kernel of a biometric image, where the biometric image includes privacy information;

a processing unit 73, configured to perform convolution processing on the biometric image by using a convolution kernel to obtain a desensitized image in which the private information is protected;

a sending unit 75, configured to send the desensitized image to a server, where the server can retrieve the biometric image from the desensitized image.

Please refer to fig. 8. The embodiment of the present specification further provides an image processing apparatus based on privacy protection, which may be disposed in a server, and specifically may include the following units.

An acquisition unit 81 configured to acquire a desensitization image obtained by performing convolution processing on a biometric image by a convolution kernel;

a construction unit 83, configured to construct a recovery matrix according to the convolution kernel;

and the recovery unit 85 is used for performing recovery processing on the desensitized image by using a recovery matrix to obtain the biological characteristic image.

Please refer to fig. 9. The embodiment of the present specification further provides an image processing apparatus based on privacy protection, which may be disposed in a server, and specifically may include the following units.

An obtaining unit 91, configured to obtain a desensitization image, where the desensitization image is used to protect privacy information in a biological feature image;

a determination unit 93 for determining a recovery parameter of the desensitized image;

and a recovery unit 95, configured to perform recovery processing on the desensitized image by using a recovery parameter, so as to obtain a biometric image including privacy information.

An embodiment of an electronic device of the present description is described below. Fig. 10 is a schematic diagram of a hardware configuration of the electronic apparatus in this embodiment. As shown in fig. 10, the electronic device may include one or more processors (only one of which is shown), memory, and a transmission module. Of course, it is understood by those skilled in the art that the hardware structure shown in fig. 10 is only an illustration, and does not limit the hardware structure of the electronic device. In practice the electronic device may also comprise more or fewer component elements than those shown in fig. 10; or have a different configuration than that shown in fig. 10.

The memory may comprise high speed random access memory; alternatively, non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory may also be included. Of course, the memory may also comprise a remotely located network memory. The memory may be used to store program instructions or modules of application software, which may be implemented, for example, based on the embodiments corresponding to fig. 2, fig. 3, fig. 4, or fig. 5.

The processor may be implemented in any suitable way. For example, the processor may take the form of, for example, a microprocessor or processor and a computer-readable medium that stores computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, an embedded microcontroller, and so forth. The processor may read and execute the program instructions or modules in the memory.

The transmission module may be used for data transmission via a network, for example via a network such as the internet, an intranet, a local area network, a mobile communication network, etc.

This specification also provides one embodiment of a computer storage medium. The computer storage medium includes, but is not limited to, a Random Access Memory (RAM), a Read-Only Memory (ROM), a Cache (Cache), a Hard Disk (HDD), a Memory Card (Memory Card), and the like. The computer storage medium stores computer program instructions. The computer program instructions when executed implement: the present specification refers to the embodiments of fig. 2, fig. 3, fig. 4 or fig. 5.

It should be noted that, in the present specification, each embodiment is described in a progressive manner, and the same or similar parts in each embodiment may be referred to each other, and each embodiment focuses on differences from other embodiments. In particular, for the method embodiment, the apparatus embodiment, the electronic device embodiment, and the computer storage medium embodiment that are implemented on a single side, since they are substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment. In addition, it is understood that one skilled in the art, after reading this specification document, may conceive of any combination of some or all of the embodiments listed in this specification without the need for inventive faculty, which combinations are also within the scope of the disclosure and protection of this specification.

In the 90 s of the 20 th century, improvements in a technology could clearly distinguish between improvements in hardware (e.g., improvements in circuit structures such as diodes, transistors, switches, etc.) and improvements in software (improvements in process flow). However, as technology advances, many of today's process flow improvements have been seen as direct improvements in hardware circuit architecture. Designers almost always obtain the corresponding hardware circuit structure by programming an improved method flow into the hardware circuit. Thus, it cannot be said that an improvement in the process flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), is an integrated circuit whose Logic functions are determined by programming the Device by a user. A digital system is "integrated" on a PLD by the designer's own programming without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Furthermore, nowadays, instead of manually making an Integrated Circuit chip, such Programming is often implemented by "logic compiler" software, which is similar to a software compiler used in program development and writing, but the original code before compiling is also written by a specific Programming Language, which is called Hardware Description Language (HDL), and HDL is not only one but many, such as abel (advanced Boolean Expression Language), ahdl (alternate Hardware Description Language), traffic, pl (core universal Programming Language), HDCal (jhdware Description Language), lang, Lola, HDL, laspam, hardward Description Language (vhr Description Language), vhal (Hardware Description Language), and vhigh-Language, which are currently used in most common. It will also be apparent to those skilled in the art that hardware circuitry that implements the logical method flows can be readily obtained by merely slightly programming the method flows into an integrated circuit using the hardware description languages described above.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

From the above description of the embodiments, it is clear to those skilled in the art that the present specification can be implemented by software plus a necessary general hardware platform. Based on such understanding, the technical solutions of the present specification may be essentially or partially implemented in the form of software products, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and include instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments of the present specification.

The description is operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

This description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

While the specification has been described with examples, those skilled in the art will appreciate that there are numerous variations and permutations of the specification that do not depart from the spirit of the specification, and it is intended that the appended claims include such variations and modifications that do not depart from the spirit of the specification.

Claims

1. An image processing method based on privacy protection comprises the following steps:

the terminal equipment collects a biological characteristic image;

the terminal equipment performs convolution processing on the biological characteristic image by using a convolution kernel to obtain a desensitized image;

the terminal equipment sends the desensitization image to a server;

the server receives the desensitized image;

the server constructs a recovery matrix according to the convolution kernel;

and the server recovers the desensitization image by using a recovery matrix to obtain the biological characteristic image.

2. An image processing method based on privacy protection comprises the following steps:

collecting a biological characteristic image;

carrying out convolution processing on the biological characteristic image by utilizing a convolution kernel to obtain a desensitized image;

and sending the desensitization image to a server, wherein the server can recover the biological characteristic image from the desensitization image.

3. The method of claim 2, the convolving the biometric image, comprising:

and performing multilayer convolution processing on the biological characteristic image by using a plurality of convolution kernels.

4. The method of claim 2, the convolving the biometric image, comprising:

identifying a privacy zone in the biometric image;

performing convolution processing on the privacy area;

the method further comprises the following steps:

and sending the position information of the privacy zone in the biological characteristic image to a server.

5. The method of claim 2, the convolution kernel is generated using a function that obeys a particular mathematical distribution.

6. The method of claim 2, further comprising:

and sending the convolution kernel to a server.

7. An image processing method based on privacy protection comprises the following steps:

obtaining a convolution kernel of a biological characteristic image, wherein the biological characteristic image comprises privacy information;

carrying out convolution processing on the biological characteristic image by utilizing a convolution core to obtain a desensitized image after protecting the private information;

8. The method of claim 7, the obtaining a convolution kernel for a biometric image, comprising:

the convolution kernel is generated using a function that obeys a particular mathematical distribution.

9. The method of claim 7, the number of convolution kernels is plural;

the method for carrying out convolution processing on the biological characteristic image by utilizing the convolution kernel comprises the following steps:

10. The method of claim 7, further comprising:

the convolution kernel is sent to the server.

11. An image processing method based on privacy protection comprises the following steps:

obtaining a desensitization image, wherein the desensitization image is obtained by performing convolution processing on a biological characteristic image by a convolution kernel;

constructing a recovery matrix according to the convolution kernel;

and restoring the desensitized image by using a restoration matrix to obtain the biological characteristic image.

12. The method of claim 11, wherein the desensitized image is obtained by performing a multi-layer convolution process on the biometric image using a plurality of convolution kernels, and wherein constructing a restoration matrix from the convolution kernels comprises:

constructing a plurality of recovery matrices from the plurality of convolution kernels;

the restoration processing of the desensitized image by using the restoration matrix includes:

and carrying out multi-layer restoration processing on the desensitized image by using a plurality of restoration matrixes.

13. The method of claim 11, said constructing a restoration matrix from said convolution kernel, comprising:

constructing a transformation matrix according to the sizes of the convolution kernel and the desensitization image;

the inverse of the transformation matrix is calculated as the recovery matrix.

14. The method of claim 11, the performing a recovery process on the desensitized image comprising:

acquiring position information, wherein the position information is used for representing the position of a privacy area in the biological characteristic image;

and performing recovery processing on the region represented by the position information in the desensitized image by using a recovery matrix.

15. The method of claim 11, further comprising:

acquiring the pre-agreed convolution kernel; alternatively, the convolution kernel is received.

16. An image processing method based on privacy protection comprises the following steps:

obtaining a desensitization image, wherein the desensitization image is used for protecting privacy information in a biological characteristic image;

determining recovery parameters for the desensitized image;

and recovering the desensitized image by utilizing recovery parameters to obtain a biological characteristic image containing private information.

17. The method of claim 16, wherein the desensitized image is obtained by convolving a biometric image with a convolution kernel, and wherein determining recovery parameters for the desensitized image comprises:

constructing a recovery matrix according to the convolution kernel;

the restoring the desensitized image by using the restoring parameters comprises the following steps:

and carrying out recovery processing on the desensitized image by using a recovery matrix.

18. The method of claim 17, wherein the desensitized image is obtained by performing a multi-layer convolution process on the biometric image using a plurality of convolution kernels, and wherein constructing a restoration matrix from the convolution kernels comprises:

19. The method of claim 17, said constructing a restoration matrix from said convolution kernel, comprising:

the inverse of the transformation matrix is calculated as the recovery matrix.

20. An image processing apparatus based on privacy protection, comprising:

the acquisition unit is used for acquiring a biological characteristic image;

the processing unit is used for carrying out convolution processing on the biological characteristic image by utilizing a convolution kernel to obtain a desensitized image;

and the sending unit is used for sending the desensitization image to a server, and the server can recover the biological characteristic image from the desensitization image.

21. An image processing apparatus based on privacy protection, comprising:

the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a convolution kernel of a biological characteristic image, and the biological characteristic image contains privacy information;

the processing unit is used for carrying out convolution processing on the biological characteristic image by utilizing a convolution core to obtain a desensitized image after privacy information is protected;

22. An image processing apparatus based on privacy protection, comprising:

the acquiring unit is used for acquiring a desensitization image, and the desensitization image is obtained by performing convolution processing on a biological characteristic image by a convolution kernel;

the construction unit is used for constructing a recovery matrix according to the convolution kernel;

and the recovery unit is used for recovering the desensitized image by using the recovery matrix to obtain the biological characteristic image.

23. An image processing apparatus based on privacy protection, comprising:

the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a desensitization image which is used for protecting privacy information in a biological characteristic image;

a determination unit configured to determine a recovery parameter of the desensitized image;

and the recovery unit is used for recovering the desensitized image by utilizing the recovery parameters to obtain the biological characteristic image containing the private information.

24. An electronic device, comprising:

at least one processor;

a memory storing program instructions configured for execution by the at least one processor, the program instructions comprising instructions for performing the method of any of claims 2-19.