CN114817607A

CN114817607A - Image detection method, device, equipment and storage medium

Info

Publication number: CN114817607A
Application number: CN202210332877.5A
Authority: CN
Inventors: 张威
Original assignee: Shenzhen Sensetime Technology Co Ltd
Current assignee: Shenzhen Sensetime Technology Co Ltd
Priority date: 2022-03-30
Filing date: 2022-03-30
Publication date: 2022-07-29

Abstract

The embodiment of the application provides an image detection method, an image detection device, image detection equipment and a storage medium, wherein the method comprises the following steps: acquiring an image to be detected; determining at least two-dimensional image feature points of the image to be detected; and screening the target image matched with the image to be detected in a preset image library based on the image feature points of at least two dimensions.

Description

Image detection method, device, equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of image processing, and relates to but is not limited to an image detection method, an image detection device, image detection equipment and a storage medium.

Background

As computer vision is more and more commonly applied to various scenes in life, face recognition belongs to a scene with a higher proportion in computer vision application. In the related art, efficiency and accuracy of face feature point positions are generally balanced, and both low-dimensional features and high-dimensional features are balanced by the efficiency and the accuracy, so that the accuracy of face feature search is not high.

Disclosure of Invention

The embodiment of the application provides an image detection technical scheme.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides an image detection method, which comprises the following steps:

acquiring an image to be detected;

determining at least two-dimensional image feature points of the image to be detected;

and screening the target image matched with the image to be detected in a preset image library based on the image feature points of at least two dimensions.

An embodiment of the present application provides an image detection apparatus, the apparatus includes:

the first acquisition module is used for acquiring an image to be detected;

the first determining module is used for determining at least two-dimensional image feature points of the image to be detected;

and the first screening module is used for screening the target image matched with the image to be detected in a preset image library based on the image feature points of at least two dimensions.

Correspondingly, the embodiment of the application provides a computer storage medium, wherein computer-executable instructions are stored on the computer storage medium, and after being executed, the image detection method can be realized.

The embodiment of the application provides an electronic device, which comprises a memory and a processor, wherein the memory is stored with computer executable instructions, and the processor can realize the image detection method when running the computer executable instructions on the memory.

The embodiment of the application provides an image detection method, device, equipment and storage medium, aiming at an acquired image to be detected; by determining the image feature points of at least two dimensions of the image to be detected, multiple searches of the target image in the preset image library can be performed conveniently through the image feature points of at least two dimensions; thus, based on the image feature points of at least two dimensions, the target image which is more matched with the image to be detected can be screened out by screening for multiple times in the preset image library; therefore, the preset image library with large data volume can be effectively screened through the characteristics of at least two dimensions, and the efficiency and the precision of screening the target images can be improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein are incorporated into and constitute a part of this specification, and illustrate embodiments consistent with the embodiments of the present disclosure and, together with the description, serve to explain the technical solutions of the embodiments of the present disclosure. It is appreciated that the following drawings depict only some embodiments of the disclosed embodiments and are therefore not to be considered limiting of scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.

Fig. 1 is a schematic view of an implementation flow of an image detection method provided in an embodiment of the present application;

fig. 2 is a schematic flowchart of another implementation of the image detection method according to the embodiment of the present application;

fig. 3 is a schematic flowchart of another implementation of the image detection method according to the embodiment of the present application;

fig. 4 is a schematic flow chart illustrating an implementation of an image detection method according to an embodiment of the present application;

fig. 5 is a schematic flowchart of another implementation of the image detection method according to the embodiment of the present application;

fig. 6 is a schematic flowchart of another implementation of the image detection method according to the embodiment of the present application;

fig. 7 is a schematic view of an application scenario of an image detection method according to an embodiment of the present application;

fig. 8 is a schematic flowchart of another implementation of the image detection method according to the embodiment of the present application;

fig. 9 is a schematic view of another application scenario of the image detection method according to the embodiment of the present application;

FIG. 10 is a schematic view of an exemplary image detecting apparatus according to the present disclosure;

fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, specific technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings in the embodiments of the present application. The following examples are intended to illustrate the present application but are not intended to limit the scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

In the following description, references to the terms "first \ second \ third" are only to distinguish similar objects and do not denote a particular order, but rather the terms "first \ second \ third" are used to interchange specific orders or sequences, where appropriate, so as to enable the embodiments of the application described herein to be practiced in other than the order shown or described herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the application.

Before further detailed description of the embodiments of the present application, terms and expressions referred to in the embodiments of the present application will be described, and the terms and expressions referred to in the embodiments of the present application will be used for the following explanation.

1) Computer vision is a science for researching how to make a machine look, and further, it refers to that a camera and a computer are used to replace human eyes to perform machine vision of identifying, tracking and measuring a target, and further to perform graphic processing, so that the computer processing becomes an image more suitable for human eyes to observe or transmitted to an instrument to detect. As a scientific discipline, computer vision research-related theories and techniques attempt to build artificial intelligence systems that can acquire information from images or multidimensional data.

2) The face recognition belongs to the biological characteristic recognition technology, and is to distinguish organism individuals from the biological characteristics of organisms (generally, specially, people). Face recognition refers to the recognition of a human face using computer techniques of analytical comparison. Face recognition is a popular research field of computer technology, including face tracking detection, automatic image magnification adjustment, night infrared detection, automatic exposure intensity adjustment, and the like.

An exemplary application of the image detection device provided in the embodiments of the present application is described below, and the device provided in the embodiments of the present application may be implemented as various types of user terminals such as a notebook computer, a tablet computer, a desktop computer, a mobile device (e.g., a personal digital assistant, a dedicated messaging device, a portable game device) and the like having a data processing function, and may also be implemented as a server. In the following, an exemplary application will be explained when the device is implemented as a terminal or a server.

The method can be applied to an electronic device, and the functions realized by the method can be realized by calling a program code by a processor in the electronic device, although the program code can be stored in a computer storage medium.

The embodiment of the present application provides an image detection method, as shown in fig. 1, which is described with reference to the steps shown in fig. 1:

and S101, acquiring an image to be detected.

In some embodiments, the image to be detected may be acquired by an image acquisition device, or may be a received image transmitted by another device. The image to be detected can be an image with simple picture content or an image with complex picture content. The image to be detected can be an image acquired in any scene in life, for example, a pedestrian image acquired in a traffic scene, a vehicle image acquired in a traffic scene, a campus image acquired in a campus scene, an image of a certain store acquired in a mall scene, one or more frames of images selected from video clips, and the like.

In some possible implementations, the image to be detected may be one or more frames of a picture type image, and may also be one or more frames of an image selected from a video. The image to be detected may be an image obtained by performing image preprocessing on an originally acquired image, for example, performing operations such as noise reduction and region of interest extraction on the originally acquired image to obtain the image to be detected. Taking a pedestrian image with a picture including a human face as an example, obtaining an image to be detected by matting a human face region in the pedestrian image; therefore, the area occupied by the face region in the image to be detected is large, and the target image matched with the face region is conveniently screened in the preset image database through the face region.

And S102, determining image characteristic points of at least two dimensions of the image to be detected.

In some embodiments, the at least two dimensions represent dimensions for selecting key points for the image feature points, and different dimensions represent different numbers of key points selected by the target object in the image to be detected. Taking the image to be detected as the face image as an example, different dimensions represent different key points of the face extracted from the face image. In some embodiments, the at least two dimensions include: the image processing method comprises a high dimension, a middle dimension and a low dimension, wherein the number of image key points corresponding to the high dimension is larger than that of image key points corresponding to the middle dimension, and the number of image key points corresponding to the middle dimension is larger than that of image key points corresponding to the low dimension. For example, taking the image to be detected as a face image as an example, the number of the image key points corresponding to the high dimension may be 81 key points or 68 key points, the number of the image key points corresponding to the high dimension may be 49 key points, and the number of the image key points corresponding to the low dimension may be 21 key points.

In some possible implementation manners, image feature point extraction can be respectively carried out on the image to be detected according to different dimensions, so as to obtain image feature points with different dimensions; for example, feature point extraction is performed on an image to be detected according to different dimensions through a neural network, and the neural network may be a network with any architecture and capable of performing feature point extraction, such as a convolutional neural network or a residual neural network. The method can also be characterized in that feature points of an image to be detected are extracted according to high dimensionality, and then sampling is carried out according to medium and low dimensionality on the basis of the high dimensionality key points, so that medium and low dimensionality feature key points can be obtained. Therefore, by acquiring the image feature points of at least two dimensions of the image to be detected, the subsequent hierarchical search in the preset image library based on the image feature points of different dimensions can be facilitated, and the target image can be searched quickly and accurately.

And S103, screening the target image matched with the image to be detected in a preset image library based on the image feature points of at least two dimensions.

In some embodiments, the predetermined image library includes a plurality of image data; the picture content of these image data may be the same as or different from the picture content type of the image to be detected. The preset image library can store a large amount of image data and video data. Taking the image to be detected as a face image as an example, the preset image library may be a created face database, or may be a person image or a person video acquired by a camera. For example, a traffic image or a traffic video collected by a traffic camera. The target image matched with the image to be detected may be an image whose similarity with the image feature point of the image to be detected is greater than a certain preset similarity threshold, where the preset similarity threshold may be set based on task requirements, for example, the task requirements are to search for a target person in a portrait base, and then the preset similarity threshold may be set to a slightly larger value, for example, the preset similarity threshold is 0.99.

In some possible implementation manners, according to each dimension of the image feature points of at least two dimensions, the images matched with the image to be detected are screened step by step in the preset image library to obtain the target image. And sequencing at least two dimensions, and screening step by step in a preset image library according to the sequence from low dimension to high dimension to obtain the target image. In some possible implementation manners, firstly, according to image feature points with low dimensionality in at least two dimensionalities, an image set with high similarity to the image feature points with the low dimensionality is screened in a preset image library; then, on the basis of the image set, an image set with higher similarity to the image feature points of the medium dimension is further screened in the image set according to the image feature points of the medium dimension until a target image is screened or the screening dimension reaches the highest dimension, the screening is stopped, and the finally screened image is used as the target image.

In the embodiment of the application, for the acquired image to be detected; by determining the image feature points of at least two dimensions of the image to be detected, the target image can be conveniently searched in a preset image library step by step through the image feature points of at least two dimensions; therefore, based on the image feature points of at least two dimensions, a target image matched with the image to be detected is screened from a preset image library; therefore, the preset image library with large data volume can be effectively screened through the characteristics of at least two dimensions, and the efficiency and the precision of screening the target images can be improved.

In some embodiments, in order to obtain image feature points of different dimensions of an image to be detected, feature point extraction may be performed according to different dimensions, and sampling may be performed on the basis of high-dimensional features to obtain image feature points of different dimensions, that is, step S102 may be implemented in the following two ways:

the first method is as follows: the feature point extraction of the image to be detected is performed in at least two dimensions, and may be implemented by the following steps S121 and S122 (not shown in the figure):

and step S121, determining at least two dimensions representing different feature point quantities based on the scene where the image to be detected is located.

In some embodiments, the scene in which the image to be detected is located represents a scene corresponding to the picture content of the image to be detected, for example, if the picture content is a pedestrian on a traffic road, the scene is a traffic scene. According to the scene where the image to be detected is located, the task to be executed by the target object in the object to be detected is determined, so that key points which need to be extracted from the target object for executing the task can be accurately set, and multi-level dimensionality is set.

In some possible implementation manners, according to a pedestrian image in a traffic scene, it can be determined that a target object targeted in a task to which an image to be detected belongs is a pedestrian, so that a multi-level dimensionality for extracting feature points of the pedestrian can be determined, and at least two-level dimensionality can be obtained. If the picture content is a teaching building, the scene of the image to be detected is a campus, and then the number of key points required for similarity matching of the teaching building is analyzed based on a target object (teaching building) in the campus scene, so that different dimensions are set. The number of key points represented by each level of dimension is different, for example, the number of key points represented by a high dimension is larger than that represented by a medium dimension.

And S122, respectively extracting the characteristic points of the image to be detected on the at least two dimensions to obtain the image characteristic points of the at least two dimensions.

In some embodiments, feature point extraction is performed on the image to be detected in each dimension, so as to obtain image feature points in the dimension. Feature point extraction can be carried out on an image to be detected in each dimension through parallel neural network branches, and one neural network branch corresponds to the first-level dimension; the method can also be used for extracting the characteristic points of the image to be detected for multiple times by adopting a neural network to obtain the image characteristic points with at least two dimensions; wherein, each time of feature point extraction corresponds to a first-level dimension.

In one particular example, at least two dimensions include: the high dimensionality of the 68 key points, the middle dimensionality of the 49 key points and the low dimensionality of the 21 key points, so that the high-dimensionality, middle-dimensionality and low-dimensionality feature point extraction can be simultaneously carried out on the image to be detected; the features of any one of high dimension, middle dimension and low dimension can be extracted successively; the feature of two dimensions of high dimension, middle dimension and low dimension can be extracted simultaneously each time to obtain the image feature points of at least two dimensions.

Through the steps S121 and S122, feature point extraction can be performed according to different dimensions, mixed extraction of multiple feature dimensions is achieved, and subsequent step-by-step search in a preset image library based on the features of mixed extraction is facilitated.

The second method comprises the following steps: the image feature points of different dimensions are obtained by extracting the image feature points of high dimensions first and then sampling the image feature points of high dimensions, that is, the step S102 can be implemented by the following steps S123 and S124 (not shown in the figure):

and S123, extracting the characteristic points of the image to be detected in the j-th dimension to obtain the image characteristic points of the j-th dimension.

In some embodiments, j is a natural number equal to or greater than 2 and equal to or less than M. In a case where the at least two-stage dimension includes a first dimension and a second dimension, the j-th-stage dimension may be a highest-level dimension of the at least two-stage dimension, and the level of the second dimension is higher than the first dimension. And according to the number of the characteristic points represented by the j-th dimension, extracting the characteristic points of the image to be detected to obtain the j-th dimension image characteristic points. For example, if the number of feature points represented by the j-th dimension is 68, 68 feature points are extracted from the image to be detected, and the 68 feature points are image feature points of the j-th dimension.

And S124, sampling the image feature points of the j-level dimension based on the j-1-level dimension to obtain the image feature points of the j-1-level dimension.

In some embodiments, the level of the j dimension is higher than the level of the j-1 dimension. And sampling the image feature points of the j-th dimension according to the number of the feature points represented by the j-1-th dimension on the basis of the image feature points of the j-th dimension, thereby obtaining the image feature points corresponding to the j-1-th dimension. For example, the number of feature points represented by the j-1 th dimension is 49, and the 49 feature points are obtained by sampling the image feature points of the j-level dimension, namely the image feature points of the j-1 th dimension. Therefore, the low-dimensional feature points can be obtained by sampling the high-dimensional feature points, so that the extraction process of the feature points is simpler and more convenient, and the calculated amount can be reduced.

In some embodiments, according to the type of the original image file, the method for processing the original image file is determined, and then the image to be detected with higher image quality is obtained, that is, the step S101 may be implemented by the following two methods:

the first method is as follows: for the file of picture type, the image to be detected is obtained after preprocessing, that is, the above step S101 can be implemented by the following steps S111 and S112 (not shown in the figure):

step S111, an input original image file is acquired.

In some embodiments, the original image file may be an image file of an arbitrary scene of the input. The original image file may be a video file or a picture input by the user. For example, a video captured by an input camera, or a plurality of frames of pictures taken by the input camera, etc.

And step S112, responding to the fact that the type of the original image file is a picture type, preprocessing the original image file, and obtaining the image to be detected.

In some embodiments, if the type of the original image file is a picture type, for example, the multi-frame picture includes a traffic image of a pedestrian. And preprocessing the traffic image to obtain an image to be detected. Wherein the pretreatment comprises: image extraction of the region of interest, image noise reduction processing, image frame content correction (for example, if a pedestrian in the image is inclined, the pedestrian region is corrected), and the like. Therefore, a series of preprocessing operations are carried out on the original image file of the picture type, so that the obtained picture of the image to be detected is clear, the picture content is more targeted, and the extraction of the characteristic points is facilitated.

The second method comprises the following steps: for an original image file of a video type, the selected video frame is preprocessed to obtain an image to be detected, that is, the above step S101 can be implemented by the following steps S113 and S114 (not shown in the figure):

step S113, in response to that the type of the original image file is a video type, selecting at least one frame of video image of which the image quality satisfies a preset condition from the original image file.

In some embodiments, if the type of the original image file is a video, for example, the original image file is an input video segment, then video frames with higher image quality are extracted from the video to obtain at least one video frame. The image quality meeting the preset condition may be higher definition of the image, less image noise, higher image resolution, etc., so that the selected at least one frame of video image is easily recognized.

Step S114, preprocessing the at least one frame of video image to obtain the image to be detected.

In some embodiments, at least one frame of video image selected from the video is subjected to preprocessing operations such as image matting of a region of interest, image denoising processing, image picture content inversion and the like, so as to obtain an image to be detected with high image quality. Therefore, the video frame with the image quality meeting the preset conditions in the video is extracted, and the extracted video frame is preprocessed, so that the image to be detected with high image quality and capable of highlighting the region of interest is obtained.

In some embodiments, the screening of the preset image library is implemented according to the screening path plan by obtaining the path plan for screening the preset image library, that is, the step S103 may be implemented by the steps shown in fig. 2:

step S201, obtaining a screening path plan based on the at least two dimensions.

In some embodiments, the filtered path plan may be stored in a path plan library, in which a plurality of filtered path plans are stored, wherein different filtered path plans correspond to different dimensional levels. And obtaining a screening path plan corresponding to the dimension series according to the required dimension series.

In some possible implementations, the screening path planning may be a step-by-step screening according to a dimension series, or a screening from a low-level dimension to a high-level dimension according to a dimension series. For example, the at least two dimensions include 5 dimensions, and the filtering path plan may be filtering step by step according to an order from 1 to 5; the screening path planning can also be implemented by removing one of the 5-level dimensions and screening the remaining four levels from low dimension to high dimension.

In some possible implementations, the screening path plan may be obtained in the following two ways, that is, step S201 may be implemented in the following two ways, where:

the first method is as follows: the target image is screened out by screening step by step in the image library according to the arrangement order of the multiple dimensions, that is, the step S201 can be realized by the following steps S211 and S211 (not shown in the figure):

step S211, arranging the dimension levels of the at least two dimensions according to the number of feature points represented by the dimensions, to obtain a first arrangement order.

Determining the level of the dimension according to the number of characteristic points represented by the dimension, and further arranging at least two dimensions according to the level of the at least two dimensions; wherein, the larger the number of feature points represented by a dimension is, the higher the level of the dimension is. In this way, at least two levels of dimensions are arranged according to the number of the characteristic points represented by the dimensions; for example, at least two dimensions may be arranged from small to large, and the dimension arranged at the first position in the obtained arrangement order, that is, the dimension with the lowest level, even if the number of feature points is the smallest, may be obtained.

Step S212, performing a step-by-step screening from a low-level dimension to a high-level dimension according to the first arrangement order, and determining the screening path as the screening path plan.

And step-by-step screening images matched with the image feature points of the dimension screened at this time in a preset image library according to a first arrangement sequence of the dimension levels of at least two dimensions, so as to realize one-step screening.

In some possible implementation manners, based on the arrangement sequence, according to the dimension levels from low to high, candidate image sets with higher image feature point similarity corresponding to the dimension searched at this time are searched in the preset image library in sequence. For example, at least two dimensions include: searching a candidate image set with high matching degree with the low-dimensional image feature points in a preset image library according to the sequence of the low-dimensional image, the medium-dimensional image and the high-dimensional image; continuously searching a candidate image set with higher matching degree with the medium-dimensional image feature points on the basis; and continuing to search a candidate image set with higher matching degree with the high-dimensional image feature points on the basis until any level dimension of a preset condition for representing the stopping of screening is searched. In this way, the multiple levels of dimensions are arranged according to the number of feature points represented by the dimensions, so that the level of each level of dimension can be indicated through the arrangement sequence; after the arrangement sequence is obtained, the preset image library is screened from a low-level dimension to a high-level dimension step by step, and the target image can be screened more accurately by screening the preset image library according to the screening path plan.

In the above steps S211 and S212, the preset image library is screened step by step from the low-level dimension until the dimension satisfying the preset condition representing the stop of screening is screened. Therefore, the step-by-step screening from the low-level dimensionality to the high-level dimensionality is used as the screening path planning, the preset image library is conveniently screened step by step according to the screening path planning in the follow-up process, and the target image matched with the image to be detected can be screened more accurately.

The second method comprises the following steps: by the screening rate and the total screening time of each level of dimensionality, the dimensionality with the larger screening rate or the larger total screening time of the multiple levels of dimensionality is filtered, so that a screening path plan is designed according to the filtered multiple levels of dimensionality, which can be realized by the following steps S213 to S218 (not shown in the figure):

step S213, obtaining the screening rate of each level of dimensionality, and obtaining the total time consumed by screening of each level of dimensionality.

In some embodiments, the greater the number of feature points represented by a dimension, the higher the level of the dimension. The screening rate of each level of dimension represents the proportion of the number of the screened image frames of the level of dimension relative to the number of the screened image frames. The total screening time of each level of dimension represents the time for the level of dimension to perform one screening in the preset image library.

Step S214, based on the screening rate of each level of dimensionality, determining at least two continuous dimensionalities with the difference value between the screening rates smaller than a preset difference value.

In some embodiments, after determining the screening rate for any level of dimension, the difference between the screening rate for any level of dimension and the screening rates for the successive dimensions that have been screened is determined. The preset difference may be set according to an experience value of a person skilled in the art, and the preset difference may be set to a smaller value (for example, the preset difference is set to a value smaller than 0.1 and equal to or larger than 0). In some possible implementations, the preset difference may be set to 0, and for any level dimension, a continuous multi-level dimension with the same screening rate as that of the any level dimension is determined. The successive multi-level dimensions may be a lower level dimension adjacent to the any level dimension, a higher level dimension, or a lower level dimension adjacent to the lower level dimension. In a specific example, the at least two dimensions of the image to be detected include 5 dimensions, and for any one dimension of the current screening, for example, the 4 th dimension, it is determined whether a difference between a screening rate of the 3 rd dimension and a screening rate of the 4 th dimension is smaller than a preset difference, and if the difference between the screening rate of the 3 rd dimension and the screening rate of the 4 th dimension is larger than the preset difference, the 5 th dimension is continuously screened.

Step S215, determining a target-level dimension with the minimum total screening time consumption in the at least two continuous dimensions.

In some embodiments, after at least two consecutive dimensions with a screening difference value smaller than a preset difference value are determined, the total screening time of each of the at least two consecutive dimensions is obtained. And comparing the total screening time consumption of each level of dimension to determine the target level dimension with the minimum total screening time consumption.

Step S216, in the at least two dimensions, removing dimensions except the target dimension in the at least two continuous dimensions to obtain remaining dimensions.

In some embodiments, the dimension other than the target dimension of the at least two consecutive dimensions is a dimension in which a difference between the screening rate and the screening rate of the target dimension is smaller than a preset difference, and the total time consumed for screening is larger than the target dimension. And removing the dimensions in at least two dimensions of the image to be detected to obtain the remaining dimensions. In a specific example, if at least two dimensions of the image to be detected include 5 dimensions, wherein at least two successive dimensions are a 3 rd dimension and a 4 th dimension, and wherein the target dimension is the 3 rd dimension, the 4 th dimension is deleted from the 5 dimensions, and thus the remaining dimensions include a 1 st dimension, a 2 nd dimension, a 3 rd dimension and a 5 th dimension.

Step S217, rank the dimension levels in the remaining dimensions to obtain a second ranking order.

In some embodiments, the remaining dimensions may be arranged from a lower-level dimension to a higher-level dimension, resulting in a second arrangement order; the second arrangement order may also be obtained by arranging from a high level dimension to a low level dimension.

And step S218, screening from a low-level dimension to a high-level dimension according to the second arrangement sequence, and determining the screened path as the screened path plan.

In some embodiments, the image to be detected is obtained by eliminating the dimension out of the target-level dimension in at least two consecutive dimensions of the at least two levels; and screening from the low-level dimensionality to the high-level dimensionality in the preset image library according to the second arrangement sequence of the residual dimensionality, so that the screening process of the dimensionality with larger total screening time consumption is reduced in the screening path planning, and the screening path planning is optimized.

In the above steps S213 to S218, by obtaining the screening rate and the total screening time of each level of dimension, first, at least two consecutive levels of dimensions in which the difference between the screening rates is smaller than the preset difference are determined; therefore, at least two continuous dimensionalities with the screening rates being not much different can be selected from the multiple dimensionalities of the image to be detected, namely the dimensionalities with the screening rates being close although the dimensionalities are different in level. Then, the stage dimension with the minimum total time consumption is screened from at least two continuous stages of dimensions to serve as a target stage dimension, and dimensions except the target stage dimension in at least two continuous stages of dimensions of the image to be detected are removed to obtain the remaining dimensions; in this way, in at least two dimensions, the dimension with the screening rate which is very different from the target dimension but the total screening time consumption which is larger than that of the target dimension is removed, so that the dimension with the larger total screening time consumption in the multiple dimensions of the image to be detected is reduced; therefore, unnecessary screening of the dimension with larger total screening time consumption can be reduced in the screening path planning, the screening path planning can be optimized, and the efficiency of screening the residual dimension of the image to be detected is improved.

In other embodiments, the dimension with larger total time consumption can be eliminated and screened by judging whether the number of the candidate image frames corresponding to two continuous dimensions is the same or similar. If the number of frames of the candidate image sets of at least two continuous dimensions is the same or similar, which indicates that the precision of detecting the images to be detected in the preset image library reaches the highest, the total screening time consumption of any one dimension can be further considered, and the dimension with the minimum total screening time consumption is determined from the continuous multi-level dimensions; therefore, the minimum dimension in total time consumption of screening is taken as the target dimension for the multistage dimensions with the same number of frames of the candidate image set, the calculated amount of the whole screening process of the image to be detected can be reduced, and therefore the minimum dimension in total time consumption of screening is determined as the target dimension for the multistage dimensions with the same number of frames of the candidate image set, the calculated amount of circular screening can be reduced, and the accuracy of the screened target image can be higher.

And S202, screening the target image matched with the image to be detected in a preset image library based on the screening path plan.

In some embodiments, the preset image library is screened for multiple times according to the screening mode indicated in the screening path plan, so as to obtain a target image matched with the image to be detected. The screening path planning is different, and the process of screening the preset image library is different. In the first mode, if the screening route is planned to be screened from a low-level dimension to a high-level dimension step by step according to the first arrangement sequence, then screening from the low-level dimension to the high-level dimension step by step in the preset image library, in the screening process, taking a candidate image set with the low-level dimension as a screened image with the high-level dimension, and taking the preset image library as a screened image with the lowest-level dimension.

In some embodiments, the preset number threshold may be set based on the currently processed task, e.g., the currently processed task is building identification, and then the preset number threshold may be set slightly larger (e.g., set to 5); if the task currently processed is pedestrian recognition, the preset number threshold is set to 2 in order to make the recognition result high. If the number of frames of the candidate image set is smaller than or equal to the preset number threshold, the number of frames of the candidate image set meets the preset number threshold, and the feature points based on the dimensionality can search images with high matching degree, so that fine screening in a preset image library is realized. And if the current level dimensionality reaches the highest dimensionality, the preset image library is subjected to multi-level screening, so that the screening workload is not too large, the screening is stopped, and a candidate image set screened from the image characteristic points under the highest dimensionality is used as a target image. Therefore, the image feature points with the multi-level dimensionality are screened step by step in the preset image library, the data amount consumed in the screening process can be reduced, and the matching degree of the screened target image and the image to be detected is higher.

In some embodiments, for any level of dimension, the screening rate of the level of dimension is determined by counting the number of image frames screened by the level of dimension and the number of image frames screened by the lower level of dimension, that is, "obtaining the screening rate of each level of dimension" in the above step S213 may be implemented by:

the method comprises the steps of firstly, obtaining a first frame number of a candidate image set corresponding to any level of dimensionality and a second frame number of a candidate image set corresponding to a lower level of dimensionality of any level of dimensionality.

Here, a candidate image set corresponding to a lower one-level dimension of any one-level dimension is used as a to-be-screened image, and screening is performed on the to-be-screened image according to an image feature point of any one-level dimension, so that a candidate image set corresponding to any one-level dimension is obtained. And counting the number of frames in the candidate image set corresponding to any one-level dimensionality to obtain a first frame number. The second frame number is the frame number of the screened image. For the lowest-level dimension, the screened image is a preset image library, the second frame number is the frame number of the image in the preset image library, and the first frame number is the frame number of a candidate image set obtained by screening the image feature point of the lowest-level dimension in the preset image library.

In some embodiments, any level of dimension is a dimension of this filtering, for example, the dimension of this filtering is a dimension of 49 keypoints, and then any level of dimension is a dimension representing the number of keypoints is 49. The lower dimension of any level dimension represents a dimension adjacent to and one level lower than the level of any level dimension, and it can be further understood that the number of feature points adjacent to and characterized by the level of any level dimension is smaller than the number of feature points characterized by any level dimension. And searching the image characteristic points of the lower-level dimensionality in the candidate image set corresponding to the lower-level dimensionality of the lower-level dimensionality to obtain the candidate image set of the lower-level dimensionality, and counting the frame number of the candidate image set.

And secondly, determining the screening rate of any level of dimensionality based on the second frame number and the first frame number.

In some embodiments, the second step can be implemented in two ways:

the first method is as follows: and before the image to be detected is screened step by step, carrying out system preheating according to the feature points represented by the dimension of the step so as to obtain the screening rate of each dimension.

Searching in a preset image library according to the characteristic points represented by the level dimension so as to count the number of image frames which can be screened out by the characteristic points of the level dimension in the preset image library; and dividing the frame number by the total frame number in a preset image library to obtain the screening rate of the dimensionality. Or determining the number of image frames screened by the level dimension and the number of the screened image frames in the process of screening in a preset image library according to the screening path plan, thereby updating the screening rate of the level dimension in real time.

In some possible implementation manners, based on the image feature point of any level of dimension, in the candidate image set corresponding to the lower level dimension of any level of dimension, the candidate image set with higher matching degree with the image feature point of any level of dimension is searched, and the number of frames of the searched candidate image sets is counted. For the current level dimension, the second frame number is the frame number of the screened image of any level dimension, and the first frame number is the frame number of the image screened from the screened image. Therefore, the screening rate of any level of dimensionality can be obtained by dividing the first frame number by the second frame number. Therefore, the screening rate of any dimension can be estimated according to the image feature points of the dimension in any level through the preset image library, and the estimated screening rate can be updated in real time in the screening process so as to provide more accurate screening rate of any dimension.

The second method comprises the following steps: and updating the preset screening rate of the system in real time in the process of screening the image to be detected step by step.

Here, based on the image feature points of the dimension, a candidate image set having a high degree of matching with the image feature points of any one level dimension is searched for among candidate image sets corresponding to a lower level dimension of the dimension, and the number of frames of the searched candidate image sets is counted. For any dimension, the first frame number is the frame number of the screened image of any dimension, and the second frame number is the frame number of the image screened from the screened image. Thus, the screening rate of any level dimension can be obtained by dividing the first frame number by the second frame number.

In the embodiment of the application, for any one of multiple levels of dimensionality, the number of frames of the candidate image set of each level of dimensionality can be rapidly counted by counting the number of frames of the candidate image set of any one level of dimensionality and taking the number of frames as the number of frames of the screened image; therefore, the screening rate of the first-level dimension can be accurately counted according to the frame number of the candidate image set of the first-level dimension and the frame number of the candidate image set of the lower-level dimension.

In some possible implementations, for any level of dimension, the image frame number filtered by any level of dimension and the filtering frequency of the level of dimension are counted to determine the filtering rate of the level of dimension, that is, "obtaining the filtering rate of each level of dimension" in step S213 above may be implemented by:

firstly, acquiring a single-frame time-consuming set and screening times of any dimension.

In some embodiments, the single frame elapsed time set comprises: the any level dimension and a lower level dimension of the any level dimension are time consuming at each single frame of the screening. The single-frame time consumption can be understood as the time consumption required for screening a frame of candidate image in the process of dimension screening at any level. In some possible implementation manners, the time consumption of a screening process of any level of dimensionality is obtained, and the time consumption of the screening process is divided by the number of frames of the screened image set, so that the single-frame time consumption of the screening process can be obtained. And summing each single-frame time consumption in the single-frame time consumption set to obtain a summation result, namely the sum of the time consumptions screened to the candidate image set by any level of dimensionality. The average time consumption of any dimension in the screening process can be obtained through the summation result and the screening times.

And secondly, determining the total screening time of any level of dimensionality based on the single-frame time consumption set and the screening times of any level of dimensionality.

In some embodiments, the total time spent screening in any level of dimension may be determined in two ways, wherein:

in the first mode, before screening at least two dimensions of an image to be detected, the total screening time of any dimension is estimated in a system preheating stage by estimating a single-frame time consumption set and screening times of any dimension.

Performing system preheating according to the feature points represented by the level dimension to obtain the total screening time of each level of dimension; searching in a preset image library according to the characteristic points represented by the level dimension, and counting the total screening time of the level dimension. The total screening time can also be updated in real time by determining the time consumption required for image screening of the dimension in the screening process. Therefore, the total screening time of any one-level dimensionality can be accurately and conveniently determined through the single-frame time consuming set of any one-level dimensionality and the screening times corresponding to any one-level dimensionality.

And in the second mode, updating the total screening time estimated at the preset stage of the system in real time in the process of screening at least two dimensions of the image to be detected through the single-frame time consumption set and the screening times of any dimension.

In the first step, the average time consumption of any level of dimensionality in the screening process is obtained through a single-frame time consumption set and the screening times. And combining the screening rate and the average time consumption of any level of dimensionality to determine the total time consumption of any level of dimensionality in the screening. Meanwhile, the screening rate and the screening time consumption of the low-level dimensionality of any level of dimensionality can be combined, and the total screening time consumption of the screened low-level dimensionality in the screening process is counted; therefore, the total screening time consumption corresponding to all screened dimensions is summed, and the total time consumption of multiple screening from the first screening to the current screening for the image to be detected can be obtained.

In some possible implementation manners, the number of image frames to be screened corresponding to any one level of dimensionality is obtained first, and the average time consumption and the screening rate of any one level of dimensionality are multiplied by the number of image frames to be screened, so that the total screening time consumption of any one level of dimensionality can be obtained. First, the average time consumption of any level of dimensionality can be obtained by combining the screening times with the screening time consumption of the low-level dimensionality of any level of dimensionality. In the implementation process, the time consumption of the 1 st screening is combined with the frame number of the screened image data according to the first screening, so that the time consumption of the dimension of the 1 st screening can be obtained; aiming at the dimension of the 2 nd screening, combining the time consumption of the previous two (namely, the 1 st screening and the 2 nd screening) and the frame number of the screened image data according to the 2 nd screening to obtain the time consumption of the dimension of the 2 nd screening; and averaging the time consumption of the dimension screened at the 1 st time and the time consumption of the dimension screened at the 2 nd time to obtain the average time consumption of the dimension screened at the 2 nd time. And by analogy, the average time consumption of each screening dimension can be obtained.

Then, based on the screening rate of any level of dimension and the average time consumption of any level of dimension, determining the total time consumption of screening of any level of dimension. For example, the total screening time of any level of dimension can be obtained by multiplying the screening rate of any level of dimension and the average screening time of any level of dimension.

In one particular example, if at least two dimensions are included: high, medium and low dimensionality, the current level dimensionality is the medium dimensionality, and then the screened dimensionality is the low dimensionality. And in the candidate image set corresponding to the low dimensionality, after searching the feature points of the middle dimensionality, obtaining the candidate image set corresponding to the middle dimensionality. The screening of the medium dimension is the 2 nd screening in the whole screening process, namely the screening frequency is 2. And multiplying the screening rate of the medium dimensionality, the screening frequency 2 and the average time consumption to obtain the total screening time consumption of the medium dimensionality.

In the embodiment of the application, for any level of dimensionality, the average time consumption of any level of dimensionality can be determined by acquiring a single-frame time consumption set and the screening times of any level of dimensionality; then, the average time consumption of any level dimension is combined with the number of the image frames to be screened of any level dimension to count the total time consumption of screening in one screening of any level dimension.

In some embodiments, according to the screening path planning, the image feature points based on multiple dimensions are screened in the image library to screen out the target image, that is, the step S202 may be implemented by the steps shown in fig. 3:

step S301, in the preset image library, screening a candidate image set matched with the image feature points of the image to be detected from a low-level dimension to a high-level dimension according to the arrangement sequence in the screening path plan.

In some embodiments, according to the arrangement sequence of the dimension levels in the screening path gauge, images matched with the image feature points of the dimension screened this time are screened step by step in a preset image library, and a dimension candidate image set screened this time is obtained. In this way, the candidate image set matched with the image feature points of each level of dimensionality can be screened for each level of dimensionality from the low level dimensionality to the high level dimensionality according to the arrangement sequence in the screening path planning.

In some possible implementation manners, based on the arrangement sequence, according to the dimension levels from low to high, candidate image sets with higher image feature point similarity corresponding to the dimension searched at this time are searched in the preset image library in sequence. For example, at least two dimensions include: and then, according to the sequence of the low dimension, the middle dimension and the high dimension, searching a candidate image set with higher matching degree with the image feature point of the low dimension in a preset image library, continuously searching a candidate image set with higher matching degree with the image feature point of the middle dimension on the basis, and continuously searching a candidate image set with higher matching degree with the image feature point of the high dimension on the basis.

Step S302, based on the candidate image set corresponding to any level of dimensionality and the dimensionality level of any level of dimensionality, determining whether any level of dimensionality meets a preset condition for representing screening stop.

In some embodiments, after determining an image feature point of the lowest-level dimension of an image to be detected, screening a candidate image set with similarity greater than a certain threshold value with the image feature point of the lowest-level dimension in a preset image library; here, the similarity threshold may be set to a slightly smaller value, for example, to 0.9; therefore, the primary screening of the image to be detected is realized in the preset image library through the image feature points of the lowest-level dimensionality, and the candidate image set corresponding to the lowest-level dimensionality is obtained. For example, if the number of image feature points represented by the lowest-level dimension is 21, the preliminary screening may be implemented in a preset image library based on the 21 feature points. If the image feature points of the low-level dimensionality are not screened to a proper number of candidate image sets, searching of the high-level dimensionality is conducted on the basis.

In some possible implementations, it is assumed that the preset condition is that a first frame number of the candidate image set corresponding to any level of dimension is less than or equal to a preset number threshold. And determining the frame number of the screened candidate image set after primary screening is carried out in a preset image library according to the image feature point of the lowest-level dimension. If the frame number is less than or equal to the preset number threshold, it indicates that the requirement of image matching is met by primary screening based on the image feature points of the lowest-level dimensionality, and the screened candidate image set can be directly used as a target image. If the number of frames is less than or equal to the preset number threshold, it indicates that the preliminary screening based on the image feature points of the lowest-level dimensionality does not meet the requirement of image matching, then on the basis of the screened candidate image set, searching is carried out according to the image feature points of the higher-level dimensionality of the lowest-level dimensionality, and the candidate image set of the higher-level dimensionality is obtained. In this way, after the candidate image set with the higher dimensionality is obtained, whether the number of frames of the candidate image set corresponding to the dimensionality is smaller than or equal to a preset number threshold value is further judged, so as to determine whether the candidate image set can be used as a target image.

In one particular example, the at least two dimensions include: taking high dimension, medium dimension and low dimension as examples, firstly, primarily screening in a preset image library according to the image feature points of the low dimension, and determining the frame number of a screened candidate image set; then, if the frame number is larger than a preset number threshold, further fine screening is carried out in a preset image library according to the image feature points of the medium dimensionality, and the frame number of the screened candidate image set is determined; and finally, if the frame number is greater than a preset number threshold, performing the final step of fine screening in a preset image library according to the image feature points with high dimensionality, and taking the candidate image set screened this time as a target image no matter whether the frame number of the screened candidate image set is less than or equal to the preset number threshold. So, can effectually carry out effectual preliminary screening to the image storehouse that the data volume is big based on multiple characteristic dimension, carry out the fine screen according to the preliminary screening result, both improved screening efficiency, still improved the accuracy of screening.

In some embodiments, the preset conditions include at least one of the following conditions:

the first condition is as follows: and the first frame number of the candidate image set corresponding to any level of dimensionality is less than or equal to a preset number threshold.

And a second condition: and comparing the total screening rate of the candidate image set corresponding to any level of dimensionality with the preset image library, and judging whether the total screening rate of the candidate image set corresponding to any level of dimensionality is smaller than or equal to a preset screening rate threshold value.

Here, in the first condition and the second condition, the preset number threshold and the preset screening rate threshold may be set based on the currently processed task, for example, if the currently processed task is building identification, the preset number threshold and the preset screening rate threshold may be set to be slightly larger (for example, the preset number threshold is set to be 5, and the preset screening rate threshold is set to be 0.7); if the task currently processed is pedestrian recognition, in order to make the recognition result higher, the preset number threshold and the preset screening rate threshold are both set to be smaller values, for example, the preset screening rate threshold is set to be 0.5, and the preset number threshold is set to be 2. If the number of frames of the candidate image set is smaller than or equal to the preset number threshold, the number of frames of the candidate image set meets the preset number threshold, and the feature points based on the dimensionality can search images with high matching degree, so that fine screening in a preset image library is realized.

And (3) carrying out a third condition: the dimension level of any dimension is the highest dimension.

Here, if any one of the dimensions reaches the highest dimension of at least two dimensions in the image to be detected, it is indicated that the preset image library has been subjected to multi-level screening, so that the screening workload is not too large, the screening is stopped, and a candidate image set obtained by screening the image feature points under the highest dimension is taken as a target image. Therefore, the image feature points with the multi-level dimensionality are screened step by step in the preset image library, the data amount consumed in the screening process can be reduced, and the matching degree of the screened target image and the image to be detected is higher.

And a fourth condition: the total time for screening in any one dimension is the smallest in at least two consecutive dimensions.

Here, a difference between the screening rates of the any one-level dimension and the at least two consecutive-level dimensions is smaller than a preset difference, and the at least two consecutive-level dimension and the any one-level dimension include a highest-level dimension. In the process of screening at least two dimensions of an image to be detected in a preset image library, counting the screening rate of the screening and the difference value between the screening rate of the screening and the screening rate of the dimension lower by one level after each screening; the difference between the screening rates of at least two consecutive dimensions is the difference between every two consecutive dimensions. If the difference value between the screening rates of at least two continuous dimensions is smaller than the preset difference value and the dimension of the highest level is included in the at least two continuous dimensions, the whole screening process of the image to be detected is completed. Therefore, if the difference value between the screening rates of at least two continuous dimensions is smaller than the preset difference value, which indicates that the screening rates of the dimensions are not much different, the screening effect of the dimensions is judged according to the total screening time of another index. Therefore, in at least two continuous dimensions with little difference in screening rate, the dimension with the minimum total screening time consumption is used as the dimension for stopping screening, and a more accurate candidate image set can be screened for the image to be detected. Therefore, the screening time consumption and the screening rate are fully considered in the preset condition for representing the screening stop, so that the screening time consumption and the screening rate are balanced, and the calculated amount can be reasonably reduced.

In the above conditions one to four, no matter which condition of the conditions one to four is satisfied by any one of the screened dimension levels, the screening is stopped, and the dimension level is taken as the final dimension, so that not only can the number of unnecessary screening be reduced, but also the accuracy of the screened candidate image set can be improved.

Step S303, in response to that any one of the dimensions satisfies the preset condition, based on the image feature point of any one of the dimensions, screening the target image in a candidate image set corresponding to a lower one of the dimensions.

In some embodiments, after determining that any one-level dimension satisfies the preset condition (for example, determining that the total filtering time is the smallest in at least two consecutive levels), the image feature point of any one-level dimension is obtained. On the basis of the candidate image set of the lower one-level dimension of the any one-level dimension, the image feature points represented by the any one-level dimension are adopted, and searching is performed in the candidate image set corresponding to the lower one-level dimension, so that a target image with higher similarity to the image feature points represented by the any one-level dimension is searched. In this way, the candidate image set of the lower one-level dimension of the any one-level dimension is used as the image to be screened, and the image to be screened is searched according to the image feature point represented by the any one-level dimension to obtain the candidate image set of the any one-level dimension, so as to rank the target image.

Through the steps S301 to S303, a candidate image set matched with the image feature points of the image to be detected is screened from a preset image library from the lowest-level dimension according to the arrangement sequence in the screening path plan; therefore, the candidate image set corresponding to each level of dimensionality for each screening can be obtained, and whether the screening is stopped at the level of dimensionality is judged conveniently according to the screening rate and the dimensionality level of the candidate image set of each level of dimensionality. For example, if the total time spent in screening in any one dimension is the smallest in at least two consecutive dimensions, then the screening is stopped in any dimension; and the candidate image set corresponding to the lower one-level dimensionality of any one-level dimensionality is used as the screened image, and the target image with higher similarity to the image to be detected can be screened in the screened image according to the image feature point of any one-level dimensionality, so that the accuracy of the screened target image is improved.

In some embodiments, the step-by-step screening is performed in the image library according to the dimension level of the M-level dimension based on the image feature points of the M-level dimension until the dimension meeting the preset condition is screened, that is, "in the preset image library, the candidate image set matched with the image feature points of the image to be detected is screened from the low-level dimension to the high-level dimension according to the arrangement sequence in the screening path plan" may be implemented by the following processes:

and screening a candidate image set matched with the image feature point of the jth level dimension from the candidate image set corresponding to the jth-1 level dimension according to the arrangement sequence in the screening path plan from the 1 st level dimension until the jth dimension meets the preset condition.

Here, the 1 st level dimension represents the lowest level dimension in the M level dimensions, and when j is 1, the candidate image set corresponding to the j-1 st level dimension is the preset image library. In this way, for the 1 st level dimension (i.e. the lowest level dimension), the preset image library is used as the screened image; in a preset image library, searching an image with higher similarity with the characteristic point according to the image characteristic point of the image to be detected corresponding to the 1 st-level dimension, wherein the searched image is a candidate image set of the 1 st-level dimension.

For the 2 nd and above 2 nd dimension, the screened image is a candidate image set corresponding to the lower dimension of the dimension. Taking the 3 rd-level dimension as an example, the screened image is a candidate image set corresponding to the 2 nd-level dimension, so that in the candidate image set corresponding to the 2 nd-level dimension, according to the image feature point of the image to be detected corresponding to the 3 rd-level dimension, an image with higher similarity to the feature point is searched, and the searched image is the candidate image set of the 3 rd-level dimension. And searching from a low-level dimension to a high-level dimension according to the arrangement sequence in the screening path planning until the dimension meeting the preset condition is searched, and stopping searching.

In the embodiment of the application, under the condition that the image to be detected comprises M-level dimensionality, starting from the lowest-level dimensionality, taking a candidate image set corresponding to the lower-level dimensionality as a screened image, and screening matched candidate images in the screened image according to image feature points of the dimension screened this time; therefore, the candidate image set with the higher one-level dimensionality is screened from the candidate image set corresponding to the lower one-level dimensionality, and the screening is stopped until the dimensionality meeting the preset condition is screened, so that the data volume consumed in the screening process can be reduced, the multi-level screening can be realized, the screening result is more accurate, and the screening efficiency can be improved.

An exemplary application of the embodiment of the present application in an actual application scenario will be described below, taking an image to be detected as a face image, and performing a multi-level face feature search on the face image as an example.

The embodiment of the application provides an image detection method, which comprises the steps of firstly, inputting a picture to be detected into a deep neural network or a human face characteristic point extraction model to obtain human face characteristics. Secondly, performing dimension reduction processing on the human face features or adopting a mature human face feature point extraction method with lower dimensionality, and searching the features which are already put in storage from low dimensionality to high dimensionality to obtain an average searching and screening ratio and performance indexes (such as time consumption); and finally, making a search strategy plan according to the average search screening ratio and the performance index. Therefore, through dynamic planning, the optimal strategy of multi-cascade search can be obtained, the problem of balance of single search dimension is effectively solved, and the parallel search is expanded in terms of architecture.

Fig. 4 is a schematic view of an implementation flow of an image detection method provided in an embodiment of the present application, and the following description is performed in conjunction with the steps shown in fig. 4:

step S401, acquiring a face image.

In some embodiments, the acquired facial image is from a user-provided picture or a live-shot video file. According to different input file types, the data input interface file types need to be unified before subsequent operations are carried out. Performing image preprocessing operation on the picture type file, and waiting for subsequent input into a network; for a video type file, image preprocessing operation needs to be performed on a picture obtained by frame selection through frame selection operation, and the picture is waited to be input into a network subsequently. And the frame selection operation is used for selecting at least one image from the partial video as an image to be detected, requiring the selected image to have higher quality, and finally detecting whether the at least one image to be detected has fake information. Wherein, the image quality can be measured by any one or more of the following criteria: whether the certificate is positioned in the center of the image, whether the edge of the certificate is completely included in the image, the proportion of the image area occupied by the surface of the certificate, the image definition, the exposure and the like, and the distance between the human face and the eyes is at least 30 pixels.

In some possible implementations, the face image may be obtained by the steps shown in fig. 5, where:

in step S501, an original image file is acquired.

Here, the original image file may be an image or video file

Step S502, it is determined whether the original image file is an image type file.

Here, if the original image file is an image type file, the process proceeds to step S504, and if the original image file is not an image type file, the process proceeds to step S503.

Step S503, at least one frame of image is selected from the video file as the image to be detected.

In step S504, an image file of a picture type is acquired.

The face image is acquired through the above steps S501 to S504.

In step S402, face feature points of different sizes and dimensions are extracted from the face image.

In some embodiments, the input is an acquired image of a human face. And outputting the human face characteristic points with different size dimensions.

In some possible implementation manners, after the face image is obtained, extracting face characteristic points with different size dimensions for the face image to be used for hierarchical searching; the facial features of the face image can be extracted according to the 21-point features, 49-point features, 68-point features and 81-point features of the face feature dimension. As shown in fig. 6, fig. 6 is a schematic flow chart of another implementation of the image detection method provided in the embodiment of the present application, and the following description is performed with reference to the steps shown in fig. 6:

step S601, an original image or video is acquired.

Step S602, image preprocessing is performed on the original image or video.

Step S603, obtaining at least one frame of image to be detected based on the preprocessed image.

Step S604, respectively extracting feature points with different dimensions in at least one frame of image to be detected.

Here, the feature points of different size dimensions include: 21-point feature, 49-point feature, 68-point feature, 81-point feature, and so on.

Step S605, feature points with different dimensions are obtained.

Feature points of different dimensions can be extracted through the above steps S601 to S605, and as shown in fig. 7, feature points of different dimensions are extracted for a face image 701, and feature points 702 of 68 key points, feature points 703 of 49 key points, and feature points 704 of 21 key points are obtained respectively.

In some possible implementation manners, the high-dimensional features may be extracted at one time, and the high-dimensional features may be subjected to dimension reduction processing to perform dimension reduction extraction to different degrees, (for example, by using some feature sampling methods). As shown in fig. 8, fig. 8 is a schematic flow chart of still another implementation of the image detection method provided in the embodiment of the present application, and the following description is performed with reference to the steps shown in fig. 8:

in step S801, an original image or video is acquired.

Step S802, image preprocessing is performed on the original image or video.

And step S803, obtaining at least one frame of image to be detected based on the preprocessed image.

Step S804, extracting high-dimensional feature points from at least one frame of image to be detected, and sampling the high-dimensional feature points to obtain low-dimensional feature points.

Step S805, feature points with different size dimensions are obtained.

Feature points with different dimensions can be extracted through the steps S801 to S805, and as shown in fig. 9, feature points with high dimensions are extracted from a face image 901 to obtain high-dimensional features 902; sampling the high-dimensional feature 902 to obtain a medium-dimensional feature 903; the mid-dimensional features 903 are sampled to obtain low-dimensional features 904.

Step S403, searching features in the image library from the low dimension to the high dimension to obtain an average search screening ratio and a performance index.

In some embodiments, before performing hierarchical search, searching is performed on features of different dimensions, and the screening rates and average time consumption of the features of different dimensions are counted (default screening similarity is greater than or equal to 90%, which may be actually changed), where the screening rates are as shown in formula (1):

wherein p is _f Representing the screening rate with corresponding extracted feature dimension f, N is the screened sample (such as an image library), N is _s The selected samples are indicated.

Average time consumption t of different dimension characteristics of face image after m times of sampling _avg As shown in equation (2):

wherein N is _m Representing the data set size, T, in m sampled data sets _nm Indicating that m screens were time consuming.

Different dimension characteristics of face imageIn the process of hierarchical searching, searching characteristics are searched from small-dimension characteristics and are increased step by step until the proper number of result sets is reached or the maximum dimension characteristic is reached, and the total time t of hierarchical searching is up to _total As shown in equation (3):

total time t consumed for pair level search _total Performing optimal planning to minimize the total time for screening in each stage search stage, so as to make t _total Satisfies the following formula (4):

wherein p is _f0 ＝1。

In the embodiment of the application, the obtained face image is subjected to mixed extraction of multiple feature dimensions, and high-dimensional features are respectively indexed; like this, multiple characteristic dimension can be effectual carry out effectual preliminary screening to the data set that the data volume is big, carries out the fine screen according to preliminary screening result, has both guaranteed efficiency and has guaranteed the precision again.

An embodiment of the present application provides an image detection apparatus, fig. 10 is a schematic structural component diagram of the image detection apparatus in the embodiment of the present application, and as shown in fig. 10, the image detection apparatus 1000 includes:

a first obtaining module 1001 configured to obtain an image to be detected;

a first determining module 1002, configured to determine image feature points of at least two dimensions of the image to be detected;

the first screening module 1003 is configured to screen, based on the image feature points of the at least two dimensions, a target image matched with the image to be detected in a preset image library.

In some embodiments, the first determining module 1002 includes:

the first determining submodule is used for determining at least two dimensions representing different feature point quantities based on a scene where the image to be detected is located;

and the first extraction submodule is used for respectively extracting the characteristic points of the image to be detected on the at least two dimensions to obtain the image characteristic points of the at least two dimensions.

In some embodiments, the at least two dimensions include M dimensions, where M is a natural number greater than or equal to 2, and the first determining module 1002 includes:

the second extraction submodule is used for extracting the characteristic points of the image to be detected on the j-th dimension to obtain the image characteristic points of the j-th dimension; wherein j is a natural number greater than or equal to 2 and less than or equal to M;

the first sampling submodule is used for sampling the image feature point of the j-th level dimension based on the j-1-th level dimension to obtain the image feature point of the j-1-th level dimension; wherein the level of the j-th level dimension is higher than the level of the j-1-th level dimension.

In some embodiments, the first screening module 1003 includes:

the first obtaining submodule is used for obtaining a screening path plan based on the at least two-stage dimensionality;

and the first screening submodule is used for screening the target image matched with the image to be detected in a preset image library based on the screening path plan.

In some embodiments, the first obtaining sub-module comprises:

the first arrangement unit is used for arranging the dimension levels of the at least two dimensions according to the number of the feature points represented by the dimensions to obtain a first arrangement sequence; wherein the larger the number of feature points represented by the dimension is, the higher the level of the dimension is;

and the first screening unit is used for screening from a low-level dimension to a high-level dimension step by step according to the first arrangement sequence and determining the screening path as the screening path plan.

In some embodiments, the first obtaining sub-module comprises:

the first acquisition unit is used for acquiring the screening rate of each level of dimensionality and acquiring the total screening time of each level of dimensionality;

the first determining unit is used for determining at least two continuous dimensionalities with the difference value smaller than a preset difference value on the basis of the screening rate of each level of dimensionality;

the second determining unit is used for determining a target-level dimension with the minimum screening total time consumption in the at least two continuous dimensions;

the first removing unit is used for removing dimensions except the target dimension in the at least two continuous dimensions to obtain residual dimensions;

the first arrangement unit is used for arranging the dimension levels in the remaining dimensions to obtain a second arrangement sequence;

and the first screening unit is used for screening from a low-level dimension to a high-level dimension according to the second arrangement sequence and determining the screened path as the screened path plan.

In some embodiments, the first screening submodule comprises:

the second screening unit is used for screening a candidate image set matched with the image feature points of the image to be detected from a low-level dimension to a high-level dimension in the preset image library according to the arrangement sequence in the screening path plan;

a third determining unit, configured to determine, based on the candidate image set corresponding to the any level dimension and a dimension level of the any level dimension, whether the any level dimension satisfies a preset condition characterizing screening stop;

and a second screening unit, configured to, in response to that any one of the level dimensions satisfies the preset condition, screen the target image in a candidate image set corresponding to a lower level dimension of the any level dimension based on the image feature point of the any level dimension.

In some embodiments, the preset condition comprises at least one of:

a first frame number of the candidate image set corresponding to the dimension of any level is less than or equal to a preset number threshold;

the total screening rate of the candidate image set corresponding to any level of dimensionality compared with the preset image library is smaller than or equal to a preset screening rate threshold value;

the dimensionality level of any one dimensionality is the highest dimensionality;

the total time for screening the targets of any one-level dimension is the minimum in at least two consecutive levels of dimensions, wherein the difference between the target screening rates of any one-level dimension and at least two consecutive levels of dimensions is smaller than a preset difference, and the at least two consecutive levels of dimensions and any one-level dimension comprise the dimension of the highest level.

In some embodiments, in a case that the at least two-level dimensions include M-level dimensions, the second filtering unit is further configured to:

starting from the 1 st dimension, screening a candidate image set matched with the image feature point of the j-th dimension from a candidate image set corresponding to the j-1 st dimension according to the arrangement sequence in the screening path plan until the j-th dimension meets the preset condition;

and the 1 st-level dimension represents the lowest-level dimension in the M-level dimensions, and under the condition that the value of j is 1, the candidate image set corresponding to the j-1 st-level dimension is the preset image library.

In some embodiments, the first obtaining unit includes:

the system comprises a first acquisition subunit, a second acquisition subunit and a third acquisition subunit, wherein the first acquisition subunit is used for acquiring a first frame number of a candidate image set corresponding to any level of dimensionality and a second frame number of a candidate image set corresponding to a lower level of dimensionality of the any level of dimensionality;

and the first determining subunit is used for determining the screening rate of the dimension of any level based on the second frame number and the first frame number.

In some embodiments, the first obtaining unit includes:

the second acquisition subunit is used for acquiring a single-frame time-consuming set and screening times of any level of dimensionality; wherein the single-frame elapsed time set comprises: the dimension of any level and the dimension of low level of the dimension of any level are time-consuming in each single frame of screening;

and the second determining subunit is configured to determine the total filtering time consumption of any level of dimension based on the single-frame time consumption set and the filtering times of any level of dimension.

It should be noted that the above description of the embodiment of the apparatus, similar to the above description of the embodiment of the method, has similar beneficial effects as the embodiment of the method. For technical details not disclosed in the embodiments of the apparatus of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.

It should be noted that, in the embodiment of the present application, if the image detection method is implemented in the form of a software functional module and sold or used as a standalone product, the image detection method may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application or portions thereof that contribute to the prior art may be embodied in the form of a software product, where the computer software product is stored in a storage medium and includes several instructions for enabling an electronic device (which may be a terminal, a server, or the like) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a hard disk drive, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.

Correspondingly, the embodiment of the present application further provides a computer program product, where the computer program product includes computer-executable instructions, and after the computer-executable instructions are executed, the steps in the image detection method provided by the embodiment of the present application can be implemented. Accordingly, an embodiment of the present application further provides a computer storage medium, where computer-executable instructions are stored on the computer storage medium, and when executed by a processor, the computer-executable instructions implement the steps of the image detection method provided in the foregoing embodiment.

Accordingly, an electronic device is provided in an embodiment of the present application, fig. 11 is a schematic structural diagram of the electronic device in the embodiment of the present application, and as shown in fig. 11, the electronic device 1100 includes: a processor 1101, at least one communication bus, a communication interface 1102, at least one external communication interface, and memory 1103. Wherein the communication interface 1102 is configured to enable connected communication between these components. The communication interface 1102 may include a display screen, and the external communication interface may include a standard wired interface and a wireless interface. Wherein the processor 1101 is configured to execute the image processing program in the memory to implement the steps of the image detection method provided by the above-mentioned embodiments. The above descriptions of the embodiments of the image detection apparatus, the electronic device and the storage medium are similar to the above descriptions of the embodiments of the method, have similar technical descriptions and advantages to the corresponding embodiments of the method, and are limited by the space. For technical details not disclosed in the embodiments of the image detection apparatus, the electronic device and the storage medium of the present application, please refer to the description of the embodiments of the method of the present application for understanding.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments. It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only one logical function division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or in other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit may be implemented in the form of hardware, or in the form of hardware plus a software functional unit. Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof that contribute to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling an electronic device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code. The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image detection method, characterized in that the method comprises:

acquiring an image to be detected;

2. The method according to claim 1, wherein the determining of image feature points of at least two dimensions of the image to be detected comprises:

determining at least two-stage dimensionality representing different feature point quantities based on the scene where the image to be detected is located;

and respectively extracting the characteristic points of the image to be detected on the at least two dimensions to obtain the image characteristic points of the at least two dimensions.

3. The method according to claim 1, wherein the at least two-level dimensions include M-level dimensions, M being a natural number greater than or equal to 2, and the determining image feature points of the at least two-level dimensions of the image to be detected includes:

extracting characteristic points of the image to be detected on the j-th dimension to obtain image characteristic points of the j-th dimension; wherein j is a natural number greater than or equal to 2 and less than or equal to M;

sampling the image feature points of the j-th level dimension based on the j-1-th level dimension to obtain the image feature points of the j-1-th level dimension; wherein the level of the j-th level dimension is higher than the level of the j-1-th level dimension.

4. The method according to any one of claims 1 to 3, wherein the screening the target image matched with the image to be detected in a preset image library based on the image feature points of at least two dimensions comprises:

obtaining a screening path plan based on the at least two-stage dimensionality;

and screening the target image matched with the image to be detected in a preset image library based on the screening path plan.

5. The method of claim 4, wherein obtaining a screening path plan based on the at least two dimensions comprises:

according to the number of feature points represented by the dimensionality, the dimensionality levels of the at least two dimensionalities are arranged to obtain a first arrangement sequence; wherein the larger the number of feature points represented by the dimension is, the higher the level of the dimension is;

and screening from a low-level dimension to a high-level dimension step by step according to the first arrangement sequence, and determining the screening path plan.

6. The method of claim 4, wherein obtaining a screening path plan based on the at least two dimensions comprises:

acquiring the screening rate of each level of dimensionality, and acquiring the total screening time of each level of dimensionality;

determining at least two continuous dimensions of which the difference value between the screening rates is smaller than a preset difference value on the basis of the screening rate of each level of dimension;

determining a target level dimension with minimum total screening time consumption in the at least two continuous levels of dimensions;

in the at least two dimensions, removing dimensions except the target dimension in the at least two continuous dimensions to obtain remaining dimensions;

ranking the dimensionalities in the residual dimensionalities to obtain a second ranking sequence;

and screening from the low-level dimension to the high-level dimension according to the second arrangement sequence, and determining the screened path plan.

7. The method according to any one of claims 4 to 6, wherein the screening the target image matched with the image to be detected in a preset image library based on the screening path plan comprises:

screening a candidate image set matched with the image feature points of the image to be detected from a low-level dimension to a high-level dimension in the preset image library according to the arrangement sequence in the screening path plan;

determining whether any level dimension meets a preset condition for representing screening stop or not based on the candidate image set corresponding to the any level dimension and the dimension level of the any level dimension;

and in response to the fact that any level of dimension meets the preset condition, screening the target image in a candidate image set corresponding to a lower level of dimension of any level of dimension based on the image feature points of any level of dimension.

8. The method of claim 7, wherein the preset condition comprises at least one of:

the total time for screening the targets of any one-level dimension is the minimum in at least two continuous levels of dimensions, wherein the difference between the target screening rates of any one-level dimension and at least two continuous levels of dimensions is smaller than a preset difference, and the at least two continuous levels of dimensions and any one continuous level of dimensions comprise the highest level of dimension.

9. The method according to claim 7 or 8, wherein in a case that the at least two dimensions include M dimensions, the screening, in the preset image library, a candidate image set matching with the image feature points of the image to be detected from a low-level dimension to a high-level dimension according to an arrangement order in the screening path plan includes:

10. The method of claim 6, wherein obtaining the screening rate for each level of dimension comprises:

acquiring a first frame number of a candidate image set corresponding to any level of dimensionality and a second frame number of a candidate image set corresponding to a lower level of dimensionality of any level;

and determining the screening rate of any level of dimensionality based on the second frame number and the first frame number.

11. The method of claim 6, wherein obtaining the total filtering time of each level of dimension comprises:

acquiring a single-frame time-consuming set and screening times of any one-level dimensionality; wherein the single-frame elapsed time set comprises: the dimension of any level and the dimension of low level of the dimension of any level are time-consuming in each single frame of screening;

and determining the total screening time consumption of any level of dimensionality based on the single-frame time consumption set and the screening times of any level of dimensionality.

12. An image detection apparatus, characterized in that the apparatus comprises:

the first acquisition module is used for acquiring an image to be detected;

13. A computer storage medium having computer-executable instructions stored thereon that, when executed, perform the method steps of any of claims 1 to 11.

14. An electronic device, comprising a memory having computer-executable instructions stored thereon and a processor capable of performing the method steps of any of claims 1-11 when executing the computer-executable instructions on the memory.