CN114387624A

CN114387624A - Pedestrian re-recognition method and device based on attitude guidance and storage medium

Info

Publication number: CN114387624A
Application number: CN202210055447.3A
Authority: CN
Inventors: 郑喜民; 翟尤; 舒畅; 陈又新
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2022-01-18
Filing date: 2022-01-18
Publication date: 2022-04-22
Also published as: WO2023137923A1

Abstract

The invention relates to an artificial intelligence technology, and provides a pedestrian re-identification method, equipment and a storage medium based on attitude guidance, wherein the method comprises the following steps: carrying out gesture recognition preprocessing on the target pedestrian image to generate local features of a plurality of body parts; inputting a plurality of local features into a human body attention module for training to generate an attention mask; processing the attention mask and the first three-dimensional feature to obtain a second three-dimensional feature; inputting the second three-dimensional characteristic into a second-order information attention module for processing to obtain a covariance matrix of a two-dimensional matrix; and performing attention calculation on the covariance matrix to obtain an output result of a second-order matrix. In the technical scheme of the embodiment, each body part is determined by the pedestrian body attention module, the influence of the shielding object on the identification of the pedestrian is avoided, the degree of association among all the parts is enhanced by the second-order information module, the information expression of the human body can be emphasized, and the information expression of the background and the shielding object is suppressed.

Description

Pedestrian re-recognition method and device based on attitude guidance and storage medium

Technical Field

The invention relates to the field of data analysis, in particular to a pedestrian re-identification method and device based on attitude guidance and a storage medium.

Background

At present, pedestrian re-identification (Person re-identification), also called pedestrian re-identification, is a technology for determining whether a specific pedestrian exists in an image or a video sequence by using a computer vision technology. The method is widely considered as a sub-problem of image retrieval, namely, a monitored pedestrian image is given, then the pedestrian image under the cross-equipment is retrieved, the method is mainly used for making up the visual limitation of a fixed camera, can be combined with a pedestrian detection/pedestrian tracking technology, and can be widely applied to the fields of intelligent video monitoring, intelligent security and the like.

Due to the difference between different camera devices, pedestrians have the characteristics of rigidity and flexibility, the appearance is easily influenced by wearing, size, shielding, posture, visual angle and the like, the pedestrian re-identification is mainly applied to places with dense pedestrian flow, such as airports, stations and the like, and the existing pedestrian re-identification technology is difficult to distinguish the pedestrians under the condition that the pedestrians are shielded by other people or other objects in the places.

Disclosure of Invention

The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

The embodiment of the invention mainly aims to provide a pedestrian re-identification method based on posture guidance, which can identify pedestrians in a sheltered scene and effectively avoid the influence of shelters on pedestrian identification.

In a first aspect, an embodiment of the present invention provides a pedestrian re-identification method based on posture guidance, including:

carrying out human body posture recognition on the target pedestrian image to obtain coordinates and confidence degrees of a plurality of body parts;

converting the coordinates and the confidence degrees to obtain heat map information;

extracting and processing the target pedestrian image through a backbone network to obtain a first three-dimensional feature;

performing calculation processing on the heat map information and the first three-dimensional features to generate local features of a plurality of body parts;

inputting the local features into a pedestrian body attention module for training to generate an attention mask;

calculating the attention mask and the first three-dimensional feature to obtain a second three-dimensional feature;

inputting the second three-dimensional characteristics to a second-order information attention module for calculation processing to obtain a covariance matrix of a two-dimensional matrix;

and carrying out attention calculation on the covariance matrix to obtain a pedestrian recognition result, wherein the pedestrian recognition result is an output result of a second-order matrix.

In one embodiment, the step of inputting a plurality of local features into the pedestrian body attention module for training and generating the attention mask is represented by a mathematical formula as follows:

wherein Pavg represents local information obtained by location attention, Cavg represents local information obtained by channel attention, Relu is an activation function, A_maskTo focus on the mask.

In an embodiment, the step of calculating the attribute mask and the first three-dimensional feature to obtain a second three-dimensional feature is represented by a mathematical formula as follows:

and F on the right side of the equation equal sign is a first three-dimensional feature, and F on the left side of the equation equal sign is a second three-dimensional feature output by the pedestrian body attention module.

In an embodiment, the step of inputting the second three-dimensional feature to a second-order information attention module for calculation processing to obtain a covariance matrix of a two-dimensional matrix is represented by a mathematical formula as:

wherein I represents a unit matrix, F 'represents a two-dimensional matrix of F, (F')^TIndicating that F' is transposed.

In an embodiment, the performing an attention calculation on the covariance matrix to obtain a pedestrian recognition result, where the pedestrian recognition result is an output result of a second-order matrix and includes:

normalizing the covariance matrix to obtain a normalized covariance matrix A_second；

A is to be_secondAnd F' are multiplied, convolution processing is carried out, and a pedestrian recognition result is obtained and is an output result of the second-order matrix.

In an embodiment, the covariance matrix is normalized to obtain a covariance matrix a after the normalization_secondIs expressed by a mathematical formula as:

A_second＝softmax(∑)

wherein softmax () represents a normalization operation function.

In one embodiment, the step A is_secondAnd F' is multiplied, convolution processing is carried out, a pedestrian identification result is obtained, and the step of identifying the pedestrian as the output result of the second-order matrix is expressed by a mathematical formula as follows:

Z＝A_secondF′W_z

wherein Wz is a parameter of the convolutional layer.

In a second aspect, an embodiment of the present invention provides a pedestrian re-identification method and apparatus based on posture guidance, including:

the recognition module is used for recognizing the human body posture of the target pedestrian image to obtain the coordinates and confidence degrees of a plurality of body parts;

the conversion module is used for converting the coordinates and the confidence degrees to obtain heat map information;

the extraction module is used for extracting and processing the target pedestrian image through a backbone network to obtain a first three-dimensional feature;

the generating module is used for calculating the heat map information and the first three-dimensional characteristic to generate local characteristics of a plurality of body parts;

the training module is used for inputting the local features into the pedestrian body attention module for training to generate an attention mask;

the multiplication module is used for calculating the attention mask and the first three-dimensional feature to obtain a second three-dimensional feature;

the calculation module is used for inputting the second three-dimensional characteristics to the second-order information attention module for calculation processing to obtain a covariance matrix of a two-dimensional matrix;

and the result module is used for carrying out attention calculation on the covariance matrix to obtain a pedestrian recognition result, and the pedestrian recognition result is an output result of a second-order matrix.

In an embodiment, the step of performing calculation processing on the attention mask and the first three-dimensional feature to obtain a second three-dimensional feature is represented by a mathematical formula as follows:

In an embodiment, the result module is further configured to perform normalization processing on the covariance matrix to obtain a covariance matrix a after the normalization processing_second(ii) a A is to be_secondAnd F' are multiplied, convolution processing is carried out, and a pedestrian recognition result is obtained and is an output result of the second-order matrix.

A_second＝softmax(∑)

wherein softmax () represents a normalization operation function.

Z＝A_secondF′W_z

wherein Wz is a parameter of the convolutional layer.

In a third aspect, an embodiment of the present invention provides a computer device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the pedestrian re-recognition method based on gesture guidance according to the first aspect when executing the computer program.

In a fourth aspect, a computer-readable storage medium stores computer-executable instructions for performing the method for pedestrian re-identification based on pose guidance of the first aspect.

The embodiment of the invention comprises the following steps: the pedestrian re-identification method based on the posture guidance comprises the following steps: carrying out human body posture recognition on the target pedestrian image to obtain coordinates and confidence degrees of a plurality of body parts; converting the coordinates and the confidence coefficient to obtain heat map information; extracting and processing the target pedestrian image through a backbone network to obtain a first three-dimensional feature; calculating the heat map information and the first three-dimensional features to generate local features of a plurality of body parts; inputting a plurality of local features into a pedestrian body attention module for training to generate an attention mask; calculating the attention mask and the first three-dimensional feature to obtain a second three-dimensional feature; inputting the second three-dimensional characteristic into a second-order information attention module for calculation processing to obtain a covariance matrix of a two-dimensional matrix; and performing attention calculation on the covariance matrix to obtain a pedestrian recognition result, wherein the pedestrian recognition result is an output result of a second-order matrix. In the technical scheme of the embodiment, each body part is determined through the pedestrian body attention module, the influence of the shielding object on pedestrian recognition is avoided, the association degree between the parts is enhanced through the second-order information module, and the information expression of the human body can be emphasized through the second-order information module training because the association degree between the human body parts is greater than the association degree between the human body and the background and is greater than the association degree between the human body and the shielding object, so that the information expression of the background and the shielding object is suppressed.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

FIG. 1 is a schematic diagram of a system architecture platform for performing a pedestrian re-identification method based on pose guidance according to an embodiment of the present invention;

FIG. 2 is a flow chart of a pedestrian re-identification method based on attitude guidance according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating a second attention calculation in the method for pedestrian re-identification based on pose guidance according to an embodiment of the present invention;

FIG. 4 is a flow chart of a pedestrian re-identification method based on attitude guidance according to another embodiment of the present invention;

fig. 5 is a schematic diagram of a pedestrian re-recognition apparatus based on posture guidance according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

It should be noted that although functional blocks are partitioned in a schematic diagram of an apparatus and a logical order is shown in a flowchart, in some cases, the steps shown or described may be performed in a different order than the partitioning of blocks in the apparatus or the order in the flowchart. The terms "first," "second," and the like in the description, in the claims, or in the drawings described above, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.

The embodiment of the application can acquire and process related data based on an artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result.

The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

In order to solve the above existing problems, an embodiment of the present invention provides a pedestrian re-identification method based on posture guidance, including the following steps: carrying out human body posture recognition on the target pedestrian image to obtain coordinates and confidence degrees of a plurality of body parts; converting the coordinates and the confidence coefficient to obtain heat map information; extracting and processing the target pedestrian image through a backbone network to obtain a first three-dimensional feature; calculating the heat map information and the first three-dimensional features to generate local features of a plurality of body parts; inputting a plurality of local features into a pedestrian body attention module for training to generate an attention mask; calculating the attention mask and the first three-dimensional feature to obtain a second three-dimensional feature; inputting the second three-dimensional characteristic into a second-order information attention module for calculation processing to obtain a covariance matrix of a two-dimensional matrix; and performing attention calculation on the covariance matrix to obtain a pedestrian recognition result, wherein the pedestrian recognition result is an output result of a second-order matrix. In the technical scheme of the embodiment, each body part is determined through the pedestrian body attention module, the influence of the shielding object on pedestrian recognition is avoided, the association degree between the parts is enhanced through the second-order information module, and the information expression of the human body can be emphasized through the second-order information module training because the association degree between the human body parts is greater than the association degree between the human body and the background and is greater than the association degree between the human body and the shielding object, so that the information expression of the background and the shielding object is suppressed.

First, several terms referred to in the present application are explained:

AlphaPose uses a top-down approach, proposing an RMPE (regional multi-person pose detection) framework. The framework mainly comprises Symmetry Spatial Transform Network (SSTN), Parametric pool Non-Maximum-suppression (NMS) and pool-Guided pro-pollutants Generator (PGPG). And three technologies of Symmetry Spatial Transform Network (SSTN), Deep Prosalsgene (DPG) and parametric pore maximum suppression (p-NMS) are used for solving the multi-person attitude estimation problem in the field scene. Adding SSTN to the SPPE structure enables high quality human body regions to be extracted in imprecise region frames. Parallel SPPE branches (SSTNs) to optimize the own network. The redundancy detection problem is solved using a parametric pos NMS, in which a self-created pose distance metric scheme is used to compare the similarity between poses. And optimizing the attitude distance parameter by a data-driven method. And finally, strengthening training data by using PGPG, simulating the generation process of a human body region frame by learning the description information of different postures in the output result, and further generating a larger training set.

Heat map, which is the most common visualization means at present, is widely used in various large data analysis scenarios due to its rich color variation and vivid and rich information expression. Meanwhile, the R language special for scenes such as big data statistical analysis, drawing and visualization also provides a series of function libraries and toolkits with strong functions and comprehensive coverage in the aspect of visualization. Therefore, mapping heatmaps in the R language is one of the most common and necessary skills for the relevant practitioner. There are many software or methods for mapping heatmaps, such as Excel, R language, HemI, Python, MATLAB, etc., with various features or advantages. Excel is simplest, interface operation is easy to operate, but functions are not strong such as R language and Python with strong operability; r, Python and MATLAB have more parameters which can be set according to the requirements, and the effect is superior to Excel; the HemI function is between Excel and R. The realization of heat map mapping using Excel was introduced at this stage.

Resnet is an abbreviation for Residual Network (Residual Network), a family of networks widely used in the field of object classification and the like and as part of the classical neural Network of the computer vision task backbone, typical networks being Resnet50, Resnet101 and the like. The Resnet50 network structure firstly performs convolution operation on input, then comprises 4 residual blocks (ResidualBlock), finally performs full connection operation to facilitate classification task, and the Resnet50 network structure comprises 50 conv2d operations.

The softmax function, softmax, is used in the multi-classification process to map a plurality of inputs into (0,1) intervals, which can be understood as probabilities, thereby performing multi-classification. The softmax function first enlarges the difference between the input values by a natural base e and then normalizes it to a probability distribution using an allocation. In the classification problem, the probability of the model being assigned to the correct class is expected to be close to 1, and the other probabilities are close to 0, and if a linear normalization method is used, the effect is difficult to achieve, while softmax has a strategy of firstly pulling the difference and then normalizing, so that the advantages existing in the classification problem are remarkable.

The embodiments of the present invention will be further explained with reference to the drawings.

As shown in fig. 1, fig. 1 is a schematic diagram of a system architecture platform 100 for executing a pedestrian re-recognition method based on posture guidance according to an embodiment of the present invention.

In the example of fig. 1, the system architecture platform 100 is provided with a processor 110 and a memory 120, wherein the processor 110 and the memory 120 may be connected by a bus or other means, and fig. 1 illustrates the connection by the bus as an example.

The memory 120, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs as well as non-transitory computer executable programs. Further, the memory 120 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 120 optionally includes memory located remotely from processor 110, which may be connected to the system architecture platform via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

It will be understood by those skilled in the art that the system architecture platform may be applied to a 5G communication network system, a mobile communication network system evolved later, and the like, and the embodiment is not limited thereto.

Those skilled in the art will appreciate that the system architecture platform illustrated in FIG. 1 does not constitute a limitation on embodiments of the invention, and may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components.

The system architecture platform 100 may be an independent system architecture platform, or may be a cloud system architecture platform 100 that provides basic cloud computing services such as cloud services, a cloud database, cloud computing, cloud functions, cloud storage, Network services, cloud communications, middleware services, domain name services, security services, a Content Delivery Network (CDN), and big data and artificial intelligence platforms.

Based on the system architecture platform, the following provides various embodiments of the pedestrian re-identification method based on posture guidance.

As shown in fig. 2, fig. 2 is a flowchart of a pedestrian re-identification method based on posture guidance according to an embodiment of the present invention, the pedestrian re-identification method based on posture guidance is applied to the above-mentioned architecture platform, and the pedestrian re-identification method based on posture guidance includes, but is not limited to, step S100, step S200, step S300, step S400, step S500, step S600, step S700, and step S800.

S100, recognizing the human body posture of the target pedestrian image to obtain the coordinates and confidence degrees of a plurality of body parts;

step S200, converting the coordinates and the confidence coefficient to obtain heat map information;

step S300, extracting and processing through a backbone network resnet50 to obtain a first three-dimensional feature;

step S400, multiplying the heat map information and the first three-dimensional characteristic to generate local characteristics of a plurality of body parts;

step S500, inputting a plurality of local features into a pedestrian body attention module for training, and generating an attention mask;

step S600, multiplying the attribute mask and the first three-dimensional feature to obtain a second three-dimensional feature;

step S700, inputting the second three-dimensional characteristic into a second-order information attention module for calculation processing to obtain a covariance matrix of a two-dimensional matrix;

and step S800, carrying out attention calculation on the covariance matrix to obtain a pedestrian recognition result, wherein the pedestrian recognition result is an output result of a second-order matrix.

In one embodiment, human body posture recognition is carried out on a target pedestrian image to obtain coordinates and confidence degrees of a plurality of body parts; converting the coordinates and the confidence coefficient to obtain heat map information; extracting and processing the target pedestrian image through a backbone network to obtain a first three-dimensional feature; calculating the heat map information and the first three-dimensional features to generate local features of a plurality of body parts; inputting a plurality of local features into a pedestrian body attention module for training to generate an attention mask; calculating the attention mask and the first three-dimensional feature to obtain a second three-dimensional feature; inputting the second three-dimensional characteristic into a second-order information attention module for calculation processing to obtain a covariance matrix of a two-dimensional matrix; and performing attention calculation on the covariance matrix to obtain a pedestrian recognition result, wherein the pedestrian recognition result is an output result of a second-order matrix. In the technical scheme of the embodiment, each body part is determined through the pedestrian body attention module, the influence of the shielding object on pedestrian recognition is avoided, the association degree between the parts is enhanced through the second-order information module, and the information expression of the human body can be emphasized through the second-order information module training because the association degree between the human body parts is greater than the association degree between the human body and the background and is greater than the association degree between the human body and the shielding object, so that the information expression of the background and the shielding object is suppressed.

It should be noted that the target pedestrian image may be one image facing the target crowd, may be multiple pedestrian images facing the target crowd at the same time at multiple angles, or may be multiple pedestrian images facing the target crowd at close times at multiple angles, which is not specifically limited in this embodiment.

It should be noted that the three-dimensional features may be obtained by extracting features from the image data using the backbone network resnet50, or may be extracted from the image data using another model, which is not limited in this embodiment.

It should be noted that the network can predict the position coordinates and confidence of each part of the body, and the coordinates and confidence can be converted into heatmap by using gaussian blur, and the present embodiment does not specifically limit the technology of converting into heatmap.

In an embodiment, the calculation method of the pedestrian body attention module is to perform channel attention and position attention on each part of a body respectively, generate an integral attribution mask for the parts of the body, and multiply the integral attribution mask and an initial backbone network to obtain an output three-dimensional feature of the part. The second-order information attention module receives the three-dimensional characteristics output by the pedestrian body attention module, processes the three-dimensional characteristics by counting the high-order relation between characteristic values, firstly, two-dimensionalizes the three-dimensional characteristics, then calculates the covariance matrix of the two-dimensional matrix, performs normalization operation after preprocessing the covariance matrix, multiplies the two-dimensional matrix and then performs convolution operation again so as to complete network learning and obtain the output result of the second-order matrix. The method comprises the steps that a pedestrian body attention module is used for determining each body part, the influence of a barrier on pedestrian recognition is avoided, the degree of association between all parts is strengthened through a second-order information module, and the degree of association between the body parts is larger than the degree of association between a human body and a background and larger than the degree of association between the human body and the barrier, so that the information expression of the human body can be emphasized through the training of the second-order information module, and the information expression of the background and the barrier is restrained.

Referring to fig. 3, in an embodiment, step S800 includes, but is not limited to, step S310 and step S320.

Step S310, normalization processing is carried out on the covariance matrix to obtain the covariance matrix A after normalization processing_second；

Step S320, A_secondAnd F' are multiplied, convolution processing is carried out, and a pedestrian recognition result is obtained and is an output result of the second-order matrix.

Specifically, after passing through the pedestrian body attention module, the output result is sent to a second-order information attention module, and the second-order information attention module processes information through the high-order relation between statistical characteristic values. Firstly, a second-order information attention module bidimensionalizes the three-dimensional characteristics, then calculates a covariance matrix of a two-dimensional matrix, and performs normalization processing on the covariance matrix to obtain A_secondThen A is added_secondPerforming a second attention calculation, namely, performing normalization on the covariance matrix A_secondAnd multiplying the two-dimensional matrix F' and then performing convolution processing again to obtain an output result of a second-order matrix.

As shown in fig. 4, fig. 4 is a flowchart of a pedestrian re-identification method based on posture guidance according to another embodiment of the present invention, and as can be seen from the flowchart, the calculation module for implementing the pedestrian re-identification method mainly includes a pedestrian body attention module and a second-order information attention module. The preprocessing step for an input pedestrian picture comprises the following steps: in the first line, alpha pos is extracted through the posture to obtain the coordinates and confidence of n body parts, and then the coordinates and confidence are converted into heatmap. On the second line, the first three-dimensional feature in the picture is extracted through the backbone network resnet 50. And then, after the information of the two lines is extracted, multiplying the two information of the heatmap and the first three-dimensional feature to obtain the local feature features of the n body parts.

And after the local feature is obtained, inputting the local feature into a pedestrian body attention module for processing. In the pedestrian body attention module, firstly, the local feature of each part of the body is respectively processed by channel attention and position attention, then the body parts generate an integral attribution mask, and the integral attribution mask is multiplied by an initial backbone network to obtain the output of the part. The formula is expressed as:

wherein, Pavg represents the local information obtained by the location attention, Cavg represents the local information obtained by the channel attention, and after the two information are multiplied, the attention mask amask (attention mask) is obtained by a convolutional layer (where Wa represents the parameter of the convolutional layer and Wa is the parameter obtained by learning) and then an activation function RELU ().

And multiplying Amask and the first three-dimensional feature F output by the backbone network to obtain a new second three-dimensional feature F, wherein the second three-dimensional feature F is the output of the whole pedestrian body attention module.

And after passing through the pedestrian body attention module, sending the second three-dimensional characteristic F of the output result into a second-order information attention module, and processing the information through the high-order relation between the statistical characteristic values.

Firstly, performing two-dimensional processing on the first three-dimensional characteristic F to obtain a two-dimensional matrix F ', and then calculating a covariance matrix of the two-dimensional matrix F'. The formula is as follows:

i represents a unit matrix, (F')^TThe target matrix is transposed, the calculation result is a covariance matrix (covariance matrix) of the target matrix F', the covariance matrix is normalized, and then the second iteration calculation is carried out:

A_second＝softmax(∑)

Z＝A_secondF′W_z

in the formula, softmax () represents a normalized operation function to obtain A_secondA is_secondAnd F' are multiplied, and then convolution operation is carried out again (wherein the parameters of the convolution layer are Wz, and Wz is the parameters obtained through network learning), so that an output result Z of the second-order matrix is obtained.

The pedestrian body attention module is used for determining each body part, the influence of the shielding object on pedestrian recognition is avoided, the second-order information module is used for enhancing the association degree of all the parts, and the association degree of the body parts is larger than the association degree of the human body and the background and larger than the association degree of the human body and the shielding object, so that the information expression of the human body can be emphasized through the training of the second-order information module, and the information expression of the background and the shielding object is restrained.

Based on the above pedestrian re-identification method based on the attitude guidance, the following respectively proposes various embodiments of the pedestrian re-identification method apparatus based on the attitude guidance, the controller and the computer readable storage medium of the invention.

Referring to fig. 5, an embodiment of the present invention also provides a pedestrian re-recognition apparatus based on posture guidance, including:

the recognition module 510 is configured to perform human posture recognition on the target pedestrian image to obtain coordinates and confidence levels of a plurality of body parts;

the conversion module 520 is configured to perform conversion processing on the coordinates and the confidence degrees to obtain heat map information;

an extracting module 530, configured to extract a target pedestrian image through a backbone network to obtain a first three-dimensional feature;

a generating module 540, configured to multiply the heat map information and the first three-dimensional feature to generate local features of a plurality of body parts;

a training module 550, configured to input the local features into a pedestrian body attention module for training, so as to generate an attention mask;

a multiplying module 560, configured to perform calculation processing on the attention mask and the first three-dimensional feature to obtain a second three-dimensional feature;

the calculation module 570 is used for inputting the second three-dimensional features to the second-order information attention module for calculation processing to obtain a covariance matrix of the two-dimensional matrix;

a result module 580, configured to perform attention calculation on the covariance matrix to obtain a pedestrian recognition result, where the pedestrian recognition result is an output result of a second-order matrix.

In one embodiment, the step of inputting a plurality of local features into the pedestrian body attention module for training and generating the attention mask is expressed by a mathematical formula as follows:

wherein Pavg represents local information obtained by location attention, Cavg represents local information obtained by channel attention, Relu is an activation function, A_maskIs the attribute mask.

In an embodiment, the step of multiplying the attribute mask and the first three-dimensional feature to obtain the second three-dimensional feature is expressed by a mathematical formula as follows:

In an embodiment, the step of inputting the second three-dimensional feature to the second-order information attention module for calculation processing to obtain the covariance matrix of the two-dimensional matrix is represented by a mathematical formula as follows:

In an embodiment, the result module 800 is further configured to perform normalization on the covariance matrix to obtain a covariance matrix a after the normalization_second(ii) a A is to be_secondAnd F' are multiplied, convolution processing is carried out, and a pedestrian recognition result is obtained and is an output result of the second-order matrix.

A_second＝softmax(∑)

wherein softmax () represents a normalization operation function.

In one embodiment, A is_secondMultiplying the result F 'by the sum F', performing convolution processing to obtain a pedestrian identification result, wherein the step of identifying the pedestrian result as an output result of a second-order matrix is represented by a mathematical formula as follows:

Z＝A_secondF′W_z

wherein Wz is a parameter of the convolutional layer.

It should be noted that, the technical means, the technical problems solved and the technical effects achieved in the embodiments of the pedestrian re-identification apparatus based on the posture guidance and the embodiments of the pedestrian re-identification method based on the posture guidance are the same, and detailed descriptions are not provided herein for details, and see the embodiments of the pedestrian re-identification method based on the posture guidance.

In addition, an embodiment of the present invention provides a computer apparatus including: a memory, a processor, and a computer program stored on the memory and executable on the processor.

The processor and memory may be connected by a bus or other means.

It should be noted that the computer device in this embodiment may be configured to include a memory and a processor as in the embodiment shown in fig. 1, and can form a part of the system architecture platform in the embodiment shown in fig. 1, and both are within the same inventive concept, so that both have the same implementation principle and beneficial effects, and are not described in detail herein.

The non-transitory software programs and instructions required to implement the device-side pedestrian re-recognition method based on pose guidance of the above-described embodiment are stored in a memory, and when executed by a processor, perform the pedestrian re-recognition method based on pose guidance of the above-described embodiment, for example, performing the above-described method steps S100 to S800 in fig. 2, and S310 to S320 in fig. 3.

The device may be a computer device comprising: radio Frequency (RF) circuit, memory, input unit, display unit, sensor, audio circuit, wireless fidelity (WiFi) module, processor, and power supply. Those skilled in the art will appreciate that the present embodiments are not exclusive as to the configuration of the computer device and may include more or fewer components than the present embodiments, or a combination of certain components, or a different arrangement of components.

The RF circuit can be used for receiving and transmitting signals in the process of information receiving and transmitting or conversation, and particularly, the downlink information of the base station is received and then is processed by the processor; in addition, the data for designing uplink is transmitted to the base station. Typically, the RF circuit includes, but is not limited to, an antenna, at least one Amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, the RF circuitry may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to Global System for Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Message Service (SMS), and the like.

The memory may be used to store software programs and modules, and the processor may execute various functional applications of the computer device and data processing by operating the software programs and modules stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the computer device, and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The input unit may be used to receive input numeric or character information and generate key signal inputs related to settings and function control of the computer device. Specifically, the input unit may include a touch panel and other input devices. The touch panel, also called a touch screen, may collect touch operations thereon or nearby (such as operations on or near the touch panel using any suitable object or accessory, such as a finger, a stylus, etc.) and drive the corresponding connection device according to a preset program. Alternatively, the touch panel may include two parts, a touch detection device and a touch controller. The touch detection device detects a touch direction, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor, and can receive and execute commands sent by the processor. In addition, the touch panel may be implemented by various types such as resistive, capacitive, infrared, and surface acoustic wave. The input unit may include other input devices in addition to the touch panel. In particular, other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

The display unit may be used to display input information or provided information as well as various menus of the computer apparatus. The display unit may include a display panel, and optionally, the display panel may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch panel may cover the display panel, and when the touch panel detects a touch operation thereon or nearby, the touch panel transmits the touch operation to the processor to determine a category of the touch event, and then the processor provides a corresponding visual output on the display panel according to the category of the touch event. Although the touch panel and the display panel are two separate components to implement the input and output functions of the computer device, in some embodiments, the touch panel and the display panel may be integrated to implement the input and output functions of the computer device.

The computer device may also include at least one sensor, such as a light sensor, a motion sensor, and other sensors. In particular, the light sensor may include an ambient light sensor that adjusts the brightness of the display panel based on the intensity of ambient light, and a proximity sensor that turns off the display panel and/or backlight when the computer device is moved to the ear. As one type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used for applications (such as horizontal and vertical screen switching, related games, magnetometer attitude calibration) for recognizing the attitude of a computer device, and related functions (such as pedometer and tapping) for vibration recognition; as for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which can be configured on the computer device, detailed descriptions thereof are omitted.

The audio circuit, speaker, microphone may provide an audio interface. The audio circuit can transmit the electric signal converted from the received audio data to the loudspeaker, and the electric signal is converted into a sound signal by the loudspeaker to be output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit and converted into audio data, which is then output to the processor for processing, and then through the RF circuit for transmission to, for example, another computer device, or for outputting the audio data to the memory for further processing.

WiFi belongs to short-distance wireless transmission technology, computer equipment can receive and send e-mails, browse webpages, access streaming media and the like through a WiFi module, and wireless broadband internet access is provided. The WiFi module is not an essential component of the computer device and can be omitted entirely as needed within the scope not changing the essence of the invention.

The processor is a control center of the computer equipment, connects various parts of the whole computer equipment by various interfaces and lines, executes various functions of the computer equipment and processes data by running or executing software programs and/or modules stored in the memory and calling the data stored in the memory, thereby monitoring the computer equipment as a whole. Alternatively, the processor may include one or more processing units; preferably, the processor may integrate an application processor and a modem processor, wherein the application processor mainly handles operating systems, operating interfaces, application programs, and the like, and the modem processor mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor.

The computer device also includes a power supply (e.g., a battery) for powering the various components, which may preferably be logically connected to the processor via a power management system, such that the power management system performs functions of managing charging, discharging, and power consumption.

Although not shown, the computer device may further include a camera, a bluetooth module, etc., which will not be described herein.

In the present embodiment, the processor included in the terminal device is capable of executing the pedestrian re-identification method of the previous embodiment.

Furthermore, an embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions, when the computer-executable instructions are used for executing the above-mentioned pedestrian re-identification method based on posture guidance on the terminal side, for example, executing the above-described method steps S100 to S800 in fig. 2 and the method steps S310 to S320 in fig. 3.

One of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.

While the preferred embodiments of the present invention have been described in detail, it will be understood by those skilled in the art that the foregoing and various other changes, omissions and deviations in the form and detail thereof may be made without departing from the scope of this invention.

Claims

1. A pedestrian re-identification method based on attitude guidance comprises the following steps:

2. The pedestrian re-recognition method based on posture guidance of claim 1, wherein the step of inputting the plurality of local features to a pedestrian body attention module for training and generating an attention mask is represented by a mathematical formula as:

3. The pedestrian re-recognition method based on pose guidance according to claim 2, wherein the step of performing calculation processing on the attention mask and the first three-dimensional feature to obtain a second three-dimensional feature is represented by a mathematical formula as follows:

4. The pedestrian re-recognition method based on posture guidance as claimed in claim 3, wherein the step of inputting the second three-dimensional feature to a second-order information attention module for calculation processing to obtain a covariance matrix of a two-dimensional matrix is represented by a mathematical formula as:

5. The pedestrian re-identification method based on the attitude guidance according to claim 4, wherein the performing attention calculation on the covariance matrix to obtain a pedestrian identification result, the pedestrian identification result being an output result of a second-order matrix comprises:

6. The pedestrian re-identification method based on attitude guidance according to claim 5, wherein the covariance matrix is normalized to obtain a covariance matrix A after normalization_secondIs expressed by a mathematical formula as:

A_second＝softmax(∑)

wherein softmax () represents a normalization operation function.

7. The pedestrian re-recognition method based on attitude guidance according to claim 5, wherein the A is_secondAnd F' is multiplied, convolution processing is carried out, a pedestrian identification result is obtained, and the step of identifying the pedestrian as the output result of the second-order matrix is expressed by a mathematical formula as follows:

Z＝A_secondF′W_z

wherein Wz is a parameter of the convolutional layer.

8. A pedestrian re-recognition method device based on attitude guidance is characterized by comprising the following steps:

9. A computer device, comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of pedestrian re-recognition based on gesture guidance according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing computer-executable instructions for performing the method for pedestrian re-identification based on gesture guidance of any one of claims 1 to 7.