CN115049816A

CN115049816A - Target identification method and device, electronic equipment and storage medium

Info

Publication number: CN115049816A
Application number: CN202110212387.7A
Authority: CN
Inventors: 刘阳; 张恒; 孙常库
Original assignee: Quantaeye Beijing Technology Co ltd
Current assignee: Quantaeye Beijing Technology Co ltd
Priority date: 2021-02-25
Filing date: 2021-02-25
Publication date: 2022-09-13

Abstract

The present disclosure relates to a target identification method and apparatus, an electronic device, and a storage medium, the method including: obtaining a first spectral image of a target area by a spectral camera; detecting the spectral image to obtain a first position of the target object in the spectral image; determining a target shooting angle of the color camera according to the first position; shooting by a color camera according to a target shooting angle to obtain a first color image of a target object; and carrying out identification processing on the target object in the first color image to obtain an identification result. According to the target identification method disclosed by the embodiment of the disclosure, the characteristic of larger field of view of the spectrum camera and the characteristic of high resolution of the color camera can be simultaneously utilized, the spectrum camera with the larger field of view is used for shooting a large-range spectrum image so as to quickly determine the position of the target object, the color camera with the high resolution is used for shooting the color image of the position of the target object, and the accuracy rate of identifying the target object is improved.

Description

Target identification method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a target identification method and apparatus, an electronic device, and a storage medium.

Background

The target identification is to find a target object of interest based on data of the sensor, and in the related art, the target object may be identified in an image using a neural network. When the distance to the target is relatively far, for example, in scenes such as unmanned aerial vehicle aerial photography, satellite remote sensing and the like, the range of the target area is large, and a camera with a large view field must be used to cover a larger range, but pixels of the target object in the camera with the large view field are relatively few on an imaging surface, and even the target object becomes a spot without a geometric shape, so that too little information can be acquired from the image, and the accuracy of target identification can be greatly reduced. One solution is to lose a part of the field of view, use a high-definition camera for photographing and object recognition, but in this solution, the same scene must be photographed and recognized many times, which can improve a certain accuracy, but reduce the time efficiency of recognition.

Disclosure of Invention

The disclosure provides a target identification method and device, an electronic device and a storage medium.

According to an aspect of the present disclosure, there is provided a target recognition method including: obtaining a first spectral image of a target area by a spectral camera; detecting the spectral image to obtain a first position of a target object in the spectral image; determining a target shooting angle of the color camera according to the first position, wherein the shooting angle is a shooting angle of the target object in a field of view of the color camera, and the field of view of the spectrum camera is larger than that of the color camera; shooting by the color camera according to the target shooting angle to obtain a first color image of the target object; and identifying the target object in the first color image to obtain an identification result.

In one possible implementation, the method further includes: calibrating the spectrum camera and the color camera to obtain a first internal reference matrix of the spectrum camera and a second internal reference matrix of the color camera; determining a transformation matrix between the spectral camera and the color camera from the first and second internal reference matrices.

In one possible implementation, the determining the transformation matrix between the spectrum camera and the color camera according to the first internal reference matrix and the second internal reference matrix includes determining the transformation matrix between the spectrum camera and the color camera according to a position transformation matrix between second position information of a target position in a second color image captured by the color camera at a plurality of capturing angles and corresponding third position information in a second spectrum image captured by the spectrum camera, and includes: determining a translation vector and a rotation matrix according to second position information in a second color image shot by a color camera at a first shooting angle, corresponding third position information in the second spectrum image and the first internal reference matrix, wherein the first shooting angle is any shooting angle of the color camera; and determining a position transformation matrix corresponding to the first shooting angle according to the second position information, the third position information, the translation vector and the rotation matrix.

In one possible implementation, determining a target shooting angle of the color camera according to the first position includes: determining the corresponding positions of the target positions in the first spectrum images in the color images shot by the color camera at a plurality of shooting angles according to the transformation matrix; determining a second position having a smallest distance from the first position among the corresponding positions; and determining the shooting angle corresponding to the second position as the target shooting angle.

In one possible implementation, the target position is a center position of a color image captured by the color camera.

In one possible implementation manner, performing recognition processing on the target object in the first color image to obtain a recognition result includes: determining a target region of the target object in the first color image; and identifying the target object in the target area to obtain an identification result.

In one possible implementation, determining a target region of the target object in the first color image includes: and determining a target area of the target object in the first color image according to the transformation matrix and the position information of the first position.

According to an aspect of the present disclosure, there is provided an object recognition apparatus including: the spectral image module is used for obtaining a first spectral image of the target area through the spectral camera; the detection module is used for detecting the spectral image to obtain a first position of a target object in the spectral image; an angle module, configured to determine a target shooting angle of the color camera according to the first position, where the shooting angle is a shooting angle of the target object in a field of view of the color camera, and the field of view of the spectrum camera is larger than the field of view of the color camera; the color image module is used for shooting according to the target shooting angle through the color camera to obtain a first color image of the target object; and the identification module is used for identifying the target object in the first color image to obtain an identification result.

In one possible implementation, the apparatus further includes: the calibration module is used for calibrating the spectrum camera and the color camera to obtain a first internal reference matrix of the spectrum camera and a second internal reference matrix of the color camera; a transform matrix module to determine a transform matrix between the spectral camera and the color camera according to the first and second internal reference matrices.

In one possible implementation, the transformation matrix between the spectral camera and the color camera includes a position transformation matrix between second position information of the target position in a second color image captured by the color camera at a plurality of capturing angles and corresponding third position information in a second spectral image captured by the spectral camera, and the transformation matrix module is further configured to: determining a translation vector and a rotation matrix according to second position information in a second color image shot by a color camera at a first shooting angle, corresponding third position information in the second spectrum image and the first internal reference matrix, wherein the first shooting angle is any shooting angle of the color camera; and determining a position transformation matrix corresponding to the first shooting angle according to the second position information, the third position information, the translation vector and the rotation matrix.

In one possible implementation, the angle module is further configured to: determining corresponding positions of target positions in the first spectrum images in color images shot by the color camera at a plurality of shooting angles according to the transformation matrix; determining a second position having a smallest distance from the first position among the corresponding positions; and determining the shooting angle corresponding to the second position as the target shooting angle.

In one possible implementation, the identification module is further configured to: determining a target region of the target object in the first color image; and identifying the target object in the target area to obtain an identification result.

In one possible implementation, the identification module is further configured to: and determining a target area of the target object in the first color image according to the transformation matrix and the position information of the first position.

According to an aspect of the present disclosure, there is provided an electronic device including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: the above object recognition method is performed.

According to an aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-mentioned object recognition method.

According to the target identification method disclosed by the embodiment of the disclosure, the characteristic of larger field of view of the spectral camera and the characteristic of high resolution of the color camera can be simultaneously utilized, the spectral camera with the larger field of view is used for shooting a wide-range spectral image so as to quickly determine the position of a target object, the shooting angle of the color camera is determined by converting a matrix, and then the color camera is used for shooting at the shooting angle to obtain a color image comprising the target object. The color image has higher resolution, the target object is clear, the target object can be further identified according to the color image, and the identification accuracy is improved. In summary, the larger field of view of the spectral camera can be used to improve the efficiency of determining the position of the target object, while the high resolution of the color camera can be used to improve the accuracy of identifying the target object.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

FIG. 1 shows a flow diagram of a target identification method according to an embodiment of the present disclosure;

FIG. 2 shows a schematic diagram of an application of a target recognition method according to an embodiment of the present disclosure;

FIG. 3 shows a block diagram of a target recognition device, according to an embodiment of the present disclosure;

FIG. 4 shows a block diagram of an electronic device according to an embodiment of the present disclosure;

fig. 5 illustrates a block diagram of an electronic device in accordance with an embodiment of the disclosure.

Detailed Description

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.

Fig. 1 shows a flow chart of a target identification method according to an embodiment of the present disclosure, as shown in fig. 1, the method comprising:

in step S11, a first spectral image of the target area is obtained by the spectral camera;

in step S12, performing detection processing on the spectral image to obtain a first position of a target object in the spectral image;

in step S13, determining a target shooting angle of the color camera according to the first position, wherein the shooting angle is a shooting angle of the target object in the field of view of the color camera, and the field of view of the spectrum camera is larger than that of the color camera;

in step S14, capturing images according to the target capturing angle by the color camera to obtain a first color image of the target object;

in step S15, a target object in the first color image is subjected to recognition processing, and a recognition result is obtained.

According to the target identification method disclosed by the embodiment of the disclosure, the characteristic of larger field of view of the spectrum camera and the characteristic of high resolution of the color camera can be simultaneously utilized, the spectrum camera with the larger field of view is used for shooting a wide-range spectrum image so as to quickly determine the position of the target object, and the color camera with the high resolution is used for shooting a color image of the position of the target object so as to further identify the target object. The larger field of view of the spectral camera may be utilized to improve the efficiency of determining the position of the target object, while the high resolution of the color camera is utilized to improve the accuracy of identifying the target object.

In a possible implementation manner, the object recognition method may be performed by an electronic device such as a terminal device or a server, the terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, or the like, and the method may be implemented by a processor calling a computer readable instruction stored in a memory. Alternatively, the object recognition method may be performed by a server.

In one possible implementation, in case of unmanned aerial vehicle aerial photography, satellite remote sensing, etc., a large-range area image is usually acquired by a camera with a large field of view (e.g., a spectral camera), and a target object in the area image can be searched. However, although the field of view of the large-range area image is large, the area of the target object in the area image may be small, and particularly, in the case that the target object itself is small, the area occupied in the large-field area image is very small, and even only one or a few pixel points exist, it is difficult to determine the geometric shape of the large-range area image, so that it is difficult to identify the target object. If a high resolution camera is used, the field of view is reduced, which may result in a less efficient search when searching for the location of the target object. Accordingly, a large field of view spectral camera may be utilized in conjunction with a high definition color camera to efficiently and accurately identify a target object.

In a possible implementation manner, the spectrum camera may obtain spectrum information of each position of the target area, that is, each pixel point in the first spectrum image captured by the spectrum camera may have spectrum information, and the spectrum information is information of a plurality of lights reflected by the pixel point, may include rich optical information, and may be used to determine the position of the target object. That is, the position of the target object can be determined in the spectral image of the large field of view without taking multiple images and finding the position of the target object one by a small field of view camera with high resolution. However, although the spectral image may provide spectral information of each pixel, there may be cases where spectra of different targets are similar or identical, and therefore, detecting a target object by using a spectrum is a large-range coarse-grained detection method, and there may be errors in the detected target object. Therefore, after the position of the target object is detected, a high-definition color image of the position can be shot by a high-definition color camera and is identified by the high-definition color image, and an accurate identification result can be obtained.

In summary, by the method of combining the large-view-field spectrum camera and the high-resolution color camera, the position of the target object can be quickly found through the large-view-field spectrum image, the image at the position can be shot through the color camera, the position of the target object can be found one by one without shooting a plurality of images through the high-resolution color camera, and the accurate recognition result can be obtained by utilizing the advantage of the high resolution of the color camera.

In one possible implementation manner, in step S11, the first spectral image may be a spectral image obtained by a large-field-of-view spectral camera disposed on an aircraft such as an unmanned aerial vehicle or a satellite, and each pixel point of the first spectral image may represent spectral information of a position corresponding to a pixel point in a wide area, that is, composition information of reflected light at the position, that is, which wavelength bands of light the reflected light at the position consists of.

In one possible implementation manner, in step S12, the first spectral image may be subjected to detection processing through the above-mentioned spectral information, and the first position of the target object in the spectral image is determined. For example, since the reflected light of different objects differs, the shapes, colors, and the like exhibited by the different objects differ, and the spectral information of the reflected light of the different objects also differs. The spectral information of each pixel point can be utilized to determine the position of the pixel point with the target spectral information. In an example, the spectral information of each pixel point may be compared to the target spectral information to determine the pixel point with the target spectral information. In another example, the target object is an object having a plurality of reflected lights, for example, a plurality of different colors exist on the target object, or the reflected lights of the target object at a plurality of photographing angles are different, and the like, in which case, the first position where the target object is located may be determined in the first spectral image by using a neural network or the like. The present disclosure does not limit the method of detecting the first position.

In one possible implementation manner, after the first position is determined, the shooting angle of the high-definition color camera may be adjusted, so that the target object may appear in the field of view of the color camera with a smaller field of view, that is, the color camera may be able to shoot the position of the target object after the shooting angle is determined.

In one possible implementation, a relationship between the spectral camera and the color camera may be determined, i.e., a shooting angle of the color camera, i.e., a target shooting angle, may be determined by means of the relationship and the first position. The relationship may be a transformation matrix, i.e. a matrix of relationships between a certain position in an image taken by a color camera and a corresponding position in an image taken by a spectral camera. The method further comprises the following steps: calibrating the spectrum camera and the color camera to obtain a first internal reference matrix of the spectrum camera and a second internal reference matrix of the color camera; determining a transformation matrix between the spectral camera and the color camera from the first and second internal reference matrices.

In one possible implementation, the spectral camera and the color camera may be calibrated separately first, and a first internal reference matrix of the spectral camera and a second internal reference matrix of the color camera may be obtained. Distortion parameters and other camera parameters of the spectrum camera and the color camera can also be determined. The elements in the first and second internal reference matrices are parameters describing the focal length and optical center position, and can be used as reference parameters of the cameras for position transformation between the two cameras. In an example, the first internal reference matrix and the second internal reference matrix may be obtained by a calibration method such as a zhangying friend plane calibration method, and the disclosure does not limit the calibration method.

In one possible implementation, a transformation matrix between the spectral camera and the color camera may be determined using the first and second internal reference matrices. In an example, the transformation matrix is generally fixed, i.e., the relative position between the two cameras is fixed, so that the relative positional relationship of the same position captured by the two cameras in the two images can be determined by the transformation matrix. However, the fields of view of the spectrum camera and the color camera are different, and the positions photographed by the spectrum camera may need to be photographed after the color camera adjusts the photographing angles, so that the position transformation matrix of the color camera and the spectrum camera at a plurality of photographing angles (i.e. the relative position relationship of the same object in two photographed images at a plurality of photographing angles) can be determined.

In one possible implementation, the determining the transformation matrix between the spectrum camera and the color camera according to the first internal reference matrix and the second internal reference matrix includes: determining a translation vector and a rotation matrix according to second position information in a second color image shot by a color camera at a first shooting angle, corresponding third position information in the second spectrum image and the first internal reference matrix, wherein the first shooting angle is any shooting angle of the color camera; and determining a position transformation matrix corresponding to the first shooting angle according to the second position information, the third position information, the translation vector and the rotation matrix.

In one possible implementation, the field of view of the spectral camera is larger than that of the color camera, and the range of the area that can be photographed is larger. The field of view of the color camera is small, and the range of the area capable of being shot is small, but the color camera can shoot a plurality of areas by adjusting the shooting angle, and the ranges of the areas can cover the range of the area shot by the spectrum camera. In an example, the long side of the area shot by the color camera is not smaller than the short side of the area shot by the spectrum camera, so that the color camera can make the range of the plurality of areas shot cover the range of the area shot by the spectrum camera only by adjusting the angle in one direction, for example, the color camera can make the range of the plurality of areas shot cover the range of the area shot by the spectrum camera only by adjusting the pitch angle in the pitch direction. Otherwise, the color camera needs to change the angles in two directions, for example, the angles in two directions of the pitch angle and the azimuth angle need to be adjusted, so that the range of the plurality of captured regions can cover the range of the captured region of the spectrum camera.

In one possible implementation, the photographing angle of the color camera may be set to the first photographing angle θ ₀ (any shooting angle) and selecting the position of the same target in the images shot by the two cameras, for example, the center position of the second color image shot by the color camera and the corresponding position of the center position in the second spectrum image shot by the spectrum camera can be selected, for example, if the target position (for example, the center position) of the second color image shot by the color camera is the position of the target a, the corresponding position is the position of the target a in the second spectrum image. The position information of the position in the second spectral image is third position information p ₁ (u ₁ ,v ₁ ) The position information of the position in the second color image is second position information p ₂ (u ₂ ,v ₂ )。

In one possible implementation, the translation vector and the rotation matrix may be determined based on the second position information, the third position information, the first internal reference matrix, and the second internal reference matrix. In an example, the rotation matrix and the translation vector may be determined by the following equation (1):

s(u ₁ ,v ₁ ,1)＝K ₂ (R(x,y,z)+t) (1)

where s is the matrix coefficient, R (x, y, z) is the rotation matrix, t is the translation vector, K ₂ Is a second reference matrix.

In an example, the matrix coefficients may be determined according to the following equation (2):

s(u ₁ ,v ₁ ,1)＝K ₁ (x,y,z) (2)

wherein, K ₁ Is a first internal reference matrix.

In one possible implementation, the expressions of the translation vector and the rotation matrix may be determined according to equations (1) and (2), and further, the position transformation matrix may be pair-wise according to the second position information, the third position information, the translation vector, and the rotation matrix.

In an example, a position transformation matrix of a relationship between pixel coordinates of two cameras may be determined according to the second position information and the third position information, for example, the base matrix may be represented by the following formula (3):

wherein F is the position transformation matrix.

Further, the relationship between the first internal reference matrix, the second internal reference matrix, the rotation matrix, and the translation vector and the base matrix may be determined according to the following equation (4):

further, the equations (3) and (4) may be integrated, the position transformation matrix F may be solved, and F may be determined as the first photographing angle θ ₀ A corresponding position transformation matrix.

In one possible implementation manner, the plurality of shooting angles θ can be obtained in the above manner _i Corresponding position transformation matrix F _i And i is a positive integer.

In one possible implementation, in step S13, the first location is a location of the target object in the spectral image. The color camera may be caused to select a target shooting angle such that the target object appears in the field of view of the color camera.

In one possible implementation, the target shooting angle may be determined according to the first position and the transformation matrix. Step S13 may include: determining the corresponding positions of the target positions in the first spectrum images in the color images shot by the color camera at a plurality of shooting angles according to the transformation matrix; determining a second position having a smallest distance from the first position among the corresponding positions; and determining the shooting angle corresponding to the second position as the target shooting angle.

In one possible implementation, after determining the transformation matrix, the corresponding position of the target position of the color image taken at each angle in the first spectral image may be determined based on the transformation matrix. For example, the angle may be determined according to the shooting angle θ ₀ Corresponding position transformation matrix F ₀ Determining the photographing angle theta of the color camera ₀ The corresponding position of the target position in the shot color image in the first spectrum image; according to the shooting angle theta ₁ Corresponding position transformation matrix F ₁ Determining the photographing angle theta of the color camera ₁ The corresponding position … … of the target position in the captured color image in the first spectral image can determine a plurality of corresponding positions corresponding to the respective capturing angles in the manner described above. In an example, the target position in the color image may be a center position of the color image captured by the color camera. The present disclosure is not limited to the selection of the target location.

In one possible implementation, the corresponding position having the smallest distance from the first position, i.e., the second position, may be determined among the plurality of corresponding positions. For example, the euclidean distance between a first location and each corresponding location may be determined, and a second location having the smallest euclidean distance from the first location may be determined. In an example, the second position is one of a plurality of corresponding positions, e.g., with respect to the photographing angle θ _k (k is a positive integer) corresponding to the second position, and the second position corresponds to the shooting angle theta _k Namely the target shooting angle.

In one possible implementation manner, in step S14, in the determination of the target shootingAfter the angle, the target shooting angle theta can be shot by the color camera _k And shooting is carried out. In an example, the second position is a corresponding position of a center point of an image captured by the color camera, and a distance between the second position and the first position is the smallest, that is, a distance between the center point of the image captured by the color camera and the first position is the smallest, so that a target object at the first position may be present near the center position of the first color image when the color camera captures the image at the target capturing angle.

In one possible implementation manner, in step S15, the target object is presented in the first color image, and the first color image may be subjected to recognition processing to obtain a recognition result. For example, the target object is a person, and the person may be identified to obtain the identity information of the target object, or the target object is an object, and the object may be identified to determine the classification information of the object, which is not limited by the present disclosure.

In one possible implementation, step S15 may include: determining a target region of the target object in the first color image; and identifying the target object in the target area to obtain an identification result.

In one possible implementation, a target area of the target object in the first color image may be determined, and in an example, the first color image may be subjected to a detection process to determine the target area where the target object is located, for example, the detection process may be performed by a neural network or the like to determine the target area where the target object is located.

In an example, the target region in the first color image may also be determined from the first location and the transformation matrix. For example, the first color image is a color camera at a target shooting angle θ _k The color image to be photographed has an angle theta with respect to the target _k Corresponding position transformation matrix is F _k Can pass through F _k And carrying out transformation processing on the first position to obtain the position of the target area in the first color image.

In one possible implementation, the target object in the target area may be identified. For example, the recognition processing may be performed by a neural network or the like to obtain a recognition result.

Fig. 2 shows an application diagram of a target recognition method according to an embodiment of the present disclosure. As shown in fig. 2, the spectral camera with a large field of view may be disposed on an unmanned aerial vehicle, a satellite, or other aircraft, and may obtain a first spectral image of a large area. The field of view of the color camera with a small field of view is smaller than that of the spectrum camera, for example, the area shot by the color camera is only a part of the area shot by the spectrum camera, but the image shot by the color camera has high resolution and high definition.

In a possible implementation manner, the first position of the target object may be determined according to the spectral information of each pixel point in the spectral image. Then, a photographing angle of the color camera may be determined according to the first position and a transformation matrix between the spectral camera and the color camera so that the target object may be presented in a field of view of the color camera. For example, according to the transformation matrix, the corresponding position of the center position of the color image shot by the color camera under a plurality of shooting angles in the spectral image can be determined, and the second position closest to the first position in the corresponding positions can be determined, and the shooting angle corresponding to the second position is the target shooting angle.

In one possible implementation, a color image may be captured by a color camera at a target capture angle, and the target object may be presented in the color image. Further, the first position may be transformed based on a position transformation matrix corresponding to the target shooting angle, a target area where the target object is located in the color image may be determined, and the target area may be subjected to recognition processing, so as to obtain a recognition result, for example, the target object is a person, and the identity information of the person may be recognized.

In one possible implementation, the target recognition method may be used in the field of target recognition, for example, in combination with a color camera in a narrow field of view push-broom type imaging spectrometer to recognize a target object. The carrier carrying the spectral camera and the color camera may also be a translation stage, a handheld device, or the like. The present disclosure does not limit the application field of the target identification method.

Fig. 3 shows a block diagram of an object recognition apparatus according to an embodiment of the present disclosure, as shown in fig. 3, the apparatus including: a spectral image module 11, configured to obtain a first spectral image of the target area through a spectral camera; the detection module 12 is configured to perform detection processing on the spectral image to obtain a first position of a target object in the spectral image; an angle module 13, configured to determine a target shooting angle of the color camera according to the first position, where the shooting angle is a shooting angle of the target object in a field of view of the color camera, and the field of view of the spectrum camera is larger than the field of view of the color camera; a color image module 14, configured to perform shooting by the color camera according to the target shooting angle, and obtain a first color image of the target object; and the identification module 15 is configured to perform identification processing on the target object in the first color image to obtain an identification result.

In one possible implementation, the angle module is further configured to: determining the corresponding positions of the target positions in the first spectrum images in the color images shot by the color camera at a plurality of shooting angles according to the transformation matrix; determining a second position having a smallest distance from the first position among the corresponding positions; and determining the shooting angle corresponding to the second position as the target shooting angle.

It is understood that the above-mentioned method embodiments of the present disclosure can be combined with each other to form a combined embodiment without departing from the logic of the principle, which is limited by the space, and the detailed description of the present disclosure is omitted.

In addition, the present disclosure also provides a target identification device, an electronic device, a computer-readable storage medium, and a program, which can be used to implement any one of the target identification methods provided by the present disclosure, and the corresponding technical solutions and descriptions and corresponding descriptions in the methods section are not repeated.

It will be understood by those of skill in the art that in the above method of the present embodiment, the order of writing the steps does not imply a strict order of execution and does not impose any limitations on the implementation, as the order of execution of the steps should be determined by their function and possibly inherent logic.

In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and for specific implementation, reference may be made to the description of the above method embodiments, and for brevity, details are not described here again

Embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the above-mentioned method. The computer readable storage medium may be a non-volatile computer readable storage medium.

An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured as the above method.

The electronic device may be provided as a terminal, server, or other form of device.

Fig. 4 is a block diagram illustrating an electronic device 800 in accordance with an example embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like terminal.

Referring to fig. 4, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile and non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 800 is in an operation mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the electronic device 800. For example, the sensor assembly 814 may detect an open/closed state of the electronic device 800, the relative positioning of components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, orientation or acceleration/deceleration of the electronic device 800, and a change in temperature of the electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object in the absence of any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 804, is also provided that includes computer program instructions executable by the processor 820 of the electronic device 800 to perform the above-described methods.

Fig. 5 is a block diagram illustrating an electronic device 1900 according to an example embodiment. For example, the electronic device 1900 may be provided as a server. Referring to fig. 5, electronic device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the above-described method.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system, such as the Microsoft Server operating system (Windows Server), stored in the memory 1932 ^TM ) Apple Inc. of the present application based on the graphic user interface operating System (Mac OS X) ^TM ) Multi-user, multi-process computer operating system (Unix) ^TM ) Free and open native code Unix-like operating System (Linux) ^TM ) Open native code Unix-like operating System (FreeBSD) ^TM ) Or the like.

In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932, is also provided that includes computer program instructions executable by the processing component 1922 of the electronic device 1900 to perform the above-described methods.

The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as punch cards or in-groove raised structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein were chosen in order to best explain the principles of the embodiments, the practical application, or technical improvements to the techniques in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method of object recognition, comprising:

obtaining a first spectral image of a target area by a spectral camera;

detecting the spectral image to obtain a first position of a target object in the spectral image;

determining a target shooting angle of the color camera according to the first position, wherein the shooting angle is a shooting angle of the target object in a field of view of the color camera, and the field of view of the spectrum camera is larger than that of the color camera;

shooting by the color camera according to the target shooting angle to obtain a first color image of the target object;

and identifying the target object in the first color image to obtain an identification result.

2. The method of claim 1, further comprising:

calibrating the spectrum camera and the color camera to obtain a first internal reference matrix of the spectrum camera and a second internal reference matrix of the color camera;

determining a transformation matrix between the spectral camera and the color camera from the first and second internal reference matrices.

3. The method of claim 2, wherein the transformation matrix between the spectral camera and the color camera comprises a position transformation matrix between second position information of a target position in a second color image captured by the color camera at a plurality of capture angles and corresponding third position information in the second spectral image captured by the spectral camera,

determining a transformation matrix between the spectral camera and the color camera from the first and second internal reference matrices, including:

determining a translation vector and a rotation matrix according to second position information in a second color image shot by a color camera at a first shooting angle, corresponding third position information in the second spectrum image and the first internal reference matrix, wherein the first shooting angle is any shooting angle of the color camera;

and determining a position transformation matrix corresponding to the first shooting angle according to the second position information, the third position information, the translation vector and the rotation matrix.

4. The method of claim 2, wherein determining the target capture angle of the color camera from the first position comprises:

determining corresponding positions of target positions in the first spectrum images in color images shot by the color camera at a plurality of shooting angles according to the transformation matrix;

determining a second position having a smallest distance from the first position among the corresponding positions;

and determining the shooting angle corresponding to the second position as the target shooting angle.

5. The method according to claim 3 or 4, wherein the target position is a center position of a color image taken by the color camera.

6. The method according to claim 2, wherein performing recognition processing on the target object in the first color image to obtain a recognition result comprises:

determining a target region of the target object in the first color image;

and identifying the target object in the target area to obtain an identification result.

7. The method of claim 6, wherein determining a target region of the target object in the first color image comprises:

and determining a target area of the target object in the first color image according to the transformation matrix and the position information of the first position.

8. An object recognition apparatus, comprising:

the spectral image module is used for obtaining a first spectral image of the target area through the spectral camera;

the detection module is used for detecting the spectral image to obtain a first position of a target object in the spectral image;

an angle module, configured to determine a target shooting angle of the color camera according to the first position, where the shooting angle is a shooting angle of the target object in a field of view of the color camera, and the field of view of the spectrum camera is larger than the field of view of the color camera;

the color image module is used for shooting according to the target shooting angle through the color camera to obtain a first color image of the target object;

and the identification module is used for identifying the target object in the first color image to obtain an identification result.

9. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to: performing the method of any one of claims 1 to 7.

10. A computer readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the method of any one of claims 1 to 7.