CN109271921B - Intelligent identification method and system for multispectral imaging - Google Patents
Intelligent identification method and system for multispectral imaging Download PDFInfo
- Publication number
- CN109271921B CN109271921B CN201811064057.2A CN201811064057A CN109271921B CN 109271921 B CN109271921 B CN 109271921B CN 201811064057 A CN201811064057 A CN 201811064057A CN 109271921 B CN109271921 B CN 109271921B
- Authority
- CN
- China
- Prior art keywords
- image
- imaging
- target object
- spectrum
- spectral
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
- G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06V10/143—Sensing or illuminating at different wavelengths
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/194—Terrestrial scenes using hyperspectral data, i.e. more or other wavelengths than RGB
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Image Analysis (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The application discloses an intelligent identification method and system of multispectral imaging, wherein the method comprises the following steps: presetting imaging response spectrums with more than or equal to two different wavelengths in imaging equipment, starting the imaging equipment to shoot a target imaging area, and acquiring spectrum images of a plurality of spectrum intervals of the target imaging area; and the spectral band images are segmented by combining preset spectral feature information of a target object by using a multispectral imaging fusion algorithm model, the spectral band images of a plurality of spectral regions are processed by using artificial intelligence or a machine learning algorithm, feature enhancement and feature fusion of the spectral band images of the plurality of spectral regions are realized, and an image consistent with the information in a discrimination strategy is selected from the fused image as an identification image through a preset discrimination strategy. The invention obtains more information by using the multispectral image, thereby improving the efficiency and the accuracy of target identification.
Description
Technical Field
The invention relates to the technical field of intelligent identification, in particular to an intelligent identification method and system of multispectral imaging.
Background
With the rapid development of social economy, the living standard and safety consciousness of people are continuously enhanced, the requirements for security monitoring and intelligent life style are increased day by day, human targets of specific occasions are identified in various interactive scenes, the flow of people in a specific time period is accurately counted, and the intelligent management and analysis of the behaviors of individuals or crowds are very necessary. Therefore, the intelligent visual system capable of automatically identifying the target (especially the human target) has wide application prospect and practical value, and becomes a challenging research hotspot. The intelligent vision system aims to identify and judge target behaviors according to imaging records of specific occasions and specific time periods, effectively count the flow of a target, and screen out abnormal behaviors of individual individuals, so that efficient and timely monitoring information is provided. However, the all-weather distributed intelligent recognition system faces different illumination conditions and place backgrounds, so the imaging quality of the system is a key factor influencing the recognition function of the intelligent vision system.
The imaging result of the common digital imaging technology is divided into panchromatic and color, wherein the panchromatic image is a single-channel image, the panchromatic image refers to the whole visible light wave band of 0.38-0.76 um, the panchromatic image is a mixed black-and-white image in the wave band range, and the color is an RGB image which is daily seen by people. The two processes the image information of the visible light wave band differently, but the three-dimensional image information is projected into a two-dimensional image, the original image information of each spectral band is not reserved, and a part of image information is lost. In addition, the spectral interval of the two imaging only covers the visible light interval, and the invisible light interval is ignored, which also causes the loss of image information. When the target detection is carried out under different light conditions, the information content contained in the image information of different spectral bands is different for different identification tasks, for example, at dark night, the image information of an infrared interval is very critical to accurately judging the position and the behavior of a human body. The spectral interval of full-color or color image imaging is the visible light interval, and the images of a plurality of spectral bands are not processed in a targeted manner according to different recognition tasks, so that the information content in the images is insufficient, and the intelligent recognition task of various interactive scenes cannot be met.
Therefore, it is an urgent technical problem to be solved in the art to provide an efficient, accurate and clear intelligent identification method and system.
Disclosure of Invention
In view of the above, the invention provides an intelligent identification method and system of multispectral imaging, which solves the problem that in the prior art, in the intelligent identification process, a target object is fused with a background and is difficult to distinguish; when object occlusion occurs, target object information is lost, so that a real boundary cannot be acquired; when multiple objects appear, the target object and other objects cannot be distinguished, and the technical problem of difficulty in identification is caused.
In order to solve the technical problem, the invention provides an intelligent identification method of multispectral imaging, which comprises the following steps:
presetting more than or equal to two imaging spectrums and a comparison table of environmental conditions and real-time imaging spectrums in more than or equal to one imaging device, and when the imaging device is started, selecting corresponding real-time imaging spectrums from the comparison table according to the environmental conditions monitored by the imaging device in real time and respectively calling the real-time imaging spectrums;
when the imaging device is started to shoot a target imaging area, acquiring spectrum images of a plurality of spectrum intervals of the target imaging area;
the spectral band images are segmented by combining preset spectral feature information of a target object through a multispectral imaging fusion algorithm model, and the spectral band images in a plurality of spectral regions are processed through a machine learning algorithm, so that feature enhancement and feature fusion of the spectral band images in the plurality of spectral regions are realized;
and selecting an image consistent with the information in the discrimination strategy from the fused image as an identification image through a preset discrimination strategy.
Optionally, the method includes segmenting a spectrum image by using a multispectral imaging fusion algorithm model in combination with preset spectral feature information of a target object, and processing a plurality of spectrum interval spectrum images by using a machine learning algorithm to realize feature enhancement and feature fusion of the plurality of spectrum interval spectrum images, and includes:
respectively fusing the spectrum images to obtain preprocessed images, acquiring target object preprocessed images in the preprocessed images according to preset target object characteristic information, calculating the image difference degree of the target object preprocessed images and other objects, sorting the images from large to small, and selecting one preprocessed image according to the sorting to perform network training to obtain an accurate target object image; and combining the position information of the target object in the spectral band image to fuse the target object image to the spectral band image to obtain a fused image.
Optionally, the step of fusing the target object image to the spectral band image in combination with the position information of the target object in the spectral band image to obtain a fused image includes:
and adopting an average gray level judgment strategy to draw the gray level distribution histograms of all the spectral band images, calculating the gray level average value of each spectral band image, selecting the spectral band image with the gray level average value consistent with a preset gray level average value judgment strategy in the average gray level judgment strategy as a background image, and combining the position information of the target object in the spectral band image to fuse the target object image into the background image to obtain a fused image.
Optionally, wherein the method further comprises:
in the imaging device, a corresponding relation between an object and an imaging spectrum is preset, when the target object is selected, the imaging spectrum of the target object is obtained by comparing the preset object with the corresponding relation between the imaging spectrum, and the target imaging area is shot by using the imaging spectrum of the target object.
Optionally wherein the wavelength range of the imaging spectrum is selected from ultraviolet, visible, near infrared, mid infrared, far infrared.
In another aspect, the present invention further provides an intelligent identification system for multispectral imaging, including: starting a setter, a shooting device, an image processor and an image discriminator; wherein the content of the first and second substances,
the starting setter is connected with the shooting device and used for presetting more than or equal to two imaging spectrums and a comparison table of environmental conditions and real-time imaging spectrums in more than or equal to one imaging device, and when the imaging device is started, corresponding real-time imaging spectrums are selected from the comparison table according to the environmental conditions monitored by the imaging device in real time, and the real-time imaging spectrums are respectively called;
the shooting device is connected with the starting setting device and the image processor and is used for starting the imaging equipment to shoot a target imaging area so as to obtain spectrum images of a plurality of spectrum intervals of the target imaging area;
the image processor is connected with the shooting device and the image discriminator and is used for combining preset spectral feature information of a target object, segmenting a spectral band image by utilizing a multispectral imaging fusion algorithm model, and processing a plurality of spectral band images in spectral intervals by utilizing a machine learning algorithm to realize feature enhancement and feature fusion of the spectral band images in the spectral intervals;
and the image discriminator is connected with the image processor and used for selecting an image which is consistent with information in the discrimination strategy from the fusion image as an identification image through a preset discrimination strategy.
Optionally, wherein the image processor is:
respectively fusing the spectrum images to obtain preprocessed images, acquiring target object preprocessed images in the preprocessed images according to preset target object characteristic information, calculating the image difference degree of the target object preprocessed images and other objects, sorting the images from large to small, and selecting one preprocessed image according to the sorting to perform network training to obtain an accurate target object image; and the image processor is used for combining the position information of the target object in the spectral band image to fuse the target object image into the spectral band image to obtain a fused image.
Optionally, wherein the image processor comprises: a target object image acquisition unit and an image fusion unit; wherein the content of the first and second substances,
the target object image acquisition unit is connected with the shooting device and used for carrying out network training on the spectral band image to obtain an accurate target object image after the spectral band image is segmented by combining preset target object characteristic information and utilizing a multispectral imaging combining algorithm model;
the image fusion unit is connected with the target object image acquisition unit and the image discriminator and is used for adopting an average gray scale judgment strategy, drawing a gray scale distribution histogram of all the spectral band images, calculating a gray scale average value of each spectral band image, selecting the spectral band image with the gray scale average value consistent with a preset gray scale average value judgment strategy in the average gray scale judgment strategy as a background image, and fusing the target object image into the background image by combining position information of the target object in the spectral band image to obtain a fused image.
Optionally, wherein the system further comprises: and the imaging spectrum selector is connected with the starting setter and the shooting device and is used for presetting the corresponding relation between the object and the imaging spectrum in the imaging equipment, comparing the preset object with the corresponding relation between the imaging spectrum to obtain the imaging spectrum of the object when the target object is selected, and shooting the target imaging area by using the imaging spectrum of the target object.
Optionally wherein the wavelength range of the imaging spectrum is selected from ultraviolet, visible, near infrared, mid infrared, far infrared.
Compared with the prior art, the intelligent identification method and system for multispectral imaging provided by the invention at least realize one of the following beneficial effects:
(1) the intelligent identification method and the intelligent identification system for the multispectral imaging utilize the multispectral imaging to acquire the image information of the target object in a plurality of spectral intervals, and greatly improve the information content of the acquired image. Meanwhile, a convolutional neural network is selected and established according to the spectral band image, so that the problems of object shielding, fusion of a target object and a background, strong light reflection and the like are solved, and the target identification efficiency is improved. The intelligent identification method has the advantages of wide application range, strong popularization and mobility and suitability for various identification algorithms.
(2) The intelligent identification method and the intelligent identification system for multispectral imaging, disclosed by the invention, have the advantages that the trained network and multispectral imaging equipment are packaged, a full-automatic human body intelligent identification system is constructed, the packaged identification system is convenient to operate, the network can be regularly and automatically upgraded along with the increase of the number of recorded images, and in practical application, the human resources can be greatly saved.
(3) According to the multispectral imaging intelligent identification method and system, multispectral imaging is combined with image processing to obtain an obvious image of a target object image, network training is carried out on the image, judgment is carried out through a preset judgment strategy, identification accuracy is greatly improved, and target object identification accuracy is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention. Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
fig. 1 is a schematic flow chart of an intelligent identification method of multispectral imaging according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a multispectral image recognition process in the intelligent multispectral imaging recognition method according to the embodiment of the present invention;
fig. 3 is a schematic flowchart of a second intelligent identification method of multispectral imaging according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a multispectral image identification process in another intelligent multispectral imaging identification method according to an embodiment of the present invention;
fig. 5 is a schematic flowchart of a third intelligent identification method of multispectral imaging according to an embodiment of the present invention;
FIG. 6 is a histogram of gray scale distribution of an image of different spectral bands according to an embodiment of the present invention;
FIG. 7 is a fused image gray distribution histogram according to an embodiment of the present invention;
FIG. 8 is a schematic flow chart of another intelligent identification method of multispectral imaging according to an embodiment of the present invention;
FIG. 9 is a schematic structural diagram of an intelligent identification system for multi-spectral imaging according to an embodiment of the present invention;
FIG. 10 is a schematic structural diagram of a second intelligent identification system for multispectral imaging according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of a third intelligent identification system for multispectral imaging according to an embodiment of the present invention;
FIG. 12 is a schematic flow chart of steps of applying multispectral imaging intelligent recognition to scenes such as off-track, unmanned driving, venue people flow statistics, and the like.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant application and are not limiting of the application. It should be noted that, for convenience of description, only the portions related to the present application are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
As shown in fig. 1, fig. 1 is a schematic flow diagram of an intelligent multispectral imaging identification method according to this embodiment, in this embodiment, multispectral imaging technology is introduced into a conventional imaging system for multispectral imaging intelligent identification, and a fully automatic intelligent human body identification system is built by relying on an image network training concept. The intelligent identification method for multispectral imaging in the embodiment comprises the following steps:
step 101, presetting more than or equal to two imaging spectrums and comparison tables of environmental conditions and real-time imaging spectrums in more than or equal to one imaging device, when the imaging device is started, selecting corresponding real-time imaging spectrums from the comparison tables according to the environmental conditions monitored by the imaging device in real time, and respectively calling the real-time imaging spectrums.
Under different environmental conditions, the image effect obtained by shooting a target by using a spectrum is different, and the environmental conditions comprise: under the conditions of light intensity, sunny days, rainy days, snowy days, foggy days and the like, different imaging spectrums are selected under different environmental conditions to obtain images with different effects, and the optimal imaging spectrums under each environmental condition are corresponding to the environmental conditions by presetting a comparison table of the environmental conditions and the real-time imaging spectrums, so that the imaging shooting efficiency can be improved.
Optionally, the wavelength range of the imaging spectrum is selected from the ultraviolet spectrum, the visible spectrum, the near infrared spectrum, the mid infrared spectrum, and the far infrared spectrum.
And more image information quantity is acquired by means of a multispectral imaging technology. For a certain shooting scene, multispectral photography is expanded towards three directions of near infrared light, infrared light and ultraviolet light on the basis of visible light, and information radiated or reflected by the same target on different narrow spectral bands is simultaneously and respectively received through combination of various optical filters or optical splitters and various photosensitive films, so that a series of photos of different spectral bands of the target are obtained, and an image cube is obtained.
And 103, segmenting the spectrum image by combining preset spectral feature information of the target object by using a multispectral imaging fusion algorithm model, and processing the spectrum images in the multiple spectral intervals by using a machine learning algorithm to realize feature enhancement and feature fusion of the spectrum images in the multiple spectral intervals.
Different objects have different oscillograms in different spectral bands, under different illumination conditions, a specific spectral band or a plurality of spectral bands can be found, the identifiability of a human body is high, the difference between the human body and other objects or backgrounds is very obvious, and the difference is very weak in a visible light region. Therefore, in the construction of an intelligent human body recognition system, the multispectral imaging technology can greatly increase the image information amount, and has great effect on improving the recognition precision and the recognition efficiency.
And constructing the target object intelligent identification system by utilizing a network training model, such as a convolutional neural network. Convolutional neural networks are used primarily to identify two-dimensional patterns of displacement, scaling and other forms of distortion invariance. Compared with the traditional method and the common neural network, the convolutional neural network has unique superiority in the aspects of voice recognition and image processing by using a special structure shared by local weights, the layout of the convolutional neural network is closer to the actual biological neural network, the complexity of the network is reduced by sharing the weights, and particularly, the complexity of data reconstruction in the processes of feature extraction and classification is avoided by the characteristic that the image of a multi-dimensional input vector can be directly input into the network. Therefore, in the intelligent human body recognition, the network training is performed by inputting the image information through the idea of the convolutional neural network, and the recognition image with high recognition accuracy is obtained.
And packaging the trained network and multispectral imaging equipment to construct a full-automatic human body intelligent recognition system. The packaged recognition system is convenient to operate, the network can be automatically upgraded regularly along with the increase of the number of recorded images, and in practical application, human resources can be greatly saved.
Optionally, all spectral band images captured by the multi-spectral imaging device are used as network input, after network training, the recognition results of all spectral band images are fused, and a more convincing recognition result is selected through voting or other discrimination strategies, so that the human body recognition accuracy can be improved. The multispectral image identification process in the multispectral imaging intelligent identification method is shown in fig. 2.
And 104, selecting an image consistent with information in a preset discrimination strategy from the fused image as an identification image.
In some optional embodiments, as shown in fig. 3, which is a schematic flowchart of the second multispectral imaging intelligent identification method in this embodiment, different from the method in fig. 1, step 103 is:
respectively fusing the spectrum images to obtain preprocessed images, acquiring target object preprocessed images in the preprocessed images according to preset target object characteristic information, calculating the image difference degree of the target object preprocessed images and other objects, sorting the images from large to small, and selecting one preprocessed image according to the sorting to perform network training to obtain an accurate target object image; and combining the position information of the target object in the spectral band image to fuse the target object image into the spectral band image to obtain a fused image. The multispectral image identification process in the multispectral imaging intelligent identification method is shown in fig. 4.
Due to the fact that imaging scenes are different and illumination angles are different, the difference between the head of a human body and surrounding objects is obvious in certain spectral bands, the difference between four limbs of the human body and the surrounding objects is obvious in certain spectral bands, when image data are used, images of a plurality of spectral bands can be fused to obtain a fused image with the obvious difference between the human body and other objects, and then the image is used as input to conduct network training, so that the problem of information loss caused by a single image in a visible light area is solved, and the accuracy of human body recognition is improved.
In some optional embodiments, as shown in fig. 5, which is a flowchart illustrating a third intelligent identification method for multispectral imaging in this embodiment, different from the method in fig. 1, the method for fusing the target object image to the spectral fragment image to obtain a fused image in combination with the position information of the target object in the spectral fragment image includes the following steps:
And 503, combining the position information of the target object in the spectral band image to fuse the target object image to the background image to obtain a fused image.
As shown in fig. 6 and fig. 7, fig. 6 is a histogram of the gray distribution of the image in different spectral bands according to the present embodiment; fig. 7 is a histogram of the fused image gray distribution according to the present embodiment.
In some optional embodiments, as shown in fig. 8, a flowchart of another multispectral imaging intelligent identification method described in this embodiment is different from the method in fig. 1, and further includes the following steps:
And step 802, when a target object is selected, comparing the target object with a corresponding relation between a preset object and an imaging spectrum to obtain an imaging spectrum of the target object, and shooting a target imaging area by using the imaging spectrum of the target object.
In some optional embodiments, as shown in fig. 9, a schematic structural diagram of an intelligent recognition system for multispectral imaging is described in this embodiment. The system is used for implementing the above-mentioned intelligent identification method of multispectral imaging, and the system 900 includes: a setter 901, a camera 902, an image processor 903 and an image discriminator 904 are started; wherein the content of the first and second substances,
and a starting setter 901 connected with the camera 902 and used for presetting more than or equal to two imaging spectrums and a comparison table of environmental conditions and real-time imaging spectrums in more than or equal to one imaging device, and when the imaging device is started, selecting corresponding real-time imaging spectrums from the comparison table according to the environmental conditions monitored by the imaging device in real time, and respectively calling the real-time imaging spectrums.
And the shooting device 902 is connected with the start setting device 901 and the image processor 903, and is used for starting the imaging device to shoot the target imaging area, so as to acquire spectrum images of a plurality of spectrum intervals of the target imaging area.
And the image processor 903 is connected with the camera 902 and the image discriminator 904, and is configured to segment the spectral band image by using a multispectral imaging fusion algorithm model in combination with preset spectral feature information of the target object, and process the spectral band images in the multiple spectral regions by using a machine learning algorithm, so as to implement feature enhancement and feature fusion on the spectral band images in the multiple spectral regions.
In some alternative embodiments, the image processor 903 may be: respectively fusing the spectrum images to obtain preprocessed images, acquiring target object preprocessed images in the preprocessed images according to preset target object characteristic information, calculating the image difference degree of the target object preprocessed images and other objects, sequencing the images from large to small, and selecting one preprocessed image according to the sequence (namely selecting the preprocessed image with the largest image difference degree) to perform network training to obtain an accurate target object image; and the image processor is used for fusing the target object image into the spectral band image by combining the position information of the target object in the spectral band image to obtain a fused image.
And the image discriminator 904 is connected with the image processor 903 and is used for selecting an image which is consistent with information in the judgment strategy from the fused image through a preset judgment strategy as an identification image.
In some alternative embodiments, as shown in fig. 10, a schematic structural diagram of the second multispectral imaging smart identification system 1000 described in this embodiment is shown. Unlike the system of fig. 9, the image processor 903 includes: a target object image acquisition unit 931 and an image fusion unit 932; wherein the content of the first and second substances,
the target object image obtaining unit 931 is connected to the camera 902, and is configured to perform network training on a spectral band image after segmenting the spectral band image by using a multispectral imaging and algorithm model in combination with preset target object feature information to obtain an accurate target object image.
The image fusion unit 932 is connected to the target object image obtaining unit 931 and the image discriminator 904, and configured to adopt an average gray level determination policy, draw a gray level distribution histogram of all spectral band images, calculate a gray level average value of each spectral band image, select a spectral band image with a gray level average value consistent with a preset gray level average value determination policy in the average gray level determination policy as a background image, and fuse the target object image to the background image according to position information of the target object in the spectral band image to obtain a fusion image.
In some alternative embodiments, as shown in fig. 11, a schematic structural diagram of an intelligent recognition system 1100 for the third multispectral imaging described in this embodiment is shown. Unlike the system of fig. 9, the system further includes: the imaging spectrum selector 905 is connected to the start setter 901 and the camera 902, and is configured to preset a correspondence between an object and an imaging spectrum in the imaging device, compare the preset object with the correspondence between the imaging spectrum when a target object is selected to obtain an imaging spectrum of the target object, and shoot a target imaging area by using the imaging spectrum of the target object.
Optionally, in the above system, the wavelength range of the imaging spectrum is selected from ultraviolet, visible, near infrared, mid infrared, and far infrared spectra.
The intelligent identification system of multispectral imaging in this embodiment can be applied to a human body intelligent identification system in a traffic junction scene, in the traffic junction, pedestrian crossing or lying rail will seriously affect the normal operation of a train to cause casualty events, as shown in fig. 12, the step flow schematic diagram of the intelligent identification of multispectral imaging in the invention is applied to scenes such as crossing rail, unmanned driving, venue pedestrian flow statistics and the like, and the step flow comprises the following steps:
The usage scenario may include: the method comprises the following scenes of rail running, vehicle driving, set area or venue people flow statistics and the like.
And 1202, presetting a comparison table of the environment conditions of the using scene and the real-time imaging spectrum in the imaging device.
And 1205, segmenting the spectrum image by using a multispectral imaging fusion algorithm model in combination with preset spectral feature information of the target object, and processing the spectrum images in the multiple spectral intervals by using a machine learning algorithm to realize feature enhancement and feature fusion of the spectrum images in the multiple spectral intervals.
And 1206, selecting an image consistent with information in the judgment strategy from the fused image as an identification image through a preset judgment strategy.
Optionally, the method may further include: comparing the obtained recognition image with a behavior early warning image preset in the system;
when the identification image is consistent with a certain behavior early warning image, starting an early warning processing instruction corresponding to the behavior early warning image;
and the system controls the corresponding device to process the early warning event according to the early warning processing instruction.
However, in practice, the laying environments of the tracks are different, and the traditional imaging technology cannot accurately identify the human bodies with different behaviors and actions at night or when objects are shielded, for example, the human body bending over to advance and the kangaroo can be identified into the same object at certain angles, so that the identification is inaccurate, and potential safety hazards are caused. The multispectral human body recognition equipment provided by the system records image information in a plurality of spectral bands, and selects or fuses the plurality of spectral band images by utilizing the characteristic that spectral distributions of different kinds of objects in multispectral imaging are different, so that a training image data set which is more beneficial to subsequent human body recognition is selected.
In the embodiment, the multispectral imager is used in the imaging stage, and the multispectral imager can capture more image information under the same imaging time and imaging conditions. In this example, even in a dark environment, the spectral band image suitable for identification can be selected from the images of multiple bands by using the multi-spectral band image information in combination with the average grayscale fusion strategy, and in practical applications, the specific selection strategy is determined according to the environment, such as the maximum grayscale selection strategy, the grayscale principal component fusion strategy, and the like.
The multispectral imaging intelligent identification system can be applied to unmanned vehicles, and the unmanned vehicles are important fields influencing future travel modes of human beings along with the continuous development of artificial intelligence technology. The most key problem in the unmanned automobile technology is how to ensure safe driving, the prior art is that sensors are basically arranged at the top, the bottom and the periphery of an automobile, the automobile is prompted to avoid pedestrians through images shot by the sensors and sensed information, and the imaging technology of the traditional sensor is imaging in a visible light area, so that a part of important information is lost.
Therefore, the human body identification intelligent system provided by the embodiment is introduced into the unmanned automobile, and more information is acquired in the data part according to the multi-spectral-band image information recorded by the hyperspectral imaging equipment, so that the subsequent human body identification part can be more efficient and accurate. Therefore, even under the condition of weak light or under the condition that pedestrians suddenly appear when objects are sheltered, the unmanned automobile can safely avoid the pedestrians, and safe driving is realized.
The multispectral imaging intelligent identification system can also perform venue people flow statistics, and when a large conference with strictly limited number of people is held, ticket checking personnel need to pay attention to people flow entering and leaving all the time, and manpower is consumed. Due to the fact that people flow and objects are carried by people, the traditional imaging technology has the defects that people in the scenes cannot be identified, and the human body identification is difficult to achieve accurately. Under this kind of scene, the human body identification intelligent system that this patent provided of application, because many map imaging device can take notes the image information of a plurality of spectral bands, then can easily discern the difference of people with the thing, the crowd overlaps the problem that gets into, accomplishes accurate human body identification, reaches the purpose to people flow statistics, and after people flow reached appointed quantity, then can self-closing venue, use manpower sparingly.
Because the imaging wave band of many map imager can be selected, can appoint the intensive degree of imaging wave band, like high spectrum imager, different wave band formation of image is only to this patent and proposes the extension of many map imaging concept, also belongs to the protective scope of this patent. The convolutional neural network in this embodiment is only a simple example, and other convolutional neural networks are all within the scope of this patent. The convolutional neural network algorithm can be transformed into other algorithms, and the method belongs to the scope of the embodiment as long as the method belongs to multispectral imaging and combines an algorithm model to perform image segmentation.
According to the embodiment, the intelligent identification method and the intelligent identification system for multispectral imaging have the following beneficial effects that:
(1) the intelligent identification method and the intelligent identification system for the multispectral imaging utilize the multispectral imaging to acquire the image information of the target object in a plurality of spectral intervals, and greatly improve the information content of the acquired image. Meanwhile, a convolutional neural network is selected and established according to the spectral band image, so that the problems of object shielding, fusion of a target object and a background, strong light reflection and the like are solved, and the target identification efficiency is improved. The intelligent identification method has the advantages of wide application range, strong popularization and mobility and suitability for various identification algorithms.
(2) The intelligent identification method and the intelligent identification system for multispectral imaging, disclosed by the invention, have the advantages that the trained network and multispectral imaging equipment are packaged, a full-automatic human body intelligent identification system is constructed, the packaged identification system is convenient to operate, the network can be regularly and automatically upgraded along with the increase of the number of recorded images, and in practical application, the human resources can be greatly saved.
(3) According to the multispectral imaging intelligent identification method and system, multispectral imaging is combined with image processing to obtain an obvious image of a target object image, network training is carried out on the image, judgment is carried out through a preset judgment strategy, identification accuracy is greatly improved, and target object identification accuracy is improved.
Although some specific embodiments of the present invention have been described in detail by way of examples, it should be understood by those skilled in the art that the above examples are for illustrative purposes only and are not intended to limit the scope of the present invention. It will be appreciated by those skilled in the art that modifications may be made to the above embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.
Claims (10)
1. An intelligent identification method of multispectral imaging is characterized by comprising the following steps:
presetting a comparison table of environmental conditions and imaging spectrums and two or more imaging spectrums in one or more imaging devices, when the imaging devices are started, selecting corresponding imaging spectrums from the comparison table according to the environmental conditions monitored by the imaging devices in real time, and respectively calling the corresponding imaging spectrums, wherein each environmental condition corresponds to at least two imaging spectrums in the comparison table of the environmental conditions and the imaging spectrums;
enabling the imaging device to shoot a target imaging area, so as to obtain spectrum images of a plurality of spectrum intervals of the target imaging area;
the spectral band images are segmented by combining preset spectral feature information of a target object through a multispectral imaging fusion algorithm model, the segmented spectral band images in a plurality of spectral regions are processed through a machine learning algorithm, feature enhancement and feature fusion of the segmented spectral band images in the plurality of spectral regions are achieved, and a fusion image is generated;
and selecting an image consistent with the information in the discrimination strategy from the fused image as an identification image through a preset discrimination strategy.
2. The intelligent multispectral imaging identification method according to claim 1, wherein the multispectral imaging fusion algorithm model is used to segment the spectral band images in combination with the spectral feature information of the preset target object, and the machine learning algorithm is used to process the segmented spectral band images in the plurality of spectral regions, so as to enhance and fuse the features of the segmented spectral band images in the plurality of spectral regions, and generate a fusion image, which is:
respectively fusing the spectrum images of the plurality of spectrum intervals to obtain a preprocessed image, acquiring a preprocessed image of a target object in the preprocessed image according to preset spectral characteristic information of the target object, calculating the image difference degree of the preprocessed image of the target object and other objects, sorting the preprocessed image of the target object from large to small, and selecting one preprocessed image according to the sorting to perform network training to obtain an accurate target object image; and combining the position information of the target object in the spectrum images of the plurality of spectral intervals to fuse the target object image into the spectrum images of the plurality of spectral intervals to obtain a fused image.
3. The intelligent multispectral imaging identification method according to claim 1, wherein the multispectral imaging fusion algorithm model is used to segment the spectral band images in combination with the spectral feature information of the preset target object, and the machine learning algorithm is used to process the segmented spectral band images in the plurality of spectral regions, so as to enhance and fuse the features of the segmented spectral band images in the plurality of spectral regions, and generate a fusion image, which is:
and adopting an average gray level judgment strategy to draw the gray level distribution histograms of all the spectral band images, calculating the gray level average value of each spectral band image, selecting the spectral band image with the gray level average value consistent with the preset gray level average value in the average gray level judgment strategy as a background image, and combining the position information of the target object in the spectral band image to fuse the target object image into the background image to obtain a fused image.
4. The intelligent identification method of multispectral imaging according to claim 1, further comprising:
in the imaging device, a corresponding relation between an object and an imaging spectrum is preset, when the target object is selected, the imaging spectrum of the target object is obtained through comparison according to the corresponding relation between the preset object and the imaging spectrum and the target object, and the imaging spectrum of the target object is utilized to shoot the target imaging area.
5. The method for intelligent identification of multispectral imaging according to any one of claims 1 to 4, wherein the wavelength range of the imaging spectrum is selected from the group consisting of ultraviolet, visible, near infrared, mid infrared, and far infrared.
6. An intelligent identification system for multispectral imaging, comprising: starting a setter, a shooting device, an image processor and an image discriminator; wherein the content of the first and second substances,
the starting setter is connected with the shooting device and is used for presetting a comparison table of environmental conditions and imaging spectrums and more than or equal to two imaging spectrums in more than or equal to one imaging device, when the imaging device is started, corresponding imaging spectrums are selected from the comparison table according to the environmental conditions monitored by the imaging device in real time, and the corresponding imaging spectrums are respectively called, wherein each environmental condition corresponds to at least two imaging spectrums in the comparison table of the environmental conditions and the imaging spectrums;
the shooting device is connected with the starting setting device and the image processor and used for starting the imaging equipment to shoot a target imaging area so as to obtain spectrum images of a plurality of spectrum intervals of the target imaging area;
the image processor is connected with the shooting device and the image discriminator and is used for combining preset spectral feature information of a target object, segmenting a spectral band image by using a multispectral imaging fusion algorithm model, processing a plurality of segmented spectral band images by using a machine learning algorithm, realizing feature enhancement and feature fusion of the segmented spectral band images and generating a fusion image;
and the image discriminator is connected with the image processor and used for selecting an image which is consistent with information in the discrimination strategy from the fusion image as an identification image through a preset discrimination strategy.
7. The intelligent multispectral imaging identification system as recited in claim 6, wherein the image processor is configured to:
respectively fusing the spectrum images of the plurality of spectrum intervals to obtain a preprocessed image, acquiring a preprocessed image of a target object in the preprocessed image according to preset spectral characteristic information of the target object, calculating the image difference degree of the preprocessed image of the target object and other objects, sorting the preprocessed image of the target object from large to small, and selecting one preprocessed image according to the sorting to perform network training to obtain an accurate target object image; and the image processor is used for combining the position information of the target object in the spectrum images of the plurality of spectrum intervals to fuse the target object image into the spectrum images of the plurality of spectrum intervals to obtain a fused image.
8. The intelligent identification system for multispectral imaging according to claim 6, wherein the image processor comprises: a target object image acquisition unit and an image fusion unit; wherein the content of the first and second substances,
the target object image acquisition unit is connected with the shooting device and used for carrying out network training on the spectral band image to obtain an accurate target object image after the spectral band image is segmented by combining preset target object characteristic information and utilizing a multispectral imaging combining algorithm model;
the image fusion unit is connected with the target object image acquisition unit and the image discriminator and is used for adopting an average gray scale judgment strategy, drawing a gray scale distribution histogram of all the spectral band images, calculating a gray scale average value of each spectral band image, selecting the spectral band image with the gray scale average value consistent with the preset gray scale average value in the average gray scale judgment strategy as a background image, and fusing the target object image to the background image by combining the position information of the target object in the spectral band image to obtain a fused image.
9. The intelligent identification system of multispectral imaging according to claim 6, further comprising: and the imaging spectrum selector is connected with the starting setter and the shooting device and is used for presetting the corresponding relation between an object and an imaging spectrum in the imaging equipment, comparing the preset object and the imaging spectrum with the target object to obtain the imaging spectrum of the target object when the target object is selected, and shooting the target imaging area by using the imaging spectrum of the target object.
10. The intelligent recognition system according to any one of claims 6 to 9, wherein the wavelength range of the imaging spectrum is selected from the group consisting of ultraviolet spectrum, visible spectrum, near infrared spectrum, mid infrared spectrum, and far infrared spectrum.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811064057.2A CN109271921B (en) | 2018-09-12 | 2018-09-12 | Intelligent identification method and system for multispectral imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811064057.2A CN109271921B (en) | 2018-09-12 | 2018-09-12 | Intelligent identification method and system for multispectral imaging |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109271921A CN109271921A (en) | 2019-01-25 |
| CN109271921B true CN109271921B (en) | 2021-01-05 |
Family
ID=65188819
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811064057.2A Active CN109271921B (en) | 2018-09-12 | 2018-09-12 | Intelligent identification method and system for multispectral imaging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109271921B (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109978806A (en) * | 2019-03-22 | 2019-07-05 | 云南谦实科技有限公司 | A kind of method, apparatus and computer media carrying out the inspection of route hidden danger based on multi-spectral imager |
| CN110188606B (en) * | 2019-04-23 | 2023-06-20 | 合刃科技（深圳）有限公司 | Lane recognition method and device based on hyperspectral imaging and electronic equipment |
| CN110441312A (en) * | 2019-07-30 | 2019-11-12 | 上海深视信息科技有限公司 | A kind of surface defects of products detection system based on multispectral imaging |
| CN110765853B (en) * | 2019-09-10 | 2023-05-05 | 中国科学院上海技术物理研究所 | Image processing method of multispectral camera |
| CN110619373B (en) * | 2019-10-31 | 2021-11-26 | 北京理工大学 | Infrared multispectral weak target detection method based on BP neural network |
| CN112188055A (en) * | 2020-09-28 | 2021-01-05 | 维沃移动通信有限公司 | Camera module, identification method, electronic device and readable storage medium |
| CN112418057A (en) * | 2020-11-19 | 2021-02-26 | 中国人民解放军空军军医大学 | Multispectral-based ground wounded person identification method and system |
| CN112799143B (en) * | 2020-12-01 | 2023-03-03 | 中国电子科技集团公司第十一研究所 | Target identification method, photoelectric system and readable storage medium |
| CN113280919A (en) * | 2021-05-08 | 2021-08-20 | 中国科学院长春光学精密机械与物理研究所 | Target object spectrum detection method, spectrum detection device and storage medium |
| CN113240643A (en) * | 2021-05-14 | 2021-08-10 | 广州广电运通金融电子股份有限公司 | Banknote quality detection method, system and medium based on multispectral image |
| CN113538285A (en) * | 2021-07-23 | 2021-10-22 | 奥比中光科技集团股份有限公司 | Method and device for determining spectral image, terminal and storage medium |
| CN113780199A (en) * | 2021-09-15 | 2021-12-10 | 江苏迪赛司自动化工程有限公司 | Double-vision imaging device and intelligent identification method for belt-transported foreign object target |
| CN114354514B (en) * | 2021-12-24 | 2024-02-09 | 南昌大学 | Non-contact multi-mode material sensing and identifying device |
| CN114878094B (en) * | 2022-04-28 | 2023-05-12 | 西安交通大学 | Multispectral excitation oil mark imaging device and detection method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101124588A (en) * | 2004-06-01 | 2008-02-13 | 光谱辨识公司 | Multispectral imaging biometrics |
| CN107578432A (en) * | 2017-08-16 | 2018-01-12 | 南京航空航天大学 | Merge visible ray and the target identification method of infrared two band images target signature |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100711110B1 (en) * | 2005-11-03 | 2007-04-27 | 연세대학교 산학협력단 | System for iris recognition against counterfeit attack using gradient based fusion of multi-spectral images and method thereof |
| CN105913040B (en) * | 2016-04-27 | 2019-04-23 | 重庆邮电大学 | The real-time pedestrian detecting system of dual camera under the conditions of noctovision |
| CN106709477A (en) * | 2017-02-23 | 2017-05-24 | 哈尔滨工业大学深圳研究生院 | Face recognition method and system based on adaptive score fusion and deep learning |
-
2018
- 2018-09-12 CN CN201811064057.2A patent/CN109271921B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101124588A (en) * | 2004-06-01 | 2008-02-13 | 光谱辨识公司 | Multispectral imaging biometrics |
| CN107578432A (en) * | 2017-08-16 | 2018-01-12 | 南京航空航天大学 | Merge visible ray and the target identification method of infrared two band images target signature |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109271921A (en) | 2019-01-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109271921B (en) | Intelligent identification method and system for multispectral imaging | |
| CN109740478B (en) | Vehicle detection and identification method, device, computer equipment and readable storage medium | |
| US20220245792A1 (en) | Systems and methods for image quality detection | |
| Neal et al. | Measuring shape | |
| US9721173B2 (en) | Machine learning approach for detecting mobile phone usage by a driver | |
| CN111462128B (en) | Pixel-level image segmentation system and method based on multi-mode spectrum image | |
| CN105930822A (en) | Human face snapshot method and system | |
| US8345100B2 (en) | Shadow removal in an image captured by a vehicle-based camera using an optimized oriented linear axis | |
| CN115170792B (en) | Infrared image processing method, device and equipment and storage medium | |
| CN111965636A (en) | Night target detection method based on millimeter wave radar and vision fusion | |
| Bell et al. | A novel system for nighttime vehicle detection based on foveal classifiers with real-time performance | |
| CN108268832A (en) | Electric operating monitoring method, device, storage medium and computer equipment | |
| US20210158108A1 (en) | Object classification method, vehicle control method, information display method, and object classification device | |
| US20190096066A1 (en) | System and Method for Segmenting Out Multiple Body Parts | |
| Kale et al. | A road sign detection and the recognition for driver assistance systems | |
| CN112613568B (en) | Target identification method and device based on visible light and infrared multispectral image sequence | |
| CN111898427A (en) | Multispectral pedestrian detection method based on feature fusion deep neural network | |
| CN109064444B (en) | Track slab disease detection method based on significance analysis | |
| Kročka et al. | Extending parking occupancy detection model for night lighting and snowy weather conditions | |
| CN115147450A (en) | Moving target detection method and detection device based on motion frame difference image | |
| Kalmukov et al. | Methods for Automated Remote Sensing and Counting of Animals | |
| CN115909254B (en) | DMS system based on camera original image and image processing method thereof | |
| CN116721093B (en) | Subway rail obstacle detection method and system based on neural network | |
| Rajesh | Deep Learning Based Object Recognition in Real Time Images Using Thermal Imaging System | |
| Winkens et al. | Automatic shadow detection using hyperspectral data for terrain classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |