WO2021238185A1 - Object detection method and apparatus, electronic device, storage medium and program - Google Patents
Object detection method and apparatus, electronic device, storage medium and program Download PDFInfo
- Publication number
- WO2021238185A1 WO2021238185A1 PCT/CN2020/137919 CN2020137919W WO2021238185A1 WO 2021238185 A1 WO2021238185 A1 WO 2021238185A1 CN 2020137919 W CN2020137919 W CN 2020137919W WO 2021238185 A1 WO2021238185 A1 WO 2021238185A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- detected
- image
- cabin
- feature
- feature map
- Prior art date
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 169
- 230000004044 response Effects 0.000 claims abstract description 25
- 238000000034 method Methods 0.000 claims description 45
- 238000013528 artificial neural network Methods 0.000 claims description 43
- 230000004927 fusion Effects 0.000 claims description 32
- 238000000605 extraction Methods 0.000 claims description 21
- 239000011159 matrix material Substances 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 description 22
- 238000012549 training Methods 0.000 description 20
- 238000012360 testing method Methods 0.000 description 13
- 239000013598 vector Substances 0.000 description 12
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012216 screening Methods 0.000 description 2
- 241000190070 Sarracenia purpurea Species 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30268—Vehicle interior
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Definitions
- the present disclosure relates to the field of tracking technology, and relates to but not limited to object detection methods, devices, electronic equipment, storage media, and computer programs.
- the embodiments of the present disclosure provide at least one object detection solution.
- the embodiments of the present disclosure provide an object detection method, including:
- the sending a prompt message in response to the presence of the object to be detected in the image of the cabin to be detected for a duration exceeding a preset time period includes:
- a first prompt message is issued, and the first prompt message is sent.
- a reminder message is used to remind passengers that items are left behind; in the case that the reduced number of people in the cabin is the driver, the duration in response to the presence of the object to be detected in the image of the cabin to be detected exceeds the second preset
- the second prompt message is sent for the duration, and the second prompt message is used to prompt the driver that the item is left behind.
- the reduced number of people in the cabin is the driver and/or passengers.
- the object Detection methods also include:
- the attribution of the object to be detected is determined; wherein the attribution of the object to be detected is the driver and/or passenger.
- the performing target detection on the image in the cabin to be detected includes:
- the first feature map corresponding to the channel and the first feature maps corresponding to other channels are fused with feature information to obtain a fused second feature map ,include:
- For multiple first feature maps for feature information fusion determine the weight matrix corresponding to the multiple first feature maps; based on the weight matrix, perform a weighted summation on the feature information of the multiple first feature maps , To obtain a second feature map containing each fusion feature information.
- the detecting the to-be-detected object in the to-be-detected cabin image based on the fused second feature map includes:
- each candidate area contains a set number of feature points; determine the candidate area based on the feature data of the feature points contained in each candidate area Corresponding confidence level; the confidence level corresponding to each candidate area is used to characterize the credibility of the candidate area containing the object to be detected; based on the confidence level corresponding to each candidate area and the overlapping area between different candidate areas, The detection area corresponding to the object to be detected is screened out from the set number of candidate areas; the detection area is used to identify the position of the object to be detected in the image of the cabin to be detected.
- the acquiring the image in the cabin to be detected includes:
- the performing target detection on the image in the cabin to be detected further includes:
- each cabin image in the cabin video stream to be detected as a to-be-tracked image, and for each non-first frame to be tracked image, based on the previous frame of the non-first frame to-be-tracked image in the to-be-tracked image
- the target detection of the image in the cabin to be detected is performed by a neural network
- the neural network is trained by using sample images in the cabin containing the sample objects to be detected and sample images in the cabin that do not contain the sample objects to be detected.
- the embodiment of the present disclosure also provides an object detection device, which includes:
- the image acquisition module is configured to acquire an image in the cabin to be detected; the image detection module is configured to perform target detection on the image in the cabin to be detected when the number of people in the cabin is reduced, and determine the to-be-detected cabin image Whether there is an object to be detected in the image of the cabin; the prompt module is configured to send a prompt message in response to the state of the object to be detected in the image of the cabin to be detected for a duration exceeding a preset period of time.
- the prompt module is configured to send a prompt message in response to a state in which the object to be detected exists in the image in the cabin to be detected exceeds a preset period of time, including:
- a first prompt message is issued, and the first prompt message is sent.
- a reminder message is used to remind passengers that items are left behind; in the case that the reduced number of people in the cabin is the driver, the duration in response to the presence of the object to be detected in the image of the cabin to be detected exceeds the second preset
- the second prompt message is sent for the duration, and the second prompt message is used to prompt the driver that the item is left behind.
- the reduced number of people in the cabin is the driver and/or passenger.
- the prompt module sends a prompt message before, the image detection module was also configured as:
- the attribution of the object to be detected is determined; wherein the attribution of the object to be detected is the driver and/or passenger.
- the image detection module configured to perform target detection on the image in the cabin to be detected includes:
- the object to be detected in the image in the cabin to be detected is detected.
- the image detection module is configured to perform feature information fusion on the first feature map corresponding to the channel and the first feature maps respectively corresponding to other channels for each of the channels, to obtain the fusion
- the second feature map afterwards includes:
- the image detection module configured to detect the object to be detected in the image in the cabin to be detected based on the fused second feature map includes:
- the confidence level corresponding to the candidate area Based on the feature data of the feature points contained in each candidate area, determine the confidence level corresponding to the candidate area; the confidence level corresponding to each candidate area is used to characterize the credibility that the candidate area contains the object to be detected;
- the detection area corresponding to the object to be detected is screened out from the set number of candidate areas; the detection area is used to identify the to-be-detected object The position of the detection object in the image in the cabin to be detected.
- the image acquisition module configured to acquire the image in the cabin to be detected includes:
- the in-cabin images to be detected are extracted at intervals.
- the image detection module configured to perform target detection on the image in the cabin to be detected, further includes:
- each cabin image in the cabin video stream to be detected as a to-be-tracked image, and for each non-first frame to be tracked image, based on the previous frame of the non-first frame to-be-tracked image in the to-be-tracked image Determining the predicted position information of the object to be detected in the non-first frame to be tracked image of the position information of the object to be detected and the non-first frame to be tracked image;
- the non-first frame of the to-be-tracked image is the to-be-detected cabin image in which the object to be detected is detected
- use the detected position information as the position information of the to-be-detected object in the non-first frame of the to-be-tracked image
- the determined predicted position information is used as the position information of the to-be-detected object in the non-first frame of the to-be-tracked image.
- the target detection on the image in the cabin to be detected is performed by a neural network
- the neural network is trained by using sample images in the cabin containing the sample objects to be detected and sample images in the cabin that do not contain the sample objects to be detected.
- the embodiments of the present disclosure also provide an electronic device, which includes a processor, a memory, and a bus.
- the memory stores machine-readable instructions executable by the processor.
- the processing The processor communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, any one of the above-mentioned object detection methods is executed.
- the embodiment of the present disclosure also provides a computer-readable storage medium having a computer program stored on the computer-readable storage medium, and when the computer program is run by a processor, any one of the aforementioned object detection methods is executed.
- the embodiment of the present disclosure also provides a computer program, the computer program includes computer readable code, when the computer readable code is run in an electronic device, the processor in the electronic device executes for realizing any one of the above Kind of object detection method.
- a method for detecting leftover objects in a cabin scene is provided.
- the image is subject to target detection, so that it can be determined whether there is an object to be detected in the image to be detected in the cabin.
- the object to be detected may be an item lost by a person in the cabin. You can make corresponding prompts when the items are in the car, so as to reduce the probability of item loss in the riding environment and improve the safety of the items in the riding environment.
- FIG. 1 is a schematic flowchart of an object detection method provided by an embodiment of the disclosure
- FIG. 2 is a flowchart of a method for detecting an object to be detected according to an embodiment of the disclosure
- FIG. 3 is a flowchart of a method for determining the detection area of the object to be detected in the image in the cabin to be detected according to an embodiment of the disclosure
- FIG. 4 is a flowchart of a method for tracking an object to be detected according to an embodiment of the disclosure
- FIG. 5 is a flowchart of a method for training a target detection network in a neural network provided by an embodiment of the disclosure
- FIG. 6 is a flowchart of another method for training a target tracking network in a neural network provided by an embodiment of the disclosure
- FIG. 7 is a schematic structural diagram of an object detection device provided by an embodiment of the disclosure.
- FIG. 8 is a schematic diagram of an electronic device provided by an embodiment of the disclosure.
- the embodiments of the present disclosure provide a method for detecting leftover items in a cabin scene.
- the acquired vehicle to be detected can be checked.
- the cabin image is subject to target detection, so that it can be determined whether there is an object to be detected in the image in the cabin to be detected.
- the object to be detected may be an item lost by a person in the cabin.
- a person loses items he can give corresponding prompts, thereby reducing the probability of item loss in the riding environment and improving the safety of items in the riding environment.
- the equipment includes, for example, a terminal device or a server or other processing equipment.
- the terminal device may be a user equipment (User Equipment, UE), a mobile device, a user terminal, and the like.
- the object detection method may be implemented by a processor invoking computer-readable instructions stored in a memory.
- FIG. 1 is a flowchart of an improved object detection method according to an embodiment of the present disclosure, including the following S101 to S103:
- S101 Acquire an image in the cabin to be detected.
- the cabin can be a cabin of a public transportation vehicle such as a taxi cabin, a train cabin, or an airplane cabin;
- the image in the cabin to be detected can be captured by an image acquisition device set in a fixed position in the cabin.
- the image in the cabin to be detected can be captured.
- S102 When the number of people in the cabin is reduced, target detection is performed on the image in the cabin to be detected, and it is determined whether there is an object to be detected in the image in the cabin to be detected.
- Object detection is performed on images in the cabin, such as detecting whether there are still items left in the cabin that reduce the number of people.
- the target detection of the image in the cabin to be detected may be used to detect preset items that are easily lost by passengers or drivers, such as mobile phones, wallets, handbags, suitcases and the like.
- the embodiments of the present disclosure provide a method for detecting leftover items in a cabin scene.
- the acquired images in the cabin can be checked.
- Target detection can determine whether there is an object to be detected in the image in the cabin to be detected.
- the object to be detected may be an item lost by a person in the cabin.
- corresponding prompts can be made to reduce the probability of items lost in the riding environment and improve the safety of items in the riding environment.
- the prompt message in response to the duration of the state in which the object to be detected exists in the image of the vehicle cabin to be detected exceeds the preset time, when the prompt message is sent, it may include:
- the reduced number of people in the cabin is a passenger
- the duration of the state of the object to be detected exceeds the first preset time period, a first prompt message is issued, and the first prompt message is used To remind passengers of items left behind;
- a second prompt message is issued, the second prompt message Used to remind the driver that items are left behind.
- the first preset duration and the second preset duration here may be the same or different. Considering that the driver may only get out of the cabin for a short time, the second preset duration here may be greater than the first preset duration. duration.
- both the first prompt information and the second prompt information may be broadcast in language, where the first prompt information is used to prompt the passenger or the driver, and the second prompt information is used to prompt the driver.
- the reduced number of persons in the cabin is the driver and/or passengers. After it is determined that there is an object to be detected in the image of the cabin to be detected, before the prompt message is issued, the object detection method further includes :
- the attribution of the object to be detected is determined; where the attribution of the object to be detected is the driver and/or passenger.
- the position of each person in the cabin in the cabin and the corresponding item to be detected in the cabin can be determined, so that the association between the object to be detected and the position can be established
- the relationship, and the association relationship between the position and the person in the cabin, and then the person who belongs to the object to be detected can be determined according to the position of the object to be detected in the cabin.
- a corresponding prompt message is issued according to the attribution.
- the attribution of the object to be detected can be determined based on the position of the object to be detected in the cabin, so as to facilitate subsequent classification prompts.
- S201 Perform feature extraction on the image in the cabin to be detected, to obtain a first feature map corresponding to each of the multiple channels; wherein the first feature map corresponding to each channel is the image corresponding to the object to be detected in the channel.
- the feature extraction of the image in the cabin to be detected can be performed by feature extraction through a feature extraction network trained in advance to obtain the first feature map corresponding to multiple preset channels.
- each channel can be understood as corresponding to the vehicle to be detected.
- An image feature category of the cabin image For example, after feature extraction of the cabin image to be detected, the first feature map corresponding to the three channels can be obtained, and the first channel can correspond to the cabin image to be detected.
- the second channel can correspond to the color feature of the image in the cabin to be detected, and the third channel can correspond to the size feature of the image in the cabin to be detected, so that the image in the cabin to be detected can be obtained Feature maps under each image feature category.
- feature extraction is performed on the image in the cabin to be detected to obtain the first feature map.
- the first feature map corresponding to each channel will represent the object to be detected.
- the feature information of the vehicle cabin is distinguished from the feature information representing the background in the cabin.
- the feature information representing the object to be detected can be enhanced, and the feature information representing the background in the cabin can be weakened.
- the feature information of the detected object is enhanced, or only the feature information representing the background in the cabin can be weakened, so that the strength of the feature information representing the object to be detected in each first feature map obtained is greater than that of the cabin.
- the strength of the characteristic information of the inner background is provided.
- S202 For each channel, perform feature information fusion on the first feature map corresponding to the channel and the first feature maps respectively corresponding to other channels to obtain a fused second feature map.
- each channel tends to indicate the feature information of the image in the cabin to be detected under the corresponding image feature category of the channel, in order to obtain a more complete feature map of the feature information, here for each channel, the first feature corresponding to the channel
- the first feature map corresponding to the image and the other channels is fused with feature information, that is, a second feature map containing multiple image feature categories can be obtained.
- the feature information in the first feature map corresponding to each channel can be represented by the feature data in the first feature map corresponding to the channel.
- Feature information fusion refers to fusing the feature data in each first feature map to obtain The second feature map after fusion.
- S203 Detect the object to be detected in the image of the cabin to be detected based on the second feature map after the fusion.
- the process of detecting the object to be detected in the image in the cabin to be detected may be based on the target detection network in the pre-trained neural network, and the image in the cabin to be detected The object to be detected is detected, that is, the fused second feature map is input to the target detection network in the pre-trained neural network, which can complete the detection of the object to be detected in the image of the cabin to be detected.
- detecting the object to be detected in the image in the cabin to be detected may refer to detecting whether there is the object to be detected in the image in the cabin to be detected, and when it is determined that there is the object to be detected in the image in the cabin to be detected , To determine the position information of the object to be detected in the image of the cabin to be detected.
- the first feature map obtained by feature extraction is the feature map after enhancement processing is performed on the feature of the object to be detected in the image feature category corresponding to the channel, that is, the feature map contained in each first feature map
- the feature information of the detected object is enhanced compared to the feature information of the non-detected object, so that the object to be detected can be clearly distinguished from the background area in the image of the cabin to be detected through the feature information; then for each channel, the channel
- the corresponding first feature map and the first feature maps corresponding to other channels are fused with feature information, thereby obtaining a more comprehensive feature information of the object to be detected, and then based on this second feature map to complete the image of the cabin to be detected
- the detection of the object to be detected can accurately detect the object to be detected in the image of the cabin to be detected.
- a first feature map with a size of h*w*c is obtained, where c represents the number of the first feature maps, that is, the cabin to be detected.
- c represents the number of the first feature maps, that is, the cabin to be detected
- h*w represents the size of each first feature map
- each first feature map contains h*w feature points corresponding to Characteristic data.
- the size of the fused second feature map is also h*w*c, that is, each channel corresponds to a second feature map, and each second feature map
- the size of the graph is h*w
- the feature data corresponding to any feature point in the second feature graph corresponds to the feature point in the same position of any feature point in the second feature graph in the first feature graph corresponding to each channel
- the weight matrix here contains the weight vectors corresponding to the c channels, and the weight value in the weight vector corresponding to each channel represents the weight value of the feature data in each first feature map when determining the second feature map corresponding to the channel .
- c is equal to 3, which means that after feature extraction of the image in the cabin to be detected, first feature maps corresponding to 3 channels are obtained, that is, 3 first feature maps are obtained, and each first feature map contains h* Feature data corresponding to w feature points, these h*w feature data can constitute a feature vector of h*w dimensions, and each feature data in the feature vector is the feature data corresponding to each feature point in the first feature map.
- the corresponding weight matrix of the channel can be used for each channel.
- the feature data in the first feature map corresponding to the channel is weighted and summed to obtain the feature data in the second feature map corresponding to the channel.
- the embodiments of the present disclosure by enriching the feature information contained in the object to be detected, and increasing the degree of discrimination between the object to be detected and the background area in the image in the cabin, it is convenient for the later based to be richer and more distinguishable from the background area.
- the large feature information accurately determines whether there is an object to be detected in the image of the cabin to be detected, and the position information of the object to be detected.
- each first feature map contains h*w feature data.
- the feature matrix formed by the feature vector corresponding to each first feature map is:
- (a 1 a 2 ... a h*w ) T can be used to represent the feature vector of the first feature map corresponding to the first channel; a 1 represents the first feature map corresponding to the first channel.
- the feature data of each feature point, a 2 represents the feature data of the second feature point in the first feature map corresponding to the first channel; a h*w represents the h*w th in the first feature map corresponding to the first channel Feature data of each feature point;
- (b 1 b 2 ... b h*w ) T can be used to represent the feature vector of the first feature map corresponding to the second channel;
- b 1 represents the first feature in the first feature map corresponding to the second channel Point feature data;
- b 2 represents the feature data of the second feature point in the first feature map corresponding to the second channel;
- b h*w represents the h*wth feature in the first feature map corresponding to the second channel Point characteristic data;
- (d 1 d 2 ... d h*w ) T can be used to represent the feature vector of the first feature map corresponding to the third channel; d 1 represents the first feature in the first feature map corresponding to the third channel Point feature data, d 2 represents the feature data of the second feature point in the first feature map corresponding to the third channel, and d h*w represents the h*wth feature in the first feature map corresponding to the third channel Point characteristic data.
- (m 1 m 2 m 3 ) T represents the weight vector corresponding to different first feature maps when determining the second feature map corresponding to the first channel, and m 1 represents the first feature map corresponding to the first channel
- the weight value of each feature data when determining the second feature map corresponding to the first channel; m 2 means that each feature data in the first feature map corresponding to the second channel determines the second feature map corresponding to the first channel
- M 3 represents the weight value of each feature data in the first feature map corresponding to the third channel when determining the second feature map corresponding to the first channel.
- (k 1 k 2 k 3 ) T represents the weight vector corresponding to different first feature maps when determining the second feature map corresponding to the second channel, and k 1 represents the first feature map corresponding to the first channel
- the weight value of each feature data when determining the second feature map corresponding to the second channel; k 2 means that each feature data in the first feature map corresponding to the second channel determines the second feature map corresponding to the second channel
- K 3 represents the weight value of each feature data in the first feature map corresponding to the third channel when determining the second feature map corresponding to the second channel.
- (l 1 l 2 l 3 ) T represents the weight vector corresponding to different first feature maps when determining the second feature map corresponding to the third channel
- l 1 represents the first feature map corresponding to the first channel
- l 2 means that each feature data in the first feature map corresponding to the second channel determines the second feature map corresponding to the third channel 13 represents the weight value of each feature data in the first feature map corresponding to the third channel when determining the second feature map corresponding to the third channel.
- the second feature map corresponding to the first channel can be determined according to the following formula (1):
- T 1 (a 1 a 2 ... a h*w ) T *m 1 +(b 1 b 2 ... b h*w ) T *m 2 +(d 1 d 2 ... d h* w ) T *m 3 (1)
- the feature data of the first feature point in the second feature map corresponding to the first channel is a 1 m 1 + b 1 m 2 + d 1 m 3 ;
- the first channel corresponding to the first channel in the second feature map is The feature data of the two feature points is a 2 m 1 + b 2 m 2 + d 2 m 3 ;
- the feature data of the h*w feature point in the second feature map corresponding to the first channel is a h* w m 1 +b h*w m 2 +d h*w m 3 .
- the second feature map corresponding to the second channel and the second feature map corresponding to the third channel can be determined in the same manner.
- the second feature map after fusion by determining the weight matrix corresponding to the first feature map, the second feature map corresponding to each channel is obtained, so that each second feature map corresponds to multiple channels
- the features under the image feature category are fused. If the image in the cabin to be detected contains the object to be detected, the second feature map after fusion can contain more feature information of the object to be detected, and because of the first feature In the figure, the feature of the detection object is enhanced, and the distinction between the feature information of the object to be detected and the feature information of the background area in the second feature map after the fusion obtained based on the first feature map is also greater, so It is convenient for the later stage to accurately determine whether there is an object to be detected and the position information of the object to be detected in the image of the cabin to be detected based on the feature information that is richer and more distinguishable from the background area.
- the object to be detected in the image of the cabin to be detected can be detected according to the fused second feature map.
- the following steps S301 to S303 may be included:
- S301 Determine a set number of candidate regions based on the fused second feature map, and each candidate region contains a set number of feature points.
- the candidate area here refers to the area that may contain the object to be detected.
- the number of candidate areas and the set number of feature points contained in each candidate area can be determined by the candidate area extraction network in the pre-trained neural network. .
- the set number of candidate regions is based on the consideration of the test accuracy of the target detection network. For example, during the network training process, the candidates are continuously adjusted for the fused second sample feature maps corresponding to a large number of sample images to be detected. The number of regions, and then in the testing process, the trained target detection network is tested, and the set number of candidate regions is determined through the test accuracy corresponding to different candidate regions.
- the number of settings contained in each candidate area can be determined in advance based on the comprehensive consideration of the test speed and test accuracy of the target detection network. For example, in the network training process, first keep the number of candidate areas unchanged, and continuously adjust each The number of feature points contained in the candidate area, and then in the test process, the target detection network is tested, and the test speed and test accuracy are comprehensively considered to determine the set number of feature points contained in each candidate area.
- the feature points contained in each candidate area correspond to feature data. According to these feature data, the credibility that the candidate area contains the object to be detected can be determined. For example, the confidence level corresponding to each candidate area can be passed
- the target detection network in the pre-trained neural network is determined, that is, the feature data in the candidate area is input into the target detection network in the pre-trained neural network, and the confidence level corresponding to the candidate area can be obtained.
- the detection area corresponding to the object to be detected is selected from a set number of candidate areas based on the confidence level corresponding to each candidate area and the overlap area between different candidate areas, you can start with The set number of candidate regions are selected to filter out the set number of target candidate regions before the confidence ranking, and then based on the preset confidence threshold and the overlap area between different candidate regions, the detection region corresponding to the object to be detected can be determined .
- the target candidate area with the corresponding confidence level higher than the confidence threshold is more likely to be the detection area corresponding to the object to be detected, and considering that there are overlapping candidate areas between the candidate areas, if an overlapping candidate occurs
- the overlap area of the region is greater than the set area threshold, which can indicate that the object to be detected contained in the overlapping candidate area may be the same object to be detected.
- the detection area corresponding to the object to be detected is further selected from the target candidate area.
- the target candidate area with a confidence higher than the confidence threshold can be reserved in the target candidate area, and the target candidate area with the highest confidence can be reserved in the target candidate area where the overlap occurs, that is, the detection area corresponding to the object to be detected is obtained.
- the above process of screening out the set number of target candidate regions before the confidence sorting from the set number of candidate regions can be determined according to the target detection network, which can be specifically based on the test speed of the target detection network and The test accuracy is determined in advance by comprehensive consideration. For example, during the network training process, the number of target candidate areas is constantly adjusted, and then during the test process, the target detection network is tested, and the test speed and test accuracy are comprehensively considered to determine the target candidate area here. The number of settings.
- the confidence level corresponding to each candidate area here is less than the set threshold, it can indicate that there is no object to be detected in the image of the cabin to be detected, and this situation is not described in detail in the embodiment of the present disclosure.
- the detection area of the object to be detected in the image of the cabin to be detected can be obtained, that is, the position of the object to be detected in the image of the cabin to be detected is obtained, and the second feature after fusion is passed here.
- Map to determine the candidate area because the second feature map after the fusion contains the feature information of the object to be detected and the feature information of the background area is more distinguishable, and the feature information of the object to be detected is more abundant, so based on the fusion
- the latter second feature map can accurately obtain the candidate area representing the position of the object to be detected in the area to be detected and the confidence of each candidate area.
- it is proposed to further consider the possible position information of the object to be detected by considering the overlapping area of the candidate area. Screening can accurately obtain whether there is an object to be detected and the position information of the object to be detected in the image in the cabin to be detected.
- the object detection method proposed in the embodiments of the present disclosure requires continuous acquisition of the images in the cabin to be detected in many application scenarios, and the detection of the images in the cabin to be detected, for example, the detection of leftover objects in transportation scenarios
- you can install an image acquisition component in the car for example, install a camera in the car, and make the camera face the set position to shoot.
- the cabin images to be detected are extracted at intervals.
- the video stream in the cabin to be detected may be a video stream captured by the image capture component at a set position in the car, and the video stream captured per second may be Contains multiple consecutive frames of images in the cabin. Taking into account the short interval between two adjacent frames of images, the similarity of the images in the adjacent two frames of the cabin is relatively high.
- the Interval extraction is performed from the frames of the cabin image to obtain the above-mentioned cabin image to be detected. For example, if the cabin video stream to be detected obtained in a certain period of time contains 1000 frames of images, follow Extracting once every frame, you can get 500 frames of images in the cabin to be detected.
- the detection of these 500 frames of images in the cabin to be detected can accomplish the purpose of detecting the remaining items in the cabin.
- the images in the cabin to be detected are extracted in an interval manner, and the images in the cabin to be detected that need to be detected are obtained from the video stream in the cabin to be detected, which can improve the detection efficiency.
- the position information of the to-be-detected object in each frame of the cabin image can also be tracked, as shown in Fig. 4 As shown, the following S401 ⁇ S404 are also included:
- S401 Use each cabin image in the cabin video stream to be detected as a to-be-tracked image, and for each non-first frame of the to-be-tracked image, based on the previous frame of the non-first-frame to-be-tracked image in the to-be-tracked image.
- the position information of the object to be detected and the non-first frame of the to-be-tracked image determine the predicted position information of the object to be detected in the non-first frame of the to-be-tracked image.
- the object detection is performed on the interval-extracted cabin image, and the position information of the object to be detected in the interval-extracted cabin image is respectively determined.
- target detection is performed on single-number frames of cabin images such as the first frame of the cabin image, the third frame of the cabin image, and the fifth frame of the cabin image.
- the object to be detected is tracked in the second frame of the cabin image
- the predicted position information of the object to be detected in the cabin image of the second frame can be determined based on the position information of the object to be detected in the cabin image of the first frame and the cabin image of the second frame .
- the object to be detected when tracking the object to be detected, it can be based on the target tracking network in the pre-trained neural network. For example, for the first frame to be tracked and the second frame to be tracked, according to the object to be detected The detection area in the image to be tracked, and the feature data of the feature points contained in the detection area, where the detection area has corresponding coordinate information, and then the detection area, the feature data of the feature points contained in the detection area, and the first Two frames of to-be-tracked images are input into the target tracking network, that is, based on the coordinate information corresponding to the detection area in the first frame of the to-be-tracked image of the object to be detected, in the local area corresponding to the coordinate information in the second frame of the to-be-tracked image Look for whether there is a detection area whose feature data similarity to the feature points contained in the detection area exceeds the threshold.
- the second frame of the image to be tracked contains the object to be detected in the first frame of the image to be tracked, and get The position information of the object to be detected in the image to be tracked in the first frame in the image to be tracked in the second frame is to complete the tracking of the object to be detected.
- the second frame of the to-be-tracked image Does not include the object to be detected in the image to be tracked in the first frame, and it can be determined that the object to be detected has moved in position.
- S402 Determine whether the non-first frame of the to-be-tracked image is a to-be-detected cabin image in which the object to be detected is detected.
- the non-first frame of the to-be-tracked image is the to-be-detected cabin image in which the object to be detected is detected, so as to consider whether the detected object is in the non-first frame of the to-be-tracked image
- the position information of the object to be detected is corrected in the predicted position information of the image to be tracked in the non-first frame, so as to track the position of the object to be detected in the image to be tracked in the next frame based on the corrected position information.
- the detected position information is used as the position information of the to-be-detected object in the non-first frame of the to-be-tracked image, that is, complete
- the predicted position information of the object to be detected in the non-first frame of the image to be tracked is corrected, and subsequently based on the position information of the object to be detected in the non-first frame of the image to be tracked, when the object to be detected is tracked , Can be more accurate.
- the object to be detected can be determined based on the predicted position information of the object to be detected in the non-first frame of the to-be-tracked image Continue to track the position in the next frame of the image to be tracked. This method can estimate the position of the object to be detected in the cabin at each moment, thereby improving the tracking efficiency.
- the non-first frame to-be-tracked image can be tracked based on the position information of the to-be-detected object in the previous frame of the non-first frame-to-be-tracked image, and it is determined that the to-be-detected object is in the non-first frame.
- the predicted location information in the image is tracked, and during the tracking process, the predicted location information can also be adjusted based on the detected location information. In this way, the efficiency and accuracy of tracking the object to be detected can be improved.
- the target detection of the image in the cabin to be detected proposed in the embodiment of the present disclosure is performed by a neural network, where the neural network uses the image in the cabin that contains the sample object to be detected and does not contain the sample to be detected.
- the sample images in the vehicle cabin of the object are obtained through training.
- the network for target detection in the neural network can be obtained by training in the following manner, as shown in FIG. 5, which specifically includes S501 to S505:
- S501 Acquire a sample image in the cabin to be detected.
- the sample image in the cabin to be detected here includes the sample image in the cabin that contains the sample object to be detected, which can be recorded as a positive sample image, and the sample image in the cabin that does not contain the sample object to be detected, can be recorded as negative Sample image.
- the appearance of the leftover objects in the sample image in the cabin may be various color blocks, such as mobile phones, suitcases, etc. can be represented by rectangular color blocks, and the water cup can be represented by cylindrical colors.
- Block representation in order to enable the neural network to better identify which are the objects to be detected and which are the background in the car, such as the background of the car seat, window, etc., here can add some non-to-be-detected items to the sample image in the cabin
- the random color patches of are used to represent the objects not to be detected.
- the real objects to be detected and the non-real random color patches and the background in the car are continuously distinguished, so as to obtain a neural network with higher accuracy.
- S502 Perform feature extraction on the sample image in the cabin to be detected to obtain a first sample feature map corresponding to each of the multiple channels; wherein the first sample feature map corresponding to each channel is the sample to be detected A sample feature map after the feature of the object in the image feature category corresponding to the channel is enhanced.
- feature extraction is performed on the sample image to be detected, and the process of obtaining the first sample feature map corresponding to each of the multiple channels is performed with the feature extraction of the image in the cabin to be detected as mentioned above, and multiple The process of the first feature map corresponding to each channel in the channel is similar, and will not be repeated here.
- S503 For each channel, perform feature information fusion on the first sample feature map corresponding to the channel and the first sample feature maps corresponding to other channels to obtain a fused second sample feature map.
- the process of obtaining the fused second sample feature map is similar to the process of obtaining the fused second feature map based on the first feature map mentioned above, and will not be repeated here. .
- S504 Predict the sample object to be detected in the sample image in the cabin to be detected based on the fused second sample feature map.
- the sample object to be detected in the sample image in the cabin is pre-stored, and the second feature map based on the fusion mentioned above is used to detect the to be detected in the cabin image to be detected.
- the process of detecting objects is similar, so I won't repeat them here.
- the position information of the sample object to be tested in the sample image in the cabin to be tested is predicted, the sample image in the cabin to be tested that contains the sample to be tested, and the sample in the cabin to be tested that does not contain the sample to be tested Image, to determine the predicted loss value of the position information of the sample object to be detected in the sample image in the cabin to be detected, and adjust the network parameter value in the neural network through the loss value.
- the training can be stopped, so as to obtain the trained neural network.
- the embodiments of the present disclosure also include a process of training the target tracking network in the neural network.
- the tracking sample image and the to-be-tracked sample image that does not contain the sample object to be detected are obtained through training.
- the sample object to be detected here may refer to the sample object that needs to be tracked.
- the sample object to be detected here can be passenger objects in various car scenes.
- the target tracking network in the neural network can be obtained by training in the following manner, as shown in FIG. 6, which specifically includes S601 to S603:
- S601 Acquire a sample image to be tracked and information about the sample object to be detected corresponding to the sample object to be detected.
- the sample image to be tracked here may refer to the sample image that needs to be tracked for the sample object to be detected.
- the sample image to be tracked here may include a positive sample image that contains the sample object to be detected and a negative sample image that does not contain the sample object to be detected.
- the detection area image of the sample object to be detected and the sample image to be tracked can be input into the neural network at the same time.
- the detection area image of the sample object to be detected contains the corresponding image of the sample object to be detected.
- the information of the sample object to be detected may include the detection area of the object to be detected and the feature data of the feature points contained in the detection area.
- S602 Based on the sample object information to be detected and the sample image to be tracked, the position of the sample object to be detected in the sample image is tracked, and the position information of the sample object to be detected in the sample image is predicted.
- the sample object to be detected in the sample images continuously acquired in the same area it can first be determined based on the detection area corresponding to the sample object to be detected in the sample object information to determine that the sample object to be detected is in the sample to be tracked.
- the local area in the image where the local area is close to the detection area corresponding to the sample object to be detected, so that the sample object to be detected can be detected in the local area based on the feature data, so as to predict that the sample object to be detected is in the sample image to be tracked Location information.
- the position information of the sample object to be detected in the sample image to be tracked can be predicted, the sample image to be tracked that contains the sample object to be detected, and the sample image to be tracked that does not contain the sample object to be detected, to determine the sample image to be tracked
- the loss value of the position information of the sample object to be detected After multiple training, the network parameter value in the neural network is adjusted through the loss value. For example, when the loss value is less than the set threshold, the training can be stopped to obtain Neural network target tracking network.
- the position of the sample object to be detected in the sample image to be tracked is tracked by acquiring the sample image to be tracked and the information of the sample object to be detected corresponding to the sample object to be detected. So as to quickly determine the position of the sample object to be detected in the sample image to be tracked, and then predict the position information of the sample object to be detected in the sample image to be tracked, the sample image to be tracked that contains the sample object to be detected, and the sample that does not contain the sample to be detected.
- the network parameter values of the neural network are adjusted to obtain a neural network with higher accuracy. Based on the neural network with higher accuracy, the object to be detected can be accurately tracked.
- the writing order of the steps does not mean a strict execution order but constitutes any limitation on the implementation process.
- the specific execution order of each step should be based on its function and possibility.
- the inner logic is determined.
- the embodiment of the present disclosure also provides an object detection device corresponding to the object detection method. Since the principle of the device in the embodiment of the present disclosure to solve the problem is similar to the above-mentioned object detection method of the embodiment of the present disclosure, the implementation of the device You can refer to the implementation of the method, and the repetition will not be repeated here.
- the object detection device 700 includes: an image acquisition module 701, an image detection module 702, and a prompt module 703.
- the image acquisition module 701 is configured to acquire an image in the cabin to be detected
- the image detection module 702 is configured to perform target detection on the image in the cabin to be detected when the number of people in the cabin is reduced, and determine whether there is an object to be detected in the image in the cabin to be detected;
- the prompting module 703 is configured to send a prompt message in response to a state in which the object to be detected exists in the image of the cabin to be detected for a duration that exceeds a preset period of time.
- the prompt module 703 is configured to send a prompt message in response to a state in which the object to be detected exists in the image in the cabin to be detected exceeds a preset period of time, including:
- a first prompt message is issued, and the first prompt message is sent.
- a reminder message is used to remind passengers that items are left behind; in the case that the reduced number of people in the cabin is the driver, the duration in response to the presence of the object to be detected in the image of the cabin to be detected exceeds the second preset
- the second prompt message is sent for the duration, and the second prompt message is used to prompt the driver that the item is left behind.
- the reduced number of people in the cabin is the driver and/or passenger.
- the prompt module issues a prompt before information, the image detection module 702 is also configured to:
- the attribution of the object to be detected is determined; wherein the attribution of the object to be detected is the driver and/or passenger.
- the image detection module 702, configured to perform target detection on the image in the cabin to be detected includes:
- the object to be detected in the image in the cabin to be detected is detected.
- the image detection module 702 is configured to perform, for each channel, a first feature map corresponding to the channel and first feature maps corresponding to other channels to perform feature information fusion to obtain The second feature map after fusion, including:
- the image detection module 702 is configured to detect the object to be detected in the image in the cabin to be detected based on the fused second feature map, including:
- the confidence level corresponding to the candidate area Based on the feature data of the feature points contained in each candidate area, determine the confidence level corresponding to the candidate area; the confidence level corresponding to each candidate area is used to characterize the credibility that the candidate area contains the object to be detected;
- the detection area corresponding to the object to be detected is screened out from the set number of candidate areas; the detection area is used to identify the to-be-detected object The position of the detection object in the image in the cabin to be detected.
- the image acquisition module 701 configured to acquire the image in the cabin to be detected includes:
- the in-cabin images to be detected are extracted at intervals.
- the image detection module 702 configured to perform target detection on the image in the cabin to be detected, further includes:
- each cabin image in the cabin video stream to be detected as a to-be-tracked image, and for each non-first frame to be tracked image, based on the previous frame of the non-first frame to-be-tracked image in the to-be-tracked image Determining the predicted position information of the object to be detected in the non-first frame to be tracked image of the position information of the object to be detected and the non-first frame to be tracked image;
- the non-first frame of the to-be-tracked image is the to-be-detected cabin image in which the object to be detected is detected
- use the detected position information as the position information of the to-be-detected object in the non-first frame of the to-be-tracked image
- the determined predicted position information is used as the position information of the to-be-detected object in the non-first frame of the to-be-tracked image.
- the object detection device further includes a neural network training module 704, and the neural network training module 704 is used to:
- a neural network for target detection on images in the cabin to be detected is trained using sample images in the cabin containing the sample objects to be detected and sample images in the cabin that do not contain the sample objects to be detected.
- an embodiment of the present disclosure also provides an electronic device 800.
- a schematic structural diagram of the electronic device 800 provided by the embodiment of the present disclosure includes:
- the processor 81 and the memory 82 communicate through the bus 83, so that the processor 81 executes the above method Any object detection method in the embodiment.
- the embodiments of the present disclosure also provide a computer-readable storage medium on which a computer program is stored, and when the computer program is run by a processor, any one of the object detection methods in the foregoing method embodiments is executed.
- the storage medium may be a volatile or nonvolatile computer readable storage medium.
- the computer program product of the object detection method provided by the embodiment of the present disclosure includes a computer-readable storage medium storing program code.
- the instructions included in the program code can be used to execute any of the object detection methods in the foregoing method embodiments. Refer to the foregoing method embodiment, which will not be repeated here.
- the embodiments of the present disclosure also provide a computer program, which, when executed by a processor, implements any one of the methods in the foregoing embodiments.
- the computer program product can be specifically implemented by hardware, software, or a combination thereof.
- the computer program product is specifically embodied as a computer storage medium.
- the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc. Wait.
- SDK software development kit
- the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
- the functional units in the various embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
- the function is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a non-volatile computer readable storage medium executable by a processor.
- the technical solution of the present disclosure essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present disclosure.
- the aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .
- Embodiments of the present disclosure provide an object detection method, device, electronic equipment, storage medium, and computer program, where the object detection method includes: acquiring an image in a cabin to be detected; when the number of people in the cabin is reduced, Perform target detection on the image in the cabin to be detected to determine whether there is an object to be detected in the image in the cabin to be detected; responding to the state of the object to be detected in the image in the cabin to be detected If the duration exceeds the preset duration, a prompt message will be issued. In this way, when an item lost by a person in the cabin is detected, a corresponding prompt can be given, thereby reducing the probability of item loss in the riding environment and improving the safety of the item in the riding environment.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
Description
Claims (21)
- 一种对象检测方法,包括:An object detection method includes:获取待检测的车舱内图像;Obtain the image in the cabin to be inspected;在车舱内人员减少的情况下,对所述待检测的车舱内图像进行目标检测,确定所述待检测的车舱内图像中是否存在待检测对象;Under the condition that the number of people in the cabin is reduced, target detection is performed on the image in the cabin to be detected, and it is determined whether there is an object to be detected in the image in the cabin to be detected;响应于所述待检测的车舱内图像中存在待检测对象的状态的持续时长超过预设时长,发出提示信息。In response to the duration of the state in which the object to be detected exists in the image in the cabin to be detected exceeds a preset time period, a prompt message is issued.
- 根据权利要求1所述的对象检测方法,其中,所述响应于所述待检测的车舱内图像中存在待检测对象的状态的持续时长超过预设时长,发出提示信息,包括:The object detection method according to claim 1, wherein the sending a prompt message in response to the state in which the object to be detected exists in the image of the cabin to be detected exceeds a preset time period, comprising:在减少的车舱内人员为乘客的情况下,响应于所述待检测的车舱内图像中存在待检测对象的状态的持续时长超过第一预设时长,发出第一提示信息,所述第一提示信息用于提示乘客物品遗留;In the case that the reduced number of persons in the cabin is a passenger, in response to the duration of the state in which the object to be detected exists in the image to be detected in the cabin exceeds a first preset time period, a first prompt message is issued, and the first prompt message is sent. A reminder message is used to remind the passenger that the item is left behind;在减少的车舱内人员为驾驶员的情况下,响应于所述待检测的车舱内图像中存在待检测对象的状态的持续时长超过第二预设时长,发出第二提示信息,所述第二提示信息用于提示驾驶员物品遗留。In the case where the reduced number of persons in the cabin is the driver, in response to the duration of the state in which the object to be detected exists in the image to be detected in the cabin exceeds the second preset duration, a second prompt message is issued, the The second prompt message is used to prompt the driver that the item is left behind.
- 根据权利要求1或2所述的对象检测方法,其中,减少的车舱内人员为驾驶员和/或乘客,在确定所述待检测的车舱内图像中存在待检测对象后,在发出提示信息之前,所述对象检测方法还包括:The object detection method according to claim 1 or 2, wherein the reduced persons in the cabin are drivers and/or passengers, and after determining that the object to be detected exists in the cabin image to be detected, a prompt is issued Before information, the object detection method further includes:根据所述待检测对象在所述车舱中位置,确定所述待检测对象的归属人员;其中,所述待检测对象的归属人员为驾驶员和/或乘客。According to the position of the object to be detected in the cabin, the attribution of the object to be detected is determined; wherein the attribution of the object to be detected is the driver and/or passenger.
- 根据权利要求1至3任一所述的对象检测方法,其中,所述对所述待检测的车舱内图像进行目标检测,包括:The object detection method according to any one of claims 1 to 3, wherein the performing target detection on the image in the cabin to be detected includes:对所述待检测的车舱内图像进行特征提取,得到与多个通道中每个通道对应的第一特征图;其中每个通道对应的第一特征图,为将待检测对象在该通道对应的图像特征类别下的特征进行增强处理后的特征图;Perform feature extraction on the image in the cabin to be detected to obtain a first feature map corresponding to each of the multiple channels; wherein the first feature map corresponding to each channel corresponds to the object to be detected in the channel The feature map after the enhancement processing is performed on the features under the image feature category;针对每个所述通道,将该通道对应的第一特征图与其它通道分别对应的第一特征图进行特征信息融合,得到融合后的第二特征图;For each of the channels, perform feature information fusion on the first feature map corresponding to the channel and the first feature maps corresponding to the other channels to obtain a fused second feature map;基于所述融合后的第二特征图,检测所述待检测的车舱内图像中的所述待检测对象。Based on the fused second feature map, the object to be detected in the image in the cabin to be detected is detected.
- 根据权利要求4所述的对象检测方法,其中,所述针对每个所述通道,将该通道对应的第一特征图与其它通道分别对应的第一特征图进行特征信息融合,得到融合后的第二特征图,包括:The object detection method according to claim 4, wherein, for each of the channels, the first feature map corresponding to the channel and the first feature maps corresponding to other channels are fused with feature information to obtain the fused The second feature map includes:针对进行特征信息融合的多个第一特征图,确定所述多个第一特征图所对应的权重矩阵;Determining a weight matrix corresponding to the plurality of first feature maps for performing feature information fusion;基于所述权重矩阵,对所述多个第一特征图的特征信息进行加权求和,得到包含各个融合特征信息的所述第二特征图。Based on the weight matrix, performing a weighted summation on the feature information of the multiple first feature maps to obtain the second feature map containing each fused feature information.
- 根据权利要求4所述的对象检测方法,其中,所述基于所述融合后的第二特征图,检测所述待检测的车舱内图像中的所述待检测对象,包括:4. The object detection method according to claim 4, wherein the detecting the object to be detected in the image of the cabin to be detected based on the fused second feature map comprises:基于所述融合后的第二特征图,确定设定个数的候选区域,每个候选区域包含设定个数的特征点;Determine a set number of candidate regions based on the fused second feature map, and each candidate region contains a set number of feature points;基于每个候选区域包含的特征点的特征数据,确定该候选区域对应的置信度;每个候选区域对应的置信度用于表征该候选区域中包含所述待检测对象的可信程度;Based on the feature data of the feature points contained in each candidate area, determine the confidence level corresponding to the candidate area; the confidence level corresponding to each candidate area is used to characterize the credibility that the candidate area contains the object to be detected;基于每个候选区域对应的置信度以及不同候选区域之间的重叠区域,从所述设定个 数的候选区域中筛选出待检测对象对应的检测区域;所述检测区域用于标识所述待检测对象在所述待检测的车舱内图像中的位置。Based on the confidence level corresponding to each candidate area and the overlapping area between different candidate areas, the detection area corresponding to the object to be detected is screened out from the set number of candidate areas; the detection area is used to identify the to-be-detected object The position of the detection object in the image in the cabin to be detected.
- 根据权利要求1所述的对象检测方法,其中,所述获取所述待检测的车舱内图像,包括:The object detection method according to claim 1, wherein said acquiring the image in the cabin to be detected comprises:获取待检测的车舱内视频流;Obtain the video stream in the cabin to be detected;从所述待检测的车舱内视频流包含的连续多帧车舱内图像中,间隔提取得到所述待检测的车舱内图像。From the continuous multiple frames of the in-cabin images contained in the video stream in the cabin to be detected, the in-cabin images to be detected are extracted at intervals.
- 根据权利要求7所述的对象检测方法,其中,所述对所述待检测的车舱内图像进行目标检测,还包括:The object detection method according to claim 7, wherein said performing target detection on the image in the cabin to be detected further comprises:将所述待检测的车舱内视频流中的每个车舱内图像作为待追踪图像,针对每个非首帧待追踪图像,基于该非首帧待追踪图像的前一帧待追踪图像中的所述待检测对象的位置信息以及该非首帧待追踪图像,确定所述待检测对象在该非首帧待追踪图像中的预测位置信息;Use each cabin image in the cabin video stream to be detected as a to-be-tracked image, and for each non-first frame to be tracked image, based on the previous frame of the non-first frame to-be-tracked image in the to-be-tracked image Determining the predicted position information of the object to be detected in the non-first frame to be tracked image of the position information of the object to be detected and the non-first frame to be tracked image;确定该非首帧待追踪图像是否为检测出待检测对象的待检测的车舱内图像;Determine whether the non-first frame of the to-be-tracked image is the to-be-detected image in the cabin where the object to be detected is detected;在确定该非首帧待追踪图像是检测出待检测对象的待检测的车舱内图像时,将检测出的位置信息作为待检测对象在该非首帧待追踪图像中的位置信息;When it is determined that the non-first frame of the to-be-tracked image is the to-be-detected cabin image in which the object to be detected is detected, use the detected position information as the position information of the to-be-detected object in the non-first frame of the to-be-tracked image;在确定该非首帧待追踪图像不是检测出待检测对象的待检测的车舱内图像时,将确定的预测位置信息作为待检测对象在该非首帧待追踪图像中的位置信息。When it is determined that the non-first frame of the to-be-tracked image is not the to-be-detected cabin image in which the object to be detected is detected, the determined predicted position information is used as the position information of the to-be-detected object in the non-first frame of the to-be-tracked image.
- 根据权利要求1-8任一所述的对象检测方法,其中,对所述待检测的车舱内图像进行目标检测由神经网络执行;8. The object detection method according to any one of claims 1-8, wherein the target detection on the image in the cabin to be detected is performed by a neural network;所述神经网络利用包含了待检测样本对象的车舱内样本图像和未包含待检测样本对象的车舱内样本图像训练得到。The neural network is trained by using sample images in the cabin containing the sample objects to be detected and sample images in the cabin that do not contain the sample objects to be detected.
- 一种对象检测装置,包括:An object detection device includes:图像获取模块,配置为获取待检测的车舱内图像;An image acquisition module, configured to acquire an image in the cabin to be detected;图像检测模块,配置为在车舱内人员减少的情况下,对所述待检测的车舱内图像进行目标检测,确定所述待检测的车舱内图像中是否存在待检测对象;An image detection module configured to perform target detection on the image in the cabin to be detected when the number of people in the cabin is reduced, and determine whether there is an object to be detected in the image in the cabin to be detected;提示模块,配置为响应于所述待检测的车舱内图像中存在待检测对象的状态的持续时长超过预设时长,发出提示信息。The prompting module is configured to send a prompt message in response to the duration of the state in which the object to be detected exists in the image of the cabin to be detected exceeds a preset period of time.
- 根据权利要求10所述的对象检测装置,其中,所述提示模块配置为响应于所述待检测的车舱内图像中存在待检测对象的状态的持续时长超过预设时长,发出提示信息,包括:The object detection device according to claim 10, wherein the prompt module is configured to send a prompt message in response to a state in which the object to be detected exists in the image in the cabin to be detected exceeds a preset period of time, including :在减少的车舱内人员为乘客的情况下,响应于所述待检测的车舱内图像中存在待检测对象的状态的持续时长超过第一预设时长,发出第一提示信息,所述第一提示信息用于提示乘客物品遗留;In the case that the reduced number of persons in the cabin is a passenger, in response to the duration of the state in which the object to be detected exists in the image to be detected in the cabin exceeds the first preset time period, a first prompt message is issued, and the first prompt message is sent. A reminder message is used to remind the passenger that the item is left behind;在减少的车舱内人员为驾驶员的情况下,响应于所述待检测的车舱内图像中存在待检测对象的状态的持续时长超过第二预设时长,发出第二提示信息,所述第二提示信息用于提示驾驶员物品遗留。In the case where the reduced number of persons in the cabin is the driver, in response to the duration of the state in which the object to be detected exists in the image to be detected in the cabin exceeds the second preset duration, a second prompt message is issued, the The second prompt message is used to prompt the driver that the item is left behind.
- 根据权利要求10或11所述的对象检测装置,其中,减少的车舱内人员为驾驶员和/或乘客,在图像检测模块确定所述待检测的车舱内图像中存在待检测对象后,在提示模块发出提示信息之前,所述图像检测模块还配置为:The object detection device according to claim 10 or 11, wherein the reduced number of people in the cabin is the driver and/or passenger, and after the image detection module determines that there is an object to be detected in the image in the cabin to be detected, Before the prompting module sends out the prompting information, the image detection module is also configured to:根据所述待检测对象在所述车舱中位置,确定所述待检测对象的归属人员;其中,所述待检测对象的归属人员为驾驶员和/或乘客。According to the position of the object to be detected in the cabin, the attribution of the object to be detected is determined; wherein the attribution of the object to be detected is the driver and/or passenger.
- 根据权利要求10至12任一所述的对象检测装置,其中,所述图像检测模块,配置为对所述待检测的车舱内图像进行目标检测,包括:The object detection device according to any one of claims 10 to 12, wherein the image detection module is configured to perform target detection on the image in the cabin to be detected, comprising:对所述待检测的车舱内图像进行特征提取,得到与多个通道中每个通道对应的第一 特征图;其中每个通道对应的第一特征图,为将待检测对象在该通道对应的图像特征类别下的特征进行增强处理后的特征图;Perform feature extraction on the image in the cabin to be detected to obtain a first feature map corresponding to each of the multiple channels; wherein the first feature map corresponding to each channel corresponds to the object to be detected in the channel The feature map after the enhancement processing of the features under the image feature category;针对每个所述通道,将该通道对应的第一特征图与其它通道分别对应的第一特征图进行特征信息融合,得到融合后的第二特征图;For each of the channels, perform feature information fusion on the first feature map corresponding to the channel and the first feature maps corresponding to the other channels to obtain a fused second feature map;基于所述融合后的第二特征图,检测所述待检测的车舱内图像中的所述待检测对象。Based on the fused second feature map, the object to be detected in the image in the cabin to be detected is detected.
- 根据权利要求13所述的对象检测装置,其中,所述图像检测模块,配置为针对每个所述通道,将该通道对应的第一特征图与其它通道分别对应的第一特征图进行特征信息融合,得到融合后的第二特征图,包括:The object detection device according to claim 13, wherein the image detection module is configured to perform, for each of the channels, a first feature map corresponding to the channel and first feature maps corresponding to other channels to perform feature information Fusion, the second feature map after fusion is obtained, including:针对进行特征信息融合的多个第一特征图,确定所述多个第一特征图所对应的权重矩阵;Determining a weight matrix corresponding to the plurality of first feature maps for performing feature information fusion;基于所述权重矩阵,对所述多个第一特征图的特征信息进行加权求和,得到包含各个融合特征信息的所述第二特征图。Based on the weight matrix, performing a weighted summation on the feature information of the multiple first feature maps to obtain the second feature map containing each fused feature information.
- 根据权利要求13所述的对象检测装置,其中,所述图像检测模块,配置为基于所述融合后的第二特征图,检测所述待检测的车舱内图像中的所述待检测对象,包括:The object detection device according to claim 13, wherein the image detection module is configured to detect the object to be detected in the image of the cabin to be detected based on the fused second feature map, include:基于所述融合后的第二特征图,确定设定个数的候选区域,每个候选区域包含设定个数的特征点;Determine a set number of candidate regions based on the fused second feature map, and each candidate region contains a set number of feature points;基于每个候选区域包含的特征点的特征数据,确定该候选区域对应的置信度;每个候选区域对应的置信度用于表征该候选区域中包含所述待检测对象的可信程度;Based on the feature data of the feature points contained in each candidate area, determine the confidence level corresponding to the candidate area; the confidence level corresponding to each candidate area is used to characterize the credibility that the candidate area contains the object to be detected;基于每个候选区域对应的置信度以及不同候选区域之间的重叠区域,从所述设定个数的候选区域中筛选出待检测对象对应的检测区域;所述检测区域用于标识所述待检测对象在所述待检测的车舱内图像中的位置。Based on the confidence level corresponding to each candidate area and the overlapping area between different candidate areas, the detection area corresponding to the object to be detected is screened out from the set number of candidate areas; the detection area is used to identify the to-be-detected object The position of the detection object in the image in the cabin to be detected.
- 根据权利要求10所述的对象检测装置,其中,所述图像获取模块,配置为获取所述待检测的车舱内图像,包括:The object detection device according to claim 10, wherein the image acquisition module, configured to acquire the image in the cabin to be detected, comprises:获取待检测的车舱内视频流;Obtain the video stream in the cabin to be detected;从所述待检测的车舱内视频流包含的连续多帧车舱内图像中,间隔提取得到所述待检测的车舱内图像。From the continuous multiple frames of the in-cabin images contained in the video stream in the cabin to be detected, the in-cabin images to be detected are extracted at intervals.
- 根据权利要求16所述的对象检测装置,其中,所述图像检测模块,配置为对所述待检测的车舱内图像进行目标检测,还包括:The object detection device according to claim 16, wherein the image detection module is configured to perform target detection on the image in the cabin to be detected, further comprising:将所述待检测的车舱内视频流中的每个车舱内图像作为待追踪图像,针对每个非首帧待追踪图像,基于该非首帧待追踪图像的前一帧待追踪图像中的所述待检测对象的位置信息以及该非首帧待追踪图像,确定所述待检测对象在该非首帧待追踪图像中的预测位置信息;Use each cabin image in the cabin video stream to be detected as a to-be-tracked image, and for each non-first frame to be tracked image, based on the previous frame of the non-first frame to-be-tracked image in the to-be-tracked image Determining the predicted position information of the object to be detected in the non-first frame to be tracked image of the position information of the object to be detected and the non-first frame to be tracked image;确定该非首帧待追踪图像是否为检测出待检测对象的待检测的车舱内图像;Determine whether the non-first frame of the to-be-tracked image is the to-be-detected image in the cabin where the object to be detected is detected;在确定该非首帧待追踪图像是检测出待检测对象的待检测的车舱内图像时,将检测出的位置信息作为待检测对象在该非首帧待追踪图像中的位置信息;When it is determined that the non-first frame of the to-be-tracked image is the to-be-detected cabin image in which the object to be detected is detected, use the detected position information as the position information of the to-be-detected object in the non-first frame of the to-be-tracked image;在确定该非首帧待追踪图像不是检测出待检测对象的待检测的车舱内图像时,将确定的预测位置信息作为待检测对象在该非首帧待追踪图像中的位置信息。When it is determined that the non-first frame of the to-be-tracked image is not the to-be-detected cabin image in which the object to be detected is detected, the determined predicted position information is used as the position information of the to-be-detected object in the non-first frame of the to-be-tracked image.
- 根据权利要求10至17任一所述的对象检测装置,其中,对所述待检测的车舱内图像进行目标检测由神经网络执行;The object detection device according to any one of claims 10 to 17, wherein the target detection on the image in the cabin to be detected is performed by a neural network;所述神经网络利用包含了待检测样本对象的车舱内样本图像和未包含待检测样本对象的车舱内样本图像训练得到。The neural network is trained by using sample images in the cabin containing the sample objects to be detected and sample images in the cabin that do not contain the sample objects to be detected.
- 一种电子设备,包括:处理器、存储器和总线,所述存储器存储有所述处理器可执行的机器可读指令,当电子设备运行时,所述处理器与所述存储器之间通过总线通信,所述机器可读指令被所述处理器执行时执行如权利要求1至9任一所述的对象检测方法。An electronic device, comprising: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus When the machine-readable instructions are executed by the processor, the object detection method according to any one of claims 1 to 9 is executed.
- 一种计算机可读存储介质,该计算机可读存储介质上存储有计算机程序,该计算机程序被处理器运行时执行如权利要求1至9任一所述的对象检测方法。A computer-readable storage medium with a computer program stored on the computer-readable storage medium, and when the computer program is run by a processor, the object detection method according to any one of claims 1 to 9 is executed.
- 一种计算机程序,包括计算机可读代码,当所述计算机可读代码在电子设备中运行时,所述电子设备中的处理器执行用于实现权利要求1至9任一所述的对象检测方法。A computer program, comprising computer-readable code, when the computer-readable code runs in an electronic device, a processor in the electronic device executes the method for implementing the object detection method of any one of claims 1 to 9 .
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2021558015A JP7224489B2 (en) | 2020-05-29 | 2020-12-21 | Target detection method, device, electronic device, storage medium and program |
KR1020217034510A KR20210149088A (en) | 2020-05-29 | 2020-12-21 | Object detection method, apparatus, electronic device, storage medium and program |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010477936.9 | 2020-05-29 | ||
CN202010477936.9A CN111652114B (en) | 2020-05-29 | 2020-05-29 | Object detection method and device, electronic equipment and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021238185A1 true WO2021238185A1 (en) | 2021-12-02 |
Family
ID=72352686
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/137919 WO2021238185A1 (en) | 2020-05-29 | 2020-12-21 | Object detection method and apparatus, electronic device, storage medium and program |
Country Status (4)
Country | Link |
---|---|
JP (1) | JP7224489B2 (en) |
KR (1) | KR20210149088A (en) |
CN (1) | CN111652114B (en) |
WO (1) | WO2021238185A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117036482A (en) * | 2023-08-22 | 2023-11-10 | 北京智芯微电子科技有限公司 | Target object positioning method, device, shooting equipment, chip, equipment and medium |
CN117152890A (en) * | 2023-03-22 | 2023-12-01 | 宁德祺朗科技有限公司 | Designated area monitoring method, system and terminal |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111626222A (en) * | 2020-05-28 | 2020-09-04 | 深圳市商汤科技有限公司 | Pet detection method, device, equipment and storage medium |
CN111652114B (en) * | 2020-05-29 | 2023-08-25 | 深圳市商汤科技有限公司 | Object detection method and device, electronic equipment and storage medium |
CN112818743B (en) * | 2020-12-29 | 2022-09-23 | 腾讯科技(深圳)有限公司 | Image recognition method and device, electronic equipment and computer storage medium |
CN113313090A (en) * | 2021-07-28 | 2021-08-27 | 四川九通智路科技有限公司 | Abandoned person detection and tracking method for abandoned suspicious luggage |
WO2023039781A1 (en) * | 2021-09-16 | 2023-03-23 | 华北电力大学扬中智能电气研究中心 | Method for detecting abandoned object, apparatus, electronic device, and storage medium |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106560836A (en) * | 2015-10-02 | 2017-04-12 | Lg电子株式会社 | Apparatus, Method And Mobile Terminal For Providing Object Loss Prevention Service In Vehicle |
CN107585096A (en) * | 2016-07-08 | 2018-01-16 | 奥迪股份公司 | Anti- forgetting system for prompting, method and vehicle |
CN108734056A (en) * | 2017-04-18 | 2018-11-02 | 深圳富泰宏精密工业有限公司 | Vehicle environmental detection device and detection method |
CN110659600A (en) * | 2019-09-19 | 2020-01-07 | 北京百度网讯科技有限公司 | Object detection method, device and equipment |
CN111652114A (en) * | 2020-05-29 | 2020-09-11 | 深圳市商汤科技有限公司 | Object detection method and device, electronic equipment and storage medium |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105894700A (en) * | 2015-11-02 | 2016-08-24 | 乐卡汽车智能科技(北京)有限公司 | Image-based in-vehicle moving object remote observing and warning device and method |
US9881234B2 (en) * | 2015-11-25 | 2018-01-30 | Baidu Usa Llc. | Systems and methods for end-to-end object detection |
US10303961B1 (en) * | 2017-04-13 | 2019-05-28 | Zoox, Inc. | Object detection and passenger notification |
US11106927B2 (en) * | 2017-12-27 | 2021-08-31 | Direct Current Capital LLC | Method for monitoring an interior state of an autonomous vehicle |
US10628667B2 (en) | 2018-01-11 | 2020-04-21 | Futurewei Technologies, Inc. | Activity recognition method using videotubes |
CN109345510A (en) * | 2018-09-07 | 2019-02-15 | 百度在线网络技术(北京)有限公司 | Object detecting method, device, equipment, storage medium and vehicle |
JP7208480B2 (en) * | 2018-10-12 | 2023-01-19 | 富士通株式会社 | Learning program, detection program, learning device, detection device, learning method and detection method |
CN110070566B (en) * | 2019-04-29 | 2021-07-30 | 武汉睿智视讯科技有限公司 | Information detection method and device, computer equipment and readable storage medium |
CN110610123A (en) * | 2019-07-09 | 2019-12-24 | 北京邮电大学 | Multi-target vehicle detection method and device, electronic equipment and storage medium |
CN110807385B (en) * | 2019-10-24 | 2024-01-12 | 腾讯科技(深圳)有限公司 | Target detection method, target detection device, electronic equipment and storage medium |
CN111144404B (en) * | 2019-12-06 | 2023-08-11 | 恒大恒驰新能源汽车科技(广东)有限公司 | Method, apparatus, system, computer device and storage medium for detecting legacy object |
-
2020
- 2020-05-29 CN CN202010477936.9A patent/CN111652114B/en active Active
- 2020-12-21 KR KR1020217034510A patent/KR20210149088A/en active Search and Examination
- 2020-12-21 JP JP2021558015A patent/JP7224489B2/en active Active
- 2020-12-21 WO PCT/CN2020/137919 patent/WO2021238185A1/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106560836A (en) * | 2015-10-02 | 2017-04-12 | Lg电子株式会社 | Apparatus, Method And Mobile Terminal For Providing Object Loss Prevention Service In Vehicle |
CN107585096A (en) * | 2016-07-08 | 2018-01-16 | 奥迪股份公司 | Anti- forgetting system for prompting, method and vehicle |
CN108734056A (en) * | 2017-04-18 | 2018-11-02 | 深圳富泰宏精密工业有限公司 | Vehicle environmental detection device and detection method |
CN110659600A (en) * | 2019-09-19 | 2020-01-07 | 北京百度网讯科技有限公司 | Object detection method, device and equipment |
CN111652114A (en) * | 2020-05-29 | 2020-09-11 | 深圳市商汤科技有限公司 | Object detection method and device, electronic equipment and storage medium |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117152890A (en) * | 2023-03-22 | 2023-12-01 | 宁德祺朗科技有限公司 | Designated area monitoring method, system and terminal |
CN117152890B (en) * | 2023-03-22 | 2024-03-08 | 宁德祺朗科技有限公司 | Designated area monitoring method, system and terminal |
CN117036482A (en) * | 2023-08-22 | 2023-11-10 | 北京智芯微电子科技有限公司 | Target object positioning method, device, shooting equipment, chip, equipment and medium |
Also Published As
Publication number | Publication date |
---|---|
KR20210149088A (en) | 2021-12-08 |
JP2022538201A (en) | 2022-09-01 |
CN111652114B (en) | 2023-08-25 |
JP7224489B2 (en) | 2023-02-17 |
CN111652114A (en) | 2020-09-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2021238185A1 (en) | Object detection method and apparatus, electronic device, storage medium and program | |
US10769459B2 (en) | Method and system for monitoring driving behaviors | |
CN102073841B (en) | Poor video detection method and device | |
US20170068863A1 (en) | Occupancy detection using computer vision | |
CN111439170B (en) | Child state detection method and device, electronic equipment and storage medium | |
Zhang et al. | Visual recognition of driver hand-held cell phone use based on hidden CRF | |
CN107862340A (en) | A kind of model recognizing method and device | |
CN111325141B (en) | Interactive relationship identification method, device, equipment and storage medium | |
CN102867188A (en) | Method for detecting seat state in meeting place based on cascade structure | |
CN104077776B (en) | A kind of visual background extracting method based on color space adaptive updates | |
US11023714B2 (en) | Suspiciousness degree estimation model generation device | |
CN104182769A (en) | Number plate detection method and system | |
JP2019106193A (en) | Information processing device, information processing program and information processing method | |
Papakis et al. | Convolutional neural network-based in-vehicle occupant detection and classification method using second strategic highway research program cabin images | |
CN112634558A (en) | System and method for preventing removal of an item from a vehicle by an improper party | |
Mathew et al. | Detecting new stable objects in surveillance video | |
JP6472504B1 (en) | Information processing apparatus, information processing program, and information processing method | |
WO2024001617A1 (en) | Method and apparatus for identifying behavior of playing with mobile phone | |
CN108985197B (en) | Automatic detection method for taxi driver smoking behavior based on multi-algorithm fusion | |
CN112381068B (en) | Method and system for detecting 'playing mobile phone' of person | |
CN115690883A (en) | Method for obtaining target training sample set and related device | |
EP4089629A1 (en) | Human detection device, human detection method, and recording medium | |
KR20200027078A (en) | Method and apparatus for detecting object independently of size using convolutional neural network | |
CN116152790B (en) | Safety belt detection method and device | |
CN113361340B (en) | Feature prompting method, device and computer storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2021558015 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 20217034510 Country of ref document: KR Kind code of ref document: A |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20937357 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 17.03.2023) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20937357 Country of ref document: EP Kind code of ref document: A1 |