CN104754311A - Device for identifying object with computer vision and system thereof - Google Patents
Device for identifying object with computer vision and system thereof Download PDFInfo
- Publication number
- CN104754311A CN104754311A CN201510205096.XA CN201510205096A CN104754311A CN 104754311 A CN104754311 A CN 104754311A CN 201510205096 A CN201510205096 A CN 201510205096A CN 104754311 A CN104754311 A CN 104754311A
- Authority
- CN
- China
- Prior art keywords
- image
- information
- unit
- real
- carry out
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Analysis (AREA)
Abstract
The invention relates to a device for identifying an object with computer vision. The device comprises an image acquisition unit, an image analyzing and processing unit, an image pyramid generating unit, a monitoring unit, a tracking unit and a motion unit, wherein the image acquisition unit is used for acquiring an image to be judged; the image analyzing and processing unit is used for removing unnecessary information of the image to be judged to generate a processed image; the image pyramid generating unit is used for generating an image pyramid according to the processed image; the monitoring unit is used for scanning each image layer of the image pyramid by using the feature information of an object to perform classification judgment, and positioning an object to be monitored in the image to be judged; the tracking unit is used for generating tracking information according to object information provided by the monitoring unit; the motion unit is used for tracking or avoiding the object to be monitored according to the tracking information.
Description
Technical field
The present invention is
the device that computer vision identifies object and system thereof, the present invention can fast monitored to object, especially there is the object of given configuration profile.
Technical background
Pedestrian monitoring is a key issue in computer vision, but in current developed computer vision supervisory control system, be adopt fixed position video camera mostly, and its background often remains unchanged.Simultaneously, although this kind of supervisory control system can judge whether to occur pedestrian or vehicle, but can not follow determinand and automatically carry out real-time tracing, therefore cannot be applied in the aspects such as such as sweeping robot, or other need running gear environmental objects being carried out to Real-Time Monitoring and tracking.
Secondly, although utilize the mode such as radar wave or infrared ray also can reach the object of surrounding being monitored, but this kind of monitoring mode need expend the long duration and carry out computing and constructing environment, therefore by unfavorable application of following the trail of at Real-Time Monitoring, and the structure of static environment often can only be applied in.
In addition, global positioning system (GPS) is utilized to arrange in pairs or groups suitable algorithm, although can the mobile status of the object such as evaluation prediction such as vehicle, the method must be arranged in pairs or groups the use of global positioning system, and cannot be applied in home environment or pedestrian monitoring.
Summary of the invention
Main purpose of the present invention is for providing a kind of computer vision that utilizes to carry out device and the application thereof of Real-Time Monitoring and tracking to certain objects, for avoiding or tracking objects.
The present invention will provide a kind of device utilizing computer vision to carry out Real-Time Monitoring and tracking, comes Real-Time Monitoring and tracking objects.Device comprises image collection unit, image analysing computer processing unit, image pyramid generation unit, monitoring means and tracing unit.Wherein, image collection unit can gather image to be determined from environment.Image analysing computer processing unit is the unnecessary information for removing image to be determined, thus produces the image after process.Image pyramid generation unit is according to image after process, produces image pyramid.Monitoring means utilizes object features information, and the pyramidal each image bearing layer of scan-image carries out classification and judges, to produce real-time object information.Tracing unit is according to this real-time object information, produces tracked information.
By the specific implementation of aforementioned techniques means, the present invention can be applicable to shorter automation action robot, such as: ball picking robot, pet robot, sweeping robot etc.Be described in detail as follows: when ball picking robot is when carrying out picking up ball, the region at sportsman place can be monitored simultaneously, just can avoid this region, allow the action simultaneously of sportsman and robot; And with regard to pet robot, it can monitor the position of the mankind, and carry out the tracking mankind, and carry out interaction with the mankind; Sweeping robot while carrying out sweeping the floor, the existence monitoring pedestrian that also can be similar to ball picking robot, so select avoid.
Accompanying drawing explanation
Fig. 1 is that the present invention utilizes computer vision, carries out the block schematic diagram of the system of Real-Time Monitoring and tracking objects.
Fig. 2 to Fig. 5 the present invention utilizes computer vision, carries out the system application schematic diagram of Real-Time Monitoring and tracking objects.
Fig. 6 is the flow chart that system carries out Real-Time Monitoring and tracking objects.
Fig. 7 system, when monitoring multiple object, carries out the flow chart of real-time tracing.
Embodiment
Concise and to the point framework of the present invention, as shown in fig. 1, this system has image collection unit 110, image analysing computer processing unit 120, image pyramid generation unit 130, training unit 140, monitoring means 150, tracing unit 160 and moving cell 170.
Image collection unit 110 is used in gather continuous print image to be determined, to judge whether there is object to be monitored from environment.
According to the image to be determined that image collection unit 110 gathers, image analysing computer processing unit 120 is used to remove the unnecessary information in image to be determined, to produce the rear image of process.Image pre-processing module 120 carries out image gray scale operation to image to be determined and wavelet conversion operates.Image gray scaleization operation is the color information removing image, and wavelet conversion operation is then the resolution reducing image.A figure in Fig. 2 is that show image pre-processing module carries out the schematic diagram of wavelet conversion to GTG image with B figure, and A figure is the image before conversion, and B figure is the image after conversion.The object of the operation of these two kinds of image procossing is to reduce image Global Information amount.
Image pyramid generation unit 130 is to produce image pyramid according to image after process.As shown in Figure 3, image pyramid is the image after foundation process, sets up multiple resolution image bearing layer decrescence continuously.Image pyramid generation unit 130 is the resolution produced by image analysing computer processing unit 120 is the image of 80X60, is decomposed into four resolution image bearing layer decrescence.The quantity of the image bearing layer of resolution and image pyramid, the demand such as feature complexity, system operations ability, Monitor in time of visual object to be monitored and adjusting.
Monitoring means 150 is according to object features information, carrys out the pyramidal each image bearing layer of scan-image and carry out classification to judge, to produce real-time object information.Object features information judges, in each image bearing layer of image pyramid, whether to there is object to be monitored with for monitoring means 150.
Training unit 140 is according to multiple object training sample and non-object training sample, produces aforesaid object characteristic information.
Tracing unit 160 is after monitoring means 150 determines to monitor object, then according to the object image model that locating information and the monitoring means 150 of the object in image to be determined is set up, produces tracked information.In object tracking process, the information such as the motion of object, edge and color can be utilized as the feature of similitude comparison.The tracked information that tracing unit 160 produces after computing, can be used for merely the motion direction of tracking objects, bumps against, also can produce warning at object before device and then when may collide with avoiding device (such as take action robot) and object.
Moving cell 170 is the tracked information produced according to tracing unit 160, and carrys out tracking objects depending on demand or avoid object.
Fig. 6 is for the system of Fig. 5 carries out the flow chart of the method for Real-Time Monitoring and tracking.As shown in FIG., first, as shown in step T510, pending image is gathered from environment.Subsequently, as shown in step T511 and T512, successively image gray scale and wavelet conversion are carried out to this pending image, to produce the rear image of process.Then, as shown in step T514, after processing according to this, image produces the image bearing layer (i.e. image pyramid) that multiple resolution is successively decreased.Next, as shown in step T516 and T518, remove the noise of image bearing layer with gaussian filtering, and strengthen its image contrast with histogram, so that subsequent classification judges.Then, as shown in step T520, each image bearing layer of the image pyramid after process is scanned, and utilize object features information to carry out classification judgement (namely determining whether object image) to the image scanned.As shown in Figure 4, with regard to example, this object features information carrys out comfortable neural network (neuron of hidden layer especially wherein and weighted value) after training.Output valve according to neural network is closer to 0 or 1, can judge whether the image scanned is object image.
As shown in Figure 6, as shown in step T530, in the training program of back propagation neural network, first object training sample and non-object training sample are adjusted to default resolution sizes.As shown in step T532, sequentially be provided to neural network to carry out back propagation neural network training by these samples, and progressively adjust the parameter of neural network, the difference of reduction network output valve and target output value, to promote the accuracy that object image judges.After completing this training step, namely the relevant parameter of this neural network can be used as and carries out classifying the object features information needed for judging also for step T520.As shown in step T521 and T522, if the result that classification judges exists object image, just produce locating information immediately, and set up object image model, use for follow-up object tracking.On the contrary, as shown in step T524, if do not monitor object image, this monitoring flow process stops immediately.As shown in step T540, after confirmation monitors object image, be confirmed whether immediately to follow the trail of object.If need follow the trail of object, then as shown in step T542, the present embodiment is the technology adopting particulate removal device, according to the locating information acquired by step T522 and object image model (object), and gather in suitable window image (material standed for) in follow-up image to be determined, carry out similarity-rough set to produce tracked information, to reach the object of dynamic tracing object.Follow the trail of object if do not need, namely this flow process only comes to an end.Fig. 6 Fig. 6 illustrates for the situation monitoring single object.
Fig. 7 then shows and monitors method for tracing practiced after multiple object.Accept the step T540 of the 6th figure, as shown in step T544, after determining to need to follow the trail of object, first judge whether there is multiple object in image to be determined.If only there is single object, then, as shown in step T542, can utilize as modes such as particulate removal devices, the similarity-rough set between the object set up by step T522 and material standed for, to produce tracked information.If monitor multiple object in image to be determined, as shown in step T546, first need judge between each object, whether to there is the situation of covering.If not, as shown in step T542, then take the processing mode as single object, each object is followed the trail of respectively, can utilize as the modes such as particulate removal device by step T522 the similarity-rough set of each material standed in each object of setting up and next image to be determined, to produce the tracked information of each tracked object.If the situation of covering, be then as shown in step T548, carry out covering process, and add moving direction feature, follow the trail of again, can utilize as the modes such as particulate removal device by step T522 the object feature set up add T548 the moving direction feature set up carry out the similarity-rough set with each material standed in next image to be determined, to produce the tracked information of each object.Finally accept back T540 to judge whether to continue to follow the trail of.
Via aforesaid structural design and explanation, allow the present invention can fast monitored to object, the in particular, for example object with given configuration profile of the leg of pedestrian, and following the trail of.For the monitoring of pedestrian lower leg, by the monitoring of pedestrian lower leg, the position of pedestrian can be judged, and not need to monitor the overall more complicated profile of pedestrian.In addition, the height of the action robot existed due to current market is usually less than the height of normal person, and the presentation content acquired by it also can be limited in the height of action robot.Therefore, the present invention is especially useful in the action robot of this type, such as sweeping robot or other need to carry out in the running gear of Real-Time Monitoring and tracking environmental objects.
Claims (5)
1. computer vision device that object is identified and a system thereof, it is characterized in that, device comprises:
Image collection unit, for gathering image to be determined;
Image analysing computer processing unit, for removing the unnecessary information of image to be determined, to produce the rear image of process;
Image pyramid generation unit, according to image after process, produces image pyramid;
Monitoring means, utilizes object features information, the pyramidal each image bearing layer of scan-image, and carries out classification judgement, to produce real-time object information;
Tracing unit, according to real-time object information, produces tracked information.
2. carry out the device of Real-Time Monitoring and tracking objects according to the computer vision that utilizes of claims 1, it is characterized in that, comprise training unit, training unit is useful according to multiple training sample, and by back propagation neural network, produces object features information.
3. carry out the device of Real-Time Monitoring and tracking objects according to the computer vision that utilizes of claims 1, it is characterized in that, comprise for foundation tracked information, and follow the trail of or avoid the moving cell of object.
4. carry out the device of Real-Time Monitoring and tracking objects according to the computer vision that utilizes of claims 1, it is characterized in that, tracing module comprises particulate removal device, its be according to real-time object information in follow-up image at least to be determined, carry out similarity-rough set to produce tracked information, and object information includes locating information and object image model in real time.
5. carry out the device of Real-Time Monitoring and tracking objects according to the computer vision that utilizes of claims 1, it is characterized in that, monitoring modular is with default window size, come the pyramidal each image bearing layer of scan-image with carry out classification judge, and then in image to be determined positioning object.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510205096.XA CN104754311A (en) | 2015-04-28 | 2015-04-28 | Device for identifying object with computer vision and system thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510205096.XA CN104754311A (en) | 2015-04-28 | 2015-04-28 | Device for identifying object with computer vision and system thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104754311A true CN104754311A (en) | 2015-07-01 |
Family
ID=53593338
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510205096.XA Pending CN104754311A (en) | 2015-04-28 | 2015-04-28 | Device for identifying object with computer vision and system thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104754311A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106773820A (en) * | 2016-12-02 | 2017-05-31 | 北京奇虎科技有限公司 | Robot interactive approach, device and robot |
| WO2017101036A1 (en) * | 2015-12-16 | 2017-06-22 | Intel Corporation | Fully convolutional pyramid networks for pedestrian detection |
| CN107671896A (en) * | 2017-05-19 | 2018-02-09 | 重庆誉鸣科技有限公司 | Fast vision localization method and system based on SCARA robots |
| WO2020097837A1 (en) * | 2018-11-15 | 2020-05-22 | Lingdong Technology (Beijing) Co.Ltd | System and method for real-time supervised machine learning in on-site environment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101739658A (en) * | 2008-11-06 | 2010-06-16 | 索尼株式会社 | Image processing apparatus, image processing method, and program |
| CN102270348A (en) * | 2011-08-23 | 2011-12-07 | 中国科学院自动化研究所 | Method for tracking deformable hand gesture based on video streaming |
| CN102426480A (en) * | 2011-11-03 | 2012-04-25 | 康佳集团股份有限公司 | Man-machine interactive system and real-time gesture tracking processing method for same |
| CN102640185A (en) * | 2009-10-20 | 2012-08-15 | 全浸公司 | Method, computer program, and device for hybrid tracking of real-time representations of objects in image sequence |
| CN104375504A (en) * | 2014-09-12 | 2015-02-25 | 中山大学 | Running accompanying robot and tracking control strategy and movement control method for running accompanying robot |
-
2015
- 2015-04-28 CN CN201510205096.XA patent/CN104754311A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101739658A (en) * | 2008-11-06 | 2010-06-16 | 索尼株式会社 | Image processing apparatus, image processing method, and program |
| CN102640185A (en) * | 2009-10-20 | 2012-08-15 | 全浸公司 | Method, computer program, and device for hybrid tracking of real-time representations of objects in image sequence |
| CN102270348A (en) * | 2011-08-23 | 2011-12-07 | 中国科学院自动化研究所 | Method for tracking deformable hand gesture based on video streaming |
| CN102426480A (en) * | 2011-11-03 | 2012-04-25 | 康佳集团股份有限公司 | Man-machine interactive system and real-time gesture tracking processing method for same |
| CN104375504A (en) * | 2014-09-12 | 2015-02-25 | 中山大学 | Running accompanying robot and tracking control strategy and movement control method for running accompanying robot |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017101036A1 (en) * | 2015-12-16 | 2017-06-22 | Intel Corporation | Fully convolutional pyramid networks for pedestrian detection |
| US10140522B2 (en) | 2015-12-16 | 2018-11-27 | Intel Corporation | Fully convolutional pyramid networks for pedestrian detection |
| CN106773820A (en) * | 2016-12-02 | 2017-05-31 | 北京奇虎科技有限公司 | Robot interactive approach, device and robot |
| CN106773820B (en) * | 2016-12-02 | 2019-07-19 | 北京奇虎科技有限公司 | Robot interactive approach, device and robot |
| CN107671896A (en) * | 2017-05-19 | 2018-02-09 | 重庆誉鸣科技有限公司 | Fast vision localization method and system based on SCARA robots |
| CN107671896B (en) * | 2017-05-19 | 2020-11-06 | 重庆誉鸣科技有限公司 | Rapid visual positioning method and system based on SCARA robot |
| WO2020097837A1 (en) * | 2018-11-15 | 2020-05-22 | Lingdong Technology (Beijing) Co.Ltd | System and method for real-time supervised machine learning in on-site environment |
| US11092968B2 (en) | 2018-11-15 | 2021-08-17 | Lingdong Technology (Beijing) Co. Ltd | System and method for real-time supervised machine learning in on-site environment |
| US11682193B2 (en) | 2018-11-15 | 2023-06-20 | Lingdong Technology (Beijing) Co. Ltd. | System and method for real-time supervised machine learning in on-site environment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wu et al. | Hybrid deep learning architecture for rail surface segmentation and surface defect detection | |
| CN108805015B (en) | Crowd abnormity detection method for weighted convolution self-coding long-short term memory network | |
| CN114693615A (en) | Deep learning concrete bridge crack real-time detection method based on domain adaptation | |
| CN104754311A (en) | Device for identifying object with computer vision and system thereof | |
| US20150279036A1 (en) | Side window detection in near-infrared images utilizing machine learning | |
| TW201405486A (en) | Real time detecting and tracing objects apparatus using computer vision and method thereof | |
| CN108416780B (en) | Object detection and matching method based on twin-region-of-interest pooling model | |
| Farley et al. | Real time IP camera parking occupancy detection using deep learning | |
| CN112800860A (en) | Event camera and visual camera cooperative high-speed scattered object detection method and system | |
| CN101576952A (en) | Method and device for detecting static targets | |
| Wang | Joint random field model for all-weather moving vehicle detection | |
| WO2008139399A2 (en) | Method of determining motion-related features and method of performing motion classification | |
| CN105976398A (en) | Daylight fire disaster video detection method | |
| CN111008994A (en) | Moving target real-time detection and tracking system and method based on MPSoC | |
| CN108710879B (en) | Pedestrian candidate region generation method based on grid clustering algorithm | |
| CN108537825B (en) | Target tracking method based on transfer learning regression network | |
| CN116977909B (en) | Deep learning fire intensity recognition method and system based on multi-modal data | |
| CN113900101A (en) | Obstacle detection method and device and electronic equipment | |
| CN111784750A (en) | Method, device and equipment for tracking moving object in video image and storage medium | |
| CN115984568A (en) | Target detection method in haze environment based on YOLOv3 network | |
| Yang et al. | A novel vision-based framework for real-time lane detection and tracking | |
| CN114529816A (en) | Tea garden pest and disease monitoring system based on image recognition | |
| CN114627183A (en) | Laser point cloud 3D target detection method | |
| Xenya et al. | Intruder detection with alert using cloud based convolutional neural network and Raspberry Pi | |
| CN112967335A (en) | Bubble size monitoring method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20150701 |
|
| WD01 | Invention patent application deemed withdrawn after publication |