Summary of the invention
In view of this, the object of the present invention is to provide a moving target detecting method and system, this method has solved the problem that interference, the shadow of moving target, the shake of video camera and the existence of blocking and blocking mutually phenomenon certainly of moving target etc. of variation, the background clutter motion of illumination bring, and has realized the moving object detection under the complex scene.
For achieving the above object, technical scheme of the present invention is achieved in that
The invention provides a kind of moving target detecting method, this method comprises the steps:
Obtain video, obtain video content obtaining scene image, and set up background model;
Pretreatment image is eliminated the influence of scene image to background model;
Marked region carries out foreground segmentation according to background model to scene image, and marks connected region;
Maintenance state is judged moving object detection system present located state, makes respective handling, and does abnormality detection where necessary;
Strengthen the zone, use shadow Detection, highlighted detection and tree filtering, reject the false areas of shade, highlighted and leaf swing;
Division merges and divides processing with merging zone, the constraint that the use background model provides and the priori of people and vehicle model to the zone, to solve target over-segmentation and the mutual occlusion issue of target.
According to the present invention, described pretreatment image comprises: Filtering Processing and global motion compensation; Wherein, described Filtering Processing comprises: image is carried out noise filtering processing, picture smooth treatment; Described global motion compensation is the image overall motion that compensation causes owing to the slight swing of camera, and in global motion compensation, motion model comprises translation, rotation, zoom.
By the regional luminance difference IDS of positive and negative 5 pixels around the rectangular area at following formula calculating prospect place, obtain distance, delta x, the Δ y of image translation in the global motion compensation, formula is as follows:
Wherein, s
xRepresent regional starting point x coordinate, s
yRepresent regional starting point y coordinate, I
(x, y)(t) expression current frame image gray scale, I
(x, y)(t-1) expression previous frame gradation of image;
In like manner calculate Δ x, the Δ y in other four zones, obtain the mean value of Δ x, Δ y at last;
Image is carried out image after translation is compensated according to the mean value of Δ x, Δ y.
According to the present invention, described marked region comprises the steps:
Foreground segmentation is cut apart scene image based on background model, to obtain the bianry image of prospect;
Morphology is handled, and uses the method for mathematical morphology to handle described bianry image, with the less false areas of removal area, and the bigger zone of filling area; And
Connected component labeling is with the zones of different in the method mark Same Scene of connected domain, to distinguish different target areas.
According to the present invention, described maintenance state comprises condition judgement and abnormality detection.
Described condition judgement is to judge moving object detection system present located state, makes respective handling; Surpass threshold value 1 when scene stabilization time, system enters duty by init state; Surpass threshold value 2 when the scene change time, system enters init state by duty.
Described abnormality detection is in the vision signal serious interference, and carries out when artificial situation of blocking camera is arranged; Judge according to the edge matching value of twice background and the shortest time of background initialization success, if the numerical value that the edge of the background of present frame and background model is complementary surpasses threshold value 4 less than the shortest time of threshold value 3 or background initialization success, then think abnormal occurrence.
According to the present invention, described enhancing zone comprises: shadow Detection, highlighted detection, tree filtering.
Shadow Detection is at each connected region, calculates the average of the pixel value in this connected region respectively, and with this average as threshold value, judge the shadow region that this is regional, then with the shadow region filtering, if pixel value less than described threshold value, then is judged to be shade;
Highlighted detection is to be used for detected image whether to be in highlighted state, if, then carrying out luminance compensation, luminance compensation makes that the average of pixel value of image is 128;
Tree filtering is used for the swing leaf of detected image and swings the leaf shade, and with its filtering from foreground image; Wherein:
The wobble detection leaf is judged realization according to one of following two features: (1) movement locus is followed the tracks of, when target corresponding region in the movement locus point belongs to the part of moving region area less than the threshold value 5 of moving region area, think that then this target is the swing leaf; (2) amplitude of center of mass motion when the change in displacement of target barycenter in the adjacent track point surpasses the threshold value 6 of target width, thinks that then this target is the swing leaf.
The method of wobble detection leaf shade is: add up respectively before and after the expansive working should the zone in before and after the expansive working pixel value be the number of the point of " 1 ", and calculate their ratio, if this ratio, thinks then that this zone is the zone of swinging the leaf shade less than threshold value 7.
According to the present invention, described division is based on the described processing procedure that strengthens the zone with merging the zone, judges whether adjacent two zones are same target areas; If belong to same target area, then these two zones are merged; Otherwise, with its division; Wherein, adjacent two zones are meant the zone of edges of regions distance less than threshold value 8.
According to a further aspect in the invention, the present invention also provides a kind of moving object detection system, and described moving object detection system comprises:
Obtain video module, be used to obtain video content obtaining scene image, and set up background model;
The pretreatment image module is used to eliminate the influence of scene image to background model;
The marked region module is used for according to background model scene image being carried out foreground segmentation, and marks connected region;
The maintenance state module is used to judge moving object detection system present located state, makes respective handling, and does abnormality detection where necessary;
Strengthen regions module, be used to use shadow Detection, highlighted detection and tree filtering, reject the false areas of shade, highlighted and leaf swing; With
Division with merge regions module, be used to use the constraint that background model provides and the priori of people and vehicle model the zone to be merged and divide processing, with solution target over-segmentation and the mutual occlusion issue of target.
According to the present invention, described pretreatment image module comprises: the Filtering Processing module is used for image is done noise filtering, conventional processing such as level and smooth, to remove the noise spot in the image; With the global motion compensation module, be used to compensate global motion owing to image translation that the slight swing of camera causes, rotation, zoom.
According to the present invention, described marked region module comprises: the foreground segmentation module is used for based on background model scene image being cut apart, to obtain the bianry image of prospect; The morphology processing module is used to use the method for mathematical morphology to handle described bianry image, with the less false areas of removal area, and the bigger zone of filling area; With the connected component labeling module, be used for zones of different, to distinguish different target areas by the method mark Same Scene of connected region.
According to the present invention, described maintenance state module comprises: condition judgement module 641 is used to judge moving object detection system present located state, and makes respective handling; With abnormality detection module 642, be used for comprising the vision signal serious interference have artificial the execution when blocking the camera situation to detect.
According to the present invention, described enhancing regions module comprises: the shadow Detection module, be used for detecting the shadow region that foreground image comprises people, car shade, and with the filtering of detected shadow region; Highlighted detection module is used for detected image and whether is in highlighted state, if then carry out luminance compensation; With the tree filtration module, be used for the leaf and the shade thereof of the swing of detected image, and with its filtering from foreground image.
According to scheme provided by the invention, realized motion target detection, solved problems such as flating, brightness variation, shade, leaf swing.
Embodiment
Below, introduce embodiments of the present invention with reference to the accompanying drawings.
Fig. 1 is the framework synoptic diagram of moving target detecting method among the present invention, as shown in Figure 1.Moving target detecting method comprises:
Step 1: obtain video, obtain video content obtaining scene image, and set up background model;
Step 2: pretreatment image, eliminate the influence of scene image to background model;
Step 3: marked region, carry out foreground segmentation according to background model to scene image, and mark connected region;
Step 4: maintenance state, judge moving object detection system present located state, make respective handling, and do abnormality detection where necessary;
Step 5: strengthen the zone, use shadow Detection, highlighted detection and tree filtering, reject the false areas of shade, highlighted and leaf swing;
Step 6: division merges and divides processing with merging zone, the constraint that the use background model provides and the priori of people and vehicle model to the zone, to solve target over-segmentation and the mutual occlusion issue of target.
At first step 1 is to obtain the content of video 1, promptly realizes that by video capture device this video capture device can be visible spectrum, near infrared or a thermal camera.Described near infrared and thermal camera allow to use under the low light level of no additional light rays.The described background model of setting up is upgraded in maintenance state 4 afterwards at first with first frame scene image model as a setting.
Step 2 is that pretreatment image is to eliminate its influence to background model then.Fig. 2 is the framework synoptic diagram of pretreatment image 2 among the present invention, as shown in Figure 2.Pretreatment image comprises: Filtering Processing and global motion compensation.
Described Filtering Processing is meant does noise filtering, conventional processing such as level and smooth to image, to remove the noise spot in the image.Filtering Processing can be passed through following document and realize, as: " image denoising mixed filtering method [J]. Chinese image graphics journal, 2005; 10 (3) ", " the improvement mean filter algorithm [J] of self-adaptation center weighting. Tsing-Hua University's journal (natural science edition), 1999,39 (9) ".
Global motion compensation is meant the image overall motion that compensation causes owing to the slight swing of camera.In global motion compensation, motion model is exactly the various motions that reflect video camera basically, comprises translation, rotation, zoom etc.The method of global motion compensation is: based on the motion compensation of region unit coupling, and four region units that in image, draw, the length and width of region unit require the zone to cover relatively fixing background between the 32-64 pixel, such as building, perhaps fixed background.
The method of conventional global motion compensation is as follows: the rectangular area size of supposing the prospect place is m * n, calculates this zone regional luminance difference IDS of positive and negative 5 pixels on every side, and formula is as follows:
Wherein, s
xRepresent regional starting point x coordinate, s
yRepresent regional starting point y coordinate, I
(x, y)(t) expression current frame image gray scale, I
(x, y)(t-1) expression previous frame gradation of image.
Can obtain the position of minimum brightness difference institute corresponding region like this, calculate this regional location variation Δ x, Δ y.In like manner calculate Δ x, the Δ y in other four zones, obtain the mean value of Δ x, Δ y at last; Image is carried out image after translation is compensated according to the mean value of Δ x, Δ y.
Step 3 is marked regions, according to background model scene image is carried out foreground segmentation, and marks connected region.As shown in Figure 3, Fig. 3 is the schematic flow sheet of marked region among the present invention, and the marked region idiographic flow is as follows: at first carry out foreground segmentation 31; Morphology handles 32 then; Carry out connected component labeling 33 again.
Wherein, foreground segmentation 31 is meant based on background model to be cut apart scene image, to obtain the bianry image of prospect.Particularly, the pixel value of the pixel that scene image is corresponding with background model subtracts each other, if this result, then is designated as " 1 " greater than preset threshold to be expressed as the foreground point; If less than threshold value, then be designated as " 0 " to be expressed as background dot, obtain the bianry image of prospect thus.
Morphology is handled 32 and is meant that the methods of using mathematical morphology promptly by corrosion after expansion earlier, handle described bianry image, removing the less false areas of area, and the bigger zone of filling area.Wherein, what corrosion parameter selected is 3 * 3 templates, and what the expansion parameter was selected is 3 * 3 templates.
Connected component labeling 233 typically refers to the zones of different in the method mark Same Scene of connected domain, to distinguish different target areas.Method for marking connected region can be realized by four connected domain methods or eight connected domain methods.Four methods that connect the connection mark of/eight connected domains are: at first, morphology is handled the 223 images executions of obtaining line by line scan, find first point in a unmarked zone, this point of mark; Check that four of this point connects/eight and connects territory points and mark and satisfy connectivity platform, and the point that is not labeled as yet, simultaneously the gauge point that increases newly is noted seed points as " region growing ".In follow-up labeling process, constantly from the array of record seed points, take out a seed, implement above-mentioned operation, so circulation is empty up to the array that writes down seed points, a connected component labeling finishes.Follow the next unlabelled zone of mark, all connected regions of handling 223 images that obtain up to morphology all are labeled again.
In the flow process of marked region, single zone and single target are not one to one.Owing to block situation, a zone has comprised a plurality of people or car; Because prospect is similar to background, a target may be a plurality of zones by over-segmentation; Because the influence of illumination may comprise shade and highlight regions in the zone; Because some non-interested motions, as leaf swing with ripple etc., also can produce false foreground area.These problems all are that the background model method is intrinsic, need solve in subsequent step.
Step 4 is maintenance states 4, promptly judges moving object detection system present located state, makes respective handling, and does abnormality detection where necessary.Fig. 4 is the framework synoptic diagram of maintenance state among the present invention, as shown in Figure 4.Maintenance state comprises: condition judgement and abnormality detection.
Condition judgement is meant judges moving object detection system present located state, and makes respective handling.Judge that moving object detection system present located state mainly judges by scene stabilization time, scene change time.Surpass threshold value 1 when scene stabilization time, system enters duty by init state; Surpass threshold value 2 when the scene change time, system enters init state by duty.
Described threshold value 1 was preferably between 0.5~2 second.Described threshold value 2 was preferably between 5~20 seconds.
When being in described duty, continue to carry out next operation, background model is constant.When being in described init state, rebuliding background model, and make abnormality detection where necessary.Described rebuliding during the background model can be carried out the zone by the frame-to-frame differences point-score and be detected realization.The frame-to-frame differences point-score subtracts each other the realization that takes absolute value by two two field pictures.
Abnormality detection is to comprise the vision signal serious interference where necessary, has the artificial situation such as camera of blocking to carry out.Judge according to the edge matching value of twice background and the shortest time of background initialization success.If the numerical value that the edge of the background of present frame and background model is complementary surpasses threshold value 4 less than the shortest time of threshold value 3 or background initialization success, then think abnormal occurrence.
Described threshold value 3 is preferably between 30~50.Described threshold value 4 was preferably between 6~20 seconds.
Step 5 is to strengthen the zone, uses shadow Detection, highlighted detection and tree filtering, rejects the false areas of shade, highlighted and leaf swing.Fig. 5 is for strengthening the framework synoptic diagram in zone, as shown in Figure 5 among the present invention.Described enhancing zone comprises: shadow Detection, highlighted detection, tree filtering.
Shadow Detection is used for detecting the shadow region of foreground image, comprises the shade of people, car, and with the filtering of detected shadow region.Described shadow Detection is at each connected region, calculates the average of the pixel value in this connected region respectively, and with this average as threshold value, judge the shadow region that this is regional, then with the shadow region filtering.The shade decision rule is as follows: if pixel value less than described threshold value, then is judged to be shade.
Highlighted detection is used for detected image and whether is in highlighted state (highlighted state refers to that promptly the pixel value of image is generally too high), if then carry out luminance compensation.Luminance compensation realizes by luminance proportion, makes that the average of pixel value of image is 128.
Tree filtering is used for the leaf and the shade thereof of the swing of detected image, and with its filtering from foreground image.
The wobble detection leaf is judged realization according to one of following two features: (1) movement locus is followed the tracks of, when target corresponding region in the movement locus point belongs to the part of moving region area less than the threshold value 5 of moving region area, think that then this target is the swing leaf; For example target has 10 tracing points, and moving in zone for once corresponding in these tracing points, then this target is considered as swinging leaf, with this target filtering.(2) amplitude of center of mass motion, if the amplitude of the center of mass motion of a certain target suddenlys change, think that then this target is the swing leaf, promptly when the change in displacement of target barycenter in the adjacent track point surpasses the threshold value 6 of target width, think that then this target is the leaf of swing, with this target filtering.
Described threshold value 5 is preferably between 5%~15%.Described threshold value 6 is preferably between 1.5~2.5.
The detection of swing leaf shade is to realize by the closeness of putting in the surveyed area, the method of wobble detection leaf shade is: the number (in promptly should the zone before and after the expansive working pixel value be the number of the point of " 1 ") of adding up the point in the zone before and after the expansive working respectively, and calculate their ratio, if this ratio is less than threshold value 7, think that then this zone is the zone of swing leaf shade, and should the zone filtering.
Described threshold value 7 is preferably between 40%~60%.
Step 6 is divisions and merge the zone, promptly uses the constraint that background model provides and the priori of people and vehicle model the zone to be merged and divide processing, with solution target over-segmentation and the mutual occlusion issue of target.Described division is based on above-mentioned enhancing zone 205 processing procedures with the method that merges the zone, judges that adjacent two zones are same target areas, or the different target zone.If belong to same target area, then these two zones are merged; Otherwise, with its division.Wherein, adjacent two zones are meant the zone of edges of regions distance less than threshold value 8, the zone of same zone index mark unanimity, the inconsistent zone of different target zone index mark.
Described threshold value 8 is preferably between 3~7 pixels.
The present invention also provides a kind of moving object detection system 6, and Fig. 6 is the structural representation of moving object detection system of the present invention, as shown in Figure 6.Described moving object detection system comprises to be obtained video module 61, pretreatment image module 62, marked region module 63, maintenance state module 64, strengthen regions module 65 and division and merges regions module 66.Wherein, obtain video module 61, be used to obtain video content obtaining scene image, and set up background model; Pretreatment image module 62 is used to eliminate the influence of scene image to background model; Marked region module 63 is used for according to background model scene image being carried out foreground segmentation, and marks connected region; Maintenance state module 64 is used to judge moving object detection system present located state, makes respective handling, and does abnormality detection where necessary; Strengthen regions module 65, be used to use shadow Detection, highlighted detection and tree filtering, reject the false areas of shade, highlighted and leaf swing; With division with merge regions module 66, be used to use the constraint that scene image provides and the priori of people and vehicle model the zone to be merged and divide processing, with solution target over-segmentation and the mutual occlusion issue of target.
Fig. 7 is the structural representation of the pretreatment image module of moving object detection system of the present invention, as shown in Figure 7.Pretreatment image module 62 comprises Filtering Processing module 621 and global motion compensation module 622.Wherein, described Filtering Processing module is used for image is done noise filtering, conventional processing such as level and smooth, to remove the noise spot in the image; Global motion compensation module 622 is used to compensate the global motion owing to image translation that the slight swing of camera causes, rotation, zoom.
Fig. 8 is the structural representation of the marked region module of moving object detection system of the present invention, as shown in Figure 8.Marked region module 63 comprises foreground segmentation module 631, morphology processing module 632, connected component labeling module 633.Wherein, foreground segmentation module 631 is used for based on background model scene image being cut apart, to obtain the bianry image of prospect; Morphology processing module 632 is used to use the method for mathematical morphology to handle described bianry image, with the less false areas of removal area, and the bigger zone of filling area; Connected component labeling module 633 is used for the zones of different by the method mark Same Scene of connected region, to distinguish different target areas.
Fig. 9 is the structural representation of the maintenance state module of moving object detection system of the present invention, as shown in Figure 9.Maintenance state module 64 comprises: condition judgement module 641 and abnormality detection module 642.Wherein, condition judgement module 641 is used to judge moving object detection system present located state, and makes respective handling; Abnormality detection module 642 is used for comprising the vision signal serious interference, has artificial the execution when blocking the camera situation to detect.
Figure 10 is the structural representation of the enhancing regions module 65 of moving object detection system of the present invention, as shown in figure 10.Strengthen regions module 65 and comprise shadow Detection module 651, highlighted detection module 652, tree filtration module 653.Wherein, shadow Detection module 651 is used for detecting the shadow region that foreground image comprises people, car shade, and with the filtering of detected shadow region; Highlighted detection module 652 is used for detected image and whether is in highlighted state, if then carry out luminance compensation; Set filtration module 653, be used for the leaf and the shade thereof of the swing of detected image, and with its filtering from foreground image.
The influence of disturbing factors such as moving object detection system 6 can the rejection image shake, shade, rain, snow accurately detects the moving target in the scene image.
Great advantage of the present invention is, can accurately detect the moving target in the scene image, comprises people, car, can ignore simultaneously the influence of the disturbing factors such as tree, brightness variation, shade, rain, snow of flating, swing.
The present invention can also be used for intelligent video monitoring system, in order to realize functions such as target classification identification, moving target warning, motion target tracking, PTZ tracking, feature shooting automatically, goal behavior detection, flow detection, crowded detection, legacy detection, stolen quality testing survey, Smoke Detection and flame detection.
The above; being preferred embodiment of the present invention only, is not to be used to limit protection scope of the present invention, is to be understood that; the present invention is not limited to implementation as described herein, and these implementation purpose of description are to help those of skill in the art to put into practice the present invention.Any those of skill in the art are easy to be further improved without departing from the spirit and scope of the present invention and perfect, therefore the present invention only is subjected to the restriction of the content and the scope of claim of the present invention, and its intention contains all and is included in alternatives and equivalent in the spirit and scope of the invention that is limited by claims.