CN1346327A - Obstruction detection system - Google Patents

Abstract

The invention provides for a method of detecting objects in an area, the method including obtaining one or more images of the area, using an edge detection technique in such a way as to highlight substantially dominant linear features in the image(s), and determining if any dominant linear features intersect linear features defining the area. The method may also include detecting parallax in at least two images, the parallax being produced by the presence of 3-dimensional objects in the area.

Description

Obstacle detection system

Invention field

The present invention relates to obstacle detection system.Particularly, the present invention relates to detect between the elevator door or near the method and apparatus of the obstacle it, but be not to be exclusively used in this occasion.The present invention also can be used for industrial environment, security personnel use, and machine works monitors, the obstacle detection under process control and people's the situation such as mobile.

Background technology

Following discussion is primarily aimed at disorder detection method and the device that elevator car door system is used.Yet, be interpreted as the application that this is not a limitation.Under some environment, other obstacle detection situations of discussing everywhere applicable to specification sheets by suitable modification the present invention.

So far, there are a large amount of technology and device to can be used for detecting static volume or the interior obstacle of variable position.International Application PCT/NZ9500067 of applicant is seen in the general discussion of these technology.

Generally speaking, these prior arts lay particular emphasis on and adopt optical device to detect the obstacle that the elevator door detection zone exists.These known system adopt infrared (IR) light source battle array and corresponding receptor usually.A kind of prior art comprises across the continuous array of elevator door inlet " irradiation " infrared beam, and the down trigger disorder event by 1 or many light beams, and this incident activator switch device makes elevator door oppositely mobile or stop to move.The advantage of these systems is that therefore its position can be particularly suitable for handling the obstacle that detects the variable geometry entrance on the plane of one or more elevator door regulations along the edge setting of dodge gate.

The generally satisfied obstacle that is located immediately between the elevator door that detects of these technology.Yet, think that at present the obstacle detection that is confined to a plane can not fully satisfy the contemporary industry safety rate.Elevator industry encourages exploitation can not only detect the obstacle in the scope between the elevator door, and has and extend to the detectability that enters the entrance hall preset distance outdoors.Thereby, need to make existing 2 dimension door obstacle sensing battle arrays to be upgraded to and comprise 3 dimension functions, new comprehensive door plane and proximity obstacle detection system perhaps can be provided.

The trial at above-mentioned industry trend before comprises the technology of setting forth in the U.S. Patent number 5387768 (Audi (Otis) elevator company).Utilize this technology, trigger disorder event by the people who walks close to elevator, rather than the quiet people who stands in before the elevator.That is, the specification sheets of this patent is mainly told about a kind of motion detector.This system adopts macking technique, removes the zone of the elevator image/area image that has nothing to do with obstacle detection.This system detects the passenger, and control gate moves and to passenger's counting, attempts to make the wait time between the elevator the shortest.

In order to realize above-mentioned functions, Audi's vision system utilizes the difference between these 2 images to judge whether elevator obstacle detection district exists mobile object after collecting image 2 different times.This technology relies on and adopts the reference picture of catching before record the 2nd obstacle detection image.Then,, and apply thresholding,, only comprise and collect the contours of objects that the 1st image moves during as the 2nd image to produce a kind of image above-mentioned 2 image subtractions.This system comprises the baffle of branch, is used for entrance hall district and elevator threshold.The variable part of entrance hall baffle masked images, institute's masked images amplify to depend in this specific region or observe the district whether detect mobile for a short time.That is, motionless if 1 people stands in the back, entrance hall, then owing to do not detect mobilely, this image area is shielded.Threshold baffle area when door is closed increases, thereby image to be processed covers door.

Therefore, the technology that most relevant prior arts are set forth carries out picture is detected all according to the response to moving, and covers the zone that does not have influence.

The system that other prior art (seeing for example U.S. Patent number 5284225 and 5182776) discloses also adopts and " effect " image of collecting afterwards reference picture relatively.These technology adopt the image subtraction method usually, and from the figure image subtraction reference picture of collecting afterwards, whether the disturbance in judgement thing enters the obstacle detection district.These technology present many shortcomings, and its time-based checking system can not be perceiveed the time gap of being longer than certain threshold value.Like this, object instantaneous fast moving after the quiescent period just may have problems.In addition, under the situation that intensity of variation is big between the elevator door environment, these known technologies also can be difficult to realize.For example, furniture (with other fixing accessory devices) variation, floor overlay pattern variation etc. all can hinder according to the time difference and detect the increase that the vision scene changes.If must be in early days or fixedly the stage set reference picture, especially like this.

Therefore, the purpose of this invention is to provide a kind of disorder detection method and device, overcome or improve at least some shortcomings of prior art, or be at least the public a kind of useful selection is provided.

Summary of the invention

With regard to broad aspect, the invention provides in a kind of zone the picture method of inspection, this method comprises and obtains the image that one or more should the zone, adopt edge detection method, its approach roughly occupies excellent lineament for giving prominence in the image, and judges whether to occupy excellent lineament and stipulate that this district's lineament is crossing.

Should the zone be the object detection district preferably, and be divided at least 2 districts, the volume that the 1st district has door and threshold to describe, the 2nd district comprises that outdoors the passenger is with a volume that will pass through.

Preferably door and threshold are the door and the thresholds of elevator, and volume is outdoors waited for the elevator platform/entrance hall of elevator for the passenger.

Preferably have 2 images at least, and this method also comprises the step that detects 2 or a plurality of image parallactics, this parallax is producing picture because of region memory; Particularly, this object is in the 2nd district.

In the 1st particular aspects, the invention provides the method for the volume detected object/obstacle of a kind of opposite house and/or threshold regulation, described method comprises the employing edge detection method, it is by way of being roughly to occupy excellent lineament in the outstanding image, and judges that whether occupy excellent lineament intersects with the lineament of described door of regulation and/or threshold.

Preferably this method comprises the preproduction phase of the feature of describing one or more images, so that set up some characteristic lineaments that this zone presents, best described characteristic occupies the lines that excellent lineament is regulation door edge and/or threshold, deposits described feature and occupies excellent position, is provided with the back reference.

This method also can comprise the operational phase, one or more images of this phase analysis, so that set up some non-characteristic features that this presents in long-pending, described non-characteristic features embodies potential object and/or obstacle in this zone.

Preproduction phase comprises at least 2 steps, and the 1st step detects the position and the size of threshold, and the 2nd step detects the position and the size at one or more edges.

Best the 1st step comprises:

Adopt approximate horizontal edge and/or approximate vertical edge detection filter device to give prominence in the image section of known threshold apparent position of living in and occupy excellent vertical bar and/or horizontal line; To produce every capable intensity values of pixels summation in the image with vertical margin and/or horizontal edge detection filter device, thereby produce vertical and/or horizontal function, its maxim and minimum value are corresponding to horizontal linearity feature and/or the special position of giving birth to of vertical linearity, and described lineament is according to the locus of horizontal properties in the image and vertical features specified thresholds.

Best the 2nd step comprises:

Physical relation between the threshold locus that utilization is understood and the threshold of understanding and the door edge obtains in the subimage of image or door;

Make subimage be subjected to the edge detection filter effect, this filter adaptation is in outstanding edge towards the angle between the known boundary;

Control this subimage producing binary picture, this binary picture comprise with the cooresponding a plurality of lineaments in door edge in one;

Derive the equation of binary picture neutral line feature.

Preferably this known boundary is approximate vertical edge and approximate horizontal edge.

Derive before the binary picture neutral line characteristic equation, the 2nd step also can comprise:

Handle binary picture with amplitude in the slant function that vertical direction increases.

Further handle this image, some occupy excellent lineament so that in the clear identification binary picture, this processing comprises with the 1st filter and leaches in the binary picture roughly isolated feature, and to the feature of binary picture with some substantial linear in the 2nd filter minimizing image.

Preferably by utilizing method of least squares or similar approach that the lineament equation is obtained in the lines location; Can have in the image to occupy excellent lineament more than 1, the equation of an arbitrary lineament of decision is promptly removed this lineament from image, and is set up another and occupy excellent lineament equation.

The most handyly always add the valuation of handling each lineament equation in aviation value, thereby improve the confidence level of this lineament equation, by following or a plurality of weighted averages are carried out normalization or quadrature, obtain this total weighted average, these weighted averages comprise:

The 1st weighted average wherein determines lineament to count derivative and variance, and unique point distance and derivative change to outside the given parameter, and representative image is interrupted, thereby the 1st weighted average or is removed this point from valuation to weighting under the described point, and/or

The 2nd weighted average makes away from the point of the some weighting in the lineament of image capture sources greater than close image capture sources in this image, and/or

The 3rd weighted average, the 3rd weighted average is the inverse of characteristic derivative, and/or

The 4th weighted average is weighted the lineament of crossing over any subimage from the vertical margin to the vertical margin.

Utilize filter, differentiator etc. to implement rim detection.

Best described rim detection is at the excellent lines that occupy in the outstanding image, these lines be oriented approximate horizontal, vertically with roughly become diagonal line; Particularly, these diagonal lines are spent for 45 degree and 135 roughly.

Operational phase may further comprise the steps:

Catch one or more real time operation images in this zone;

The position of door in the detected image;

Detect and represent the obstacle that occurs in the image-region of threshold;

Detect the obstacle that occurs in the image-region of representing the door edge.

Preferably, obtain the position of door, the locus of door in the Strength Changes specified image by the variation of intensity in the approximate horizontal feature that detects threshold.

Preferably occupy excellent vertical features, judge in the image-region of representing threshold to have obstacle by adopting at least in the image that the approximate vertical edge detection filter is outstanding and the threshold lineament is crossing.

Preferably, judge in the image-region of representing the door edge to have obstacle by adopting dominant character in the image that edge detection filter is outstanding and the door lineament is crossing at least.

The best operated step comprises that the image transformation of measuring the edge is a statistical graph, and the peak value of this statistical graph is equivalent to the feature in the image, and described feature is represented door and/or threshold, and/or obstacle on door edge and/or the threshold.

Operational phase can be with above-mentioned any image processing means.

The best operated stage can repeat repeatedly.

Another particular aspects of the present invention is a kind of obstacle and/or obstacle movement detection method, and this method comprises the parallax that detects in the obstacle detection district 2 or a plurality of images, and this parallax produces because of there being object in the zone.

This method can comprise the step of surveyed area image transient change.

This method can comprise the step of the vertical and horizontal parallax that is positioned at this regional object generation.

Particularly, the invention provides a kind of object detection method, the method includes the steps of: the background of a plurality of images in 1 zone of aliging; There is object in image subtraction so that represent this zone by parallax in pairs.

More specifically, the invention provides a kind of object detection method, the method includes the steps of: the background of the 1st image and the 2nd image in 1 zone of aliging; From the 2nd figure image subtraction the 1st image, thereby represent existence 3 dimensions to object by parallax.

In a preferred embodiment, the method includes the steps of:

Collect the 1st image in 1 zone from the 1st viewpoint;

Collect the 2nd image in this zone from the 2nd viewpoint;

Calculate the skew between these 2 image backgrounds;

The align background of these 2 images;

These 2 image subtractions are to produce the 3rd difference image;

Analyze the 3rd difference image, measure parallax, have 3 dimensional objects thereby represent this zone.

After being preferably in subtraction step, before the analytical procedure, exist thresholding to apply step, thus, difference image is applied thresholding,, thereby produce binary picture with the eliminating noise.

Preferably the 3rd difference image is handled, so that the profile of some 3 dimensional objects in the inclusion region only roughly.

In another embodiment, image division is background image and door edge image; The image of background during according to no object calculates the skew that needs between 2 image backgrounds.

Preferably adopt crosscorrelation to calculate this skew.

Preferably utilize Gaussian filter, median filter or similar filter to use image blurringization, so that reduce the pixelation effect in the image.

The present invention also provides the obstacle detecting device in a kind of obstacle detection zone, and described device comprises described image is handled at least 1 imaging device and 1 adaptation according to above-mentioned any part micro processor, apparatus.

Object test equipment in a kind of regional internal area, this device comprises: at least 1 imaging device adapts to the image that the viewpoint of separating from least 2 spaces forms identical scene;

Micro processor, apparatus adapt to be handled described image, and its method is for roughly occupying excellent lineament in the outstanding described image, and judges whether that the excellent lineament of some residents represents that there is object in this zone.

Obstacle detecting device comprises in this obstacle detection district:

At least 1 imaging device adapts to the image that the viewpoint of separating from least 2 spaces forms roughly the same scene;

Micro processor, apparatus adapt to be handled described image, so that calculate the skew between the right background of 2 images or image, according to described offset alignment background image, the gained image subtraction, to produce difference image, thereby can detect the parallax effect in the difference image, represent that there is object in this zone.

Preferably microprocessor also adapts to the processing image, roughly to occupy excellent lineament in the outstanding image.

Available optical mode, mathematical way or other similar fashion are handled image, and this mode represents and occupies excellent lineament and/or parallax in the area image.

Preferably microprocessor also adapts to difference image is applied thresholding.

The form of this microprocessor can be solid-state device, optical device or device similar with it.

When adopting 1 pick up camera, this device comprises optical branch and reflective devices, and this optical branch and reflective devices adaptation are carried out relaying to the image from the viewpoint of leaving this pick up camera actual position.

The collection of anaglyph can be implemented by the optical unit that comprises prism, coherent optic fiber wave guide etc., perhaps can be transformed into as device proper or makes the suitable displacement of this device proper.

Among one embodiment, can add dummy feature, to help roughly regularly in the outstanding image of microprocessor to occupy excellent feature.Also can comprise input media, this input media makes the user can be the position input microprocessor of regular dominant character.

From following only as an example explanation, can understand the more aspect of the present invention.

Summary of drawings

Now consult description of drawings the present invention, these illustrate only as an example.In the accompanying drawing,

Fig. 1 illustrates the planar view (a) of the elevator entrance of band pick up camera in the preferred embodiment of the present invention; End elevation (b) and lateral plan (c);

Fig. 2 illustrates the scheme drawing of 2 embodiment of parallax imaging system of the present invention;

Fig. 3 illustrates the connection diagram of 2 imaging devices (pick up camera), computing machine and door controller interface;

Fig. 4 illustrates the 1st detection zone;

Fig. 5 illustrates a series of images that middle elevator pick up camera embodiment illustrated in fig. 1 is taken.

Fig. 6 illustrates the edge detecting technology that is used for judging image elevator threshold level attitude and upright position;

Fig. 7 illustrates the step that used algorithm is located at an edge;

Fig. 8 illustrates the step 4 and the subimage in 5 processing times 5 among Fig. 7;

Fig. 9 illustrates 9 * 9 filtering techniques of removing the isolated feature of black and white image;

Figure 10 illustrates the door edge lines that image breaks;

Figure 11 illustrates black and white image is applied slant function;

Figure 12 illustrates the thresholding edge is applied the weighting array;

Figure 13 illustrates the broken string image is applied the weighting array;

Figure 14 illustrates the valuation of black and white image lines equation;

Figure 15 illustrates and calculates the equation that the door vanishing point is used;

Figure 16 illustrates by examination running clearance intensity profile map detecting gate position;

Figure 17 illustrates according to the method that determines the door position from the statistical graph of vertical margin and horizontal edge detected image;

Figure 18 illustrates the used statistical graph structure example of some objects on decision gate position and threshold or the door edge;

Figure 19 illustrates the method for available statistical graph detecting gate position;

Figure 20 illustrates the method that available statistical graph is determined the door position and had object or obstacle location;

Figure 21 illustrates the diagram of circuit that detects each step of parallax method of obstacle in the obstacle detection district;

The method that Figure 22 illustrates Figure 21 is used for the data that elevator door obstacle example (ladder) is produced;

Figure 23 illustrates the discernible parallax of some obstacle example detection machine;

Figure 24 illustrates the ability that filtering technique reduces institute's altimetric image false picture that pixelation produces;

Figure 25 illustrates the data example of a limbic disorder incident.

The description of preferred embodiment

Below, under the situation of elevator car door system obstacle detection, describe.These explanations are interpreted as not limiting feature of the present invention.Apparatus and method of the present invention can be used for various obstacle detection, and for example industrial machine detects; Security personnel's application etc.

Under the situation of elevator door obstacle detection system, there are 2 different districts, the 1st district has the crucial volume on threshold and 2 groups of edge limited borders of door, is called the 1st barrier.Must detect object in this district with height reliability, almost clash into any obstacle certainly because close the door.The 2nd district for the passenger near and wait for platform/entrance hall district before the elevator of elevator, be called the 2nd barrier.The obstacle detection in this district requires not strict, and obstacle detection system must be able to be sheltered irrelevant object or mistake object.Obstacle detection system of the present invention is based on light detection method, and has various imaging methods to be used to provide respectively to distinguish required obstacle detection degree.

To the 1st obstacle detection district, obstacle detection system adopts edge detection method, whether has object (obstacle) between the decision gate edge or on the threshold (inserting this car and the flat-bed ground plate portion of elevator cab door and platform door).Under the situation of elevator door, rim detection is even more important.Long-term next, people have got used to hand being put into crack between a door and its frame or stepping on threshold, close to stop elevator door.Therefore, importantly any obstacle detection system can detect the hand that is placed between the door close or the object on other objects and the threshold.

Whether intersect with the lines of describing door or threshold edge by some lines of judging regulation obstacle edge, native system can reach above-mentioned requirements.This is also to adopt various standard imaging methods, wherein adopts reference picture, makes to detect obstacle.

In order to detect obstacle and object in the 2nd barrier, adopt parallax method.This parallax method adopt with edge detection method in the optical imagery of identical shooting, but relate to 3 dimensional objects of identification elevator platform/entrance hall district existence.Parallax method also can be used for detecting object or obstacle in the 1st district, yet, found that the method does not possess the required accuracy of key area.Its reason has two: one, and the big parallax effect that an edge produces may be flooded the parallax that little 3 dimensional objects produce; The 2nd, applicant is found can not detect the following object (this problem hereinafter illustrates) of threshold top 200mm with parallax method.

Practical layout

In the preferred embodiment, native system may comprise 2 pick up cameras observing lift port from 2 independent viewpoints, and 2 independent viewpoints can work the 2nd detecting device based on parallax.Realize that this puts available known optical imaging system, such as the electronic imaging apparatus of charge-coupled device (CCD) pick up camera, cmos camera (applicant adopts valid pixel sensor (or APS) cmos camera at present) or any kind.

In addition, also available single pick up camera, this single pick up camera of image orientation of the 1st observation (from the observation of the 1st commanding elevation) and the 2nd observation (from the observation of the 2nd commanding elevation), perhaps available coherent optic fiber wave guide is sent to single pick up camera to the observation that separates by one or more optical branchings.Imaging also can be controlled by optical unit, and this unit alternately inserts the truncated picture device of parasite or other types in camera coverage, thereby makes observation forward the commanding elevation of removing on the space to.The major defect of this system is: pick up camera must be enough fast, so that after collecting image at every turn, can and then present static; Photosystem may be invaded by dust in the elevator environment.

Fig. 2 illustrates the rough schematic view that the present invention adopts the embodiment of electron camera.The embodiment on Fig. 2 top illustrates and is arranged in the single electron camera 110 that can collect obstacle detection district image from different viewpoints.This scheme is implemented by mirror 111,112,113,114.For clear, shorten the horizontal length of optical branch.If adopt the charge-coupled device (CCD) pick up camera, can comprise 2 gathering-devices that separate, perhaps comprise 1 ccd array that separates.The lower part of Fig. 2 illustrates the scheme drawing of single pick up camera parallax detecting device.Here, each independent image of a scene is by each branch of detector.Selecting earlier by some electric photoswitches that are subjected to switch 119 controls of concrete viewpoint controlled.Pick up camera is collected the image of " watching " by staggered optical branch, and carries out the parallax detection according to switching the scene observation, and optical branch constitutes with mirror 116,117 and 118.

Yet the applicant's preferred systems adopts opens the layout standard camera of (as shown in Figure 1) by splayed.Fig. 1 a elevator entrance planar view that draws illustrates entrance hall side 1 and bridge railway carriage or compartment side 2.These 2 zones are separated by the threshold 3 that running clearance 8 is divided into two.9a, 9b and 10a, 10b represent an edge.Fig. 1 b 1 end elevation of seeing into car 2 that draws from the entrance hall.This figure is clearly shown that and is contained in 2 pick up camera 6 and 7 of inlet on the back timber 4.Pick up camera makes pick up camera 6 towards the 1st side door edge 9a, 9b by splayed expanded configuration layout, and pick up camera 7 is towards the 2nd edge 10a, 10b.There is the overlay region 5 that covers entrance hall district 1, is used for parallax method.In order can to strengthen the observation volume and/or to strengthen the parallax district, pick up camera 6 and 7 position are interchangeable.In other words during cameras view door group, pick up camera 6 observation door groups 10, pick up camera 7 observation door groups 9.Fig. 1 c illustrates the elevator entrance lateral plan.

The reason of choice criteria electron camera is in this layout: most of lift port (the camera lens costliness in the wide visual field of cameras view that available rates is inexpensive, be more difficult to get, and the image of Pai Sheing has distortion probably), and can provide 2 detection zones of parallax (being overlay region, the visual field) in lift port central authorities.This is provided with the image example of gained shown in Fig. 5 a and Fig. 5 b.These 2 images it should be noted that there is the overlay region in elevator the place ahead.This is the 2nd barrier of adopting parallax method to use, and is equivalent to the district 5 of Fig. 1 b and Fig. 1 c.

Fig. 3 bonded assembly scheme drawing between 2 pick up cameras 6 and 7, computing machine 11 and the door controller interface 12 that draws.Send energizing signal 13 to door controller from door controller interface 12, but this controller is opened the relay of elevator door when for example there is obstacle in the start-up system detection.

Rim detection

The 1st aspect of the present invention is to discern when elevator door edge and threshold have obstacle the lineament that the 1st obstacle detection is used.This feature of hypographous region representation among Fig. 4.

Edge detection method is divided into 2 independent sectors.Section 1 is automatic calibration algorithm, is used to determine the position of image Zhong Men edge and threshold.Move this algorithm when expecting this device head dress, provide a description some parameters of elevator door and pick up camera geometry.The part 2 of edge detection method is an operative algorithm, detects the object that exists on the door edge and threshold at closing time.These algorithms are called the 1st calibration algorithm and the 1st operative algorithm.

The 1st calibration algorithm

The edge detecting technology that the 1st calibration algorithm is used is divided into 2 steps.The 1st step examination image is so that detect the threshold of numeral 3 indications among Fig. 1 a and Fig. 4.The door edge that digital 9a, 9b and 10a, 10b represent among the 2nd step identification Fig. 1 a, 1b and Fig. 4.

Consult Fig. 5 a and Fig. 5 b (pick up camera 6 and 7 images of taking among Fig. 1 are shown), in the image threshold the identification of corresponding lineament comprise and fully open elevator door, and right side (image 5a) and left side (image 5b) the dense lines of vertical and level separately in water orle and the outstanding image of vertical margin detection filter device.This is the estimating position of threshold in this 2 image.

Picture specification filtering technique among Fig. 6, this image is taken with the single pick up camera of getting a bird's eye view threshold.Yet this technology also can be used for each image of Fig. 5 a and Fig. 5 b.Horizontal door track can be seen in the bottom of Fig. 6 b.Owing to have threshold and door track, can see also among Fig. 6 b that horizontal edge detects outstanding horizontal line.

In case obtain the horizontal edge image of Fig. 6 b, just ask the summation of every capable pixel intensity value in the image.From Fig. 6 f as can be known, its result is positioned at the function (vertical scan direction) of threshold track and edge for peak value.Can measure these peak values easily, so that the position of threshold is provided.

In order to determine threshold width (horizontal component of elevator door zone), image shown in Figure 6 is subjected to the effect of vertical margin detection filter device.The gained image is shown in Fig. 6 c, a wherein outstanding threshold and a vertical bar of junction, edge appearance.Identical with the used mode of track, always add the intensity of every row pixel among Fig. 6, to produce the function shown in Fig. 6 e.Peak value among Fig. 6 e is equivalent to the level attitude at threshold edge.

Above method provides the level attitude and the upright position of threshold, thereby can distinguish threshold from image (or 2 images under Fig. 5 a and Fig. 5 b situation).

In a word, the step of decision threshold range can reduce:

1, fully open elevator door, and no foreign object exists on threshold or the door edge.

2, water orle detection filter device decision threshold vertical range, running clearance upright position and some track upright positions.

3, with vertical margin detection filter device decision threshold horizontal extent.That is, decision illustrates that threshold is not held and the vertical bar at door top.

4, threshold position digital in each image of explanation is deposited nonvolatile memory.

Owing to fit with the lower part at door edge in the threshold edge, know that an edge just can tell the subimage at offside door edge and offside door edge.These subimages shown in Fig. 5 c, 5d, 5e and the 5f.Then, in the 2nd step of this method with this subimage recognitiion gate edge.Below conclude position and the vanishing point that how to determine an edge.

1, extracting the door edge with the threshold scope that determines previously is 4 number of sub images of principal character.This 4 number of sub images comprises 2 left-hand door edge images and 2 right door edge images (Fig. 5 c, 5d, 5e, 5f).Image when the image that every side is 2 comprises an edge for about miter angle or 135 degree angles.

2, adopt the edge detection filter that to give prominence to the door edge.For example level and 45 is spent or the edge detection filters of 135 degree.

3, addition after whole frontier inspection altimetric images that previous step is produced suddenly take absolute value (for example the absolute value additions of horizontal edge detected image and 45 degree or 135 degree edge-detected image).

4, edge-detected image is applied thresholding,, do not belong to the isolated stain of an edge lines, reduce an edge lines, the cleaning black and white image by getting rid of so that obtain the black and white image at an edge.

5, black and white image is handled with slant function, the value of this function vertically reduces with the threshold vertical displacement.

6, ask each row maxim.That is, ask the point of the door edge lines of close threshold of explanation.Comprise the peaked battle array of every row example and be called the maxim array.

7, obtain the weighting matrix of the explanation each point accuracy of asking.

8, obtain the linear equation of explanation lines with the linear equation algorithm for estimating of least square.

9, the width of the current lines of decision, and from image, wipe these lines.

10, return step 6, continue to seek lines up to maximum array vacant occupy excellent.

The joining of whole the edge lines of finding when 11, asking every side door open with method of least squares.Can get 2 vanishing points with a little methods, 1 in each pick up camera.

Fig. 7 comprises the diagram of circuit that the elevator running door is organized the 2nd stage in used the 1st calibration algorithm.Now describe each step in detail.

Step 1,2 and 3

Initial step is divided into 4 number of sub images with the known threshold scope that above obtains with image, wherein comprises toward the lines of figure father top or bottom angled.Draw a little image shown in Fig. 5 c, 5d, 5e and the 5f.These subimages are subjected to edge detection filter (identical with the filter that determines the threshold scope to use) effect, and these filter adaptation are outstanding to be about 45 degree or 135 edges of spending with angle in horizontal direction.

Step 4

Now by applying thresholding, the subimage at door edge is transformed to black and white (b/w) image.What apply thresholding the results are shown in Fig. 8 a and Fig. 8 b, and this is respectively the image of Fig. 5 e and Fig. 5 d to be applied the black and white image that thresholding produces.This algorithm also application program separates close especially threshold just as the lines that will combine with it, and removal obviously is not the little isolated feature (the promptly isolated pixel of deceiving) of lines component part.

The employing method of reducing is separated lines.Among the present invention, this is reduced method and removes pixel from object bounds.By the black pixel near white pixel is bleached, accomplish this point.Object will be removed some pixels of bridge joint adjacent lines, and an edge lines is attenuated.The image of Fig. 8 a and Fig. 8 b is in case through reducing, be exactly the image of Fig. 8 c and Fig. 8 d.Can find out that a little processes have the effect that the lines that make explanation door edge attenuate and separate.

In order to remove isolated feature, adopt by 9 * 9 image subsection filter operating.If all the summation of unit is below 9 in 9 * 9 subareas, then center pixel is set at zero, otherwise it is output as the value of center pixel.Therefore, this algorithm notices observing whether at least one complete lines may pass through 9 * 9 subareas.Some is random for the scale of filter (being 9 * 9 in this example), can change, to get rid of small object or to comprise big object.Fig. 9 a and Fig. 9 b explanation the method.Among Fig. 9 b, center pixel is set at zero.

The image of Fig. 8 e and 8f is to be used the result who obtains behind 9 * 9 filters of removing isolated feature to deleting image 8c and 8d.Can be clear that the ability of the isolated feature of this filter filtering among the image 8e.

Step 5

Now with the slant function black and white image that converts, so that can describe the lines that the door edge that will judge produces with linear equation.The value of slant function reduces with the vertical shift to the bisector that produced image.Utilize the slant function of this mode, its reason is in the subimage that the door edge lines of close this bisector tend to level, and from the vertical margin to the vertical margin across subimage.In addition, in the subimage that the door edge upper part is divided, the edge lines tend to be inclined upwardly, and the lower part at door edge, the edge lines tend to be inclined upwardly, and the lower part at door edge, the edge lines are downward-sloping.These inclination lines stop at the upper limb (going up subimage) of image or the lower edge (following subimage) of subimage then from the vertical margin of contact threshold, thereby shorter.

Figure 11 illustrates an example of using inclined function, the particular image of last left half that this diagram is gone out or following left half.Figure 11 a and Figure 11 c are the door edge particular image after applying rim detection, isolated pixel is removed and deleting the filtering that disappears.The direction of tilt of slant function is shown at the A of left row and B row to these figure.Figure 11 b and Figure 11 d illustrate filtered image applications slant function, can see that slant function is inclined upwardly toward the bisector between 2 images.The 1st row maxim array (method of least squares produces the equation of scanning lines with this array) is shown in the bottom of Figure 11 b and Figure 11 d.Row maxim array among Figure 11 b and Figure 11 d regulation is the door edge lines of close bisector.

The particular image of Figure 11 is the representative form of the gained image when carrying out multiply by slant function after the filtering of the image to Fig. 5 d and Fig. 5 e.

Get back to Fig. 8, the picture specification of 8g and 8h to the image applications slant function of 8e and 8f after the image of gained.As mentioned above, slant function is with linear mode these images that convert.To the image (being Fig. 5 e and 5f) of elevator door bottom, slant function descends from the top of image, and to the image (being Fig. 5 c and Fig. 5 d) at elevator door top, then the top from image rises.Its reason is, when finding the longest lines the most clearly, at first strengthens the performance of this algorithm.

Therefore, value increases with the vertical image size and the horizontal image size is kept constant slant function.Have maxim along bisector now, amplitude drops to the bottom or the top of subimage then.In this way, the lines judgment part of algorithm (being step 6-10) carries out the longest lines the most clearly earlier, claims short more unsharp lines then.

Step 7,8 and 9

Step then is to ask the equation of each edge lines in the image.This is from the row maximum array (seeing Figure 11 b and Figure 11 d) at the edge of the most close image bisector of regulation.

Stipulate some point in the row maximum array of the current lines of determining equation, its accuracy provides the confidence level greater than other points.Therefore, algorithm produces and puts the weighting array that the instrument degree influences the linear equation result according to data point.

The instrument degree of putting that decision row maximum array is used is subjected to some factor affecting.These factors comprise following 4 kinds:

1, away from the point of threshold and poor contrast.That is, along with the lines at regulation door edge move on to bridge compartment top portion from threshold, since the contrast ratio loss, these svelteness degree variation.Therefore, this point on the lines of close threshold has than the high weighting of the most this point of close car top.

2, counting below the subimage lateral dimension in the row maximum array.When line is to begin and stop at the image vertical margin, but in vertical margin (this place fits with threshold) beginning, when the image level edge stops, a little situations appear then.The example of this lines can see in Figure 11 a that the lines of this figure upper end stop at the upper limb of image.In this case, the row maximum array is 54211111, and wherein last 4 values are 1, and this is because these row comprise zero now fully.These values can influence encloses and lines is estimated as upwards.

3, another factor that influences weighting array pixel confidence is that bar is broken into a little lines, thereby not exclusively across image.It is very common that lines have little broken string.Figure 10 illustrates an example of this problem, can see among this figure that the lines of bottommost are broken into 2 root lines.As a result, the unit that the row maximum array has 2 top line contributions now obviously destroys linear Equation for Calculating.This point can be seen in Unit the 3rd and the 4th in Figure 10 b bottom row maximum array.If the derivative of calculated column maximum array and variance can be obtained these partly by the derivative sudden change with to the distance that previous estimation line is calculated greater than variance.By descending weighting, from the linear equation estimation, remove these parts.

4, the have the greatest impact last factor of array confidence level is a noise data.The weighting function of the inverse by producing row maximum array derivative can descend weighting to this noise.That is, when asking the lines of equation level and smooth, the row maximum array is also level and smooth, thereby any sudden change of derivative may be exactly a noise.

In order to overcome above-mentioned factor, calculate each weighting array, overcome above-mentioned influence respectively.These weighting arrays are called the distance weighted array of threshold, short-term weighting array, broken string weighting array and derivative weighting array.After respectively its largest unit being carried out normalisation by these arrays, each array multiplies each other, and obtains summation power array.

Consult Figure 12, the application that short-term weighting array and threshold are distance weighted is described.Figure 12 a and Figure 12 b relate to the weighting valuation of Figure 13 a middle and upper part lines group the 1st lines, and there is the top of image in these lines, rather than the left side of image.Above-mentioned the 1st lines are called short-term.So, produce weighting function, guarantee that the valuation of lines equation only is subjected to reach the top that has image at the most, rather than the left side of image.Above-mentioned the 1st lines are called short-term.So, produce weighting function, guarantee that the valuation of lines equation only is subjected to reach at the most the lines data influence of the point on the lines that have the image bottom.The curve on Figure 12 a top illustrates the row maximum array 20 of this short-term, has the 1st linear equation valuation 21 on it.The curve at Figure 12 a middle part is the derivative of row maximum array, and the curve of this figure bottom is the distance weighted product of short-term weighting threshold.Can see threshold distance weighted when there is the point of image top in short-term weighting function be zero, thereby the later data of this point do not influence the linear equation valuation.When this curve also is illustrated in lines and further leaves threshold, the distance weighted data that force the linear equation estimation not emphasize to constitute current lines of threshold.Can find that the distance weighted function of threshold is along with the increase of distance reduces weighting with linear mode.

Least square to lines is judged, adopts the standard weighted least squares, from the equation of the disconnected number of row maximum array image Zhong Men edge lines.Now should inferior in cash least square algorithm to maximum array to each.When using algorithm for the 1st time, whether only the 1st valuation of lines equation is asked in distance weighted and short-term weighting with threshold, suitable in order to judge the 1st valuation accuracy of lines equation, the intersection point of judgement lines valuation and row maximum array.If 2 non-intersect or its angle of line is greater than certain thresholding, then this valuation nonconformity.

If the lines valuation is qualified, then begin to calculate the broken string weighting.This work begins and shifts to each termination back end of row maximum array from intersection point.So the broken string weighting is descended weighting to some points that significantly depart from the sudden change of the 1st lines valuation and row maximum array derivative in the row maximum array.If row maximum array derivative has sudden change in addition, and the distance of the point of row maximum array and lines valuation is little, then stops weighting down.Like this, just start, stop weighting down.

The data of current lines broken cord association are removed in the application of weighting down of this broken string of explanation among Figure 13 shown in the figure, and making afterwards, the data of gained lines are included in the row maximum array.Figure 13 a, 13c and 13e illustrate and are included in the tilted image that diverse location has the lines of broken string.Among Figure 13 a, in each end disconnection of lines.Among Figure 13 c, there are 2 places to disconnect at the middle part of lines, have 1 place to disconnect in the end of lines.Among Figure 13 e, the feature association that broken string and previous lines stay.The superpose curve of the 1st lines valuation of the row maximum array that Figure 13 b, 13d and 13f are respectively Figure 13 a, 13c and 13e.These curves also illustrate the weighting function that the row maximum array derivative of asking current lines broken string usefulness and the data of removing the broken string appearance are used.

If lines valuation nonconformity, then algorithm is led and is found maximum array, so that obtain the nose section.Derivative is maximum to be changed by seeking, and carries out this work.So, begin to calculate the broken string weighting from this nose bar middle part.Then, with row maximum array and all weighted calculation the 2nd lines valuations.

Get back to Figure 12, the lines among Figure 12 b illustrate total weighting function and component part (except threshold is distance weighted) thereof.The linear equation estimation utilizes this total weighting function when obtaining the improved valuation of lines equation its 2nd time.

Figure 14 illustrates the valuation of linear equation, and obtains equation from current lines, just removes the situation of lines from the tilted image of Fig. 8 h.Figure 14 a illustrates the tilted image of Fig. 8 h, and the curve on Figure 14 b top is the content of row maximum array (value of the value gained that the every row of process decision chart picture are maximum).The curve of Figure 14 b bottom has the value that the linear equation estimation obtains after its 1st time estimation.This is the data of coming from the equation of describing Figure 14 a lines bottom, from the data of Figure 14 b upper curve being used the least square program and deriving.The result of Figure 14 c for after the related data of the lines of judging more than the image wipe of Figure 14 a, obtaining.This result represents can be in order to calculate the data of next lines.Figure 14 a and Figure 14 b comparison shows that then these lines are used as one, and all wipe if exist between current lines and the contiguous lines and be connected pixel.The curve on the top of Figure 14 b be row from Figure 14 c at array, the curve of the bottom of Figure 14 b is the 1st valuation of the linear equation of data in the explanation upper curve.The process that obtains the lines data shown in Figure 14 e to Figure 14 k and wipe each continuous lines.Last Figure 14 l is the original black and white image in an edge, the lines valuation (grey) that stack is calculated on it.

Step 11

It is useful understanding vanishing point, because this point can make the position at tracking gate edge at closing time.At closing time, the vanishing point maintenance is static, thereby according to the position on the threshold of understanding, decidable is the position of door at closing time, that is, if the point of contact threshold bottom the energy decision gate, picture is put by this and the straight line of vanishing point just can be derived the edge of door.Can develop a kind of technology for detection mis-calculate and present the lines of impassabitity vanishing point.If success, then the least-square estimate of vanishing point can be because of these lines deviations.

In order to obtain the vanishing point valuation of each pick up camera, that adopts the least square algorithm to ask to describe the door edge feature before calculates whole lines intersection point valuations.That is,, ask the intersection point of the linear equation of the door edge feature of describing every side door by separating the equation of form shown in Figure 15 a and Figure 15 b.In these equations, x is the vanishing point level attitude, and y is the vanishing point upright position, and ai is the slope of equation, and bi is the intercept of equation, and n is an equation number.

In case terrible each edge lines and vanishing point data then deposit nonvolatile memory in and use for the back.

More than automatic calibrating method is discussed in explanation, and this method is by adopting autoselect process regulation threshold, door edge and vanishing point.Also can be during installation by for example adopting computing machine and the mouse apparatus with the checking system interface, these features of artificial selection.

Also can help this automatic calibrating method by the emulation boundary mark being set at threshold and door edge key character.For example available band or gummed label mark doorway middle part or more outstanding features, such as the guide rail of door or the threshold lines in conjunction with elevator door.

The 1st detection algorithm is divided into 2 independent sectors.Section 1 is automatic calibration algorithm, is used in the position of image decision gate edge and threshold, and is as indicated above.The expectation device moves this algorithm, some parameters of furnish an explanation elevator door and pick up camera geometry when just adorning.Part 2 is an operative algorithm, is used to detect at closing time the object that exists on door edge and the threshold.

Below, the 1st operative algorithm is described.

The 1st operative algorithm

The 1st operative algorithm comprises following steps.Hereinafter will describe these steps in detail.

1, obtains new (in real time) image in doorway.

2, detecting gate position:

● utilize understanding extraction to occupy the excellent subimage that comprises this threshold to the threshold position.

● utilize understanding, extract the image of these features running clearance and guide rail position.

● by judging the position of horizontal direction intensity sudden change, decision door position.That is, running clearance and track be black feature always almost, and the intensity of these features changes at itself and a point that door intersects.

● if desired, threshold is adopted vertical margin detection filter device, to confirm the door position.

3, the object on the detection threshold;

● judge some dense vertical bars that whether exist cutting to describe 2 lines of threshold vertical range in the threshold subimage with vertical margin detection filter device.

4, the object on the detecting gate edge:

● with previous door position and the vanishing point decision gate boundary position of understanding.

● if desired, confirm the door position with edge detecting technology.

Each concrete steps condition of knowing clearly now is described.

Step 1

At first, obtain the application drawing picture in elevator door, threshold and entrance hall.When pick up camera and elevator threshold were fixed relative to one another, the position of these two parts in image kept constant.The process of the image extraction threshold subimage of therefore, catching when the elevator checking system is operated is fairly simple.Determined the position of these subimages at above-mentioned calibration phase.

Step 2

As threshold itself, running clearance (bisection of the threshold between elevator platform/floor, entrance hall and the lift car floor can be known when observing the threshold image and see at interval) and door track (inserting the groove of door guide rail in the threshold) remain on the same position in the application drawing picture.Therefore, by utilizing the position of these features that calibration phase understands, can extract the subimage of these features from the threshold image.

When elevator door cuts out, the subimage of running clearance and door track be characterized as its do not hold near intensity sudden change.This is because highly the door of light is covered these black features.Shown in particular image among Figure 16, the Strength Changes of seeing is for from secretly to bright.Along with door is closed, also move the position of this Strength Changes, thereby the position of energy tracking gate bottom.Among Figure 16, the operation crack when numeral 31 represent and fully opened, 32 running clearances when representing door section to close.

The principle that the another kind of method of seeking the door position adopts be door when closing in the image horizontal line shorten, and door when closing door edge lines become more vertical, the 2nd kind of method comprises following steps.

1, water orle detection filter device filters image.

2, the edge-detected image value transform is become absolute value.

3, always add each row of above-mentioned image, produce the horizontal edge statistical graph.

4, with vertical margin detection filter device image is filtered.

5, be the edge-detected image value transform absolute value.

6, always add each row of above-mentioned image, produce the vertical margin statistical graph.

7, the energy to vertical margin statistical graph and horizontal edge statistical graph carries out equilibrium.

8, the addition of 2 balancing energy statistical graphs, to produce single statistical graph.

9, obtain statistical graph peak value between known door release position and the door off position, new door position is provided.

Use above-mentioned algorithm when closing, obtain an off position (promptly being generally elevator door central authorities) by door.In order to obtain the good approximation of global maximum and opposite house off position, the point of peak maximum value and both sides thereof is mixed parabola, wherein peak value is near bias light intensity.

The method is better than the 1st method part and is to prevent more reliably that at closing time noise and characteristics of image from moving.This is because always add and on average fall any noise, and strengthens at closing time all variations of horizontal properties width on the threshold.In contrast, the 1st method relies in following factors very much.

● the position of running clearance keeps same position (firm if imaging system is installed, this point is not too big problem in the image between image.

● big brightness variation does not appear in horizontal properties at closing time.

● only several pixels on the horizontal properties height in each image.

● horizontal properties is suitably accurately towards level (being that image can not tilt too big to horizontal direction).

Yet the advantage of the 1st method is to calculate faster than the 3rd method after more.

Provide the example of the 3rd method among Figure 17, wherein Figure 17 a, 17c, 17e and 17g illustrate the original image in threshold district at closing time, and Figure 17 b, 17d, 17f and 17h illustrate the curve of ASSOCIATE STATISTICS figure.This statistical graph curve comprise this two statistical graph behind horizontal edge detected image statistical graph, vertical margin detected image statistical graph and the balancing energy and.Can see with the cooresponding position of door on, always add and have intensity sudden change in the statistical graph.Therefore, use the method detecting gate position automatically.

Step 3

Then, this algorithm need detect the object on the threshold.Object on the threshold can cut 1 or 2 horizons of regulation threshold vertical range.Also may cut the horizontal line of describing running clearance and track upright position.

Be positioned at some objects on the threshold by the threshold subimage being applied level picture and vertical margin detection filter, can giving prominence to.So the cutting by in the horizontal line of seeking regulation vertical threshold scope, running clearance and door track can detect.In case adopt vertical margin detection filter device, the object on the threshold produces ripple vertical bar (otherwise former threshold has dense horizontal line) on the spot.Therefore, by in filtering gained threshold image, seeking the dense vertical bar that intersects with the horizontal line of stipulating threshold, running clearance or door track, also can detected object.Obtain the understanding of opposite house position during previous step is rapid, meaning can be the wrong object detection of working as of elevator door.

Step 4

Object on the detecting gate edge.The call understanding of bottom position (top firm acquisition) makes the equation that can revise regulation door edge at closing time to vanishing point (obtaining during calibration phase) and threshold.Therefore, at closing time, where decidable image Zhong Men edge is positioned at.

Above-mentioned understanding can be used for extracting the subimage that comprises an edge.Subimage is applied the vertical margin detection filter.The dense vertical bar of the outstanding object of vertical margin detection filter device because of finding easily towards pick up camera.Lines by stack regulation door edge can judge whether these lines are subjected to cut with some dense lines of object association.Therefore, but detected object.

The extension of the other method of decision gate position (top be illustrated with reference to Figure 17), some objects that make available single job detecting gate position and appearance.The method is based on vertical margin, horizontal edge and the statistical graph of diagonal line (for example 45 degree or 135 degree) edge detected image roughly.This method is carried out following step after comprising the statistical graph of calculation procedure 1 to 9 described horizontal edge and vertical margin.

10, calculation Design becomes in the outstanding image roughly the edge detection filter of catercorner lines to handle the edge of image statistical graph.For example this image is the 45 degree edge-detected image and the offside door 135 degree edge-detected image of offside door (when seeing outward in the lift stand).

11, according to the product of angle edge statistics figure and vertical edge statistical graph divided by the horizontal edge statistical graph, calculate new statistical graph.

12, follow the tracks of the peak of this statistical graph at closing time.If the guard plot is clear, these peaks just belong to a position.If object appears in threshold or door edge, the peak that big additional peak and its position do not correspond to door then appears in this statistical graph, and there is object in expression.

The outstanding door of roughly catercorner rim detection edge the statistical graph height is raise toward the door locality from an image left side (right side) side, thereby this method indicates a position.By the edge that door bottom and threshold junction produce, vertical margin detects also to provide according to the door position peak is located.Therefore, the position of the peak of 2 statistical graph products indication door.When there was object at threshold or door edge, vertical statistical graph comprised the peak of many denoted objects position.In this case, the product of statistical graph comprises a plurality of peaks, and some belongs to object, and some belongs to door.

The statistical graph product is with horizontal statistical graph because show like this and reduce bias light intensity level in the statistical graph, thereby peak of prominence, the image of threshold comprise form in some way from the horizontal properties of threshold 41 time, the bias light intensity level can be very high.

The image of Figure 18 illustrates the statistical graph that how to make up various edge-detected image, so that the statistical graph of energy detecting gate position to be provided.Figure 18 a, 18b and 18c are for applying the image after 45 degree, the vertical and horizontal edge detection filter respectively, and among Figure 18 d, uppermost 3 curves be the primary statistics figure that obtains from edge-detected image, and following curve is used for the decision gate position for making up statistical graph.

Image in threshold district when Figure 19 closes with some (Figure 19 a, 19c, 19e and 19g) and the statistical graph of following (Figure 19 b, 19d, 19f and 19h) thereof show that this further method can be used for the method for decision gate position.

The image of Figure 20 and curve illustrate how to make up statistical graph make the door and to the picture can locate.Among Figure 20 to similarly being pin and the elevator door edge last arm on the elevator threshold.Close the threshold original image in each stage among Figure 20 a, 20c, 20e and the 20g for door, then be its statistical graph of following among Figure 20 b, 20d, 20f and the 20h.Can see having the used object of this example, then the peak of this object association is much larger than the peak of edge association.

Except that the statistic map method of above-mentioned detected object, also available following method detects or confirms and detects;

(a) broken string in the lines of a searching description door edge, threshold/running clearance interface or threshold/floor interface, and/or.

(b) seek from threshold vertical bar or the vertically extending lines of door edge angle lines.

To the method for above decision door positions, the door position that available door symmetry is open or the look-ahead procedure validation algorithm is provided.That is, the distance from the left-hand door position estimated to line of centers should be approximately equal to the distance of the You Men position of estimation to line of centers.In addition, also the estimation when position, Qianmen, service direction and opposite house speed of available understanding confirms the door position of two field picture down.

Above-mentioned edge formation method considers that judgement can be to the door edge of the door bump of closing and the object on the threshold.This also helps and alarms near the object of elevator door in advance mobile when the required safety performance of an obstacle detection is provided.At this moment electric life controller can be predicted the someone and will enter lift car.The parallax method that will illustrate below the imagination can be used as this early warning and prediction unit.Therefore, object appears at before threshold or the door edge, and door can example move back.

The parallax detection method

The principal character of the 2nd method of inspection is obstacle detection application parallax effect.Consult Figure 21, this illustrates the step of a preferred embodiment.The commanding elevation that separates from the space is collected 2 images.These images are corresponding to the scene of overlooking the elevator doorway from 2 diverse locations (seeing Figure 22 a and Figure 22 b).This image comprises elevator door mouth close vicinity, and promptly common elevator passenger is walked close to the zone of elevator door.This close vicinity can be divided into the 1st barrier (above stating) and the 2nd bigger barrier, this district's (see figure 4) of process when the passenger walks close to elevator.

Consult Figure 22, by place ladder explanation parallax method near the outside, the 1st barrier (promptly in the 2nd barrier) of elevator door.From different 2 image 22a of commanding elevation record and 22b.As condition precedent, take the camera head that these 2 images use and be different from the camera head that the anterior branch of explanation is introduced.Previous camera head adopts 2 ones of separating 100mm to open pick up camera outward.With this previous camera head, 2 images that produced are shown in Fig. 5 a and Fig. 5 b.To be Figure 22 a illustrate identical door image with image among the 22b to main lime light, and the image among Figure 15 a and the 5b illustrates the cooresponding image of each side door.This point and haveing nothing to do of begging for below.In case, only consider the subimage (being marked with numeral 5 among Fig. 4) of the 2nd barrier because the position of calibration algorithm decision gate edge and threshold in fact just covers these zones.These subimages correspond respectively to the last right side and the last left part of going up image among middle part or Fig. 5 a and the 5b of image among Figure 22 a and the 2b.

In case obtain the image among Figure 22 a and the 22b, promptly calculate 2 skews between image background, and the background between scene of being used to align.Preferably when it is made, as far as possible accurately locate, make the background calibration amount minimum by the optical unit of guaranteeing system.Can compensate alignment some little defectives in the background with the appropriate mathematical image processing techniques.In the preferred embodiment, the method for correcting this defective adopt crosscorrelation or least energy by way of.Minimizing energy method comprises makes image (2 dimension) " skew " pixel (in the mode of all directions ordering) at every turn.Behind 2 image subtractions of gained, always add whole pixel values by difference image.The skew that the most accurate required skew of framing is the total value added minimum of gained.That is, the difference image summation hour, calibration is best.

Crosscorrelation is a kind of statistic law, usually reliable the and fast operation than minimizing energy method.And, in the time of crossing Fast Fourier Transform (FFT) enforcement crosscorrelation, found that degree of processing significantly improves.Because the skew that the decision parallax causes is inaccurate, the incomplete counteracting that noise causes, pixelation figure image effect, nonlinear distortion, the rotation of camera review plane and the luminance difference of observing from 2 different commandings elevation, it is unpractical having found to offset fully background.

The error that the framing result introduces depends on that 3 dimensional objects are with respect to the size of background and the size of parallax that object produces.In order to reduce this error, the available required skew of a part of image calculation calibration image background that does not comprise parallax or contained parallax minimum and background maximum.

The mathematical method hypothesis image of skew usefulness is unlimited between the computed image background.Realtime graphic is limited, thereby these mathematical methods are tended to estimating skew.Before calculating skew with mathematical method, forcing the image girth is zero, often can overcome this influence.Utilizing image to adopt with border (girth) value is zero function (such as the binary cosine function), realizes this point, when doing like this, finds and can correct calculation be offset that the less of background that difference image only is subjected to not offset fully influences.

Other source of error is a noise.Yet its total influence is generally very little.

The another source of error of background offset is the pixelation of photo unit.Realtime graphic is because its characteristic, and is discrete on basic border, thereby when 2 different commandings elevation are observed, accurately registration image background or offset background fully.This is because the edge to picture can always really not be positioned at pixel boundary in the image.Overlapping with pixel boundary usually to the edge of picture, thereby be offset and can always do not equal whole pixel.

Image blurring by making, thus each pixel intensity value is the aviation value of its surrounding pixel value, can major part overcome this error.Find that in example gaussian filtering and medium filtering are effective especially in this respect.

Accurate collimation optical device during the manufacturing can make image rotate the error that causes and greatly reduce.The system that adopts single pick up camera to constitute, thus exposure is identical with the aperture control system, so that obtain 2 images that are not subjected to the different influence of illumination intensity.Can make illumination error minimum like this.So, adopt single pick up camera and reflector/lens system obtain the ken of separating on the space, the gained image is focused on half and half part of imaging device in the single pick up camera, so just can obtain parallax effect.Show and on half and half part of imaging device, to separate image.Available switching device focuses on the pick up camera after selecting required image.This point is above discussed.

In case calculate 2 skews between the initial collection image background, promptly this information left in nonvolatile memory and use for the back.Because a pick up camera and a door left side are maintained fixed the position mutually, do not need each application drawing is calculated background offset as severe.So just save the processing time.Be preferably in calibration phase and calculate background offset.

Subtract each other after the alignment background, so that produce difference image.This point shown in Figure 22 c.As seen, background on the image and elevator threshold are roughly offset among the figure, and outstanding obstacle (this example is a ladder).In order to strengthen the parallax effect, the gained image preferably applies thresholding, thereby produces the parallax performance that intensity significantly improves, shown in Figure 22 d.

Preferably difference image only comprises the profile of 3 dimensional objects.Yet outstanding image on the gained parallax in fact is shown in Figure 22 d.This image has the component part of door and threshold.As indicated above, from the position of right these unit of calibration phase, thereby covered.

This method to detect the passenger near or to enter elevator particularly useful.This is because along with the increase of object height, it is remarkable more that parallax effect becomes, thus more accurate and cognitive disorders more clearly.

Figure 23 illustrates the result who places various obstacle samples near the elevator door outside.Figure 23 a, 23d, 23j and 23m illustrate chest, club, the soft toy (representing animal) on chest, the carpet respectively and the passenger's that walks close to leg.The difference image (Figure 23 c, 23f, 23i, 23l and 23o) that corresponding difference image (Figure 23 b, 23e, 23h, 23k and 23n) also is shown and roughly applies thresholding.As seen, the carpet that has factor can the overslaugh background effectively subtract each other among the figure.Yet even to significant background, the step that applies thresholding also can significantly be strengthened machine recognizable obstacle location.

Therefore, when placement of obstacle detection district or moving disorder, produce machine recognizable parallax effect.Shown in Figure 23 a, 23b and 23c, mainly produce parallax by image section corresponding to the vertical edge of chest, rather than cooresponding people of its horizontal edge.This is because pick up camera is shifted in elevator doorway top, horizontal, thus horizontal parallax effect minimum.The parallax that the edge of left-hand door produces is also clearly visible, and can see that the degree of parallax reduces, even disappears during near threshold or floor area at the door edge.

Figure 24 illustrates the ability that filtering technique (above discussing) reduces the pixelation fault image of image example same as shown in Figure 23, and Figure 24 c, 24f and 24i (when comparing with Figure 23 f, 23i and 23o) illustrate and reduce the feature that background is offset remaining degree and do not suppressed the parallax generation.But can see to the get on the right track horizon that causes of the threshold of previous wind and adopt now and disappear, thereby the validity of this technology is remarkable.If these features belong to background rather than belong to the parallax effect that obstacle causes, then cater to the need.

Found that described parallax disorder detection method can be used for hand or other obstacles on the detecting gate edge.In the experiment, the parallax machine that the hand on the door edge produces can be known detection.Obviously, if the method is put to practicality, needs to distinguish the door edge and originally penetrate the parallax of generation and have hand or the parallax of other obstacles generation.Image inside door mentioned above edge method of identification can be used for this purpose.

The another kind of method that can be used for discerning above-mentioned obstacle obtains the reference picture at elevator door edge under clog-free situation.The car door edge image of record compares continuously in this reference picture and the elevator utilization.If hand is placed between the door, " operation " figure image subtraction reference picture just from newly obtaining.If there is obstacle, with regard to the poor visibility partial image, otherwise difference image is zero.The example of this subtraction shown in Figure 25 d to 25f.Reference picture 25b and 25e illustrate clog-free situation, and image 25a and 25d are respectively the application drawing picture.Subtracting each other gained image 25f and 25c represents and has hand and in the reflection of closing the door pillar edge.

In the experiment, found that the present invention can detect the various suitable parallaxes of objects greatly by machine.There is the theoretic limit in the parallax that the camera chain of known various camera parameters (such as maximal oxygen camera interval and height) can detect.Therefore, the object that is positioned at the following object of top, floor 200mm and is placed on elevator doorway drift angle appears being difficult to detect.Yet,, might overcome this limit by exploitation camera optics device and geometry.

Developed said method, automatically the intramarginal threshold of detecting gate.According to the requirement of the 1st detection zone, parallax method and reference picture method are all worked well, but all produce direct machine recognition feature corresponding to the obstacle that exists in the door obstacle detection district, yet edge detection method mentioned above is firm more credible to key area.During the object of general parallax method in detecting the 2nd barrier, especially effective.

Therefore, the invention provides a kind of obstacle detection system of remarkable improvement, the object in energy reliable Detection door edge and the big guard plot.The change of imaging parameters only improves the thresholding of this detection, especially to parallax method.Most of background can also be removed reliably by this system from image, help to handle further.In addition, also find the hand or other obstacles that are placed on an edge by image being divided into the 1st detection zone and the 2nd detection zone, can detecting reliably.It will be understood by those skilled in the art that and have many conversion and modification.Comprising replacing various types of pick up cameras or imaging device.And, also can reduce to 1 to image gathering device by all optical systems as indicated above.This can significantly save cost requiring to provide aspect the viewpoint of separating on the space.

Although under the situation of elevator door, the present invention is described, by suitable modification, be used for other obstacle detection among the present invention, such as related obstacle detection such as heavy-duty machine tool, process processing controls, security personnel.

More than in the explanation,, then, include these efficacious prescriptions faces at this by such with indivedual explanations with reference to the specific composition of known equivalents of the present invention aspect part or whole complete.

Although by example and with reference to embodiment the present invention is described, should understands and to carry out various modifications or improvement to it and do not depart from scope of the present invention or spirit.

Claims (44)

1, object detection method in a kind of zone is characterized in that, this method comprises and obtains the image that one or more should the zone; Adopt edge detection method, its approach roughly occupies excellent lineament for giving prominence in the image, and judges whether to occupy excellent lineament and stipulate that this district's lineament is crossing.

2, object detection method in the zone as claimed in claim 1 is characterized in that, this zone is the object detection district, and is divided at least 2 districts, the volume that the 1st district has door and threshold to describe, and the 2nd district comprises that outdoors the passenger is with a volume that will pass through.

3, object detection method in the zone as claimed in claim 2 is characterized in that, the 1st district is the door and the threshold of elevator, and the 2nd district waits for the elevator platform/entrance hall of elevator for the passenger.

4, as object detection method in the described zone of each claim in the claim 1 to 3, it is characterized in that having 2 images at least, and this method also comprises the step that detects 2 or a plurality of image parallactics, this parallax is producing picture because of region memory; Particularly, this object is in the 2nd district.

5, object detection method in a kind of zone, this zone is by door and/or threshold regulation, it is characterized in that, described method comprises the employing edge detection method, it is by way of being roughly to occupy excellent lineament in the outstanding image, and judges that whether occupy excellent lineament intersects with the lineament of described door of regulation and/or threshold.

6, object detection method in the zone as claimed in claim 5, it is characterized in that, this method comprises the preproduction phase of the feature of describing one or more images, so that set up some characteristic lineaments that this zone presents, best described characteristic occupies the lines that excellent lineament is regulation door edge and/or threshold, deposit described feature and occupy excellent position, be provided with the back reference.

7, object detection method in the zone as claimed in claim 5, it is characterized in that, this method also can comprise the operational phase, one or more images of this phase analysis, so that set up some non-characteristic features that this presents in long-pending, described non-characteristic features embodies potential object and/or obstacle in this zone.

8, as object detection method in claim 6 or the 7 described zones, it is characterized in that the preproduction phase comprises at least 2 steps, the 1st step detects the position and the size of threshold, and the 2nd step detects the position and the size at one or more edges.

9, object detection method in the zone as claimed in claim 8 is characterized in that, the 1st step comprises:

Adopt approximate horizontal edge and/or approximate vertical edge detection filter device to give prominence in the image section of known threshold apparent position of living in and occupy excellent vertical bar and/or horizontal line; To produce every capable intensity values of pixels summation in the image with vertical margin and/or horizontal edge detection filter device, thereby produce vertical and/or horizontal function, its maxim and minimum value are corresponding to horizontal linearity feature and/or the special position of giving birth to of vertical linearity, and described lineament is according to the locus of horizontal properties in the image and vertical features specified thresholds.

10, regional as claimed in claim 8 or 9 interior object detection method is characterized in that the 2nd step comprises:

Physical relation between the threshold locus that utilization is understood and the threshold of understanding and the door edge obtains in the subimage of image or door;

Make subimage be subjected to the edge detection filter effect, this filter adaptation is in outstanding edge towards the angle between the known boundary;

Control this subimage producing binary picture, this binary picture comprise with the cooresponding a plurality of lineaments in door edge in one;

Derive the equation of binary picture neutral line feature.

11, object detection method in the zone as claimed in claim 10 is characterized in that, this known boundary is approximate vertical edge and approximate horizontal edge.

As object detection method in claim 10 or the 11 described zones, it is characterized in that 12, derive before the binary picture neutral line characteristic equation, the 2nd step also can comprise:

Handle binary picture with amplitude in the slant function that vertical direction increases.

Further handle this image, some occupy excellent lineament so that in the clear identification binary picture, this processing comprises with the 1st filter and leaches in the binary picture roughly isolated feature, and to the feature of binary picture with some substantial linear in the 2nd filter minimizing image.

13, annotate object detection method in the zone described in 10 to 12 as the right village, it is characterized in that, the lineament equation is obtained in the lines location by utilizing method of least squares or similar approach; Occupy excellent lineament more than 1 if having in the image, the equation of an arbitrary lineament of decision is promptly removed this lineament from image, and is set up another and occupy excellent lineament equation.

14, as object detection method in the described zone of claim 10 to 13, it is characterized in that, with always adding the valuation of handling each lineament equation in aviation value, thereby improve the confidence level of this lineament equation, by following or a plurality of weighted averages are carried out normalization or quadrature, obtain this total weighted average, these weighted averages comprise

The 1st weighted average wherein determines lineament to count derivative and variance, and unique point distance and derivative change to outside the given parameter, and representative image is interrupted, thereby the 1st weighted average or is removed this point from valuation to weighting under the described point, and/or

The 2nd weighted average makes away from the point of the some weighting in the lineament of image capture sources greater than close image capture sources in this image, and/or

The 3rd weighted average, the 3rd weighted average is the inverse of characteristic derivative, and/or

The 4th weighted average is weighted the lineament of crossing over any subimage from the vertical margin to the vertical margin.

15, as object detection method in the described zone of above arbitrary claim, it is characterized in that, utilize filter, differentiator etc. to implement rim detection.

16, object detection method in the zone as previously discussed is characterized in that, described rim detection is at the excellent lines that occupy in the outstanding image, these lines be oriented approximate horizontal, vertically with roughly become diagonal line; Particularly, these diagonal lines are spent for 45 degree and 135 roughly.

17, object detection method in the zone as claimed in claim 8 is characterized in that,

Operational phase may further comprise the steps:

Catch one or more real time operation images in this zone;

The position of door in the detected image;

Detect and represent the object that occurs in the image-region of threshold;

Detect the object that occurs in the image-region of representing the door edge.

18, object detection method in the zone as claimed in claim 17 is characterized in that, by the variation of intensity in the approximate horizontal feature that detects threshold, obtains the position of door, the locus of door in the Strength Changes specified image.

19, object detection method in the zone as claimed in claim 17, it is characterized in that, occupy excellent vertical features by adopting at least in the image that the approximate vertical edge detection filter is outstanding and the threshold lineament is crossing, judge in the image-region of representing threshold to have object.

20, object detection method in the zone as claimed in claim 17 is characterized in that, by adopting dominant character in the image that edge detection filter is outstanding and the door lineament is crossing at least, judges in the image-region of representing the door edge to have object.

21, object detection method in the zone described in claim 17 to 20, it is characterized in that, control step comprises that the image transformation of measuring the edge is a statistical graph, the peak value of this statistical graph is equivalent to the feature in the image, described feature is represented door and/or threshold, and/or obstacle on door edge and/or the threshold.

22, object detection method in the zone described in claim 17 to 21 is characterized in that the operational phase can repeat repeatedly.

23, a kind of object and/or object movement detection method is characterized in that, this method comprises the step that detects in 1 zone 2 or a plurality of image parallactics, and this parallax produces because of there is object in this zone.

24, method as claimed in claim 23 is characterized in that this method comprises the step that detects this area image transient change.

25, as claim 23 or 24 described sides, this method comprises detection and is positioned at the vertical and horizontal that this regional object produces, the step of parallax.

26, method as claimed in claim 25 is characterized in that, comprises following steps: the background of a plurality of images in 1 zone of aliging; There is object in image subtraction so that represent this zone by parallax in pairs.

27, as object detection method in claim 23 or the 26 described zones, it is characterized in that this object detection method comprises following steps: the background of the 1st image and the 2nd image in 1 zone of aliging; From the 2nd figure image subtraction the 1st image, thereby represent existence 3 dimensions to object by parallax.

28, as object detection method in claim 23, the 26 or 27 described zones, it is characterized in that the method includes the steps of:

Collect the 1st image in 1 zone from the 1st viewpoint;

Collect the 2nd image in this zone from the 2nd viewpoint;

Calculate the skew between these 2 image backgrounds;

The align background of these 2 images;

These 2 image subtractions are to produce the 3rd difference image;

Analyze the 3rd difference image, measure parallax, have 3 dimensional objects thereby represent this zone.

29, object detection method in the zone as claimed in claim 28 is characterized in that, after the subtraction step, before the analytical procedure, exists thresholding to apply step, thus, difference image is applied thresholding, with the eliminating noise, thereby produces binary picture.

30, as object detection method in claim 28 or the 29 described zones, it is characterized in that, the 3rd difference image is handled, so that the profile of some 3 dimensional objects in the inclusion region only roughly.

As the described method of each claim in the claim 28 to 30, it is characterized in that 31, image division is background image and door edge image; The image of background during according to no object calculates the skew that needs between 2 image backgrounds.

32, object detection method in the zone described in claim 28 to 31 is characterized in that, adopts crosscorrelation to calculate this skew.

33, object detection method in the zone described in claim 28 to 32 is characterized in that, utilizes Gaussian filter, median filter or similar filter to use image blurringization, so that reduce the pixelation effect in the image.

34, object test equipment in a kind of zone is characterized in that, described device comprises at least 1 imaging device and 1 micro processor, apparatus that adapts to the above described method of arbitrary claim of realization.

35, object test equipment in a kind of zone is characterized in that, this device comprises: at least 1 imaging device adapts to the image that the viewpoint of separating from least 2 spaces forms identical scene;

Micro processor, apparatus adapt to be handled described image, and its method is for roughly occupying excellent lineament in the outstanding described image, and Duan more whether not occupy excellent lineament and represent that there is object in this zone.

36, object test equipment in a kind of zone is characterized in that, this device comprises:

At least 1 imaging device adapts to the image that the viewpoint of separating from least 2 spaces forms roughly the same scene;

Micro processor, apparatus adapt to be handled described image, so that calculate the skew between the right background of 2 images or image, according to described offset alignment background image, the gained image subtraction, to produce difference image, thereby can detect the parallax effect in the difference image, represent that there is object in this zone.

37, as object test equipment in the described zone of each claim in the claim 34 to 36, it is characterized in that microprocessor also adapts to the processing image, roughly to occupy excellent lineament in the outstanding image.

38, object detection method in the zone described in claim 34 to 37 is characterized in that, available optical mode, and mathematical way or other similar fashion are handled image, and this mode represents and occupies excellent lineament and/or parallax in the area image.

39, object detection method in the zone described in claim 34 to 38 is characterized in that microprocessor also adapts to difference image is applied thresholding.

40, as object detection method in the described zone of claim 34 to 39, it is characterized in that the form of microprocessor can be solid-state device, optical device or device similar with it.

41, object detection method in the zone described in claim 34 to 40, it is characterized in that, when adopting 1 pick up camera, this device comprises optical branch and reflective devices, and this optical branch and reflective devices adaptation are carried out relaying to the image from the viewpoint of leaving this pick up camera actual position.

42, object detection method in the zone described in claim 34 to 41, it is characterized in that, the collection of 2 or a plurality of images can be implemented by the optical unit that comprises prism, coherent optic fiber wave guide etc., perhaps can be transformed into as device proper or makes the suitable displacement of this device proper.

43, object detection method in the zone described in claim 34 to 42 is characterized in that, can add dummy feature, to help roughly regularly in the outstanding image of microprocessor to occupy excellent feature.

44, object detection method in the zone described in claim 34 to 42 is characterized in that, also can comprise input media, this input media makes the user can be the position input microprocessor of regular dominant character.

Images