CN110009650B - Escalator handrail boundary area border crossing detection method and system - Google Patents

Escalator handrail boundary area border crossing detection method and system Download PDF

Info

Publication number
CN110009650B
CN110009650B CN201811568311.2A CN201811568311A CN110009650B CN 110009650 B CN110009650 B CN 110009650B CN 201811568311 A CN201811568311 A CN 201811568311A CN 110009650 B CN110009650 B CN 110009650B
Authority
CN
China
Prior art keywords
roi
point
image
quadrilateral
region
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811568311.2A
Other languages
Chinese (zh)
Other versions
CN110009650A (en
Inventor
毕举
施行
王超
吴磊磊
朱鲲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Xinzailing Technology Co ltd
Original Assignee
Zhejiang Xinzailing Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Xinzailing Technology Co ltd filed Critical Zhejiang Xinzailing Technology Co ltd
Priority to CN201811568311.2A priority Critical patent/CN110009650B/en
Publication of CN110009650A publication Critical patent/CN110009650A/en
Application granted granted Critical
Publication of CN110009650B publication Critical patent/CN110009650B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method for detecting the border area border crossing of an escalator handrail, which comprises the following steps: acquiring escalator operation video images from the right front of the escalator through a video image acquisition unit, and using the acquired image data for subsequent unit analysis; analyzing according to the video image information transmitted by the video image acquisition unit through the border crossing detection analysis unit to obtain a border crossing alarm signal value; and outputting voice reminding information through the multimedia prompting unit according to the received alarm signal. The invention only processes the ROI, thus improving the processing precision and speed.

Description

Escalator handrail boundary area border crossing detection method and system
Technical Field
The invention belongs to the technical field of escalator safety, and particularly relates to a method and a system for detecting border area border crossing of an escalator handrail.
Background
At present, the escalator is applied to places with dense flows, such as various malls, subway stations, railway stations, airports and the like, and the escalator brings great convenience to people and can cause loss of lives and properties of people due to improper use. For example, when someone stretches his hand out of the boundary of the handrail or stretches his upper body out of the boundary, if there is a shelter in front of the handrail, if the warning voice is not played or the operation speed of the escalator is slowed down, the passenger is likely to collide with the barrier, the personal safety of the passenger is damaged, and even the passenger at the back falls down, and falls down, so that the domino effect is caused, and a more serious casualty event is caused. Therefore, the border crossing event of the escalator handrail is detected, corresponding voice and security measures are taken at the first time, and tragedies can be avoided.
The chinese patent application CN 101695983 a discloses an escalator energy-saving and safety monitoring system based on omnidirectional computer vision, which comprises a microprocessor, an omnidirectional vision sensor and an escalator PLC controller. Wherein the microprocessor includes: the system comprises a video image reading module, a concerned area customizing module (including the determination of the handrail boundary), an out-of-range detecting module and an energy-saving control module, wherein the out-of-range detecting module is mainly used for detecting whether passengers and carried objects of the escalator exceed the handrail boundary, and comprehensively judging the panoramic video image stored in a dynamic memory by adopting an edge detection algorithm and a Gaussian mixture background modeling algorithm: and under the condition that the non-handrail edge is detected around the handrail boundary, whether the non-handrail edge belongs to the foreground object is determined through a Gaussian foreground object modeling algorithm, and if the outer edge of the foreground object is detected to cross over the outer edge of the handrail of the escalator by more than 5 pixels, the boundary crossing is determined. The technical scheme has the following defects: the whole image is used for detection and modeling, CPU and memory resources are wasted, and the cost is increased; the modeling judgment is carried out only under the condition that the non-handrail edge is detected around the handrail boundary, and the condition of missing report can occur if the non-handrail edge cannot be detected; the outer edge of the foreground object crosses over 5 pixels of the outer edge of the handrail of the escalator to be judged as out-of-range, the judgment condition is too simple, only space factors are considered, time is not considered, and more false alarms can occur.
Disclosure of Invention
The invention aims to solve the technical problem of providing an escalator border crossing detection method and system, which only process an ROI (region of interest) region and improve the processing precision and speed.
In order to solve the technical problems, the invention adopts the following technical scheme:
a method for detecting border area border crossing of an escalator handrail comprises the following steps:
acquiring escalator operation video images from the right front of the escalator through a video image acquisition unit, and using the acquired image data for subsequent unit analysis; analyzing according to the video image information transmitted by the video image acquisition unit through the border crossing detection analysis unit to obtain a border crossing alarm signal value; outputting voice reminding information according to the received alarm signal through a multimedia prompting unit; the border crossing detection and analysis unit analyzes the video image information transmitted by the video image acquisition unit and specifically comprises the following steps:
acquiring an ROI area image of a region of interest (ROI), wherein the ROI area image comprises an ROI rectangular area image and an ROI quadrilateral area template; establishing a gray level image background for the ROI rectangular region by adopting a Gaussian Mixture Modeling (GMM) method; carrying out background difference and binarization; morphological processing of the binary image; counting the foreground proportion value in the ROI quadrilateral template and the duration time of which the area ratio is greater than a threshold value;
and if the foreground proportion in the current frame ROI quadrilateral template is larger than a set threshold and the duration time is larger than the set threshold, sending out an out-of-range warning, selecting a region which is easy to have potential safety hazards outside the handrail in the ROI rectangular region, wherein the region is represented by a quadrilateral formed by four points, the connecting lines of two points are parallel and close to the handrail boundary, and the connecting lines of the other two points are outwards far away from the handrail boundary.
Preferably, the acquiring the ROI rectangular region image specifically includes: according to the vertex of the input quadrangle, the minimum x coordinate value min _ x and the minimum y coordinate value min _ y, and the maximum x coordinate value max _ x and the maximum y coordinate value max _ y are found, so that an ROI rectangular region is obtained.
Preferably, the obtaining of the ROI quadrilateral region template specifically includes:
drawing straight lines on the input quadrilateral vertexes in pairs in sequence to form a closed ROI quadrilateral, wherein a Bresenham algorithm is adopted when drawing the straight lines; and (4) performing quick scanning line filling algorithm on the quadrilateral operation.
Preferably, the establishment of the gray level image background for the ROI rectangular region by using the gaussian mixture modeling GMM method specifically includes:
assigning an initial mean value, standard deviation and weight to each pixel point of the gray level image;
collecting N frames of images, and obtaining the mean value, the standard deviation and the weight of each pixel point by using an online EM (effective noise) algorithm;
starting detection from an N +1 frame, wherein the detection method comprises the following steps:
for each pixel point: sorting all Gaussian kernels according to omega/sigma descending order; selecting the first M gaussian kernels satisfying the following formula: m ═ argmin (ω/σ > T); if one of the pixel values of the current pixel point satisfies: the (| x-mu _ i |)/sigma _ i < K can be regarded as a background point;
the background image is updated using the online EM algorithm.
Preferably, the background difference and binarization specifically comprises:
subtracting the background image from the current frame to obtain a differential image Pd;
and selecting a threshold th1, and binarizing the difference image Pd to obtain a binary image Bd.
An escalator handrail boundary region out-of-range detection system, comprising:
the video image acquisition unit is used for acquiring escalator operation video images from the right front of the escalator and using the acquired image data for subsequent unit analysis; the border crossing detection and analysis unit is used for analyzing according to the video image information transmitted by the video image acquisition unit to acquire a border crossing alarm signal value; the multimedia prompt unit is used for outputting voice prompt information according to the received alarm signal; the border crossing detection and analysis unit analyzes the video image information transmitted by the video image acquisition unit and specifically comprises the following steps:
acquiring a Region Of Interest (ROI) Region image comprising an ROI rectangular Region image and an ROI quadrilateral Region template; establishing a gray level image background for the ROI rectangular region by adopting a Gaussian Mixture Modeling (GMM) method; carrying out background difference and binarization; morphological processing of the binary image; counting the foreground proportion value in the ROI quadrilateral template and the duration time of which the area ratio is greater than a threshold value;
and if the foreground proportion in the current frame ROI quadrilateral template is larger than a set threshold and the duration time is larger than the set threshold, sending out an out-of-range warning, selecting a region which is easy to have potential safety hazards outside the handrail in the ROI rectangular region, wherein the region is represented by a quadrilateral formed by four points, the connecting lines of two points are parallel and close to the handrail boundary, and the connecting lines of the other two points are outwards far away from the handrail boundary.
Preferably, the acquiring the ROI rectangular region image specifically includes: according to the vertex of the input quadrangle, the minimum x coordinate value min _ x and the minimum y coordinate value min _ y, and the maximum x coordinate value max _ x and the maximum y coordinate value max _ y are found, so that an ROI rectangular region is obtained.
Preferably, the obtaining of the ROI quadrilateral region template specifically includes:
drawing straight lines on the input quadrilateral vertexes in pairs in sequence to form a closed ROI quadrilateral, wherein a Bresenham algorithm is adopted when drawing the straight lines; and (4) performing quick scanning line filling algorithm on the quadrilateral operation.
Preferably, the establishment of the gray level image background for the ROI rectangular region by using the gaussian mixture modeling GMM method specifically includes:
assigning an initial mean value, standard deviation and weight to each pixel point of the gray level image;
collecting N frames of images, and obtaining the mean value, the standard deviation and the weight of each pixel point by using an online EM (effective noise) algorithm;
starting detection from an N +1 frame, wherein the detection method comprises the following steps:
for each pixel point: sorting all Gaussian kernels according to omega/sigma descending order; selecting the first M gaussian kernels satisfying the following formula: m ═ argmin (ω/σ > T); if one of the pixel values of the current pixel point satisfies: the (| x-mu _ i |)/sigma _ i < K can be regarded as a background point;
the background image is updated using the online EM algorithm.
Preferably, the background difference and binarization specifically comprises:
subtracting the background image from the current frame to obtain a differential image Pd;
and selecting a threshold th1, and binarizing the difference image Pd to obtain a binary image Bd.
The invention has the following beneficial effects: only processing an ROI area, mainly providing a rapid scanning line filling method aiming at a quadrangle to acquire an ROI quadrangle area template through four vertexes of the input ROI (region of interest) quadrangle, then detecting a motion foreground by using a Gaussian mixture model for an ROI rectangular area image, then calculating the area ratio of the motion foreground in the ROI quadrangle template and the duration time of which the area ratio is greater than a set threshold, finally judging whether border crossing exists according to whether the duration time is greater than the set threshold, sending a border crossing alarm to a multimedia prompting unit, and performing behavior induction on an on-site passenger by the multimedia prompting unit according to a received alarm value; meanwhile, the escalator control unit adjusts the running speed of the escalator according to the alarm value, slows down or gradually stops, and avoids causing adverse consequences.
Drawings
FIG. 1 is a schematic block diagram of an escalator out-of-range detection system according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a ROI quadrilateral photo in the escalator border crossing detection system according to the embodiment of the invention;
FIG. 3 is a schematic view of a photograph of ROI rectangular region determination in the escalator border crossing detection system according to the embodiment of the present invention;
FIG. 4 is a schematic view of an ROI rectangular region image in an escalator border crossing detection system implemented by the present invention;
FIG. 5 is a schematic diagram of a ROI quadrilateral region template of the escalator border crossing detection system according to the embodiment of the invention;
fig. 6 is a flowchart of a rapid scan line filling algorithm for a quadrilateral for an escalator border crossing detection system according to an embodiment of the present invention;
FIG. 7 is a schematic view of the corrosion principle of the escalator out-of-range detection system according to the embodiment of the present invention;
fig. 8 is an expansion principle schematic diagram of the escalator border crossing detection system according to the embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1 to 8, an escalator border crossing detection system according to an embodiment of the present invention includes:
and the video image acquisition unit is used for acquiring a video image of the escalator operation from the right front of the escalator and using the acquired image data for subsequent unit analysis, wherein the video image acquisition unit comprises but is not limited to a CCD (charge coupled device) camera, a network camera and the like, is arranged right front of an entrance and exit of the escalator, and horizontally shoots the video image of the escalator operation forwards.
And the boundary crossing detection and analysis unit is used for analyzing according to the video image information transmitted by the video image acquisition unit to acquire a boundary crossing alarm signal value, and comprises but not limited to general processing equipment such as a CPU, an ARM, a DSP, a GPU, an FPGA, an ASIC and the like.
And the multimedia prompt unit is used for outputting voice prompt information according to the received alarm signal. The multimedia prompting unit comprises but not limited to a liquid crystal display screen, a loudspeaker and other equipment with video and audio display capability, and plays reminding persuasion video information on the display screen and audio reminding information in the loudspeaker after receiving the alarm signal.
The border crossing detection and analysis unit analyzes the video image information transmitted by the video image acquisition unit and specifically comprises the following steps:
acquiring an ROI area image of a region of interest (ROI), wherein the ROI area image comprises an ROI rectangular area image and an ROI quadrilateral area template; establishing a gray level image background for the ROI rectangular region by adopting a Gaussian Mixture Modeling (GMM) method; carrying out background difference and binarization; morphological processing of the binary image; counting the foreground proportion value in the ROI quadrilateral template and the duration time of which the area ratio is greater than a threshold value;
and if the foreground proportion in the current frame ROI quadrilateral template is larger than a set threshold and the duration time is larger than the set threshold, sending out an out-of-range warning, selecting a region which is easy to have potential safety hazards outside the handrail in the ROI rectangular region, wherein the region is represented by a quadrilateral formed by four points, the connecting lines of two points are parallel and close to the handrail boundary, and the connecting lines of the other two points are outwards far away from the handrail boundary.
The obtaining of the ROI rectangular region image specifically comprises the following steps: according to the vertex of the input quadrangle, the minimum x coordinate value min _ x and the minimum y coordinate value min _ y, and the maximum x coordinate value max _ x and the maximum y coordinate value max _ y are found, so that an ROI rectangular region is obtained.
The method for acquiring the ROI quadrilateral region template specifically comprises the following steps:
drawing straight lines on the input quadrilateral vertexes in pairs in sequence to form a closed ROI quadrilateral, wherein a Bresenham algorithm is adopted when drawing the straight lines; and (4) performing quick scanning line filling algorithm on the quadrilateral operation.
The method for establishing the gray level image background for the ROI rectangular region by adopting a Gaussian Mixture Modeling (GMM) method specifically comprises the following steps:
assigning an initial mean value, standard deviation and weight to each pixel point of the gray level image;
collecting N frames of images, and obtaining the mean value, the standard deviation and the weight of each pixel point by using an online EM (effective noise) algorithm;
starting detection from an N +1 frame, wherein the detection method comprises the following steps:
for each pixel point: sorting all Gaussian kernels according to omega/sigma descending order; selecting the first M gaussian kernels satisfying the following formula: m ═ argmin (ω/σ > T); if one of the pixel values of the current pixel point satisfies: the (| x-mu _ i |)/sigma _ i < K can be regarded as a background point;
the background image is updated using the online EM algorithm.
The background difference and binarization specifically comprises the following steps:
subtracting the background image from the current frame to obtain a differential image Pd;
and selecting a threshold th1, and binarizing the difference image Pd to obtain a binary image Bd.
In a specific application example, the escalator border-crossing detection system provided by the embodiment of the invention has the following working process:
A. the video image acquisition unit acquires an escalator operation video image from the right front of the escalator, the input ROI quadrangle vertex is shown in figure 2, the ROI quadrangle and the ROI rectangular region are shown in figure 3, wherein the input ROI quadrangle vertex is marked with 1, 2, 3 and 4 in figure 2, the quadrangle is the ROI quadrangle region drawn according to the input vertex in figure 3, the rectangle is the ROI rectangular region calculated according to the input vertex, and the ROI rectangular image module is acquired from the border-crossing detection analysis unit by a specific calculation method.
B. The image collected by the video image collecting unit is sent to the border crossing detection and analysis unit, and the analysis process is as follows:
1. and acquiring an ROI area image comprising an ROI rectangular area image and an ROI quadrilateral area template.
1.1. Obtaining ROI rectangular region image
According to the vertex of the input quadrangle, the minimum x coordinate value min _ x and the minimum y coordinate value min _ y, and the maximum x coordinate value max _ x and the maximum y coordinate value max _ y are found, so that a ROI rectangular region is obtained, in order to obtain a subsequent ROI quadrangle region template, the ROI rectangular region is expanded by one pixel outwards, and then a sub-image of the ROI rectangular region is obtained from a source image, as shown in the image in FIG. 4.
1.2. Obtaining ROI quadrilateral region template
In order to obtain the ROI quadrilateral region template, a rapid scanning line filling method for a quadrilateral is provided for filling the ROI quadrilateral region, the method is low in consumed memory and CPU resource, low in space complexity, free of constructing a complex data structure and low in time complexity, and the algorithm can complete filling only by traversing the ROI region once. The traditional ordered edge table scanning line filling method is high in space complexity, an Edge Table (ET), an Active Edge Table (AET) and other data structures need to be established, time complexity is high, intersection points of scanning lines and all sides of a polygon need to be required, floating point number operation is involved, time is consumed, steps of ordering, intersection point matching, interval filling and the like are required to be performed on the intersection points, and operation time is increased.
1.2.1. Inputting the vertex of the quadrangle, drawing straight lines two by two in sequence, thereby enclosing a closed ROI quadrangle. And a Bresenham algorithm is adopted when a straight line is drawn, the operation speed of the algorithm is high, and floating point number operation is not required. Assuming that the slope k of the line is > 0 and the line is in the first quadrant, the Bresenham algorithm principle is as follows:
a. drawing starting point (x)1，y1).
b. And (5) preparing to draw the next point, adding 1 to the x coordinate, and finishing if the terminal point is reached. Otherwise, finding the next point, which is either the right adjacent point of the current point or the upper right adjacent point of the current point. Subtracting the distances from the two points to the point on the straight line, judging the positive and negative of the two points, if the distance from the lower point to the actual point of the straight line is far, then d1-d2 > -0, then taking the upper point y1+1, so that the next point can be selected according to the sign of the Δ d directly, and the calculation rule of the Δ d is as follows:
(1) the initial value of Δ d is 2 × dy-dx
(2) When Δ d < 0, Δ d +2 dy
(3) When Δ d ═ 0, Δ d ═ Δ d +2 dyd-2 dx
c. Drawing points
d. Jump back to step b
e. And (6) ending.
1.2.2. A rapid scanning line filling algorithm is provided for a quadrangle, a filled ROI quadrangle template is shown in FIG. 5, a flow chart is shown in FIG. 6, and the method comprises the following specific steps:
a. scanning the ROI rectangular area in lines;
b. recording a current pixel value curval and a previous pixel value pre _ val using two variables, and a state variable change _ state to determine when to start and end filling;
c. if cur _ val is 0 and pre _ val is not 0, change _ state is incremented by 1. If changestate is 1, the current pixel value is set to 255, and the x-coordinate value xs at the start of filling is saved. If change _ state is 2, stopping the line scanning and filling;
d. if change _ state is 1 at this time, it means that there is only one intersection point between the scanning line and the quadrangle, and it is necessary to trace back the previously filled pixel to xs, and set the pixel value to 0;
e. scanning is completed until the whole ROI area is finished.
2. Background modeling
And establishing a gray level image background for the ROI rectangular region by adopting a GMM (Gaussian mixture modeling) method. The modeling process of the Gaussian mixture background is as follows
2.1. An initial mean, standard deviation and weight are assigned to each pixel point of the gray scale image.
2.2. Collecting N (generally more than 200, otherwise, the result of the image sample is difficult to obtain) frame images, and obtaining the mean value, the standard deviation and the weight of each pixel point by using an online EM algorithm.
2.3. Starting detection from the N +1 frame, and the detection method comprises the following steps:
for each pixel point:
2.3.1. all Gaussian kernels are expressed in
Figure BDA0001913434380000091
Sort in descending order
2.3.2. Selecting the first M gaussian kernels satisfying the following formula:
Figure BDA0001913434380000092
2.3.3. if one of the pixel values of the current pixel point satisfies:
Figure BDA0001913434380000093
it can be considered as a background point.
2.4. The background image is updated using the online EM algorithm.
3. Background differencing and binarization
3.1. Subtracting the background image from the current frame to obtain a difference image Pd；
3.2. Selecting threshold th1 for difference image PdCarrying out binarization to obtain a binary image Bd。
4. Binary image morphological processing
4.1. For binary image BdPerforming corrosion treatment, adopting a 3x3 template, removing impurity points to obtain a binary image Be；
The corrosion principle is as follows:
firstly, defining a convolution kernel B, wherein the kernel (also called template or mask) can be in any shape and size and has a separately defined reference point-anchor point (anchorpoint);
then, performing convolution on the kernel B and the image A, namely calculating the minimum value of pixel points in the coverage area of the kernel B;
and finally, assigning the minimum value to the pixel specified by the reference point.
4.2. For binary image BePerforming expansion treatment, adopting a 3x3 template to compensate and expand the original foreground part to obtain a binary image Bf。
The principle of expansion is as follows:
firstly, defining a convolution kernel B, wherein the kernel (also called template or mask) can be in any shape and size and has a separately defined reference point-anchor point (anchorpoint);
then, performing convolution on the kernel B and the image A, and calculating the maximum value of pixel points in the coverage area of the kernel B;
finally, the maximum value is assigned to the pixel specified by the reference point.
5. And counting the foreground proportion value in the ROI quadrilateral template and the duration of the area ratio larger than a threshold value.
Traversing a binary image B obtained by performing operations such as Gaussian mixture modeling on ROI rectangular region imagefIf the current pixel is 255 and the pixel value of the position corresponding to the ROI quadrilateral template is 255, the foreground number mask _ fg _ count in the template is increased by 1, and in order to count the pixel number mask _ count in the ROI quadrilateral template, the mask _ count is increased by 1 as long as the pixel value of the position corresponding to the ROI quadrilateral template image is 255. And then calculating mask _ fg _ count/mask _ count, namely the foreground proportion fg _ ratio in the ROI quadrilateral template, and recording the continuous time continuance _ time when fg _ ratio is greater than a set threshold.
6. And if the foreground proportion fg _ ratio in the current frame ROI quadrilateral template is larger than a set threshold value and the continuous time continuance _ time is larger than the set threshold value, sending out the boundary crossing warning.
C. The multimedia prompt unit receives the alarm signal, plays the induction voice to remind passengers not to cross the border, carefully shields objects in front, and slows down the running speed of the escalator after the escalator control unit receives the alarm signal. If the alarm signal is not received, the escalator is in a normal speed running state and continues to be maintained.
Corresponding to the escalator border crossing detection system in the embodiment of the invention, the embodiment of the invention also provides an escalator border crossing detection method, which comprises the following steps:
acquiring escalator operation video images from the right front of the escalator through a video image acquisition unit, and using the acquired image data for subsequent unit analysis; analyzing according to the video image information transmitted by the video image acquisition unit through the border crossing detection analysis unit to obtain a border crossing alarm signal value; outputting voice reminding information according to the received alarm signal through a multimedia prompting unit; the border crossing detection and analysis unit analyzes the video image information transmitted by the video image acquisition unit and specifically comprises the following steps:
acquiring an ROI area image of a region of interest (ROI), wherein the ROI area image comprises an ROI rectangular area image and an ROI quadrilateral area template; establishing a gray level image background for the ROI rectangular region by adopting a Gaussian Mixture Modeling (GMM) method; carrying out background difference and binarization; morphological processing of the binary image; counting the foreground proportion value in the ROI quadrilateral template and the duration time of which the area ratio is greater than a threshold value;
and if the foreground proportion in the current frame ROI quadrilateral template is larger than a set threshold and the duration time is larger than the set threshold, sending out an out-of-range warning, selecting a region which is easy to have potential safety hazards outside the handrail in the ROI rectangular region, wherein the region is represented by a quadrilateral formed by four points, the connecting lines of two points are parallel and close to the handrail boundary, and the connecting lines of the other two points are outwards far away from the handrail boundary.
Further, the obtaining of the ROI rectangular region image specifically includes: according to the vertex of the input quadrangle, the minimum x coordinate value min _ x and the minimum y coordinate value min _ y, and the maximum x coordinate value max _ x and the maximum y coordinate value max _ y are found, so that an ROI rectangular region is obtained.
Further, the obtaining of the ROI quadrilateral region template specifically includes:
drawing straight lines on the input quadrilateral vertexes in pairs in sequence to form a closed ROI quadrilateral, wherein a Bresenham algorithm is adopted when drawing the straight lines; and (4) performing quick scanning line filling algorithm on the quadrilateral operation.
Further, the establishment of the gray level image background for the ROI rectangular region by adopting a Gaussian mixture modeling GMM method specifically comprises the following steps:
assigning an initial mean value, standard deviation and weight to each pixel point of the gray level image;
collecting N frames of images, and obtaining the mean value, the standard deviation and the weight of each pixel point by using an online EM (effective noise) algorithm;
starting detection from an N +1 frame, wherein the detection method comprises the following steps:
for each pixel point: sorting all Gaussian kernels according to omega/sigma descending order; selecting the first M gaussian kernels satisfying the following formula: m ═ argmin (ω/σ > T); if one of the pixel values of the current pixel point satisfies: the (| x-mu _ i |)/sigma _ i < K can be regarded as a background point;
the background image is updated using the online EM algorithm.
Further, the background difference and binarization specifically comprises:
subtracting the background image from the current frame to obtain a differential image Pd;
and selecting a threshold th1, and binarizing the difference image Pd to obtain a binary image Bd.
The specific implementation process corresponding to the above escalator border crossing detection method is the same as that of the above escalator border crossing detection system, and is not repeated herein.
It is to be understood that the exemplary embodiments described herein are illustrative and not restrictive. Although one or more embodiments of the present invention have been described with reference to the accompanying drawings, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the following claims.

Claims (6)

1. A method for detecting border area border crossing of an escalator handrail is characterized by comprising the following steps:
acquiring escalator operation video images from the right front of the escalator through a video image acquisition unit, and using the acquired image data for subsequent unit analysis; analyzing according to the video image information transmitted by the video image acquisition unit through the border crossing detection analysis unit to obtain a border crossing alarm signal value; outputting voice reminding information according to the received alarm signal through a multimedia prompting unit; the border crossing detection and analysis unit analyzes the video image information transmitted by the video image acquisition unit and specifically comprises the following steps:
acquiring an ROI area image of a region of interest (ROI), wherein the ROI area image comprises an ROI rectangular area image and an ROI quadrilateral area template; establishing a gray level image background for the ROI rectangular region by adopting a Gaussian Mixture Modeling (GMM) method; carrying out background difference and binarization; morphological processing of the binary image; counting the foreground proportion value in the ROI quadrilateral template and the duration time of which the area ratio is greater than a threshold value;
the method for acquiring the ROI quadrilateral region template specifically comprises the following steps: drawing straight lines on the input quadrilateral vertexes pairwise in sequence to form a closed ROI quadrilateral, wherein a Bresenham algorithm is adopted when drawing the straight lines; filling the quadrangle by using a fast scanning line filling algorithm;
drawing lines on the input quadrilateral vertexes pairwise in sequence to form a closed ROI quadrilateral, wherein the drawing of the lines by adopting a Bresenham algorithm specifically comprises the following steps:
assuming that the slope k of the line is > 0, the line is in the first quadrant, the Bresenham algorithm is as follows:
a. drawing starting point (x)1，y1)；
b. Preparing to draw the next point, adding 1 to the x coordinate, and finishing if the terminal point is reached; otherwise, finding the next point, wherein the next point is either the right adjacent point of the current point or the upper right adjacent point of the current point; subtracting the distances from the two points to the point on the straight line, judging the positive and negative of the two points, and if the distance from the lower point to the actual point of the straight line is far, taking the upper point y as d1-d2 ═ 01+1, therefore, the next point is selected directly according to the sign of Δ d, and the calculation rule of Δ d is as follows:
(1) the initial value of Δ d is 2 × dy-dx;
(2) Δ d +2 dy when Δ d < 0;
(3) Δ d +2 dy-2 dx when Δ d > ═ 0;
c. drawing points;
d. jumping back to the step b;
e. finishing;
filling the quadrangle by using a fast scanning line filling algorithm, and specifically comprising the following steps:
A. scanning the ROI rectangular area in lines;
B. recording a current pixel value curval and a previous pixel value pre _ val using two variables, and a state variable change _ state to determine when to start and end filling;
C. if cur _ val is 0 and pre _ val is not 0, change _ state is increased by 1; if change _ state is 1, setting the current pixel value to be 255, and simultaneously saving the x coordinate value xs when filling is started; if change _ state is 2, stopping the line scanning and filling;
D. if change _ state is 1 at this time, it means that there is only one intersection point between the scanning line and the quadrangle, and it is necessary to trace back the previously filled pixel to xs, and set the pixel value to 0;
E. scanning to the end of the whole ROI area;
if the foreground proportion in the ROI quadrilateral template of the current frame is larger than a set threshold and the duration is larger than the set threshold, sending out an out-of-range warning, selecting a region which is easy to have potential safety hazard outside the armrest from the ROI rectangular region, wherein the quadrilateral region is represented by a quadrilateral formed by four points, the connecting lines of two points are parallel and close to the boundary of the armrest, and the connecting line of the other two points is outwards far away from the boundary of the armrest,
the obtaining of the ROI rectangular region image specifically comprises the following steps: according to the vertex of the input quadrangle, the minimum x coordinate value min _ x and the minimum y coordinate value min _ y, and the maximum x coordinate value max _ x and the maximum y coordinate value max _ y are found, so that an ROI rectangular region is obtained.
2. The escalator handrail boundary region border crossing detection method according to claim 1, characterized in that a Gaussian mixture modeling GMM method is adopted to establish a grayscale image background for the ROI rectangular region, specifically:
assigning an initial mean value, standard deviation and weight to each pixel point of the gray level image;
collecting N frames of images, and obtaining a mean value mu _ i, a standard deviation sigma _ i and a weight omega _ i of each pixel point by using an online EM (effective memory) algorithm;
starting detection from an N +1 frame, wherein the detection method comprises the following steps:
for each pixel point: sorting all Gaussian kernels according to the descending order of omega _ i/sigma _ i; selecting the first M gaussian kernels satisfying the following formula: m ═ argmin (ω _ i/σ _ i > T); if one of the pixel values of the current pixel point meets the following conditions: the (| x-mu _ i |)/sigma _ i < K can be regarded as a background point;
and updating the background image.
3. The escalator handrail boundary region out-of-range detection method according to claim 1, characterized in that the background difference and binarization specifically comprises:
subtracting the background image from the current frame to obtain a differential image Pd;
and selecting a threshold th1, and binarizing the difference image Pd to obtain a binary image Bd.
4. An escalator handrail border region out-of-range detection system, comprising:
the video image acquisition unit is used for acquiring escalator operation video images from the right front of the escalator and using the acquired image data for subsequent unit analysis; the border crossing detection and analysis unit is used for analyzing according to the video image information transmitted by the video image acquisition unit to acquire a border crossing alarm signal value; the multimedia prompt unit is used for outputting voice prompt information according to the received alarm signal; the border crossing detection and analysis unit analyzes the video image information transmitted by the video image acquisition unit and specifically comprises the following steps:
acquiring an ROI area image of a region of interest (ROI), wherein the ROI area image comprises an ROI rectangular area image and an ROI quadrilateral area template; establishing a gray level image background for the ROI rectangular region by adopting a Gaussian Mixture Modeling (GMM) method; carrying out background difference and binarization; morphological processing of the binary image; counting the foreground proportion value in the ROI quadrilateral template and the duration time of which the area ratio is greater than a threshold value;
the method for acquiring the ROI quadrilateral region template specifically comprises the following steps: drawing straight lines on the input quadrilateral vertexes pairwise in sequence to form a closed ROI quadrilateral, wherein a Bresenham algorithm is adopted when drawing the straight lines; filling the quadrangle by using a fast scanning line filling algorithm;
drawing lines on the input quadrilateral vertexes pairwise in sequence to form a closed ROI quadrilateral, wherein the drawing of the lines by adopting a Bresenham algorithm specifically comprises the following steps:
assuming that the slope k of the line is > 0, the line is in the first quadrant, the Bresenham algorithm is as follows:
a. drawing starting point (x)1，y1)；
b. Preparing to draw the next point, adding 1 to the x coordinate, and finishing if the terminal point is reached; otherwise, finding the next point, wherein the next point is either the right adjacent point of the current point or the upper right adjacent point of the current point; subtracting the distances from the two points to the point on the straight line, judging the positive and negative of the two points, and if the distance from the lower point to the actual point of the straight line is far, taking the upper point y as d1-d2 ═ 01+1, therefore, the next point is selected directly according to the sign of Δ d, and the calculation rule of Δ d is as follows:
(1) the initial value of Δ d is 2 × dy-dx;
(2) Δ d +2 dy when Δ d < 0;
(3) Δ d +2 dy-2 dx when Δ d > ═ 0;
c. drawing points;
d. jumping back to the step b;
e. finishing;
the quick scanning line filling algorithm for quadrilateral operation specifically comprises the following steps:
A. scanning the ROI rectangular area in lines;
B. recording a current pixel value curval and a previous pixel value pre _ val using two variables, and a state variable change _ state to determine when to start and end filling;
C. if cur _ val is 0 and pre _ val is not 0, change _ state is increased by 1; if change _ state is 1, setting the current pixel value to be 255, and simultaneously saving the x coordinate value xs when filling is started; if change _ state is 2, stopping the line scanning and filling;
D. if change _ state is 1 at this time, it means that there is only one intersection point between the scanning line and the quadrangle, and it is necessary to trace back the previously filled pixel to xs, and set the pixel value to 0;
E. scanning to the end of the whole ROI area;
if the foreground proportion in the ROI quadrilateral template of the current frame is larger than a set threshold and the duration is larger than the set threshold, sending out an out-of-range warning, selecting a region which is easy to have potential safety hazard outside the handrail in the ROI rectangular region, wherein the region is represented by a quadrilateral formed by four points, the connecting lines of two points are parallel and close to the boundary of the handrail, the connecting lines of the other two points are outwards far away from the boundary of the handrail,
the obtaining of the ROI rectangular region image specifically comprises the following steps: according to the vertex of the input quadrangle, the minimum x coordinate value min _ x and the minimum y coordinate value min _ y, and the maximum x coordinate value max _ x and the maximum y coordinate value max _ y are found, so that an ROI rectangular region is obtained.
5. The escalator handrail boundary region boundary crossing detection system according to claim 4, wherein the establishment of the gray level image background for the ROI rectangular region by adopting a Gaussian mixture modeling GMM method specifically comprises the following steps:
assigning an initial mean value, standard deviation and weight to each pixel point of the gray level image;
collecting N frames of images, and obtaining a mean value mu _ i, a standard deviation sigma _ i and a weight omega _ i of each pixel point by using an online EM (effective memory) algorithm;
starting detection from an N +1 frame, wherein the detection method comprises the following steps:
for each pixel point: sorting all Gaussian kernels according to the descending order of omega _ i/sigma _ i; selecting the first M gaussian kernels satisfying the following formula: m ═ argmin (ω _ i/σ _ i > T); if one of the pixel values of the current pixel point satisfies: the (| x-mu _ i |)/sigma _ i < K can be regarded as a background point;
and updating the background image.
6. The escalator handrail boundary region out-of-range detection system of claim 4, wherein the background difference and binarization specifically comprises:
subtracting the background image from the current frame to obtain a differential image Pd;
and selecting a threshold th1, and binarizing the difference image Pd to obtain a binary image Bd.
CN201811568311.2A 2018-12-20 2018-12-20 Escalator handrail boundary area border crossing detection method and system Active CN110009650B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811568311.2A CN110009650B (en) 2018-12-20 2018-12-20 Escalator handrail boundary area border crossing detection method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811568311.2A CN110009650B (en) 2018-12-20 2018-12-20 Escalator handrail boundary area border crossing detection method and system

Publications (2)

Publication Number Publication Date
CN110009650A CN110009650A (en) 2019-07-12
CN110009650B true CN110009650B (en) 2021-06-29

Family

ID=67165186

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811568311.2A Active CN110009650B (en) 2018-12-20 2018-12-20 Escalator handrail boundary area border crossing detection method and system

Country Status (1)

Country Link
CN (1) CN110009650B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110472550A (en) * 2019-08-02 2019-11-19 南通使爱智能科技有限公司 A kind of text image shooting integrity degree judgment method and system
CN112700368A (en) * 2019-10-22 2021-04-23 华为技术有限公司 Image processing method and device and electronic equipment
CN111275910B (en) * 2019-11-05 2021-05-11 杭州安脉盛智能技术有限公司 Method and system for detecting border crossing behavior of escalator based on Gaussian mixture model
CN113344874B (en) * 2021-06-04 2024-02-09 温州大学 Pedestrian boundary crossing detection method based on Gaussian mixture modeling

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101670973A (en) * 2008-09-09 2010-03-17 东芝电梯株式会社 Escalator monitoring system
CN101695983A (en) * 2009-10-23 2010-04-21 浙江工业大学 Omnibearing computer vision based energy-saving and safety monitoring system of escalator
CN102087707A (en) * 2009-12-03 2011-06-08 索尼株式会社 Image processing equipment and image processing method
CN106503632A (en) * 2016-10-10 2017-03-15 南京理工大学 A kind of escalator intelligent and safe monitoring method based on video analysis
CN107368786A (en) * 2017-06-16 2017-11-21 华南理工大学 A kind of passenger based on machine vision crosses handrail detection algorithm
CN108182381A (en) * 2017-12-05 2018-06-19 华南理工大学 Escalator occupant detection algorithm based on quick Adaboost training algorithms
CN108230305A (en) * 2017-12-27 2018-06-29 浙江新再灵科技股份有限公司 Method based on the detection of video analysis staircase abnormal operating condition

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7801330B2 (en) * 2005-06-24 2010-09-21 Objectvideo, Inc. Target detection and tracking from video streams
CN101408987A (en) * 2007-10-12 2009-04-15 中兴通讯股份有限公司 Method for generating deuce-symmetrical straight line
JP4792069B2 (en) * 2008-09-03 2011-10-12 株式会社日立製作所 Image recognition device
CN101920603A (en) * 2010-08-06 2010-12-22 东莞市升力智能科技有限公司 NURBS figure laser broom method
CN105701760A (en) * 2016-01-12 2016-06-22 中国人民解放军国防科学技术大学 Histogram real-time generation method of geographic raster data optional polygon area
CN107662868B (en) * 2016-07-29 2022-01-04 奥的斯电梯公司 Monitoring system of passenger conveyer, passenger conveyer and monitoring method thereof
CN107662872B (en) * 2016-07-29 2021-03-12 奥的斯电梯公司 Monitoring system and monitoring method for passenger conveyor
CN107273852A (en) * 2017-06-16 2017-10-20 华南理工大学 Escalator floor plates object and passenger behavior detection algorithm based on machine vision
CN108805093B (en) * 2018-06-19 2020-09-22 华南理工大学 Escalator passenger tumbling detection method based on deep learning
CN110294397A (en) * 2018-07-05 2019-10-01 江苏瑞奇海力科技有限公司 A kind of staircase passenger safety prior-warning device and method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101670973A (en) * 2008-09-09 2010-03-17 东芝电梯株式会社 Escalator monitoring system
CN101695983A (en) * 2009-10-23 2010-04-21 浙江工业大学 Omnibearing computer vision based energy-saving and safety monitoring system of escalator
CN102087707A (en) * 2009-12-03 2011-06-08 索尼株式会社 Image processing equipment and image processing method
CN106503632A (en) * 2016-10-10 2017-03-15 南京理工大学 A kind of escalator intelligent and safe monitoring method based on video analysis
CN107368786A (en) * 2017-06-16 2017-11-21 华南理工大学 A kind of passenger based on machine vision crosses handrail detection algorithm
CN108182381A (en) * 2017-12-05 2018-06-19 华南理工大学 Escalator occupant detection algorithm based on quick Adaboost training algorithms
CN108230305A (en) * 2017-12-27 2018-06-29 浙江新再灵科技股份有限公司 Method based on the detection of video analysis staircase abnormal operating condition

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于计算机视觉的自动扶梯节能***;汤一平 等;《计算机测量与控制》;20111231;第19卷(第7期);1659-1661,1677 *

Also Published As

Publication number Publication date
CN110009650A (en) 2019-07-12

Similar Documents

Publication Publication Date Title
CN110009650B (en) Escalator handrail boundary area border crossing detection method and system
US20220076444A1 (en) Methods and apparatuses for object detection, and devices
JP6719457B2 (en) Method and system for extracting main subject of image
EP3306562B1 (en) Image processing method and device
US7936926B2 (en) Apparatus, method, and program for face feature point detection
WO2019052318A1 (en) Method, apparatus and system for monitoring elevator car
US8548247B2 (en) Image processing apparatus and method, and program
JP2015070359A (en) Person counting device
CN112528861B (en) Foreign matter detection method and device applied to ballast bed in railway tunnel
JP2012048484A (en) Image processing apparatus, image processing method, and program
CN113468918A (en) Package detection method, device, equipment and storage medium
CN104766330B (en) A kind of image processing method and electronic equipment
CN110570441B (en) Ultra-high definition low-delay video control method and system
CN112001336A (en) Pedestrian boundary crossing alarm method, device, equipment and system
KR101600617B1 (en) Method for detecting human in image frame
CN113343856B (en) Image recognition method and system
CN114022468B (en) Method for detecting article left-over and lost in security monitoring
JP2013011950A (en) Image processing apparatus, image processing method, and program
CN115546764A (en) Obstacle detection method, device, equipment and storage medium
JP3578321B2 (en) Image normalizer
JP2008130015A (en) Photographic object identification program and photographic object identification device
JPH0718812B2 (en) Foreign object detection method
CN111008595A (en) Private car interior rear row baby/pet groveling window distinguishing and car interior atmosphere identifying method
JP2001283374A (en) Traffic flow measuring system
JPH08221549A (en) Intrusion object recognizing device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant