CN110316630B - Deviation early warning method and system for installation angle of elevator camera - Google Patents
Deviation early warning method and system for installation angle of elevator camera Download PDFInfo
- Publication number
- CN110316630B CN110316630B CN201910475330.9A CN201910475330A CN110316630B CN 110316630 B CN110316630 B CN 110316630B CN 201910475330 A CN201910475330 A CN 201910475330A CN 110316630 B CN110316630 B CN 110316630B
- Authority
- CN
- China
- Prior art keywords
- car
- camera
- image
- door
- elevator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000009434 installation Methods 0.000 title claims abstract description 31
- 230000011218 segmentation Effects 0.000 claims abstract description 67
- 238000001514 detection method Methods 0.000 claims abstract description 51
- 238000010191 image analysis Methods 0.000 claims abstract description 7
- 238000012549 training Methods 0.000 claims description 17
- 238000005259 measurement Methods 0.000 claims description 16
- 230000006835 compression Effects 0.000 claims description 6
- 238000007906 compression Methods 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims 2
- 238000012544 monitoring process Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B5/00—Applications of checking, fault-correcting, or safety devices in elevators
- B66B5/0006—Monitoring devices or performance analysers
- B66B5/0018—Devices monitoring the operating condition of the elevator system
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B5/00—Applications of checking, fault-correcting, or safety devices in elevators
- B66B5/0087—Devices facilitating maintenance, repair or inspection tasks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Indicating And Signalling Devices For Elevators (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a deviation early warning method and a system for an installation angle of an elevator camera, wherein the method comprises the following steps: detecting the state of the door; a picture is grabbed, and a camera at the top of the car is used for grabbing an elevator image; detecting whether people exist or not, detecting the captured picture by using a yolov3 detection model, if the head in the picture and a target frame for shielding a human body can be detected, indicating that people exist, otherwise, detecting that no people exist, when an unmanned signal is output, performing an image analysis process, and when an unmanned signal is output, waiting for a next elevator door closing signal; performing semantic segmentation on the image; estimating the vertexes of the rectangle, namely fitting an external rectangle aiming at the ground area of the elevator car obtained by segmentation to obtain the positions of four vertexes of the external rectangle; tilt and coverage metrics. The invention solves the problem of the inclination of the camera at the top of the elevator car caused by some reasons, can predict which cameras incline in an image mode, and estimates the inclination degree and the car ground area coverage rate.
Description
Technical Field
The invention belongs to the technical field of data analysis, and particularly relates to a deviation early warning method and system for an installation angle of an elevator camera.
Background
The camera in the elevator car is an important medium for carrying out elevator car picture acquisition and elevator car environment monitoring, and the installation angle of the camera can greatly influence the picture acquisition effect.
Therefore, in practical application, the early warning of the deviation of the installation angle of the elevator camera in the car is necessary.
Disclosure of Invention
In view of the above technical problems, the present invention is directed to providing a method and a system for warning deviation of an installation angle of an elevator camera.
In order to solve the technical problems, the invention adopts the following technical scheme:
in one aspect of the invention, a deviation early warning method for an installation angle of an elevator camera is provided, which comprises the following steps:
detecting a door state, acquiring an image sequence by a camera at the top of a car, detecting the position of an alarm prompt tag from the whole image when the car runs for the first time, defining a subsequent detected local area according to the position of the alarm prompt tag, analyzing the subsequent image sequence to define the local area, and judging the opening and closing state of the door according to the distance between the tags, wherein the opening state, the closing state and the closing state are four states, the state is judged once every 200ms, the door state change refers to the state from closing to opening, otherwise, a door closing signal is output when the door state is changed from opening to closing, and other state changes do not output signals any more;
a picture is grabbed, and a camera at the top of the car is used for grabbing an elevator image;
detecting whether people exist or not, detecting the captured picture by using a yolov3 detection model, if the head in the picture and a target frame for shielding a human body can be detected, indicating that people exist, otherwise, detecting that no people exist, when an unmanned signal is output, performing an image analysis process, and when an unmanned signal is output, waiting for a next elevator door closing signal;
performing image semantic segmentation, namely performing image semantic segmentation on the captured picture to obtain the ground area of the elevator car; the semantic segmentation network output is defined herein as two classes, car floor and background, respectively. The pixel area occupied by the car ground is calibrated manually by collecting data when different elevator cars have sundries. Training the semantic segmentation network by using the calibrated data, and obtaining a semantic segmentation network model after the training is finished; because the target outputs two types, the sigmod cross entropy loss function is used for guiding training, the original picture can be input by installing and deploying the semantic segmentation model, and the corresponding mask area is output;
estimating the vertexes of the rectangle, namely fitting an external rectangle aiming at the ground area of the elevator car obtained by segmentation to obtain the positions of four vertexes of the external rectangle;
the inclination and coverage rate measurement is carried out, wherein the inclination measurement is to determine which vertex is used for measurement with the corresponding image boundary according to the installation position of the camera and the position relation between the circumscribed rectangle and the picture, and if the camera is installed at a corner, the intersection angle between the circumscribed rectangle and the image boundary is large; if the camera is arranged in the middle, the intersection angle of the circumscribed rectangle and the image boundary is small, if the vertex of the circumscribed rectangle is outside the image boundary, the distance between the circumscribed rectangle and the corresponding image boundary is defined as negative, otherwise, the distance is defined as positive; for one corner of the camera installed inside the top of the car, whether the elevator camera is installed properly is judged according to the distance from the left point and the right point of the bottom of the car to the image boundary and the distance from the top point of the lower part of the bottom of the car to the image lower boundary; for the camera to be installed in the middle of one side inside the top of the car, the distance from the two top points at the bottom to the bottom edge of the image and the distance from the two top points to the two sides of the image are required to be used for judging whether the camera is installed properly; and the coverage rate measurement definition defines the ratio of the bottom area of the divided car to the area of the external rectangle, and the coverage condition of the camera on the bottom visual angle of the car is obtained.
Preferably, the steps of door state detection, image capture, human detection, image semantic segmentation, rectangular vertex estimation and inclination and coverage rate measurement are performed once every other time, and whether the angle of the elevator camera is changed or not is judged according to the calculation result of each time.
Preferably, in the process of judging whether a person exists in the captured picture in the car, the target is defined as a head and a sheltered human body, and whether the head and the human body exist in the picture is identified through target detection; the image semantic segmentation area is the ground at the bottom of the car, and the picture is segmented semantically to obtain the ground area of the car, so that an external rectangle of the picture is obtained, and four vertexes are obtained; the door state identification also utilizes a target detection mode, corresponding alarm prompt tags are pasted at the higher position of the elevator door, and the door state detection is realized by detecting the positions among the alarm prompt tags.
Preferably, the semantic segmentation is classification on a pixel level, pixels belonging to the same class are classified into one class, a semantic segmentation network is formed by adopting an Encoder-Decoder architecture of U-net and simultaneously borrowing a spatial pyramid formatting module in PSPnet, a single-model single-scale method is adopted, an encoding part Enencoder input picture passes through 4 convolution blocks to be subjected to 4 times of downsampling, then spatial pyramid formatting operation is carried out on a 16 x 16 feature map, a newly generated feature map and a previous 16 x 16 feature map are spliced, and a decoding part Decode performs channel compression by utilizing 1 x 1 channel convolution. The method comprises the steps of utilizing bilinear interpolation to conduct upsampling, conducting adding operation on a feature graph obtained through upsampling and a previous 32 x 32 feature graph, adjusting to be consistent through 1 x 1 channel convolution if the number of channels of the previous 32 x 32 feature graph is inconsistent, conducting repeated operation twice subsequently to obtain a 128 x 128 feature graph, conducting upsampling for the last time to obtain a 256 x 256 feature graph, finally obtaining a 2 x 256 prediction template through a plurality of convolution layers, and comparing with a calibration template value, wherein 2 represents two categories, namely car ground and background.
In another aspect of the present invention, there is provided a deviation warning system for an installation angle of an elevator camera, including:
the door state detection unit is used for acquiring an image sequence by utilizing a camera at the top of the car, detecting the position of an alarm prompt tag from the whole image when the car is in operation for the first time, defining a subsequent detected local area according to the position of the alarm prompt tag, analyzing the subsequent image sequence to define the local area, and judging the opening and closing state of the door according to the distance between the tags, wherein the opening and closing state comprises four states of opening the door, closing the door and closing the door, the state change is judged once every 200ms, the door state change refers to the state from closing the door to opening the door, otherwise, a door closing signal is output when the door state is changed from opening the door to closing the door, and;
the image grabbing unit is used for grabbing an elevator image by using a camera on the top of the car;
the device comprises a presence and absence detection unit, a door closing detection unit and a door closing detection unit, wherein the presence and absence detection unit is used for detecting a captured picture by using a yolov3 detection model, if the head and a target frame for shielding a human body in the picture can be detected, the presence of the person is indicated, otherwise, the absence of the person is indicated, when an absence signal is output, an image analysis process is carried out, and when an absence signal is output, a door closing signal of the elevator needs to wait for the next time;
the image semantic segmentation unit is used for performing image semantic segmentation on the captured picture to obtain an elevator car ground area; the semantic segmentation network output is defined herein as two classes, car floor and background, respectively. The pixel area occupied by the car ground is calibrated manually by collecting data when different elevator cars have sundries. Training the semantic segmentation network by using the calibrated data, and obtaining a semantic segmentation network model after the training is finished; because the target outputs two types, the sigmod cross entropy loss function is used for guiding training, the original picture can be input by installing and deploying the semantic segmentation model, and the corresponding mask area is output;
the rectangular vertex estimation unit is used for fitting an external rectangle aiming at the ground area of the elevator car obtained by segmentation to obtain four vertex positions of the external rectangle;
the inclination and coverage rate measuring unit is used for judging which vertex is used and the corresponding image boundary to measure according to the installation position of the camera and the position relation between the circumscribed rectangle and the picture, and if the camera is installed at a corner, the intersection angle between the circumscribed rectangle and the image boundary is large; if the camera is arranged in the middle, the intersection angle of the circumscribed rectangle and the image boundary is small, if the vertex of the circumscribed rectangle is outside the image boundary, the distance between the circumscribed rectangle and the corresponding image boundary is defined as negative, otherwise, the distance is defined as positive; for one corner of the camera installed inside the top of the car, whether the elevator camera is installed properly is judged according to the distance from the left point and the right point of the bottom of the car to the image boundary and the distance from the top point of the lower part of the bottom of the car to the image lower boundary; for the camera to be installed in the middle of one side inside the top of the car, the distance from the two top points at the bottom to the bottom edge of the image and the distance from the two top points to the two sides of the image are required to be used for judging whether the camera is installed properly; and the coverage rate measurement definition defines the ratio of the bottom area of the divided car to the area of the external rectangle, and the coverage condition of the camera on the bottom visual angle of the car is obtained.
Preferably, the steps of door state detection, image capture, human detection, image semantic segmentation, rectangular vertex estimation and inclination and coverage rate measurement are performed once every other time, and whether the angle of the elevator camera is changed or not is judged according to the calculation result of each time.
Preferably, in the process of judging whether a person exists in the captured picture in the car, the target is defined as a head and a sheltered human body, and whether the head and the human body exist in the picture is identified through target detection; the image semantic segmentation area is the ground at the bottom of the car, and the picture is segmented semantically to obtain the ground area of the car, so that an external rectangle of the picture is obtained, and four vertexes are obtained; the door state identification also utilizes a target detection mode, corresponding alarm prompt tags are pasted at the higher position of the elevator door, and the door state detection is realized by detecting the positions among the alarm prompt tags.
Preferably, the semantic segmentation is classification on a pixel level, pixels belonging to the same class are classified into one class, a semantic segmentation network is formed by adopting an Encoder-Decoder architecture of U-net and simultaneously borrowing a spatial pyramid formatting module in PSPnet, a single-model single-scale method is adopted, an encoding part Enencoder input picture passes through 4 convolution blocks to be subjected to 4 times of downsampling, then spatial pyramid formatting operation is carried out on a 16 x 16 feature map, a newly generated feature map and a previous 16 x 16 feature map are spliced, and a decoding part Decode performs channel compression by utilizing 1 x 1 channel convolution. The method comprises the steps of utilizing bilinear interpolation to conduct upsampling, conducting adding operation on a feature graph obtained through upsampling and a previous 32 x 32 feature graph, adjusting to be consistent through 1 x 1 channel convolution if the number of channels of the previous 32 x 32 feature graph is inconsistent, conducting repeated operation twice subsequently to obtain a 128 x 128 feature graph, conducting upsampling for the last time to obtain a 256 x 256 feature graph, finally obtaining a 2 x 256 prediction template through a plurality of convolution layers, and comparing with a calibration template value, wherein 2 represents two categories, namely car ground and background.
The invention has the following beneficial effects: the problem of the camera slope at elevator car top because some reasons cause is solved, can predict which camera heads take place the slope through the mode of image, estimate inclination degree and car visual angle coverage to and provide real-time correction evaluation for the maintainer, guide maintainer angle adjustment.
Drawings
Fig. 1 is a flowchart illustrating steps of a method for correcting an installation angle of an elevator camera according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a camera mounted at a corner with a large intersection angle between a circumscribed rectangle and an image boundary according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a case where the ground area of the car is blocked when the angle deviation of the camera is large, and the area of the case is small compared with that of the circumscribed rectangle according to the embodiment of the present invention;
FIG. 4 is a flow chart illustrating semantic segmentation according to an embodiment of the present invention;
fig. 5 is a flowchart of steps of a system for correcting an installation angle of an elevator camera according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a flow chart of steps of a method for correcting an installation angle of an elevator camera according to an embodiment of the present invention is shown, which includes the following steps:
detecting a door state, acquiring an image sequence by a camera at the top of a car, detecting the position of an alarm prompt tag from the whole image when the car runs for the first time, defining a subsequent detected local area according to the position of the alarm prompt tag, analyzing the subsequent image sequence to define the local area, and judging the opening and closing state of the door according to the distance between the tags, wherein the opening state, the closing state and the closing state are four states, the state is judged once every 200ms, the door state change refers to the state from closing to opening, otherwise, a door closing signal is output when the door state is changed from opening to closing, and other state changes do not output signals any more; the door state analysis needs to analyze the door state in real time through videos, in order to reduce the calculated amount, firstly, the position of an alarm prompt label is obtained through the global visual field, then a local detection area is set according to the position of the alarm prompt label, and then only the local detection area is detected, and finally, the local detection area can be automatically adjusted according to the central position of the label, so that the situation that the camera head deviates due to some reasons is prevented.
A picture is grabbed, and a camera at the top of the car is used for grabbing an elevator image;
detecting whether people exist or not, detecting the captured picture by using a yolov3 detection model, if the head in the picture and a target frame for shielding a human body can be detected, indicating that people exist, otherwise, detecting that no people exist, when an unmanned signal is output, performing an image analysis process, and when an unmanned signal is output, waiting for a next elevator door closing signal;
performing image semantic segmentation, namely performing image semantic segmentation on the captured picture to obtain the ground area of the elevator car; the semantic segmentation network output is defined herein as two classes, car floor and background, respectively. The pixel area occupied by the car ground is calibrated manually by collecting data when different elevator cars have sundries. Training the semantic segmentation network by using the calibrated data, and obtaining a semantic segmentation network model after the training is finished; because the target outputs two types, the sigmod cross entropy loss function is used for guiding training, the original picture can be input by installing and deploying the semantic segmentation model, and the corresponding mask area is output;
estimating the vertexes of the rectangle, namely fitting an external rectangle aiming at the ground area of the elevator car obtained by segmentation to obtain the positions of four vertexes of the external rectangle;
and a tilt and coverage measure, wherein the tilt measure is used for determining which vertices are used to measure with corresponding image boundaries according to the camera mounting position and the position relation of the circumscribed rectangle and the picture, namely h1, h2 and h 3. If the camera is installed at a corner, the intersection angle of the circumscribed rectangle and the image boundary is large, as shown in FIG. 2; if the camera is arranged in the middle, the intersection angle of the circumscribed rectangle and the image boundary is small. If the vertex of the circumscribed rectangle is outside the image boundary, the distance from the vertex to the corresponding image boundary is defined as negative, otherwise, the distance is defined as positive. For one corner of the camera installed inside the top of the car, whether the elevator camera is installed properly is judged through the distances from the left point and the right point of the bottom of the car to the image boundary, namely h1 and h2 (the left distance and the right distance are required to be equal in sign in principle) and the distance from the top point of the lower part of the bottom of the car to the image lower boundary, namely h3 (the distance is required to be smaller and close to zero in principle, and the sign is positive). For the camera to be installed in the middle of one side inside the top of the car, the distances from the two top points at the bottom to the bottom edge of the image and the distances from the two top points to the two sides of the image (the distance is required to be smaller and close to zero and the sign is positive in principle) are respectively utilized to judge whether the camera is installed properly; and the coverage rate measurement defines the ratio of the bottom area of the divided car to the area of the external rectangle to obtain the coverage condition of the camera on the view angle of the car. For example, when the angular deviation of the camera is large, the floor area of the car is blocked, which is small compared with the area of the circumscribed rectangle, as shown in fig. 3. Whether the installation angle of the elevator camera is suitable is analyzed, the elevator video monitoring is facilitated to shoot the concerned area to the maximum extent, and the elevator video monitoring system is also facilitated to inform a cloud management platform of arrangement personnel to maintain in time due to vibration or angle change caused by people in real time.
In an actual application scene, whether the angle installation of the elevator camera is proper or not can be analyzed by detecting four vertexes at the bottom of the car. It should be noted that most cameras are installed at one corner of the top of the car, so the circumscribed rectangle obtained by the method is an erected rhombus; of course, part of the camera is arranged in the middle of one side of the inner part of the top of the car, and the obtained external rectangle is a rectangle with one side parallel to the bottom edge of the image. For both cases, the camera installation location may be identified by the installer input system, or may be programmatically analyzed. In the embodiment of the invention, the installation position of the camera is identified for an installer.
In a specific application example, the steps of door state detection, image capture, human existence detection, image semantic segmentation, rectangular vertex estimation and inclination and coverage rate measurement are detected once every other period of time, and whether the angle of the elevator camera is changed or not is judged according to the calculation result of each time.
In a specific application example, in the process of judging whether a person exists in a captured picture in a car, a target is defined as a head and a sheltered human body, and whether the head and the human body exist in the picture is identified through target detection; the image semantic segmentation area is the ground at the bottom of the car, and the picture is segmented semantically to obtain the ground area of the car, so that an external rectangle of the picture is obtained, and four vertexes are obtained; the door state identification also utilizes a target detection mode, corresponding alarm prompt tags are pasted at the higher position of the elevator door, and the door state detection is realized by detecting the positions among the alarm prompt tags.
Further, semantic segmentation is classification at a pixel level, pixels belonging to the same class are classified into one class, referring to fig. 4, a semantic segmentation network is formed by adopting an Encoder-Decoder architecture of U-net and simultaneously using a spatial pyramid clustering module in PSPnet, a single-model single-scale method is adopted, an encoding part Encoder input picture passes through 4 convolution blocks to perform 4 times of downsampling, then, a spatial pyramid clustering operation is performed on a 16 × 16 feature map, a newly generated feature map is spliced with the previous 16 × 16 feature map, and a decoding part Decoder performs channel compression by using 1 × 1 channel convolution. The method comprises the steps of utilizing bilinear interpolation to conduct upsampling, conducting adding operation on a feature graph obtained through upsampling and a previous 32 x 32 feature graph, adjusting to be consistent through 1 x 1 channel convolution if the number of channels of the previous 32 x 32 feature graph is inconsistent, conducting repeated operation twice subsequently to obtain a 128 x 128 feature graph, conducting upsampling for the last time to obtain a 256 x 256 feature graph, finally obtaining a 2 x 256 prediction template through a plurality of convolution layers, and comparing with a calibration template value, wherein 2 represents two categories, namely car ground and background.
Corresponding to the embodiment of the method of the invention, referring to fig. 5, the invention provides a deviation early warning system for the installation angle of an elevator camera, which comprises:
the door state detection unit is used for acquiring an image sequence by utilizing a camera at the top of the car, detecting the position of an alarm prompt tag from the whole image when the car is in operation for the first time, defining a subsequent detected local area according to the position of the alarm prompt tag, analyzing the subsequent image sequence to define the local area, and judging the opening and closing state of the door according to the distance between the tags, wherein the opening and closing state comprises four states of opening the door, closing the door and closing the door, the state change is judged once every 200ms, the door state change refers to the state from closing the door to opening the door, otherwise, a door closing signal is output when the door state is changed from opening the door to closing the door, and;
the image grabbing unit is used for grabbing an elevator image by using a camera on the top of the car;
the device comprises a presence and absence detection unit, a door closing detection unit and a door closing detection unit, wherein the presence and absence detection unit is used for detecting a captured picture by using a yolov3 detection model, if the head and a target frame for shielding a human body in the picture can be detected, the presence of the person is indicated, otherwise, the absence of the person is indicated, when an absence signal is output, an image analysis process is carried out, and when an absence signal is output, a door closing signal of the elevator needs to wait for the next time;
the image semantic segmentation unit is used for performing image semantic segmentation on the captured picture to obtain an elevator car ground area; the semantic segmentation network output is defined herein as two classes, car floor and background, respectively. The pixel area occupied by the car ground is calibrated manually by collecting data when different elevator cars have sundries. Training the semantic segmentation network by using the calibrated data, and obtaining a semantic segmentation network model after the training is finished; because the target outputs two types, the sigmod cross entropy loss function is used for guiding training, the original picture can be input by installing and deploying the semantic segmentation model, and the corresponding mask area is output.
The rectangular vertex estimation unit is used for fitting an external rectangle aiming at the ground area of the elevator car obtained by segmentation to obtain four vertex positions of the external rectangle;
and a tilt and coverage measure, wherein the tilt measure is used for determining which vertices are used to measure with corresponding image boundaries according to the camera mounting position and the position relation of the circumscribed rectangle and the picture, namely h1, h2 and h 3. If the camera is installed at a corner, the intersection angle of the circumscribed rectangle and the image boundary is large, as shown in FIG. 2; if the camera is arranged in the middle, the intersection angle of the circumscribed rectangle and the image boundary is small. If the vertex of the circumscribed rectangle is outside the image boundary, the distance from the vertex to the corresponding image boundary is defined as negative, otherwise, the distance is defined as positive. For one corner of the camera installed inside the top of the car, whether the elevator camera is installed properly is judged through the distances from the left point and the right point of the bottom of the car to the image boundary, namely h1 and h2 (the left distance and the right distance are required to be equal in sign in principle) and the distance from the top point of the lower part of the bottom of the car to the image lower boundary, namely h3 (the distance is required to be smaller and close to zero in principle, and the sign is positive). For the camera to be installed in the middle of one side inside the top of the car, the distances from the two top points at the bottom to the bottom edge of the image and the distances from the two top points to the two sides of the image (the distance is required to be smaller and close to zero and the sign is positive in principle) are respectively utilized to judge whether the camera is installed properly; and the coverage rate measurement defines the ratio of the bottom area of the divided car to the area of the external rectangle to obtain the coverage condition of the camera on the view angle of the car. For example, when the angular deviation of the camera is large, the floor area of the car is blocked, which is small compared with the area of the circumscribed rectangle, as shown in fig. 3. Whether the installation angle of the elevator camera is suitable is analyzed, the elevator video monitoring is facilitated to shoot the concerned area to the maximum extent, and the elevator video monitoring system is also facilitated to inform a cloud management platform of arrangement personnel to maintain in time due to vibration or angle change caused by people in real time.
In an actual application scene, whether the angle installation of the elevator camera is proper or not can be analyzed by detecting four vertexes at the bottom of the car. It should be noted that most cameras are installed at one corner of the top of the car, so the circumscribed rectangle obtained by the method is an erected rhombus; of course, part of the camera is arranged in the middle of one side of the inner part of the top of the car, and the obtained external rectangle is a rectangle with one side parallel to the bottom edge of the image. For both cases, the camera installation location may be identified by the installer input system, or may be programmatically analyzed. In the embodiment of the invention, the installation position of the camera is identified for an installer.
In a specific application example, the steps of door state detection, image capture, human existence detection, image semantic segmentation, rectangular vertex estimation and inclination and coverage rate measurement are detected once every other period of time, and whether the angle of the elevator camera is changed or not is judged according to the calculation result of each time.
In a specific application example, in the process of judging whether a person exists in a captured picture in a car, a target is defined as a head and a sheltered human body, and whether the head and the human body exist in the picture is identified through target detection; the image semantic segmentation area is the ground at the bottom of the car, and the picture is segmented semantically to obtain the ground area of the car, so that an external rectangle of the picture is obtained, and four vertexes are obtained; the door state identification also utilizes a target detection mode, corresponding alarm prompt tags are pasted at the higher position of the elevator door, and the door state detection is realized by detecting the positions among the alarm prompt tags.
Further, semantic segmentation is classification at a pixel level, pixels belonging to the same class are classified into one class, referring to fig. 4, a semantic segmentation network is formed by adopting an Encoder-Decoder architecture of U-net and simultaneously using a spatial pyramid clustering module in PSPnet, a single-model single-scale method is adopted, an encoding part Encoder input picture passes through 4 convolution blocks to perform 4 times of downsampling, then, a spatial pyramid clustering operation is performed on a 16 × 16 feature map, a newly generated feature map is spliced with the previous 16 × 16 feature map, and a decoding part Decoder performs channel compression by using 1 × 1 channel convolution. The method comprises the steps of utilizing bilinear interpolation to conduct upsampling, conducting adding operation on a feature graph obtained through upsampling and a previous 32 x 32 feature graph, adjusting to be consistent through 1 x 1 channel convolution if the number of channels of the previous 32 x 32 feature graph is inconsistent, conducting repeated operation twice subsequently to obtain a 128 x 128 feature graph, conducting upsampling for the last time to obtain a 256 x 256 feature graph, finally obtaining a 2 x 256 prediction template through a plurality of convolution layers, and comparing with a calibration template value, wherein 2 represents two categories, namely car ground and background.
It is to be understood that the exemplary embodiments described herein are illustrative and not restrictive. Although one or more embodiments of the present invention have been described with reference to the accompanying drawings, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the following claims.
Claims (8)
1. A deviation early warning method for an installation angle of an elevator camera is characterized by comprising the following steps:
detecting a door state, acquiring an image sequence by a camera at the top of a car, detecting the position of an alarm prompt tag from the whole image when the car runs for the first time, defining a subsequent detected local area according to the position of the alarm prompt tag, analyzing the subsequent image sequence to define the local area, and judging the opening and closing state of the door according to the distance between the tags, wherein the opening state, the closing state and the closing state are four states, the state is judged once every 200ms, the door state change refers to the state from closing to opening, otherwise, a door closing signal is output when the door state is changed from opening to closing, and other state changes do not output signals any more;
a picture is grabbed, and a camera at the top of the car is used for grabbing an elevator image;
detecting whether people exist or not, detecting the captured picture by using a yolov3 detection model, if the head in the picture and a target frame for shielding a human body can be detected, indicating that people exist, otherwise, detecting that no people exist, when an unmanned signal is output, performing an image analysis process, and when an unmanned signal is output, waiting for a next elevator door closing signal;
performing image semantic segmentation, namely performing image semantic segmentation on the captured picture to obtain the ground area of the elevator car; the semantic segmentation network output is defined as two types, namely car ground and background, the pixel area occupied by the car ground is calibrated manually by collecting data when different elevator cars are provided with sundries, the semantic segmentation network is trained by using the calibrated data, and a semantic segmentation network model is obtained after the training is finished; because the target outputs two types, the sigmod cross entropy loss function is used for guiding training, the original picture can be input by installing and deploying the semantic segmentation model, and the corresponding mask area is output;
estimating the vertexes of the rectangle, namely fitting an external rectangle aiming at the ground area of the elevator car obtained by segmentation to obtain the positions of four vertexes of the external rectangle;
the inclination and coverage rate measurement is carried out, wherein the inclination measurement is to determine which vertex is used for measurement with the corresponding image boundary according to the installation position of the camera and the position relation between the circumscribed rectangle and the picture, and if the camera is installed at a corner, the intersection angle between the circumscribed rectangle and the image boundary is large; if the camera is arranged in the middle, the intersection angle of the circumscribed rectangle and the image boundary is small, if the vertex of the circumscribed rectangle is outside the image boundary, the distance between the circumscribed rectangle and the corresponding image boundary is defined as negative, otherwise, the distance is defined as positive; for one corner of the camera installed inside the top of the car, whether the elevator camera is installed properly is judged according to the distance from the left point and the right point of the bottom of the car to the image boundary and the distance from the top point of the lower part of the bottom of the car to the image lower boundary; for the camera to be installed in the middle of one side inside the top of the car, the distance from the two top points at the bottom to the bottom edge of the image and the distance from the two top points to the two sides of the image are required to be used for judging whether the camera is installed properly; and the coverage rate measurement definition defines the ratio of the bottom area of the divided car to the area of the external rectangle, and the coverage condition of the camera on the bottom visual angle of the car is obtained.
2. The method for warning the deviation of the installation angle of the elevator camera according to claim 1, wherein the steps of detecting the state of the upper door, grabbing a picture, detecting the existence of people, segmenting the semantic meaning of the image, estimating the vertex of the rectangle and measuring the inclination and the coverage rate are detected once every period of time, and whether the angle of the elevator camera is changed or not is judged according to the calculation result of each time.
3. The deviation early warning method for the installation angle of the elevator camera according to claim 1 or 2, characterized in that in the process of judging whether a person exists in the captured picture in the car, the target is defined as the head and the sheltered human body, and whether the head and the human body exist in the picture is identified through target detection; the image semantic segmentation area is the ground at the bottom of the car, and the picture is segmented semantically to obtain the ground area of the car, so that an external rectangle of the picture is obtained, and four vertexes are obtained; the door state identification also utilizes a target detection mode, corresponding alarm prompt tags are pasted at the higher position of the elevator door, and the door state detection is realized by detecting the positions among the alarm prompt tags.
4. The deviation warning method of the installation angle of the elevator camera as claimed in claim 3, characterized in that the semantic segmentation is classification at the pixel level, the pixels belonging to the same class are classified into one class, the Encoder-Decoder architecture of U-net is adopted, and the spatial pyramid boosting module in PSPnet is simultaneously used to form the semantic segmentation network, the method of single model and single scale is adopted, the Encoder input picture passes through 4 convolution blocks to perform 4 times of downsampling, then the spatial pyramid boosting operation is performed on the 16 x 16 feature map, the newly generated feature map is spliced with the previous 16 x 16 feature map, the Decoder performs channel compression by using 1 x 1 channel convolution, the upsampling is performed by using bilinear interpolation, the upsampling obtained feature map is summed with the previous 32 x 32 feature map, if the number of the previous 32 x 32 feature map channels is not consistent, adjusting to be consistent through 1 × 1 channel convolution, repeating the operation twice subsequently to obtain a 128 × 128 feature map, then performing last up-sampling to obtain a 256 × 256 feature map, finally obtaining a 2 × 256 × 256 prediction template through a plurality of convolution layers, and comparing the prediction template with a calibration template value, wherein 2 represents two categories, namely car ground and background.
5. The utility model provides an elevator camera installation angle's skew early warning system which characterized in that includes:
the door state detection unit is used for acquiring an image sequence by utilizing a camera at the top of the car, detecting the position of an alarm prompt tag from the whole image when the car is in operation for the first time, defining a subsequent detected local area according to the position of the alarm prompt tag, analyzing the subsequent image sequence to define the local area, and judging the opening and closing state of the door according to the distance between the tags, wherein the opening and closing state comprises four states of opening the door, closing the door and closing the door, the state change is judged once every 200ms, the door state change refers to the state from closing the door to opening the door, otherwise, a door closing signal is output when the door state is changed from opening the door to closing the door, and;
the image grabbing unit is used for grabbing an elevator image by using a camera on the top of the car;
the device comprises a presence and absence detection unit, a door closing detection unit and a door closing detection unit, wherein the presence and absence detection unit is used for detecting a captured picture by using a yolov3 detection model, if the head and a target frame for shielding a human body in the picture can be detected, the presence of the person is indicated, otherwise, the absence of the person is indicated, when an absence signal is output, an image analysis process is carried out, and when an absence signal is output, a door closing signal of the elevator needs to wait for the next time;
the image semantic segmentation unit is used for performing image semantic segmentation on the captured picture to obtain an elevator car ground area; the semantic segmentation network output is defined as two types, namely car ground and background; the method comprises the steps that data of different elevator cars with or without sundries are collected, and pixel areas occupied by the car ground are calibrated manually; training the semantic segmentation network by using the calibrated data, and obtaining a semantic segmentation network model after the training is finished; because the target outputs two types, the sigmod cross entropy loss function is used for guiding training, the original picture can be input by installing and deploying the semantic segmentation model, and the corresponding mask area is output;
the rectangular vertex estimation unit is used for fitting an external rectangle aiming at the ground area of the elevator car obtained by segmentation to obtain four vertex positions of the external rectangle;
the inclination and coverage rate measuring unit is used for judging which vertex is used and the corresponding image boundary to measure according to the installation position of the camera and the position relation between the circumscribed rectangle and the picture, and if the camera is installed at a corner, the intersection angle between the circumscribed rectangle and the image boundary is large; if the camera is arranged in the middle, the intersection angle of the circumscribed rectangle and the image boundary is small, if the vertex of the circumscribed rectangle is outside the image boundary, the distance between the circumscribed rectangle and the corresponding image boundary is defined as negative, otherwise, the distance is defined as positive; for one corner of the camera installed inside the top of the car, whether the elevator camera is installed properly is judged according to the distance from the left point and the right point of the bottom of the car to the image boundary and the distance from the top point of the lower part of the bottom of the car to the image lower boundary; for the camera to be installed in the middle of one side inside the top of the car, the distance from the two top points at the bottom to the bottom edge of the image and the distance from the two top points to the two sides of the image are required to be used for judging whether the camera is installed properly; and the coverage rate measurement definition defines the ratio of the bottom area of the divided car to the area of the external rectangle, and the coverage condition of the camera on the bottom visual angle of the car is obtained.
6. The system of claim 5, wherein the steps of detecting the state of the door, capturing the image, detecting the existence of people, segmenting the semantic of the image, estimating the vertex of the rectangle, and measuring the inclination and the coverage rate are detected once every a period of time, and whether the angle of the elevator camera is changed or not is judged according to the calculation result of each time.
7. The deviation early warning system of the installation angle of the elevator camera according to claim 5 or 6, characterized in that in the process of judging whether a person exists in the captured picture in the car, the target is defined as a head and a sheltered human body, and whether the head and the human body exist in the picture is identified through target detection; the image semantic segmentation area is the ground at the bottom of the car, and the picture is segmented semantically to obtain the ground area of the car, so that an external rectangle of the picture is obtained, and four vertexes are obtained; the door state identification also utilizes a target detection mode, corresponding alarm prompt tags are pasted at the higher position of the elevator door, and the door state detection is realized by detecting the positions among the alarm prompt tags.
8. The system of claim 7, wherein the semantic segmentation is a classification at a pixel level, pixels belonging to the same class are classified into one class, an Encoder-Decoder architecture of U-net is adopted, and a spatial pyramid boosting module in PSPnet is used to form a semantic segmentation network, a single-model single-scale method is adopted, an encoding portion Enencoder input picture is passed through 4 convolution blocks to perform 4 times of downsampling, then a spatial pyramid boosting operation is performed on a 16 x 16 feature map, a newly generated feature map is spliced with a previous 16 x 16 feature map, a decoding portion Decode is convolved by a 1 x 1 channel to perform channel compression, bilinear interpolation is used to perform upsampling, an upsampling operation is performed on the upsampled feature map and a previous 32 x 32 feature map, and if the number of the previous 32 x 32 feature maps is inconsistent, adjusting to be consistent through 1 × 1 channel convolution, repeating the operation twice subsequently to obtain a 128 × 128 feature map, then performing last up-sampling to obtain a 256 × 256 feature map, finally obtaining a 2 × 256 × 256 prediction template through a plurality of convolution layers, and comparing the prediction template with a calibration template value, wherein 2 represents two categories, namely car ground and background.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910475330.9A CN110316630B (en) | 2019-06-03 | 2019-06-03 | Deviation early warning method and system for installation angle of elevator camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910475330.9A CN110316630B (en) | 2019-06-03 | 2019-06-03 | Deviation early warning method and system for installation angle of elevator camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110316630A CN110316630A (en) | 2019-10-11 |
| CN110316630B true CN110316630B (en) | 2020-12-29 |
Family
ID=68119427
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910475330.9A Active CN110316630B (en) | 2019-06-03 | 2019-06-03 | Deviation early warning method and system for installation angle of elevator camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110316630B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112179318A (en) * | 2020-10-14 | 2021-01-05 | 刘灿灿 | Method for detecting inclination angle of track robot based on characteristics of battery panel |
| JP6962439B1 (en) * | 2020-12-24 | 2021-11-05 | 三菱電機株式会社 | Elevator display control device |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1427660B1 (en) * | 2001-09-18 | 2006-03-29 | Inventio Ag | Monitoring system |
| JP2007186329A (en) * | 2006-01-16 | 2007-07-26 | Mitsubishi Electric Corp | Crime prevention device for elevator |
| CN108040242A (en) * | 2017-12-18 | 2018-05-15 | 横琴峰云视觉技术有限公司 | A kind of bearing calibration at angle of inclination of Double-camera device and device |
| CN108222749B (en) * | 2017-12-29 | 2020-10-02 | 安徽慧视金瞳科技有限公司 | Intelligent automatic door control method based on image analysis |
| CN109368434B (en) * | 2018-11-23 | 2020-09-01 | 张勇 | Elevator control system with video monitoring function |
| CN109379586B (en) * | 2019-01-16 | 2019-08-06 | 歌尔股份有限公司 | The tilt angle test method and device of camera module |
-
2019
- 2019-06-03 CN CN201910475330.9A patent/CN110316630B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110316630A (en) | 2019-10-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110255318B (en) | Method for detecting idle articles in elevator car based on image semantic segmentation | |
| US20210073573A1 (en) | Ship identity recognition method based on fusion of ais data and video data | |
| CN101458871B (en) | Intelligent traffic analysis system and application system thereof | |
| US9070023B2 (en) | System and method of alerting a driver that visual perception of pedestrian may be difficult | |
| CN112800860B (en) | High-speed object scattering detection method and system with coordination of event camera and visual camera | |
| US20030190058A1 (en) | Apparatus and method for measuring queue length of vehicles | |
| KR102253989B1 (en) | object tracking method for CCTV video by use of Deep Learning object detector | |
| CN105512720A (en) | Public transport vehicle passenger flow statistical method and system | |
| CN111832489A (en) | Subway crowd density estimation method and system based on target detection | |
| CN109711322A (en) | A kind of people's vehicle separation method based on RFCN | |
| CN110316630B (en) | Deviation early warning method and system for installation angle of elevator camera | |
| CN113780270B (en) | Target detection method and device | |
| CN110189355A (en) | Safe escape channel occupies detection method, device, electronic equipment and storage medium | |
| CN113449632B (en) | Vision and radar perception algorithm optimization method and system based on fusion perception and automobile | |
| CN113111722A (en) | Automatic driving target identification method based on improved Mask R-CNN | |
| CN112084892B (en) | Road abnormal event detection management device and method thereof | |
| CN116311084B (en) | Crowd gathering detection method and video monitoring equipment | |
| CN106778540A (en) | Parking detection is accurately based on the parking event detecting method of background double layer | |
| US11580659B2 (en) | Method for size estimation by image recognition of specific target using given scale | |
| CN112446246A (en) | Image occlusion detection method and vehicle-mounted terminal | |
| CN111553500A (en) | Railway traffic contact net inspection method based on attention mechanism full convolution network | |
| KR100532058B1 (en) | Traffic information acquisition method and apparatus using camera calibration | |
| KR102492290B1 (en) | Drone image analysis system based on deep learning for traffic measurement | |
| CN115719464A (en) | Water meter durability device water leakage monitoring method based on machine vision | |
| CN111160224B (en) | High-speed rail contact net foreign matter detection system and method based on FPGA and horizon line segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |