CN109484935B - Elevator car monitoring method, device and system - Google Patents
Elevator car monitoring method, device and system Download PDFInfo
- Publication number
- CN109484935B CN109484935B CN201710822052.0A CN201710822052A CN109484935B CN 109484935 B CN109484935 B CN 109484935B CN 201710822052 A CN201710822052 A CN 201710822052A CN 109484935 B CN109484935 B CN 109484935B
- Authority
- CN
- China
- Prior art keywords
- human body
- point cloud
- depth image
- elevator car
- cloud data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000012544 monitoring process Methods 0.000 title claims abstract description 50
- 238000009434 installation Methods 0.000 claims description 29
- 238000004590 computer program Methods 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 9
- 230000003287 optical effect Effects 0.000 claims description 8
- 238000012806 monitoring device Methods 0.000 claims description 5
- 238000004422 calculation algorithm Methods 0.000 description 40
- 238000010801 machine learning Methods 0.000 description 12
- 238000012549 training Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 238000007637 random forest analysis Methods 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000013145 classification model Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 238000012706 support-vector machine Methods 0.000 description 3
- 241000023320 Luma <angiosperm> Species 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B5/00—Applications of checking, fault-correcting, or safety devices in elevators
- B66B5/0006—Monitoring devices or performance analysers
- B66B5/0012—Devices monitoring the users of the elevator system
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B5/00—Applications of checking, fault-correcting, or safety devices in elevators
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B5/00—Applications of checking, fault-correcting, or safety devices in elevators
- B66B5/02—Applications of checking, fault-correcting, or safety devices in elevators responsive to abnormal operating conditions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
Landscapes
- Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Indicating And Signalling Devices For Elevators (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides a method, a device and a system for monitoring an elevator car, wherein the method comprises the following steps: when the closing of an elevator car door is detected, obtaining a depth image acquired by image acquisition equipment aiming at an elevator car scene; judging whether the obtained depth image contains a human body target or not; when the human body target is judged to be included, determining human body height information of the included human body target based on the obtained depth image; judging whether the determined height information of the human body is lower than a preset height threshold value or not; and when the determined height information of the human body is judged to be lower than a preset height threshold value, alarming. By applying the embodiment of the invention, the intelligent monitoring of the condition that only children take the elevator is realized, the labor is saved, and the possible omission phenomenon caused by limited labor energy or human negligence and other factors is avoided.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a method, a device and a system for monitoring an elevator car.
Background
Elevators have found increasingly widespread use in life as an important vertical conveyance. The elevator brings convenience to people to go out and brings potential safety hazards to people. In one case, the potential safety hazards may include the situation where only a child rides in the elevator car, and since the child is active and the security awareness is weak, if not discovered in time, it is likely that tragedy may occur. Although image acquisition equipment is installed in elevator cars of many buildings, the video monitoring system where the image acquisition equipment is located at present can only provide an image acquisition function; subsequently, the monitoring personnel watch the image collected by the image collecting device through manpower, and identify whether the condition that only children take the elevator appears. Such buildings may include, but are not limited to, residential areas, hotels, and buildings, among others.
The mode of monitoring by manually watching the images needs monitoring personnel to watch the images collected by the image collecting equipment in real time, consumes manpower, is not intelligent enough, and can have the omission phenomenon caused by limited manual energy or human negligence and other factors.
Disclosure of Invention
The embodiment of the invention aims to provide a method, a device and a system for monitoring an elevator car, so as to realize intelligent monitoring on the condition that only children take the elevator, save labor and avoid possible omission phenomenon caused by limited labor energy or human negligence and other factors. The specific technical scheme is as follows:
in one aspect, an embodiment of the present invention provides an elevator car monitoring method, where the method includes:
when the closing of an elevator car door is detected, obtaining a depth image acquired by image acquisition equipment aiming at an elevator car scene;
determining whether the obtained depth image contains a human body target;
when the human body target is determined to be contained, determining human body height information of the contained human body target based on the obtained depth image;
judging whether the determined height information of the human body is lower than a preset height threshold value or not;
and when the determined height information of the human body is judged to be lower than the preset height threshold value, alarming.
Optionally, the step of determining whether the obtained depth image includes a human target includes:
extracting image features from the obtained depth image;
and determining whether the obtained depth image contains the human body target or not based on the extracted image features and a first preset classifier.
Optionally, the step of determining whether the obtained depth image includes a human target includes:
converting the obtained depth image into point cloud data under a world coordinate system;
and determining whether the point cloud data comprises point cloud data corresponding to the human body target, wherein when the point cloud data comprises the point cloud data corresponding to the human body target, the obtained depth image is represented to comprise the human body target.
Optionally, the step of determining whether the point cloud data includes point cloud data corresponding to a human target includes:
clustering the point cloud data to obtain various point cloud subdata;
and determining whether the obtained various types of point cloud sub-data contain point cloud sub-data corresponding to the human body target or not based on the obtained various types of point cloud sub-data and a second preset classifier.
Optionally, the step of determining whether the point cloud data includes point cloud data corresponding to a human target includes:
clustering the point cloud data to obtain various point cloud subdata;
and determining whether the obtained various point cloud subdata contains point cloud subdata corresponding to the human body target or not based on the obtained various point cloud subdata and a preset human body model.
Optionally, the image acquisition device acquires a depth image by using the included depth image acquisition sub-device;
the step of converting the obtained depth image into point cloud data in a world coordinate system includes:
obtaining parameter information of the depth image acquisition sub-device, wherein the parameter information comprises: focal length information, image principal point information, installation height information and installation angle information;
converting the obtained depth image into point cloud data under an equipment coordinate system by using the focal length information and the image principal point information, wherein the equipment coordinate system is as follows: a coordinate system established based on the optical center of the depth image acquisition sub-device;
and converting the point cloud data under the equipment coordinate system into the point cloud data under the world coordinate system by using the installation height information and the installation angle information.
Optionally, the step of obtaining the depth image captured by the image capturing device for the scene of the elevator car includes:
acquiring a depth image acquired by the image acquisition equipment aiming at an elevator car scene and a color image corresponding to the acquired depth image;
the step of determining whether the obtained depth image contains a human body target includes:
and determining whether the obtained depth image contains the human body target or not by identifying whether the color image contains the human body target or not.
Optionally, before the step of determining the height information of the included human target based on the obtained depth image, the method further includes:
when the human body targets are determined to be contained, judging whether the contained human body targets are one or not;
and when the number of the included human body targets is judged to be one, the step of determining the height information of the included human body targets based on the obtained depth image is executed.
Optionally, before the step of alarming, the method further comprises:
when the determined human body height information is judged to be lower than the preset height threshold value, determining the number of the contained human body targets;
and when the determined number of the included human body targets is lower than the preset number, executing the step of alarming.
Optionally, after the step of alarming, the method further comprises:
controlling the elevator car to move to a designated floor;
when the elevator car moves to the appointed floor, controlling the elevator car to stop moving;
and after a door opening instruction for the elevator car is obtained, controlling the elevator car to open the car door.
In another aspect, an embodiment of the present invention provides an elevator car monitoring apparatus, where the apparatus includes:
the first obtaining module is used for obtaining a depth image which is collected by the image collecting equipment aiming at an elevator car scene when the fact that the elevator car door is closed is detected;
the first determining module is used for determining whether the obtained depth image contains a human body target or not;
a second determination module, configured to determine, based on the obtained depth image, human body height information of the included human body target when it is determined that the human body target is included;
the first judgment module is used for judging whether the determined height information of the human body is lower than a preset height threshold value or not;
and the alarm module is used for giving an alarm when the determined height information of the human body is judged to be lower than the preset height threshold value.
Optionally, the first determining module is specifically configured to
Extracting image features from the obtained depth image;
and determining whether the obtained depth image contains the human body target or not based on the extracted image features and a first preset classifier.
Optionally, the first determining module includes a converting unit and a determining unit;
the conversion unit is used for converting the obtained depth image into point cloud data under a world coordinate system;
the determining unit is used for determining whether the point cloud data comprises point cloud data corresponding to a human body target, wherein when the point cloud data comprises the point cloud data corresponding to the human body target, the obtained depth image is represented to comprise the human body target.
Optionally, the determination unit is specifically configured to
Clustering the point cloud data to obtain various point cloud subdata;
and determining whether the obtained various types of point cloud sub-data contain point cloud sub-data corresponding to the human body target or not based on the obtained various types of point cloud sub-data and a second preset classifier.
Optionally, the determination unit is specifically configured to
Clustering the point cloud data to obtain various point cloud subdata;
and determining whether the obtained various point cloud subdata contains point cloud subdata corresponding to the human body target or not based on the obtained various point cloud subdata and a preset human body model.
Optionally, the image acquisition device acquires a depth image by using the included depth image acquisition sub-device;
the conversion unit is particularly used for
Obtaining parameter information of the depth image acquisition sub-device, wherein the parameter information comprises: focal length information, image principal point information, installation height information and installation angle information;
converting the obtained depth image into point cloud data under an equipment coordinate system by using the focal length information and the image principal point information, wherein the equipment coordinate system is as follows: a coordinate system established based on the optical center of the depth image acquisition sub-device;
and converting the point cloud data under the equipment coordinate system into the point cloud data under the world coordinate system by using the installation height information and the installation angle information.
Optionally, the first obtaining module is specifically configured to
Acquiring a depth image acquired by the image acquisition equipment aiming at an elevator car scene and a color image corresponding to the acquired depth image;
the first determining module is specifically used for
And determining whether the obtained depth image contains the human body target or not by identifying whether the color image contains the human body target or not.
Optionally, the apparatus further includes a second determining module;
the second judging module is used for judging whether the included human body targets are one or not when the included human body targets are determined before the height information of the included human body targets is determined based on the obtained depth image; and when the number of the included human body targets is one, triggering the second determining module.
Optionally, the apparatus further comprises a third determining module;
the third determining module is used for determining the number of the included human body targets when the determined human body height information is judged to be lower than the preset height threshold value before the alarm is given;
and when the determined number of the included human body targets is lower than a preset number, triggering the alarm module.
Optionally, the apparatus further comprises a first control module, a second control module, and a third control module;
the first control module is used for controlling the elevator car to move to a specified floor after the alarm is given;
the second control module is used for controlling the elevator car to stop moving after the elevator car moves to the appointed floor;
and the third control module is used for controlling the elevator car to open the car door after obtaining the door opening instruction for the elevator car.
In another aspect, an electronic device is provided, comprising a processor and a memory, wherein the memory is configured to store a computer program;
the processor is used for realizing any elevator car monitoring method when executing the computer program stored in the memory.
In another aspect, an elevator system is provided, comprising: elevator, and any above-mentioned elevator car monitoring device.
Optionally, the elevator system includes a camera for collecting a depth image in the elevator car and sending the depth image to the elevator car monitoring device.
In the embodiment of the invention, when the closing of the elevator car door is detected, the depth image acquired by the image acquisition equipment for the elevator car scene is obtained; judging whether the obtained depth image contains a human body target or not; when the human body target is judged to be included, determining human body height information of the included human body target based on the obtained depth image; judging whether the determined height information of the human body is lower than a preset height threshold value or not; and when the determined height information of the human body is judged to be lower than a preset height threshold value, alarming.
When the closing of the elevator car door is detected, the depth image acquired by the image acquisition equipment for the elevator car scene is obtained, whether the depth image contains a human body target or not is determined, when the depth image contains the human body target, the height information of the human body target is further determined, a preset height threshold value is combined, whether the contained human body target is a child or not is judged, when the height information of the human body target is judged to be lower than the preset height threshold value, the contained human body target can be determined to be the child, and the situation that only the child rides in the elevator car at the moment can be determined, namely, the situation that only the child rides the elevator. Under the condition that only child took the elevator in the control, need report to the police to warning supervisory personnel and pay close attention to, subsequent, supervisory personnel can carry out corresponding processing, and the child of avoiding taking the elevator appears dangerously, with the realization only has the intelligent monitoring of the condition that child took in the elevator, uses manpower sparingly. Moreover, the situation that monitoring omission occurs due to limited manual energy or human negligence and other factors when monitoring personnel monitor manually, and further tragic situations are avoided. Of course, it is not necessary for any product or method of practicing the invention to achieve all of the above-described advantages at the same time.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of an elevator car monitoring method according to an embodiment of the present invention;
fig. 2 is another schematic flow chart of an elevator car monitoring method according to an embodiment of the present invention;
fig. 3 is a schematic structural view of an elevator car monitoring apparatus according to an embodiment of the present invention;
fig. 4 is another schematic structural view of an elevator car monitoring apparatus according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention provides a method, a device and a system for monitoring an elevator car, which are used for realizing intelligent monitoring on the condition that only children take the elevator, saving labor and avoiding possible omission phenomenon caused by limited labor energy or human negligence and other factors.
As shown in fig. 1, an embodiment of the present invention provides an elevator car monitoring method, which may include the following steps:
s101: when the closing of an elevator car door is detected, obtaining a depth image acquired by image acquisition equipment aiming at an elevator car scene;
it can be understood that the elevator car monitoring method provided by the embodiment of the invention can be applied to any electronic device which can obtain the depth image acquired by the image acquisition device, and the electronic device can be a computer, a smart phone, a monitoring server and the like.
The image capturing device may be mounted within the elevator car. When installing above-mentioned image acquisition equipment in elevator car, can install perpendicularly, also can the slope installation, wherein, perpendicular installation can be: the image acquisition equipment can be arranged in the middle of the top of the elevator car and is right opposite to the ground of the elevator car so as to acquire images of scenes in the elevator car; the oblique installation can be installed in the direction that has certain contained angle with the direction that elevator car goes up and down, and above-mentioned image acquisition equipment can be installed in any corner at elevator car top to carry out image acquisition to the scene in the elevator car. It can be understood that no matter how the image acquisition equipment is installed, the image acquisition equipment is ensured to acquire images of all corners in the elevator car to the maximum extent.
The image capturing device may be any device that can capture depth information, i.e., three-dimensional information, of an object in a scene, and may include, but is not limited to, a binocular depth camera, a TOF (Time of Flight) camera, and a structured light camera.
The depth image can contain depth information of each point in the elevator car scene, namely distance information between each point in the elevator car scene and the image acquisition equipment. In an implementation manner, the pixel value of each pixel point in the depth image is: depth information of points in the elevator car scene corresponding to each pixel point. As can be seen, the depth image has the characteristic of being unaffected by factors such as illumination, shading, and chromaticity.
S102: determining whether the obtained depth image contains a human body target;
after the electronic device obtains the depth image, whether the human body target is included in the depth image can be identified. In one implementation, the electronic device may first detect whether a target exists in the depth image, where the target may include a human target and an object target. When the fact that the target exists in the depth image is detected, the electronic equipment determines each target contained in the depth image, subsequently, the electronic equipment identifies each target, and whether each target is a human body target or not is judged.
In one case, the electronic device may pre-store the background depth image of the elevator car scene, that is, the depth image of the elevator car scene when the elevator car is empty; when the electronic equipment detects that the door of the elevator car is closed, the depth image acquired by the image acquisition equipment aiming at the scene of the elevator car is obtained, the obtained depth image is subjected to subtraction with the background depth image to obtain a difference image, and when foreground pixel points exist in the difference image, the fact that the obtained depth image contains a target can be determined. Of course, when the obtained depth image is subtracted from the background depth image, it is required to ensure that the viewing angle of the image capturing device is not changed, that is, the position and/or angle of the image capturing device is not changed.
In one implementation, the electronic device identifies each target, and the process of determining whether each target is a human target may be: and matching each target with a preset three-dimensional human body model, wherein when the matching is successful, the target can be determined to be a human body target, and when the matching is failed, the target can be determined not to be the human body target.
In another implementation manner, the electronic device identifies each target, and the process of determining whether each target is a human target may also be: and classifying each target in the depth image by using a pre-trained classifier, and determining whether each target is a human body target. It is understood that the above-mentioned pre-trained classifier may be: and training the obtained machine learning model based on a training sample and a preset algorithm, wherein the training sample can comprise sample depth images of a human body target and a non-human body target. For the preset algorithm, the preset algorithm may be: in the neural network algorithm including the convolutional neural network correlation algorithm, the machine learning model may be: a neural network model. Alternatively, the preset algorithm may be: a random forest algorithm, in this case, the machine learning model may be: and (4) a random forest classification model. Alternatively, the preset algorithm may be: and a vector machine algorithm is supported, in this case, the machine learning model may be: support vector machine classification models, and so on.
S103: when the human body target is determined to be contained, determining human body height information of the contained human body target based on the obtained depth image;
when the electronic device determines that the depth image contains the human body target, the electronic device may determine human body height information of the contained human body target by using the depth image. In one implementation, the electronic device may convert the obtained depth image into point cloud data in a world coordinate system, and then determine human height information of the human target by using the point cloud data in the world coordinate system corresponding to the human target.
In one case, the highest point cloud data (i.e., the point cloud data corresponding to the top of the head of the human body target) and the lowest point cloud data (i.e., the point cloud data corresponding to the bottom of the foot of the human body target) in the point cloud data of the world coordinate system corresponding to the human body target may be determined, and the height difference between the highest point cloud data and the lowest point cloud data may be used as the human body height information of the human body target.
In another case, the electronic device may sort the point cloud data in the world coordinate system corresponding to each human target based on the height of each point in the point cloud data in the world coordinate system corresponding to each human target, calculate an average value of the heights of a predetermined number of points in the sorting order as a first average value, calculate an average value of the heights of a predetermined number of points in the sorting order as a second average value, and use an absolute value of the first average value and the second average value as the human height information of the human target.
S104: judging whether the determined height information of the human body is lower than a preset height threshold value or not;
s105: and when the determined height information of the human body is judged to be lower than a preset height threshold value, alarming.
After determining the height information of the human body of the included human body target, the electronic device may compare the determined height information of the human body with a preset height threshold value, and determine whether the determined height information of the human body is lower than the preset height threshold value. In one implementation, the depth image may include one or more human body targets, and when there are multiple human body targets, the human body height information of each human body target may be respectively compared with a preset height threshold, and whether the human body height information of each human body target is lower than the preset height threshold is determined.
And when the electronic equipment judges that the determined height information of the human body is lower than a preset height threshold value, alarming. When the electronic device gives an alarm, the electronic device may give an alarm in a form of outputting prompt information, giving an alarm in a form of sending a prompt sound, giving an alarm in a form of changing screen brightness, and the like.
In one implementation, when the electronic device determines that there are a plurality of human body targets included in the depth image, the electronic device may alarm only when the human body height information of the included human body targets is lower than a preset height threshold.
The preset height threshold may be set according to the height of the population as children, and in one implementation, the population as children may include a population under 10 years old.
By applying the embodiment of the invention, when the closing of the elevator car door is detected, the depth image acquired by the image acquisition equipment for the elevator car scene is obtained, whether the depth image contains the human body target or not is determined, when the human body target is determined to be contained, the human body height information of the human body target is further determined, and whether the contained human body target is child or not is judged by combining the preset height threshold value, when the human body height information of the human body target is judged to be lower than the preset height threshold value, the contained human body target can be determined to be child, and the condition that only child rides in the elevator car at the moment can be determined, namely the condition that only child rides in the elevator car. Under the condition that only child took the elevator in the control, need report to the police to warning supervisory personnel and pay close attention to, subsequent, supervisory personnel can carry out corresponding processing, and the child of avoiding taking the elevator appears dangerously, with the realization only has the intelligent monitoring of the condition that child took in the elevator, uses manpower sparingly. Moreover, the situation that monitoring omission occurs due to limited manual energy or human negligence and other factors when monitoring personnel monitor manually, and further tragic situations are avoided.
When the electronic device determines whether the depth image includes the human body target, it may determine whether the depth image includes the human body target by directly using the image features in the depth image, or determine whether the depth image includes the human body target by using the point cloud data features corresponding to the depth image.
In one implementation, the step of determining whether the obtained depth image includes a human target (S102) may include:
extracting image features from the obtained depth image;
and determining whether the obtained depth image contains the human body target or not based on the extracted image features and a first preset classifier.
It is understood that the first preset classifier may be: training a machine learning model based on training samples and a preset algorithm, wherein the training samples can be: the sample depth image including the human target and the non-human target, that is, the image features used for training the first preset classifier may be: image features in the sample depth image. For the preset algorithm, the preset algorithm may be: in the neural network algorithm including the convolutional neural network correlation algorithm, the machine learning model may be: a neural network model. Alternatively, the preset algorithm may be: a random forest algorithm, in this case, the machine learning model may be: and (4) a random forest classification model. Alternatively, the preset algorithm may be: and a vector machine algorithm is supported, in this case, the machine learning model may be: support vector machine classification models, and so on.
In one implementation, the electronic device may perform image feature extraction on the depth image, such as: and (5) utilizing Canny operator and the like to extract edge features of the depth image. Subsequently, the extracted image features are input into a first preset classifier, and the first preset classifier classifies the extracted image features to determine whether the depth image contains the human body target.
Alternatively, the electronic device may be pre-stored with human image features. After the electronic equipment extracts the image features from the depth image, the extracted image features are matched with the pre-stored human body image features, and when the matching is successful, the fact that the depth image comprises the human body target can be determined.
In another implementation, the step of determining whether the obtained depth image includes a human target (S102) may include:
converting the obtained depth image into point cloud data under a world coordinate system;
and determining whether the point cloud data comprises point cloud data corresponding to the human body target, wherein when the point cloud data comprises the point cloud data corresponding to the human body target, the obtained depth image is represented to comprise the human body target.
In the embodiment of the invention, the electronic device may first convert the depth image into point cloud data in a world coordinate system, and then determine whether the depth image includes a human body target based on the point cloud data in the world coordinate system. The image acquisition equipment can acquire a depth image by using the included depth image acquisition sub-equipment, and when the electronic equipment converts the depth image into point cloud data under a world coordinate system, the electronic equipment can convert the depth image into the point cloud data under the world coordinate system by using the parameter information of the depth image acquisition sub-equipment in the image acquisition equipment for acquiring the depth image. The parameter information may include internal parameters and external parameters of the depth image acquisition sub-device, and the electronic device may calibrate the image acquisition device based on a zhangyingyou calibration method, so as to determine the parameter information, or the electronic device may directly calibrate the internal parameters calibrated when the image acquisition device leaves the factory, and the internal parameters and the external parameters may be obtained by measurement and pre-calibration in the parameter information provided in the embodiment of the present invention. In one implementation, the step of converting the obtained depth image into point cloud data in a world coordinate system may include:
obtaining parameter information of the depth image acquisition sub-device, wherein the parameter information comprises: focal length information, image principal point information, installation height information and installation angle information;
converting the obtained depth image into point cloud data under an equipment coordinate system by using the focal length information and the image principal point information, wherein the equipment coordinate system can be as follows: a coordinate system established based on the optical center of the depth image acquisition sub-device;
and converting the point cloud data under the equipment coordinate system into point cloud data under a world coordinate system by using the installation height information and the installation angle information.
It is understood that the above-mentioned image principal points may be: and the intersection point of the optical axis of the depth image acquisition sub-device and the image plane. In the embodiment of the present invention, the information about the image principal point may include: two-dimensional coordinates of the image principal point in the depth image. The two-dimensional coordinates of each pixel point in the depth image can be determined by using the two-dimensional coordinates of the image principal point.
In one implementation, the two-dimensional coordinates (u, v) of each pixel point in the depth image are converted to three-dimensional coordinates (X) in the device coordinate systemC,YC,ZC) To obtain point cloud data of the depth image. The preset three-dimensional rectangular coordinate system may be: based on a coordinate system established by the optical center of the depth image acquisition sub-device, a formula used in coordinate conversion is as follows:
wherein f isDxAnd fDyAll are the focal lengths of the depth image acquisition sub-devices; (u)D0,vD0) Two-dimensional coordinates of the image principal points; zCAnd distance information from the point in the scene corresponding to the pixel point (u, v) in the depth image to the depth image acquisition sub-device, namely the pixel value of the pixel point (u, v). Wherein f isDxRepresenting the determined focal distance in the x-axis direction of the above-mentioned depth-image acquisition sub-apparatus, fDyIndicating the determined focal length in the y-axis direction of the above-described depth image acquisition sub-apparatus. F aboveDxAnd fDyAll included in the above-mentioned focal length information, can be directly calibrated and determined by Zhang Zhengyou calibration method. The two-dimensional coordinates of the image principal points can also be directly calibrated and determined by a Zhang Zhengyou calibration method.
The device coordinate system may be a three-dimensional rectangular coordinate system. And subsequently, determining a conversion relation between the equipment coordinate system and a world coordinate system by utilizing the installation height information and the installation angle information of the depth image acquisition sub-equipment, including the pitch angle and the deflection angle of the depth image acquisition sub-equipment, and converting point cloud data under the equipment coordinate system into point cloud data under the world coordinate system based on the determined conversion relation.
After the electronic device converts the depth image into point cloud data in a world coordinate system, clustering the point cloud data by using a preset clustering algorithm to determine each target included in the depth image, and further, determining whether each target is a human target, wherein in one case, the step of determining whether the point cloud data includes point cloud data corresponding to the human target may include:
clustering point cloud data to obtain various point cloud subdata;
and determining whether the obtained various types of point cloud sub-data contain point cloud sub-data corresponding to the human body target or not based on the obtained various types of point cloud sub-data and a second preset classifier.
In another case, the step of determining whether the point cloud data includes point cloud data corresponding to a human target may include:
clustering point cloud data to obtain various point cloud subdata;
and determining whether the obtained various point cloud subdata contains point cloud subdata corresponding to the human body target or not based on the obtained various point cloud subdata and a preset human body model.
In an implementation manner, the preset clustering algorithm may be a density-based clustering algorithm, for example: DBSCAN (Density-Based Spatial Clustering of Applications with Noise) Clustering algorithm, wherein the DBSCAN Clustering algorithm requires two parameters, namely scanning radius (eps) and minimum contained points (minPts). After the electronic device obtains the point cloud data, selecting an unvisited point (i.e. a point which is not marked as visited or not marked as noise) from the point cloud data by using the DBSCAN clustering algorithm as a current point, and determining all points which are within an eps distance (including eps) from the current point as nearby points; the electronic device determines whether the number of nearby points of the current point is equal to or greater than minPts, and if the electronic device determines that the number of nearby points of the current point is equal to or greater than minPts, the electronic device determines the current point and the determined nearby points as a cluster, and the electronic device marks the current point as visited (visited); the electronic device recursively processes all the points in the cluster that are not marked as visited (visited) or not marked as noise in the manner described above, thereby expanding the cluster; if the cluster is sufficiently expanded, i.e., all points within the cluster are marked as visited; and the electronic equipment selects an unaccessed point again as the current point and executes the subsequent flow. When the electronic equipment determines that the number of the nearby points of the current point is less than minPts, the electronic equipment marks the current point as noise; and finishing clustering until the point cloud data does not contain the points which are not accessed. At this time, the electronic device may determine various types of point cloud sub-data included in the point cloud data, and subsequently, the electronic device may process each type of point cloud sub-data to determine whether each type of point cloud sub-data is point cloud sub-data corresponding to the human body target.
In one case, the electronic device may input each type of the obtained point cloud sub-data into a second preset classifier, so that the second preset classifier classifies each type of the point cloud sub-data, and determines whether each type of the point cloud sub-data is the point cloud sub-data corresponding to the human body target. It is understood that the second preset classifier may be: training a machine learning model based on training samples and a preset algorithm, wherein the training samples can be: the method comprises the steps of obtaining point cloud data corresponding to sample depth images of human body targets and non-human body targets. For the preset algorithm, the preset algorithm may be: in the neural network algorithm including the convolutional neural network correlation algorithm, the machine learning model may be: a neural network model. Alternatively, the preset algorithm may be: a random forest algorithm, in this case, the machine learning model may be: and (4) a random forest classification model. Alternatively, the preset algorithm may be: and a vector machine algorithm is supported, in this case, the machine learning model may be: support vector machine classification models, and so on.
In another case, the electronic device may match each type of acquired point cloud sub-data with a preset human body model, determine that the point cloud sub-data is the point cloud sub-data corresponding to the human body target when the matching is successful, and determine that the point cloud sub-data is not the point cloud sub-data corresponding to the human body target when the matching is failed.
In an implementation manner, the preset clustering algorithm may also be a grid-based clustering algorithm, for example: STING clustering algorithm, etc., and may also be a partition-based clustering algorithm, such as: the embodiment of the invention does not limit the algorithm capable of clustering the point cloud data, and all the algorithms capable of clustering the point cloud data can be applied to the embodiment of the invention.
In one implementation, it may be determined whether the depth image includes a human target by using a color image corresponding to the depth image acquired for the elevator car, where the step of obtaining the depth image acquired by the image acquisition device for the elevator car scene may include:
acquiring a depth image acquired by an image acquisition device aiming at an elevator car scene and a color image corresponding to the acquired depth image;
the step of determining whether the obtained depth image contains the human body target may include:
and determining whether the obtained depth image contains the human body target or not by identifying whether the color image contains the human body target or not.
Whether the human body target is contained in the depth image can be determined by identifying whether the human body target is contained in the color image through a face identification algorithm, wherein when the human body target is contained in the color image, the human body target is contained in the depth image; when the color image does not contain the human body target, the depth image does not contain the human body target.
In an embodiment of the present invention, the color image may be an image in any color mode, for example: an image of RGB (Red Green Blue ) color mode, or a YUV color mode image, and so on. Among them, YUV (also called YCrCb) is a color coding method, and "Y" represents brightness (Luma or Luma), i.e., a gray level value; and "U" and "V" represent Chroma (Chroma) and function to describe image color and saturation for specifying the color of a pixel. "chroma" defines two aspects of the color of a pixel-hue and saturation, which can be expressed as Cr and Cb, respectively.
In one case, when only one child takes the elevator, the probability of danger may be greater, in order to better avoid the danger when only one child takes the elevator, and when only one child takes the elevator, the monitoring person needs to pay more attention and take corresponding measures to avoid the danger of the child taking the elevator alone. In one implementation, as shown in fig. 2, the method may include the steps of:
s201: when the closing of an elevator car door is detected, obtaining a depth image acquired by image acquisition equipment aiming at an elevator car scene;
s202: determining whether the obtained depth image contains a human body target;
here, S201 is the same as S101 shown in fig. 1, and S202 is the same as S102 shown in fig. 1.
S203: when the human body targets are determined to be contained, judging whether the contained human body targets are one or not;
s204: determining human body height information of the included human body target based on the obtained depth image;
s205: judging whether the determined height information of the human body is lower than a preset height threshold value or not;
s206: and when the determined height information of the human body is judged to be lower than a preset height threshold value, alarming.
Here, S204 is the same as S103 shown in fig. 1, S205 is the same as S104 shown in fig. 1, and S206 is the same as S105 shown in fig. 1.
In one implementation manner, when the electronic device determines that the human height information of the human body targets contained in the elevator car is lower than the preset height threshold, that is, when it is determined that the human body targets contained in the elevator car are all children, and when the number of the human body targets is lower than the preset number, the monitoring personnel needs to be reminded to pay attention to the human body targets, and the monitoring personnel takes corresponding measures according to the above situation to avoid danger of the human body targets, that is, children. Before the step of alarming (S105), the method may further include:
when the determined human body height information is judged to be lower than a preset height threshold value, determining the number of the included human body targets;
and when the determined number of the included human body targets is lower than the preset number, executing the step of alarming.
The step of determining the number of included human body targets may be performed between the step of judging whether the determined human body height information is lower than a preset height threshold and the step of alarming, that is, after the determined human body height information is judged to be lower than the preset height threshold, the number of included human body targets is determined; and when the determined number of the included human body targets is lower than the preset number, giving an alarm. The preset number can be any positive integer, for example, the preset number can be 3, that is, when the electronic device determines that the number of the human body targets contained in the elevator car is less than 3, that is, the number of the children is 2 or 1, the electronic device gives an alarm. For another example, the preset number may be 2, that is, when the electronic device determines that the number of the human targets contained in the elevator car is less than 2, that is, the number of the children is 1, the electronic device gives an alarm.
In one implementation, after the step of alerting (S105), the method may further include:
controlling the elevator car to move to a designated floor;
when the elevator car moves to the appointed floor, controlling the elevator car to stop moving;
and controlling the elevator car to open the car door after obtaining the door opening instruction for the elevator car.
In the embodiment of the invention, after the electronic equipment gives an alarm, the electronic equipment can receive the movement control instruction of a monitoring person aiming at the elevator car, or the electronic equipment gives an alarm and automatically triggers the movement control instruction and the like. When the electronic equipment obtains the movement control command, the electronic equipment can control the elevator car to move to a specified floor and control the elevator car to stop at the specified floor.
Wherein, the appointed floor can be a preset floor, such as a floor; or, the floor may be a floor which is in the moving direction of the elevator car and is closest to the current position of the elevator car, and the current position of the elevator car may be: and the electronic equipment obtains the position of the elevator car when the movement control command is obtained. Wherein the direction of movement of the elevator car may comprise ascending or descending.
When the electronic equipment controls the elevator car to stop at the designated floor, the car door of the elevator car can be controlled to be in a closed state until a door opening instruction for the elevator car is obtained, and the elevator car is controlled to open the car door.
Corresponding to the above method embodiment, an embodiment of the present invention further provides an elevator car monitoring apparatus, as shown in fig. 3, the apparatus may include:
a first obtaining module 310, configured to obtain a depth image, which is collected by an image collecting device for an elevator car scene, when it is detected that an elevator car door is closed;
a first determining module 320, configured to determine whether the obtained depth image includes a human target;
a second determining module 330, configured to, when it is determined that a human body target is included, determine human body height information of the included human body target based on the obtained depth image;
a first judging module 340, configured to judge whether the determined height information of the human body is lower than a preset height threshold;
and an alarm module 350, configured to alarm when it is determined that the determined height information of the human body is lower than the preset height threshold.
By applying the embodiment of the invention, when the closing of the elevator car door is detected, the depth image acquired by the image acquisition equipment for the elevator car scene is obtained, whether the depth image contains the human body target or not is determined, when the human body target is determined to be contained, the human body height information of the human body target is further determined, and whether the contained human body target is child or not is judged by combining the preset height threshold value, when the human body height information of the human body target is judged to be lower than the preset height threshold value, the contained human body target can be determined to be child, and the condition that only child rides in the elevator car at the moment can be determined, namely the condition that only child rides in the elevator car. Under the condition that only child took the elevator in the control, need report to the police to warning supervisory personnel and pay close attention to, subsequent, supervisory personnel can carry out corresponding processing, and the child of avoiding taking the elevator appears dangerously, with the realization only has the intelligent monitoring of the condition that child took in the elevator, uses manpower sparingly. Moreover, the situation that monitoring omission occurs due to limited manual energy or human negligence and other factors when monitoring personnel monitor manually, and further tragic situations are avoided.
In one implementation, the first determining module 320 is specifically configured to
Extracting image features from the obtained depth image;
and determining whether the obtained depth image contains the human body target or not based on the extracted image features and a first preset classifier.
In one implementation, the first determining module 320 includes a converting unit and a determining unit;
the conversion unit is used for converting the obtained depth image into point cloud data under a world coordinate system;
the determining unit is used for determining whether the point cloud data comprises point cloud data corresponding to a human body target, wherein when the point cloud data comprises the point cloud data corresponding to the human body target, the obtained depth image is represented to comprise the human body target.
In one implementation, the determining unit is specifically configured to
Clustering the point cloud data to obtain various point cloud subdata;
and determining whether the obtained various types of point cloud sub-data contain point cloud sub-data corresponding to the human body target or not based on the obtained various types of point cloud sub-data and a second preset classifier.
In one implementation, the determining unit is specifically configured to
Clustering the point cloud data to obtain various point cloud subdata;
and determining whether the obtained various point cloud subdata contains point cloud subdata corresponding to the human body target or not based on the obtained various point cloud subdata and a preset human body model.
In one implementation, the image acquisition device acquires a depth image using an included depth image acquisition sub-device;
the conversion unit is particularly used for
Obtaining parameter information of the depth image acquisition sub-device, wherein the parameter information comprises: focal length information, image principal point information, installation height information and installation angle information;
converting the obtained depth image into point cloud data under an equipment coordinate system by using the focal length information and the image principal point information, wherein the equipment coordinate system is as follows: a coordinate system established based on the optical center of the depth image acquisition sub-device;
and converting the point cloud data under the equipment coordinate system into the point cloud data under the world coordinate system by using the installation height information and the installation angle information.
In one implementation, the first obtaining module 310 is specifically configured to
Acquiring a depth image acquired by the image acquisition equipment aiming at an elevator car scene and a color image corresponding to the acquired depth image;
the first determining module 320 is specifically configured to
And determining whether the obtained depth image contains the human body target or not by identifying whether the color image contains the human body target or not.
In one implementation, as shown in fig. 4, the apparatus may further include a second determining module 410;
the second determining module 410 is configured to determine whether there is one included human body target when it is determined that there is a human body target before determining the height information of the included human body target based on the obtained depth image; when it is determined that there is one included human target, the second determining module 330 is triggered.
In one implementation, the apparatus may further include a third determining module;
the third determining module is used for determining the number of the included human body targets when the determined human body height information is judged to be lower than the preset height threshold value before the alarm is given;
triggering the alarm module 350 when the determined number of included human targets is below a preset number.
In one implementation, the apparatus may further include a first control module, a second control module, and a third control module;
the first control module is used for controlling the elevator car to move to a specified floor after the alarm is given;
the second control module is used for controlling the elevator car to stop moving after the elevator car moves to the appointed floor;
and the third control module is used for controlling the elevator car to open the car door after obtaining the door opening instruction for the elevator car.
Corresponding to the above method embodiment, an embodiment of the present invention further provides an elevator system, including: elevator, and any above-mentioned elevator car monitoring device.
Optionally, the elevator system includes a camera for collecting a depth image in the elevator car and sending the depth image to the elevator car monitoring device.
Corresponding to the above method embodiment, an embodiment of the present invention further provides an electronic device, including a processor and a memory, where the memory is used for storing a computer program;
the processor is used for realizing any elevator car monitoring method when executing the computer program stored in the memory.
Corresponding to the above method embodiments, the embodiment of the present invention further provides an electronic device, as shown in fig. 5, including a processor 510, a communication interface 520, a memory 530 and a communication bus 540, where the processor 510, the communication interface 520, and the memory 530 complete mutual communication through the communication bus 540,
a memory 530 for storing a computer program;
the processor 510, when executing the computer program stored in the memory 530, implements any of the elevator car monitoring methods provided by the embodiments of the present invention, and the method may include the following steps:
when the closing of an elevator car door is detected, obtaining a depth image acquired by image acquisition equipment aiming at an elevator car scene;
determining whether the obtained depth image contains a human body target;
when the human body target is determined to be contained, determining human body height information of the contained human body target based on the obtained depth image;
judging whether the determined height information of the human body is lower than a preset height threshold value or not;
and when the determined height information of the human body is judged to be lower than the preset height threshold value, alarming.
By applying the embodiment of the invention, when the closing of the elevator car door is detected, the depth image acquired by the image acquisition equipment for the elevator car scene is obtained, whether the depth image contains the human body target or not is determined, when the human body target is determined to be contained, the human body height information of the human body target is further determined, and whether the contained human body target is child or not is judged by combining the preset height threshold value, when the human body height information of the human body target is judged to be lower than the preset height threshold value, the contained human body target can be determined to be child, and the condition that only child rides in the elevator car at the moment can be determined, namely the condition that only child rides in the elevator car. Under the condition that only child took the elevator in the control, need report to the police to warning supervisory personnel and pay close attention to, subsequent, supervisory personnel can carry out corresponding processing, and the child of avoiding taking the elevator appears dangerously, with the realization only has the intelligent monitoring of the condition that child took in the elevator, uses manpower sparingly. Moreover, the situation that monitoring omission occurs due to limited manual energy or human negligence and other factors when monitoring personnel monitor manually, and further tragic situations are avoided.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
Corresponding to the above method embodiments, the embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements any of the elevator car monitoring methods provided by the embodiments of the present invention, and the method may include the following steps:
when the closing of an elevator car door is detected, obtaining a depth image acquired by image acquisition equipment aiming at an elevator car scene;
determining whether the obtained depth image contains a human body target;
when the human body target is determined to be contained, determining human body height information of the contained human body target based on the obtained depth image;
judging whether the determined height information of the human body is lower than a preset height threshold value or not;
and when the determined height information of the human body is judged to be lower than the preset height threshold value, alarming.
By applying the embodiment of the invention, when the closing of the elevator car door is detected, the depth image acquired by the image acquisition equipment for the elevator car scene is obtained, whether the depth image contains the human body target or not is determined, when the human body target is determined to be contained, the human body height information of the human body target is further determined, and whether the contained human body target is child or not is judged by combining the preset height threshold value, when the human body height information of the human body target is judged to be lower than the preset height threshold value, the contained human body target can be determined to be child, and the condition that only child rides in the elevator car at the moment can be determined, namely the condition that only child rides in the elevator car. Under the condition that only child took the elevator in the control, need report to the police to warning supervisory personnel and pay close attention to, subsequent, supervisory personnel can carry out corresponding processing, and the child of avoiding taking the elevator appears dangerously, with the realization only has the intelligent monitoring of the condition that child took in the elevator, uses manpower sparingly. Moreover, the situation that monitoring omission occurs due to limited manual energy or human negligence and other factors when monitoring personnel monitor manually, and further tragic situations are avoided.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (21)
1. A method of monitoring an elevator car, the method comprising:
when the closing of an elevator car door is detected, obtaining a depth image acquired by image acquisition equipment aiming at an elevator car scene;
determining whether the obtained depth image contains a human body target;
when the human body target is determined to be contained, determining human body height information of the contained human body target based on the obtained depth image;
judging whether the determined height information of the human body is lower than a preset height threshold value or not;
when the determined height information of the human body is judged to be lower than the preset height threshold value, alarming;
automatically triggering a movement control command to control the elevator car to move to a designated floor;
after the elevator car moves to the appointed floor, controlling the elevator car to stop moving, and controlling a car door of the elevator car to be in a closed state until a door opening instruction for the elevator car is obtained;
and after a door opening instruction for the elevator car is obtained, controlling the elevator car to open the car door.
2. The method of claim 1, wherein the step of determining whether the obtained depth image contains a human target comprises:
extracting image features from the obtained depth image;
and determining whether the obtained depth image contains the human body target or not based on the extracted image features and a first preset classifier.
3. The method of claim 1, wherein the step of determining whether the obtained depth image contains a human target comprises:
converting the obtained depth image into point cloud data under a world coordinate system;
and determining whether the point cloud data comprises point cloud data corresponding to the human body target, wherein when the point cloud data comprises the point cloud data corresponding to the human body target, the obtained depth image is represented to comprise the human body target.
4. The method according to claim 3, wherein the step of determining whether the point cloud data includes point cloud data corresponding to a human target comprises:
clustering the point cloud data to obtain various point cloud subdata;
and determining whether the obtained various types of point cloud sub-data contain point cloud sub-data corresponding to the human body target or not based on the obtained various types of point cloud sub-data and a second preset classifier.
5. The method according to claim 3, wherein the step of determining whether the point cloud data includes point cloud data corresponding to a human target comprises:
clustering the point cloud data to obtain various point cloud subdata;
and determining whether the obtained various point cloud subdata contains point cloud subdata corresponding to the human body target or not based on the obtained various point cloud subdata and a preset human body model.
6. The method of claim 3, wherein the image capture device captures a depth image using an included depth image capture sub-device;
the step of converting the obtained depth image into point cloud data in a world coordinate system includes:
obtaining parameter information of the depth image acquisition sub-device, wherein the parameter information comprises: focal length information, image principal point information, installation height information and installation angle information;
converting the obtained depth image into point cloud data under an equipment coordinate system by using the focal length information and the image principal point information, wherein the equipment coordinate system is as follows: a coordinate system established based on the optical center of the depth image acquisition sub-device;
and converting the point cloud data under the equipment coordinate system into the point cloud data under the world coordinate system by using the installation height information and the installation angle information.
7. The method of claim 1, wherein the step of obtaining a depth image captured by an image capture device for an elevator car scene comprises:
acquiring a depth image acquired by the image acquisition equipment aiming at an elevator car scene and a color image corresponding to the acquired depth image;
the step of determining whether the obtained depth image contains a human body target includes:
and determining whether the obtained depth image contains the human body target or not by identifying whether the color image contains the human body target or not.
8. The method according to any of claims 1-7, wherein prior to the step of determining height information of the included human target based on the obtained depth image, the method further comprises:
when the human body targets are determined to be contained, judging whether the contained human body targets are one or not;
and when the number of the included human body targets is judged to be one, the step of determining the height information of the included human body targets based on the obtained depth image is executed.
9. The method of any of claims 1-7, wherein prior to the step of alerting, the method further comprises:
when the determined human body height information is judged to be lower than the preset height threshold value, determining the number of the contained human body targets;
and when the determined number of the included human body targets is lower than the preset number, executing the step of alarming.
10. An elevator car monitoring apparatus, the apparatus comprising:
the first obtaining module is used for obtaining a depth image which is collected by the image collecting equipment aiming at an elevator car scene when the fact that the elevator car door is closed is detected;
the first determining module is used for determining whether the obtained depth image contains a human body target or not;
a second determination module, configured to determine, based on the obtained depth image, human body height information of the included human body target when it is determined that the human body target is included;
the first judgment module is used for judging whether the determined height information of the human body is lower than a preset height threshold value or not;
the alarm module is used for giving an alarm when the determined height information of the human body is judged to be lower than the preset height threshold value;
the first control module is used for automatically triggering a movement control command after the alarm is given out so as to control the elevator car to move to a specified floor;
the second control module is used for controlling the elevator car to stop moving and controlling a car door of the elevator car to be in a closed state until a door opening instruction for the elevator car is obtained after the elevator car moves to the appointed floor;
and the third control module is used for controlling the elevator car to open the car door after the door opening instruction for the elevator car is obtained.
11. The apparatus according to claim 10, wherein the first determining means is specifically configured to determine the first threshold value
Extracting image features from the obtained depth image;
and determining whether the obtained depth image contains the human body target or not based on the extracted image features and a first preset classifier.
12. The apparatus of claim 10, wherein the first determining module comprises a converting unit and a determining unit;
the conversion unit is used for converting the obtained depth image into point cloud data under a world coordinate system;
the determining unit is used for determining whether the point cloud data comprises point cloud data corresponding to a human body target, wherein when the point cloud data comprises the point cloud data corresponding to the human body target, the obtained depth image is represented to comprise the human body target.
13. Device according to claim 12, characterized in that the determination unit is specifically configured to
Clustering the point cloud data to obtain various point cloud subdata;
and determining whether the obtained various types of point cloud sub-data contain point cloud sub-data corresponding to the human body target or not based on the obtained various types of point cloud sub-data and a second preset classifier.
14. Device according to claim 12, characterized in that the determination unit is specifically configured to
Clustering the point cloud data to obtain various point cloud subdata;
and determining whether the obtained various point cloud subdata contains point cloud subdata corresponding to the human body target or not based on the obtained various point cloud subdata and a preset human body model.
15. The apparatus of claim 12, wherein the image capture device captures a depth image using an included depth image capture sub-device;
the conversion unit is particularly used for
Obtaining parameter information of the depth image acquisition sub-device, wherein the parameter information comprises: focal length information, image principal point information, installation height information and installation angle information;
converting the obtained depth image into point cloud data under an equipment coordinate system by using the focal length information and the image principal point information, wherein the equipment coordinate system is as follows: a coordinate system established based on the optical center of the depth image acquisition sub-device;
and converting the point cloud data under the equipment coordinate system into the point cloud data under the world coordinate system by using the installation height information and the installation angle information.
16. The apparatus according to claim 10, characterized in that said first obtaining means are in particular adapted to
Acquiring a depth image acquired by the image acquisition equipment aiming at an elevator car scene and a color image corresponding to the acquired depth image;
the first determining module is specifically used for
And determining whether the obtained depth image contains the human body target or not by identifying whether the color image contains the human body target or not.
17. The apparatus according to any one of claims 10-16, further comprising a second determining module;
the second judging module is used for judging whether the included human body targets are one or not when the included human body targets are determined before the height information of the included human body targets is determined based on the obtained depth image; and when the number of the included human body targets is one, triggering the second determining module.
18. The apparatus according to any of claims 10-16, wherein the apparatus further comprises a third determining module;
the third determining module is used for determining the number of the included human body targets when the determined human body height information is judged to be lower than the preset height threshold value before the alarm is given;
and when the determined number of the included human body targets is lower than a preset number, triggering the alarm module.
19. An electronic device comprising a processor and a memory, wherein the memory is configured to store a computer program;
the processor, when executing a computer program stored in a memory, implements the method of monitoring an elevator car of any of claims 1-9.
20. An elevator system, comprising: elevator and elevator car monitoring arrangement according to any of claims 10-18.
21. The elevator system of claim 20, comprising:
the camera is used for acquiring depth images in the elevator car and sending the depth images to the elevator car monitoring device.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710822052.0A CN109484935B (en) | 2017-09-13 | 2017-09-13 | Elevator car monitoring method, device and system |
PCT/CN2018/101545 WO2019052318A1 (en) | 2017-09-13 | 2018-08-21 | Method, apparatus and system for monitoring elevator car |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710822052.0A CN109484935B (en) | 2017-09-13 | 2017-09-13 | Elevator car monitoring method, device and system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN109484935A CN109484935A (en) | 2019-03-19 |
CN109484935B true CN109484935B (en) | 2020-11-20 |
Family
ID=65688677
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710822052.0A Active CN109484935B (en) | 2017-09-13 | 2017-09-13 | Elevator car monitoring method, device and system |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN109484935B (en) |
WO (1) | WO2019052318A1 (en) |
Families Citing this family (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111031278B (en) * | 2019-11-25 | 2021-02-05 | 广州恒龙信息技术有限公司 | Monitoring method and system based on structured light and TOF |
CN111689327A (en) * | 2020-06-19 | 2020-09-22 | 迅达(中国)电梯有限公司 | Elevator car control system |
CN114063572B (en) * | 2020-07-31 | 2024-05-31 | 北京熵行科技有限公司 | Non-perception intelligent device control method, electronic device and control system |
CN112184737B (en) * | 2020-10-10 | 2024-06-14 | 上海阅目科技有限公司 | Image analysis-based child independent elevator detection and alarm system |
CN112257570B (en) * | 2020-10-20 | 2021-07-27 | 江苏濠汉信息技术有限公司 | Method and device for detecting whether safety helmet of constructor is not worn based on visual analysis |
CN112693981B (en) * | 2020-12-17 | 2023-04-18 | 日立楼宇技术(广州)有限公司 | Adjusting method, device and equipment of touch control box and storage medium |
CN112529953B (en) * | 2020-12-17 | 2022-05-03 | 深圳市普渡科技有限公司 | Elevator space state judgment method and device and storage medium |
CN113104687A (en) * | 2021-04-14 | 2021-07-13 | 合肥工业大学 | Method and device for judging elevator space state based on computer vision recognition |
CN113269111B (en) * | 2021-06-03 | 2024-04-05 | 昆山杜克大学 | Video monitoring-based elevator abnormal behavior detection method and system |
CN113361410B (en) * | 2021-06-07 | 2022-08-26 | 上海数川数据科技有限公司 | Child detection box filtering algorithm based on grid clustering |
CN113492405B (en) * | 2021-06-17 | 2022-08-02 | 达闼机器人股份有限公司 | Robot door opening method and device, storage medium and electronic equipment |
CN113838097B (en) * | 2021-09-29 | 2024-01-09 | 成都新潮传媒集团有限公司 | Camera lens angle deviation detection method, device and storage medium |
CN114155557B (en) * | 2021-12-07 | 2022-12-23 | 美的集团(上海)有限公司 | Positioning method, positioning device, robot and computer-readable storage medium |
CN114180431B (en) * | 2021-12-22 | 2023-12-15 | 佛山市和众电梯技术有限公司 | Safety operation monitoring system for elevator car |
WO2023245402A1 (en) * | 2022-06-21 | 2023-12-28 | Kone Corporation | A method and a triggering system for triggering an acquisition of image data of an empty elevator car |
CN116137074A (en) * | 2023-02-22 | 2023-05-19 | 常熟理工学院 | Automatic detection method and system for passengers in elevator car |
CN115909321B (en) * | 2023-03-08 | 2023-05-12 | 成都睿瞳科技有限责任公司 | Identification method, system and storage medium for elevator car |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103971135A (en) * | 2014-05-05 | 2014-08-06 | 中国民航大学 | Human body target detection method based on head and shoulder depth information features |
CN104751146A (en) * | 2015-04-13 | 2015-07-01 | 中国科学技术大学 | Indoor human body detection method based on 3D (three-dimensional) point cloud image |
CN105923481A (en) * | 2016-06-30 | 2016-09-07 | 爱默生电梯有限公司 | Elevator capable of preventing risk of children accidental entering |
CN106315324A (en) * | 2015-06-16 | 2017-01-11 | 奥的斯电梯公司 | Elevator system capable of monitoring child's use and control method for same |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5274714A (en) * | 1990-06-04 | 1993-12-28 | Neuristics, Inc. | Method and apparatus for determining and organizing feature vectors for neural network recognition |
CN100595121C (en) * | 2007-12-10 | 2010-03-24 | 中国科学院计算技术研究所 | Target place optimized dispatching method and system |
CN102657532B (en) * | 2012-05-04 | 2014-09-17 | 深圳泰山在线科技有限公司 | Height measuring method and device based on body posture identification |
CN104176570A (en) * | 2013-05-22 | 2014-12-03 | 日立电梯(中国)有限公司 | Control method and control system for lift |
CN103413145B (en) * | 2013-08-23 | 2016-09-21 | 南京理工大学 | Intra-articular irrigation method based on depth image |
CN203728342U (en) * | 2014-03-05 | 2014-07-23 | 中国石油大学(华东) | Mistaken stopping preventive system for elevator |
US20170308736A1 (en) * | 2014-10-28 | 2017-10-26 | Hewlett-Packard Development Company, L.P. | Three dimensional object recognition |
CN104598890B (en) * | 2015-01-30 | 2017-07-28 | 南京邮电大学 | A kind of Human bodys' response method based on RGB D videos |
CN105129550B (en) * | 2015-08-03 | 2016-09-14 | 罗普特(厦门)科技集团有限公司 | The elevator safety preventing minor from individually taking controls platform |
CN206486130U (en) * | 2017-01-19 | 2017-09-12 | 北京北迅电梯公司 | Elevator early warning system |
-
2017
- 2017-09-13 CN CN201710822052.0A patent/CN109484935B/en active Active
-
2018
- 2018-08-21 WO PCT/CN2018/101545 patent/WO2019052318A1/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103971135A (en) * | 2014-05-05 | 2014-08-06 | 中国民航大学 | Human body target detection method based on head and shoulder depth information features |
CN104751146A (en) * | 2015-04-13 | 2015-07-01 | 中国科学技术大学 | Indoor human body detection method based on 3D (three-dimensional) point cloud image |
CN106315324A (en) * | 2015-06-16 | 2017-01-11 | 奥的斯电梯公司 | Elevator system capable of monitoring child's use and control method for same |
CN105923481A (en) * | 2016-06-30 | 2016-09-07 | 爱默生电梯有限公司 | Elevator capable of preventing risk of children accidental entering |
Also Published As
Publication number | Publication date |
---|---|
WO2019052318A1 (en) | 2019-03-21 |
CN109484935A (en) | 2019-03-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109484935B (en) | Elevator car monitoring method, device and system | |
WO2020078229A1 (en) | Target object identification method and apparatus, storage medium and electronic apparatus | |
CN110930353B (en) | Method and device for detecting state of hole site protection door, computer equipment and storage medium | |
CN109117827B (en) | Video-based method for automatically identifying wearing state of work clothes and work cap and alarm system | |
CN108446630B (en) | Intelligent monitoring method for airport runway, application server and computer storage medium | |
CN107662872B (en) | Monitoring system and monitoring method for passenger conveyor | |
US20080136934A1 (en) | Flame Detecting Method And Device | |
CN109766779B (en) | Loitering person identification method and related product | |
US8712149B2 (en) | Apparatus and method for foreground detection | |
CN104599427B (en) | A kind of intelligent image type fire alarm system for vcehicular tunnel | |
KR101374139B1 (en) | Monitoring method through image fusion of surveillance system | |
CN112733690B (en) | High-altitude parabolic detection method and device and electronic equipment | |
KR101204259B1 (en) | A method for detecting fire or smoke | |
CN103679704B (en) | Video motion shadow detecting method based on lighting compensation | |
US10762372B2 (en) | Image processing apparatus and control method therefor | |
CN109508576B (en) | Abnormal driving behavior detection method and device and electronic equipment | |
CN112270253A (en) | High-altitude parabolic detection method and device | |
CN108805184B (en) | Image recognition method and system for fixed space and vehicle | |
CN110598596A (en) | Dangerous behavior monitoring method and device and electronic equipment | |
CN110602446A (en) | Garbage recovery reminding method and system and storage medium | |
CN115937746A (en) | Smoke and fire event monitoring method and device and storage medium | |
CN114332925A (en) | Method, system and device for detecting pets in elevator and computer readable storage medium | |
CN209657454U (en) | Coal-mine fire identifying system | |
CN108460357B (en) | Windowing alarm detection system and method based on image recognition | |
CN113936252A (en) | Battery car intelligent management system and method based on video monitoring |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |