CN109484935B

CN109484935B - Elevator car monitoring method, device and system

Info

Publication number: CN109484935B
Application number: CN201710822052.0A
Authority: CN
Inventors: 龚晖; 任烨
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2017-09-13
Filing date: 2017-09-13
Publication date: 2020-11-20
Anticipated expiration: 2037-09-13
Also published as: WO2019052318A1; CN109484935A

Abstract

The embodiment of the invention provides a method, a device and a system for monitoring an elevator car, wherein the method comprises the following steps: when the closing of an elevator car door is detected, obtaining a depth image acquired by image acquisition equipment aiming at an elevator car scene; judging whether the obtained depth image contains a human body target or not; when the human body target is judged to be included, determining human body height information of the included human body target based on the obtained depth image; judging whether the determined height information of the human body is lower than a preset height threshold value or not; and when the determined height information of the human body is judged to be lower than a preset height threshold value, alarming. By applying the embodiment of the invention, the intelligent monitoring of the condition that only children take the elevator is realized, the labor is saved, and the possible omission phenomenon caused by limited labor energy or human negligence and other factors is avoided.

Description

Elevator car monitoring method, device and system

Technical Field

The invention relates to the technical field of computer vision, in particular to a method, a device and a system for monitoring an elevator car.

Background

Elevators have found increasingly widespread use in life as an important vertical conveyance. The elevator brings convenience to people to go out and brings potential safety hazards to people. In one case, the potential safety hazards may include the situation where only a child rides in the elevator car, and since the child is active and the security awareness is weak, if not discovered in time, it is likely that tragedy may occur. Although image acquisition equipment is installed in elevator cars of many buildings, the video monitoring system where the image acquisition equipment is located at present can only provide an image acquisition function; subsequently, the monitoring personnel watch the image collected by the image collecting device through manpower, and identify whether the condition that only children take the elevator appears. Such buildings may include, but are not limited to, residential areas, hotels, and buildings, among others.

The mode of monitoring by manually watching the images needs monitoring personnel to watch the images collected by the image collecting equipment in real time, consumes manpower, is not intelligent enough, and can have the omission phenomenon caused by limited manual energy or human negligence and other factors.

Disclosure of Invention

The embodiment of the invention aims to provide a method, a device and a system for monitoring an elevator car, so as to realize intelligent monitoring on the condition that only children take the elevator, save labor and avoid possible omission phenomenon caused by limited labor energy or human negligence and other factors. The specific technical scheme is as follows:

in one aspect, an embodiment of the present invention provides an elevator car monitoring method, where the method includes:

when the closing of an elevator car door is detected, obtaining a depth image acquired by image acquisition equipment aiming at an elevator car scene;

determining whether the obtained depth image contains a human body target;

when the human body target is determined to be contained, determining human body height information of the contained human body target based on the obtained depth image;

judging whether the determined height information of the human body is lower than a preset height threshold value or not;

and when the determined height information of the human body is judged to be lower than the preset height threshold value, alarming.

Optionally, the step of determining whether the obtained depth image includes a human target includes:

extracting image features from the obtained depth image;

and determining whether the obtained depth image contains the human body target or not based on the extracted image features and a first preset classifier.

converting the obtained depth image into point cloud data under a world coordinate system;

and determining whether the point cloud data comprises point cloud data corresponding to the human body target, wherein when the point cloud data comprises the point cloud data corresponding to the human body target, the obtained depth image is represented to comprise the human body target.

Optionally, the step of determining whether the point cloud data includes point cloud data corresponding to a human target includes:

clustering the point cloud data to obtain various point cloud subdata;

and determining whether the obtained various types of point cloud sub-data contain point cloud sub-data corresponding to the human body target or not based on the obtained various types of point cloud sub-data and a second preset classifier.

clustering the point cloud data to obtain various point cloud subdata;

and determining whether the obtained various point cloud subdata contains point cloud subdata corresponding to the human body target or not based on the obtained various point cloud subdata and a preset human body model.

Optionally, the image acquisition device acquires a depth image by using the included depth image acquisition sub-device;

the step of converting the obtained depth image into point cloud data in a world coordinate system includes:

obtaining parameter information of the depth image acquisition sub-device, wherein the parameter information comprises: focal length information, image principal point information, installation height information and installation angle information;

converting the obtained depth image into point cloud data under an equipment coordinate system by using the focal length information and the image principal point information, wherein the equipment coordinate system is as follows: a coordinate system established based on the optical center of the depth image acquisition sub-device;

and converting the point cloud data under the equipment coordinate system into the point cloud data under the world coordinate system by using the installation height information and the installation angle information.

Optionally, the step of obtaining the depth image captured by the image capturing device for the scene of the elevator car includes:

acquiring a depth image acquired by the image acquisition equipment aiming at an elevator car scene and a color image corresponding to the acquired depth image;

the step of determining whether the obtained depth image contains a human body target includes:

and determining whether the obtained depth image contains the human body target or not by identifying whether the color image contains the human body target or not.

Optionally, before the step of determining the height information of the included human target based on the obtained depth image, the method further includes:

when the human body targets are determined to be contained, judging whether the contained human body targets are one or not;

and when the number of the included human body targets is judged to be one, the step of determining the height information of the included human body targets based on the obtained depth image is executed.

Optionally, before the step of alarming, the method further comprises:

when the determined human body height information is judged to be lower than the preset height threshold value, determining the number of the contained human body targets;

and when the determined number of the included human body targets is lower than the preset number, executing the step of alarming.

Optionally, after the step of alarming, the method further comprises:

controlling the elevator car to move to a designated floor;

when the elevator car moves to the appointed floor, controlling the elevator car to stop moving;

and after a door opening instruction for the elevator car is obtained, controlling the elevator car to open the car door.

In another aspect, an embodiment of the present invention provides an elevator car monitoring apparatus, where the apparatus includes:

the first obtaining module is used for obtaining a depth image which is collected by the image collecting equipment aiming at an elevator car scene when the fact that the elevator car door is closed is detected;

the first determining module is used for determining whether the obtained depth image contains a human body target or not;

a second determination module, configured to determine, based on the obtained depth image, human body height information of the included human body target when it is determined that the human body target is included;

the first judgment module is used for judging whether the determined height information of the human body is lower than a preset height threshold value or not;

and the alarm module is used for giving an alarm when the determined height information of the human body is judged to be lower than the preset height threshold value.

Optionally, the first determining module is specifically configured to

Extracting image features from the obtained depth image;

Optionally, the first determining module includes a converting unit and a determining unit;

the conversion unit is used for converting the obtained depth image into point cloud data under a world coordinate system;

the determining unit is used for determining whether the point cloud data comprises point cloud data corresponding to a human body target, wherein when the point cloud data comprises the point cloud data corresponding to the human body target, the obtained depth image is represented to comprise the human body target.

Optionally, the determination unit is specifically configured to

Clustering the point cloud data to obtain various point cloud subdata;

Optionally, the determination unit is specifically configured to

Clustering the point cloud data to obtain various point cloud subdata;

the conversion unit is particularly used for

Optionally, the first obtaining module is specifically configured to

the first determining module is specifically used for

Optionally, the apparatus further includes a second determining module;

the second judging module is used for judging whether the included human body targets are one or not when the included human body targets are determined before the height information of the included human body targets is determined based on the obtained depth image; and when the number of the included human body targets is one, triggering the second determining module.

Optionally, the apparatus further comprises a third determining module;

the third determining module is used for determining the number of the included human body targets when the determined human body height information is judged to be lower than the preset height threshold value before the alarm is given;

and when the determined number of the included human body targets is lower than a preset number, triggering the alarm module.

Optionally, the apparatus further comprises a first control module, a second control module, and a third control module;

the first control module is used for controlling the elevator car to move to a specified floor after the alarm is given;

the second control module is used for controlling the elevator car to stop moving after the elevator car moves to the appointed floor;

and the third control module is used for controlling the elevator car to open the car door after obtaining the door opening instruction for the elevator car.

In another aspect, an electronic device is provided, comprising a processor and a memory, wherein the memory is configured to store a computer program;

the processor is used for realizing any elevator car monitoring method when executing the computer program stored in the memory.

In another aspect, an elevator system is provided, comprising: elevator, and any above-mentioned elevator car monitoring device.

Optionally, the elevator system includes a camera for collecting a depth image in the elevator car and sending the depth image to the elevator car monitoring device.

In the embodiment of the invention, when the closing of the elevator car door is detected, the depth image acquired by the image acquisition equipment for the elevator car scene is obtained; judging whether the obtained depth image contains a human body target or not; when the human body target is judged to be included, determining human body height information of the included human body target based on the obtained depth image; judging whether the determined height information of the human body is lower than a preset height threshold value or not; and when the determined height information of the human body is judged to be lower than a preset height threshold value, alarming.

When the closing of the elevator car door is detected, the depth image acquired by the image acquisition equipment for the elevator car scene is obtained, whether the depth image contains a human body target or not is determined, when the depth image contains the human body target, the height information of the human body target is further determined, a preset height threshold value is combined, whether the contained human body target is a child or not is judged, when the height information of the human body target is judged to be lower than the preset height threshold value, the contained human body target can be determined to be the child, and the situation that only the child rides in the elevator car at the moment can be determined, namely, the situation that only the child rides the elevator. Under the condition that only child took the elevator in the control, need report to the police to warning supervisory personnel and pay close attention to, subsequent, supervisory personnel can carry out corresponding processing, and the child of avoiding taking the elevator appears dangerously, with the realization only has the intelligent monitoring of the condition that child took in the elevator, uses manpower sparingly. Moreover, the situation that monitoring omission occurs due to limited manual energy or human negligence and other factors when monitoring personnel monitor manually, and further tragic situations are avoided. Of course, it is not necessary for any product or method of practicing the invention to achieve all of the above-described advantages at the same time.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic flow chart of an elevator car monitoring method according to an embodiment of the present invention;

fig. 2 is another schematic flow chart of an elevator car monitoring method according to an embodiment of the present invention;

fig. 3 is a schematic structural view of an elevator car monitoring apparatus according to an embodiment of the present invention;

fig. 4 is another schematic structural view of an elevator car monitoring apparatus according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The embodiment of the invention provides a method, a device and a system for monitoring an elevator car, which are used for realizing intelligent monitoring on the condition that only children take the elevator, saving labor and avoiding possible omission phenomenon caused by limited labor energy or human negligence and other factors.

As shown in fig. 1, an embodiment of the present invention provides an elevator car monitoring method, which may include the following steps:

s101: when the closing of an elevator car door is detected, obtaining a depth image acquired by image acquisition equipment aiming at an elevator car scene;

it can be understood that the elevator car monitoring method provided by the embodiment of the invention can be applied to any electronic device which can obtain the depth image acquired by the image acquisition device, and the electronic device can be a computer, a smart phone, a monitoring server and the like.

The image capturing device may be mounted within the elevator car. When installing above-mentioned image acquisition equipment in elevator car, can install perpendicularly, also can the slope installation, wherein, perpendicular installation can be: the image acquisition equipment can be arranged in the middle of the top of the elevator car and is right opposite to the ground of the elevator car so as to acquire images of scenes in the elevator car; the oblique installation can be installed in the direction that has certain contained angle with the direction that elevator car goes up and down, and above-mentioned image acquisition equipment can be installed in any corner at elevator car top to carry out image acquisition to the scene in the elevator car. It can be understood that no matter how the image acquisition equipment is installed, the image acquisition equipment is ensured to acquire images of all corners in the elevator car to the maximum extent.

The image capturing device may be any device that can capture depth information, i.e., three-dimensional information, of an object in a scene, and may include, but is not limited to, a binocular depth camera, a TOF (Time of Flight) camera, and a structured light camera.

The depth image can contain depth information of each point in the elevator car scene, namely distance information between each point in the elevator car scene and the image acquisition equipment. In an implementation manner, the pixel value of each pixel point in the depth image is: depth information of points in the elevator car scene corresponding to each pixel point. As can be seen, the depth image has the characteristic of being unaffected by factors such as illumination, shading, and chromaticity.

S102: determining whether the obtained depth image contains a human body target;

after the electronic device obtains the depth image, whether the human body target is included in the depth image can be identified. In one implementation, the electronic device may first detect whether a target exists in the depth image, where the target may include a human target and an object target. When the fact that the target exists in the depth image is detected, the electronic equipment determines each target contained in the depth image, subsequently, the electronic equipment identifies each target, and whether each target is a human body target or not is judged.

In one case, the electronic device may pre-store the background depth image of the elevator car scene, that is, the depth image of the elevator car scene when the elevator car is empty; when the electronic equipment detects that the door of the elevator car is closed, the depth image acquired by the image acquisition equipment aiming at the scene of the elevator car is obtained, the obtained depth image is subjected to subtraction with the background depth image to obtain a difference image, and when foreground pixel points exist in the difference image, the fact that the obtained depth image contains a target can be determined. Of course, when the obtained depth image is subtracted from the background depth image, it is required to ensure that the viewing angle of the image capturing device is not changed, that is, the position and/or angle of the image capturing device is not changed.

In one implementation, the electronic device identifies each target, and the process of determining whether each target is a human target may be: and matching each target with a preset three-dimensional human body model, wherein when the matching is successful, the target can be determined to be a human body target, and when the matching is failed, the target can be determined not to be the human body target.

In another implementation manner, the electronic device identifies each target, and the process of determining whether each target is a human target may also be: and classifying each target in the depth image by using a pre-trained classifier, and determining whether each target is a human body target. It is understood that the above-mentioned pre-trained classifier may be: and training the obtained machine learning model based on a training sample and a preset algorithm, wherein the training sample can comprise sample depth images of a human body target and a non-human body target. For the preset algorithm, the preset algorithm may be: in the neural network algorithm including the convolutional neural network correlation algorithm, the machine learning model may be: a neural network model. Alternatively, the preset algorithm may be: a random forest algorithm, in this case, the machine learning model may be: and (4) a random forest classification model. Alternatively, the preset algorithm may be: and a vector machine algorithm is supported, in this case, the machine learning model may be: support vector machine classification models, and so on.

S103: when the human body target is determined to be contained, determining human body height information of the contained human body target based on the obtained depth image;

when the electronic device determines that the depth image contains the human body target, the electronic device may determine human body height information of the contained human body target by using the depth image. In one implementation, the electronic device may convert the obtained depth image into point cloud data in a world coordinate system, and then determine human height information of the human target by using the point cloud data in the world coordinate system corresponding to the human target.

In one case, the highest point cloud data (i.e., the point cloud data corresponding to the top of the head of the human body target) and the lowest point cloud data (i.e., the point cloud data corresponding to the bottom of the foot of the human body target) in the point cloud data of the world coordinate system corresponding to the human body target may be determined, and the height difference between the highest point cloud data and the lowest point cloud data may be used as the human body height information of the human body target.

In another case, the electronic device may sort the point cloud data in the world coordinate system corresponding to each human target based on the height of each point in the point cloud data in the world coordinate system corresponding to each human target, calculate an average value of the heights of a predetermined number of points in the sorting order as a first average value, calculate an average value of the heights of a predetermined number of points in the sorting order as a second average value, and use an absolute value of the first average value and the second average value as the human height information of the human target.

S104: judging whether the determined height information of the human body is lower than a preset height threshold value or not;

s105: and when the determined height information of the human body is judged to be lower than a preset height threshold value, alarming.

After determining the height information of the human body of the included human body target, the electronic device may compare the determined height information of the human body with a preset height threshold value, and determine whether the determined height information of the human body is lower than the preset height threshold value. In one implementation, the depth image may include one or more human body targets, and when there are multiple human body targets, the human body height information of each human body target may be respectively compared with a preset height threshold, and whether the human body height information of each human body target is lower than the preset height threshold is determined.

And when the electronic equipment judges that the determined height information of the human body is lower than a preset height threshold value, alarming. When the electronic device gives an alarm, the electronic device may give an alarm in a form of outputting prompt information, giving an alarm in a form of sending a prompt sound, giving an alarm in a form of changing screen brightness, and the like.

In one implementation, when the electronic device determines that there are a plurality of human body targets included in the depth image, the electronic device may alarm only when the human body height information of the included human body targets is lower than a preset height threshold.

The preset height threshold may be set according to the height of the population as children, and in one implementation, the population as children may include a population under 10 years old.

By applying the embodiment of the invention, when the closing of the elevator car door is detected, the depth image acquired by the image acquisition equipment for the elevator car scene is obtained, whether the depth image contains the human body target or not is determined, when the human body target is determined to be contained, the human body height information of the human body target is further determined, and whether the contained human body target is child or not is judged by combining the preset height threshold value, when the human body height information of the human body target is judged to be lower than the preset height threshold value, the contained human body target can be determined to be child, and the condition that only child rides in the elevator car at the moment can be determined, namely the condition that only child rides in the elevator car. Under the condition that only child took the elevator in the control, need report to the police to warning supervisory personnel and pay close attention to, subsequent, supervisory personnel can carry out corresponding processing, and the child of avoiding taking the elevator appears dangerously, with the realization only has the intelligent monitoring of the condition that child took in the elevator, uses manpower sparingly. Moreover, the situation that monitoring omission occurs due to limited manual energy or human negligence and other factors when monitoring personnel monitor manually, and further tragic situations are avoided.

When the electronic device determines whether the depth image includes the human body target, it may determine whether the depth image includes the human body target by directly using the image features in the depth image, or determine whether the depth image includes the human body target by using the point cloud data features corresponding to the depth image.

In one implementation, the step of determining whether the obtained depth image includes a human target (S102) may include:

extracting image features from the obtained depth image;

It is understood that the first preset classifier may be: training a machine learning model based on training samples and a preset algorithm, wherein the training samples can be: the sample depth image including the human target and the non-human target, that is, the image features used for training the first preset classifier may be: image features in the sample depth image. For the preset algorithm, the preset algorithm may be: in the neural network algorithm including the convolutional neural network correlation algorithm, the machine learning model may be: a neural network model. Alternatively, the preset algorithm may be: a random forest algorithm, in this case, the machine learning model may be: and (4) a random forest classification model. Alternatively, the preset algorithm may be: and a vector machine algorithm is supported, in this case, the machine learning model may be: support vector machine classification models, and so on.

In one implementation, the electronic device may perform image feature extraction on the depth image, such as: and (5) utilizing Canny operator and the like to extract edge features of the depth image. Subsequently, the extracted image features are input into a first preset classifier, and the first preset classifier classifies the extracted image features to determine whether the depth image contains the human body target.

Alternatively, the electronic device may be pre-stored with human image features. After the electronic equipment extracts the image features from the depth image, the extracted image features are matched with the pre-stored human body image features, and when the matching is successful, the fact that the depth image comprises the human body target can be determined.

In another implementation, the step of determining whether the obtained depth image includes a human target (S102) may include:

In the embodiment of the invention, the electronic device may first convert the depth image into point cloud data in a world coordinate system, and then determine whether the depth image includes a human body target based on the point cloud data in the world coordinate system. The image acquisition equipment can acquire a depth image by using the included depth image acquisition sub-equipment, and when the electronic equipment converts the depth image into point cloud data under a world coordinate system, the electronic equipment can convert the depth image into the point cloud data under the world coordinate system by using the parameter information of the depth image acquisition sub-equipment in the image acquisition equipment for acquiring the depth image. The parameter information may include internal parameters and external parameters of the depth image acquisition sub-device, and the electronic device may calibrate the image acquisition device based on a zhangyingyou calibration method, so as to determine the parameter information, or the electronic device may directly calibrate the internal parameters calibrated when the image acquisition device leaves the factory, and the internal parameters and the external parameters may be obtained by measurement and pre-calibration in the parameter information provided in the embodiment of the present invention. In one implementation, the step of converting the obtained depth image into point cloud data in a world coordinate system may include:

converting the obtained depth image into point cloud data under an equipment coordinate system by using the focal length information and the image principal point information, wherein the equipment coordinate system can be as follows: a coordinate system established based on the optical center of the depth image acquisition sub-device;

and converting the point cloud data under the equipment coordinate system into point cloud data under a world coordinate system by using the installation height information and the installation angle information.

It is understood that the above-mentioned image principal points may be: and the intersection point of the optical axis of the depth image acquisition sub-device and the image plane. In the embodiment of the present invention, the information about the image principal point may include: two-dimensional coordinates of the image principal point in the depth image. The two-dimensional coordinates of each pixel point in the depth image can be determined by using the two-dimensional coordinates of the image principal point.

In one implementation, the two-dimensional coordinates (u, v) of each pixel point in the depth image are converted to three-dimensional coordinates (X) in the device coordinate system^C,Y^C,Z^C) To obtain point cloud data of the depth image. The preset three-dimensional rectangular coordinate system may be: based on a coordinate system established by the optical center of the depth image acquisition sub-device, a formula used in coordinate conversion is as follows:

wherein f is_DxAnd f_DyAll are the focal lengths of the depth image acquisition sub-devices; (u)_D0,v_D0) Two-dimensional coordinates of the image principal points; z^CAnd distance information from the point in the scene corresponding to the pixel point (u, v) in the depth image to the depth image acquisition sub-device, namely the pixel value of the pixel point (u, v). Wherein f is_DxRepresenting the determined focal distance in the x-axis direction of the above-mentioned depth-image acquisition sub-apparatus, f_DyIndicating the determined focal length in the y-axis direction of the above-described depth image acquisition sub-apparatus. F above_DxAnd f_DyAll included in the above-mentioned focal length information, can be directly calibrated and determined by Zhang Zhengyou calibration method. The two-dimensional coordinates of the image principal points can also be directly calibrated and determined by a Zhang Zhengyou calibration method.

The device coordinate system may be a three-dimensional rectangular coordinate system. And subsequently, determining a conversion relation between the equipment coordinate system and a world coordinate system by utilizing the installation height information and the installation angle information of the depth image acquisition sub-equipment, including the pitch angle and the deflection angle of the depth image acquisition sub-equipment, and converting point cloud data under the equipment coordinate system into point cloud data under the world coordinate system based on the determined conversion relation.

After the electronic device converts the depth image into point cloud data in a world coordinate system, clustering the point cloud data by using a preset clustering algorithm to determine each target included in the depth image, and further, determining whether each target is a human target, wherein in one case, the step of determining whether the point cloud data includes point cloud data corresponding to the human target may include:

clustering point cloud data to obtain various point cloud subdata;

In another case, the step of determining whether the point cloud data includes point cloud data corresponding to a human target may include:

clustering point cloud data to obtain various point cloud subdata;

In an implementation manner, the preset clustering algorithm may be a density-based clustering algorithm, for example: DBSCAN (Density-Based Spatial Clustering of Applications with Noise) Clustering algorithm, wherein the DBSCAN Clustering algorithm requires two parameters, namely scanning radius (eps) and minimum contained points (minPts). After the electronic device obtains the point cloud data, selecting an unvisited point (i.e. a point which is not marked as visited or not marked as noise) from the point cloud data by using the DBSCAN clustering algorithm as a current point, and determining all points which are within an eps distance (including eps) from the current point as nearby points; the electronic device determines whether the number of nearby points of the current point is equal to or greater than minPts, and if the electronic device determines that the number of nearby points of the current point is equal to or greater than minPts, the electronic device determines the current point and the determined nearby points as a cluster, and the electronic device marks the current point as visited (visited); the electronic device recursively processes all the points in the cluster that are not marked as visited (visited) or not marked as noise in the manner described above, thereby expanding the cluster; if the cluster is sufficiently expanded, i.e., all points within the cluster are marked as visited; and the electronic equipment selects an unaccessed point again as the current point and executes the subsequent flow. When the electronic equipment determines that the number of the nearby points of the current point is less than minPts, the electronic equipment marks the current point as noise; and finishing clustering until the point cloud data does not contain the points which are not accessed. At this time, the electronic device may determine various types of point cloud sub-data included in the point cloud data, and subsequently, the electronic device may process each type of point cloud sub-data to determine whether each type of point cloud sub-data is point cloud sub-data corresponding to the human body target.

In one case, the electronic device may input each type of the obtained point cloud sub-data into a second preset classifier, so that the second preset classifier classifies each type of the point cloud sub-data, and determines whether each type of the point cloud sub-data is the point cloud sub-data corresponding to the human body target. It is understood that the second preset classifier may be: training a machine learning model based on training samples and a preset algorithm, wherein the training samples can be: the method comprises the steps of obtaining point cloud data corresponding to sample depth images of human body targets and non-human body targets. For the preset algorithm, the preset algorithm may be: in the neural network algorithm including the convolutional neural network correlation algorithm, the machine learning model may be: a neural network model. Alternatively, the preset algorithm may be: a random forest algorithm, in this case, the machine learning model may be: and (4) a random forest classification model. Alternatively, the preset algorithm may be: and a vector machine algorithm is supported, in this case, the machine learning model may be: support vector machine classification models, and so on.

In another case, the electronic device may match each type of acquired point cloud sub-data with a preset human body model, determine that the point cloud sub-data is the point cloud sub-data corresponding to the human body target when the matching is successful, and determine that the point cloud sub-data is not the point cloud sub-data corresponding to the human body target when the matching is failed.

In an implementation manner, the preset clustering algorithm may also be a grid-based clustering algorithm, for example: STING clustering algorithm, etc., and may also be a partition-based clustering algorithm, such as: the embodiment of the invention does not limit the algorithm capable of clustering the point cloud data, and all the algorithms capable of clustering the point cloud data can be applied to the embodiment of the invention.

In one implementation, it may be determined whether the depth image includes a human target by using a color image corresponding to the depth image acquired for the elevator car, where the step of obtaining the depth image acquired by the image acquisition device for the elevator car scene may include:

acquiring a depth image acquired by an image acquisition device aiming at an elevator car scene and a color image corresponding to the acquired depth image;

the step of determining whether the obtained depth image contains the human body target may include:

Whether the human body target is contained in the depth image can be determined by identifying whether the human body target is contained in the color image through a face identification algorithm, wherein when the human body target is contained in the color image, the human body target is contained in the depth image; when the color image does not contain the human body target, the depth image does not contain the human body target.

In an embodiment of the present invention, the color image may be an image in any color mode, for example: an image of RGB (Red Green Blue ) color mode, or a YUV color mode image, and so on. Among them, YUV (also called YCrCb) is a color coding method, and "Y" represents brightness (Luma or Luma), i.e., a gray level value; and "U" and "V" represent Chroma (Chroma) and function to describe image color and saturation for specifying the color of a pixel. "chroma" defines two aspects of the color of a pixel-hue and saturation, which can be expressed as Cr and Cb, respectively.

In one case, when only one child takes the elevator, the probability of danger may be greater, in order to better avoid the danger when only one child takes the elevator, and when only one child takes the elevator, the monitoring person needs to pay more attention and take corresponding measures to avoid the danger of the child taking the elevator alone. In one implementation, as shown in fig. 2, the method may include the steps of:

s201: when the closing of an elevator car door is detected, obtaining a depth image acquired by image acquisition equipment aiming at an elevator car scene;

s202: determining whether the obtained depth image contains a human body target;

here, S201 is the same as S101 shown in fig. 1, and S202 is the same as S102 shown in fig. 1.

S203: when the human body targets are determined to be contained, judging whether the contained human body targets are one or not;

s204: determining human body height information of the included human body target based on the obtained depth image;

s205: judging whether the determined height information of the human body is lower than a preset height threshold value or not;

s206: and when the determined height information of the human body is judged to be lower than a preset height threshold value, alarming.

Here, S204 is the same as S103 shown in fig. 1, S205 is the same as S104 shown in fig. 1, and S206 is the same as S105 shown in fig. 1.

In one implementation manner, when the electronic device determines that the human height information of the human body targets contained in the elevator car is lower than the preset height threshold, that is, when it is determined that the human body targets contained in the elevator car are all children, and when the number of the human body targets is lower than the preset number, the monitoring personnel needs to be reminded to pay attention to the human body targets, and the monitoring personnel takes corresponding measures according to the above situation to avoid danger of the human body targets, that is, children. Before the step of alarming (S105), the method may further include:

when the determined human body height information is judged to be lower than a preset height threshold value, determining the number of the included human body targets;

The step of determining the number of included human body targets may be performed between the step of judging whether the determined human body height information is lower than a preset height threshold and the step of alarming, that is, after the determined human body height information is judged to be lower than the preset height threshold, the number of included human body targets is determined; and when the determined number of the included human body targets is lower than the preset number, giving an alarm. The preset number can be any positive integer, for example, the preset number can be 3, that is, when the electronic device determines that the number of the human body targets contained in the elevator car is less than 3, that is, the number of the children is 2 or 1, the electronic device gives an alarm. For another example, the preset number may be 2, that is, when the electronic device determines that the number of the human targets contained in the elevator car is less than 2, that is, the number of the children is 1, the electronic device gives an alarm.

In one implementation, after the step of alerting (S105), the method may further include:

controlling the elevator car to move to a designated floor;

and controlling the elevator car to open the car door after obtaining the door opening instruction for the elevator car.

In the embodiment of the invention, after the electronic equipment gives an alarm, the electronic equipment can receive the movement control instruction of a monitoring person aiming at the elevator car, or the electronic equipment gives an alarm and automatically triggers the movement control instruction and the like. When the electronic equipment obtains the movement control command, the electronic equipment can control the elevator car to move to a specified floor and control the elevator car to stop at the specified floor.

Wherein, the appointed floor can be a preset floor, such as a floor; or, the floor may be a floor which is in the moving direction of the elevator car and is closest to the current position of the elevator car, and the current position of the elevator car may be: and the electronic equipment obtains the position of the elevator car when the movement control command is obtained. Wherein the direction of movement of the elevator car may comprise ascending or descending.

When the electronic equipment controls the elevator car to stop at the designated floor, the car door of the elevator car can be controlled to be in a closed state until a door opening instruction for the elevator car is obtained, and the elevator car is controlled to open the car door.

Corresponding to the above method embodiment, an embodiment of the present invention further provides an elevator car monitoring apparatus, as shown in fig. 3, the apparatus may include:

a first obtaining module 310, configured to obtain a depth image, which is collected by an image collecting device for an elevator car scene, when it is detected that an elevator car door is closed;

a first determining module 320, configured to determine whether the obtained depth image includes a human target;

a second determining module 330, configured to, when it is determined that a human body target is included, determine human body height information of the included human body target based on the obtained depth image;

a first judging module 340, configured to judge whether the determined height information of the human body is lower than a preset height threshold;

and an alarm module 350, configured to alarm when it is determined that the determined height information of the human body is lower than the preset height threshold.

In one implementation, the first determining module 320 is specifically configured to

Extracting image features from the obtained depth image;

In one implementation, the first determining module 320 includes a converting unit and a determining unit;

In one implementation, the determining unit is specifically configured to

Clustering the point cloud data to obtain various point cloud subdata;

In one implementation, the determining unit is specifically configured to

Clustering the point cloud data to obtain various point cloud subdata;

In one implementation, the image acquisition device acquires a depth image using an included depth image acquisition sub-device;

the conversion unit is particularly used for

In one implementation, the first obtaining module 310 is specifically configured to

the first determining module 320 is specifically configured to

In one implementation, as shown in fig. 4, the apparatus may further include a second determining module 410;

the second determining module 410 is configured to determine whether there is one included human body target when it is determined that there is a human body target before determining the height information of the included human body target based on the obtained depth image; when it is determined that there is one included human target, the second determining module 330 is triggered.

In one implementation, the apparatus may further include a third determining module;

triggering the alarm module 350 when the determined number of included human targets is below a preset number.

In one implementation, the apparatus may further include a first control module, a second control module, and a third control module;

Corresponding to the above method embodiment, an embodiment of the present invention further provides an elevator system, including: elevator, and any above-mentioned elevator car monitoring device.

Corresponding to the above method embodiment, an embodiment of the present invention further provides an electronic device, including a processor and a memory, where the memory is used for storing a computer program;

Corresponding to the above method embodiments, the embodiment of the present invention further provides an electronic device, as shown in fig. 5, including a processor 510, a communication interface 520, a memory 530 and a communication bus 540, where the processor 510, the communication interface 520, and the memory 530 complete mutual communication through the communication bus 540,

a memory 530 for storing a computer program;

the processor 510, when executing the computer program stored in the memory 530, implements any of the elevator car monitoring methods provided by the embodiments of the present invention, and the method may include the following steps:

determining whether the obtained depth image contains a human body target;

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

Corresponding to the above method embodiments, the embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements any of the elevator car monitoring methods provided by the embodiments of the present invention, and the method may include the following steps:

determining whether the obtained depth image contains a human body target;

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method of monitoring an elevator car, the method comprising:

determining whether the obtained depth image contains a human body target;

when the determined height information of the human body is judged to be lower than the preset height threshold value, alarming;

automatically triggering a movement control command to control the elevator car to move to a designated floor;

after the elevator car moves to the appointed floor, controlling the elevator car to stop moving, and controlling a car door of the elevator car to be in a closed state until a door opening instruction for the elevator car is obtained;

2. The method of claim 1, wherein the step of determining whether the obtained depth image contains a human target comprises:

extracting image features from the obtained depth image;

3. The method of claim 1, wherein the step of determining whether the obtained depth image contains a human target comprises:

4. The method according to claim 3, wherein the step of determining whether the point cloud data includes point cloud data corresponding to a human target comprises:

clustering the point cloud data to obtain various point cloud subdata;

5. The method according to claim 3, wherein the step of determining whether the point cloud data includes point cloud data corresponding to a human target comprises:

clustering the point cloud data to obtain various point cloud subdata;

6. The method of claim 3, wherein the image capture device captures a depth image using an included depth image capture sub-device;

7. The method of claim 1, wherein the step of obtaining a depth image captured by an image capture device for an elevator car scene comprises:

8. The method according to any of claims 1-7, wherein prior to the step of determining height information of the included human target based on the obtained depth image, the method further comprises:

9. The method of any of claims 1-7, wherein prior to the step of alerting, the method further comprises:

10. An elevator car monitoring apparatus, the apparatus comprising:

the alarm module is used for giving an alarm when the determined height information of the human body is judged to be lower than the preset height threshold value;

the first control module is used for automatically triggering a movement control command after the alarm is given out so as to control the elevator car to move to a specified floor;

the second control module is used for controlling the elevator car to stop moving and controlling a car door of the elevator car to be in a closed state until a door opening instruction for the elevator car is obtained after the elevator car moves to the appointed floor;

and the third control module is used for controlling the elevator car to open the car door after the door opening instruction for the elevator car is obtained.

11. The apparatus according to claim 10, wherein the first determining means is specifically configured to determine the first threshold value

Extracting image features from the obtained depth image;

12. The apparatus of claim 10, wherein the first determining module comprises a converting unit and a determining unit;

13. Device according to claim 12, characterized in that the determination unit is specifically configured to

Clustering the point cloud data to obtain various point cloud subdata;

14. Device according to claim 12, characterized in that the determination unit is specifically configured to

Clustering the point cloud data to obtain various point cloud subdata;

15. The apparatus of claim 12, wherein the image capture device captures a depth image using an included depth image capture sub-device;

the conversion unit is particularly used for

16. The apparatus according to claim 10, characterized in that said first obtaining means are in particular adapted to

the first determining module is specifically used for

17. The apparatus according to any one of claims 10-16, further comprising a second determining module;

18. The apparatus according to any of claims 10-16, wherein the apparatus further comprises a third determining module;

19. An electronic device comprising a processor and a memory, wherein the memory is configured to store a computer program;

the processor, when executing a computer program stored in a memory, implements the method of monitoring an elevator car of any of claims 1-9.

20. An elevator system, comprising: elevator and elevator car monitoring arrangement according to any of claims 10-18.

21. The elevator system of claim 20, comprising:

the camera is used for acquiring depth images in the elevator car and sending the depth images to the elevator car monitoring device.