CN111275923B

CN111275923B - Man-machine collision early warning method and system for construction site

Info

Publication number: CN111275923B
Application number: CN202010074149.XA
Authority: CN
Inventors: 方东平; 郭红领; 古博韬; 张沛尧; 黄玥诚; 苗春刚; 周予启
Original assignee: Tsinghua University; China Construction First Group Construction and Development Co Ltd
Current assignee: Tsinghua University; China Construction First Group Construction and Development Co Ltd
Priority date: 2020-01-22
Filing date: 2020-01-22
Publication date: 2020-11-24
Anticipated expiration: 2040-01-22
Also published as: CN111275923A

Abstract

The disclosure relates to a man-machine collision early warning method and a system for a construction site, wherein the system comprises the following steps: the acquisition module is used for acquiring image data of a construction site through a plurality of binocular cameras arranged on mobile mechanical equipment of the construction site; the processing module is connected to the acquisition module and used for selecting target image data containing workers in the image from the image data and determining the actual distance between the workers and the mechanical equipment according to each target image data; and the alarm module is used for determining a corresponding human-computer collision early warning prompting mode for prompting according to the actual distance between the worker and the mechanical equipment. Through the technical scheme, the actual distance between workers and mechanical equipment on a construction site can be automatically determined through the image data, and then early warning prompt is carried out according to the actual distance, so that potential hidden dangers during construction are effectively eliminated, workers do not need to be relied on, and time is saved.

Description

Man-machine collision early warning method and system for construction site

Technical Field

The disclosure relates to the technical field of safety monitoring of construction sites, in particular to a man-machine collision early warning method and system for a construction site.

Background

The construction industry is still the most dangerous industry worldwide, a great amount of casualties are caused every year, and China also faces challenges when the safety situation is more and more. In recent years, as the degree of mechanization of a construction site has been gradually increased, various kinds of construction machinery such as a crane and an excavator are used in construction work, which brings convenience to construction, but also brings more accidents related to the construction machinery. Among all accidents related to construction machines, man-machine collision accidents are one important type of accidents. Workers need to work near the machine due to work task requirements or site condition limitations, and when the workers and the machine are close to each other in the same space-time range and are smaller than a certain threshold value, a human-machine collision risk is formed.

At present, the early warning of human-computer collision is generally carried out by monitoring and evaluating on the basis of manual inspection. The specific operation is that safety related personnel, such as project managers, safety officers and team leaders, are managed by a safety check list method. Safety checklists can effectively eliminate potential hazards during construction, but they are highly dependent on the personal abilities of safety-related personnel and require extensive time for on-site supervision. When the safety-related personnel are not present or real-time and sudden risk sources exist, no method for dealing with the risks is provided.

Disclosure of Invention

In order to overcome the problems in the related art, the invention provides a man-machine collision early warning method and system for a construction site.

According to a first aspect of the embodiments of the present disclosure, a human-computer collision early warning system for a construction site is provided, which includes:

the acquisition module is used for acquiring image data of a construction site through a plurality of binocular cameras arranged on mobile mechanical equipment of the construction site;

the processing module is connected to the acquisition module and used for selecting target image data containing workers in the image from the image data and determining the actual distance between the workers and the mechanical equipment according to each target image data;

and the alarm module is used for determining a corresponding human-computer collision early warning prompting mode for prompting according to the actual distance between the worker and the mechanical equipment.

In one embodiment, preferably, the processing module includes:

the recognition unit is used for recognizing target image data containing workers in the image from the image data through a deep learning algorithm and determining the coordinates of pixel points corresponding to the human body range of the workers in each target image data;

the depth calculation unit is used for calculating the depth distance from an object corresponding to each pixel point in the target image data to the camera through a binocular vision algorithm;

the distance determining unit is used for determining a central point coordinate of each human body range, taking 5 multiplied by 5 target pixel points in the human body range at equal intervals by taking the central point coordinate as a reference coordinate, generating a frequency distribution histogram according to the distance between the extreme difference of the depth distances corresponding to all the target pixel points and the quotient of 5, and determining the average value of the depth distances corresponding to all the target pixel points contained in the histogram with the maximum frequency as the distance between the worker and the binocular camera;

the acquisition unit is used for acquiring an absolute angle formed by each binocular camera and the due north direction and a relative angle between each binocular camera and a corresponding worker on a construction site;

and the distance calculation unit is used for calculating the distance between the worker and the mobile mechanical equipment according to the distance between the worker and the binocular camera, the absolute angle and the relative angle.

In one embodiment, preferably, the deep learning algorithm includes: CNN algorithm, ResNet algorithm, and Unet algorithm.

In one embodiment, preferably, the alarm module includes:

the first warning unit is used for prompting in a first human-computer collision early warning prompting mode when the actual distance between the worker and the mechanical equipment is smaller than a preset distance threshold value, wherein the first human-computer collision early warning prompting mode comprises interface display and sound warning;

and the second alarm unit is used for prompting through a second human-computer collision early warning prompting mode when the actual distance between the worker and the mechanical equipment is greater than or equal to a preset distance threshold value, wherein the second human-computer collision early warning prompting mode comprises interface display.

In one embodiment, preferably, the system further comprises:

and the threshold value determining module is used for determining the preset distance threshold value by a Delphi method.

According to a second aspect of the embodiments of the present disclosure, there is provided a human-computer collision warning method for a construction site, including:

acquiring image data of a construction site through a plurality of binocular cameras arranged on mobile mechanical equipment of the construction site;

selecting target image data containing workers in the image from the image data, and determining the actual distance between the workers and the mechanical equipment according to each target image data;

and determining a corresponding human-computer collision early warning prompting mode for prompting according to the actual distance between the worker and the mechanical equipment.

In one embodiment, preferably, the selecting target image data of the worker included in the image from the image data and determining the actual distance between the worker and the mechanical device according to each target image data includes:

identifying target image data containing workers in the image from the image data through a deep learning algorithm, and determining coordinates of pixel points corresponding to the human body range of the workers in each target image data, wherein the deep learning algorithm comprises the following steps: CNN algorithm, ResNet algorithm, and Unet algorithm;

calculating the depth distance from an object corresponding to each pixel point in the target image data to the camera through a binocular vision algorithm;

determining a central point coordinate of each human body range, taking the central point coordinate as a reference coordinate, taking 5 multiplied by 5 target pixel points in the human body range at equal intervals, generating a frequency distribution histogram according to the range difference of the depth distances corresponding to all the target pixel points and the quotient of 5 as a distance, and determining the average value of the depth distances corresponding to all the target pixel points contained in the histogram with the maximum frequency as the distance between the worker and the binocular camera;

acquiring an absolute angle formed by each binocular camera and the due north direction and a relative angle between each binocular camera and a corresponding worker on a construction site;

and calculating the distance between the worker and the mobile mechanical equipment according to the distance between the worker and the binocular camera, the absolute angle and the relative angle.

In one embodiment, preferably, the determining, according to the actual distance between the worker and the mechanical device, a corresponding human-computer collision warning prompting manner for prompting includes:

when the actual distance between the worker and the mechanical equipment is smaller than a preset distance threshold value, prompting is carried out through a first human-computer collision early warning prompting mode, wherein the first human-computer collision early warning prompting mode comprises interface display and sound alarm;

and when the actual distance between the worker and the mechanical equipment is greater than or equal to a preset distance threshold value, prompting is carried out through a second human-computer collision early warning prompting mode, wherein the second human-computer collision early warning prompting mode comprises interface display.

In one embodiment, preferably, the method further comprises:

and determining the preset distance threshold value by a Delphi method.

According to a third aspect of the embodiments of the present disclosure, there is provided a human-computer collision warning system for a construction site, including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

positioning the position of each data acquisition module, and determining the coordinates of each data acquisition module;

acquiring an absolute angle set by each data acquisition module and a relative angle between each data acquisition module and a worker on a construction site;

acquiring image data of a construction site through the data acquisition module, and selecting target image data from the image data, wherein the target image data contains workers, and the distance between the workers and the mobile mechanical equipment is within a preset distance range;

and storing the target image data according to a storage mode of image data and label data corresponding to the target image data, wherein the label data comprises coordinates of the data acquisition module, acquisition time, the absolute angle, the relative angle and the distance between the worker and the mobile mechanical equipment.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:

in the embodiment of the invention, the actual distance between workers and mechanical equipment on the construction site can be automatically determined through the image data, and the early warning prompt is carried out according to the actual distance, so that potential hidden dangers during construction can be effectively eliminated, workers do not need to be relied on, and the time is saved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

Fig. 1 is a schematic structural diagram illustrating a man-machine collision warning system of a construction site according to an exemplary embodiment.

Fig. 2 is a schematic structural diagram illustrating processing modules in a human-computer collision warning system at a construction site according to an exemplary embodiment.

Fig. 3-a is a diagram of a Resnet network shown in accordance with an exemplary embodiment.

Fig. 3-B is a schematic diagram of the structure of the uet shown in accordance with an exemplary embodiment.

FIG. 4 is a diagram illustrating a FPN, according to an exemplary embodiment.

Fig. 5 is a diagram illustrating a binocular vision algorithm according to an exemplary embodiment.

FIG. 6 is a schematic diagram illustrating a moving average filtering algorithm according to an exemplary embodiment.

Fig. 7 is a schematic diagram illustrating a structure of an alarm module in a human-machine collision warning system of a construction site according to an exemplary embodiment.

FIG. 8 is an interface interaction forewarning diagram shown in accordance with an exemplary embodiment.

Fig. 9 is a schematic structural diagram illustrating another human-machine collision warning system for a construction site according to an exemplary embodiment.

Fig. 10 is a schematic structural diagram illustrating another human-machine collision warning system for a construction site according to an exemplary embodiment.

Fig. 11 is a schematic diagram illustrating a structure of a device tri-proof apparatus according to an exemplary embodiment.

FIG. 12 is a schematic diagram illustrating a shock protection design in a three proofing apparatus of a device according to an exemplary embodiment.

FIG. 13 is a schematic illustration of a mobile crane forewarning range determination in accordance with an exemplary embodiment.

FIG. 14 is a mobile crane worker identification schematic according to an exemplary embodiment.

FIG. 15 is a schematic diagram of a mobile crane worker early warning system according to an exemplary embodiment.

FIG. 16 is a flow chart illustrating a method for human-machine collision warning at a job site according to an exemplary embodiment.

Fig. 17 is a flowchart illustrating step S1602 in a man-machine collision warning method for a construction site according to an exemplary embodiment.

Fig. 18 is a flowchart illustrating step S1603 of a man-machine collision warning method for a construction site according to an exemplary embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

In a construction site, the application of the mechanical equipment for hoisting, transporting and excavating the building can effectively shorten the construction period and improve the construction efficiency. The total quantity of mechanical equipment on a construction site is increased year by year from 2002, which is a necessary result that China becomes a large-scale building country and is a new requirement proposed by modern construction. With the ever-increasing production scale and the continuous reduction of production cycles in the construction industry, construction machines are playing an increasingly important role on site. The production efficiency is improved, and accidents related to machines on construction sites are increased continuously. About one-fourth of the accidents at present on-site are machine related. Among all accidents related to construction machines, man-machine collision accidents are one important type of accidents. Human-to-machine collision accidents at construction sites are often caused or initiated by worker improper contact with all or part of the construction machine. At the construction site, the machine may be in a state of being wholly or partially moved as required by a work task, such as a forklift moving forward or backward as it shovels earth, a crane extending a boom as it is being hoisted, or a rotating crane. Workers can also work near the machine due to work task requirements or site condition constraints, such as the rope workers binding the materials hoisted by the crane, the workers and the machine working in a narrow site at the same time. When the worker and the machine are close to each other in the same space-time range and are smaller than a certain threshold value, the human-machine collision risk is formed. Therefore, effective technical or management means are required to prevent the collision accident caused by the continuous approach of the two.

The invention introduces the deep learning technology and the binocular stereo vision technology which are rapidly developed in the field of computers into the research of the safety management of a construction site by taking the relative distance between a constructor and construction mechanical equipment as an index by means of the deep learning vision technology, and solves the problems in the safety management of construction by using an updated technical means with better performance.

The man-machine collision early warning system and method for a construction site based on deep learning vision technology according to the embodiment of the invention are described below with reference to the accompanying drawings, and firstly, the man-machine collision early warning system for a construction site according to the embodiment of the invention will be described with reference to the accompanying drawings.

Fig. 1 is a schematic structural diagram illustrating a human-machine collision warning system of a construction site according to an exemplary embodiment, and as shown in fig. 1, the human-machine collision warning system 100 of the construction site includes: the system comprises an acquisition module 101, a processing module 102 and an alarm module 103;

and the acquisition module 101 is connected with the processing module 102 and is used for acquiring image data of a construction site through a plurality of binocular cameras arranged on mobile mechanical equipment of the construction site.

And the processing module 102 is connected to the acquisition module 101 and is used for selecting target image data containing workers in the images from the image data and determining the actual distance between each worker and the mechanical equipment according to each target image data. Wherein, processing module can be realized through the industrial computer.

And the alarm module 103 is connected to the processing module 102 and is used for determining a corresponding human-computer collision early warning prompting mode to prompt according to the actual distance between the worker and the mechanical equipment. The alarm module can be realized by a display screen on mechanical equipment or other display equipment.

In the embodiment, the actual distance between workers and mechanical equipment on the construction site can be automatically determined through the image data, and then early warning prompt is carried out according to the actual distance, so that potential hidden dangers during construction are effectively eliminated, workers do not need to be relied on, and time is saved.

As shown in fig. 2, in one embodiment, the processing module 102 preferably includes:

the recognition unit 201 is used for recognizing target image data containing workers in the image from the image data through a deep learning algorithm and determining coordinates of pixel points corresponding to the human body range of the workers in each target image data;

In order to solve the problem that the equipment records redundant data when no person is in construction operation, the invention adopts a deep learning technology and records the working scene after people are identified. The algorithm selects a Convolutional Neural Network (CNN) algorithm, and particularly combines ResNet and Unet on network design.

The classification accuracy is reduced due to the gradient disappearance problem as the depth of the neural network is deepened, the Resnet network solves the problem to a certain extent by using a residual error unit, and high-quality high-level information is extracted. The specific units are shown in fig. 3-a. Input x output h (x) ═ f (x) + x.

As the depth of the neural network is pooled for multiple times, the spatial resolution of the features is lost, the recognition accuracy is reduced, and the boundary details are easily lost. The Unet structure fuses high-level information and low-level information, so that the boundary of the object frame is more accurate. The schematic is shown in fig. 3-B.

In the Detection structure, the limited memory and power supply of the mobile equipment are considered, an algorithm with low calculation amount needs to be selected, and the consumption is reduced. The algorithm combines FPN and One-stage algorithms, and aims to reduce calculation consumption and accelerate the detection process. The FPN utilizes hierarchical semantic features of the convolutional network to construct a feature pyramid, and high-level semantic features of all scales are constructed through a hierarchical structure from top to bottom and with lateral connection. FPN is as the feature extractor, is showing the performance that has promoted. Compared with the traditional image pyramid, the two images perform equally well on different distant and near objects, but the FPN avoids excessive calculation consumption in the image pyramid, and a schematic diagram is shown in FIG. 4.

The One-stage detection algorithm avoids additional operations such as a regionproposal network in the second stage, directly generates the class probability and the position coordinate value of an object, and can directly obtain a final detection result through single detection, so that the detection speed is higher, and the detection speed is more typical algorithms such as YOLO, SSD and Retina-Net.

The Convolutional Neural Network (CNN) algorithm realizes real-time detection of the worker target, and the algorithm does not use manually set rules to identify the characteristics of the worker target in the image, but uses a large number of picture samples marked with workers to train, so that the algorithm obtains the capability of identifying the workers through learning, and the accuracy is continuously improved.

And the depth calculating unit 202 is configured to calculate a depth distance from an object corresponding to each pixel point in the target image data to the camera through a binocular vision algorithm.

Wherein, fig. 5 shows a schematic view of a binocular vision algorithm, as shown in fig. 5, an object P in the diagram is an object to be measured, O_lAnd O_rIs two cameras of a binocular camera, T is the distance between the actual two cameras, f is the camera focal length, so T and f are known parameters. According to the principle of similar triangles

Therefore, the object distance problem is solved by determining the parallax D ═ X_l-X_rTo a problem of (a).

The parallax is the difference value of corresponding x coordinates of the same space point in the imaging of the two cameras, each pixel point has a gray value through encoding in the imaging, and the gray value closer to the lens is brighter. Corresponding pixel points can be found in the two photos through the gray value so as to calculate the gray value to obtain the parallax D, and therefore the depth image can be obtained through a binocular distance measurement algorithm.

The distance determining unit 203 is configured to determine a center point coordinate of each human body range, take 5 × 5 target pixel points in the human body range at equal intervals by using the center point coordinate as a reference coordinate, generate a frequency distribution histogram according to a distance between a range difference of depth distances corresponding to all target pixel points and a quotient of 5, and determine an average value of the depth distances corresponding to all target pixel points included in the histogram with the highest frequency as a distance between the worker and the binocular camera.

It should be noted that, because a general binocular camera can obtain 90 frames of images in one second, human eyes can only recognize 24 frames in one second, and actually 0.1s in one frame can meet the anti-collision requirement, based on the consideration of power consumption, the embodiment of the present invention sets 0.1s to perform distance calculation once.

Because the points are not taken on workers, the overall accuracy is improved, the specific algorithm is as shown in fig. 6, the worker distance data obtained by each frame is regarded as a queue, 5 values are continuously taken as a circular queue, the length of the queue is fixed to be 5, a frame of data is placed at the tail of the queue every time, a piece of data at the head of the original queue is thrown away, and the data output by the filter each time is always the arithmetic average value of 5 data in the current queue.

The binocular camera transmits 10 frames of pictures containing distance data to the processing module 102 of the system every second. In order to ensure the transmission speed, the embodiment of the present invention uses the optical fiber for transmission, and those skilled in the art can select the optical fiber according to specific situations, and is not limited specifically herein.

The obtaining unit 204 obtains an absolute angle formed by each binocular camera and the due north direction, and a relative angle between each binocular camera and a corresponding worker at the construction site. The absolute angle formed by each binocular camera and the due north direction and the relative angle between each binocular camera and the corresponding worker on the construction site can be obtained through the IMU sensor.

A distance calculating unit 205, configured to calculate a distance between the worker and the mobile mechanical device according to a distance between the worker and the binocular camera, the absolute angle, and the relative angle.

Through the technical scheme, the distance between the calculated worker and the mobile mechanical equipment can be more accurate.

As shown in fig. 7, in one embodiment, preferably, the alarm module 103 includes:

a first alarm unit 701, configured to prompt in a first human-machine collision early warning prompting manner when an actual distance between the worker and the mechanical device is smaller than a preset distance threshold, where the first human-machine collision early warning prompting manner includes interface display and sound alarm;

and a second alarm unit 702, configured to prompt in a second human-machine collision early warning prompting manner when the actual distance between the worker and the mechanical device is greater than or equal to a preset distance threshold, where the second human-machine collision early warning prompting manner includes interface display.

Wherein the interface interaction may be a display provided on the mechanical device. And when the distance between the person and the mechanical equipment is greater than a preset distance threshold value, triggering interface interaction to enable the color of the display to be displayed in a wire frame in the display. When the distance between the human and the mechanical equipment is smaller than a preset distance threshold value, interface interaction and sound interaction are triggered simultaneously, the sound interaction gives out a buzzer sound to warn, and red is displayed in a display line frame. Specifically, as shown in fig. 8, the early warning interaction is divided into an interface interaction and a sound interaction. The interface interaction is directed to a driver seated within the machine cab. The driver can see the picture shot by the binocular camera in real time through the display. The method comprises the following steps that people in a picture are selected to be highlighted by a rectangular wire frame, and when the distance between the people in the picture and mechanical equipment is larger than a preset distance threshold value, the wire frame is green (small rectangles in the picture); when the distance between the person and the mechanical equipment in the picture is smaller than the preset distance threshold, the wire frame turns red (large rectangle in the figure), and voice interaction aims at workers on site and warns for sending out buzzing sound when the distance between the person and the mechanical equipment is smaller than the preset distance threshold.

As shown in fig. 9, in one embodiment, preferably, the system further comprises:

a threshold determining module 901, configured to determine the preset distance threshold by using a delphi method.

The method of the Delphi method comprises the following steps: the problem of the proposed survey is the preset distance threshold of the construction machinery, and the design questionnaire comprises common construction machinery such as an excavator, a mobile crane and the like. Questionnaires can be sent to the panel via the client, where the panel's personnel can include first party safety chief, second party safety chief, third party proctoring and workers, and related professional experts. And performing a first round of questionnaire survey on the members of the expert group, so that the experts write down the early warning threshold value in the client and clearly write the reason obtained by the threshold value. The opinions of each expert are collected for the first time through the server, listed into a chart, compared and distributed to each expert, so that the experts can compare different opinions of the experts with other people and modify the opinions and judgment of the experts. The opinions of the experts may also be collated or reviewed by other experts of higher identity and then re-distributed to the experts so that they revise their opinions after reference. Collecting the modification opinions of all experts, collecting the modification opinions, and distributing the modification opinions to each expert again so as to carry out second modification. Collecting opinions and feeding back information for experts one by one is a main link of the Delphi method. Collecting opinions and information feedback typically takes three or four rounds. When feedback is given to the experts, only various opinions are given, but the specific names of the experts who issue the various opinions are not described. This process is repeated until each expert no longer changes his opinion. From this, a reasonable preset distance threshold for each device is derived.

As shown in fig. 10, in one embodiment, preferably, the system further comprises:

equipment three proofings module 1000, processing module 102 can install to inside equipment three proofings module 1000 for avoid processing module 102 to receive the influence of rainwater, dust and vibration, the guarantee normally works.

Particularly, the working environment of a construction site is complex, a large amount of dust can fly upwards in the construction process, and in order to guarantee good construction environment and environmental requirements, a constructor can reduce the dust in the air in a water spraying mode. The embodiment of the invention depends on an industrial personal computer which takes a CPU and a GPU as cores as carriers for realizing the algorithm, and the chassis of the industrial personal computer is required to meet the requirements of dustproof and waterproof of the computing cores. Meanwhile, the equipment is arranged on construction machinery equipment such as hoisting, transportation, excavation and the like, and the equipment can bring high-frequency vibration during operation, so the calculation core also has the requirement of shock resistance. And finally, the heat dissipation problem of the equipment is also considered due to the high power of the CPU and the GPU. Therefore, as shown in fig. 11, a device three-proofing device is designed to solve the existing problems.

Further, in one embodiment of the present invention, the equipment tri-proof module includes a waterproof and dustproof unit and a shockproof unit. The shockproof unit is characterized in that a steel wire shock absorber is arranged between an inner box and a fixed support in the processing module to buffer three-dimensional shock, and a spring shock absorption structure is arranged between a main board and the inner box in the processing module to eliminate residual shock.

Specifically, first, in terms of the waterproof and dustproof problems, a plurality of structural designs are adopted to avoid. The outer cover case is adopted for preventing water and dust, a special downward air outlet is reserved on the side face of the case, rainwater is prevented from flowing backwards, and enough space is reserved between the inner case and the outer cover. And a dustproof filter screen is additionally arranged at an air outlet below the outer cover. The ventilation opening of the inner box is also provided with a dustproof filter screen. Magnitude filtering, and effective dust prevention.

As shown in fig. 12, next to the shock-proof problem, the embodiment of the present invention employs two-stage shock-proof measures. The large structure of the case is divided into an inner case and an outer cover. The inner box and the fixed bracket are connected and fixed by a steel wire shock absorber. The steel wire shock absorber can effectively buffer three-dimensional vibration. The steel wire shock absorber can absorb the vibration of the upper, lower, left, right, front and back through the deformation of the spiral line structure through the wound spiral steel wire rope, and has stable structure, good elasticity and toughness. The second level of shock absorption measures are installed between the main plate and the inner box. The mainboard is fixed and has used spring shock-absorbing structure at the inner box, further eliminates remaining vibrations, guarantees industrial computer normal operating.

And finally, aiming at the heat dissipation problem, the air cooling circulation of the case is optimally designed. 5 fans are arranged in the inner box, wherein one fan is positioned at the upper part of the CPU, and the heat dissipation of the processor is ensured. In addition, a fan is arranged on the GPU to ensure the heat dissipation of the graphics card. The other three steps are positioned at one side of the inner box, and the whole case and the outer box are ventilated and circulated. In addition, the situation that high temperature and sunlight are always directly irradiated in a construction site is considered, the heat dissipation outer cover can effectively isolate the sunlight to directly irradiate the inner box to cause temperature rise, and heat dissipation performance is improved.

The embodiment of the invention is applied to the construction process of actual mechanical equipment, and specifically comprises the following steps:

(1) select early warning area erection equipment

And selecting a range needing early warning according to the operation content of the construction machinery equipment, and installing a camera at a proper position of the equipment. Taking a mobile crane as an example, the early warning range of the current activity is the tail side of the vehicle, the specific early warning range is shown in fig. 13, and an industrial personal computer case containing computing power such as a CPU and a GPU is installed at a proper position, preferably in a cab. The vehicle-mounted storage battery is used as a power supply, and the power is supplied to the industrial personal computer uninterruptedly under the condition that the crane is started. The industrial personal computer is connected with the camera through optical fibers, so that data transmission delay is reduced. The display is fixed at a proper position selected in the cab and is connected with the industrial personal computer.

(2) Setting alarm threshold

And determining a threshold value for selecting the pre-warning equipment by a Delphi method (combination of questionnaire survey, semi-structured method and the like). The alarm thresholds may be different for different camera coverage areas. The last determined alarm threshold is entered in the system.

(3) Risk situation alarm

As shown in fig. 14, during the operation of the system, each worker who is present in the camera within 10 meters of the mechanical device and greater than the threshold value will automatically frame out with a green frame.

As shown in fig. 15, when the worker approaches the construction machine by a distance less than a threshold value, the worker is framed in a red frame in the display and beeps.

Through field tests, the alarm delay is less than 1s, the alarm distance error in 5 meters is less than 10cm, the coverage area of each camera is 72 degrees, and the industrial personal computer can be connected with a plurality of binocular cameras. The camera processes more than 10 input pictures per second, and the early warning requirement is met.

According to the construction site human-computer collision early warning system based on the deep learning vision technology provided by the embodiment of the invention, an industrial personal computer meeting the calculation requirement is designed based on a mature deep learning vision algorithm, and a binocular camera and a buzzing alarm device are matched, so that the construction human-computer collision early warning system based on the deep learning vision technology is realized. The invention takes a mobile crane as a research object, is intended to be installed on mobile mechanical equipment of a construction site represented by the mobile crane, takes video stream data acquired by a binocular camera as input, identifies people in a video and the distance between the people and a camera through an algorithm, and alarms the risk condition that the distance between the people and the mechanical equipment exceeds a threshold value, thereby effectively eliminating potential hidden dangers during construction, avoiding dependence on workers, saving time and easily coping with real-time and sudden conditions.

Next, a man-machine collision warning method for a construction site according to an embodiment of the present invention will be described with reference to the drawings.

As shown in fig. 16, the method for warning human-computer collision in construction site includes steps S1601-S1603:

in step S1601, image data of a construction site is acquired by a plurality of binocular cameras provided on a mobile mechanical apparatus of the construction site;

in step S1602, target image data including a worker in an image is selected from the image data, and an actual distance between the worker and the mechanical device is determined according to each target image data;

in step S1603, a corresponding human-computer collision warning prompting mode is determined to prompt according to the actual distance between the worker and the mechanical equipment.

As shown in fig. 17, in one embodiment, the step S1602 includes steps S1701 to S1705:

in step S1701, target image data including a worker in an image is recognized from the image data by a deep learning algorithm, and coordinates of pixel points corresponding to a human body range of the worker in each target image data are determined, the deep learning algorithm including: CNN algorithm, ResNet algorithm, and Unet algorithm;

in step S1702, a depth distance from an object corresponding to each pixel point in the target image data to the camera is calculated through a binocular vision algorithm;

in step S1703, a center point coordinate of each human body range is determined, 5 × 5 target pixel points are taken from the human body range at equal intervals by taking the center point coordinate as a reference coordinate, a frequency distribution histogram is generated according to the distance between the range difference of the depth distances corresponding to all the target pixel points and the quotient of 5, and the average value of the depth distances corresponding to all the target pixel points contained in the histogram with the highest frequency is determined as the distance between the worker and the binocular camera;

in step S1704, an absolute angle formed by each binocular camera and the due north direction and a relative angle between each binocular camera and a corresponding worker at the construction site are obtained;

in step S1705, a distance between the worker and the mobile mechanical apparatus is calculated based on the distance between the worker and the binocular camera, and the absolute angle and the relative angle.

As shown in fig. 18, in one embodiment, preferably, the step S1603 includes steps S1801-S1802:

in step S1801, when the actual distance between the worker and the mechanical device is smaller than a preset distance threshold, performing a prompt in a first human-machine collision early warning prompting manner, where the first human-machine collision early warning prompting manner includes interface display and sound alarm;

in step S1802, when the actual distance between the worker and the mechanical device is greater than or equal to the preset distance threshold, a second human-machine collision warning prompting mode is used for prompting, where the second human-machine collision warning prompting mode includes interface display.

In one embodiment, preferably, the method further comprises:

and determining a preset distance threshold value by a Delphi method.

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

The processor is further configured to:

And determining the preset distance threshold value by a Delphi method.

A non-transitory computer-readable storage medium having instructions therein, which when executed by a processor of a mobile terminal, enable the mobile terminal to perform a human-machine collision warning method for a construction site, the method comprising:

In one embodiment, preferably, the method further comprises:

and determining the preset distance threshold value by a Delphi method.

It is further understood that the use of "a plurality" in this disclosure means two or more, as other terms are analogous. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. The singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It will be further understood that the terms "first," "second," and the like are used to describe various information and that such information should not be limited by these terms. These terms are only used to distinguish one type of information from another and do not denote a particular order or importance. Indeed, the terms "first," "second," and the like are fully interchangeable. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure.

It is further to be understood that while operations are depicted in the drawings in a particular order, this is not to be understood as requiring that such operations be performed in the particular order shown or in serial order, or that all illustrated operations be performed, to achieve desirable results. In certain environments, multitasking and parallel processing may be advantageous.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A man-machine collision early warning system of job site, its characterized in that includes:

the alarm module is used for determining a corresponding human-computer collision early warning prompting mode for prompting according to the actual distance between the worker and the mechanical equipment;

the processing module comprises:

2. The human-computer collision warning system for a construction site according to claim 1, wherein the deep learning algorithm comprises: CNN algorithm, ResNet algorithm, and Unet algorithm.

3. The human-computer collision warning system of a construction site according to claim 1, wherein the alarm module comprises:

4. The human-computer collision warning system for a construction site according to claim 3, further comprising:

5. A man-machine collision early warning method for a construction site is characterized by comprising the following steps:

determining a corresponding human-computer collision early warning prompting mode for prompting according to the actual distance between the worker and the mechanical equipment;

the selecting target image data of workers contained in the image from the image data, and determining the actual distance between the workers and the mechanical equipment according to each target image data comprises the following steps:

6. The human-computer collision early warning method for the construction site according to claim 5, wherein the step of determining a corresponding human-computer collision early warning prompting mode for prompting according to the actual distance between the worker and the mechanical equipment comprises the following steps:

7. The human-computer collision warning method for the construction site according to claim 6, further comprising:

and determining the preset distance threshold value by a Delphi method.

8. A man-machine collision early warning system of job site, its characterized in that includes:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

selecting target image data containing workers in the image from the image data, and determining the actual distance between the workers and the mechanical equipment according to each target image data, wherein the target image data comprises: