CN111105047B

CN111105047B - Operation and maintenance monitoring method and device, electronic equipment and storage medium

Info

Publication number: CN111105047B
Application number: CN201911276146.8A
Authority: CN
Inventors: 吴宁海; 何阳
Original assignee: Shandong Hairui Smart Data Technology Co ltd
Current assignee: Shandong Hairui Smart Data Technology Co ltd
Priority date: 2019-12-12
Filing date: 2019-12-12
Publication date: 2024-04-16
Anticipated expiration: 2039-12-12
Also published as: CN111105047A

Abstract

The embodiment of the disclosure discloses an operation and maintenance monitoring method, an operation and maintenance monitoring device, electronic equipment and a storage medium, and relates to the technical field of automatic operation and maintenance monitoring. Wherein the method comprises the following steps: real-time monitoring and processing are carried out on the operation state of the standardized industrial place and/or industrial equipment through a machine vision system; when the machine vision system finds that abnormal conditions occur in the monitoring video/image, alarming and/or generating operation and maintenance instructions; automatically monitoring an operation and maintenance flow for executing the operation and maintenance instruction through a machine vision system; after the alarm is released and/or the operation and maintenance are finished, the machine vision system and the sensor are used for carrying out cross verification on the operation state of the standardized industrial place and/or the industrial equipment; by adopting the method, the machine vision system replaces personnel for inspection, the labor cost is greatly reduced, and the safe and effective operation of the standardized industrial site is ensured by the omnibearing monitoring of the standardized industrial site and/or industrial equipment.

Description

Operation and maintenance monitoring method and device, electronic equipment and storage medium

Technical Field

The disclosure relates to the technical field of automatic operation and maintenance monitoring, in particular to an operation and maintenance monitoring method, an operation and maintenance monitoring device, electronic equipment and a storage medium.

Background

The traditional industrial place or industrial equipment needs a large amount of personnel for inspection and maintenance, the personnel cost is high, and inspection dead angles or conditions of personnel negative idle work and missing error detection are easy to occur; and once an emergency happens, the emergency is difficult to quickly detect and treat, a certain potential safety hazard is easily caused, and even a certain economic loss is caused.

Some automatic monitoring methods based on machine vision are proposed, and intrusion of external personnel and safety equipment of maintenance personnel can be automatically identified through the machine vision method. However, the operation and maintenance of industrial sites or equipment is the result of a series of interactions between personnel and equipment, and no method is known in the art that allows automated monitoring of the entire process. Therefore, there is a need for a standardized operation and maintenance monitoring method for industrial sites or industrial equipment.

Disclosure of Invention

Aiming at the technical problems in the prior art, the embodiment of the disclosure provides an operation and maintenance monitoring method, an operation and maintenance monitoring device, electronic equipment and a storage medium, so as to solve the problems that the personnel cost is relatively high, and inspection dead angles are easy to exist or personnel are out of work negatively, and leakage detection occurs in the prior art; and once an emergency occurs, the emergency is difficult to quickly detect and treat, and a certain economic loss is caused.

A first aspect of an embodiment of the present disclosure provides an operation and maintenance monitoring method, including:

real-time monitoring and processing are carried out on the operation state of the standardized industrial place and/or industrial equipment through a machine vision system;

when the machine vision system finds that abnormal conditions occur in the monitoring video/image, alarming and/or generating operation and maintenance instructions;

automatically monitoring an operation and maintenance flow for executing the operation and maintenance instruction through the machine vision system;

and after the alarm is released and/or the operation and maintenance are finished, the machine vision system and the sensor are used for carrying out cross verification on the operation state of the standardized industrial place and/or the industrial equipment.

In some embodiments, the abnormal situation includes an abnormal situation of a person in the surveillance video/image and/or an abnormal situation of a target object in the surveillance video/image.

In some embodiments, the method further comprises: establishing a database of standardized industrial sites or industrial equipment; and forming an operation and maintenance record by the abnormal condition, the alarm information, the operation and maintenance flow and the generated operation and maintenance result, and updating the operation and maintenance record into a database.

In some embodiments, the method further comprises: if the abnormal situation is eliminated, the operation and maintenance verification database in the database is linked for verification, and the operation and maintenance knowledge base in the database is linked for classified storage; and if the abnormal condition is not eliminated, continuing to send the operation and maintenance task until the operation and maintenance is finished.

In some embodiments, the machine vision system may monitor the standardized industrial site and/or the industrial equipment operating state in real-time using a trained neural network.

In some embodiments, the operation and maintenance flow specifically includes: and identifying the running state of the industrial equipment, the equipment state of the operation and maintenance personnel, the running path of the operation and maintenance personnel, the maintenance operation standard of the operation and maintenance personnel and the corresponding state of the operation and maintenance personnel and the fault equipment through the machine vision system.

In some embodiments, the method further comprises automatically monitoring the alarm release result and/or the operation and maintenance result after the alarm release and/or the operation and maintenance are finished.

A second aspect of an embodiment of the present disclosure provides an operation and maintenance monitoring device, including:

the real-time monitoring module is used for carrying out real-time monitoring treatment on the operation state of the standardized industrial place and/or industrial equipment through the machine vision system;

the abnormal condition processing module is used for alarming and/or generating operation and maintenance instructions when the machine vision system finds that the abnormal condition occurs in the monitoring video/image;

the operation and maintenance flow monitoring module is used for automatically monitoring the operation and maintenance flow for executing the operation and maintenance instruction through the machine vision system;

And the verification module is used for carrying out cross verification on the operation state of the standardized industrial place and/or the industrial equipment by using the machine vision system and the sensor after the alarm is released and/or the operation and maintenance are finished.

A third aspect of the disclosed embodiments provides an electronic device, comprising:

a memory and one or more processors;

wherein the memory is communicatively coupled to the one or more processors, and instructions executable by the one or more processors are stored in the memory, which when executed by the one or more processors, are operable to implement the methods as described in the previous embodiments.

A fourth aspect of the disclosed embodiments provides a computer-readable storage medium having stored thereon computer-executable instructions which, when executed by a computing device, are operable to implement the methods of the previous embodiments.

A fifth aspect of the disclosed embodiments provides a computer program product comprising a computer program stored on a computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, are operable to implement a method as described in the previous embodiments.

The invention has the beneficial effects that: the inspection is carried out by replacing personnel by a machine vision system, so that the labor cost is greatly reduced, and the safe and effective operation of the standardized industrial place is ensured by the omnibearing monitoring of the standardized industrial place and/or industrial equipment; meanwhile, the operation and maintenance flow can be automatically monitored; and the combination of the machine vision system and the sensor is used for cross verification, so that the monitoring of operation and maintenance at a higher level is realized, and the normal operation state of the standardized industrial place and/or industrial equipment is ensured.

Drawings

The features and advantages of the present disclosure will be more clearly understood by reference to the accompanying drawings, which are schematic and should not be construed as limiting the disclosure in any way, in which:

FIG. 1 is a flow chart of an operation and maintenance monitoring method according to some embodiments of the present disclosure;

FIG. 2 is a block diagram of an operation and maintenance monitoring module according to some embodiments of the present disclosure;

FIG. 3 is a schematic diagram of an operation and maintenance monitoring flow of a power distribution room according to some embodiments of the present disclosure;

FIG. 4 is a schematic diagram of a power distribution room alarm operation and maintenance process shown in accordance with some embodiments of the present disclosure;

FIG. 5 is a schematic diagram of an operation and maintenance monitoring flow for a gas station according to some embodiments of the present disclosure;

FIG. 6 is a schematic illustration of a gas station alarm operation and maintenance flow shown in accordance with some embodiments of the present disclosure;

FIG. 7 is a schematic diagram of an operation and maintenance monitoring flow of an oil recovery machine according to some embodiments of the present disclosure;

FIG. 8 is a schematic diagram of an oil extractor alarm operation and maintenance process shown in accordance with some embodiments of the present disclosure;

FIG. 9 is a schematic diagram of an operation and maintenance monitoring flow of a substation, shown according to some embodiments of the present disclosure;

FIG. 10 is a schematic diagram of a substation alarm operation and maintenance process shown in accordance with some embodiments of the present disclosure;

FIG. 11 is a schematic diagram of an operation and maintenance monitoring flow of a power tunnel, shown in accordance with some embodiments of the present disclosure;

FIG. 12 is a schematic diagram of a power tunnel alert operation flow shown in accordance with some embodiments of the present disclosure;

FIG. 13 is a schematic illustration of an operation and maintenance monitoring flow of an automobile charging station according to some embodiments of the present disclosure;

FIG. 14 is a schematic illustration of an automobile charging station alert operation flow shown in accordance with some embodiments of the present disclosure;

FIG. 15 is a schematic diagram of an operation and maintenance monitoring flow of a data room according to some embodiments of the present disclosure;

FIG. 16 is a schematic diagram of a data room alarm operation and maintenance process shown in accordance with some embodiments of the present disclosure;

Fig. 17 is a schematic diagram of an electronic device, according to some embodiments of the present application.

Detailed Description

In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant disclosure. However, it will be apparent to one of ordinary skill in the art that the present disclosure may be practiced without these specific details. It should be appreciated that the use of "system," "apparatus," "unit," and/or "module" terms in this disclosure is one method for distinguishing between different parts, elements, portions, or components at different levels in a sequential arrangement. However, these terms may be replaced with other expressions if the other expressions can achieve the same purpose.

It will be understood that when a device, unit, or module is referred to as being "on," "connected to," or "coupled to" another device, unit, or module, it can be directly on, connected to, or coupled to, or in communication with the other device, unit, or module, or intervening devices, units, or modules may be present unless the context clearly indicates an exception. For example, the term "and/or" as used in this disclosure includes any and all combinations of one or more of the associated listed items.

The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to limit the scope of the present disclosure. As used in the specification and the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" are intended to cover only those features, integers, steps, operations, elements, and/or components that are explicitly identified, but do not constitute an exclusive list, as other features, integers, steps, operations, elements, and/or components may be included.

These and other features and characteristics of the present disclosure, as well as the methods of operation, functions of the related elements of structure, combinations of parts and economies of manufacture, may be better understood with reference to the following description and the accompanying drawings, all of which form a part of this specification. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the disclosure. It will be understood that the figures are not drawn to scale.

In the invention, a standardized industrial site daily operation and maintenance system is established, which is applicable to most industrial sites, such as: substation, power tunnel, power channel, electricity distribution room, gas station, charging station, oil extraction machine etc..

As shown in fig. 1, an embodiment of the present disclosure provides an operation and maintenance monitoring method, which specifically includes:

s101, carrying out real-time monitoring treatment on the operation state of a standardized industrial place and/or industrial equipment through a machine vision system;

s102, when the machine vision system finds that an abnormal condition occurs in the monitoring video/image, alarming and/or generating an operation and maintenance instruction are carried out;

s103, automatically monitoring an operation and maintenance flow for executing the operation and maintenance instruction through the machine vision system;

and S104, after the alarm is released and/or the operation and maintenance are finished, cross-verifying the operation state of the standardized industrial place and/or the industrial equipment by using the machine vision system and the sensor.

Further, abnormal conditions of personnel in the monitoring video/image include illegal invasion of personnel, nonstandard dressing of personnel, personnel travelling paths, illegal behaviors of personnel entering and the like;

the abnormal conditions of the target object in the monitoring video/image mainly comprise the faults of industrial equipment; for example, oil extraction machine leaks oil, oiling machine data is abnormal, and a charging pile display screen is broken;

In some embodiments, if an abnormal condition of a target object in the surveillance video/image occurs, manual maintenance and repair is required. Optionally, the operation maintenance further comprises periodic operation maintenance requirements.

Furthermore, the abnormal situation monitored by the machine vision also comprises the abnormal situation of the personnel and the target object in the monitoring video/image at the same time, and the abnormal situation mainly comprises the following steps: the corresponding states of personnel and industrial equipment to be maintained are not matched, and abnormal conditions such as non-correspondence and the like exist between personnel maintenance equipment and fault equipment; meanwhile, the machine vision can also identify the equipment state, personnel traveling path, personnel maintenance operation standard and personnel maintenance and fault equipment corresponding state in the area.

Alternatively, the status and state transitions of the industrial equipment may be identified by the machine vision system and compared to a predefined maintenance flow to see if there is a match.

In some embodiments, the discovered abnormal situation is uploaded to a server, and a corresponding operation and maintenance instruction is generated according to the abnormal situation, so as to inform an operation and maintenance person to start an operation and maintenance process.

In some embodiments, the method further comprises:

establishing a database of standardized industrial sites or industrial equipment;

And forming an operation and maintenance record by the abnormal condition, the alarm information, the operation and maintenance flow and the operation and maintenance result, and updating the operation and maintenance record into a database.

Specifically, the database comprises one or more of a personnel database, an alarm type database, an operation and maintenance flow database, an operation and maintenance verification database and an operation and maintenance knowledge database.

Further, when an abnormal condition of a target object in the monitoring video/image occurs, updating the alarm type data to an alarm type database;

further, after operation and maintenance are finished, the machine vision system and the sensor are used for cross verification, if abnormal conditions are eliminated, the linkage operation and maintenance verification database is used for verification, and meanwhile the linkage operation and maintenance knowledge base is used for classified storage; if the abnormal situation is not eliminated, continuing to send down the operation and maintenance task until the operation and maintenance is finished.

In some embodiments, the present disclosure also runs a convolutional neural network algorithm based on the frontmost TensorFlow framework, which performs an extraction process for each frame of picture in the video/image.

Specifically, the present disclosure uses a region pro-post (candidate region) method in a neural network to perform target extraction on each frame of image, so as to achieve real-time monitoring of the target object by extracting features of the target object, and in the processing process, the real-time position and name of the target object can be obtained by an arrow mechanism and a softmax classification method, respectively.

In some embodiments, the monitoring video/image is monitored in real time by a machine vision system, which specifically includes:

real-time monitoring of various faults and anomalies in standardized industrial sites and/or industrial equipment is achieved through a machine vision system.

More specifically, the machine vision system can detect and process a target object by using a trained neural network structure, and can realize novel functions such as illegal personnel intrusion judgment, standardized dressing detection, violation detection in an operation and maintenance process, personnel track tracking, voice alarm, information transmission alarm and the like.

As shown in fig. 2, the embodiment of the present disclosure further provides an operation and maintenance monitoring device 200, which specifically includes:

the real-time monitoring module 201 is configured to perform real-time monitoring processing on the operation state of the standardized industrial location and/or the industrial equipment through the machine vision system;

the abnormal situation processing module 202 is configured to alarm and/or generate an operation and maintenance instruction when the machine vision system finds that an abnormal situation occurs in the monitoring video/image;

the operation and maintenance flow monitoring module 203 is configured to automatically monitor the operation and maintenance flow through the machine vision system in an operation and maintenance process;

And the verification module 204 is used for performing cross verification on the operation state of the standardized industrial place and/or industrial equipment by using the machine vision system and the sensor after the alarm is released and/or the operation is finished.

In some embodiments, the management and control flow of the standardized industrial site in daily operation is generally divided into: daily monitoring, fault discovery, operation and maintenance notification, process monitoring, result monitoring and authentication feedback.

In some embodiments, taking operation and maintenance monitoring of a power distribution room as an example, the operation and maintenance monitoring method provided by the invention is described, an image/video of equipment of the power distribution room is obtained through a machine vision system, and an operation state of the equipment is identified, and the state can be cross-verified with states identified by other sensors of the internet of things. When a fault exists in the identification system, such as smoke alarm or tripping, an image of the distribution room is obtained, and the operation and maintenance process is identified and monitored. At this time, the state of the personnel entering the distribution room is identified, the result can be a maintainer or a non-maintainer, if the personnel is not maintained, the abnormal invasion is considered, and the identification result is alarmed. And when the identification result is that the maintenance personnel are identified, identifying the safety equipment, the travelling path and the maintenance equipment of the maintenance personnel. When the maintenance equipment does not correspond to the fault equipment, a maintenance flow error alarm is provided. Finally, after the operation and maintenance are identified, for example, after the operation and maintenance personnel leave the industrial place, the system identifies the state of the equipment, and when the state of the equipment is different from the expected state after the operation and maintenance, an alarm is provided; FIG. 3 shows the operation and maintenance monitoring flow of the distribution room, wherein the distribution room is monitored and processed in real time through a machine vision system in daily life, and meanwhile, the cross verification is carried out by combining with a sensor; if the machine vision system finds that the personnel are abnormal, personnel are identified by acquiring personnel database data, the personnel abnormal types are automatically defined, and classified into abnormal 1, 2, 3 … N and the like, for example, abnormal 1: illegal personnel intrusion; anomaly 2: non-standardized wear; anomaly 3: violating rules in the operation and maintenance process; anomaly 4: the operation and maintenance task is not matched with the employee type; anomaly N: other anomalies, etc.; meanwhile, when an alarm occurs, abnormal type pushing is automatically carried out, and a monitoring center is notified. Carrying out local alarm and long-term transmission through an alarm; in addition, the machine vision system is used for monitoring the alarm result, for example, staff can correct the rule violations and the abnormality until the staff returns to normal after the alarm gives an alarm; preferably, the operation and maintenance records are stored in a one-to-one correspondence to the operation and maintenance database.

Further, if the machine vision system finds that the equipment class has abnormal conditions, the machine vision system alarms and automatically defines fault types, and classifies the fault types into faults 1, 2, 3 … N and the like, for example, fault 1: the A-type high-voltage cabinet monitors abnormal data, changes in appearance and changes in indicator lights; monitoring data abnormality, appearance change and indicator lamp change of the No. A low-voltage cabinet by faults 2; fault 3: the standard arrangement of the distribution room is not in place, and the necessary articles are lost; fault 4: the cabinet door A is arranged in the power distribution room and is not closed; fault N: other faults, etc.; after uploading the fault abnormal information to a server, generating an operation and maintenance instruction to inform operation and maintenance personnel to go to clear the fault; in the operation and maintenance process, obtaining image data through a machine vision system, and automatically monitoring the process identification of fault types and the whole operation and maintenance process aiming at operation and maintenance personnel; for example, fault 1: personnel arrive at the A-type high-voltage cabinet to be positioned and monitored, and a maintenance flow is monitored; fault 2: personnel arrive at the A-type low-voltage cabinet to be positioned and monitored, and the maintenance flow is monitored; fault 3: whether personnel carry the missing standard appliance to a designated position for placement; fault 4: whether the personnel arrive at the cabinet A or not, and whether the personnel cross other areas; fault N: identifying other fault processes; meanwhile, the machine vision system is used for monitoring operation and maintenance results, for example, result 1: the A-type high-voltage cabinet monitors the data, the appearance and the indicator lights and returns to normal; results 2: the monitoring data, the appearance and the indicator lights of the No. A low-voltage cabinet are recovered to be normal; results 3: the standard distribution room arrangement is normal; results 4: all cabinet doors of the cabinet A in the power distribution room are normally closed; results N: other normal conditions;

Furthermore, the alarm types corresponding to the abnormality or the fault of the equipment abnormality and the personnel abnormality are classified, each alarm type has corresponding flow control identification, and when the alarm is triggered, the management and the monitoring are automatically switched, so that a plurality of alarms can be simultaneously carried out; and meanwhile, after the operation and maintenance flow corresponding to each alarm is finished, cross verification is carried out on the alarm and the sensor through a convolutional neural network algorithm, and whether the alarm is released is judged. If the alarm is released, the operation and maintenance results are stored in a classified mode, and if the process is not continued to be monitored, the operation and maintenance tasks are distributed until the alarm information is released.

And further, after the alarm is released and/or the operation and maintenance are finished, the machine vision system and the sensor are used for carrying out cross verification and feedback on the operation state of the standardized industrial place and/or the industrial equipment.

In some embodiments, as shown in fig. 4, a schematic diagram of a distribution room alarm operation and maintenance process is correspondingly presented. When the daily monitoring finds that personnel identification abnormality exists, the alarm types are classified according to the abnormality details, for example, personnel alarm: illegal personnel enter, are not worn in a standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the employee types; then starting an operation and maintenance process, and simultaneously, automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: personnel intrusion recognition, no safety helmet, no work clothes wearing, smoking and the like; and finally, classifying and storing the alarm processing result, for example, correcting the rule violations by personnel after the alarm processing result is the alarm, and returning to normal, and forming an operation and maintenance record by the alarm processing result and the corresponding alarm record to store the operation and maintenance record into an operation and maintenance database. When the daily monitoring finds that the equipment is abnormal, firstly, the alarm types are classified according to the details of the abnormality, for example, the fault 1: the A-type high-voltage cabinet monitors abnormal data, changes in appearance and changes in indicator lights; then, starting an operation and maintenance process, and simultaneously, automatically identifying and monitoring the operation and maintenance process, for example, a process identification 1: personnel arrive at the A-type high-voltage cabinet to be positioned and monitored, and a maintenance flow is monitored; finally, the alarm processing results are classified and stored, for example, the alarm processing result is a result 1: and (3) the monitoring data, the appearance and the indicator lights of the No. A high-voltage cabinet are recovered to be normal, and an operation and maintenance record is formed by the alarm processing result and the corresponding alarm record and is stored into an operation and maintenance database.

In some embodiments, taking operation and maintenance monitoring of a gas station as an example, the operation and maintenance monitoring method provided by the invention is described, when an abnormality occurs in a site, for example, smoke/fire-fighting equipment is absent, and the like, alarm information is sent to inform staff of processing. At the moment, the system recognizes the state of the personnel entering the equipment area of the gas station, if the personnel is recognized as non-maintenance personnel by the system, the abnormal invasion is considered, and the recognition result is alarmed; if identified by the system as a serviceman, the serviceman's safety wear, travel path and maintenance equipment are identified. When the maintenance equipment does not correspond to the fault equipment, a maintenance flow error alarm is provided. After the system identifies operation and maintenance is finished, for example, after the operator leaves the industrial place, the system carries out sensing cross verification on the maintenance result and the Internet of things through the deep neural network and records the result. An alarm is provided when the device status differs from the expected status after operation and maintenance. As shown in particular in fig. 5; FIG. 5 shows the operation and maintenance monitoring flow of a gas station, wherein the gas station is monitored and processed in real time through a machine vision system in daily life, and meanwhile, the gas station is cross-validated by combining with a sensor; if the machine vision system finds that the personnel are abnormal, personnel are identified by acquiring personnel database data, the personnel abnormal types are automatically defined, and classified into abnormal 1, 2, 3 … N and the like, for example, abnormal 1: illegal personnel intrusion; anomaly 2: smoking in the personnel gas station; anomaly 3: violating rules in the operation and maintenance process; anomaly 4: the operation and maintenance task is not matched with the employee type; anomaly N: other anomalies, etc.; meanwhile, when an alarm occurs, abnormal type pushing is automatically carried out, and a monitoring center is notified. Carrying out local alarm and long-term transmission through an alarm; in addition, the machine vision system is used for monitoring the alarm result, for example, staff can correct the rule violations and the abnormality until the staff returns to normal after the alarm gives an alarm;

Correspondingly, if the machine vision system finds that the equipment class has abnormal conditions, alarming and automatically defining fault types, and classifying the fault types into faults 1, 2, 3 … N and the like, for example, fault 1: the A-type oiling machine has abnormal data, shape change and indicator lamp change; the fault 2:B oiling machine oiling gun is not placed at the designated position; fault 3: the standard arrangement of the gas station is not in place, and the necessary articles are lost; fault 4: the cabinet door of the No. A fueling cabinet is not closed in the fueling station; fault N: other faults, etc.; after uploading the fault abnormal information to a server, generating an operation and maintenance instruction to inform operation and maintenance personnel to go to clear the fault; in the operation and maintenance process, obtaining image data through a machine vision system, and automatically monitoring the process identification of fault types and the whole operation and maintenance process aiming at operation and maintenance personnel; for example, fault 1: personnel arrive at the A-type oiling machine to locate and monitor, and the maintenance flow is monitored; fault 2: personnel arrive at the B-number oiling machine oiling gun to locate and monitor, the maintenance flow is monitored; fault 3: whether personnel carry the missing standard appliance to a designated position for placement; fault 4: whether the personnel arrive at the A-type oiling machine or not and whether the personnel cross other areas; fault N: identifying other fault processes; meanwhile, the machine vision system is used for monitoring operation and maintenance results, for example, result 1: the A-type high-voltage cabinet monitors the data, the appearance and the indicator lights and returns to normal; results 2: the oiling gun of the No. B oiling machine returns to normal; results 3: the arrangement of the fire-fighting equipment of the gas station is normal; results 4: all cabinet doors of the No. A cabinet in the gas station are normally closed; results N: other normal conditions;

Preferably, the operation and maintenance records are stored in a one-to-one correspondence to the operation and maintenance database.

Further, the alarm types corresponding to the abnormality or the fault occurring in the equipment abnormality and the personnel abnormality are classified, each alarm type has corresponding flow management and control identification, and when the alarm is triggered, the management and the monitoring are automatically switched, so that a plurality of alarms can be synchronously performed; and meanwhile, after the operation and maintenance flow corresponding to each alarm is finished, cross verification is carried out on the alarm and the sensor through a convolutional neural network algorithm, and whether the alarm is released is judged. If the alarm is released, the operation and maintenance results are stored in a classified mode, and if the process is not continued to be monitored, the operation and maintenance tasks are distributed until the alarm information is released.

Furthermore, after the alarm is released and/or the operation and maintenance is finished, the machine vision system can be used for carrying out cross verification and feedback on the operation state of the standardized industrial place and/or the industrial equipment with other sensors of the Internet of things.

In some embodiments, as shown in fig. 6, a schematic diagram of a gas station alarm operation and maintenance procedure is given. When the daily monitoring finds that personnel identification abnormality exists, the alarm types are classified according to the abnormality details, for example, personnel alarm: illegal personnel enter, are not worn in a standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the employee types; then starting an operation and maintenance process, and simultaneously, automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: personnel intrusion recognition, no safety helmet, no work clothes wearing, smoking and the like; finally, classifying and storing the alarm processing result, for example, correcting the rule violations by personnel after the alarm processing result is the alarm, and returning to normal, and forming an operation and maintenance record by the alarm processing result and the corresponding alarm record to store the operation and maintenance record into an operation and maintenance database; when the daily monitoring finds that the equipment is abnormal, firstly, the alarm types are classified according to the details of the abnormality, for example, the fault 1: the A-type oiling machine has abnormal data, shape change and indicator lamp change; then, starting an operation and maintenance process, and simultaneously, automatically identifying and monitoring the operation and maintenance process, for example, a process identification 1: personnel arrive at the A-type oiling machine to locate and monitor, and the maintenance flow is monitored; finally, the alarm processing results are classified and stored, for example, the alarm processing result is a result 1: and (3) the monitoring data, the appearance and the indicator lights of the A-number oiling machine are recovered to be normal, and an operation and maintenance record is formed by the alarm processing result and the corresponding alarm record and is stored into an operation and maintenance database.

In some embodiments, taking operation and maintenance monitoring of an oil extraction machine as an example, the operation and maintenance monitoring method provided by the invention is described, cameras are additionally arranged at each key monitoring system position of the oil extraction machine equipment, the system obtains environment images of the oil extraction machine and operation images of the equipment, and the operation states of the equipment are identified, and the states and the self-checking program of the equipment are mutually complemented and can be cross-verified. When the oil extraction machine is abnormal, the corresponding fault type can be alarmed according to the equipment fault cause, and related responsible personnel can be informed to the site for equipment maintenance; firstly, identity information detection is carried out on in-place personnel, the result can be maintenance personnel or non-maintenance personnel, when the in-place personnel are judged to be non-maintenance personnel, abnormal personnel are considered to invade, and the recognition result is alarmed; when the identification result is a maintenance person, carrying out next step of standard dressing and safety equipment detection, and monitoring a travelling path and a maintenance flow; when the maintenance items do not correspond to the fault information, a maintenance flow error alarm is provided to prompt maintenance personnel to correct in time. Finally, after the system identifies the operation and maintenance, for example, the system identifies the state of the equipment after identifying that the operation and maintenance personnel leave the industrial place, and when the state of the equipment is different from the expected state after the operation and maintenance, an alarm is provided. FIG. 7 shows in detail the operation and maintenance monitoring process of the oil extraction machine, wherein the oil extraction machine is monitored and treated in real time through a machine vision system in daily life, and meanwhile, the oil extraction machine is cross-validated by combining with a sensor; if the machine vision system finds that the personnel are abnormal, personnel are identified by acquiring personnel database data, the personnel abnormal types are automatically defined, and classified into abnormal 1, 2, 3 … N and the like, for example, abnormal 1: illegal personnel intrusion; anomaly 2: non-standardized wear; anomaly 3: violating rules in the operation and maintenance process; anomaly 4: the operation and maintenance task is not matched with the employee type; anomaly N: other anomalies, etc.; meanwhile, when an alarm occurs, abnormal type pushing is automatically carried out, and a monitoring center is notified. Carrying out local alarm and long-term transmission through an alarm; in addition, the machine vision system is used for monitoring the alarm result, for example, staff can correct the rule violations and the abnormality until the staff returns to normal after the alarm gives an alarm;

Correspondingly, if the machine vision system finds that the equipment class has abnormal conditions, alarming and automatically defining fault types, and classifying the fault types into faults 1, 2, 3 … N and the like, for example, fault 1: power system failure (motor, speed reducer); failure 2: drive train failure (belt, steel rope); fault 3: working system failure (master link, tail shaft, walking beam); fault 4: delivery system failure (oil line); fault N: other faults, etc.; after uploading the fault abnormal information to a server, generating an operation and maintenance instruction, and informing operation and maintenance personnel to go to clear the fault; in the operation and maintenance process, obtaining image data through a machine vision system, and automatically monitoring the process identification of fault types and the whole operation and maintenance process aiming at operation and maintenance personnel; for example, failure 1: monitoring personnel positions and paths, and monitoring a process of maintaining a speed reducer and a motor; fault 2: monitoring personnel positions and paths, and monitoring the process of replacing a belt and maintaining a steel rope; fault 3: monitoring personnel positions and paths, and monitoring the process of replacing a main connecting rod, a tail shaft and a walking beam; fault 4: monitoring personnel positions and paths, and monitoring the process of replacing oil pipeline sealing and maintaining pipelines; fault N: monitoring personnel positions and paths, and identifying other fault processes; meanwhile, the machine vision system is used for monitoring operation and maintenance results, for example, result 1: the output of the speed reducer is stable; results 2: the belt pulley is moderate in tensioning degree, free of jumping, neat in steel rope arrangement, free of scattered strands and broken strands; results 3: the deformation-free motion precision of the main connecting rod, the tail shaft and the walking beam reaches the standard; results 4: the oil pipeline has no oil leakage and vibration; results N: other normal conditions;

Furthermore, after the alarm is released and/or the operation and maintenance is finished, the machine vision system and the sensor are used for carrying out cross verification and feedback on the operation state of the standardized industrial place and/or the industrial equipment.

In some embodiments, as shown in fig. 8, a schematic diagram of an alarm operation and maintenance flow of an oil production machine is provided. When the daily monitoring finds that personnel identification abnormality exists, the alarm types are classified according to the abnormality details, for example, personnel alarm: illegal personnel enter, are not worn in a standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the employee types; then starting an operation and maintenance process, and simultaneously, automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: personnel intrusion recognition, no safety helmet, no work clothes wearing, smoking and the like; finally, classifying and storing the alarm processing result, for example, correcting the rule violations by personnel after the alarm processing result is the alarm, and returning to normal, and forming an operation and maintenance record by the alarm processing result and the corresponding alarm record to store the operation and maintenance record into an operation and maintenance database; when the daily monitoring finds that the equipment is abnormal, firstly, the alarm types are classified according to the details of the abnormality, for example, the fault 1: power system failure (motor, speed reducer); then, starting an operation and maintenance process, and simultaneously, automatically identifying and monitoring the operation and maintenance process, for example, a process identification 1: monitoring personnel positions and paths, and monitoring a process of maintaining a speed reducer and a motor; finally, the alarm processing results are classified and stored, for example, the alarm processing result is a result 1: the speed reducer outputs steadily, and forms an operation and maintenance record with the alarm processing result and the corresponding alarm record to be stored in the operation and maintenance database.

In some embodiments, the operation and maintenance monitoring method provided by the invention is described by taking operation and maintenance monitoring of a transformer substation as an example, and by identifying the field image, the condition that surrounding illegal invasion, loss of fixed equipment, equipment failure and the like are found to give an alarm can be detected, and the condition can be cross-verified with the condition identified by other sensors of the internet of things; reporting staff to process when abnormality is found, managing and controlling wearing standardization, maintenance area standardization, maintenance process standardization and the like after the staff enters the transformer substation, alarming when the abnormality occurs, and recording; after maintenance is completed, cross verification is carried out through the identification image and the states identified by other sensors of the Internet of things; feeding back and storing maintenance results; an alarm is provided when the device status differs from the expected status after operation and maintenance. Fig. 9 shows in detail an operation and maintenance monitoring flow of the transformer substation, wherein the transformer substation is monitored and treated in real time through a machine vision system in daily life, and meanwhile, the transformer substation is cross-validated by combining with a sensor; if the machine vision system finds that the personnel are abnormal, personnel are identified by acquiring personnel database data, the personnel abnormal types are automatically defined, and classified into abnormal 1, 2, 3 … N and the like, for example, abnormal 1: illegal personnel intrusion; anomaly 2: non-standardized wear; anomaly 3: violating rules in the operation and maintenance process; anomaly 4: the operation and maintenance task is not matched with the employee type; anomaly N: other anomalies, etc.; meanwhile, when an alarm occurs, abnormal type pushing is automatically carried out, and a monitoring center is notified. Carrying out local alarm and long-term transmission through an alarm; in addition, the machine vision system is used for monitoring the alarm result, for example, staff can correct the rule violations and the abnormality until the staff returns to normal after the alarm gives an alarm;

Correspondingly, if the machine vision system finds that the equipment class has abnormal conditions, alarming and automatically defining fault types, and classifying the fault types into faults 1, 2, 3 … N and the like, for example, fault 1: the equipment monitors abnormal data and changes of the indicator lights; fault 2: attaching foreign matters to the bus; fault 3: the standard arrangement of the transformer substation is not in place, and necessary articles are absent; fault 4: the equipment area gate is not closed; fault N: other faults; uploading the fault type to a server, generating an operation and maintenance instruction, and notifying corresponding operation and maintenance personnel to go to the site for operation and maintenance work; in the operation and maintenance process, obtaining image data through a machine vision system, and automatically monitoring the process identification of fault types and the whole operation and maintenance process aiming at operation and maintenance personnel; for example, failure 1: personnel arrive at the equipment area for positioning and monitoring, and maintenance flow monitoring; fault 2: personnel arrive at the bus for positioning and monitoring, and maintenance flow monitoring; fault 3: whether personnel carry the missing standard appliance to a designated position for placement; fault 4: whether the personnel arrive at the equipment area or not, and whether the personnel cross the other areas; fault N: identifying other fault processes; meanwhile, the machine vision system is used for monitoring operation and maintenance results, for example, result 1: the equipment monitors the data and the indicator lights are recovered to normal; results 2: the foreign matters on the bus are cleaned; results 3: the standard arrangement of the transformer substation is normal; results 4: the equipment area gate is closed and normal; results N: other normal conditions;

In some embodiments, as shown in fig. 10, a schematic diagram of a substation alarm operation and maintenance flow is given. When the daily monitoring finds that personnel identification abnormality exists, the alarm types are classified according to the abnormality details, for example, personnel alarm: illegal personnel enter, are not worn in a standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the employee types; then starting an operation and maintenance process, and simultaneously, automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: personnel intrusion recognition, no safety helmet, no work clothes wearing, smoking and the like; finally, classifying and storing the alarm processing result, for example, correcting the rule violations by personnel after the alarm processing result is the alarm, and returning to normal, and forming an operation and maintenance record by the alarm processing result and the corresponding alarm record to store the operation and maintenance record into an operation and maintenance database; when the daily monitoring finds that the equipment is abnormal, firstly, the alarm types are classified according to the details of the abnormality, for example, the fault 1: the equipment monitors abnormal data and changes of the indicator lights; then, starting an operation and maintenance process, and simultaneously, automatically identifying and monitoring the operation and maintenance process, for example, a process identification 1: personnel arrive at the equipment area for positioning and monitoring, and maintenance flow monitoring; finally, the alarm processing results are classified and stored, for example, the alarm processing result is a result 1: and the equipment monitoring data and the indicator lights are recovered to be normal, and the alarm processing result and the corresponding alarm record form an operation and maintenance record which is stored in an operation and maintenance database.

In some embodiments, taking operation and maintenance monitoring of the electric power tunnel as an example, the operation and maintenance monitoring method provided by the invention is described, and the system discovers that illegal invasion steals and cuts a cable, illegal invasion lays an optical cable, fixed equipment is absent, the cable drops and the like to give an alarm by identifying on-site images, and the state can also be cross-verified with states identified by other sensors of the internet of things, and reports staff to handle when abnormality occurs. After entering the power tunnel, the staff manages and controls wearing standardization, maintenance area standardization, maintenance process standardization and the like, alarms when violating rules and records. And after maintenance is finished, cross verification is carried out through the identification image and the states identified by other Internet of things sensors. And feeding back and storing the maintenance result. An alarm is provided when the device status differs from the expected status after operation and maintenance. FIG. 11 shows the operation and maintenance monitoring flow of the power tunnel in detail, wherein the power tunnel is monitored and treated in real time through a machine vision system in daily life, and meanwhile, the power tunnel is cross-validated by combining with a sensor; if the machine vision system finds that the personnel are abnormal, personnel are identified by acquiring personnel database data, the personnel abnormal types are automatically defined, and classified into abnormal 1, 2, 3 … N and the like, for example, abnormal 1: illegal personnel intrusion; anomaly 2: non-standardized wear; anomaly 3: violating rules in the operation and maintenance process; anomaly 4: the operation and maintenance task is not matched with the employee type; anomaly N: other anomalies, etc.; meanwhile, when an alarm occurs, abnormal type pushing is automatically carried out, and a monitoring center is notified. Carrying out local alarm and long-term transmission through an alarm; in addition, the machine vision system is used for monitoring the alarm result, for example, staff can correct the rule violations and the abnormality until the staff returns to normal after the alarm gives an alarm;

Correspondingly, if the machine vision system finds that the equipment class has abnormal conditions, alarming and automatically defining fault types, and classifying the fault types into faults 1, 2, 3 … N and the like, for example, fault 1: the cable on the bridge drops off; fault 2: the standard arrangement in the electric tunnel is not in place, and the necessary articles are missing; fault 3: the power tunnel protection door is not closed; fault N: other faults; uploading the fault type to a server, generating an operation and maintenance instruction, notifying corresponding operation and maintenance personnel to go to the site for operation and maintenance work, acquiring image data through a machine vision system in the operation and maintenance process, and automatically monitoring the process identification of the fault type and the whole operation and maintenance process aiming at the operation and maintenance personnel; for example, failure 1: the personnel arrive at the position of the drop cable for positioning monitoring and maintenance flow monitoring; fault 2: whether personnel carry the missing standard appliance to a designated position for placement; fault 3: whether the personnel arrive at the power tunnel or cross the line to other areas; fault N: identifying other fault processes; meanwhile, the machine vision system is used for monitoring operation and maintenance results, for example, result 1: resetting the drop cable; results 2: the standard arrangement in the power tunnel is normal; results 3: the power tunnel protection door is closed and normal; results N: other normal conditions;

In some embodiments, as shown in fig. 12, a schematic diagram of a power tunnel alarm operation and maintenance flow is provided. When the daily monitoring finds that personnel identification abnormality exists, the alarm types are classified according to the abnormality details, for example, personnel alarm: illegal personnel enter, are not worn in a standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the employee types; then starting an operation and maintenance process, and simultaneously, automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: personnel intrusion recognition, no safety helmet, no work clothes wearing, smoking and the like; finally, classifying and storing the alarm processing result, for example, correcting the rule violations by personnel after the alarm processing result is the alarm, and returning to normal, and forming an operation and maintenance record by the alarm processing result and the corresponding alarm record to store the operation and maintenance record into an operation and maintenance database; when the daily monitoring finds that the equipment is abnormal, firstly, the alarm types are classified according to the details of the abnormality, for example, the fault 1: the cable on the bridge drops, and simultaneously, the operation and maintenance process is automatically identified and monitored, for example, the process identification 1: the personnel arrive at the position of the drop cable for positioning monitoring and maintenance flow monitoring; finally, the alarm processing results are classified and stored, for example, the alarm processing result is a result 1: and returning the drop cable, and forming an operation and maintenance record by the alarm processing result and the corresponding alarm record, and storing the operation and maintenance record into an operation and maintenance database.

In some embodiments, taking operation and maintenance monitoring of a vehicle charging station as an example, describing the operation and maintenance monitoring method provided by the invention, adding cameras in each key monitoring area of the vehicle charging station, obtaining video images and equipment operation image data in the charging station by a system, and identifying the operation state of the equipment; automatically identifying the vehicles entering the automobile charging station, displaying the vehicles with new energy license plates normally, and recording the in-out time; when other license plates or unlicensed vehicles enter, the system automatically carries out on-site voice alarm to drive out abnormal intrusion vehicles, records vehicle information, automatically identifies license plate numbers, and simultaneously reserves vehicle entrance and exit information and videos; when the machine vision detects equipment faults such as the charging gun head shell damage of the charging pile, the exposed gun head cable, abnormal monitoring data, abnormal indicator lamp change and the like, an operation and maintenance task is automatically issued to inform a worker to the site for maintenance. After the worker arrives at the site, the system firstly automatically judges whether the operation and maintenance task is matched with the type of the worker, whether the operation position is correct or not and records the action path of the worker, secondly, identifies the standard dressing specification, standard operation and the like of the operation and maintenance worker, and after the maintenance work is finished, the system performs cross verification with the sensor of the Internet of things through the deep neural network, and reports and records the maintenance result. An alarm is provided when the device status differs from the expected status after operation and maintenance. FIG. 13 shows in detail the operation and maintenance monitoring process of the vehicle charging station, wherein the vehicle charging station is monitored and processed in real time through a machine vision system in daily life, and meanwhile, the vehicle charging station is cross-validated by combining with a sensor; if the machine vision system finds that the personnel are abnormal, the personnel are identified by acquiring the data of the maintenance personnel database, the abnormal types of the personnel are automatically defined, and the abnormal types of the personnel are classified into abnormal 1, 2, 3 … N and the like, for example, abnormal 1: illegal personnel invade the equipment to steal and destroy the equipment; anomaly 2: the maintenance personnel wear out-of-specification; anomaly 3: operation violations exist in the operation and maintenance process of maintenance personnel; anomaly N: other anomalies, etc.; moreover, whether the operation and maintenance task is matched with the employee type, whether the operation position is correct, the automatic identification of the personnel path and the like can be judged; meanwhile, when an alarm occurs, abnormal type pushing is automatically carried out, and a monitoring center is notified. Carrying out local alarm and long-term transmission through an alarm; in addition, the machine vision system is used for monitoring the alarm result, for example, staff can correct the rule violations and the abnormality until the staff returns to normal after the alarm gives an alarm;

Accordingly, if the machine vision system finds that the equipment state and the vehicle are abnormal, the equipment and/or the vehicle is identified by acquiring the equipment state database and the vehicle database data, alarming and automatically defining the equipment fault/vehicle abnormality type, classifying the equipment fault/vehicle abnormality, and classifying the equipment fault/vehicle abnormality into equipment faults 1, 2, 3 … N, vehicle abnormality 1 (other vehicle abnormalities may exist, and only abnormality 1 is taken as an example in the figure), and the equipment fault may be, for example, fault 1: the charging pile A is damaged in the charging gun head shell and the gun head cable is exposed; fault 2: monitoring data and abnormal change of an indicator lamp by the charging pile A equipment; fault 3: the display screen of the charging pile A is damaged; fault 4: the standard arrangement in the charging station is not in place, and the fire-fighting equipment is absent; fault N: other faults; the vehicle abnormality may be, for example, abnormality 1: the entering vehicle is a non-new energy charging vehicle or a non-license vehicle; if the equipment fails, uploading failure/abnormality details to a server, and informing corresponding operation and maintenance personnel to go to the site for operation and maintenance work, acquiring image data through a machine vision system in the operation and maintenance process, and automatically monitoring the equipment failure type, the abnormal flow identification of the vehicle and the whole operation and maintenance process by the operation and maintenance personnel; for example, the flow identification corresponding to the fault type may be fault 1: personnel arrive at the charging pile A, the charging gun is maintained, and the maintenance flow is monitored; fault 2: the maintenance personnel arrive at the charging pile A for positioning monitoring and maintenance flow monitoring; fault 3: the maintenance personnel arrive at the charging pile A, the display screen is replaced, and the maintenance process is monitored; fault 4: whether personnel carry the missing standard appliance to a designated position for placement; fault N: identifying other fault processes; the operation and maintenance monitoring for vehicle anomalies may be, for example, anomaly 1: the system automatically sends out an on-site voice alarm to drive a non-new energy vehicle or a license plate-free vehicle, records vehicle information, automatically identifies license plate numbers, and simultaneously reserves the information and video of the entrance and exit of the vehicle; in addition, in the monitoring of the operation and maintenance process, whether the operation and maintenance task is matched with the employee type, whether the operation position is correct, automatic identification of the personnel path and the like are continuously monitored. Meanwhile, the machine vision system is used for monitoring operation and maintenance results, for example, the monitoring of the results aiming at equipment faults can be as follows: results 1: the charging gun of the charging pile A is maintained and then returns to normal operation; results 2: the charging pile A is maintained and then is subjected to normal change of monitoring data and indicator lamps; results 3: the display screen of the charging pile A is restored to normal operation after maintenance; results 4: the standard arrangement of the charging station is normal; results N: other normal conditions; the result monitoring for vehicle anomalies may be result 1: the vehicle is driven away from the charging station by voice.

In some embodiments, as shown in fig. 14, a schematic diagram of an alarm operation and maintenance flow of a vehicle charging station is provided. When intelligent monitoring finds that personnel identification abnormality exists, the intelligent monitoring firstly classifies alarm types according to abnormality details, for example, personnel alarm: illegal personnel enter, are not worn in a standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the employee types; then starting an operation and maintenance process, and simultaneously, automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: personnel intrusion recognition, absence of safety helmet, absence of work clothes, etc.; finally, classifying and storing the alarm processing result, for example, correcting the rule violations by personnel after the alarm processing result is the alarm, and returning to normal, and forming an operation and maintenance record by the alarm processing result and the corresponding alarm record to store the operation and maintenance record into an operation and maintenance database; when the intelligent monitoring finds that the vehicle management is abnormal, firstly, the alarm type is classified according to the abnormal details, for example, the alarm type is abnormal 1: the entering vehicle is a non-new energy charging vehicle or a non-license vehicle; then starting an operation and maintenance process, and simultaneously, automatically identifying and monitoring the operation and maintenance process; for example, a process identification may be: the system automatically drives off the vehicle in an on-site voice alarm mode, records vehicle information and reserves the information and video of the driving in and driving out of the vehicle; and finally, classifying and storing the alarm processing result, for example, driving the vehicle away from the charging station after the vehicle is driven away by voice, forming an operation and maintenance record by the alarm processing result and the corresponding alarm record, and storing the operation and maintenance record into an operation and maintenance database.

In some embodiments, taking operation and maintenance monitoring of a data machine room as an example, the operation and maintenance monitoring method provided by the invention is described, the system obtains machine room equipment and environment images, and identifies the running state of the equipment, and the state can also be cross-validated with the states identified by other sensors of the internet of things; when the abnormality occurs in the machine room, the system informs maintenance personnel, and identifies and monitors the operation and maintenance process. At this time, the system recognizes the state of the personnel entering the site, the result can be a maintainer or a non-maintainer, if the system is a non-maintainer, the system considers that the system is abnormal, and the recognition result is alarmed. And when the identification result is that the maintenance personnel are identified, identifying safety equipment, maintenance procedures, travelling paths and maintenance equipment of the maintenance personnel. When the maintenance equipment does not correspond to the fault equipment or must be operated before equipment maintenance (such as an anti-static bracelet is required to be worn when the equipment board card is hot plugged), a maintenance flow error alarm is provided. Finally, after the system identifies the operation and maintenance, for example, the system identifies the state of the equipment after identifying that the operation and maintenance personnel leave the industrial place, and when the state of the equipment is different from the expected state after the operation and maintenance, an alarm is provided. Fig. 15 shows in detail an operation and maintenance monitoring flow of the data room, wherein the data room is monitored and processed in real time through a machine vision system in daily life, and meanwhile, the data room is cross-validated by combining with a sensor; if the machine vision system finds that the personnel are abnormal, personnel are identified by acquiring personnel database data, the personnel abnormal types are automatically defined, and classified into abnormal 1, 2, 3 … N and the like, for example, abnormal 1: illegal personnel intrusion; anomaly 2: non-standardized wear; anomaly 3: violating rules in the operation and maintenance process; anomaly 4: the operation and maintenance task is not matched with the employee type; anomaly N: other anomalies, etc.; meanwhile, when an alarm occurs, abnormal type pushing is automatically carried out, and a monitoring center is notified. Carrying out local alarm and long-term transmission through an alarm; in addition, the machine vision system is used for monitoring the alarm result, for example, staff can correct the rule violations and the abnormality until the staff returns to normal after the alarm gives an alarm;

Correspondingly, if the machine vision system finds that the equipment class has abnormal conditions, alarming and automatically defining fault types, and classifying the fault types into faults 1, 2, 3 … N and the like, for example, fault 1: monitoring data abnormality, appearance change and indicator lamp change by the server cabinet; fault 2: the cabinet door of the No. A cabinet of the data machine room is not closed; fault 3: the standard arrangement of the data machine room is not in place, and fire-fighting articles are needed to be lost; fault 4: the equipment cables are scattered, the labels are missing and are not bound according to the regulations; fault N: other faults; uploading the fault type to a server, generating an operation and maintenance instruction, notifying corresponding operation and maintenance personnel to go to the site for operation and maintenance work, acquiring image data through a machine vision system in the operation and maintenance process, and automatically monitoring the process identification of the fault type and the whole operation and maintenance process aiming at the operation and maintenance personnel; for example, failure 1: the operation and maintenance personnel arrive at the server cabinet for positioning and monitoring, and the maintenance flow is monitored; fault 2: whether the operation and maintenance personnel arrive at the cabinet A or not, and whether the operation and maintenance personnel cross other areas; fault 3: whether the operation and maintenance personnel carry the missing standard appliance to a designated position for placement; fault 4: whether the operation and maintenance personnel rectify and change the cable according to the specification; fault N: identifying other fault processes; meanwhile, the machine vision system is used for monitoring operation and maintenance results, for example, result 1: monitoring data, appearance and indicator lights of the server cabinet to be normal; results 2: the cabinet door of the cabinet A is normally closed; results 3: the data machine room is normally arranged, and missing objects are filled up; results 4: the cables are bound neatly, and the label is complete; results N: other normal conditions;

In some embodiments, as shown in fig. 16, a schematic diagram of a data room alarm operation and maintenance process is provided. When the daily monitoring finds that personnel identification abnormality exists, the alarm types are classified according to the abnormality details, for example, personnel alarm: illegal personnel enter, are not worn in a standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the employee types; then starting an operation and maintenance process, and simultaneously, automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: personnel intrusion identification, not wearing a safety helmet, not wearing working clothes, smoking, not wearing an antistatic bracelet and the like; finally, classifying and storing the alarm processing result, for example, correcting the rule violations by personnel after the alarm processing result is the alarm, and returning to normal, and forming an operation and maintenance record by the alarm processing result and the corresponding alarm record to store the operation and maintenance record into an operation and maintenance database; when the daily monitoring finds that the equipment is abnormal, firstly, the alarm types are classified according to the details of the abnormality, for example, the fault 1: the server cabinet monitors abnormal data, appearance change and indicator light change, and simultaneously automatically identifies and monitors the operation and maintenance process, for example, the process identification 1: the operation and maintenance personnel arrive at the server cabinet for positioning and monitoring, and the maintenance flow is monitored; finally, the alarm processing results are classified and stored, for example, the alarm processing result is a result 1: and the server cabinet monitors the data, the appearance and the indicator lights to be normal, and forms an operation and maintenance record with the alarm processing result and the corresponding alarm record to be stored in an operation and maintenance database.

Referring to fig. 17, a schematic diagram of an electronic device according to an embodiment of the disclosure is provided. As shown in fig. 3, the electronic device 500 includes:

memory 530 and one or more processors 510;

wherein the memory 530 is communicatively coupled to the one or more processors 510, and instructions 532 executable by the one or more processors are stored in the memory 530, where the instructions 532 are executable by the one or more processors 510 to cause the one or more processors 510 to perform the methods of the foregoing embodiments of the present application.

In particular, processor 510 and memory 530 may be connected by a bus or otherwise, as illustrated in FIG. 13 by bus 540. The processor 510 may be a central processing unit (Central Processing Unit, CPU). Processor 510 may also be a chip such as other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination thereof.

Memory 530, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as a cascading progressive network in embodiments of the present application, and the like. Processor 510 performs various functional applications of the processor as well as data processing by running non-transitory software programs, instructions, and modules 532 stored in memory 530.

Memory 530 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created by the processor 510, etc. In addition, memory 530 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 530 may optionally include memory located remotely from processor 510, which may be connected to processor 510 via a network, such as via communication interface 520. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

An embodiment of the present application further provides a computer-readable storage medium having stored therein computer-executable instructions that, when executed, perform the method of the previous embodiments of the present application.

The foregoing computer-readable storage media includes both physical volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media includes, but is not limited to, U disk, removable hard disk, read-Only Memory (ROM), random access Memory (RAM, random Access Memory), erasable programmable Read-Only Memory (EPROM), electrically erasable programmable Read-Only Memory (EEPROM), flash Memory or other solid state Memory technology, CD-ROM, digital Versatile Disks (DVD), HD-DVD, blue-Ray or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium capable of storing the desired information and that can be accessed by a computer.

While the subject matter described herein is provided in the general context of operating systems and application programs that execute in conjunction with the execution of a computer system, those skilled in the art will recognize that other implementations may also be performed in combination with other types of program modules. Generally, program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types. Those skilled in the art will appreciate that the subject matter described herein may be practiced with other computer system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like, as well as distributed computing environments that have tasks performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Those of ordinary skill in the art will appreciate that the elements and method steps of the examples described in connection with the embodiments of the application herein may be implemented as electronic hardware, or as a combination of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or a part of the technical solution, or in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present application.

The inspection is carried out by replacing personnel through the machine vision system, so that the labor cost is greatly reduced, and the safe and effective operation of the standardized industrial place is ensured through the omnibearing monitoring of the standardized industrial place and/or industrial equipment; meanwhile, the operation and maintenance flow can be automatically monitored; and the combination of the machine vision system and the sensor is used for cross verification, so that the monitoring of operation and maintenance at a higher level is realized, and the normal operation state of the standardized industrial place and/or industrial equipment is ensured.

It is to be understood that the above-described embodiments of the present disclosure are merely illustrative or explanatory of the principles of the disclosure and are not restrictive of the disclosure. Accordingly, any modifications, equivalent substitutions, improvements, or the like, which do not depart from the spirit and scope of the present disclosure, are intended to be included within the scope of the present disclosure. Furthermore, the appended claims of this disclosure are intended to cover all such changes and modifications that fall within the scope and boundary of the appended claims, or the equivalents of such scope and boundary.

Claims

1. An operation and maintenance monitoring method, comprising:

after the alarm is released and/or the operation and maintenance are finished, the machine vision system and the sensor are used for carrying out cross verification on the operation state of the standardized industrial place and/or the industrial equipment;

establishing a database of standardized industrial sites or industrial equipment; forming an operation and maintenance record by the abnormal condition, the alarm information, the operation and maintenance flow and the generated operation and maintenance result, and updating the operation and maintenance record into a database;

if the abnormal situation is eliminated, the operation and maintenance verification database in the database is linked for verification, and the operation and maintenance knowledge base in the database is linked for classified storage; if the abnormal condition is not eliminated, continuing to send down the operation and maintenance task until the operation and maintenance is finished;

the database comprises a personnel database, an alarm type database, an operation and maintenance flow database, an operation and maintenance verification database and an operation and maintenance knowledge database,

when the abnormal condition of the target object in the monitoring video/image occurs, the alarm type is classified according to the abnormal details, the alarm type data is updated to an alarm type database,

The machine vision system is used for carrying out real-time monitoring treatment on a standardized industrial place, if abnormal conditions of personnel are found through the machine vision system, personnel are identified through acquiring personnel database data, personnel abnormal types are automatically defined, the personnel abnormal types are classified, abnormal type pushing is automatically carried out after an alarm occurs, a monitoring center is notified, and local alarm is carried out through an alarm and long-term transmission is carried out; monitoring the alarm result by using the machine vision system, correcting the illegal abnormality until the normal state is recovered, storing the operation and maintenance records into an operation and maintenance database in a one-to-one correspondence manner,

if the machine vision system finds that the equipment has abnormal conditions, alarming and automatically defining fault types, classifying the fault types, uploading fault abnormal information to a server, and generating an operation and maintenance instruction to inform operation and maintenance personnel to clear the fault; in the operation and maintenance process, obtaining image data through the machine vision system, automatically monitoring the process identification of fault types and the whole operation and maintenance process aiming at operation and maintenance personnel, monitoring operation and maintenance results by utilizing the machine vision system, correcting rule violations until the operation and maintenance processes are recovered to be normal, and storing operation and maintenance records in a one-to-one correspondence manner into an operation and maintenance database;

Classifying alarm types corresponding to abnormality or fault occurring in equipment abnormality and personnel abnormality, wherein each alarm type comprises corresponding process management and control identification, when an alarm is triggered, automatically switching, identifying, managing and monitoring, simultaneously carrying out a plurality of alarms, and after each alarm corresponds to an operation and maintenance process, carrying out cross verification with a sensor through a convolutional neural network algorithm to judge whether the alarm is released; if the alarm is released, the operation and maintenance results are classified and stored, if the process supervision is not carried out continuously, the operation and maintenance tasks are distributed until the alarm information is released, and the alarm processing result and the corresponding alarm record form an operation and maintenance record which is stored into an operation and maintenance database.

2. The method of claim 1, wherein the abnormal situation comprises an abnormal situation of a person in the surveillance video/image and/or an abnormal situation of a target object in the surveillance video/image.

3. The method of claim 1, wherein the machine vision system can monitor the standardized industrial site and/or the industrial equipment operating state in real time by using a trained neural network.

4. The method according to claim 1, wherein the operation and maintenance procedure specifically comprises: and identifying the running state of the industrial equipment, the equipment state of the operation and maintenance personnel, the running path of the operation and maintenance personnel, the maintenance operation standard of the operation and maintenance personnel and the corresponding state of the operation and maintenance personnel and the fault equipment through the machine vision system.

5. The method of claim 1, further comprising automatically monitoring the alarm release result and/or the operation and maintenance result after the alarm release and/or operation and maintenance is completed.

6. An operation and maintenance monitoring device, comprising:

the verification module is used for carrying out cross verification on the operation state of the standardized industrial place and/or the industrial equipment by using the machine vision system and the sensor after the alarm is released and/or the operation is finished;

a database module for establishing a database of a standardized industrial location or industrial equipment; forming an operation and maintenance record by the abnormal condition, the alarm information, the operation and maintenance flow and the generated operation and maintenance result, and updating the operation and maintenance record into a database;

7. An electronic device, comprising:

a memory and one or more processors;

wherein the memory is communicatively coupled to the one or more processors, the memory having stored therein instructions executable by the one or more processors, the instructions, when executed by the one or more processors, for implementing the method of any of claims 1-5.

8. A computer readable storage medium having stored thereon computer executable instructions which, when executed by a computing device, are operable to implement a method as claimed in any one of claims 1 to 5.