CN111105047A

CN111105047A - Operation and maintenance monitoring method and device, electronic equipment and storage medium

Info

Publication number: CN111105047A
Application number: CN201911276146.8A
Authority: CN
Inventors: 吴宁海; 何阳
Original assignee: Shaanxi Ruihai Engineering Intelligence Data Technology Co ltd
Current assignee: Shaanxi Ruihai Engineering Intelligence Data Technology Co ltd
Priority date: 2019-12-12
Filing date: 2019-12-12
Publication date: 2020-05-05
Anticipated expiration: 2039-12-12
Also published as: CN111105047B

Abstract

The embodiment of the disclosure discloses an operation and maintenance monitoring method and device, electronic equipment and a storage medium, and relates to the technical field of automatic operation and maintenance monitoring. Wherein, the method comprises the following steps: the running state of a standardized industrial place and/or industrial equipment is monitored and processed in real time through a machine vision system; when the machine vision system finds that an abnormal condition occurs in the monitoring video/image, alarming and/or generating an operation and maintenance instruction; automatically monitoring an operation and maintenance flow for executing the operation and maintenance instruction through a machine vision system; after the alarm is relieved and/or the operation and maintenance are finished, a machine vision system and a sensor are used for carrying out cross validation on the operation state of the standardized industrial place and/or the industrial equipment; by adopting the method, the machine vision system replaces personnel to carry out routing inspection, the labor cost is greatly reduced, and the safe and effective operation of the standardized industrial place is ensured through the omnibearing monitoring of the standardized industrial place and/or the industrial equipment.

Description

Operation and maintenance monitoring method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of automated operation and maintenance monitoring technologies, and in particular, to an operation and maintenance monitoring method and apparatus, an electronic device, and a storage medium.

Background

In the traditional industrial places or industrial equipment, a large amount of personnel are needed for routing inspection and maintenance, the personnel cost is high, and routing inspection dead corners or passive idling of personnel and missing detection and error detection are easy to occur; and once the emergency happens, the emergency is difficult to quickly detect and process, so that certain potential safety hazard is easily caused, and even certain economic loss is brought.

Some automatic monitoring methods based on machine vision are proposed, and the method can automatically identify the invasion of external personnel and the safety equipment of maintenance personnel. However, the operation and maintenance of industrial sites or equipment is a result of a series of interactions between personnel and equipment, and the prior art does not provide a method for automatically monitoring the whole process. Therefore, a method for monitoring operation and maintenance of a standardized industrial site or industrial equipment is needed.

Disclosure of Invention

In view of the above technical problems in the prior art, the embodiments of the present disclosure provide an operation and maintenance monitoring method, apparatus, electronic device and storage medium, so as to solve the problems in the prior art that personnel cost is high, inspection dead corners are easy to occur, or personnel are in a passive idle state, and missing inspection and error detection occur; in addition, in case of an emergency, it is difficult to quickly detect and process the emergency, which causes a certain economic loss.

A first aspect of the embodiments of the present disclosure provides an operation and maintenance monitoring method, including:

the running state of a standardized industrial place and/or industrial equipment is monitored and processed in real time through a machine vision system;

when the machine vision system finds that an abnormal condition occurs in a monitoring video/image, alarming and/or generating an operation and maintenance instruction;

automatically monitoring an operation and maintenance flow for executing the operation and maintenance instruction through the machine vision system;

and after the alarm is relieved and/or the operation and maintenance are finished, the machine vision system and the sensor are used for carrying out cross validation on the operation state of the standardized industrial place and/or the industrial equipment.

In some embodiments, the abnormal condition includes an abnormal condition of a person in the surveillance video/image and/or an abnormal condition of a target object in the surveillance video/image.

In some embodiments, the method further comprises: establishing a database of standardized industrial places or industrial equipment; and forming an operation and maintenance record by the abnormal condition, the alarm information, the operation and maintenance flow and the generated operation and maintenance result, and updating the operation and maintenance record into a database.

In some embodiments, the method further comprises: if the abnormal condition is eliminated, the operation and maintenance verification database in the database is linked for verification, and meanwhile, the operation and maintenance knowledge base in the database is linked for classified storage; and if the abnormal condition is not eliminated, continuing to issue the operation and maintenance task until the operation and maintenance are finished.

In some embodiments, the machine vision system may perform real-time monitoring processing on the standardized industrial site and/or the operating state of the industrial equipment by using a trained neural network.

In some embodiments, the operation and maintenance process specifically includes: and identifying the running state of the industrial equipment, the equipment state of operation and maintenance personnel, the traveling path of the operation and maintenance personnel, the maintenance and operation specification of the operation and maintenance personnel and the corresponding state of the operation and maintenance personnel and the fault equipment through the machine vision system.

In some embodiments, the method further comprises automatically monitoring the alarm release result and/or the operation and maintenance result after the alarm release and/or the operation and maintenance are finished.

A second aspect of the embodiments of the present disclosure provides an operation and maintenance monitoring apparatus, including:

the real-time monitoring module is used for monitoring and processing the running state of the standardized industrial place and/or the industrial equipment in real time through the machine vision system;

the abnormal condition processing module is used for alarming and/or generating an operation and maintenance instruction when the machine vision system finds that the abnormal condition occurs in the monitoring video/image;

the operation and maintenance flow monitoring module is used for automatically monitoring the operation and maintenance flow for executing the operation and maintenance instruction through the machine vision system;

and the verification module is used for performing cross verification on the operation state of the standardized industrial place and/or the industrial equipment by using the machine vision system and the sensor after the alarm is relieved and/or the operation and maintenance are finished.

A third aspect of the embodiments of the present disclosure provides an electronic device, including:

a memory and one or more processors;

wherein the memory is communicatively coupled to the one or more processors, and the memory stores instructions executable by the one or more processors, and when the instructions are executed by the one or more processors, the electronic device is configured to implement the method according to the foregoing embodiments.

A fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium having stored thereon computer-executable instructions, which, when executed by a computing device, may be used to implement the method according to the foregoing embodiments.

A fifth aspect of embodiments of the present disclosure provides a computer program product comprising a computer program stored on a computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, are operable to implement a method as in the preceding embodiments.

The invention achieves the following beneficial effects: the machine vision system is used for replacing personnel to carry out inspection, so that the labor cost is greatly reduced, and the safe and effective operation of a standardized industrial site is ensured through the omnibearing monitoring of the standardized industrial site and/or industrial equipment; meanwhile, the operation and maintenance process can be automatically monitored; and the combination of the machine vision system and the sensor is utilized to carry out cross validation, thereby realizing higher-level monitoring of operation and maintenance and ensuring the normal operation state of the standardized industrial site and/or the industrial equipment.

Drawings

The features and advantages of the present disclosure will be more clearly understood by reference to the accompanying drawings, which are illustrative and not to be construed as limiting the disclosure in any way, and in which:

FIG. 1 is a flow diagram of an operation and maintenance monitoring method according to some embodiments of the present disclosure;

FIG. 2 is a block diagram of an operation and maintenance monitoring module according to some embodiments of the present disclosure;

FIG. 3 is a schematic diagram illustrating an operation and maintenance monitoring process for a power distribution room according to some embodiments of the present disclosure;

FIG. 4 is a schematic diagram of a distribution room alarm operation and maintenance flow, according to some embodiments of the present disclosure;

FIG. 5 is a schematic view of an operation and maintenance monitoring process for a gasoline station according to some embodiments of the present disclosure;

FIG. 6 is a schematic illustration of a gas station alert operation and maintenance procedure, according to some embodiments of the present disclosure;

FIG. 7 is a schematic illustration of an operation and maintenance monitoring flow for a petroleum production machine according to some embodiments of the present disclosure;

FIG. 8 is a schematic illustration of a pump production alarm operation and maintenance procedure in accordance with certain embodiments of the present disclosure;

FIG. 9 is a schematic view of an operation and maintenance monitoring process for a substation according to some embodiments of the present disclosure;

FIG. 10 is a schematic diagram of a substation alarm operation and maintenance flow, according to some embodiments of the present disclosure;

fig. 11 is a schematic view of an operation and maintenance monitoring process of a power tunnel according to some embodiments of the present disclosure;

FIG. 12 is a schematic diagram of a power tunnel alarm operation and maintenance flow, according to some embodiments of the present disclosure;

FIG. 13 is a schematic view of an operation and maintenance monitoring flow for a vehicle charging station, according to some embodiments of the present disclosure;

FIG. 14 is a schematic illustration of a vehicle charging station alert operation and maintenance procedure in accordance with some embodiments of the present disclosure;

fig. 15 is a schematic operation and maintenance monitoring flow diagram of a data room according to some embodiments of the present disclosure;

FIG. 16 is a schematic diagram of a data room alarm operation and maintenance flow, according to some embodiments of the present disclosure;

FIG. 17 is a schematic diagram of an electronic device shown in accordance with some embodiments of the present application.

Detailed Description

In the following detailed description, numerous specific details of the disclosure are set forth by way of examples in order to provide a thorough understanding of the relevant disclosure. However, it will be apparent to one of ordinary skill in the art that the present disclosure may be practiced without these specific details. It should be understood that the use of the terms "system," "apparatus," "unit" and/or "module" in this disclosure is a method for distinguishing between different components, elements, portions or assemblies at different levels of sequence. However, these terms may be replaced by other expressions if they can achieve the same purpose.

It will be understood that when a device, unit or module is referred to as being "on" … … "," connected to "or" coupled to "another device, unit or module, it can be directly on, connected or coupled to or in communication with the other device, unit or module, or intervening devices, units or modules may be present, unless the context clearly dictates otherwise. For example, as used in this disclosure, the term "and/or" includes any and all combinations of one or more of the associated listed items.

The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to limit the scope of the present disclosure. As used in the specification and claims of this disclosure, the terms "a," "an," "the," and/or "the" are not intended to be inclusive in the singular, but rather are inclusive in the plural, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" are intended to cover only the explicitly identified features, integers, steps, operations, elements, and/or components, but not to constitute an exclusive list of such features, integers, steps, operations, elements, and/or components.

These and other features and characteristics of the present disclosure, as well as the methods of operation and functions of the related elements of structure and the combination of parts and economies of manufacture, will be better understood by reference to the following description and drawings, which form a part of this specification. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the disclosure. It will be understood that the figures are not drawn to scale.

In the invention, a standard daily operation and maintenance system for industrial places is established, and the system is suitable for most industrial places, such as: transformer substation, electric power tunnel, electric power channel, electricity distribution room, gas station, charging station, oil extraction machine etc..

As shown in fig. 1, an embodiment of the present disclosure provides an operation and maintenance monitoring method, which specifically includes:

s101, monitoring and processing the running state of a standardized industrial place and/or industrial equipment in real time through a machine vision system;

s102, when the machine vision system finds that an abnormal condition occurs in a monitoring video/image, alarming and/or generating an operation and maintenance instruction;

s103, automatically monitoring an operation and maintenance flow for executing the operation and maintenance instruction through the machine vision system;

and S104, after the alarm is relieved and/or the operation and maintenance are finished, the machine vision system and the sensors are used for carrying out cross validation on the operation state of the standardized industrial place and/or the industrial equipment.

Furthermore, the abnormal situations of the personnel in the monitoring video/image include situations of illegal invasion of the personnel, nonstandard dressing of the personnel, personnel traveling paths, illegal behaviors of the personnel entering and the like;

the abnormal condition of the target object in the monitoring video/image mainly comprises that the industrial equipment has a fault; for example, the oil leakage of the oil extraction machine, the data abnormality of the oil extraction machine, the breakage of the display screen of the charging pile and the like;

in some embodiments, if an abnormal condition occurs in the target object in the surveillance video/image, manual maintenance and repair is required. Optionally, the operation and maintenance further comprises periodic operation and maintenance requirements.

Furthermore, the abnormal condition monitored by the machine vision also includes the abnormal condition of the person and the target object in the monitoring video/image, which mainly includes: the corresponding state of personnel and industrial equipment to be maintained is not consistent, and abnormal conditions such as non-correspondence between personnel maintenance equipment and fault equipment exist; meanwhile, the machine vision can also identify the equipment state, the personnel traveling path, the personnel maintenance operation specification and the personnel maintenance and fault equipment corresponding state in the area.

Optionally, the state of the identified industrial equipment and the state conversion can be performed by the machine vision system, and compared with the predefined maintenance process to see whether the state is matched.

In some embodiments, the found abnormal situation is uploaded to a server, and a corresponding operation and maintenance instruction is generated according to the abnormal situation, so as to inform operation and maintenance personnel to start an operation and maintenance flow.

In some embodiments, the method further comprises:

establishing a database of standardized industrial places or industrial equipment;

and forming an operation and maintenance record by the abnormal condition, the alarm information, the operation and maintenance flow and the operation and maintenance result, and updating the operation and maintenance record into a database.

Specifically, the database comprises one or more of a personnel database, an alarm type database, an operation and maintenance flow database, an operation and maintenance verification database and an operation and maintenance knowledge database.

Further, when the target object in the monitoring video/image has abnormal conditions, the alarm type data is updated to the alarm type database;

furthermore, after the operation and maintenance are finished, the machine vision system and the sensor are used for cross verification, if abnormal conditions are eliminated, the operation and maintenance verification database is linked for verification, and meanwhile, the operation and maintenance knowledge base is linked for classified storage; and if the abnormal condition is not eliminated, continuing to issue the operation and maintenance task until the operation and maintenance are finished.

In some embodiments, the present disclosure further operates a convolutional neural network algorithm based on the frontmost tensrflow framework to perform extraction processing on each frame of picture in the video/image.

Specifically, the method utilizes a region pro-potential method in the neural network to extract the target of each frame of picture, and the real-time monitoring of the target object is achieved by extracting the characteristics of the target object, and the real-time position and name of the target object can be obtained by an archor mechanism and a softmax classification method in the processing process.

In some embodiments, the real-time monitoring processing of the monitoring video/image by the machine vision system specifically includes:

real-time monitoring of various faults and anomalies of standardized industrial sites and/or industrial equipment is achieved by a machine vision system.

More specifically, the machine vision system can detect and process a target object by using a trained neural network structure, and can realize novel functions of illegal personnel intrusion judgment, standardized dressing detection, illegal operation and maintenance process detection, personnel trajectory tracking, voice alarm, information transmission alarm and the like.

As shown in fig. 2, an embodiment of the present disclosure further provides an operation and maintenance monitoring apparatus 200, which specifically includes:

the real-time monitoring module 201 is used for monitoring and processing the operation state of the standardized industrial place and/or the industrial equipment in real time through a machine vision system;

an abnormal situation processing module 202, configured to alarm and/or generate an operation and maintenance instruction when the machine vision system finds that an abnormal situation occurs in the monitoring video/image;

the operation and maintenance flow monitoring module 203 is used for automatically monitoring the operation and maintenance flow through the machine vision system in the operation and maintenance process;

and the verification module 204 is used for performing cross verification on the operation state of the standardized industrial place and/or the industrial equipment by using the machine vision system and the sensor after the alarm is relieved and/or the operation and maintenance are finished.

In some embodiments, the process of managing and controlling the field in daily operation of the standardized industrial site is generally divided into: daily monitoring, fault finding, operation and maintenance notification, process monitoring, result monitoring and authentication feedback.

In some embodiments, taking operation and maintenance monitoring of an electricity distribution room as an example, the operation and maintenance monitoring method provided by the present invention is described, and an image/video of a device in the electricity distribution room is obtained through a machine vision system, and an operation state of the device is identified, and the state can be cross-verified with states identified by other internet of things sensors. When the identification system has faults, such as smoke alarm or tripping, images of the power distribution room are obtained, and the operation and maintenance process is identified and monitored. At the moment, the state of the personnel entering the power distribution room is identified, the result can be maintenance personnel or non-maintenance personnel, if the personnel are non-maintenance personnel, the abnormal invasion is considered, and the identification result is alarmed. And when the identification result is a maintenance person, identifying the safety equipment, the travel path and the maintenance equipment of the maintenance person. When the maintenance equipment does not correspond to the fault equipment, a maintenance flow error alarm is provided. Finally, after the operation and maintenance are identified, for example, after the operation and maintenance personnel leave the industrial place, the system identifies the equipment state, and provides an alarm when the equipment state is different from the expected state after the operation and maintenance; FIG. 3 shows a process of operation and maintenance monitoring of a power distribution room, wherein the power distribution room is monitored and processed in real time by a machine vision system in daily life, and cross validation is performed by combining a sensor; if the machine vision system finds that the person class is abnormal, the person class is identified by acquiring data of a staff database, the abnormal type of the person is automatically defined, and the abnormal type of the person is classified into abnormity 1, 2, 3 … N and the like, wherein the abnormity 1: illegal personnel invasion; anomaly 2: non-standardized wearing; anomaly 3: violation in operation and maintenance; anomaly 4: the operation and maintenance task is not matched with the employee type; and (3) exception N: other anomalies, etc.; meanwhile, after the alarm occurs, the abnormal type pushing is automatically carried out, and a monitoring center is informed. Carrying out local alarm and long-term transmission through an alarm; in addition, a machine vision system is used for monitoring an alarm result, for example, an alarm can be used for correcting violation abnormity by staff after alarming until the violation abnormity is recovered to be normal; preferably, the operation and maintenance records are stored and recorded into the operation and maintenance database in a one-to-one correspondence manner.

Further, if the machine vision system finds that the equipment class has an abnormal condition, the machine vision system gives an alarm and automatically defines a fault type, and classifies the fault type into faults 1, 2, 3 … N and the like, for example, fault 1: the A high-voltage cabinet monitors data abnormity, appearance change and indicator lamp change; the failure 2 is that the A-type low-voltage cabinet monitors data abnormity, appearance change and indicator lamp change; failure 3: the standard arrangement of the distribution room is not in place, and necessary articles are lost; and 4, fault: the door of the A-size cabinet in the power distribution room is not closed; and (4) fault N: other faults, etc.; after the fault abnormal information is uploaded to the server, an operation and maintenance instruction is generated to inform operation and maintenance personnel to go to clear the fault; in the operation and maintenance process, image data are obtained through a machine vision system, and the flow identification of the fault type and the whole operation and maintenance process are automatically monitored by operation and maintenance personnel; for example, failure 1: positioning and monitoring when a person arrives at the A-grade high-voltage cabinet, and monitoring a maintenance process; and (3) failure 2: positioning and monitoring when a person arrives at the A-type low-voltage cabinet, and monitoring a maintenance process; failure 3: whether a person carries a missing standard appliance to a specified position for placement; and 4, fault: whether the personnel arrive at the A cabinet or not and whether the personnel cross the other areas or not; and (4) fault N: identifying other fault processes; meanwhile, the operation and maintenance result is monitored by using a machine vision system, for example, the result 1: the monitoring data, the appearance and the indicator lamp of the A high-voltage cabinet are recovered to be normal; results 2: the monitoring data, the appearance and the indicator lamp of the No. A low-voltage cabinet are recovered to be normal; results 3: the standard arrangement of the distribution room is normal; results 4: the door of the No. A cabinet in the power distribution room is normally closed; results N: the others are normal;

furthermore, alarm types corresponding to the abnormity or fault occurring in equipment abnormity and personnel abnormity are classified, each alarm type is provided with corresponding flow management and control identification, and after the alarm is triggered, the identification management and monitoring are automatically switched, so that a plurality of alarms can be simultaneously performed; meanwhile, after the operation and maintenance process corresponding to each alarm is completed, cross validation is carried out on the alarm and the sensor through a convolutional neural network algorithm, and whether the alarm is relieved or not is judged. If the alarm is released, the operation and maintenance results are stored in a classified mode, if the operation and maintenance results are not solved, the process supervision is continued, and the operation and maintenance tasks are distributed until the alarm information is released.

And further, after the alarm is relieved and/or the operation and maintenance are finished, the machine vision system and the sensor are used for carrying out cross verification and feedback on the operation state of the standardized industrial place and/or the industrial equipment.

In some embodiments, as shown in fig. 4, a schematic diagram of an alarm operation and maintenance process of a power distribution room is correspondingly provided. When people identification abnormality is found by daily monitoring, firstly, alarm types are classified according to abnormality details, for example, people alarm: illegal personnel enter, are worn in an un-standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the types of the personnel; then starting an operation and maintenance process, and automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: personnel intrusion identification, no safety helmet, no working clothes, smoking and the like; and finally, classifying and storing the alarm processing result, for example, after the alarm processing result is an alarm, a person corrects the violation and returns to normal, and the alarm processing result and the corresponding alarm record form an operation and maintenance record and store the operation and maintenance record into the operation and maintenance database. When the equipment abnormality is found by daily monitoring, firstly, the alarm type is classified according to the abnormality details, for example, the fault 1: the A high-voltage cabinet monitors data abnormity, appearance change and indicator lamp change; then, starting an operation and maintenance process, and simultaneously automatically identifying and monitoring the operation and maintenance process, for example, flow identification 1: positioning and monitoring when a person arrives at the A-grade high-voltage cabinet, and monitoring a maintenance process; and finally classifying and storing the alarm processing result, wherein the alarm processing result is a result 1: the monitoring data, the appearance and the indicator lamp of the A high-voltage cabinet are recovered to be normal, and an operation and maintenance record is formed by the alarm processing result and the corresponding alarm record and is stored in the operation and maintenance database.

In some embodiments, the operation and maintenance monitoring method provided by the present invention is described by taking the operation and maintenance monitoring of a gas station as an example, and when an abnormality occurs in the identification site, such as a person smoking, smoke/fire equipment missing, etc., an alarm message is sent to notify a worker to process the abnormality. At the moment, the system identifies the state of personnel entering the equipment area of the gas station, if the personnel are identified as non-maintenance personnel by the system, the system considers abnormal invasion and alarms the identification result; if identified by the system as a serviceman, the serviceman's safe dressing, travel path, and equipment for maintenance are identified. When the maintenance equipment does not correspond to the fault equipment, a maintenance flow error alarm is provided. After the system identifies the operation and maintenance, for example, after the operation and maintenance personnel leave the industrial place, the system cross-verifies and records the maintenance result through the deep neural network and the sensing of the internet of things. And when the equipment state is different from the expected state after operation and maintenance, providing an alarm. As shown in particular in fig. 5; FIG. 5 shows the operation and maintenance monitoring process of the gas station, which is performed daily by a machine vision system to perform real-time monitoring processing on the gas station, and meanwhile, cross validation is performed in combination with a sensor; if the machine vision system finds that the person class is abnormal, the person class is identified by acquiring data of a staff database, the abnormal type of the person is automatically defined, and the abnormal type of the person is classified into abnormity 1, 2, 3 … N and the like, wherein the abnormity 1: illegal personnel invasion; anomaly 2: smoking in the personnel gas station; anomaly 3: violation in operation and maintenance; anomaly 4: the operation and maintenance task is not matched with the employee type; and (3) exception N: other anomalies, etc.; meanwhile, after the alarm occurs, the abnormal type pushing is automatically carried out, and a monitoring center is informed. Carrying out local alarm and long-term transmission through an alarm; in addition, a machine vision system is used for monitoring an alarm result, for example, an alarm can be used for correcting violation abnormity by staff after alarming until the violation abnormity is recovered to be normal;

correspondingly, when the machine vision system finds that the equipment class has an abnormal condition, the machine vision system gives an alarm and automatically defines fault types, and classifies the fault types into faults 1, 2, 3 … N and the like, for example, the fault 1: the No. A oiling machine has abnormal data, changed appearance and changed indicator lights; no. B oiling machine oil gun is not placed at the designated position when the fault is 2; failure 3: the standard arrangement of the gas station is not in place, and necessary goods are lost; and 4, fault: the No. A refueling cabinet door in the gas station is not closed; and (4) fault N: other faults, etc.; after the fault abnormal information is uploaded to the server, an operation and maintenance instruction is generated to inform operation and maintenance personnel to go to clear the fault; in the operation and maintenance process, image data are obtained through a machine vision system, and the flow identification of the fault type and the whole operation and maintenance process are automatically monitored by operation and maintenance personnel; for example, failure 1: the personnel arrive at the oiling machine A for positioning monitoring and maintenance flow monitoring; and (3) failure 2: the personnel arrive at the refueling gun of the B oiling machine for positioning monitoring and maintenance flow monitoring; failure 3: whether a person carries a missing standard appliance to a specified position for placement; and 4, fault: whether the personnel arrive at the oiling machine A or not and whether the personnel cross the other areas or not; and (4) fault N: identifying other fault processes; meanwhile, the operation and maintenance result is monitored by using a machine vision system, for example, the result 1: the monitoring data, the appearance and the indicator lamp of the A high-voltage cabinet are recovered to be normal; results 2: the refueling gun of the refueling machine B returns to normal; results 3: the arrangement of the fire-fighting equipment of the gas station is normal; results 4: the door of the No. A cabinet in the gas station is normally closed; results N: the others are normal;

preferably, the operation and maintenance records are stored and recorded into the operation and maintenance database in a one-to-one correspondence manner.

Furthermore, alarm types corresponding to the abnormity or fault occurring in equipment abnormity and personnel abnormity are classified, each alarm type is provided with corresponding flow management and control identification, and after the alarm is triggered, the identification management and monitoring are automatically switched, so that a plurality of alarms can be synchronously performed; meanwhile, after the operation and maintenance process corresponding to each alarm is completed, cross validation is carried out on the alarm and the sensor through a convolutional neural network algorithm, and whether the alarm is relieved or not is judged. If the alarm is released, the operation and maintenance results are stored in a classified mode, if the operation and maintenance results are not solved, the process supervision is continued, and the operation and maintenance tasks are distributed until the alarm information is released.

Furthermore, after the alarm is relieved and/or the operation and maintenance are finished, the machine vision system and other internet of things sensors can be used for carrying out cross verification on the operation state of the standardized industrial site and/or the industrial equipment and feeding back the operation state.

In some embodiments, as shown in fig. 6, a schematic diagram of a gas station alarm operation and maintenance process is provided. When people identification abnormality is found by daily monitoring, firstly, alarm types are classified according to abnormality details, for example, people alarm: illegal personnel enter, are worn in an un-standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the types of the personnel; then starting an operation and maintenance process, and automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: personnel intrusion identification, no safety helmet, no working clothes, smoking and the like; finally, classifying and storing the alarm processing result, for example, after the alarm processing result is an alarm, a person corrects the violation and returns to normal, and the alarm processing result and the corresponding alarm record form an operation and maintenance record and store the operation and maintenance record into an operation and maintenance database; when the equipment abnormality is found by daily monitoring, firstly, the alarm type is classified according to the abnormality details, for example, the fault 1: the No. A oiling machine has abnormal data, changed appearance and changed indicator lights; then, starting an operation and maintenance process, and simultaneously automatically identifying and monitoring the operation and maintenance process, for example, flow identification 1: the personnel arrive at the oiling machine A for positioning monitoring and maintenance flow monitoring; and finally classifying and storing the alarm processing result, wherein the alarm processing result is a result 1: the monitoring data, the appearance and the indicator lamp of the No. A oiling machine are recovered to be normal, and an operation and maintenance record is formed by the alarm processing result and the corresponding alarm record and is stored into an operation and maintenance database.

In some embodiments, the operation and maintenance monitoring method provided by the present invention is described by taking the operation and maintenance monitoring of an oil production machine as an example, a camera is additionally installed at each key monitoring system position of oil production machine equipment, the system obtains an environment image of the oil production machine and an equipment operation image, and identifies an operation state of the equipment, and the state and an equipment self-checking program complement each other and can be cross-verified. When the oil extraction machine is abnormal, corresponding fault types can be alarmed according to equipment fault reasons, and related responsible personnel can be informed to carry out equipment maintenance on site; firstly, identity information detection is carried out on in-place personnel, the result can be maintenance personnel or non-maintenance personnel, and when the non-maintenance personnel are judged, the abnormal personnel are considered to invade, and the recognition result is alarmed; when the identification result is that the maintenance personnel are present, carrying out the next step of standard dressing and safety equipment detection, and monitoring the advancing path and the maintenance flow; and when the maintenance items do not correspond to the fault information, providing a maintenance flow error alarm to prompt maintenance personnel to correct in time. Finally, after the system identifies the operation and maintenance, for example, after the operation and maintenance personnel leave the industrial place, the system identifies the equipment state, and provides an alarm when the equipment state is different from the expected state after the operation and maintenance. Fig. 7 shows the operation and maintenance monitoring process of the oil extraction machine in detail, wherein the oil extraction machine is monitored and processed in real time by a machine vision system in daily life, and cross validation is performed by combining a sensor; if the machine vision system finds that the person class is abnormal, the person class is identified by acquiring data of a staff database, the abnormal type of the person is automatically defined, and the abnormal type of the person is classified into abnormity 1, 2, 3 … N and the like, wherein the abnormity 1: illegal personnel invasion; anomaly 2: non-standardized wearing; anomaly 3: violation in operation and maintenance; anomaly 4: the operation and maintenance task is not matched with the employee type; and (3) exception N: other anomalies, etc.; meanwhile, after the alarm occurs, the abnormal type pushing is automatically carried out, and a monitoring center is informed. Carrying out local alarm and long-term transmission through an alarm; in addition, a machine vision system is used for monitoring an alarm result, for example, an alarm can be used for correcting violation abnormity by staff after alarming until the violation abnormity is recovered to be normal;

correspondingly, when the machine vision system finds that the equipment class has an abnormal condition, the machine vision system gives an alarm and automatically defines fault types, and classifies the fault types into faults 1, 2, 3 … N and the like, for example, the fault 1: power system faults (motor, reducer); failure 2 transmission system failure (belt, steel rope); failure 3: work system failures (main link, tail shaft, walking beam); and 4, fault: a delivery system failure (oil pipeline); and (4) fault N: other faults, etc.; after the fault abnormal information is uploaded to the server, an operation and maintenance instruction is generated, and operation and maintenance personnel are notified to go to clear the fault; in the operation and maintenance process, image data are obtained through a machine vision system, and the flow identification of the fault type and the whole operation and maintenance process are automatically monitored by operation and maintenance personnel; for example, failure 1: monitoring personnel positions and paths, maintaining a speed reducer and monitoring the process of a motor; and (3) failure 2: monitoring the position and path of personnel, and monitoring the process of replacing a belt and maintaining a steel rope; failure 3: monitoring personnel positions and paths, and monitoring the processes of replacing a main connecting rod, a tail shaft and a walking beam; and 4, fault: monitoring personnel positions and paths, replacing oil pipeline seals and monitoring the process of maintaining the pipelines; and (4) fault N: monitoring the positions and paths of personnel, and identifying other fault processes; meanwhile, the operation and maintenance result is monitored by using a machine vision system, for example, the result 1: the output of the speed reducer is stable; results 2: the belt pulley has moderate tensioning degree and no jumping, the steel ropes are arranged neatly, and the steel ropes have no loose strands or broken strands; results 3: the precision of the deformation-free motion of the main connecting rod, the tail shaft and the walking beam reaches the standard; results 4: the oil pipeline has no oil leakage and vibration; results N: the others are normal;

Furthermore, after the alarm is relieved and/or the operation and maintenance are finished, the machine vision system and the sensor are used for carrying out cross validation and feedback on the operation state of the standardized industrial place and/or the industrial equipment.

In some embodiments, as shown in fig. 8, a schematic diagram of an operation and maintenance process of a pumping unit alarm is provided. When people identification abnormality is found by daily monitoring, firstly, alarm types are classified according to abnormality details, for example, people alarm: illegal personnel enter, are worn in an un-standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the types of the personnel; then starting an operation and maintenance process, and automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: personnel intrusion identification, no safety helmet, no working clothes, smoking and the like; finally, classifying and storing the alarm processing result, for example, after the alarm processing result is an alarm, a person corrects the violation and returns to normal, and the alarm processing result and the corresponding alarm record form an operation and maintenance record and store the operation and maintenance record into an operation and maintenance database; when the equipment abnormality is found by daily monitoring, firstly, the alarm type is classified according to the abnormality details, for example, the fault 1: power system faults (motor, reducer); then, starting an operation and maintenance process, and simultaneously automatically identifying and monitoring the operation and maintenance process, for example, flow identification 1: monitoring personnel positions and paths, maintaining a speed reducer and monitoring the process of a motor; and finally classifying and storing the alarm processing result, wherein the alarm processing result is a result 1: the output of the speed reducer is stable, and the alarm processing result and the corresponding alarm record form an operation and maintenance record to be stored in the operation and maintenance database.

In some embodiments, the operation and maintenance monitoring of the transformer substation is taken as an example to illustrate the operation and maintenance monitoring method provided by the invention, and the alarm is given out by identifying field images and finding out peripheral illegal intrusion, fixed equipment loss, equipment failure and the like, and the state can also be cross-verified with the state identified by other internet-of-things sensors; reporting to a worker for processing when the abnormity is found, managing and controlling the wearing standardization, the maintenance area standardization, the maintenance process standardization and the like of the worker after the worker enters the transformer substation, and alarming and recording when the abnormity is violated; after maintenance is finished, cross verification is carried out through the recognition image and the states recognized by other IOT sensors; feeding back and storing the maintenance result; and when the equipment state is different from the expected state after operation and maintenance, providing an alarm. Fig. 9 shows the operation and maintenance monitoring process of the transformer substation in detail, wherein the transformer substation is monitored and processed in real time by a machine vision system in daily life, and is cross-verified by combining with a sensor; if the machine vision system finds that the person class is abnormal, the person class is identified by acquiring data of a staff database, the abnormal type of the person is automatically defined, and the abnormal type of the person is classified into abnormity 1, 2, 3 … N and the like, wherein the abnormity 1: illegal personnel invasion; anomaly 2: non-standardized wearing; anomaly 3: violation in operation and maintenance; anomaly 4: the operation and maintenance task is not matched with the employee type; and (3) exception N: other anomalies, etc.; meanwhile, after the alarm occurs, the abnormal type pushing is automatically carried out, and a monitoring center is informed. Carrying out local alarm and long-term transmission through an alarm; in addition, a machine vision system is used for monitoring an alarm result, for example, an alarm can be used for correcting violation abnormity by staff after alarming until the violation abnormity is recovered to be normal;

correspondingly, when the machine vision system finds that the equipment class has an abnormal condition, the machine vision system gives an alarm and automatically defines fault types, and classifies the fault types into faults 1, 2, 3 … N and the like, for example, the fault 1: the equipment monitors data abnormity and indicator light change; and (3) failure 2: foreign matters are attached to the bus; failure 3: the standard arrangement of the transformer substation is not in place, and necessary articles are lost; and 4, fault: the equipment area gate is not closed; and (4) fault N: other failures; uploading the fault type to a server, generating an operation and maintenance instruction, and informing corresponding operation and maintenance personnel to carry out operation and maintenance work on site; in the operation and maintenance process, image data are obtained through a machine vision system, and the flow identification of the fault type and the whole operation and maintenance process are automatically monitored by operation and maintenance personnel; for example, failure 1: personnel arrive at the equipment area for positioning monitoring and maintenance flow monitoring; and (3) failure 2: positioning and monitoring when personnel arrive at the bus, and monitoring the maintenance process; failure 3: whether a person carries a missing standard appliance to a specified position for placement; and 4, fault: whether the personnel arrive at the equipment area or cross the other areas; and (4) fault N: identifying other fault processes; meanwhile, the operation and maintenance result is monitored by using a machine vision system, for example, the result 1: the equipment monitors data and the indicator light returns to normal; results 2: cleaning up foreign matters on the bus; results 3: the standard arrangement of the transformer substation is normal; results 4: the gate of the equipment area is closed and normal; results N: the others are normal;

In some embodiments, as shown in fig. 10, a schematic diagram of a substation alarm operation and maintenance flow is provided. When people identification abnormality is found by daily monitoring, firstly, alarm types are classified according to abnormality details, for example, people alarm: illegal personnel enter, are worn in an un-standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the types of the personnel; then starting an operation and maintenance process, and automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: personnel intrusion identification, no safety helmet, no working clothes, smoking and the like; finally, classifying and storing the alarm processing result, for example, after the alarm processing result is an alarm, a person corrects the violation and returns to normal, and the alarm processing result and the corresponding alarm record form an operation and maintenance record and store the operation and maintenance record into an operation and maintenance database; when the equipment abnormality is found by daily monitoring, firstly, the alarm type is classified according to the abnormality details, for example, the fault 1: the equipment monitors data abnormity and indicator light change; then, starting an operation and maintenance process, and simultaneously automatically identifying and monitoring the operation and maintenance process, for example, flow identification 1: personnel arrive at the equipment area for positioning monitoring and maintenance flow monitoring; and finally classifying and storing the alarm processing result, wherein the alarm processing result is a result 1: and the equipment monitoring data and the indicator light are recovered to be normal, and an operation and maintenance record is formed by the alarm processing result and the corresponding alarm record and is stored into the operation and maintenance database.

In some embodiments, the operation and maintenance monitoring of the power tunnel is taken as an example to illustrate the operation and maintenance monitoring method provided by the present invention, the system identifies the field image, finds that the cable is cut illegally, the optical cable is laid illegally, the fixed equipment is lost, the cable falls off, and the like, and sends out an alarm, and the state can also be cross-verified with the state identified by other sensors of the internet of things, and is reported to a worker for processing when an abnormality occurs. The staff gets into behind the electric power tunnel to its dress standardization, maintenance area standardization, maintenance process standardization etc. manage and control, reports to the police in case of violation of rules and regulations, and the record. And after the maintenance is finished, cross verification is carried out through the recognition image and the states recognized by other IOT sensors. And feeding back and storing the maintenance result. And when the equipment state is different from the expected state after operation and maintenance, providing an alarm. Fig. 11 shows in detail an operation and maintenance monitoring process of an electric power tunnel, which daily monitors and processes the electric power tunnel in real time through a machine vision system, and performs cross validation by combining a sensor; if the machine vision system finds that the person class is abnormal, the person class is identified by acquiring data of a staff database, the abnormal type of the person is automatically defined, and the abnormal type of the person is classified into abnormity 1, 2, 3 … N and the like, wherein the abnormity 1: illegal personnel invasion; anomaly 2: non-standardized wearing; anomaly 3: violation in operation and maintenance; anomaly 4: the operation and maintenance task is not matched with the employee type; and (3) exception N: other anomalies, etc.; meanwhile, after the alarm occurs, the abnormal type pushing is automatically carried out, and a monitoring center is informed. Carrying out local alarm and long-term transmission through an alarm; in addition, a machine vision system is used for monitoring an alarm result, for example, an alarm can be used for correcting violation abnormity by staff after alarming until the violation abnormity is recovered to be normal;

correspondingly, when the machine vision system finds that the equipment class has an abnormal condition, the machine vision system gives an alarm and automatically defines fault types, and classifies the fault types into faults 1, 2, 3 … N and the like, for example, the fault 1: the cable on the bridge frame falls off; and (3) failure 2: the standard arrangement in the electric tunnel is not in place, and necessary articles are lost; failure 3: the electric tunnel protective door is not closed; and (4) fault N: other failures; uploading the fault type to a server, generating an operation and maintenance instruction, informing corresponding operation and maintenance personnel to carry out operation and maintenance work on site, acquiring image data through a machine vision system in the operation and maintenance process, and carrying out automatic monitoring on the flow identification of the fault type and the whole operation and maintenance process aiming at the operation and maintenance personnel; for example, failure 1: positioning and monitoring when personnel arrive at the drop cable, and monitoring the maintenance process; and (3) failure 2: whether a person carries a missing standard appliance to a specified position for placement; failure 3: whether people arrive at the electric power tunnel or cross other areas; and (4) fault N: identifying other fault processes; meanwhile, the operation and maintenance result is monitored by using a machine vision system, for example, the result 1: returning the fallen cable; results 2: the standard arrangement in the electric tunnel is normal; results 3: the protection door of the electric tunnel is closed and normal; results N: the others are normal;

In some embodiments, as shown in fig. 12, a schematic diagram of an operation and maintenance process of power tunnel alarm is provided. When people identification abnormality is found by daily monitoring, firstly, alarm types are classified according to abnormality details, for example, people alarm: illegal personnel enter, are worn in an un-standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the types of the personnel; then starting an operation and maintenance process, and automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: personnel intrusion identification, no safety helmet, no working clothes, smoking and the like; finally, classifying and storing the alarm processing result, for example, after the alarm processing result is an alarm, a person corrects the violation and returns to normal, and the alarm processing result and the corresponding alarm record form an operation and maintenance record and store the operation and maintenance record into an operation and maintenance database; when the equipment abnormality is found by daily monitoring, firstly, the alarm type is classified according to the abnormality details, for example, the fault 1: and (3) the cable on the bridge frame falls off, and meanwhile, the operation and maintenance process is automatically identified and monitored, for example, the process identification 1: positioning and monitoring when personnel arrive at the drop cable, and monitoring the maintenance process; and finally classifying and storing the alarm processing result, wherein the alarm processing result is a result 1: and returning the fallen cable, and forming an operation and maintenance record by the alarm processing result and the corresponding alarm record and storing the operation and maintenance record into the operation and maintenance database.

In some embodiments, the operation and maintenance monitoring of the vehicle charging station is taken as an example to explain the operation and maintenance monitoring method provided by the invention, cameras are additionally arranged in each key monitoring area of the vehicle charging station, and the system obtains video images and equipment operation image data in the charging station and identifies the operation state of the equipment; automatically identifying vehicles entering the vehicle charging station, displaying vehicles with new energy license plates normally, and recording access time; when other license plates or unlicensed vehicles enter, the system automatically gives an alarm in situ by voice to drive away abnormal vehicles entering, records vehicle information, automatically identifies license plate numbers, and simultaneously retains vehicle entering and exiting information and videos; when machine vision detects equipment faults such as damage of a charging gun head shell of the charging pile, exposure of a gun head cable, abnormal monitoring data, abnormal change of an indicator lamp and the like, an operation and maintenance task is automatically issued to inform a worker to maintain on site. After the staff arrives at the site, the system automatically judges whether the operation and maintenance task is matched with the type of the staff or not, whether the operation position is correct or not, records the action path of the staff, identifies the standard dressing standard, standard operation and the like of the operation and maintenance staff, performs cross verification with the sensor of the Internet of things through the deep neural network after the maintenance work is completed, and reports and records the maintenance result. And when the equipment state is different from the expected state after operation and maintenance, providing an alarm. Fig. 13 shows the operation and maintenance monitoring process of the vehicle charging station in detail, wherein the vehicle charging station is monitored and processed in real time by the machine vision system in daily life, and meanwhile, cross validation is performed by combining the sensor; if the machine vision system finds that the person class is abnormal, the person class is identified by acquiring data of a maintenance person database, the abnormal type of the person is automatically defined, and the abnormal type of the person is classified into abnormity 1, 2, 3 … N and the like, such as abnormity 1: illegal personnel invade to steal and destroy the equipment; anomaly 2: the dressing of maintainers is not standard; anomaly 3: operation violation exists in the operation and maintenance process of maintenance personnel; and (3) exception N: other anomalies, etc.; moreover, whether the operation and maintenance tasks are matched with the types of the staff, whether the operation positions are correct, automatic identification of the staff paths and the like can be judged; meanwhile, after the alarm occurs, the abnormal type pushing is automatically carried out, and a monitoring center is informed. Carrying out local alarm and long-term transmission through an alarm; in addition, a machine vision system is used for monitoring an alarm result, for example, an alarm can be used for correcting violation abnormity by staff after alarming until the violation abnormity is recovered to be normal;

correspondingly, when the machine vision system finds that the equipment state and the vehicle identification are abnormal, the equipment and/or the vehicle are identified by acquiring the equipment state database and the vehicle database data, an alarm is given, the type of the equipment fault/vehicle abnormality is automatically defined, the equipment fault/vehicle abnormality is classified into equipment faults 1, 2, 3 … N, vehicle abnormality 1 (other vehicle abnormalities may exist, and only abnormality 1 is taken as an example in the figure), and the like, wherein the equipment fault may be, for example, fault 1: the charging gun head shell of the charging pile A is damaged, and a gun head cable is exposed; and (3) failure 2: charging pile A equipment monitoring data and an indicator light change abnormally; failure 3: a display screen of the charging pile A is damaged; and 4, fault: the standard arrangement in the charging station is not in place, and fire-fighting equipment is lost; and (4) fault N: other failures; the vehicle abnormality may be, for example, abnormality 1: the entering vehicle is a non-new energy charging vehicle or a unlicensed vehicle; if the equipment fault is detected, the fault/abnormal details are uploaded to a server, and operation and maintenance personnel are informed to carry out operation and maintenance work on site correspondingly, image data are obtained through a machine vision system in the operation and maintenance process, and the operation and maintenance personnel carry out automatic monitoring on the equipment fault type, the vehicle abnormal flow identification and the whole operation and maintenance process; for example, the process identification for the corresponding fault type may be fault 1: when a person arrives at the charging pile number A, the charging gun is maintained, and the maintenance process is monitored; and (3) failure 2: a maintenance worker arrives at the charging pile number A for positioning monitoring, and a maintenance process is monitored; failure 3: a maintenance worker arrives at the charging pile number A, the display screen is replaced, and the maintenance process is monitored; and 4, fault: whether a person carries a missing standard appliance to a specified position for placement; and (4) fault N: identifying other fault processes; the operation and maintenance monitoring for vehicle abnormality may be, for example, abnormality 1: the system automatically alarms in situ to drive away vehicles which abnormally break into non-new energy or unlicensed vehicles, records vehicle information, automatically identifies license plate numbers, and retains vehicle driving-in and driving-out information and videos; in addition, in the monitoring of the operation and maintenance process, whether the operation and maintenance task is matched with the type of the staff, whether the operation position is correct, automatic identification of the staff path and the like are continuously monitored. Meanwhile, the operation and maintenance result is monitored by using a machine vision system, for example, the result monitoring for the equipment failure may be: results 1: the charging gun of the charging pile A is maintained and then returns to normal work; results 2: after maintenance, the charging pile A monitors data and the indicator lamp changes normally; results 3: the display screen of the charging pile A is maintained and then is restored to normal work; results 4: the standard arrangement of the charging station is normal; results N: the others are normal; the result monitoring for vehicle anomalies may be result 1: the vehicle is driven away from the charging station after being driven away by voice.

In some embodiments, as shown in fig. 14, a schematic diagram of a vehicle charging station alarm operation and maintenance process is provided. When the intelligent monitoring finds that the person identification is abnormal, firstly, the alarm type is classified according to the abnormal details, for example, the person alarm: illegal personnel enter, are worn in an un-standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the types of the personnel; then starting an operation and maintenance process, and automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: personnel intrusion identification, no safety helmet, no working clothes, and the like; finally, classifying and storing the alarm processing result, for example, after the alarm processing result is an alarm, a person corrects the violation and returns to normal, and the alarm processing result and the corresponding alarm record form an operation and maintenance record and store the operation and maintenance record into an operation and maintenance database; when intelligent monitoring finds that vehicle management abnormity exists, firstly, alarm types are classified according to abnormity details, for example, abnormity 1: the entering vehicle is a non-new energy charging vehicle or a unlicensed vehicle; then starting an operation and maintenance process, and automatically identifying and monitoring the operation and maintenance process; for example, the process identification may be: the system automatically gives an alarm in situ by voice to drive away the vehicle, records the vehicle information and retains the vehicle driving-in and driving-out information and video; and finally, classifying and storing the alarm processing result, for example, the alarm processing result is that the vehicle drives away from a charging station after being driven away by voice, and the alarm processing result and the corresponding alarm record form an operation and maintenance record and are stored into an operation and maintenance database.

In some embodiments, the operation and maintenance monitoring of a data machine room is taken as an example to illustrate the operation and maintenance monitoring method provided by the present invention, a system obtains machine room equipment and environment images, and identifies the running state of the equipment, and the state can also be cross-verified with the states identified by other internet of things sensors; when the abnormity appears in the machine room, the system informs maintenance personnel and identifies and monitors the operation and maintenance process. At the moment, the system identifies the state of personnel entering the site, the result can be maintenance personnel or non-maintenance personnel, if the system is the non-maintenance personnel, the system considers abnormal invasion, and the identification result is alarmed. And when the identification result is a maintenance person, identifying the safety equipment, the maintenance program, the traveling path and the maintenance equipment of the maintenance person. When the maintenance equipment does not correspond to the fault equipment or necessary operation is performed before the equipment is maintained (for example, an antistatic bracelet must be worn when the equipment board card is subjected to hot plug), a maintenance flow error alarm is provided. Finally, after the system identifies the operation and maintenance, for example, after the operation and maintenance personnel leave the industrial place, the system identifies the equipment state, and provides an alarm when the equipment state is different from the expected state after the operation and maintenance. Fig. 15 shows in detail an operation and maintenance monitoring process of a data room, which is performed daily by a machine vision system to perform real-time monitoring processing on the data room, and meanwhile, cross validation is performed in combination with a sensor; if the machine vision system finds that the person class is abnormal, the person class is identified by acquiring data of a staff database, the abnormal type of the person is automatically defined, and the abnormal type of the person is classified into abnormity 1, 2, 3 … N and the like, wherein the abnormity 1: illegal personnel invasion; anomaly 2: non-standardized wearing; anomaly 3: violation in operation and maintenance; anomaly 4: the operation and maintenance task is not matched with the employee type; and (3) exception N: other anomalies, etc.; meanwhile, after the alarm occurs, the abnormal type pushing is automatically carried out, and a monitoring center is informed. Carrying out local alarm and long-term transmission through an alarm; in addition, a machine vision system is used for monitoring an alarm result, for example, an alarm can be used for correcting violation abnormity by staff after alarming until the violation abnormity is recovered to be normal;

correspondingly, when the machine vision system finds that the equipment class has an abnormal condition, the machine vision system gives an alarm and automatically defines fault types, and classifies the fault types into faults 1, 2, 3 … N and the like, for example, the fault 1: the server cabinet monitors data abnormity, appearance change and indicator lamp change; and (3) failure 2: the cabinet door of the No. A cabinet of the data machine room is not closed; failure 3: the standard arrangement of the data machine room is not in place, and fire-fighting articles are necessary to be lost; and 4, fault: equipment cables are scattered, labels are lost and are not bound according to the regulations; and (4) fault N: other failures; uploading the fault type to a server, generating an operation and maintenance instruction, informing corresponding operation and maintenance personnel to carry out operation and maintenance work on site, acquiring image data through a machine vision system in the operation and maintenance process, and carrying out automatic monitoring on the flow identification of the fault type and the whole operation and maintenance process aiming at the operation and maintenance personnel; for example, failure 1: the operation and maintenance personnel arrive at the server cabinet for positioning monitoring and maintenance flow monitoring; and (3) failure 2: whether the operation and maintenance personnel arrive at the A cabinet or not and whether the operation and maintenance personnel cross the other areas or not; failure 3: whether the operation and maintenance personnel carry the missing standard appliance to a specified position for placement; and 4, fault: whether the operation and maintenance personnel amend the cable according to the regulations; and (4) fault N: identifying other fault processes; meanwhile, the operation and maintenance result is monitored by using a machine vision system, for example, the result 1: the server cabinet monitors data, appearance and indicator lights to be recovered to normal; results 2: the door of the No. A cabinet is normally closed; results 3: the data machine room is normally arranged in a standard mode, and missing articles are completely supplemented; results 4: the cables are bound orderly, and the label is complete; results N: the others are normal;

In some embodiments, as shown in fig. 16, a schematic diagram of an alarm operation and maintenance flow of a data room is provided. When people identification abnormality is found by daily monitoring, firstly, alarm types are classified according to abnormality details, for example, people alarm: illegal personnel enter, are worn in an un-standardized way, violate rules in the operation and maintenance process, and the operation and maintenance tasks are not matched with the types of the personnel; then starting an operation and maintenance process, and automatically identifying and monitoring the operation and maintenance process; for example, personnel flow identification 1: people are subjected to intrusion identification, do not wear safety helmets, do not wear work clothes, smoke, do not wear anti-static bracelets and the like; finally, classifying and storing the alarm processing result, for example, after the alarm processing result is an alarm, a person corrects the violation and returns to normal, and the alarm processing result and the corresponding alarm record form an operation and maintenance record and store the operation and maintenance record into an operation and maintenance database; when the equipment abnormality is found by daily monitoring, firstly, the alarm type is classified according to the abnormality details, for example, the fault 1: the server cabinet monitors data abnormity, appearance change and indicator lamp change, and simultaneously automatically identifies and monitors the operation and maintenance process, for example, flow identification 1: the operation and maintenance personnel arrive at the server cabinet for positioning monitoring and maintenance flow monitoring; and finally classifying and storing the alarm processing result, wherein the alarm processing result is a result 1: and the server cabinet monitoring data, the appearance and the indicator lamp are recovered to be normal, and an operation and maintenance record is formed by the alarm processing result and the corresponding alarm record and is stored in the operation and maintenance database.

Referring to fig. 17, a schematic diagram of an electronic device is provided for one embodiment of the present disclosure. As shown in fig. 3, the electronic device 500 includes:

memory 530 and one or more processors 510;

wherein the memory 530 is communicatively coupled to the one or more processors 510, and instructions 532 executable by the one or more processors are stored in the memory 530, and the instructions 532 are executed by the one or more processors 510 to cause the one or more processors 510 to perform the methods of the previous embodiments of the present application.

In particular, processor 510 and memory 530 may be connected by a bus or other means, such as bus 540 in FIG. 13. Processor 510 may be a Central Processing Unit (CPU). The Processor 510 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or combinations thereof.

The memory 530, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as the cascaded progressive network in the embodiments of the present application. The processor 510 performs various functional applications of the processor and data processing by executing non-transitory software programs, instructions, and modules 532 stored in the memory 530.

The memory 530 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor 510, and the like. Further, memory 530 may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 530 may optionally include memory located remotely from processor 510, which may be connected to processor 510 via a network, such as through communication interface 520. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are executed to perform the method in the foregoing embodiment of the present application.

The foregoing computer-readable storage media include physical volatile and nonvolatile, removable and non-removable media implemented in any manner or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. The computer-readable storage medium specifically includes, but is not limited to, a USB flash drive, a removable hard drive, a Read-Only Memory (ROM), a Random Access Memory (RAM), an erasable programmable Read-Only Memory (EPROM), an electrically erasable programmable Read-Only Memory (EEPROM), flash Memory or other solid state Memory technology, a CD-ROM, a Digital Versatile Disk (DVD), an HD-DVD, a Blue-Ray or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer.

While the subject matter described herein is provided in the general context of execution in conjunction with the execution of an operating system and application programs on a computer system, those skilled in the art will recognize that other implementations may also be performed in combination with other types of program modules. Generally, program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types. Those skilled in the art will appreciate that the subject matter described herein may be practiced with other computer system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like, as well as distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Those of ordinary skill in the art will appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to perform all or part of the steps of the method according to the embodiments of the present application.

The machine vision system is used for replacing personnel to carry out routing inspection, so that the labor cost is greatly reduced, and the safe and effective operation of a standardized industrial site is ensured through the omnibearing monitoring of the standardized industrial site and/or industrial equipment; meanwhile, the operation and maintenance process can be automatically monitored; and the combination of the machine vision system and the sensor is utilized to carry out cross validation, thereby realizing higher-level monitoring of operation and maintenance and ensuring the normal operation state of the standardized industrial site and/or the industrial equipment.

It is to be understood that the above-described specific embodiments of the present disclosure are merely illustrative of or illustrative of the principles of the present disclosure and are not to be construed as limiting the present disclosure. Accordingly, any modification, equivalent replacement, improvement or the like made without departing from the spirit and scope of the present disclosure should be included in the protection scope of the present disclosure. Further, it is intended that the following claims cover all such variations and modifications that fall within the scope and bounds of the appended claims, or equivalents of such scope and bounds.

Claims

1. An operation and maintenance monitoring method is characterized by comprising the following steps:

2. The method according to claim 1, wherein the abnormal situation comprises an abnormal situation of a person in the monitoring video/image and/or an abnormal situation of a target object in the monitoring video/image.

3. The method of claim 1, further comprising: establishing a database of standardized industrial places or industrial equipment; and forming an operation and maintenance record by the abnormal condition, the alarm information, the operation and maintenance flow and the generated operation and maintenance result, and updating the operation and maintenance record into a database.

4. The method of claim 3, further comprising: if the abnormal condition is eliminated, the operation and maintenance verification database in the database is linked for verification, and meanwhile, the operation and maintenance knowledge base in the database is linked for classified storage; and if the abnormal condition is not eliminated, continuing to issue the operation and maintenance task until the operation and maintenance are finished.

5. The method of claim 1, wherein the machine vision system can monitor the operating status of the standardized industrial site and/or the industrial equipment in real time by using a trained neural network.

6. The method according to claim 1, wherein the operation and maintenance process specifically comprises: and identifying the running state of the industrial equipment, the equipment state of operation and maintenance personnel, the traveling path of the operation and maintenance personnel, the maintenance and operation specification of the operation and maintenance personnel and the corresponding state of the operation and maintenance personnel and the fault equipment through the machine vision system.

7. The method according to claim 1, further comprising automatically monitoring the result of alarm release and/or the result of operation and maintenance after the alarm release and/or the operation and maintenance are finished.

8. An operation and maintenance monitoring device, comprising:

9. An electronic device, comprising:

a memory and one or more processors;

wherein the memory is communicatively coupled to the one or more processors and has stored therein instructions executable by the one or more processors, the electronic device being configured to implement the method of any of claims 1-7 when the instructions are executed by the one or more processors.

10. A computer-readable storage medium having stored thereon computer-executable instructions operable, when executed by a computing device, to implement the method of any of claims 1-7.