WO2022037281A1 - Real-time substation field operation monitoring and alarm system based on machine vision - Google Patents

Real-time substation field operation monitoring and alarm system based on machine vision Download PDF

Info

Publication number
WO2022037281A1
WO2022037281A1 PCT/CN2021/103959 CN2021103959W WO2022037281A1 WO 2022037281 A1 WO2022037281 A1 WO 2022037281A1 CN 2021103959 W CN2021103959 W CN 2021103959W WO 2022037281 A1 WO2022037281 A1 WO 2022037281A1
Authority
WO
WIPO (PCT)
Prior art keywords
apriltag
icon
real
machine vision
camera
Prior art date
Application number
PCT/CN2021/103959
Other languages
French (fr)
Chinese (zh)
Inventor
朱明增
胡凯博
黄新华
孙春日
覃剑永
岑建军
刘荣洲
张炜
黎华
梁兆庭
周承秀
覃秋勤
刘小兰
周虹妤
丘浩
林翔宇
莫梓樱
陈少暖
黄承伟
陈琴
蒙亮
李永栈
杨芳
韦妙香
陈极万
罗小波
杨波
谢辉
张展声
吕鸣
胡清智
朱敦森
Original Assignee
广西电网有限责任公司贺州供电局
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广西电网有限责任公司贺州供电局 filed Critical 广西电网有限责任公司贺州供电局
Priority to JP2021575905A priority Critical patent/JP2022554044A/en
Publication of WO2022037281A1 publication Critical patent/WO2022037281A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Programme-control systems
    • G05B19/02—Programme-control systems electric
    • G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
    • G05B19/048—Monitoring; Safety
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C11/00—Arrangements, systems or apparatus for checking, e.g. the occurrence of a condition, not provided for elsewhere

Definitions

  • the invention relates to the technical field of safety monitoring, and more particularly, to a real-time monitoring and alarm system for substation field operations based on machine vision.
  • the substation undertakes the tasks of regional voltage level transformation, use and power transmission.
  • on-site operations must strictly follow safety regulations to ensure safe construction in compliance with regulations, and avoid casualties and economic losses caused by illegal construction. Due to the wide area of substation operations and the irregular construction location, fixed cameras cannot cover every scene.
  • the existing video surveillance system only has the functions of collection, storage and playback, and lacks in intelligent analysis and processing capabilities.
  • the purpose of the present invention is to aim at the deficiencies in the prior art, to provide a real-time monitoring and alarm system for substation field operations based on machine vision, using AprilTag image processing technology to determine the working position and work content of electric power workers through positioning, real-time monitoring Worker's work station information.
  • a real-time monitoring and alarm system for substation field operations based on machine vision including:
  • an AprilTag icon is provided on the wearable device, and the AprilTag icon has a unique ID number
  • the processor processes the AprilTag icon on the wearable device based on the AprilTag image processing technology, and identifies the ID number of the icon;
  • the camera is used to collect image information, capture the AprilTag icon on the wearable device, and transmit it to the processor;
  • Voice system the voice system is used for voice prompting of the operation process and operation precautions or to issue a voice alarm signal
  • the communication system is used to upload alarm information and video, and accept the designation of the host computer.
  • the camera includes a plurality of cameras with independent angles, and the range of collecting images covers all workstations in the work area.
  • the wearable device includes insulating gloves, insulating boots, work clothes and safety helmets.
  • a plurality of AprilTag icons are installed on the wearable device, which are respectively arranged on the front, the back or the left and right sides of the wearable device.
  • the processor includes an AprilTag icon encoding library.
  • the processor determines the position and workstation of the operator according to the image information collected by the camera, and controls the voice system to make a relative response according to the determined situation.
  • the determination process includes the following steps:
  • R ij and T k can be calculated. s as a scale factor is unknown. Since the columns of the rotation matrix must be normalized, the size of s can be constrained according to R ij ; the columns of the rotation matrix are orthogonal, and the third column can be based on the cross of the first two columns. Multiply to get; get a rotation matrix and a translation matrix; the translation matrix T x , T y , T z is the coordinate value of the tag relative to the camera center, T X is the lateral offset of the tag center relative to the camera center, T z is the depth of the tag center from the camera;
  • T X and T z values Use the T X and T z values to compare with the station coordinate system in the database to obtain the position and working station information.
  • step S1 also includes comparing the ID value of the AprilTag icon with the ID value of the workstation's dressing requirements in the database, and it is judged whether it is Dress code required.
  • step S1 the camera can be switched to perform image sampling from another angle for comparison.
  • step S1 the voice system sends out a voice alarm signal, and the communication system uploads alarm information and video images.
  • the invention can intelligently analyze and process the collected image information, determine the work position and work content of the electric power workers, and prompt the work flow and work precautions through voice. hidden danger.
  • multiple comparisons can be performed based on omnidirectional cameras, the monitoring system is reliable, and the determination result is accurate.
  • Figure 1 is a wearable device with an AprilTag icon
  • Fig. 2 is a flow chart for determining the working position of the operator
  • Figure 3 is a flow chart for the detection and automatic alarm of workers' dress violations.
  • this embodiment provides a real-time monitoring and alarm system for substation field operations based on machine vision, including:
  • an AprilTag icon is provided on the wearable device, and the AprilTag icon has a unique ID number
  • the processor processes the AprilTag icon on the wearable device based on the AprilTag image processing technology, and identifies the ID number of the icon;
  • the camera is used to collect image information, capture the AprilTag icon on the wearable device, and transmit it to the processor;
  • Voice system the voice system is used for voice prompting of the operation process and operation precautions or to issue a voice alarm signal
  • the communication system is used to upload alarm information and video, and accept the designation of the host computer.
  • a plurality of cameras with independent angles are installed in the working area, and the range of collecting images covers all the workstations in the working area, so as to achieve full coverage of the working area.
  • the wearable equipment used by the operator includes insulating gloves, insulating boots, work clothes and safety helmets.
  • Multiple AprilTag icons are installed on the above-mentioned wearable equipment, which are respectively arranged on the front, back or left and right sides of the wearable equipment.
  • the processor includes the AprilTag icon encoding library, which determines the position and workstation of the operator according to the image information collected by the camera, and controls the voice system to make a relative response according to the judgment.
  • the determination process in this embodiment includes the following steps:
  • R ij and T k can be calculated. s as a scale factor is unknown. Since the columns of the rotation matrix must be normalized, the size of s can be constrained according to R ij ; the columns of the rotation matrix are orthogonal, and the third column can be based on the cross of the first two columns. Multiply to get; get a rotation matrix and a translation matrix; the translation matrix T x , T y , T z is the coordinate value of the tag relative to the camera center, T X is the lateral offset of the tag center relative to the camera center, T z is the depth of the tag center from the camera;
  • T X and T z values Use the T X and T z values to compare with the station coordinate system in the database to obtain the position and working station information, and monitor the situation of the on-site operation.
  • the present embodiment provides a method for determining the working position of an operator.
  • the ID values of the devices that are required to be dressed by different workstations are stored in the database.
  • Step S1 also includes comparing the ID values of the AprilTag icons and the database The ID value of the dress requirements of the station is compared to determine whether it is the required dress.
  • the camera can be switched to another angle to perform image sampling for comparison; after multiple comparisons fail in step S1, the voice system sends a voice alarm signal, and the communication system uploads the alarm information and the video image .

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Alarm Systems (AREA)
  • Emergency Alarm Devices (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

A real-time substation field operation monitoring and alarm system based on machine vision. The system comprises: a wearable device which is provided with an AprilTag icon having a unique ID number; a processor which processes the AprilTag icon on the wearable device on the basis of AprilTag image processing technology, so as to identify the ID number of the icon; a camera, which is used for collecting image information, capturing the AprilTag icon on the wearable device, and transmitting same to the processor; a voice system which is used for giving a voice prompt regarding an operation flow and operation considerations or for sending a voice alarm signal; and a communication system which is used for uploading alarm information and a video, and for accepting the designation by an upper computer. Intelligent analysis and processing can be performed on collected image information, operation space positioning can be performed on an electrical operator, and operation content of the operator and a corresponding operation clothing requirement can be determined by means of spacial positioning, such that potential safety risks are reduced, a monitoring system is reliable, and a determination result is accurate.

Description

一种基于机器视觉的变电现场作业实时监控与报警***A real-time monitoring and alarm system for substation field operations based on machine vision 技术领域technical field
本发明涉及安全监测技术领域，更具体地，涉及一种基于机器视觉的变电现场作业实时监控与报警***。The invention relates to the technical field of safety monitoring, and more particularly, to a real-time monitoring and alarm system for substation field operations based on machine vision.
背景技术Background technique
变电站作为电力***的关键基础设施，承担着地区电压等级变换、使用及输电任务。为了保证变电站的可靠稳定运行，现场作业必须严格遵循安全规范，确保符合规范安全施工，避免违规施工造成的人员伤亡和经济损失。由于变电站作业具有地域广、施工位置不固定等特点，导致固定设置的摄像头无法覆盖各个场景，此外现有视频监控***只具有采集、存储和回放功能，在智能分析和处理能力方面有所欠缺。As the key infrastructure of the power system, the substation undertakes the tasks of regional voltage level transformation, use and power transmission. In order to ensure the reliable and stable operation of the substation, on-site operations must strictly follow safety regulations to ensure safe construction in compliance with regulations, and avoid casualties and economic losses caused by illegal construction. Due to the wide area of substation operations and the irregular construction location, fixed cameras cannot cover every scene. In addition, the existing video surveillance system only has the functions of collection, storage and playback, and lacks in intelligent analysis and processing capabilities.
因此，传统的变电站现场作业还需要通过人工值班员现场巡检来保证安全，而人工方式无法实现全方位无死角管控，现场作业有短时违规行为不易察觉，存在现场作业安全隐患。由于工作场景点多面广，固定摄像头无法覆盖多个业务场景，因此需要研制多维度视觉智能终端弥补固定摄像头不足，实现可控、多维度视频采集与穿透。Therefore, traditional on-site operations of substations also need to ensure safety through on-site inspections by human on-duty personnel. However, manual methods cannot achieve all-round control without dead ends. Short-term violations of on-site operations are difficult to detect, and there are potential safety hazards in on-site operations. Due to the wide variety of work scenes, fixed cameras cannot cover multiple business scenarios. Therefore, it is necessary to develop multi-dimensional visual intelligent terminals to make up for the lack of fixed cameras and achieve controllable, multi-dimensional video capture and penetration.
发明内容SUMMARY OF THE INVENTION
本发明的目的在于针对现有技术中的不足，提供一种基于机器视觉的变电现场作业实时监控与报警***，利用AprilTag图像处理技术，通过定位确定电力作业人员作业位置与作业内容，实时监控作业人员的工位信息。The purpose of the present invention is to aim at the deficiencies in the prior art, to provide a real-time monitoring and alarm system for substation field operations based on machine vision, using AprilTag image processing technology to determine the working position and work content of electric power workers through positioning, real-time monitoring Worker's work station information.
本发明的目的通过以下技术方案实现：The object of the present invention is achieved through the following technical solutions:
一种基于机器视觉的变电现场作业实时监控与报警***，包括：A real-time monitoring and alarm system for substation field operations based on machine vision, including:
穿戴装备，所述穿戴设备上设置有AprilTag图标，所述AprilTag图标具有唯一ID号；Wearing equipment, an AprilTag icon is provided on the wearable device, and the AprilTag icon has a unique ID number;
处理器，所述处理器基于AprilTag图像处理技术对穿戴设备上的AprilTag图标进行处理，识别图标的ID号，；a processor, the processor processes the AprilTag icon on the wearable device based on the AprilTag image processing technology, and identifies the ID number of the icon;
摄像头，所述摄像头用于采集图像信息，捕捉穿戴设备上的AprilTag图标，并传送至处理器；a camera, the camera is used to collect image information, capture the AprilTag icon on the wearable device, and transmit it to the processor;
语音***：所述语音***用于语音提示作业流程与作业注意事项或发出语音 报警信号；Voice system: the voice system is used for voice prompting of the operation process and operation precautions or to issue a voice alarm signal;
通讯***：所述通信***用于上传报警信息与视频，接受上位机指定。Communication system: The communication system is used to upload alarm information and video, and accept the designation of the host computer.
进一步地，所述摄像头包括多个独立角度的摄像头，采集图像的范围覆盖工作区域所有工位。Further, the camera includes a plurality of cameras with independent angles, and the range of collecting images covers all workstations in the work area.
进一步地，所述穿戴设备包括绝缘手套、绝缘靴、工作服和安全帽。Further, the wearable device includes insulating gloves, insulating boots, work clothes and safety helmets.
进一步地，所述穿戴装备上安装有多个AprilTag图标，分别设置在穿戴设备的正面、背面或左右两侧。Further, a plurality of AprilTag icons are installed on the wearable device, which are respectively arranged on the front, the back or the left and right sides of the wearable device.
进一步地，所述处理器中包括AprilTag图标编码库。Further, the processor includes an AprilTag icon encoding library.
进一步地，所述处理器根据摄像头采集的图像信息对作业人员的位置和工位进行判定，并根据判定情况控制语音***做出相对响应。Further, the processor determines the position and workstation of the operator according to the image information collected by the camera, and controls the voice system to make a relative response according to the determined situation.
进一步地，所述判定过程包括以下步骤：Further, the determination process includes the following steps:
S1.将摄像头捕捉的AprilTag图标的ID值与处理器中的AprilTag图标编码库相匹配；S1. Match the ID value of the AprilTag icon captured by the camera with the AprilTag icon encoding library in the processor;
S2.用矩形框将检测到的图像tag部分框出来，得到tag的角点在图像像素坐标系下的坐标，根据tag的实际大小，计算tag各角点的世界坐标系坐标，具体计算公式如下：S2. Use a rectangular frame to frame the detected image tag part to obtain the coordinates of the corner points of the tag in the image pixel coordinate system, and calculate the world coordinate system coordinates of each corner point of the tag according to the actual size of the tag. The specific calculation formula is as follows :
Figure PCTCN2021103959-appb-000001
Figure PCTCN2021103959-appb-000001
Figure PCTCN2021103959-appb-000002
Figure PCTCN2021103959-appb-000002
S3.将第三列去掉，使tag坐标系转化二维坐标系，补充平移矩阵得到E，具体计算公式如下：
Figure PCTCN2021103959-appb-000003
S3. Remove the third column, convert the tag coordinate system into a two-dimensional coordinate system, and supplement the translation matrix to obtain E. The specific calculation formula is as follows:
Figure PCTCN2021103959-appb-000003
Figure PCTCN2021103959-appb-000004
Figure PCTCN2021103959-appb-000004
式中，已知h ij和f x，可以计算出R ij和T k。s作为尺度因子是未知的，由于旋转矩阵的列必须是单位化的，所以可以根据R ij约束s的大小；旋转矩阵的列之间是正交的，第三列可以根据前两列的叉乘得到；得旋转矩阵和平移矩阵；所 述平移矩阵T x,T y,T z即为tag相对于摄像机中心的坐标值，T X为tag中心相对于摄像机中心的横向偏移量，T z为tag中心距离摄像机的深度； In the formula, with known h ij and f x , R ij and T k can be calculated. s as a scale factor is unknown. Since the columns of the rotation matrix must be normalized, the size of s can be constrained according to R ij ; the columns of the rotation matrix are orthogonal, and the third column can be based on the cross of the first two columns. Multiply to get; get a rotation matrix and a translation matrix; the translation matrix T x , T y , T z is the coordinate value of the tag relative to the camera center, T X is the lateral offset of the tag center relative to the camera center, T z is the depth of the tag center from the camera;
S5.利用T X、T z值与数据库中工位坐标系比较，得到位置和作业工位信息。 S5. Use the T X and T z values to compare with the station coordinate system in the database to obtain the position and working station information.
进一步地，所述数据库中储存有不同工位对着装要求的装置的ID值，步骤S1中还包括对AprilTag图标的ID值与数据库中该工位着装需求的ID值进行比对，判断是否是规定的着装。Further, the ID value of the device of different workstations is stored in the database, and step S1 also includes comparing the ID value of the AprilTag icon with the ID value of the workstation's dressing requirements in the database, and it is judged whether it is Dress code required.
进一步地，步骤S1中可切换摄像头换个角度进行图像采样进行比对。Further, in step S1, the camera can be switched to perform image sampling from another angle for comparison.
进一步地，步骤S1中多次比对失败后，所述语音***发出语音报警信号，所述通信***上传报警信息与视频图像。Further, after multiple comparisons fail in step S1, the voice system sends out a voice alarm signal, and the communication system uploads alarm information and video images.
与现有技术相比，本发明的有益效果如下：Compared with the prior art, the beneficial effects of the present invention are as follows:
本发明可对采集的图像信息进行智能化分析和处理，确定电力作业人员作业位置与作业内容，通过语音提示作业流程与作业注意事项，如果作业人员穿戴违规，可及时发出语音报警信号，减少安全隐患。本发明在判定过程中，可依据全方位的摄像头进行多次比对，监控***可靠，判定结果准确。The invention can intelligently analyze and process the collected image information, determine the work position and work content of the electric power workers, and prompt the work flow and work precautions through voice. hidden danger. During the determination process of the present invention, multiple comparisons can be performed based on omnidirectional cameras, the monitoring system is reliable, and the determination result is accurate.
附图说明Description of drawings
图1为设有AprilTag图标的穿戴装置；Figure 1 is a wearable device with an AprilTag icon;
图2为作业人员作业位置确定流程图；Fig. 2 is a flow chart for determining the working position of the operator;
图3为作业人员着装违规检测与自动报警流程图。Figure 3 is a flow chart for the detection and automatic alarm of workers' dress violations.
具体实施方式detailed description
为了便于理解本发明，下文将结合实施例对本发明作更全面、细致地描述，但本发明的保护范围并不限于以下具体的实施例。In order to facilitate the understanding of the present invention, the present invention will be described more comprehensively and in detail below with reference to the embodiments, but the protection scope of the present invention is not limited to the following specific embodiments.
除非另有定义，下文中所使用的所有专业术语与本领域技术人员通常理解的含义相同。本文中所使用的专业术语只是为了描述具体实施例的目的，并不旨在限制本发明的保护范围。Unless otherwise defined, all technical terms used hereinafter have the same meaning as commonly understood by those skilled in the art. The technical terms used herein are only for the purpose of describing specific embodiments, and are not intended to limit the protection scope of the present invention.
除非另有特别说明，本发明中用到的各种原材料、试剂、仪器和设备等均可通过市场购买得到或者可通过现有方法制备得到。Unless otherwise specified, various raw materials, reagents, instruments and equipment used in the present invention can be purchased from the market or can be prepared by existing methods.
实施例1Example 1
如图1～2所示，本实施例提供一种基于机器视觉的变电现场作业实时监控与报警***，包括：As shown in Figures 1-2, this embodiment provides a real-time monitoring and alarm system for substation field operations based on machine vision, including:
穿戴装备，所述穿戴设备上设置有AprilTag图标，所述AprilTag图标具有 唯一ID号；Wearing equipment, an AprilTag icon is provided on the wearable device, and the AprilTag icon has a unique ID number;
处理器，所述处理器基于AprilTag图像处理技术对穿戴设备上的AprilTag图标进行处理，识别图标的ID号，；a processor, the processor processes the AprilTag icon on the wearable device based on the AprilTag image processing technology, and identifies the ID number of the icon;
摄像头，所述摄像头用于采集图像信息，捕捉穿戴设备上的AprilTag图标，并传送至处理器；a camera, the camera is used to collect image information, capture the AprilTag icon on the wearable device, and transmit it to the processor;
语音***：所述语音***用于语音提示作业流程与作业注意事项或发出语音报警信号；Voice system: the voice system is used for voice prompting of the operation process and operation precautions or to issue a voice alarm signal;
通讯***：所述通信***用于上传报警信息与视频，接受上位机指定。Communication system: The communication system is used to upload alarm information and video, and accept the designation of the host computer.
本实施例中，在工作区域安装多个独立角度的摄像头，采集图像的范围覆盖工作区域所有工位，实现对工作区域的全覆盖。作业人员所用的穿戴设备包括绝缘手套、绝缘靴、工作服和安全帽，在上述穿戴装备上安装有多个AprilTag图标，分别设置在穿戴设备的正面、背面或左右两侧。在处理器中包括AprilTag图标编码库，根据摄像头采集的图像信息对作业人员的位置和工位进行判定，并根据判定情况控制语音***做出相对响应。In this embodiment, a plurality of cameras with independent angles are installed in the working area, and the range of collecting images covers all the workstations in the working area, so as to achieve full coverage of the working area. The wearable equipment used by the operator includes insulating gloves, insulating boots, work clothes and safety helmets. Multiple AprilTag icons are installed on the above-mentioned wearable equipment, which are respectively arranged on the front, back or left and right sides of the wearable equipment. The processor includes the AprilTag icon encoding library, which determines the position and workstation of the operator according to the image information collected by the camera, and controls the voice system to make a relative response according to the judgment.
具体地，本实施例中的判定过程包括以下步骤：Specifically, the determination process in this embodiment includes the following steps:
S1.将摄像头捕捉的AprilTag图标的ID值与处理器中的AprilTag图标编码库相匹配；S1. Match the ID value of the AprilTag icon captured by the camera with the AprilTag icon encoding library in the processor;
S2.用矩形框将检测到的图像tag部分框出来，得到tag的角点在图像像素坐标系下的坐标，根据tag的实际大小，计算tag各角点的世界坐标系坐标，具体计算公式如下：S2. Use a rectangular frame to frame the detected image tag part to obtain the coordinates of the corner points of the tag in the image pixel coordinate system, and calculate the world coordinate system coordinates of each corner point of the tag according to the actual size of the tag. The specific calculation formula is as follows :
Figure PCTCN2021103959-appb-000005
Figure PCTCN2021103959-appb-000005
Figure PCTCN2021103959-appb-000006
Figure PCTCN2021103959-appb-000006
S3.将第三列去掉，使tag坐标系转化二维坐标系，补充平移矩阵得到E，具体计算公式如下：
Figure PCTCN2021103959-appb-000007
S3. Remove the third column, convert the tag coordinate system into a two-dimensional coordinate system, and supplement the translation matrix to obtain E. The specific calculation formula is as follows:
Figure PCTCN2021103959-appb-000007
Figure PCTCN2021103959-appb-000008
Figure PCTCN2021103959-appb-000008
式中，已知h ij和f x，可以计算出R ij和T k。s作为尺度因子是未知的，由于旋转矩阵的列必须是单位化的，所以可以根据R ij约束s的大小；旋转矩阵的列之间是正交的，第三列可以根据前两列的叉乘得到；得旋转矩阵和平移矩阵；所述平移矩阵T x,T y,T z即为tag相对于摄像机中心的坐标值，T X为tag中心相对于摄像机中心的横向偏移量，T z为tag中心距离摄像机的深度； In the formula, if h ij and f x are known, R ij and T k can be calculated. s as a scale factor is unknown. Since the columns of the rotation matrix must be normalized, the size of s can be constrained according to R ij ; the columns of the rotation matrix are orthogonal, and the third column can be based on the cross of the first two columns. Multiply to get; get a rotation matrix and a translation matrix; the translation matrix T x , T y , T z is the coordinate value of the tag relative to the camera center, T X is the lateral offset of the tag center relative to the camera center, T z is the depth of the tag center from the camera;
S5.利用T X、T z值与数据库中工位坐标系比较，得到位置和作业工位信息，监测现场作业的情况。 S5. Use the T X and T z values to compare with the station coordinate system in the database to obtain the position and working station information, and monitor the situation of the on-site operation.
实施例2Example 2
本实施例参照实施例1，提供一种作业人员作业位置的确定方法，在数据库中储存有不同工位对着装要求的装置的ID值，步骤S1中还包括对AprilTag图标的ID值与数据库中该工位着装需求的ID值进行比对，判断是否是规定的着装。本实施例中，在判定过程中可切换摄像头换个角度进行图像采样进行比对；步骤S1中多次比对失败后，所述语音***发出语音报警信号，所述通信***上传报警信息与视频图像。Referring to Embodiment 1, the present embodiment provides a method for determining the working position of an operator. The ID values of the devices that are required to be dressed by different workstations are stored in the database. Step S1 also includes comparing the ID values of the AprilTag icons and the database The ID value of the dress requirements of the station is compared to determine whether it is the required dress. In this embodiment, during the determination process, the camera can be switched to another angle to perform image sampling for comparison; after multiple comparisons fail in step S1, the voice system sends a voice alarm signal, and the communication system uploads the alarm information and the video image .
显然，上述实施例仅仅是为清楚地说明本发明的技术方案所作的举例，而并非是对本发明的实施方式的限定。对于所属领域的普通技术人员来说，在上述说明的基础上还可以做出其它不同形式的变化或变动。这里无需也无法对所有的实施方式予以穷举。凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明权利要求的保护范围之内。Obviously, the above-mentioned embodiments are only examples for clearly illustrating the technical solutions of the present invention, but are not intended to limit the embodiments of the present invention. For those of ordinary skill in the art, changes or modifications in other different forms can also be made on the basis of the above description. There is no need and cannot be exhaustive of all implementations here. Any modifications, equivalent replacements and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the claims of the present invention.

Claims (10)

  1. 一种基于机器视觉的变电现场作业实时监控与报警***，其特征在于，包括：A real-time monitoring and alarm system for substation field operations based on machine vision, characterized in that it includes:
    穿戴装备，所述穿戴设备上设置有AprilTag图标，所述AprilTag图标具有唯一ID号；Wearing equipment, an AprilTag icon is provided on the wearable device, and the AprilTag icon has a unique ID number;
    处理器，所述处理器基于AprilTag图像处理技术对穿戴设备上的AprilTag图标进行处理，识别图标的ID号；a processor, wherein the processor processes the AprilTag icon on the wearable device based on the AprilTag image processing technology, and identifies the ID number of the icon;
    摄像头，所述摄像头用于采集图像信息，捕捉穿戴设备上的AprilTag图标，并传送至处理器；a camera, the camera is used to collect image information, capture the AprilTag icon on the wearable device, and transmit it to the processor;
    语音***：所述语音***用于语音提示作业流程与作业注意事项或发出语音报警信号；Voice system: the voice system is used for voice prompting of the operation process and operation precautions or to issue a voice alarm signal;
    通讯***：所述通信***用于上传报警信息与视频，接受上位机指定。Communication system: The communication system is used to upload alarm information and video, and accept the designation of the host computer.
  2. 根据权利要求1所述的基于机器视觉的变电现场作业实时监控与报警***，其特征在于，所述摄像头包括多个独立角度的摄像头，采集图像的范围覆盖工作区域所有工位。The machine vision-based real-time monitoring and alarm system for substation field operations according to claim 1, wherein the camera includes a plurality of cameras with independent angles, and the range of collecting images covers all workstations in the work area.
  3. 根据权利要求1所述的基于机器视觉的变电现场作业实时监控与报警***，其特征在于，所述穿戴设备包括绝缘手套、绝缘靴、工作服和安全帽。The real-time monitoring and alarm system for substation field operations based on machine vision according to claim 1, wherein the wearable device comprises insulating gloves, insulating boots, work clothes and safety helmets.
  4. 根据权利要求1所述的基于机器视觉的变电现场作业实时监控与报警***，其特征在于，所述穿戴装备上安装有多个AprilTag图标，分别设置在穿戴设备的正面、背面或左右两侧。The real-time monitoring and alarm system for substation field operations based on machine vision according to claim 1, wherein a plurality of AprilTag icons are installed on the wearable device, which are respectively arranged on the front, back or left and right sides of the wearable device .
  5. 根据权利要求1所述的基于机器视觉的变电现场作业实时监控与报警***，其特征在于，所述处理器中包括AprilTag图标编码库。The real-time monitoring and alarm system for substation field operations based on machine vision according to claim 1, wherein the processor includes an AprilTag icon encoding library.
  6. 根据权利要求5所述的基于机器视觉的变电现场作业实时监控与报警***，其特征在于，所述处理器根据摄像头采集的图像信息对作业人员的位置和工位进行判定，并根据判定情况控制语音***做出相对响应。The real-time monitoring and alarm system for substation field operations based on machine vision according to claim 5, characterized in that, the processor determines the position and workstation of the operator according to the image information collected by the camera, and determines the position and station according to the determined situation. Control the voice system to respond relatively.
  7. 根据权利要求6所述的基于机器视觉的变电现场作业实时监控与报警***，其特征在于，所述判定过程包括以下步骤：The real-time monitoring and alarm system for substation field operations based on machine vision according to claim 6, wherein the determination process comprises the following steps:
    S1.将摄像头捕捉的AprilTag图标的ID值与处理器中的AprilTag图标编码库相匹配；S1. Match the ID value of the AprilTag icon captured by the camera with the AprilTag icon encoding library in the processor;
    S2.用矩形框将检测到的图像tag部分框出来，得到tag的角点在图像像素坐 标系下的坐标，根据tag的实际大小，计算tag各角点的世界坐标系坐标，具体计算公式如下：S2. Use a rectangular frame to frame the detected image tag part to obtain the coordinates of the corner points of the tag in the image pixel coordinate system, and calculate the world coordinate system coordinates of each corner point of the tag according to the actual size of the tag. The specific calculation formula is as follows :
    Figure PCTCN2021103959-appb-100001
    Figure PCTCN2021103959-appb-100001
    Figure PCTCN2021103959-appb-100002
    Figure PCTCN2021103959-appb-100002
    S3.将第三列去掉，使tag坐标系转化二维坐标系，补充平移矩阵得到E，具体计算公式如下：
    Figure PCTCN2021103959-appb-100003
    S3. Remove the third column, convert the tag coordinate system into a two-dimensional coordinate system, and supplement the translation matrix to obtain E. The specific calculation formula is as follows:
    Figure PCTCN2021103959-appb-100003
    Figure PCTCN2021103959-appb-100004
    Figure PCTCN2021103959-appb-100004
    式中，已知h ij和f x，可以计算出R ij和T k。s作为尺度因子是未知的，由于旋转矩阵的列必须是单位化的，所以可以根据R ij约束s的大小；旋转矩阵的列之间是正交的，第三列可以根据前两列的叉乘得到；得旋转矩阵和平移矩阵；所述平移矩阵T x,T y,T z即为tag相对于摄像机中心的坐标值，T X为tag中心相对于摄像机中心的横向偏移量，T z为tag中心距离摄像机的深度； In the formula, with known h ij and f x , R ij and T k can be calculated. s as a scale factor is unknown. Since the columns of the rotation matrix must be normalized, the size of s can be constrained according to R ij ; the columns of the rotation matrix are orthogonal, and the third column can be based on the cross of the first two columns. Multiply to get; get a rotation matrix and a translation matrix; the translation matrix T x , T y , T z is the coordinate value of the tag relative to the camera center, T X is the lateral offset of the tag center relative to the camera center, T z is the depth of the tag center from the camera;
    S5.利用T X、T z值与数据库中工位坐标系比较，得到位置和作业工位信息。 S5. Use the T X and T z values to compare with the station coordinate system in the database to obtain the position and working station information.
  8. 根据权利要求7所述的基于机器视觉的变电现场作业实时监控与报警***，其特征在于，所述数据库中储存有不同工位对着装要求的装置的ID值，步骤S1中还包括对AprilTag图标的ID值与数据库中该工位着装需求的ID值进行比对，判断是否是规定的着装。The real-time monitoring and alarm system for substation field operations based on machine vision according to claim 7, wherein the database stores the ID values of the devices required by different workstations for dressing, and the step S1 also includes an AprilTag The ID value of the icon is compared with the ID value of the station's dress requirements in the database to determine whether it is the prescribed dress.
  9. 根据权利要求8所述的基于机器视觉的变电现场作业实时监控与报警***，其特征在于，步骤S1中可切换摄像头换个角度进行图像采样进行比对。The machine vision-based real-time monitoring and alarm system for substation field operations according to claim 8, characterized in that, in step S1, the camera can be switched from another angle to perform image sampling for comparison.
  10. 根据权利要求9所述的基于机器视觉的变电现场作业实时监控与报警***，其特征在于，步骤S1中多次比对失败后，所述语音***发出语音报警信号，所述通信***上传报警信息与视频图像。The machine vision-based real-time monitoring and alarm system for substation field operations according to claim 9, wherein after multiple comparisons fail in step S1, the voice system sends out a voice alarm signal, and the communication system uploads an alarm Information and video images.
PCT/CN2021/103959 2020-08-19 2021-07-01 Real-time substation field operation monitoring and alarm system based on machine vision WO2022037281A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2021575905A JP2022554044A (en) 2020-08-19 2021-07-01 Real-time work monitoring and alarm system at substation site based on machine vision

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010837588 2020-08-19
CN202010837588.1 2020-08-19

Publications (1)

Publication Number Publication Date
WO2022037281A1 true WO2022037281A1 (en) 2022-02-24

Family

ID=80322525

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/103959 WO2022037281A1 (en) 2020-08-19 2021-07-01 Real-time substation field operation monitoring and alarm system based on machine vision

Country Status (2)

Country Link
JP (1) JP2022554044A (en)
WO (1) WO2022037281A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114998271A (en) * 2022-06-10 2022-09-02 海南浙江大学研究院 Transformer substation high tension switchgear secondary bin panel image recognition system
CN115497268A (en) * 2022-09-19 2022-12-20 中铁第四勘察设计院集团有限公司 Personnel positioning management system based on intelligent construction site and application
CN115567690A (en) * 2022-09-22 2023-01-03 国网山东省电力公司莒县供电公司 Intelligent monitoring system capable of automatically identifying dangerous points of field operation

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070097211A1 (en) * 2003-01-02 2007-05-03 Covi Technologies, Inc. Systems and methods for location of objects
CN104318732A (en) * 2014-10-27 2015-01-28 国网冀北电力有限公司张家口供电公司 Transformer substation field worker monitoring and management system and method based on video analysis and RFID
CN105844846A (en) * 2016-04-19 2016-08-10 唐山新质点科技有限公司 Safety monitoring method, device and system based on Ibeacon technology
CN105957169A (en) * 2016-04-19 2016-09-21 唐山新质点科技有限公司 Detection method, apparatus and system based on iBeacon technology
WO2017167813A1 (en) * 2016-03-30 2017-10-05 Koninklijke Philips N.V. An arm position tracking system and method for use during a shoulder flexion exercise
CN107544326A (en) * 2017-08-30 2018-01-05 深圳市易联智道科技有限公司 Ensure the multifunction monitoring system of construction site security
CN109189073A (en) * 2018-09-28 2019-01-11 华南理工大学 A kind of robot and patrol method based on apriltag positioning

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011227619A (en) * 2010-04-16 2011-11-10 Chugoku Electric Power Co Inc:The Area management system
US10699147B2 (en) * 2017-06-22 2020-06-30 BITO Robotics, Inc. Systems and methods for fast identification and processing of an image area of interest
CN107240063B (en) * 2017-07-04 2020-05-26 武汉大学 Autonomous take-off and landing method of rotor unmanned aerial vehicle facing mobile platform
US11113819B2 (en) * 2019-01-15 2021-09-07 Nvidia Corporation Graphical fiducial marker identification suitable for augmented reality, virtual reality, and robotics
CN111273679A (en) * 2020-02-14 2020-06-12 南京航空航天大学 Visual-guided network-collision recovery longitudinal guidance method for small fixed-wing unmanned aerial vehicle

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070097211A1 (en) * 2003-01-02 2007-05-03 Covi Technologies, Inc. Systems and methods for location of objects
CN104318732A (en) * 2014-10-27 2015-01-28 国网冀北电力有限公司张家口供电公司 Transformer substation field worker monitoring and management system and method based on video analysis and RFID
WO2017167813A1 (en) * 2016-03-30 2017-10-05 Koninklijke Philips N.V. An arm position tracking system and method for use during a shoulder flexion exercise
CN105844846A (en) * 2016-04-19 2016-08-10 唐山新质点科技有限公司 Safety monitoring method, device and system based on Ibeacon technology
CN105957169A (en) * 2016-04-19 2016-09-21 唐山新质点科技有限公司 Detection method, apparatus and system based on iBeacon technology
CN107544326A (en) * 2017-08-30 2018-01-05 深圳市易联智道科技有限公司 Ensure the multifunction monitoring system of construction site security
CN109189073A (en) * 2018-09-28 2019-01-11 华南理工大学 A kind of robot and patrol method based on apriltag positioning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LI YANG, XIN JING;JIAYU GAO: "Research on Visual Localization based AprilTag System", PROCEEDINGS OF THE 10TH ANNUAL CHINA SATELLITE NAVIGATION CONFERENCE, S10 PNT SYSTEM AND MULTI-SOURCE FUSION NAVIGATION, 22 May 2019 (2019-05-22), pages 55 - 59, XP055901469 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114998271A (en) * 2022-06-10 2022-09-02 海南浙江大学研究院 Transformer substation high tension switchgear secondary bin panel image recognition system
CN115497268A (en) * 2022-09-19 2022-12-20 中铁第四勘察设计院集团有限公司 Personnel positioning management system based on intelligent construction site and application
CN115567690A (en) * 2022-09-22 2023-01-03 国网山东省电力公司莒县供电公司 Intelligent monitoring system capable of automatically identifying dangerous points of field operation

Also Published As

Publication number Publication date
JP2022554044A (en) 2022-12-28

Similar Documents

Publication Publication Date Title
WO2022037281A1 (en) Real-time substation field operation monitoring and alarm system based on machine vision
US10445887B2 (en) Tracking processing device and tracking processing system provided with same, and tracking processing method
CN110850723B (en) Fault diagnosis and positioning method based on transformer substation inspection robot system
CN110650316A (en) Intelligent patrol and early warning processing method and device, electronic equipment and storage medium
CN110874866B (en) Video-based three-dimensional monitoring method and system for transformer substation
CN106791700B (en) Enterprise critical area personnel path safety monitoring system and method
CN107368014A (en) Machine room monitoring system and method
CN109409395A (en) Using the method for template matching method identification target object region electrical symbol in power monitoring
CN113066195A (en) Power equipment inspection method and device, AR glasses and storage medium
WO2016143976A1 (en) Method for recognizing operator in work site image data
CN115880722A (en) Intelligent identification method, system and medium worn by power distribution operating personnel
CN106744332A (en) A kind of visual crane hangs and takes alignment system
CN115620192A (en) Method and device for detecting wearing of safety rope in aerial work
CN108460357B (en) Windowing alarm detection system and method based on image recognition
CN113627321A (en) Image identification method and device based on artificial intelligence and computer equipment
CN111523423B (en) Power equipment identification method and device
CN114007047B (en) Machine vision-based real-time monitoring and alarming system for operation of power transformation site
CN116311034A (en) Robot inspection system based on contrast detection
CN115937983A (en) Patient falling detection method and system based on multi-view depth image
CN115690687A (en) Safe wearing standard detection system based on deep learning technology
CN213457742U (en) Welding operation monitoring system
CN105376523A (en) Stereoscopic vision detection method and system
JPWO2022037281A5 (en)
CN114821444A (en) Unmanned overhead traveling crane operation area safety detection method based on visual perception
CN113920535A (en) Electronic region detection method based on YOLOv5

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2021575905

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21857377

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21857377

Country of ref document: EP

Kind code of ref document: A1