CN114612825B - Target detection method based on edge equipment - Google Patents

Target detection method based on edge equipment Download PDF

Info

Publication number
CN114612825B
CN114612825B CN202210230959.9A CN202210230959A CN114612825B CN 114612825 B CN114612825 B CN 114612825B CN 202210230959 A CN202210230959 A CN 202210230959A CN 114612825 B CN114612825 B CN 114612825B
Authority
CN
China
Prior art keywords
deep learning
detection
target detection
model
learning frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210230959.9A
Other languages
Chinese (zh)
Other versions
CN114612825A (en
Inventor
何臻力
索珈顺
王汝欣
何婧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yunnan University YNU
Original Assignee
Yunnan University YNU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yunnan University YNU filed Critical Yunnan University YNU
Priority to CN202210230959.9A priority Critical patent/CN114612825B/en
Publication of CN114612825A publication Critical patent/CN114612825A/en
Application granted granted Critical
Publication of CN114612825B publication Critical patent/CN114612825B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a target detection method based on edge equipment, which comprises the steps of selecting a target detection model according to actual needs, setting detection time, detection power consumption and detection precision priority, setting detection time upper limit, acquiring deep learning frames supporting different model input resolutions for the edge equipment needing target detection, forming a deep learning frame set, acquiring inference time and average power consumption of each deep learning frame, optimizing the deep learning frames based on the detection time upper limit and three performance priorities, optimizing and configuring the target detection model according to the screened deep learning frames, training the target detection model, and then deploying the target detection model on the edge equipment for target detection. According to the invention, the deep learning framework of the target detection model on the edge equipment is optimized, and the detection time, the detection power consumption and the detection precision of the target detection model are comprehensively considered, so that the performance of target detection is improved.

Description

Target detection method based on edge equipment
Technical Field
The invention belongs to the technical field of edge calculation, and particularly relates to a target detection method based on edge equipment.
Background
The scheme belongs to the field of edge calculation. Currently, many AI algorithms are deployed on edge devices to achieve the goals of latency reduction, bandwidth saving, and privacy protection. The target detection algorithm is widely applied to the fields of industrial production, city monitoring, pedestrian detection and the like. It would be of great value to design a target detection method that would perform well on low cost edge devices.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a target detection method based on edge equipment, which improves the performance of target detection by optimizing a deep learning frame of a target detection model on the edge equipment.
In order to achieve the above object, the edge device-based object detection method of the present invention includes the steps of:
s1: selecting a target detection model according to actual requirements, and then setting the priority of three detection performance indexes of detection time, detection power consumption and detection precision of the target detection model and the upper limit T of the detection time of the target detection model;
s2: for edge equipment needing target detection, firstly determining deep learning frames supported by the edge equipment, and recording the number of the supported deep learning frames as N; obtaining the input resolutions of the models supported by the target detection models when the target detection models run on the edge equipment and are used for each deep learning framework, and recording the number M of the input resolutions of the models supported by the nth deep learning framework n N=1, 2, …, N, the nth deep learning frame supporting the mth model input resolution is denoted as f n,m ,m=1,2,…,M n All deep learning frames f n,m Forming a deep learning frame set F, and then acquiring each deep learning frame F n,m Time of inference at edge device t n,m And average power consumption w n,m
S3: for the deep learning framework set F, the time t is inferred n,m Deleting the deep learning frames exceeding the upper limit T of the detection time to obtain a deep learning frame set F' after preliminary screening;
if the highest priority in the detection performance indexes is the detection time, the deep learning frame with the least detection time in the deep learning frame set F' is taken as the deep learning frame to be usedTaking the input resolution of the corresponding model as the input resolution of the target detection model +.>If the number of the deep learning frames is greater than 1, continuing to screen according to the detection precision and the priority of the detection power consumption to obtain an optimal deep learning frame;
if the highest priority in the detection performance indexes is the detection power consumption, the deep learning frame with the least average power consumption in the deep learning frame set F' is taken as the deep learning frame to be usedTaking the input resolution of the corresponding model as the input resolution of the target detection model +.>If the number of the deep learning frames is greater than 1, continuing to screen according to the detection precision and the priority of the detection time to obtain an optimal deep learning frame;
if the highest priority among the detection performance indexes is the detection accuracy, the deep learning frame with the largest model input resolution in the deep learning frame set F' is taken as the deep learning frame usedTaking the input resolution of the corresponding model as the input resolution of the target detection model +.>If the number of the deep learning frames is greater than 1, continuing to screen according to the detection time and the priority of the detection power consumption to obtain an optimal deep learning frame;
s4: deep learning frame obtained through screening in step S3And input resolution of the object detection model +.>Optimally configuring a target detection model, and then collecting training samples to train the target detection model;
s5: and (3) deploying the target detection model trained in the step (S4) on edge equipment, and carrying out target detection on the video image acquired by the camera.
According to the target detection method based on the edge equipment, a target detection model is selected according to actual requirements, the detection time, the detection power consumption and the detection precision priority are set, for the edge equipment needing target detection, a deep learning frame supporting different model input resolutions is obtained to form a deep learning frame set, the deducing time and the average power consumption of each deep learning frame are obtained, the deep learning frame is optimized based on the detection time upper limit and the three performance priorities, the target detection model is optimally configured according to the deep learning frame obtained through screening, and the target detection model is deployed on the edge equipment after training.
According to the invention, the deep learning framework of the target detection model on the edge equipment is optimized, and the detection time, the detection power consumption and the detection precision of the target detection model are comprehensively considered, so that the performance of target detection is improved.
Drawings
FIG. 1 is a flow chart of an embodiment of an edge device-based object detection method of the present invention;
FIG. 2 is a schematic diagram of a model input resolution preference supported by a deep learning framework.
Detailed Description
The following description of the embodiments of the invention is presented in conjunction with the accompanying drawings to provide a better understanding of the invention to those skilled in the art. It is to be expressly noted that in the description below, detailed descriptions of known functions and designs are omitted here as perhaps obscuring the present invention.
Examples
Fig. 1 is a flowchart of an embodiment of an edge device-based object detection method of the present invention. As shown in fig. 1, the method for detecting an object based on an edge device of the present invention comprises the following specific steps:
s101: setting a target detection model:
selecting a target detection model according to actual requirements, and then setting the priority of three detection performance indexes of detection time, detection power consumption and detection precision of the target detection model and the upper limit T of the detection time of the target detection model;
s102: determining a deep learning framework supported by the edge device:
for the edge device needing target detection, firstly determining the deep learning frames supported by the edge device, and recording the number of the supported deep learning frames as N. Obtaining the input resolutions of the models supported by the target detection models when the target detection models run on the edge equipment and are used for each deep learning framework, and recording the number M of the input resolutions of the models supported by the nth deep learning framework n N=1, 2, …, N, the nth deep learning frame supporting the mth model input resolution is denoted as f n,m ,m=1,2,…,M n All deep learning frames f n,m Forming a deep learning frame set F, and then acquiring each deep learning frame F n,m Time of inference at edge device t n,m And average power consumption w n,m
The invention aims at a target detection method based on deep learning, and before starting a deep learning project, the selection of a proper framework is very important, because the selection of the proper framework can play a role in half effort. Therefore, before the edge device is started to detect the target, the invention firstly determines the deep learning framework supported by the edge device. Currently, there are more popular deep learning frameworks of PaddlePaddle, tensorflow, caffe, theano, MXNet, torch, pyTorch, tensorRT 16-bit and TensoorRT 32-bit.
When determining the model input resolution supported by the deep learning frame, since the resolution of the image acquired by the camera is generally greater than the model input resolution, scaling is required, how to make the scaled acquired image contain more effective information as much as possible, and primary screening can be performed on the model input resolution supported by the deep learning frame according to the resolution of the image acquired by the camera, that is, the supported model input resolution of the deep learning frame selects the resolution with the aspect ratio closest to the aspect ratio of the resolution of the image acquired by the camera, that is, the resolution with the difference between the ratios smaller than the preset threshold. FIG. 2 is a schematic diagram of a model input resolution preference supported by a deep learning framework. As shown in fig. 2, assuming that the size ratio of the camera captured images is 4:3, the model input resolution supported by the deep learning framework is 416×416 (1:1), 416×320 (13:10), 416×256 (13:8). As can be seen, when the model input resolution is 416×416, the scaled camera acquires an image as shown in fig. 2 (a), the image is filled up and down by solid color blocks, and the actual effective resolution of the image is 416×312; when the model input resolution is 416×320, the scaled camera acquired image is fig. 2 (b), and the actual effective resolution of the image is 416×312; when the model input resolution is 416×256, the scaled camera capture image is fig. 2 (c), and the effective resolution of the image is reduced to 341×256. The accuracy of the different inputs is measured with the visdrop dataset assuming the target detection model is the YOLOv3 model. Table 1 is a table of statistics of detection accuracy after scaling of different model input resolutions in this embodiment.
Model input resolution AP precision (%) AP50 precision (%) AP75 precision (%)
416x416 8 18.39 6.02
416x320 8.05 18.49 5.99
416x256 7.65 17.59 5.63
TABLE 1
As shown in table 1, there was little difference in accuracy between 1:1 resolution 416 x 416 and nearest 4:3 resolution 416 x 320, while accuracy was reduced when inferred at nearest 16:9 resolution 416 x 256. And the calculated amount of resolution 416 x 320 is smaller than resolution 416 x 416. Therefore, when the aspect ratio of the camera acquisition image is closest to the aspect ratio of the input resolution of the model, the scaling of the camera acquisition image can ensure that the accuracy is not reduced and the calculation amount of the model is reduced, so that the estimation speed is increased.
In this embodiment, the edge device is a GPU-based edge device Jetson Nano, and the supported deep learning frameworks include TensorFlow, pyTorch, tensoorRT-bit and TensoorRT 32-bit. Table 2 is an evaluation index of the ten flow framework on the edge device Jetson Nano in this embodiment.
TABLE 2
Table 3 shows the evaluation index of the pyretch framework on the edge device Jetson Nano in this embodiment.
TABLE 3 Table 3
Table 4 shows the evaluation index of the TensoorRT 16-bit framework on the edge device Jetson Nano in this example.
TABLE 4 Table 4
Table 5 shows the evaluation index of the TensoorRT 32-bit framework on the edge device Jetson Nano in this example.
TABLE 5
S103: determining an optimization mode of a target detection model:
in the invention, the target detection model is optimized from two aspects of an inference frame and input picture resolution, so that the performance of the target detection model can reach the expected as much as possible, and the specific method comprises the following steps:
for the deep learning framework set F, the time t is inferred n,m And deleting the deep learning frames exceeding the upper limit T of the detection time to obtain a deep learning frame set F' after preliminary screening.
If the highest priority in the detection performance indexes is the detection time, the deep learning frame with the least detection time in the deep learning frame set F' is taken as the deep learning frame to be usedThe input resolution of the corresponding model is used as the input of the target detection modelResolution of->If the number of the deep learning frames is greater than 1, continuing to screen according to the detection precision and the priority of the detection power consumption to obtain the optimal deep learning frames.
If the highest priority in the detection performance indexes is the detection power consumption, the deep learning frame with the least average power consumption in the deep learning frame set F' is taken as the deep learning frame to be usedTaking the input resolution of the corresponding model as the input resolution of the target detection model +.>If the number of the deep learning frames is greater than 1, continuing to screen according to the detection precision and the priority of the detection time to obtain the optimal deep learning frames.
If the highest priority among the detection performance indexes is the detection accuracy, the deep learning frame with the largest model input resolution in the deep learning frame set F' is taken as the deep learning frame usedTaking the input resolution of the corresponding model as the input resolution of the target detection model +.>This is because the greater the input resolution, the higher the detection accuracy in the neural network model. If the number of the deep learning frames is greater than 1, continuing to screen according to the detection time and the priority of the detection power consumption to obtain the optimal deep learning frames.
S104: optimizing a target detection model:
and (3) optimally configuring the target detection model according to the target detection model optimization mode determined in the step (S103), and then collecting training samples to train the target detection model.
S105: and (3) target detection:
and (3) deploying the target detection model trained in the step (S104) on edge equipment, and carrying out target detection on the video image acquired by the camera.
Besides the edge device Jetson Nano in the embodiment, experiments are also performed on the edge device Jetson Xavier NX, and the experiments show that the invention can well operate on both edge devices.
While the foregoing describes illustrative embodiments of the present invention to facilitate an understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the embodiments, but is to be construed as protected by the accompanying claims insofar as various changes are within the spirit and scope of the present invention as defined and defined by the appended claims.

Claims (2)

1. An edge device-based target detection method is characterized by comprising the following steps:
s1: selecting a target detection model according to actual requirements, and then setting the priority of three detection performance indexes of detection time, detection power consumption and detection precision of the target detection model and the upper limit T of the detection time of the target detection model;
s2: for edge equipment needing target detection, firstly determining deep learning frames supported by the edge equipment, and recording the number of the supported deep learning frames as N; obtaining the input resolutions of the models supported by the target detection models when the target detection models run on the edge equipment and are used for each deep learning framework, and recording the number M of the input resolutions of the models supported by the nth deep learning framework n N=1, 2, …, N, the nth deep learning frame supporting the mth model input resolution is denoted as f n,m ,m=1,2,…,M n All deep learning frames f n,m Forming a deep learning frame set F, and then acquiring each deep learning frame F n,m At edge devicesTime t is inferred n,m And average power consumption w n,m
S3: for the deep learning framework set F, the time t is inferred n,m Deleting the deep learning frames exceeding the upper limit T of the detection time to obtain a deep learning frame set F' after preliminary screening;
if the highest priority in the detection performance indexes is the detection time, the deep learning frame with the least detection time in the deep learning frame set F' is taken as the deep learning frame to be usedTaking the input resolution of the corresponding model as the input resolution of the target detection model +.>If the number of the deep learning frames is greater than 1, continuing to screen according to the detection precision and the detection power consumption preference level to obtain an optimal deep learning frame;
if the highest priority in the detection performance indexes is the detection power consumption, the deep learning frame with the least average power consumption in the deep learning frame set F' is taken as the deep learning frame to be usedTaking the input resolution of the corresponding model as the input resolution of the target detection model +.>If the number of the deep learning frames is greater than 1, continuing to screen according to the detection precision and the priority of the detection time to obtain an optimal deep learning frame;
if the highest priority among the detection performance indexes is the detection accuracy, the deep learning frame with the largest model input resolution in the deep learning frame set F' is taken as the deep learning frame usedTaking the input resolution of the corresponding model as the input resolution of the target detection model +.>If the number of the deep learning frames is greater than 1, continuing to screen according to the detection time and the priority of the detection power consumption to obtain an optimal deep learning frame;
s4: deep learning frame obtained through screening in step S3And input resolution of the object detection model +.>Optimally configuring a target detection model, and then collecting training samples to train the target detection model;
s5: and (3) deploying the target detection model trained in the step (S4) on edge equipment, and carrying out target detection on the video image acquired by the camera.
2. The method according to claim 1, wherein the supported model input resolution of the deep learning framework in step S1 is selected from resolutions having a difference between an aspect ratio and an aspect ratio of the resolution of the image acquired by the camera less than a preset threshold.
CN202210230959.9A 2022-03-09 2022-03-09 Target detection method based on edge equipment Active CN114612825B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210230959.9A CN114612825B (en) 2022-03-09 2022-03-09 Target detection method based on edge equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210230959.9A CN114612825B (en) 2022-03-09 2022-03-09 Target detection method based on edge equipment

Publications (2)

Publication Number Publication Date
CN114612825A CN114612825A (en) 2022-06-10
CN114612825B true CN114612825B (en) 2024-03-19

Family

ID=81860641

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210230959.9A Active CN114612825B (en) 2022-03-09 2022-03-09 Target detection method based on edge equipment

Country Status (1)

Country Link
CN (1) CN114612825B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116055778B (en) * 2022-05-30 2023-11-21 荣耀终端有限公司 Video data processing method, electronic device and readable storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112257500A (en) * 2020-09-16 2021-01-22 江苏方天电力技术有限公司 Intelligent image recognition system and method for power equipment based on cloud edge cooperation technology
AU2020103494A4 (en) * 2020-11-17 2021-01-28 China University Of Mining And Technology Handheld call detection method based on lightweight target detection network
CN113158794A (en) * 2021-03-16 2021-07-23 西安天和防务技术股份有限公司 Object detection method, edge device, and computer-readable storage medium
CN113159166A (en) * 2021-04-19 2021-07-23 国网山东省电力公司威海供电公司 Embedded image identification detection method, system, medium and equipment based on edge calculation
WO2021189507A1 (en) * 2020-03-24 2021-09-30 南京新一代人工智能研究院有限公司 Rotor unmanned aerial vehicle system for vehicle detection and tracking, and detection and tracking method
CN113610024A (en) * 2021-08-13 2021-11-05 天津大学 Multi-strategy deep learning remote sensing image small target detection method
WO2021238826A1 (en) * 2020-05-26 2021-12-02 苏宁易购集团股份有限公司 Method and apparatus for training instance segmentation model, and instance segmentation method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021189507A1 (en) * 2020-03-24 2021-09-30 南京新一代人工智能研究院有限公司 Rotor unmanned aerial vehicle system for vehicle detection and tracking, and detection and tracking method
WO2021238826A1 (en) * 2020-05-26 2021-12-02 苏宁易购集团股份有限公司 Method and apparatus for training instance segmentation model, and instance segmentation method
CN112257500A (en) * 2020-09-16 2021-01-22 江苏方天电力技术有限公司 Intelligent image recognition system and method for power equipment based on cloud edge cooperation technology
AU2020103494A4 (en) * 2020-11-17 2021-01-28 China University Of Mining And Technology Handheld call detection method based on lightweight target detection network
CN113158794A (en) * 2021-03-16 2021-07-23 西安天和防务技术股份有限公司 Object detection method, edge device, and computer-readable storage medium
CN113159166A (en) * 2021-04-19 2021-07-23 国网山东省电力公司威海供电公司 Embedded image identification detection method, system, medium and equipment based on edge calculation
CN113610024A (en) * 2021-08-13 2021-11-05 天津大学 Multi-strategy deep learning remote sensing image small target detection method

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
路艳巧 ; 孙翠英 ; 曹红卫 ; 闫红伟 ; .基于边缘计算与深度学习的输电设备异物检测方法.中国电力.(06),31-37. *
陈佳林 ; 和青 ; 李云波 ; 潘志松 ; .基于边缘计算和深度学习的园区目标检测方法.电子技术与软件工程.2020,(16),152-154. *

Also Published As

Publication number Publication date
CN114612825A (en) 2022-06-10

Similar Documents

Publication Publication Date Title
CN110929569B (en) Face recognition method, device, equipment and storage medium
TWI395483B (en) Motion object detection method using adaptive background model and computer program product thereof
US10609355B2 (en) Dynamically adjusting sampling of a real-time depth map
TWI543610B (en) Electronic device and image selection method thereof
TWI399703B (en) Forward and backward resizing method
WO2020177723A1 (en) Image processing method, night photographing method, image processing chip and aerial camera
CN114612825B (en) Target detection method based on edge equipment
CN111242127A (en) Vehicle detection method with granularity level multi-scale characteristics based on asymmetric convolution
WO2017113917A1 (en) Imaging method, imaging apparatus, and terminal
US20150187051A1 (en) Method and apparatus for estimating image noise
JP2008236389A5 (en)
WO2013185695A1 (en) Method and device for processing mobile terminal images
WO2021047492A1 (en) Target tracking method, device, and computer system
WO2023155482A1 (en) Identification method and system for quick gathering behavior of crowd, and device and medium
CN101690160A (en) Methods, systems and apparatuses for motion detection using auto-focus statistics
CN102984463A (en) Method and system for image processing
CN103020580B (en) Fast face detecting method
CN104809706A (en) Single lens computational imaging method based on gentle image color change priori
US20100061650A1 (en) Method And Apparatus For Providing A Variable Filter Size For Providing Image Effects
CN114885074B (en) Event camera denoising method based on space-time density
CN104639714A (en) Test method of mobile phone response time
US8467630B2 (en) Image processing device and processing method for generating a super-resolution image
CN103096117A (en) Video noise detecting method and device
WO2017143654A1 (en) Method for selecting photo to be outputted, photographing method, device and storage medium
CN103856729A (en) Image processing device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant