CN117950355B - Reconnaissance unmanned aerial vehicle supervision control system and reconnaissance unmanned aerial vehicle supervision control method - Google Patents
Reconnaissance unmanned aerial vehicle supervision control system and reconnaissance unmanned aerial vehicle supervision control method Download PDFInfo
- Publication number
- CN117950355B CN117950355B CN202410354463.1A CN202410354463A CN117950355B CN 117950355 B CN117950355 B CN 117950355B CN 202410354463 A CN202410354463 A CN 202410354463A CN 117950355 B CN117950355 B CN 117950355B
- Authority
- CN
- China
- Prior art keywords
- information
- task
- task execution
- operator
- suspected target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 14
- 238000004891 communication Methods 0.000 claims abstract description 17
- 230000003993 interaction Effects 0.000 claims description 70
- 230000004424 eye movement Effects 0.000 claims description 8
- 238000012216 screening Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
- G05B19/042—Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
- G05B19/0428—Safety, monitoring
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/20—Pc systems
- G05B2219/24—Pc safety
- G05B2219/24024—Safety, surveillance
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention relates to the field of control systems, in particular to a reconnaissance unmanned aerial vehicle supervision control system and a reconnaissance unmanned aerial vehicle supervision control method. The system comprises a target identification module, an autonomous agent module and a user interface module; the input end of the target identification module is in communication connection with an external unmanned aerial vehicle; the output end of the target identification module is respectively connected with the input end of the autonomous agent module and the first input end of the user interface module; the output end of the autonomous agent module is connected with the second input end of the user interface module; the user interface module is used for displaying the return image of the unmanned aerial vehicle and the suspected target return image respectively. According to the invention, the target recognition module is used for screening the returned image of the unmanned aerial vehicle to obtain the suspected target returned image, the autonomous agent module is used for further processing the suspected target returned image, and finally the returned image of the unmanned aerial vehicle and the suspected target returned image are respectively displayed through the user interface, so that the number of images required to be processed by an operator is reduced.
Description
Technical Field
The invention relates to a control system and a control method, in particular to a reconnaissance unmanned aerial vehicle supervision control system and a supervision control method based on the same.
Background
Along with the continuous improvement of the autonomous level of the unmanned aerial vehicle system, the roles of unmanned aerial vehicle operators are changed into high-level supervision control from bottom control, and the conventional common unmanned aerial vehicle cluster control environments comprise aerostats, patrol, emergency rescue and the like, and all face the requirements of a single operator for managing a plurality of unmanned aerial vehicles. With the help of automatic navigation, when a task management loop adopts a consent management mode, the maximum number of unmanned aerial vehicles which can be managed by an operator is 4-5; when the mission management loop adopts a higher autonomous mode-an exception management mode, the maximum number of drones that the operator can manage will increase to 8-12 frames.
As the number of unmanned aerial vehicles managed by the operator increases, the number of unmanned aerial vehicle return images that the operator needs to process increases. Therefore, when a single operator simultaneously manipulates multiple unmanned aerial vehicles, it is often necessary to use an unmanned aerial vehicle supervisory control system to assist the operator in processing the unmanned aerial vehicle return images.
However, the existing unmanned aerial vehicle supervision control system has a low automation level, and still is difficult to assist a controller to process unmanned aerial vehicle return images, so that the workload of the controller is heavy.
Disclosure of Invention
The invention aims to solve the technical problems that an existing unmanned aerial vehicle supervision control system is low in automation level and is difficult to assist a controller to process unmanned aerial vehicle feedback images, so that the workload of the controller is heavy.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
the reconnaissance unmanned aerial vehicle supervision control system is characterized in that:
The system comprises a target identification module, an autonomous agent module and a user interface module; the input end of the target recognition module is communicated with an external unmanned aerial vehicle and is used for receiving and detecting a return image of the unmanned aerial vehicle and acquiring a suspected target return image; the output end of the target identification module is respectively connected with the input end of the autonomous agent module and the first input end of the user interface module, and is used for transmitting the suspected target return image to the autonomous agent module and transmitting the return image of the unmanned aerial vehicle to the user interface module; the output end of the autonomous agent module is connected with the second input end of the user interface module, and the autonomous agent module is used for acquiring current attention target information, task execution time information and rest time information of the operator, acquiring task execution sequence information of each corresponding suspected target return image according to the current attention target information, task execution time information and rest time information of the operator, and acquiring task execution time information of each suspected target return image and rest time information of the operator; simultaneously, sequentially sending all suspected target return images to a user interface module according to the task execution sequence information and the task execution time information, and stopping sending the suspected target return images to the user interface module according to rest time information of a manipulator; the user interface module is used for displaying the return image of the unmanned aerial vehicle and the suspected target return image respectively.
Further, the autonomous agent module comprises an information classification storage unit, a task scheduling unit, a task time distribution unit and a man-machine interaction unit; the input end of the information classification storage unit is used as the input end of the autonomous agent module, is connected with the output end of the target recognition module, and is used for classifying and storing all suspected target return images output by the target recognition module and storing the related information of all the suspected target return images; the man-machine interaction unit is used for acquiring current attention target information, task execution time information and rest time information of the operator; the communication end of the task scheduling unit is respectively connected with the communication end of the information classification storage unit and the first output end of the human-computer interaction unit, and is used for reading the related information of all the corresponding suspected target return images according to the current attention target information of the operator in the human-computer interaction unit, acquiring the image task weights and the task execution time upper and lower bounds of all the corresponding suspected target return images according to the related information of all the corresponding suspected target return images, and acquiring the task execution sequence information according to the image task weights and the task execution time upper and lower bounds of all the corresponding suspected target return images; the input end of the task time distribution unit is respectively connected with the output end of the task scheduling unit and the second output end of the man-machine interaction unit, and is used for acquiring task execution time information of each suspected target return image and rest time information of the operator according to current task execution time information and rest time information of the operator in the man-machine interaction unit and task execution sequence information output by the task scheduling unit; the input end of the man-machine interaction unit is connected with the output end of the task time distribution unit, the third output end of the man-machine interaction unit is connected with the second input end of the user interface module, and the man-machine interaction unit is used for sequentially sending all suspected target return images to the user interface module for display according to task execution sequence information and task execution time information; and stopping sending the suspected target return image information to the user interface module according to the rest time information of the operator.
Further, the task scheduling unit comprises a task pool subunit and a heuristic scheduling subunit which are sequentially connected; the communication end of the task pool subunit is respectively connected with the communication end of the information classification storage unit and the first output end of the man-machine interaction unit, and is used for reading the related information of all the corresponding suspected target return images according to the current attention target information of the operator, modeling the related information of all the corresponding suspected target return images and obtaining the image task weight and the task execution time upper and lower bounds of all the corresponding suspected target return images; the output end of the heuristic scheduling subunit is connected with the input end of the task time distribution unit, and is used for acquiring task execution sequence information through a weighted shortest processing time priority rule according to the image task weights of all the corresponding suspected target return images and the upper and lower boundaries of task execution time.
Further, the related information of the suspected target return image comprises the number of the shooting unmanned aerial vehicle, the shooting time point, the storage path of the image and the related information of the suspected target.
Further, the man-machine interaction unit comprises an eye tracker and a man-machine interaction interface; the eye movement instrument is used for acquiring the current sight line coordinate point of the operator, judging whether the current sight line coordinate point of the operator is in the coordinate system range of the user interface module, acquiring the current task execution time information and rest time information of the operator according to the judging result, and acquiring the current attention target information of the operator according to the current sight line coordinate point of the operator; the first output end and the second output end of the eye movement instrument are respectively connected with the communication end of the task scheduling unit and the input end of the task time distribution unit, and are used for sending the current attention target information of the operator to the task scheduling unit and sending the current task execution time information and rest time information of the operator to the task time distribution unit; the input end of the man-machine interaction interface is connected with the output end of the task time distribution unit, the output end of the man-machine interaction interface is connected with the second input end of the user interface module, and the man-machine interaction interface is used for sequentially sending all suspected target return images to the user interface module for display according to task execution sequence information and task execution time information, and stopping sending the suspected target return image information to the user interface module according to rest time information of an operator.
Further, the man-machine interaction interface is a touch display device; the first output end of the man-machine interaction interface is connected with the communication end of the task scheduling unit, the second output end of the man-machine interaction interface is connected with the second input end of the user interface module, the man-machine interaction interface is used for acquiring current target information of interest of the operator according to the coordinate range manually framed by the operator on the touch display area of the operator, sending the current target information to the task scheduling unit, sequentially sending all suspected target return images to the user interface module for display according to the task execution sequence information and the task execution time information, and stopping sending the suspected target return image information to the user interface module according to the rest time information of the operator.
Further, the user interface module includes a first display area and a second display area; the first display area is used for displaying the return image of the unmanned aerial vehicle, and the second display area is used for displaying the return image of the suspected target.
Meanwhile, the invention also provides a reconnaissance unmanned aerial vehicle supervision and control method, which is based on the reconnaissance unmanned aerial vehicle supervision and control system and is characterized by comprising the following steps:
Step 1, receiving and detecting a return image of the unmanned aerial vehicle through a target recognition module, acquiring suspected target return images, transmitting all the suspected target return images to an autonomous agent module, and transmitting the return images of the unmanned aerial vehicle to a user interface module;
step 2, acquiring current attention target information, task execution time information and rest time information of the operator through an autonomous agent module, and acquiring task execution sequence information of each corresponding suspected target return image, task execution time information of each suspected target return image and rest time information of the operator according to the current attention target information, task execution time information and rest time information of the operator; simultaneously, sequentially sending all suspected target return images to a user interface module according to the task execution sequence information and the task execution time information, and stopping sending the suspected target return images to the user interface module according to rest time information of a manipulator;
and 3, respectively displaying the return image of the unmanned aerial vehicle and the return image of the suspected target through the user interface module.
Further, the step 2 specifically comprises:
Step 2.1, classifying and storing all suspected target return images output by the target recognition module through an information classification storage unit, and storing relevant information of all the suspected target return images;
Step 2.2, acquiring current attention target information, task execution time information and rest time information of an operator through a human-computer interaction unit;
Step 2.3, reading the related information of all the corresponding suspected target return images through a task scheduling unit according to the current attention target information of the operator in the man-machine interaction unit, acquiring the image task weights and the upper and lower task execution time bounds of all the corresponding suspected target return images according to the related information of all the corresponding suspected target return images, and acquiring the task execution sequence information according to the image task weights and the upper and lower task execution time bounds of all the corresponding suspected target return images;
Step 2.4, acquiring task execution time information of each suspected target return image and rest time information of the operator through a task time distribution unit according to current task execution time information and rest time information of the operator in the man-machine interaction unit and task execution sequence information output by a task scheduling unit;
Step 2.5, sequentially sending all suspected target return images to a user interface module for display through a man-machine interaction unit according to the task execution sequence information and the task execution time information; and stopping sending the suspected target return image information to the user interface module according to the rest time information of the operator.
The beneficial effects of the invention are as follows:
1. According to the reconnaissance unmanned aerial vehicle supervision control system provided by the invention, the return images of the unmanned aerial vehicle can be screened through the target recognition module to obtain the suspected target return images, the suspected target return images are further processed through the autonomous agent module to obtain the suspected target return images corresponding to the current focused target of the operator, and finally the return images of the unmanned aerial vehicle and the suspected target return images are respectively displayed through the user interface module, so that the operator can focus on the suspected target return images corresponding to the current focused target of the operator, and the working intensity of the operator is greatly reduced.
2. According to the reconnaissance unmanned aerial vehicle supervision control system provided by the invention, the optimal task execution sequence can be allocated to each suspected target return image under the condition that the task time is uncertain through the heuristic scheduling subunit.
3. According to the reconnaissance unmanned aerial vehicle supervision control method provided by the invention, the optimal task execution time can be allocated for each suspected target return image, and the rest time is allocated for the operator, so that the operator always keeps in the optimal working state, and the task execution quality and efficiency of the operator are ensured.
4. According to the reconnaissance unmanned aerial vehicle supervision control method provided by the invention, the current attention target information, task execution time information and rest time information of the operator can be acquired through acquiring the sight coordinate point of the operator, so that the attention target information, the task execution time information and the rest time information are correspondingly fed back to the task scheduling unit and the task time distribution unit, further the suspected target return images sent to the user interface are dynamically selected, and the task execution sequence, the task execution time and the rest time of the operator of each suspected target return image are dynamically adjusted.
Drawings
Fig. 1 is a schematic structural diagram of an embodiment of a reconnaissance unmanned aerial vehicle supervisory control system according to the present invention;
fig. 2 is a schematic structural diagram of a task scheduling unit in an embodiment of a reconnaissance unmanned aerial vehicle supervision control system according to the present invention;
Fig. 3 is a schematic structural diagram of a man-machine interaction unit and a task time allocation unit in an embodiment of a reconnaissance unmanned aerial vehicle supervision control system according to the present invention;
fig. 4 is a schematic diagram of an implementation of a man-machine interaction unit in an embodiment of a reconnaissance unmanned aerial vehicle supervision control system according to the present invention.
Detailed Description
In order to make the objects, advantages and features of the present invention more apparent, the following describes in further detail a reconnaissance unmanned aerial vehicle supervisory control system and method according to the present invention with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following detailed description.
Referring to fig. 1, a reconnaissance unmanned aerial vehicle supervision control system of this embodiment mainly includes three major modules, namely a target identification module, an autonomous agent module and a user interface module.
The input end of the target recognition module is communicated with an external unmanned aerial vehicle, and when the external unmanned aerial vehicle executes a reconnaissance task, the reconnaissance returned image is transmitted to the target recognition module through task load equipment such as a photoelectric ball. Specifically, firstly, a return image of the unmanned aerial vehicle is received through the target recognition module, the return image of the unmanned aerial vehicle is processed through the target recognition module, and a suspected target return image is screened out from the return image of the unmanned aerial vehicle. And the output end of the target recognition module is respectively connected with the input end of the autonomous agent module and the first input end of the user interface module, so that the suspected target return image is conveniently sent to the autonomous agent module for further processing through the target recognition module, and the return image of the unmanned aerial vehicle is simultaneously sent to the user interface module for real-time display.
The output end of the autonomous agent module is connected with the second input end of the user interface module, and the autonomous agent module is mainly used for acquiring current attention target information, task execution time information and rest time information of the operator, acquiring task execution sequence information of each corresponding suspected target return image according to the current attention target information, task execution time information and rest time information of the operator, and acquiring task execution time information of each suspected target return image and rest time information of the operator; and simultaneously, sequentially sending all the suspected target return images to the user interface module according to the task execution sequence information and the task execution time information, and stopping sending the suspected target return images to the user interface module according to the rest time information of the operator.
Specifically, in this embodiment, referring to fig. 1, the autonomous agent module mainly includes an information classification storage unit, a task scheduling unit, a task time allocation unit, a man-machine interaction unit, and the like.
The input end of the information classification storage unit is used as the input end of the autonomous agent module and is connected with the output end of the target recognition module, the information classification storage unit stores all the suspected target return images output by the target recognition module in a classified mode, stores relevant information of all the suspected target return images, and is convenient for quickly matching the corresponding suspected target return images through the relevant information of the suspected target return images. The related information of the suspected target return image specifically comprises the number of the shooting unmanned aerial vehicle, the shooting time point, the storage path of the image, the related information of the suspected target and the like.
The communication end of the task scheduling unit is respectively connected with the communication end of the information classification storage unit and the first output end of the human-computer interaction unit, the task scheduling unit reads the related information of all the corresponding suspected target return images according to the current attention target information of the operator in the human-computer interaction unit, obtains the image task weight and the upper and lower task execution time bounds of all the corresponding suspected target return images according to the related information of all the corresponding suspected target return images, and obtains the task execution sequence information, namely the sequence of all the corresponding suspected target return images sent to the user interface module for display according to the image task weight and the upper and lower task execution time bounds of all the corresponding suspected target return images.
Specifically, in this embodiment, referring to fig. 2, the task scheduling unit specifically includes a task pool subunit and a heuristic scheduling subunit that are sequentially connected. The task pool unit reads the relevant information of all the corresponding suspected target return images according to the current attention target information of the operator, models the relevant information of all the corresponding suspected target return images, and accordingly obtains the image task weight and the task execution time upper and lower bounds of all the corresponding suspected target return images; because the task execution time is uncertain before the operator executes the task, the task execution time can be determined only after the operator executes the task once, and therefore, the embodiment obtains the optimal task execution sequence under the condition of uncertain task time through the set heuristic scheduling subunit. The output end of the heuristic scheduling subunit is connected with the input end of the task time distribution unit, and the heuristic scheduling subunit acquires task execution sequence information through a weighted shortest processing time priority rule according to the image task weights of all corresponding suspected target return images and the upper and lower boundaries of task execution time.
The input end of the task time distribution unit is respectively connected with the output end of the task scheduling unit and the second output end of the man-machine interaction unit, referring to fig. 3, according to the current task execution time information and rest time information of the operators in the man-machine interaction unit and the task execution sequence information output by the task scheduling unit, the task execution time information of each suspected target return image and the rest time information of the operators are obtained by the task time distribution unit, so that the optimal task execution time is distributed for each suspected target return image, the rest time is distributed for the operators, the working intensity of the operators is kept within a certain degree, the working state of the operators is kept in the optimal state all the time, and the problems of lower working efficiency, poor working quality and the like of the operators due to overlarge working intensity are avoided.
The man-machine interaction unit is mainly used for acquiring current attention target information, task execution time information and rest time information of an operator; the current attention target information, task execution time information and rest time information of the operator are fed back to the task pool subunit and the task time distribution unit; the input end of the man-machine interaction unit is connected with the output end of the task time distribution unit, the third output end of the man-machine interaction unit is connected with the second input end of the user interface module, and each suspected target return image is sequentially sent to the user interface module for display according to the task execution sequence information and the task execution time information; and stopping sending the suspected target return image information to the user interface module according to the rest time information of the operator.
Specifically, in this embodiment, referring to fig. 3 and 4, the man-machine interaction unit adopts a man-machine interaction interface based on an eye tracker.
The eye movement instrument is used for distinguishing three states of eye fixation, glance and moving vision of the operator through an eye movement classification algorithm based on Kalman filtering, so that the eye observation point of the operator is obtained, the current sight coordinate point of the operator is obtained, whether the current sight coordinate point of the operator is in the coordinate system range of the user interface module is judged through the eye movement instrument, the current task execution time information and the rest time information of the operator are obtained according to the judging result, and meanwhile the current attention target information of the operator is obtained according to the current sight coordinate point of the operator. The first output end and the second output end of the eye movement instrument are respectively connected with the communication end of the task scheduling unit and the input end of the task time distribution unit, so that the current attention target information of the operator is conveniently sent to the task scheduling unit, and the current task execution time information and rest time information of the operator are conveniently sent to the task time distribution unit.
The input end of the man-machine interaction interface is connected with the output end of the task time distribution unit, the output end of the man-machine interaction interface is connected with the second input end of the user interface module, and each suspected target return image is sequentially sent to the user interface module for display according to the task execution sequence information and the task execution time information; and stopping sending the suspected target return image information to the user interface module according to the rest time information of the operator.
In a preferred embodiment of the invention, the human-machine interaction interface may also be configured as a touch display device. The first output end of the man-machine interaction interface is connected with the communication end of the task scheduling unit, and the second output end of the man-machine interaction interface is connected with the second input end of the user interface module. Referring to fig. 4, based on the foregoing man-machine interaction interface, the man-machine interaction interface may further obtain current information of the target of interest of the operator according to the coordinate range manually framed on the touch display area of the operator, and send the information to the task scheduling unit, so as to provide different man-machine interaction modes when the information is used in cooperation with the eye tracker.
The user interface module is mainly used for displaying the return image of the unmanned aerial vehicle and the suspected target return image respectively. Specifically, the user interface module includes a first display area and a second display area; the return image of the unmanned aerial vehicle is displayed through the first display area, the suspected target return image is displayed through the second display area, and operators can pay attention to different display areas according to different requirements.
The reconnaissance unmanned aerial vehicle supervision control method specifically comprises the following steps:
Step 1, receiving and detecting a return image of the unmanned aerial vehicle through a target recognition module, acquiring suspected target return images, transmitting all the suspected target return images to an autonomous agent module, and transmitting the return images of the unmanned aerial vehicle to a user interface module;
step 2, acquiring current attention target information, task execution time information and rest time information of the operator through an autonomous agent module, and acquiring task execution sequence information of each corresponding suspected target return image, task execution time information of each suspected target return image and rest time information of the operator according to the current attention target information, task execution time information and rest time information of the operator; simultaneously, sequentially sending all suspected target return images to a user interface module according to the task execution sequence information and the task execution time information, and stopping sending the suspected target return images to the user interface module according to rest time information of a manipulator;
Specifically, step 2 includes the following steps:
Step 2.1, classifying and storing all suspected target return images output by the target recognition module through an information classification storage unit, and storing relevant information of all the suspected target return images;
Step 2.2, acquiring current attention target information, task execution time information and rest time information of an operator through a human-computer interaction unit;
Step 2.3, reading the related information of all the corresponding suspected target return images through a task scheduling unit according to the current attention target information of the operator in the man-machine interaction unit, acquiring the image task weights and the upper and lower task execution time bounds of all the corresponding suspected target return images according to the related information of all the corresponding suspected target return images, and acquiring the task execution sequence information according to the image task weights and the upper and lower task execution time bounds of all the corresponding suspected target return images;
Step 2.4, acquiring task execution time information of each suspected target return image and rest time information of the operator through a task time distribution unit according to current task execution time information and rest time information of the operator in the man-machine interaction unit and task execution sequence information output by a task scheduling unit;
Step 2.5, sequentially sending all suspected target return images to a user interface module for display through a man-machine interaction unit according to the task execution sequence information and the task execution time information; and stopping sending the suspected target return image information to the user interface module according to the rest time information of the operator.
And 3, respectively displaying the return image of the unmanned aerial vehicle and the return image of the suspected target through the user interface module.
Claims (7)
1. A reconnaissance unmanned aerial vehicle supervisory control system which characterized in that:
the system comprises a target identification module, an autonomous agent module and a user interface module;
the input end of the target recognition module is communicated with an external unmanned aerial vehicle and is used for receiving and detecting a return image of the unmanned aerial vehicle and obtaining a suspected target return image; the output end of the target identification module is respectively connected with the input end of the autonomous agent module and the first input end of the user interface module, and is used for transmitting the suspected target return image to the autonomous agent module and transmitting the unmanned aerial vehicle return image to the user interface module;
The output end of the autonomous agent module is connected with the second input end of the user interface module, and the autonomous agent module is used for acquiring current attention target information, task execution time information and rest time information of the operator, acquiring task execution sequence information of each corresponding suspected target return image according to the current attention target information, task execution time information and rest time information of the operator, and acquiring task execution time information of each suspected target return image and rest time information of the operator; simultaneously, sequentially sending all suspected target return images to a user interface module according to the task execution sequence information and the task execution time information, and stopping sending the suspected target return images to the user interface module according to rest time information of a manipulator;
the autonomous agent module comprises an information classification storage unit, a task scheduling unit, a task time distribution unit and a man-machine interaction unit;
The input end of the information classification storage unit is used as the input end of the autonomous agent module, is connected with the output end of the target recognition module, and is used for classifying and storing all suspected target return images output by the target recognition module and storing the related information of all the suspected target return images;
The man-machine interaction unit is used for acquiring current attention target information, task execution time information and rest time information of an operator;
The communication end of the task scheduling unit is respectively connected with the communication end of the information classification storage unit and the first output end of the human-computer interaction unit, and is used for reading the related information of all the corresponding suspected target return images according to the current attention target information of the operator in the human-computer interaction unit, acquiring the image task weight and the task execution time upper and lower bounds of all the corresponding suspected target return images according to the related information of all the corresponding suspected target return images, and acquiring the task execution sequence information according to the image task weight and the task execution time upper and lower bounds of all the corresponding suspected target return images;
the input end of the task time distribution unit is respectively connected with the output end of the task scheduling unit and the second output end of the man-machine interaction unit, and is used for acquiring task execution time information of each suspected target feedback image and rest time information of the operator according to current task execution time information and rest time information of the operator in the man-machine interaction unit and task execution sequence information output by the task scheduling unit;
The input end of the man-machine interaction unit is connected with the output end of the task time distribution unit, and the third output end of the man-machine interaction unit is connected with the second input end of the user interface module; the man-machine interaction unit is used for sequentially sending all suspected target feedback images to the user interface module for display according to the task execution sequence information and the task execution time information; stopping sending the suspected target return image information to the user interface module according to rest time information of the operator;
The user interface module is used for displaying the return image of the unmanned aerial vehicle and the return image of the suspected target respectively.
2. A reconnaissance unmanned aerial vehicle supervisory control system according to claim 1, wherein:
The task scheduling unit comprises a task pool subunit and a heuristic scheduling subunit which are sequentially connected;
The communication end of the task pool subunit is respectively connected with the communication end of the information classification storage unit and the first output end of the man-machine interaction unit, and is used for reading the related information of all the corresponding suspected target return images according to the current attention target information of the operator, modeling the related information of all the corresponding suspected target return images and obtaining the image task weight and the task execution time upper and lower bounds of all the corresponding suspected target return images;
The output end of the heuristic scheduling subunit is connected with the input end of the task time distribution unit, and is used for acquiring task execution sequence information through a weighted shortest processing time priority rule according to the image task weights of all the corresponding suspected target return images and the upper and lower boundaries of task execution time.
3. A reconnaissance unmanned aerial vehicle supervisory control system according to claim 2, wherein:
The related information of the suspected target return image comprises the number of the shooting unmanned aerial vehicle, the shooting time point, the storage path of the image and the related information of the suspected target.
4. A reconnaissance unmanned aerial vehicle supervisory control system according to any of claims 1 to 3, wherein:
The man-machine interaction unit comprises an eye tracker and a man-machine interaction interface;
The eye movement instrument is used for acquiring a current sight line coordinate point of the operator, judging whether the current sight line coordinate point of the operator is in a coordinate system range of the user interface module, acquiring current task execution time information and rest time information of the operator according to a judging result, and acquiring current attention target information of the operator according to the current sight line coordinate point of the operator; the first output end and the second output end of the eye movement instrument are respectively connected with the communication end of the task scheduling unit and the input end of the task time distribution unit, and are used for sending current attention target information of the operator to the task scheduling unit and sending current task execution time information and rest time information of the operator to the task time distribution unit;
The input end of the man-machine interaction interface is connected with the output end of the task time distribution unit, the output end of the man-machine interaction interface is connected with the second input end of the user interface module, and the man-machine interaction interface is used for sequentially sending all suspected target return images to the user interface module for display according to task execution sequence information and task execution time information, and stopping sending the suspected target return image information to the user interface module according to rest time information of an operator.
5. The reconnaissance unmanned aerial vehicle supervisory control system of claim 4, wherein:
The man-machine interaction interface is a touch display device;
The first output end of the man-machine interaction interface is connected with the communication end of the task scheduling unit, the second output end of the man-machine interaction interface is connected with the second input end of the user interface module, the man-machine interaction interface is used for acquiring current target information of interest of the operator according to the coordinate range manually framed by the operator on the touch display area of the operator, sending the target information to the task scheduling unit, sequentially sending all suspected target return images to the user interface module for display according to the task execution sequence information and the task execution time information, and stopping sending the suspected target return image information to the user interface module according to the rest time information of the operator.
6. A reconnaissance unmanned aerial vehicle supervisory control system according to claim 1, wherein:
the user interface module comprises a first display area and a second display area;
The first display area is used for displaying the return image of the unmanned aerial vehicle, and the second display area is used for displaying the return image of the suspected target.
7. A reconnaissance unmanned aerial vehicle supervision control method based on the reconnaissance unmanned aerial vehicle supervision control system according to any one of claims 1 to 6, characterized by comprising the steps of:
Step 1, receiving and detecting a return image of the unmanned aerial vehicle through a target recognition module, acquiring suspected target return images, transmitting all the suspected target return images to an autonomous agent module, and transmitting the return images of the unmanned aerial vehicle to a user interface module;
step 2, acquiring current attention target information, task execution time information and rest time information of the operator through an autonomous agent module, and acquiring task execution sequence information of each corresponding suspected target return image, task execution time information of each suspected target return image and rest time information of the operator according to the current attention target information, task execution time information and rest time information of the operator; simultaneously, sequentially sending all suspected target return images to a user interface module according to the task execution sequence information and the task execution time information, and stopping sending the suspected target return images to the user interface module according to rest time information of a manipulator;
Step 2.1, classifying and storing all suspected target return images output by the target recognition module through an information classification storage unit, and storing relevant information of all the suspected target return images;
Step 2.2, acquiring current attention target information, task execution time information and rest time information of an operator through a human-computer interaction unit;
Step 2.3, reading the related information of all the corresponding suspected target return images through a task scheduling unit according to the current attention target information of the operator in the man-machine interaction unit, acquiring the image task weights and the upper and lower task execution time bounds of all the corresponding suspected target return images according to the related information of all the corresponding suspected target return images, and acquiring the task execution sequence information according to the image task weights and the upper and lower task execution time bounds of all the corresponding suspected target return images;
Step 2.4, acquiring task execution time information of each suspected target return image and rest time information of the operator through a task time distribution unit according to current task execution time information and rest time information of the operator in the man-machine interaction unit and task execution sequence information output by a task scheduling unit;
Step 2.5, sequentially sending all suspected target return images to a user interface module for display through a man-machine interaction unit according to the task execution sequence information and the task execution time information; stopping sending the suspected target return image information to the user interface module according to rest time information of the operator;
and 3, respectively displaying the return image of the unmanned aerial vehicle and the return image of the suspected target through the user interface module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410354463.1A CN117950355B (en) | 2024-03-27 | 2024-03-27 | Reconnaissance unmanned aerial vehicle supervision control system and reconnaissance unmanned aerial vehicle supervision control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410354463.1A CN117950355B (en) | 2024-03-27 | 2024-03-27 | Reconnaissance unmanned aerial vehicle supervision control system and reconnaissance unmanned aerial vehicle supervision control method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117950355A CN117950355A (en) | 2024-04-30 |
| CN117950355B true CN117950355B (en) | 2024-06-11 |
Family
ID=90801880
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410354463.1A Active CN117950355B (en) | 2024-03-27 | 2024-03-27 | Reconnaissance unmanned aerial vehicle supervision control system and reconnaissance unmanned aerial vehicle supervision control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117950355B (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106295613A (en) * | 2016-08-23 | 2017-01-04 | 哈尔滨理工大学 | A kind of unmanned plane target localization method and system |
| CN108037770A (en) * | 2017-11-22 | 2018-05-15 | 国网山东省电力公司济宁供电公司 | Unmanned plane power transmission line polling system and method based on artificial intelligence |
| CN109902141A (en) * | 2017-12-08 | 2019-06-18 | 三星电子株式会社 | The method and autonomous agents of motion planning |
| CN110221620A (en) * | 2019-05-30 | 2019-09-10 | 中国人民解放军国防科技大学 | MAS-based multi-unmanned system supervision control station |
| CN112257621A (en) * | 2020-10-28 | 2021-01-22 | 贵州电网有限责任公司 | Equipment image identification method for unmanned aerial vehicle inspection |
| KR102212232B1 (en) * | 2020-07-23 | 2021-02-04 | (주)올포랜드 | System for photography, charge and return of specific region using drone |
| KR102345508B1 (en) * | 2021-02-18 | 2022-01-03 | (주)에어로텍 | Online maintenance and monitoring system for drone |
| CN114115289A (en) * | 2021-12-07 | 2022-03-01 | 湖南大学 | Autonomous unmanned cluster reconnaissance system |
| CN114237281A (en) * | 2021-11-26 | 2022-03-25 | 国网北京市电力公司 | Control method and device for unmanned aerial vehicle inspection and inspection system |
| CN114564049A (en) * | 2022-03-21 | 2022-05-31 | 厦门大学 | Unmanned aerial vehicle wide area search device and method based on deep learning |
| CN116017509A (en) * | 2022-12-21 | 2023-04-25 | 南京航空航天大学 | Task unmanned plane network-oriented lightweight asynchronous provable consensus method and application |
| CN116193398A (en) * | 2023-02-22 | 2023-05-30 | 中电华鸿科技有限公司 | Remote investigation and striking system and method based on multi-rotor unmanned aerial vehicle |
| CN117389300A (en) * | 2023-11-10 | 2024-01-12 | 苏州科技大学 | Urban war unmanned plane formation reconnaissance task allocation method |
| CN117456092A (en) * | 2023-10-24 | 2024-01-26 | 徐州硕祥信息科技有限公司 | Three-dimensional live-action modeling system and method based on unmanned aerial vehicle aerial survey |
-
2024
- 2024-03-27 CN CN202410354463.1A patent/CN117950355B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106295613A (en) * | 2016-08-23 | 2017-01-04 | 哈尔滨理工大学 | A kind of unmanned plane target localization method and system |
| CN108037770A (en) * | 2017-11-22 | 2018-05-15 | 国网山东省电力公司济宁供电公司 | Unmanned plane power transmission line polling system and method based on artificial intelligence |
| CN109902141A (en) * | 2017-12-08 | 2019-06-18 | 三星电子株式会社 | The method and autonomous agents of motion planning |
| CN110221620A (en) * | 2019-05-30 | 2019-09-10 | 中国人民解放军国防科技大学 | MAS-based multi-unmanned system supervision control station |
| KR102212232B1 (en) * | 2020-07-23 | 2021-02-04 | (주)올포랜드 | System for photography, charge and return of specific region using drone |
| CN112257621A (en) * | 2020-10-28 | 2021-01-22 | 贵州电网有限责任公司 | Equipment image identification method for unmanned aerial vehicle inspection |
| KR102345508B1 (en) * | 2021-02-18 | 2022-01-03 | (주)에어로텍 | Online maintenance and monitoring system for drone |
| CN114237281A (en) * | 2021-11-26 | 2022-03-25 | 国网北京市电力公司 | Control method and device for unmanned aerial vehicle inspection and inspection system |
| CN114115289A (en) * | 2021-12-07 | 2022-03-01 | 湖南大学 | Autonomous unmanned cluster reconnaissance system |
| CN114564049A (en) * | 2022-03-21 | 2022-05-31 | 厦门大学 | Unmanned aerial vehicle wide area search device and method based on deep learning |
| CN116017509A (en) * | 2022-12-21 | 2023-04-25 | 南京航空航天大学 | Task unmanned plane network-oriented lightweight asynchronous provable consensus method and application |
| CN116193398A (en) * | 2023-02-22 | 2023-05-30 | 中电华鸿科技有限公司 | Remote investigation and striking system and method based on multi-rotor unmanned aerial vehicle |
| CN117456092A (en) * | 2023-10-24 | 2024-01-26 | 徐州硕祥信息科技有限公司 | Three-dimensional live-action modeling system and method based on unmanned aerial vehicle aerial survey |
| CN117389300A (en) * | 2023-11-10 | 2024-01-12 | 苏州科技大学 | Urban war unmanned plane formation reconnaissance task allocation method |
Non-Patent Citations (1)
| Title |
|---|
| 警用无人机空中实时图像侦察***的设计与应用;刘丹;田银枝;王红闯;梁涛;;地理空间信息;20150228(01);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117950355A (en) | 2024-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111427320A (en) | Intelligent industrial robot distributed unified scheduling management platform | |
| US7557825B2 (en) | Camera system, camera, and camera control method | |
| KR102399425B1 (en) | Data labelling pre-processing, distributing and checking system | |
| CN112541426B (en) | Communication bandwidth self-adaptive data processing method based on unmanned aerial vehicle cluster cooperative sensing | |
| CN110673647B (en) | Omnidirectional obstacle avoidance method and unmanned aerial vehicle | |
| CN109544548A (en) | Defect inspection method, device, server, equipment and the storage medium of cutlery box | |
| US20210390303A1 (en) | Method and device for monitoring an industrial process step | |
| CN111985352A (en) | AI front-end substation inspection video real-time identification method and system | |
| CN111884333A (en) | Unmanned inspection unmanned aerial vehicle cooperative work system and method thereof | |
| CN115542951B (en) | Unmanned aerial vehicle centralized management and control method, system, equipment and medium based on 5G network | |
| CN109993234A (en) | A kind of unmanned training data classification method, device and electronic equipment | |
| CN114035606A (en) | Pole tower inspection system, pole tower inspection method, control device and storage medium | |
| CN109242762A (en) | Image-recognizing method, device, storage medium and electronic equipment | |
| CN117950355B (en) | Reconnaissance unmanned aerial vehicle supervision control system and reconnaissance unmanned aerial vehicle supervision control method | |
| CN114637321A (en) | Cable monitoring system based on image data optimization path | |
| CN114637320A (en) | Cable monitoring system for determining routing inspection parameters according to historical fault data | |
| CN109270957A (en) | A kind of plant protection system and its flying vehicles control method and apparatus | |
| CN108536156A (en) | Target Tracking System and method for tracking target | |
| US20230150661A1 (en) | Horizon detection to support an aircraft on a mission in an environment | |
| CN113642425A (en) | Multi-mode-based image detection method and device, electronic equipment and storage medium | |
| CN112906538A (en) | Ship target image recognition system and method based on unattended platform | |
| CN113114331A (en) | Method, system, device and medium for merging and transmitting multi-channel serial interfaces | |
| CN111667626B (en) | Safety helmet and safety belt detection and identification system based on edge equipment | |
| CN104994342A (en) | Intelligent and dynamic high-definition video face recognition pre-warning system | |
| JP2020198024A (en) | Remote control system and remote control method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |