CN112492281A - Remote auxiliary maintenance method, device, equipment and storage medium - Google Patents
Remote auxiliary maintenance method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112492281A CN112492281A CN202011455534.5A CN202011455534A CN112492281A CN 112492281 A CN112492281 A CN 112492281A CN 202011455534 A CN202011455534 A CN 202011455534A CN 112492281 A CN112492281 A CN 112492281A
- Authority
- CN
- China
- Prior art keywords
- information
- mark
- frame
- image
- client device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 90
- 238000012423 maintenance Methods 0.000 title abstract description 24
- 239000003550 marker Substances 0.000 claims abstract description 104
- 238000012217 deletion Methods 0.000 claims description 32
- 230000037430 deletion Effects 0.000 claims description 32
- 230000001133 acceleration Effects 0.000 claims description 20
- 230000003190 augmentative effect Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 2
- 238000009877 rendering Methods 0.000 claims 4
- 238000004891 communication Methods 0.000 abstract description 23
- 230000008569 process Effects 0.000 description 13
- 230000008859 change Effects 0.000 description 8
- 230000008439 repair process Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application provides a remote auxiliary maintenance method, a remote auxiliary maintenance device, a remote auxiliary maintenance equipment and a storage medium, and relates to the technical field of instant messaging. The method comprises the following steps: displaying a first frame of image acquired by first client equipment; drawing first marking information in a first frame image according to input marking operation aiming at a first marking position in the first frame image, wherein the first marking information is marking information corresponding to a fault point of an industrial field; collecting first state data when the first marking information is drawn and second state data after the first marking information is drawn; calculating a second marking position according to the first marking position, the first state data and the second state data; drawing first marker information at a second marker position in at least one frame of image subsequent to the first frame of image; a first video stream including a first frame image in which the first marker information is drawn and at least one frame image subsequent to the first frame image is transmitted to the second client device. The remote auxiliary maintenance system can improve the communication efficiency of remote auxiliary maintenance and improve the maintenance progress.
Description
Technical Field
The invention relates to the technical field of instant messaging, in particular to a remote auxiliary maintenance method, a remote auxiliary maintenance device, a remote auxiliary maintenance equipment and a storage medium.
Background
With the development of industrial technology, more and more industrial equipment is applied in production transportation environment. Along with the use time of the industrial equipment, the use environment and other factors, the industrial equipment can be in failure in the application. In order to ensure industrial productivity and avoid damage caused by equipment failure, a fault point of an industrial field needs to be overhauled in time.
Although in some industrial sites, service robots or other automated means are used to perform the service. The cost of the maintenance robot or the automatic maintenance is usually higher, so the manual maintenance is still the mainstream mode of the equipment maintenance at present. And manual maintenance, wherein detection personnel are required to enter an industrial field for standby maintenance. If the inspection personnel can not work well, the inspection personnel need to ask for help from other personnel with higher inspection capability. At present, the method of repeatedly inquiring by the telephone to know the corresponding failure solving method is more adopted.
However, repeated inquiry by the telephone not only makes it difficult to accurately describe the fault point condition, but also makes communication speed slow and affects the overhaul progress.
Disclosure of Invention
The present invention aims to provide a remote auxiliary maintenance method, device, equipment and storage medium to improve the communication efficiency of remote auxiliary maintenance and improve the maintenance progress.
In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solutions:
in a first aspect, an embodiment of the present invention provides a remote auxiliary overhaul method, including:
displaying a first frame of image acquired by first client equipment;
drawing first marking information in the first frame image according to input marking operation aiming at a first marking position in the first frame image, wherein the first marking information is marking information marked by a first user and corresponding to a fault point of an industrial field;
acquiring first state data of the first client device when the first mark information is drawn and second state data of the first client device after the first mark information is drawn;
calculating a second marker position based on the first marker position, the first state data, and the second state data;
drawing the first marker information at the second marker position in at least one frame image subsequent to the first frame image;
sending a first video stream to a second client device, the first video stream comprising: drawing the first frame image of the first mark information, and drawing at least one frame image subsequent to the first frame image of the first mark information.
Optionally, the calculating a second mark position according to the first mark position, the first state data and the second state data includes:
calculating a mark offset according to the first state data and the second state data;
and calculating the second mark position according to the first mark position and the mark offset.
Optionally, the acquiring first state data of the first client device when the first marker information is drawn and second state data of the first client device after the first marker information is drawn includes:
acquiring first attitude data of the first client device when the first marker information is drawn, wherein the first state data comprises: the first pose data;
acquiring acceleration data and second posture data of the first client device after the first marker information is drawn, wherein the second state data comprises: the acceleration data and the second pose data.
Optionally, the drawing, according to an input marking operation for a first marking position in the first frame image, first marking information in the first frame image includes:
and according to the marking operation, displaying the first marking information as augmented reality information at the first marking position in the first frame image in an overlapping mode.
Optionally, the drawing the first annotation information at the second marker position in at least one frame of image after the first frame of image includes:
and displaying the first marker information as augmented reality information at the second marker position in at least one frame of image after the first frame of image in an overlapping mode.
Optionally, the drawing the first annotation information at the second marker position in at least one frame of image after the first frame of image includes:
judging whether the second mark position exceeds the display range of at least one frame of image behind the first frame of image;
and if the second mark position does not exceed the display range, drawing the first mark information at the second mark position in at least one frame of image after the first frame of image.
Optionally, the method further includes:
if the second mark position exceeds the display range of a second frame image after the first frame image, sending a second video stream to the second client device, wherein the second video stream comprises: the second frame image of the first marker information is not drawn.
Optionally, the method further includes:
deleting the first mark information drawn in a third frame image after the first frame image according to an input first mark deletion operation for the first mark information;
sending a third video stream to the second client device, the third video stream comprising: deleting the third frame image after the first mark information.
Optionally, the method further includes:
receiving a fourth frame of image which is sent by the second client device and is drawn with second mark information, wherein the second mark information is mark information of a solution corresponding to the fault point of the fourth frame of image mark displayed by the second client device by a second user;
acquiring third state data of the first client device when the fourth frame of image is displayed and fourth state data of the first client device after the fourth frame of image is displayed;
calculating a fourth mark position according to a third mark position, the third state data and the fourth state data, wherein the third mark position is a display position of the second mark information in the fourth frame image;
drawing the second marker information at the fourth marker position in at least one frame image subsequent to the fourth frame image;
displaying a fourth video stream, the fourth video stream comprising: drawing the fourth frame image of the second mark information, and drawing at least one frame image subsequent to the fourth frame image of the second mark information.
Optionally, the method further includes:
receiving mark deletion indication information sent by the second client device, wherein the mark deletion indication information is information sent by the second client device according to input second mark deletion operation aiming at the second mark information;
deleting the second mark information drawn in a fifth frame image after the fourth frame image according to the mark deletion indication information;
displaying a fifth video stream, the fifth video stream comprising: deleting the fifth frame image after the second mark information.
In a second aspect, an embodiment of the present application further provides a remote auxiliary service device, including:
the display module is used for displaying a first frame image acquired by first client equipment;
the system comprises a drawing module, a first marking module and a second marking module, wherein the drawing module is used for drawing first marking information in a first frame image according to input marking operation aiming at a first marking position in the first frame image, and the first marking information is marking information marked by a first user and corresponding to a fault point of an industrial field;
the acquisition module is used for acquiring first state data of the first client device when the first mark information is drawn and second state data of the first client device after the first mark information is drawn;
a calculation module, configured to calculate a second mark position according to the first mark position, the first state data, and the second state data;
the drawing module is further used for drawing the first mark information at the second mark position in at least one frame of image after the first frame of image;
a sending module, configured to send a first video stream to a second client device, where the first video stream includes: drawing the first frame image of the first mark information, and drawing at least one frame image subsequent to the first frame image of the first mark information.
In a third aspect, an embodiment of the present application further provides a client device, including: the memory stores a computer program executable by the processor, and the processor implements any one of the remote auxiliary service methods provided by the first aspect when executing the computer program.
In a fourth aspect, an embodiment of the present application further provides a storage medium, where a computer program is stored on the storage medium, and when the computer program is read and executed, the remote auxiliary service method provided in the first aspect is implemented.
The beneficial effect of this application is:
the remote auxiliary overhaul method, the device, the equipment and the storage medium provided by the application can be used. In this method, the following is performed.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a network schematic diagram of a remote auxiliary service system according to an embodiment of the present application;
fig. 2 is a schematic flow chart of a remote auxiliary overhaul method provided in an embodiment of the present application;
fig. 3 is a schematic flowchart of calculating a second mark position in a remote auxiliary repair method according to an embodiment of the present application;
FIG. 4 is a flowchart of a method for mapping first marker information at a second marker position in a remote auxiliary service method according to an embodiment of the present disclosure;
fig. 5 is a schematic flowchart of deleting first marker information in a remote auxiliary overhaul method provided in an embodiment of the present application;
fig. 6 is a schematic flowchart of drawing second marking information in a remote auxiliary overhaul method provided in an embodiment of the present application;
fig. 7 is a flowchart of a method for deleting second flag information in a remote auxiliary repair method according to an embodiment of the present application;
FIG. 8 is a schematic view of a remote auxiliary service device provided in an embodiment of the present application;
fig. 9 is a schematic diagram of a client device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention.
The remote auxiliary overhaul method, the device, the equipment and the storage medium provided by the embodiment of the application can be applied to a remote auxiliary overhaul system. The network architecture of the remote auxiliary service system is first illustrated as follows. Fig. 1 is a network schematic diagram of a remote auxiliary service system according to an embodiment of the present application. As shown in fig. 1, the remote auxiliary service system may include: a server 11, a first client device 12 and a second client device 13. The first client device 12 and the second client device 13 may be electronic devices, such as portable mobile terminals, for example, mobile phones or tablet computers, on which client applications of the remote auxiliary service software are installed and run, respectively. The server 11 may be installed and run a server application program of the remote auxiliary repair software, that is, a server application program corresponding to the client device application program, and therefore, the server 11 may be an application server corresponding to the client application program. The client application program can also become a client application program.
The client application may be a client application having an instant messaging function, and the first client device 12 may establish a wireless communication connection with the second client device 13 through the server 11, that is, instant messaging messages of the first client device 12 and the second client device 13 when running the client application may be transmitted through the server 11.
The client application program may have a preset communication directory, where the communication directory may be a user list, and a client device installed and running with the client application program may select a target user from the communication directory, and establish an instant messaging connection with the client device of the target user through the server 11.
For example, a first user may establish an instant messaging connection between the first client device 12 and the second client device 13 by selecting a target user, such as a second user, from a correspondence list on the first client device 12 to enable instant messaging between the first user and the second user.
The first client device 12 may transmit an instant message, which may be at least one of a file, a picture, text, an audio file, or video call data, with the second client device 13 through the established instant communication connection.
The remote auxiliary overhaul method provided by each embodiment described below of the application mainly aims at an application scene of realizing video call between users by transmitting video call data. The remote auxiliary service method provided by the application is illustrated by a plurality of examples as follows.
Fig. 2 is a schematic flowchart of a remote auxiliary service method provided in an embodiment of the present application, where the remote auxiliary service method may be implemented by a first client device through software and/or hardware. The first client device may be a client device of a user of a help seeker at an industrial site. As shown in fig. 2, the method may include:
s201, displaying a first frame image acquired by first client equipment.
The first frame image may be a frame image in a video stream captured by a camera of the first client device when the first client device establishes an instant messaging connection based on a video call with the second client device. In this embodiment, the video call may be initiated by a first user, that is, a user of a help seeker, or may be initiated by a second user, that is, a user of a help seeker.
The video stream captured by the first client device should include at least a frame image of the industrial field, where the first frame image may be, for example, a frame image of the industrial field captured by the first client device, such as a field frame image of a fault in the industrial field.
S202, drawing first mark information in the first frame image according to the input mark operation aiming at the first mark position in the first frame image.
The first marking information is marking information corresponding to a fault point of the industrial field marked by the first user.
After the first client device displays the first frame image, the first user can determine that the first mark position in the first frame image is the display position corresponding to the fault point of the industrial field by viewing the first frame image, so that a marking operation can be input for the first mark position to mark at the first mark position. The first client device may render the first marking information at a first marking location in the first frame image after receiving a marking operation of the first marking location.
The first mark information is mark information corresponding to a fault point of the industrial field marked by the first user and can be used for indicating the fault point of the industrial field. The first flag information may include, for example: the shape of the first mark information, if including graphics, may include, for example, indication information in the form of text, graphics, or icons such as arrow icons: lines, boxes or circles. The color of the first mark information may be, for example, a color selected by the first user or a preset mark color.
After the first mark information is drawn in the first frame image, the color and/or shape information and the position information of the first mark information, i.e. the first mark position, are also recorded.
For example, as shown above, the drawing of the first mark information in the first frame image according to the input marking operation for the first mark position in the first frame image may include:
according to the marking operation, the first marking information is displayed as Augmented Reality (AR) information at the first marking position in the first frame image in an overlapping manner.
In a possible implementation manner, AR technology may be adopted, and the first marker information is displayed as AR information in an overlapping manner at the first marker position in the first frame image. By adopting the AR technology, the first mark information is displayed in the first frame image, so that the display of the first mark information is more flexible, the identification degree of the first mark information is improved, and the communication efficiency aiming at the fault point indicated by the first identification information is improved.
S203, collecting first state data of the first client device when the first mark information is drawn and second state data of the first client device after the first mark information is drawn.
In the method, the drawing time of the first marking information is also recorded, and the state data acquired by the state sensor at the drawing time is acquired from the state sensor on the first client device according to the drawing time, namely the first state data.
After the first mark information is drawn, state data acquired by detecting the state sensor in real time can be acquired, and state data acquired by the state sensor after the first mark information is drawn, namely the second state data, can be acquired from the state sensor at a preset detection frequency.
The first state data may be used to characterize a device state of the first client device when the first mark information is drawn, and the first state data may be used to characterize a device state of the first client device after the first mark information is drawn.
Optionally, as shown above, the acquiring first state data of the first client device when the first marker information is drawn and second state data of the first client device after the first marker information is drawn may include:
acquiring first attitude data of the first client device when the first mark information is drawn, wherein the first state data comprises: the first attitude data;
acquiring acceleration data and second posture data of the first client device after the first marker information is plotted, the second state data comprising: the acceleration data and the second pose data.
For example, the first client device is pre-installed with a status sensor such as: the accelerometer, the gyroscope, the accelerometer and the gyroscope can be called as a six-axis sensor together, and can acquire first attitude data acquired by the gyroscope on the first client device when the first mark information is drawn, acquire acceleration data acquired by the accelerometer on the first client device after the first mark information is drawn, and acquire second attitude data acquired by the gyroscope. Wherein the first pose data may include: the first yaw angle, the first roll angle, and the first pitch angle, and the second attitude data may include: a second yaw angle, a second roll angle, and a second pitch angle. The acceleration data may include: and the accelerometer acquires acceleration data of the first client device on three coordinate axes in a preset three-dimensional coordinate system.
In the following, the manner of acquiring the acceleration data collected by the accelerometer and the attitude data collected by the gyroscope will be described as an example.
For the acceleration data acquired by the accelerometer, a Push mode can be adopted to acquire the acceleration data, that is, after the accelerometer acquires the acceleration data, the acceleration data is actively pushed to the processor of the first client device.
Before acquiring the acceleration data collected by the accelerometer, the accelerometer may be configured in advance, for example, the motion management object may be initialized as the accelerometer, whether the accelerometer is available or not may be determined, if the accelerometer is available, the update frequency of the accelerometer is set, the update frequency is in seconds, and after the accelerometer is configured, the acceleration data pushed by the set update frequency may be acquired in real time.
For the acceleration data collected by the gyroscope, a Pull (Pull) manner may be adopted to obtain the acceleration data, that is, the processor of the first client device pulls the collected number of gestures from the gyroscope.
Accordingly, before acquiring the attitude data acquired by the gyroscope, the gyroscope may be configured to acquire the data in advance, for example, the motion management object may be initialized as the gyroscope, whether the gyroscope is available or not may be determined, if the gyroscope is available, the update frequency of the gyroscope may be set, the update frequency may be set in seconds, and after the gyroscope is configured, the acquired attitude data may be pulled from the gyroscope in real time with the set update frequency.
S204, calculating a second marking position according to the first marking position, the first state data and the second state data.
For example, it may be determined whether the state of the first client device changes before and after the first marker information is drawn according to the first state data and the second state data, and if so, it may also be determined that the state of the second client device changes before and after the first marker information is drawn based on the first state data and the second state data, determine a change of a display position in a frame image displayed by the first client device based on the state change, and then determine a marker position after the change, that is, the first marker position, according to the first marker position and the change of the display position.
In a possible implementation, the drawing the first annotation information at the second marker position in at least one frame of image after the first frame of image as shown above may include:
and displaying the first marker information as augmented reality information in an overlapping mode at the position of the second marker in at least one frame of image behind the first frame of image.
For at least one frame of image after the first frame of image, the AR technology may also be adopted to display the first marker information in an overlapping manner on the at least one frame of image after the first frame of image, which not only can ensure stable display of the first marker information in the frame of image after the first frame of image, but also can ensure the recognition degree of the first marker information, thereby improving the communication efficiency for the fault point indicated by the first identification information.
S205, drawing the first mark information at the second mark position in at least one frame of image after the first frame of image.
For each frame of image after the first frame of image, before the first mark information is rendered, the target display position of the first mark information in each frame of image, that is, the corresponding second mark position of each frame of image, may be calculated in S204. After the second marker position is calculated, the first marker information may be rendered based on the second marker position in each frame of image subsequent to the first frame of image.
In the method, the second mark position corresponding to each frame of image after the first frame of image adopts the first state data before the first mark information is drawn and the second state data after the first mark information is drawn, and the position of the first marker information in the first frame image, i.e., the first marker position, the calculated position, and, therefore, the second mark position is determined based on the state data of the first client device before and after the first mark information is drawn, so that even if the first client device moves and the state changes, the first mark information can be ensured to be displayed at the corresponding position in each frame of image after the first frame of image, the first marking information is always kept at the display position corresponding to the fault point of the industrial field and is always used for indicating the fault point of the industrial field, and the first marking information is prevented from being lost.
S206, sending a first video stream to the second client device, wherein the first video stream comprises: a first frame image drawing the first mark information, and at least one frame image following the first frame image drawing the first mark information.
The second client device may be a client device of the second user, i.e., the sought after user. The first client device may send the first video stream to a second client device through a server.
It should be further noted that, in the video call application scenario in the instant messaging, after the first client device collects the frame images with the preset frame number, the frame images with the preset frame number may be transmitted to the second client device, so as to implement the video call between the first client device and the second client device. Therefore, the first video stream transmitted by the first client device may include at least: the specific number of frame images after the first frame image can be determined based on a preset communication protocol of the remote auxiliary overhaul software.
Of course, in a case that the first mark information is drawn in at least one frame of image after the first frame of image, the first client device displays, in addition to the first frame of image in which the first mark information is drawn, at least one frame of image after the first frame of image in which the first mark information is drawn, so as to realize display of the first video stream in which the first mark information is drawn.
Accordingly, the second client device may sequentially play and display the first video stream based on the sequence of the frame images in the first video stream after receiving the first video stream.
The remote auxiliary overhaul method provided by the embodiment of the application can display the first frame of image collected by the first client device, drawing first marker information in the first frame image in accordance with an input marking operation for a first marker position in the first frame image, wherein the first marking information is marking information corresponding to a fault point of the industrial field marked by the first user, and collects first state data of the first client device when the first marker information is drawn and second state data of the first client device after the first marker information is drawn, and calculating a second mark position according to the first mark position, the first state data and the second state data, then drawing first mark information at the second mark position in at least one frame of image after the first frame of image, and sending a first video stream to a second client device, wherein the first video stream comprises: drawing a first frame image of the first mark information, and drawing at least one frame image following the first frame image of the first mark information. In the remote auxiliary overhaul method provided by this embodiment, when the first mark information is drawn in the first frame image, the first state data, the first mark information, and the first mark information of the first client device may be collected, and after the first mark information is drawn in the first frame image, the second state data of the first client device is also collected, so that the second mark position may be calculated based on the state data before and after the mark and the first mark position, and the first mark information is drawn at the second mark position in the frame image after the first frame image, so that even if the first client device moves and the state changes, the first mark information drawn in each frame image after the first frame image may be ensured to be displayed at the display position corresponding to the fault point, and the accuracy of the display position of the first mark information in the video stream is ensured, therefore, the accuracy of the display position of the first mark information in the first video stream displayed by the second client device is ensured, the second user can accurately know the condition of the fault point of the industrial field based on the display position of the first mark information, the communication description flow of the first user and the second user based on the fault point is reduced, the communication efficiency of the fault point of the industrial field is improved, and the overhaul progress is improved.
The remote auxiliary maintenance method provided by the embodiment can help other users with higher maintenance capability through video call if the user on the industrial site cannot solve the fault, and accurately indicate the fault point on the industrial site through the first mark information drawn in the frame image in the video call process, thereby avoiding the defect that the fault is difficult to describe in telephone communication.
Optionally, the embodiment of the present application may further provide an implementation manner of calculating the position of the second mark. Fig. 3 is a schematic flowchart of calculating a second mark position in a remote auxiliary repair method according to an embodiment of the present application. As shown in fig. 3, calculating the second mark position according to the first mark position, the first state data and the second state data in S204 as shown above may include:
s301, calculating the mark offset according to the first state data and the second state data.
For example, a state change of the second client device before and after drawing the first mark information may be determined based on the first state data and the second state data, and a target position offset corresponding to the first mark information may be determined based on the state change, that is, the mark offset. The mark offset amount may be used to represent a shift of the first mark information from the first frame image to a display position in each frame image subsequent to the first frame image, i.e., a mark offset amount from the first mark position.
For at least one frame of image after the first frame of image, each frame of image has corresponding second state data, and in a possible implementation, the mark offset corresponding to each frame of image can be calculated according to the first state data and the second state data corresponding to each frame of image.
S302, calculating the second mark position according to the first mark position and the mark offset.
For example, a second marker position corresponding to each frame of image after the first frame of image may be calculated according to the first marker position and the marker offset corresponding to each frame of image. The second mark position is the target display position of the first mark information in each frame of image after the first frame of image.
In the method of the embodiment, the mark offset can be calculated first through the first state data and the second state data, and then based on the first mark position and the mark offset, the second mark position can ensure that the display position of the first mark information in each frame of image after the first frame of image can be more accurate, thereby avoiding the technical defect that the mark of the first mark information is inaccurate due to the movement of the first client device.
On the basis of any one of the remote auxiliary overhaul methods shown above, the embodiments of the present application may also provide an example implementation of the remote auxiliary overhaul method, as follows for an explanation of an example by drawing a possible implementation of the first marker information at the second marker position. Fig. 4 is a flowchart of a method for mapping first marker information at a second marker position in a remote auxiliary service method according to an embodiment of the present application. As shown in fig. 4, the drawing the first mark information at the second mark position in at least one frame image after the first frame image in S205 may include:
s401, judging whether the second mark position exceeds the display range of at least one frame of image behind the first frame of image.
S402, if the second mark position does not exceed the display range, drawing the first mark information at the second mark position in at least one frame of image after the first frame of image.
If the second mark position does not exceed the display range, the position where the first mark information should be displayed can be determined, and the second mark position is still within the interface display range of the first client device, then the fault point of the industrial field can be determined to still display the second mark position in the frame image, that is, the first mark information can be drawn at the second mark position to indicate the fault point of the industrial field.
Optionally, the method may further include:
s403, if the second mark position exceeds the display range of a second frame image after the first frame image, sending a second video stream to the second client device, where the second video stream includes: the second frame image of the first mark information is not drawn.
Otherwise, if the second mark position exceeds the display range of the second frame image, the position where the first mark information should be displayed may be determined, the second mark position has exceeded the interface display range of the first client device, and the drawing of the first mark information is not required, meanwhile, the above-mentioned operation of S204 is also performed, the second mark position is recalculated, and if the recalculated second mark position is within the display range of the corresponding frame image, the drawing of the first mark information is performed again at the recalculated second mark position in the corresponding frame image.
If the second mark position exceeds the display range of the second frame image, the second video stream including the second frame image without the first mark information is displayed without drawing the first mark information in the second frame image.
It should be noted that the second video stream at least includes: the second frame image in which the first mark information is not drawn may further include: a frame image before the second frame image, and/or other frame images such as a frame image after the second frame image. Similar to the first video stream, the number of frames of the frame images included in the second video stream can be determined based on the predetermined communication protocol of the remote auxiliary service software.
The embodiment provides a method, in the case that the second mark position does not exceed the display range of at least one frame image after the first frame image, the first marker information is rendered at a second marker position in at least one frame of image subsequent to the first frame of image, and in the case where the second mark position is out of the display range of the second frame image, transmitting a second video stream including the second frame image in which the first mark information is not drawn to the second client apparatus, and as such, the first mark information displayed in the video stream picture can be displayed during the process of the first client device and the second client device carrying out the video call, the first mark information can move along with the movement of the first client device, once the first mark information moves out of the display range, the first mark information is not displayed any more, and if the first mark information moves into the display range, the first mark information is displayed again, so that the flexible dynamic display of the first mark information in the video call process is realized.
Optionally, an implementation manner of deleting the first flag information may also be provided in the embodiments of the present application. Fig. 5 is a schematic flowchart of deleting first flag information in a remote auxiliary overhaul method provided in an embodiment of the present application. As shown in fig. 5, the method may further include:
s501, according to an input first marker deleting operation for the first marker information, deleting the first marker information drawn in a third frame image after the first frame image.
The mark deleting operation may be a touch operation of a deleting control preset in a third frame image on which the first mark information is drawn by the first user through display, an input deleting operation, a mark deleting operation input through voice, a mark deleting operation input through an external input device, or a mark deleting operation input through a preset soft keyboard or a preset shortcut key of an external keyboard. Of course, the first mark deletion operation may be implemented in other forms, and the embodiments of the present application are not limited thereto.
After receiving the first mark deletion operation, the third frame image, that is, the first mark information drawn in the current frame image when receiving the first mark deletion operation, can be deleted, and the pre-recorded recording information of the first mark information can also be deleted. The recording information of the first mark information may include at least one of an initial display position of the first mark information, i.e., a first mark position, a display form of the first mark information, a shape of the first mark information, a color of the first mark information, and the like, for example.
S502, sending a third video stream to the second client device, where the third video stream includes: deleting the third frame image after the first mark information.
It should be noted that the third video stream at least includes: deleting the third frame image after the first mark information, in addition, the method may further include: a frame image before the third frame image, and/or other frame images such as a frame image after the third frame image. Similar to the first video stream, the number of frames of the frame images included in the third video stream can be determined based on the predetermined communication protocol of the remote auxiliary service software.
Optionally, in a case where the first mark information drawn in the third frame image is deleted, the first client device further displays a third video stream including the third frame image after the deletion of the first mark information.
The second client device may display the third video stream including the third frame image after deleting the first mark information in a case where the third video stream is received.
The remote auxiliary overhaul method provided by the embodiment can delete the first marker information in the third frame of image based on the received first marker deletion operation of the first marker information, and transmit the third video stream including the third frame of image after the first marker information is deleted to the second client device, so that the first marker information in the picture in the video stream displayed by the second client device is deleted, thereby implementing synchronous deletion of the first marker information in the video call, ensuring that the second user knows the update of the first marker information in time in the video call process, and implementing the communication instantaneity.
Optionally, an implementation manner of drawing the first mark information may also be provided in the embodiments of the present application. Fig. 6 is a schematic flow chart illustrating drawing of second marker information in a remote auxiliary overhaul method according to an embodiment of the present application. As shown in fig. 6, the method may further include:
s601, receiving a fourth frame image which is sent by the second client device and drawn with the second mark information.
The second mark information is mark information of a solution corresponding to the fault point marked by the fourth frame of image displayed by the second user through the second client device.
The fourth frame image is a frame image corresponding to the second marking message drawn in the video stream of the industrial site acquired by the first client device. The second user may input a marking operation for a third marking position in the fourth frame image in a case where the second client apparatus displays the fourth frame image, and the second client apparatus may draw the second marking information at the third marking position in the fourth frame image according to the marking operation. The second flag information may include, for example: information indicative of at least one solution step in the solution corresponding to the failed point.
The second client device may also display a fourth frame image on which the second mark information is drawn, in a case where the second mark information is drawn. Meanwhile, the fourth frame image drawing the second mark information is also transmitted to the first client device through the server, so that the second client device can display the second mark information marked by the second user.
In the case of sending the fourth frame image on which the second mark information is drawn to the first client device, the second client device may also send record information of the second mark information to the first client device, and the record information may include, for example: the display position of the second mark information in the fourth frame image is at least one of a third mark position, a display form of the second mark information, a shape of the second mark information, a color of the second mark information, and the like.
S602, collecting third state data of the first client device when the fourth frame image is displayed and fourth state data of the first client device after the fourth frame image is displayed.
And under the condition that a fourth frame image drawn with second mark information is received, acquiring the frame identifier or time information of the fourth frame image, and acquiring third state data when the fourth frame image is acquired when the fourth frame image is displayed and fourth state data after the fourth frame image is displayed on the basis of the frame identifier or the time information.
In a possible implementation manner, the process of acquiring the third state data and the fourth state data may be similar to the above implementation process of acquiring the first state data and the second state data, which is specifically referred to above and is not described herein again.
And S603, calculating a fourth mark position according to the third mark position, the third state data and the fourth state data, wherein the third mark position is the display position of the second mark information in the fourth frame image.
The specific implementation process of S603 may be similar to the specific process of S204, and for specific description, refer to the foregoing, and will not be described herein again.
And S604, drawing the second mark information at the fourth mark position in at least one frame of image after the fourth frame of image.
The specific implementation process of S604 may be similar to the specific process of S205, and for specific description, refer to the above, which is not described herein again.
S605, displaying a fourth video stream, wherein the fourth video stream comprises: drawing a fourth frame image of the second mark information, and drawing at least one frame image following the fourth frame image of the second mark information.
The first client device may display in sequence based on the order of the frame images in the fourth video stream. At the same time, the first client device may also transmit the fourth video stream to the second client device, such that the second client device displays the video stream including the frame image on which the second marker information is drawn.
According to the method provided by the embodiment, the drawing position of the second marker information in at least one frame of image after the fourth frame of image can be updated according to the third state data and the fourth state data of the first client device, and the display position of the second marker information in the fourth video stream displayed on the interface of the client device can be guaranteed to change along with the movement of the first client device, so that the flexible and accurate change of the second marker information is realized, the first user can accurately know the solution indicated by the second marker information, the communication frequency of the first user and the second user for the solution of the fault point can be reduced, the communication efficiency of the solution is improved, meanwhile, the labor cost increased by newly dispatching maintenance personnel is avoided, the fault is solved as fast as possible, and the time length of waiting for maintenance of the client in an industrial field is reduced.
It should be noted that, in the actual application process of the remote auxiliary overhaul method, the first marking information and the second marking information may be marked for multiple times respectively to achieve multiple marking communication, and in order to avoid interference caused by multiple marking, the method may further perform functions such as deleting the first marking information and the second marking information respectively.
As follows, the deletion of the second flag information is illustrated by an example, and fig. 7 is a flowchart of a method for deleting the second flag information in a remote auxiliary service method according to an embodiment of the present application. Optionally, on the basis of the method shown in fig. 6, the method may further include:
and S701, receiving mark deletion indication information sent by the second client device, wherein the mark deletion indication information is information sent by the second client device according to input second mark deletion operation aiming at the second mark information.
The second deletion marking operation may be a touch operation of the second user by displaying a deletion control preset in the frame image on which the second marking information is drawn, an input deletion operation, or a deletion operation input by the second user by voice or in another form.
In the case of receiving the second delete marker operation, the second client device may delete the second marker information in the fourth frame image or the subsequent other frame image, for example, the fifth frame image.
S702, according to the mark deletion instruction information, deleting the second mark information drawn in the fifth frame image after the fourth frame image.
The first client apparatus may delete the drawn second marker information in the fifth frame image and may also delete the previously recorded recording information of the second marker information in the case where the marker deletion instruction information is received.
S703, displaying a fifth video stream, where the fifth video stream includes: deleting the fifth frame image following the second mark information.
The first client device may display in sequence based on an order of frame images in the fifth video stream. At the same time, the first client device may also transmit the fifth video stream to the second client device, such that the second client device displays the video stream including the frame image from which the second marker information was deleted.
The method provided by the embodiment provides a function of deleting second mark information marked by the second client device in the process of carrying out the video call by the first client device and the second client device, so that the first client device and the second client device can conveniently carry out marking communication for multiple times aiming at the solution indicated by the second mark information, and the solution of the fault point of the industrial field is ensured.
It should be noted that, even if the first user does not input the first tag deletion operation, or the second user does not input the second tag deletion operation, when receiving the end operation input by any user through the corresponding client device, the video call between the first client device and the second client device is ended, and all the tag information previously tagged and the recorded information of the recorded tag information are deleted together, which is convenient for realizing the video call based on the tag information again.
The following describes a device, equipment, a storage medium, and the like for executing the remote auxiliary repair method provided by the present application, and specific implementation processes and technical effects thereof are referred to above, and will not be described again below.
Fig. 8 is a schematic view of a remote auxiliary service apparatus provided in an embodiment of the present application, and as shown in fig. 8, the remote auxiliary service apparatus 800 may include:
a display module 801, configured to display a first frame image acquired by a first client device;
a drawing module 802, configured to draw first marking information in a first frame image according to an input marking operation for a first marking position in the first frame image, where the first marking information is marking information corresponding to a fault point of an industrial field marked by a first user;
the collecting module 803 is configured to collect first state data of the first client device when the first marker information is drawn and second state data of the first client device after the first marker information is drawn;
a calculating module 804, configured to calculate a second mark position according to the first mark position, the first state data, and the second state data;
a drawing module 802, further configured to draw the first marker information at a second marker position in at least one frame of image after the first frame of image;
a sending module 805, configured to send a first video stream to a second client device, where the first video stream includes: drawing a first frame image of the first mark information, and drawing at least one frame image following the first frame image of the first mark information.
Optionally, the calculating module 804 is specifically configured to: calculating a mark offset according to the first state data and the second state data; a second mark position is calculated based on the first mark position and the mark offset.
Optionally, the collecting module 803 is specifically configured to: acquiring first attitude data of a first client device when drawing first marker information, wherein the first state data comprises: first attitude data; acquiring acceleration data and second attitude data of the first client device after the first marker information is drawn, the second state data including: acceleration data and the second attitude data.
Optionally, the drawing module 802 is specifically configured to, according to the marking operation, superimpose and display the first marking information as augmented reality information at a first marking position in the first frame image.
Optionally, the drawing module 802 is specifically configured to display the first marker information as augmented reality information in a manner of being superimposed on a second marker position in at least one frame of image after the first frame of image.
Optionally, the drawing module 802 is specifically configured to determine whether the second mark position exceeds a display range of at least one frame of image after the first frame of image; and if the second mark position does not exceed the display range, drawing the first mark information at the second mark position in at least one frame of image after the first frame of image.
Optionally, the sending module 805 is further configured to send a second video stream to the second client device if the second mark position exceeds a display range of a second frame image after the first frame image, where the second video stream includes: the second frame image of the first mark information is not drawn.
Optionally, the remote auxiliary service apparatus 800 may further include:
the first deleting module is used for deleting first mark information drawn in a third frame image after the first frame image according to input first mark deleting operation aiming at the first mark information;
the sending module 805 is further configured to send a third video stream to the second client device, where the third video stream includes: deleting the third frame image after the first flag information.
Optionally, the remote auxiliary service apparatus 800 may further include:
the receiving module is used for receiving a fourth frame image which is sent by the second client device and is drawn with second mark information, wherein the second mark information is mark information of a solution corresponding to a fault point of a fourth frame image mark displayed by the second client device by a second user;
the collecting module 803 is specifically configured to collect third state data of the first client device when the fourth frame image is displayed and fourth state data of the first client device after the fourth frame image is displayed;
the calculating module 804 is further configured to calculate a fourth mark position according to the third mark position, the third state data, and the fourth state data, where the third mark position is a display position of the second mark information in the fourth frame image;
a drawing module 802, further configured to draw the second marker information at a fourth marker position in at least one frame of image after the fourth frame of image;
the display module 801 is further configured to display a fourth video stream, where the fourth video stream includes: drawing a fourth frame image of the second mark information, and drawing at least one frame image following the fourth frame image of the second mark information.
Optionally, the receiving module is further configured to receive mark deletion instruction information sent by the second client device, where the mark deletion instruction information is information sent by the second client device according to an input second mark deletion operation for the second mark information;
the remote auxiliary service apparatus 800 may further include:
the second deleting module is used for deleting second mark information drawn in a fifth frame image after the fourth frame image according to the mark deletion indication information;
the display module 801 is further configured to display a fifth video stream, where the fifth video stream includes: and deleting the fifth frame image after the second mark information.
The above-mentioned apparatus is used for executing the method provided by the foregoing embodiment, and the implementation principle and technical effect are similar, which are not described herein again.
These above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Fig. 9 is a schematic diagram of a client device according to an embodiment of the present application. The client device 900 includes: memory 901, processor 902. The memory 901 and the processor 902 are connected by a bus.
The memory 901 is used for storing programs, and the processor 902 calls the programs stored in the memory 901 to execute the above method embodiments. The specific implementation and technical effects are similar, and are not described herein again.
Optionally, the invention also provides a program product, for example a computer-readable storage medium, comprising a program which, when being executed by a processor, is adapted to carry out the above-mentioned method embodiments.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (13)
1. A remote auxiliary overhaul method is characterized by comprising the following steps:
displaying a first frame of image acquired by first client equipment;
drawing first marking information in the first frame image according to input marking operation aiming at a first marking position in the first frame image, wherein the first marking information is marking information marked by a first user and corresponding to a fault point of an industrial field;
acquiring first state data of the first client device when the first mark information is drawn and second state data of the first client device after the first mark information is drawn;
calculating a second marker position based on the first marker position, the first state data, and the second state data;
drawing the first marker information at the second marker position in at least one frame of image subsequent to the first frame of image;
sending a first video stream to a second client device, the first video stream comprising: drawing the first frame image of the first mark information, and drawing at least one frame image subsequent to the first frame image of the first mark information.
2. The method of claim 1, wherein calculating a second marker position based on the first marker position, the first state data, and the second state data comprises:
calculating a mark offset according to the first state data and the second state data;
and calculating the second mark position according to the first mark position and the mark offset.
3. The method of claim 1, wherein collecting first state data of the first client device while rendering the first markup information and second state data of the first client device after rendering the first markup information comprises:
acquiring first attitude data of the first client device when the first marker information is drawn, wherein the first state data comprises: the first pose data;
acquiring acceleration data and second posture data of the first client device after the first marker information is drawn, wherein the second state data comprises: the acceleration data and the second pose data.
4. The method according to claim 1, wherein said drawing first marking information in the first frame image according to an input marking operation for a first marking position in the first frame image comprises:
and according to the marking operation, displaying the first marking information as augmented reality information at the first marking position in the first frame image in an overlapping mode.
5. The method according to claim 1, wherein said rendering the first marker information at the second marker position in at least one frame of image subsequent to the first frame of image comprises:
and displaying the first marker information as augmented reality information at the second marker position in at least one frame of image after the first frame of image in an overlapping mode.
6. The method according to claim 1, wherein said rendering the first marker information at the second marker position in at least one frame of image subsequent to the first frame of image comprises:
judging whether the second mark position exceeds the display range of at least one frame of image behind the first frame of image;
and if the second mark position does not exceed the display range, drawing the first mark information at the second mark position in at least one frame of image after the first frame of image.
7. The method of claim 6, further comprising:
if the second mark position exceeds the display range of a second frame image after the first frame image, sending a second video stream to the second client device, wherein the second video stream comprises: the second frame image of the first marker information is not drawn.
8. The method of claim 1, further comprising:
deleting the first mark information drawn in a third frame image after the first frame image according to an input first mark deletion operation for the first mark information;
sending a third video stream to the second client device, the third video stream comprising: deleting the third frame image after the first mark information.
9. The method according to any one of claims 1-8, further comprising:
receiving a fourth frame of image which is sent by the second client device and is drawn with second mark information, wherein the second mark information is mark information of a solution corresponding to the fault point of the fourth frame of image mark displayed by the second client device by a second user;
acquiring third state data of the first client device when the fourth frame of image is displayed and fourth state data of the first client device after the fourth frame of image is displayed;
calculating a fourth mark position according to a third mark position, the third state data and the fourth state data, wherein the third mark position is a display position of the second mark information in the fourth frame image;
drawing the second marker information at the fourth marker position in at least one frame of image subsequent to the fourth frame of image;
displaying a fourth video stream, the fourth video stream comprising: drawing the fourth frame image of the second mark information, and drawing at least one frame image subsequent to the fourth frame image of the second mark information.
10. The method of claim 9, further comprising:
receiving mark deletion indication information sent by the second client device, wherein the mark deletion indication information is information sent by the second client device according to input second mark deletion operation aiming at the second mark information;
deleting the second mark information drawn in a fifth frame image after the fourth frame image according to the mark deletion indication information;
displaying a fifth video stream, the fifth video stream comprising: deleting the fifth frame image after the second mark information.
11. A remote auxiliary service device, comprising:
the display module is used for displaying a first frame image acquired by first client equipment;
the system comprises a drawing module, a first marking module and a second marking module, wherein the drawing module is used for drawing first marking information in a first frame image according to input marking operation aiming at a first marking position in the first frame image, and the first marking information is marking information marked by a first user and corresponding to a fault point of an industrial field;
the acquisition module is used for acquiring first state data of the first client device when the first mark information is drawn and second state data of the first client device after the first mark information is drawn;
a calculation module, configured to calculate a second mark position according to the first mark position, the first state data, and the second state data;
the drawing module is further used for drawing the first mark information at the second mark position in at least one frame of image after the first frame of image;
a sending module, configured to send a first video stream to a second client device, where the first video stream includes: drawing the first frame image of the first mark information, and drawing at least one frame image subsequent to the first frame image of the first mark information.
12. A client device, comprising: a memory storing a computer program executable by the processor, and a processor implementing the remote assisted service method of any of the preceding claims 1-10 when executing the computer program.
13. A storage medium having stored thereon a computer program which, when read and executed, implements a remote assisted service method as claimed in any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011455534.5A CN112492281A (en) | 2020-12-10 | 2020-12-10 | Remote auxiliary maintenance method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011455534.5A CN112492281A (en) | 2020-12-10 | 2020-12-10 | Remote auxiliary maintenance method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112492281A true CN112492281A (en) | 2021-03-12 |
Family
ID=74916684
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011455534.5A Pending CN112492281A (en) | 2020-12-10 | 2020-12-10 | Remote auxiliary maintenance method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112492281A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106339094A (en) * | 2016-09-05 | 2017-01-18 | 山东万腾电子科技有限公司 | Interactive remote expert cooperation maintenance system and method based on augmented reality technology |
| CN110189416A (en) * | 2019-05-29 | 2019-08-30 | 国家电网有限公司 | The remote guide method and system of Overhaul site |
| CN110322024A (en) * | 2019-06-21 | 2019-10-11 | 上海翊视皓瞳信息科技有限公司 | A kind of job guide system and method based on wearable device |
| CN110708384A (en) * | 2019-10-12 | 2020-01-17 | 西安维度视界科技有限公司 | Interaction method, system and storage medium of AR-based remote assistance system |
| US20200143593A1 (en) * | 2018-11-02 | 2020-05-07 | General Motors Llc | Augmented reality (ar) remote vehicle assistance |
| CN111723646A (en) * | 2020-04-24 | 2020-09-29 | 四川科华天府科技有限公司 | Method and device for remotely assisting in maintaining equipment |
-
2020
- 2020-12-10 CN CN202011455534.5A patent/CN112492281A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106339094A (en) * | 2016-09-05 | 2017-01-18 | 山东万腾电子科技有限公司 | Interactive remote expert cooperation maintenance system and method based on augmented reality technology |
| US20200143593A1 (en) * | 2018-11-02 | 2020-05-07 | General Motors Llc | Augmented reality (ar) remote vehicle assistance |
| CN110189416A (en) * | 2019-05-29 | 2019-08-30 | 国家电网有限公司 | The remote guide method and system of Overhaul site |
| CN110322024A (en) * | 2019-06-21 | 2019-10-11 | 上海翊视皓瞳信息科技有限公司 | A kind of job guide system and method based on wearable device |
| CN110708384A (en) * | 2019-10-12 | 2020-01-17 | 西安维度视界科技有限公司 | Interaction method, system and storage medium of AR-based remote assistance system |
| CN111723646A (en) * | 2020-04-24 | 2020-09-29 | 四川科华天府科技有限公司 | Method and device for remotely assisting in maintaining equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108830894B (en) | Remote guidance method, device, terminal and storage medium based on augmented reality | |
| US9639988B2 (en) | Information processing apparatus and computer program product for processing a virtual object | |
| JP6296056B2 (en) | Image processing apparatus, image processing method, and program | |
| US20180376104A1 (en) | Method and apparatus for sharing information during video call | |
| CN114140528A (en) | Data annotation method and device, computer equipment and storage medium | |
| JP2019153274A (en) | Position calculation device, position calculation program, position calculation method, and content addition system | |
| CN113031591B (en) | Exception handling method and device for material pushing robot, server and storage medium | |
| WO2019085081A1 (en) | Method and apparatus for correcting target position in map, computer device, and storage medium | |
| CN111882634A (en) | Image rendering method, device and equipment and storage medium | |
| EP4336385A1 (en) | Method and apparatus for updating target detection model | |
| CN114363161B (en) | Abnormal equipment positioning method, device, equipment and medium | |
| KR20230088332A (en) | Data annotation methods, devices, systems, devices and storage media | |
| CN111078819A (en) | Application sharing method and electronic equipment | |
| CN112818898B (en) | Model training method and device and electronic equipment | |
| WO2023237065A1 (en) | Loop closure detection method and apparatus, and electronic device and medium | |
| CN112492281A (en) | Remote auxiliary maintenance method, device, equipment and storage medium | |
| CN115314684B (en) | Method, system and equipment for inspecting railroad bridge and readable storage medium | |
| CN113961161B (en) | Data display method, system, mobile terminal, storage medium and program product | |
| CN114740854A (en) | Robot obstacle avoidance control method and device | |
| US11189050B2 (en) | Learning template representation libraries | |
| CN109922100B (en) | Information processing method, terminal and server | |
| US20220341737A1 (en) | Method and device for navigating | |
| CN113542332B (en) | Customer service video interaction method and equipment based on positioning labels | |
| WO2024057779A1 (en) | Information processing device, program, and information processing system | |
| CN113610987B (en) | Mixed reality space labeling method and system based on three-dimensional reconstruction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210312 |