CN115674180A

CN115674180A - Control method, control device, electronic device and storage medium

Info

Publication number: CN115674180A
Application number: CN202110851039.4A
Authority: CN
Inventors: 陈子柯
Original assignee: Ronovo Shanghai Medical Science and Technology Ltd
Current assignee: Ronovo Shanghai Medical Science and Technology Ltd
Priority date: 2021-07-27
Filing date: 2021-07-27
Publication date: 2023-02-03

Abstract

The embodiment of the invention discloses a control method, a control device, electronic equipment and a storage medium. The method comprises the following steps: monitoring a trigger signal sent to an operation component on operation equipment; wherein the trigger signal is generated when the operation part is operated, and comprises a trigger type and a trigger parameter; generating a corresponding operation signal based on a trigger type and a trigger parameter in the trigger signal, wherein the operation signal comprises an operation object control signal and an object switching signal, the object switching signal is used for controlling a display device to switch an information display interface and triggering an operation object corresponding to the information display interface, and the operation object control signal is used for controlling the state of the corresponding operation object; and sending the operation signal to corresponding equipment so as to enable the equipment to execute the operation signal. By the technical scheme disclosed by the embodiment of the invention, the operation convenience of the operation equipment is improved and the efficiency of task operation is improved.

Description

Control method, control device, electronic device and storage medium

Technical Field

The embodiment of the invention relates to the technical field of medical instrument control, in particular to a control method, a control device, electronic equipment and a storage medium.

Background

The surgical robot in the prior art remotely controls the robot to perform surgical operations through a doctor operating system for a user, wherein the doctor operating system includes a 3D surgical interface, a 2D configuration interface, a handle, a foot pedal and other pieces of equipment.

Under the current system setting, when a doctor needs to set the relevant setting of a doctor operating system in the operation process, the doctor needs to leave the 3D view and leave the operation operating system operating equipment, the doctor uses fingers to perform the relevant setting on the 2D touch screen, and after the setting is finished, the doctor needs to re-enter the 3D view and put the glove into an operating handle to continue the operation.

The existing operating equipment can only realize the control of the operation through a handle and a pedal plate, and an independent control device is required to be configured for the control of a configuration interface; therefore, the operation and the configuration are separately arranged, so that the operation complexity in the operation process is increased, and the smoothness of the operation is influenced.

Disclosure of Invention

The invention provides a control method, a control device, an electronic device and a storage medium, which are used for increasing the operation convenience of an operation device and improving the efficiency of task operation.

In a first aspect, an embodiment of the present invention provides a control method, which is applied to an operating device, and the method includes:

monitoring a trigger signal sent to an operation component on operation equipment; wherein the trigger signal is generated when the operation part is operated, and comprises a trigger type and a trigger parameter;

generating a corresponding operation signal based on a trigger type and a trigger parameter in the trigger signal, wherein the operation signal comprises an operation object control signal and an object switching signal, the object switching signal is used for controlling a display device to switch an information display interface and triggering an operation object corresponding to the information display interface, and the operation object control signal is used for controlling the state of the corresponding operation object;

and sending the operation signal to corresponding equipment so as to enable the equipment to execute the operation signal, wherein the equipment comprises display equipment and task execution equipment.

Optionally, the operating device comprises two operating regions, each operating region is provided with a plurality of operating components,

the generating of the corresponding operation signal based on the trigger type and the trigger parameter in the trigger signal comprises:

and generating a corresponding operation signal based on the first trigger signal sent by the operation part in the first operation area and/or the second trigger signal sent by the operation part in the second operation area.

Optionally, the generating a corresponding operation signal based on the first trigger signal sent by the operation component in the first operation region and/or the second trigger signal sent by the operation component in the second operation region includes:

and if the second trigger signal is received within a preset time interval after the first trigger signal is received, or the first trigger signal is received within a preset time interval after the second trigger signal is received, determining a corresponding operation signal based on the first trigger signal, the second trigger signal and a preset operation signal library.

Optionally, the determining the corresponding operation signal based on the first trigger signal, the second trigger signal and a preset operation signal library includes:

if a combined operation signal of the first trigger signal and the second trigger signal is matched in the preset operation signal library, determining the combined operation signal as a corresponding operation signal;

if the combined operation signal of the first trigger signal and the second trigger signal is not matched in the preset operation signal library, determining the first trigger signal and the second trigger signal as operation signals corresponding to the first operation area and the second operation area respectively.

Optionally, the operating device includes at least two sub-devices, each of the sub-devices is provided with a plurality of operating components, and the sub-devices generate operating signals based on the trigger signals sent by the operating components and send the operating signals to corresponding devices independently.

Optionally, the operating device includes a master device and at least one slave device, each of the master device and the slave device is provided with a plurality of operating components, the slave device sends a trigger signal to the master device, and the master device is configured to generate an operating signal based on the trigger signal sent by each slave device and the trigger signal sent by its own operating component.

Optionally, the information display interface includes a 2D display interface and a 3D display interface, where an operation object corresponding to the 2D display interface includes each content object in the display interface, and an operation object corresponding to the 3D display interface is a task execution device.

Optionally, the 2D display interface includes a configuration interface.

In a second aspect, an embodiment of the present invention further provides a control method, applied to a display device, where the method includes:

receiving operation signals sent by each operation device;

determining an operation signal sent by any one operation device as a target operation signal, or determining the target operation signal based on the operation signals sent by at least two operation devices;

executing the target operation signal; the target operation signal comprises an operation object control signal and an object switching signal, the object switching signal is used for controlling and switching an information display interface and triggering an operation object corresponding to the information display interface, and the operation object control signal is used for controlling the state of the operation object in the information display interface.

Optionally, the number of the operating devices is at least two;

the determining a target operation signal based on operation signals sent by at least two operation devices comprises:

if a fourth operation signal sent by the second operation device is received within a preset time interval after a third operation signal sent by the first operation device is received, or the third operation signal is received within a preset time interval after the fourth operation signal is received, determining a target operation signal based on the third operation signal, the fourth operation signal and a preset operation signal library.

Optionally, the determining a target operation signal based on the third operation signal, the fourth operation signal and a preset operation signal library includes:

if a combined operation signal of the third operation signal and the fourth operation signal is matched in the preset operation signal library, determining the combined operation signal as a target operation signal;

if the combined operation signal of the third operation signal and the fourth operation signal is not matched in the preset operation signal library, determining the third operation signal and the fourth operation signal as a target operation signal corresponding to the first operation area or the first device and corresponding to the second operation area or the second device respectively.

Optionally, if the target operation signal includes an object switching signal;

correspondingly, the executing the target operation signal includes:

switching the information display interface between a 2D display interface and a 3D display interface based on the object switching signal; or,

and additionally displaying the 2D display interface on the basis of the 3D display interface based on the object switching signal, or switching to the 3D display interface from a mixed interface of the 2D display interface and the 3D display interface.

Optionally, the executing the target operation signal further includes:

controlling on-off states of at least two channels based on the object switching signal, wherein the at least two channels are respectively used for transmitting the 2D display interface and the 3D display interface.

Optionally, the mixed interface of the 2D display interface and the 3D display interface includes: and the 2D display interface and the 3D display interface are superposed to display a formed mixed interface, or the 2D display interface and the 3D display interface are combined to display the formed mixed interface in the display interface.

In a third aspect, an embodiment of the present invention further provides a control apparatus, which is applied to an operating device, and includes:

the trigger signal monitoring module is used for monitoring a trigger signal sent to an operation component on the operation equipment; wherein the trigger signal is generated when the operation part is operated, and comprises a trigger type and a trigger parameter;

an operation signal generation module, configured to generate a corresponding operation signal based on a trigger type and a trigger parameter in the trigger signal, where the operation signal includes an operation object control signal and an object switching signal, the object switching signal is used to control a display device to switch an information display interface and trigger an operation object corresponding to the information display interface, and the operation object control signal is used to control a state of the corresponding operation object;

and the operation object sending module is used for sending the operation signal to corresponding equipment so as to enable the equipment to execute the operation signal, wherein the equipment comprises display equipment and task execution equipment.

In a fourth aspect, an embodiment of the present invention further provides a control apparatus, applied to a display device, where the apparatus includes:

the operation signal receiving module is used for receiving operation signals sent by each operation device;

the target operation signal generation module is used for determining the operation signals sent by the equipment as target operation signals or determining the target operation signals based on the operation signals sent by at least two pieces of operation equipment;

the target operation signal execution module is used for executing the target operation signal; the target operation signal comprises an operation object control signal and an object switching signal, the object switching signal is used for controlling and switching an information display interface and triggering an operation object corresponding to the information display interface, and the operation object control signal is used for controlling the state of the operation object in the information display interface.

In a fifth aspect, an embodiment of the present invention further provides an electronic device, where the electronic device includes:

one or more processors;

a storage device for storing one or more programs,

when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the control method as provided by any of the embodiments of the present invention.

In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor, and the computer program implements the control method provided in any embodiment of the present invention.

The technical scheme of the embodiment of the invention specifically comprises the following steps: monitoring a trigger signal sent to an operation component on operation equipment so as to generate a corresponding operation signal based on the trigger signal subsequently, wherein the trigger signal is generated when the operation component is operated, and the specific trigger signal comprises a trigger type and a trigger parameter; generating a corresponding operation signal based on a trigger type and a trigger parameter in the trigger signal, wherein the operation signal includes an operation object control signal and an object switching signal, the object switching signal is used for controlling the display device to switch the information display interface and triggering the operation object corresponding to the information display interface, and the operation object control signal is used for controlling the state of the corresponding operation object; and further sending the operation signal to corresponding equipment to enable the equipment to execute the operation signal, wherein the equipment comprises display equipment and task execution equipment. According to the technical scheme of the embodiment of the invention, the interface switching of the information display interface and the control of the operation object corresponding to the switched interface are realized by monitoring the operation action of the operation component and generating the corresponding operation signal, so that the switching of the interface and the operation of the operation object under different interfaces are realized based on the operation equipment, the operation convenience of the operation equipment is improved, and the efficiency of task operation is improved.

Drawings

In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, a brief description is given below of the drawings used in describing the embodiments. It should be clear that the described figures are only views of some of the embodiments of the invention to be described, not all, and that for a person skilled in the art, other figures can be derived from these figures without inventive effort.

Fig. 1 is a schematic flow chart of a control method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a configuration interface according to an embodiment of the invention;

fig. 3 is a schematic flow chart of a control method according to a second embodiment of the present invention;

fig. 4 is a schematic flow chart of a control method according to a third embodiment of the present invention;

fig. 5 is a schematic structural diagram of a control device according to a fourth embodiment of the present invention;

fig. 6 is a schematic structural diagram of a control device according to a fifth embodiment of the present invention;

fig. 7 is a schematic structural diagram of an electronic device according to a sixth embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some structures related to the present invention are shown in the drawings, not all of them.

Example one

Fig. 1 is a flowchart of a control method according to an embodiment of the present invention, which is applied to an operating device. The present embodiment is applicable to a case where an operation device is operated, and more specifically, to a case where an operation motion to an operation member in the operation device is detected and an operation signal corresponding to the operation member is generated. The method may be performed by a control device, which may be implemented by means of software and/or hardware.

Before describing the technical solution of the present embodiment, an application scenario of the present embodiment is described in an exemplary manner, and of course, the following application scenario is only an optional application scenario, and the technical solution of the present embodiment may also be applied to other application scenarios, which is not limited in the present embodiment. Specifically, exemplary application scenarios include:

the surgical robot of the prior art is a user who remotely controls the robot to perform surgical operations through a doctor operating system, which includes a display device and a series of operation input devices such as a handle and a foot pedal. The display device comprises a 3D view and an independent touch screen, the 3D view is used for displaying 3D images and related instrument information in the operation process of a doctor, and a user obtains the operation process information by observing the images of the 3D view, so that the instrument can be remotely operated to perform an operation. The independent touch screen displays and sets doctor related doctor operating system settings such as motion scaling, 3D view display angle, arm bracket height, pedal fore-and-aft position and the like through a configuration interface, so that parameters (height and the like) of the operating system conform to human engineering.

Under the current system setting, when a doctor needs to set the relevant setting of a doctor operating system in the operation process, the doctor needs to leave the 3D view, meanwhile, the hand needs to leave the operation handle, the relevant setting is carried out on the touch screen by fingers, and after the setting is finished, the doctor needs to re-enter the 3D view and put the glove into the operation handle to continue the operation. Therefore, the operation in the operation process is complicated, and the safety of the operation is affected.

In order to solve the technical problem, before the technical scheme of the embodiment of the present invention is executed, an operating system for executing the technical scheme of the embodiment of the present invention is built, and specifically includes an operating device and a display device. On the basis of the operating system, the control method applied to the operating device provided by the technical solution of the embodiment specifically includes: monitoring a trigger signal sent to an operation component on operation equipment so as to generate a corresponding operation signal based on the trigger signal subsequently, wherein the trigger signal is generated when the operation component is operated, and the specific trigger signal comprises a trigger type and a trigger parameter; generating a corresponding operation signal based on a trigger type and a trigger parameter in the trigger signal, wherein the operation signal includes an operation object control signal and an object switching signal, the object switching signal is used for controlling the display device to switch the information display interface and triggering the operation object corresponding to the information display interface, and the operation object control signal is used for controlling the state of the corresponding operation object; and further sending the operation signal to corresponding equipment to enable the equipment to execute the operation signal, wherein the equipment comprises display equipment and task execution equipment. The technical scheme of the embodiment of the invention realizes the interface switching of the information display interface and the control of the operation object corresponding to the switched interface by monitoring the operation action of the operation component and generating the corresponding operation signal, realizes the switching of the interface and the operation of the operation object under different interfaces based on the operation equipment, increases the operation convenience of the operation equipment and improves the efficiency of task operation, thereby improving the operation efficiency of an operation user.

As shown in fig. 1, the technical solution of this embodiment specifically includes the following steps:

s110, monitoring a trigger signal sent to an operation component on the operation equipment; wherein the trigger signal is generated when the operation part is operated, and comprises a trigger type and a trigger parameter.

In this embodiment, the operation device may be a device used by the operation user to perform an operation. In particular, the operating device may be at least one operating handle. The operation member includes a member provided on the surface of the operation device or the entire operation device is used as one member. The number of operating parts is at least one. The types of the operation components comprise an operation switching component and an operation object control component, which are respectively used for realizing the switching of the information display interface and controlling the operation object corresponding to the switched information display interface. Specifically, when any operation component is operated, a corresponding trigger signal is generated according to the operation action; the trigger signal comprises a trigger type and a trigger parameter. The triggering type comprises the types of movement, pressing or rotation, and the triggering parameters comprise the parameters of distance, angle and the like.

Specifically, the operation of the operation member may include, but is not limited to, pressing the operation member, moving or rotating the operation member. Optionally, if the operation component is an operation key arranged on the surface of the operation device, a corresponding trigger signal is generated when the operation key is pressed or moved; if the whole operation equipment is operated when the component is operated, a corresponding trigger signal is generated when the whole operation equipment is pressed or moved. Taking the operating part as the whole operating device as an example, monitoring a trigger signal sent to the operating part on the operating device is introduced: specifically, the operation action on the operation device includes six-degree-of-freedom operation, namely translation of X, Y and Z axes and rotation of the X, Y and Z axes; for example, if the operating device is operated to move K millimeters in the positive Y-axis direction, the trigger signal monitored at this time includes that the trigger type is movement in the positive Y-axis direction, and the trigger parameter is K millimeters. Of course, the above embodiments are only optional embodiments, and the trigger signal of the operation component may also be obtained based on other manners, which is not limited in this embodiment.

And S120, generating a corresponding operation signal based on the trigger type and the trigger parameter in the trigger signal.

In this embodiment, the operation signal includes an operation object control signal and an object switching signal, the object switching signal is used to control the display device to switch the information display interface and trigger the operation object corresponding to the information display interface, and the operation object control signal is used to control the state of the corresponding operation object.

Optionally, generating the corresponding operation signal based on the trigger signal may include generating the corresponding operation signal based on the trigger signal of any one of the operation components, that is, generating the corresponding operation signal based on the trigger type and the trigger parameter of the trigger signal of any one of the operation components; generating the corresponding operation signal based on the trigger signal may further include generating the corresponding operation signal based on the trigger signals of the plurality of operation members, i.e., generating the corresponding combined operation signal based on the trigger type and the trigger parameter corresponding to each trigger signal, respectively.

For example, if the operating device is an operating handle, the corresponding operating signal may be generated based on the monitored trigger signal sent by the single handle; the corresponding combined operating signal may also be generated based on the monitored trigger signals sent by multiple (e.g., two) handles. Specifically, the corresponding combined operation signal may be determined based on the time difference between the plurality of monitored trigger signals and a preset operation signal library.

And S130, transmitting the operation signal to corresponding equipment so as to enable the equipment to execute the operation signal, wherein the equipment comprises display equipment and task execution equipment.

In the embodiment of the present invention, the operation signal may only include an operation switching signal, and correspondingly, the operation switching signal is sent to the corresponding device, that is, to the display device, so that the display device performs switching of the information display interface; specifically, the information display interface includes a 2D display interface and a 3D display interface, and correspondingly, the display device performs switching from the 2D display interface to the 3D display interface, or from the 3D display interface to the 2D display interface.

Optionally, the operation signal may also only include an operation object control signal, and correspondingly, the operation object control signal of the operation object corresponding to the information display interface is generated according to the currently displayed information display interface, and the operation object control signal is sent to the corresponding device. Specifically, the operation object corresponding to the 2D display interface includes each content object in the display interface, and the operation object corresponding to the 3D display interface is the task execution device.

Optionally, if the currently displayed information display interface is a 2D display interface, and correspondingly, the operation object corresponding to the current interface includes each content object in the display interface, the operation object control signal is sent to the display device, so that the display device controls each content object in the current information display interface. Each content object in this embodiment includes, but is not limited to, a cursor, a text, a picture, an interface, and other content objects.

Optionally, if the currently displayed information display interface is a 3D display interface, and correspondingly, the operation object corresponding to the current interface is each device in the task execution device, the operation object control signal is sent to the task execution device, so that each device in the execution device can operate the task object. The task execution device in the present embodiment may be a device for performing endoscopic surgery, a device for performing fine carving, or the like.

In some embodiments, the 2D display interface includes a configuration interface, for example, the configuration interface shown in fig. 2, which specifically includes, but is not limited to, a configuration interface corresponding to configuring user data, configuring parameters of an operating device, and configuring parameters of a display device.

Example two

Fig. 3 is a flowchart of a control method according to a second embodiment of the present invention, where in this embodiment, on the basis of the foregoing embodiments, the step "generating a corresponding operation signal based on a trigger type and a trigger parameter in a trigger signal" is refined to "generating a corresponding operation signal based on a trigger type and a trigger parameter in a trigger signal; the operating device includes two operating regions, and a plurality of operating members are provided in each of the operating regions. The explanation of the terms identical or corresponding to the above embodiments is not repeated here. Referring to fig. 3, the control method provided in this embodiment includes:

s210, monitoring a trigger signal sent to an operation component on operation equipment; wherein the trigger signal is generated when the operation part is operated, and comprises a trigger type and a trigger parameter.

S220, generating a corresponding operation signal based on the trigger type and the trigger parameter in the trigger signal; the operating device comprises two operating areas, and a plurality of operating components are arranged in each operating area.

In this embodiment, the operating device includes two operating regions, and for example, the operating device is an operating handle, and for convenience of single-hand operation of a user, the operating regions may be respectively disposed at the top of the handle and at the side end of the handle, or may be respectively disposed in the left region and the right region of the operating handle, so that the user can operate the operating handle with both hands simultaneously. Of course, other multiple regions may be provided in the operation handle, and the number and positions of the operation regions on the operation device are not limited in this embodiment.

Optionally, in this embodiment, when the trigger signals sent to the operation components in different operation areas on the operation device are monitored, the corresponding operation signal is generated based on the first trigger signal sent by the operation component in the first operation area and/or the second trigger signal sent by the operation component in the second operation area. In other words, the corresponding operation signal may be generated based only on the first trigger signal transmitted by the operation member within the first operation region of the first region; the corresponding operation signal can be generated only based on the second trigger signal sent by the operation component in the second operation area; of course, the corresponding operation signal may also be generated based on the first trigger signal sent by the operation member in the first operation region and the second trigger signal sent by the operation member in the second operation region. It can also be stated that the corresponding operation signals generated based on the first trigger signal of the first operation region and the second trigger signal of the second operation region may be a single operation signal or a combined operation signal.

Specifically, the method for determining whether the corresponding operation signal generated based on the first trigger signal of the first operation region and the second trigger signal of the second operation region is a single operation signal or a combined operation signal may be based on a time difference between a timestamp corresponding to the monitored first trigger signal and a timestamp corresponding to the monitored second trigger signal.

Optionally, the determining method specifically includes determining the corresponding operation signal based on the first trigger signal, the second trigger signal, and the preset operation signal library if the second trigger signal is received within a preset time interval after the first trigger signal is received, or the first trigger signal is received within a preset time interval after the second trigger signal is received.

Optionally, the method for determining a corresponding operation signal based on the first trigger signal, the second trigger signal, and the preset operation signal library specifically includes: and if the combined operation signal of the first trigger signal and the second trigger signal is matched in the preset operation signal library, determining the combined operation signal as a corresponding operation signal.

Of course, in some embodiments, if the combined operation signal of the first trigger signal and the second trigger signal is not matched in the preset operation signal library, the first trigger signal and the second trigger signal are respectively determined as the operation signals corresponding to the first operation area and the second operation area respectively.

In some other embodiments, the operation device further includes a master device and at least one slave device, each of the master device and the slave device is provided with a plurality of operation components, the slave device transmits the trigger signal to the master device, and the master device is configured to generate the operation signal based on the trigger signal transmitted by each of the slave devices and the trigger signal transmitted by the own operation component.

In this embodiment, the master device uses the trigger signal sent by the self-operation part as the first trigger signal, uses the trigger signal sent by each slave device as the second trigger signal according to the number of each slave device, and sequentially sorts the trigger signals according to the number of the slave devices. Further, the method for determining the corresponding operation signal according to the trigger signals sent by different operation areas determines the corresponding operation signal for each trigger signal in the master device, and the specific determination method is not described herein again.

In some other embodiments, the operation device further includes at least two sub-devices, each of the sub-devices is provided with a plurality of operation components, and the sub-devices generate operation signals based on the trigger signals sent by the operation components and respectively and independently send the operation signals to corresponding devices, so that the corresponding devices execute the received operation signals.

And S230, sending the operation signal to corresponding equipment so as to enable the equipment to execute the operation signal, wherein the equipment comprises display equipment and task execution equipment.

According to the technical scheme of the embodiment of the invention, the interface switching of the information display interface and the control of the operation object corresponding to the switched interface are realized by monitoring the operation action of the operation component and generating the corresponding operation signal, so that the switching of the interface and the operation of the operation object under different interfaces are realized based on the operation equipment, the operation convenience of the operation equipment is improved, and the efficiency of task operation is improved.

On the basis of the foregoing embodiments, the embodiments of the present invention further provide a preferred embodiment, where when the current information display interface is a 2D display interface, that is, a configuration interface: the screen cursor is moved up and down and left and right by operating the handle as an example: specifically, the operating handle is controlled to rotate anticlockwise by X degrees towards an X axis so as to move up the cursor; controlling the operating handle to rotate clockwise to X degrees towards the Y axis so as to realize downward movement of the cursor; and controlling the operating handle to rotate anticlockwise by X degrees towards the Z axis so as to realize the left movement of the operating handle. And controlling the operating handle to rotate clockwise by X degrees towards the Z axis so as to realize the right movement of the cursor. Of course, the present embodiment also includes control of the operation object, such as determination of selection, cancellation of selection, and dragging, based on other operations performed on the operation handle. Specifically, the operating handle is controlled to move Xmm towards the Z-axis square to realize the determination and selection; the operation handle is controlled to move Xmm to the Z-axis negative square shape to realize the cancellation of selection; of course, after the selection is determined, the operation handle may be moved in the corresponding direction to implement dragging, and of course, text input and image import may also be performed based on the operation handle, which is not limited herein.

On the basis of the above embodiments, the embodiment of the present invention provides another preferred embodiment, that is, in order to facilitate user operation, the technical solution of the embodiment may further adopt two hands to control the operation handle, so as to control the operation object corresponding to the current information display interface. Similarly, when the current information display interface is a 2D display interface, that is, a configuration interface: the up-down and left-right movement of the screen cursor is realized by operating the handle as an example: specifically, in order to realize the upward movement of the cursor, the left operating handle is controlled to move Xmm towards the Y-axis square. And in order to realize the upward movement of the cursor, the left operating handle is controlled to move Xmm towards the negative direction of the Y axis. The left operating handle is controlled to move Xmm towards the negative direction of the X axis in order to realize the downward movement of the cursor. In order to realize the left movement of the cursor, the left operating handle is controlled to move Xmm towards the positive direction of the X axis. In order to realize the right movement of the cursor, the right handle is controlled and operated to move Xmm towards the positive direction of the X axis. Similarly, the present embodiment also includes control of the operation object, such as determination of selection, cancellation of selection, and dragging, based on other operations performed on different operation handles. Specifically, in order to realize the definite selection, a button A of the right handle is pressed; to cancel the selection, the button B of the right handle is pressed; to realize dragging: after the selection is determined, the handle is moved in the corresponding direction; to achieve image, interface or text zooming: after the selection is determined, the left and right handles simultaneously press the B button and move Xmm relative to or in the opposite square. Of course, the above preferred embodiments are only optional embodiments of the embodiments in the present application, and other preferred embodiments may also be mentioned in the technical solution of the present application, and the embodiments are not described herein.

EXAMPLE III

Fig. 4 is a flowchart of a control method according to a third embodiment of the present invention, and is applied to a display device. The present embodiment is applicable to a case where an operation device is operated, and more specifically, to a case where a received operation signal is displayed. The method may be performed by a control device, which may be implemented by means of software and/or hardware. As shown in fig. 4, the method specifically includes the following steps:

and S310, receiving operation signals sent by the operation devices.

In the present embodiment, the operation signal includes an operation switching signal and an operation object control signal.

Specifically, the operation signal transmitted by each operation device may include a single operation signal generated by an operation action of a single operation member; it may be a combined operation signal generated based on operation actions of a plurality of operation members, which is not limited by the present embodiment.

And S320, determining the operation signal sent by any one operation device as a target operation signal, or determining the target operation signal based on the operation signals sent by at least two operation devices.

In this embodiment, if the number of the operation devices is one, an operation signal sent by any one of the operation devices is received, and the operation signal is taken as a target signal. And if the number of the operating devices is multiple, acquiring time stamps corresponding to the received operating signals, and determining the target operating signals according to the time stamps. Optionally, the method for determining the target operation signal based on each timestamp includes: and if the fourth operation signal sent by the second operation device is received within a preset time interval after the third operation signal sent by the first operation device is received, or the third operation signal is received within a preset time interval after the fourth operation signal is received, determining the target operation signal based on the third operation signal, the fourth operation signal and a preset operation signal library.

Optionally, the method for determining the target operation signal based on the third operation signal, the fourth operation signal and the preset operation signal library includes: and if the combined operation signal of the third operation signal and the fourth operation signal is matched in the preset operation signal library, determining the combined operation signal as a target operation signal.

In some other embodiments, if a combined operation signal of the third operation signal and the fourth operation signal is not matched in the preset operation signal library, the third operation signal and the fourth operation signal are respectively determined as a target operation signal corresponding to the first operation area or the first device, and the second operation area or the second device.

And S330, executing the target operation signal.

In the embodiment of the present invention, the target operation signal includes an operation object control signal and an object switching signal, the object switching signal is used to control switching of the information display interface and trigger an operation object corresponding to the information display interface, and the operation object control signal is used to control a state of the operation object in the information display interface.

Specifically, corresponding operation is executed according to the type of the target operation signal, specifically, when the target operation signal is an operation object control signal, that is, according to the currently displayed interface, an operation object corresponding to the interface is determined, and the operation object control signal is executed for the operation object.

Specifically, when the target operation object comprises a pair of switching signals, switching the information display interface between the 2D display interface and the 3D display interface based on the object switching signals; or, the 2D display interface is additionally displayed on the basis of the 3D display interface based on the object switching signal, or the mixed interface of the 2D display interface and the 3D display interface is switched to the 3D display interface. In this embodiment, the hybrid interface of the 2D display interface and the 3D display interface includes: the 2D display interface and the 3D display interface are overlapped to display a formed mixed interface, or the 2D display interface and the 3D display interface are combined in the display interface to display the formed mixed interface, wherein the 2D display interface comprises a configuration interface, and the 3D display interface comprises a task execution interface.

In some embodiments, when the information display interface is switched, the on-off states of at least two channels are also controlled based on the object switching signal, wherein the at least two channels are respectively used for transmitting the 2D display interface and the 3D display interface.

Specifically, a first channel corresponding to the 2D display interface and a second channel corresponding to the 3D display interface are respectively determined, and further, the corresponding information display interface is determined according to the operation switching signal, and the on-off state of each channel is further controlled. Illustratively, when the information display interface includes a 2D display interface, the first channel is controlled to be in an open state and the second channel is controlled to be in a closed state, so that an operation user performs configuration operation on a configuration interface corresponding to the 2D display interface, and certainly, when the information display interface is a 3D display interface, the second channel is controlled to be in an open state and the first channel is controlled to be in a closed state, so that the operation user performs task operation on a task screen display interface corresponding to the 2D display interface.

In some embodiments, when the channel corresponding to the information display interface is in the closed-on state, the operation object corresponding to the information display interface is also in the inoperable state, so as to avoid the safety problem caused by the operation object of the task screen display interface being operated by mistake.

The technical scheme of the embodiment of the invention specifically comprises the following steps: receiving operation signals of each operation device, determining the operation signals sent by any operation device as target operation signals, acquiring the operation signals sent by at least two operation devices, determining the operation signals as the target signals, and further executing the target operation signals; the target operation signal comprises an operation object control signal and an object switching signal, the object switching signal is used for controlling and switching the information display interface and triggering an operation object corresponding to the information display interface, and the operation object control signal is used for controlling the state of the operation object in the information display interface. According to the technical scheme of the embodiment of the invention, the interface switching of the information display interface and the control of the operation object corresponding to the switched interface are realized by receiving the operation signals and determining the signal type of the operation signals, namely, the target operation signal is executed, so that the interface switching and the operation of the operation object under different interfaces are realized based on the operation equipment, the operation convenience of the operation equipment is improved, and the task operation efficiency is improved.

The following is an embodiment of a control device provided in an embodiment of the present invention, which belongs to the same inventive concept as the control method in each of the above embodiments, and reference may be made to the above embodiment of the control method for details that are not described in detail in the embodiment of the control device.

Example four

Fig. 5 is a schematic structural diagram of a control device according to a fourth embodiment of the present invention, which is applied to an operating device. The present embodiment is applicable to a case where an operation device is operated, and more specifically, to a case where an operation motion to an operation member in the operation device is detected and an operation signal corresponding to the operation member is generated. The specific structure of the control device comprises: a trigger signal monitoring module 410, an operation signal generating module 420 and an operation object sending module 430; wherein,

a trigger signal monitoring module 410, configured to monitor a trigger signal sent to an operation component on an operation device; wherein the trigger signal is generated when the operation part is operated, and comprises a trigger type and a trigger parameter;

an operation signal generating module 420, configured to generate a corresponding operation signal based on a trigger type and a trigger parameter in the trigger signal, where the operation signal includes an operation object control signal and an object switching signal, the object switching signal is used to control a display device to switch an information display interface and trigger an operation object corresponding to the information display interface, and the operation object control signal is used to control a state of the corresponding operation object;

an operation object sending module 430, configured to send the operation signal to a corresponding device, so that the device executes the operation signal, where the device includes a display device and a task execution device.

The technical scheme of the embodiment of the invention specifically comprises the following steps: monitoring a trigger signal sent to an operation component on operation equipment so as to generate a corresponding operation signal based on the trigger signal subsequently, wherein the trigger signal is generated when the operation component is operated, and the specific trigger signal comprises a trigger type and a trigger parameter; generating a corresponding operation signal based on a trigger type and a trigger parameter in the trigger signal, wherein the operation signal includes an operation object control signal and an object switching signal, the object switching signal is used for controlling a display device to switch an information display interface and triggering an operation object corresponding to the information display interface, and the operation object control signal is used for controlling a state of the corresponding operation object; and further sending the operation signal to corresponding equipment to enable the equipment to execute the operation signal, wherein the equipment comprises display equipment and task execution equipment. According to the technical scheme of the embodiment of the invention, the interface switching of the information display interface and the control of the operation object corresponding to the switched interface are realized by monitoring the operation action of the operation component and generating the corresponding operation signal, so that the switching of the interface and the operation of the operation objects under different interfaces are realized based on a single operation device, the operation convenience of the operation device is improved, and the efficiency of task operation is improved.

On the basis of the technical scheme, the operating device comprises two operating areas, a plurality of operating components are arranged in each operating area,

the operation signal generating module 420 includes:

and the operation signal generation submodule is used for generating a corresponding operation signal based on the first trigger signal sent by the operation component in the first operation area and/or the second trigger signal sent by the operation component in the second operation area.

On the basis of the above technical solution, the operation signal generation submodule includes:

and the operation signal generation unit is used for determining a corresponding operation signal based on the first trigger signal, the second trigger signal and a preset operation signal library if the second trigger signal is received within a preset time interval after the first trigger signal is received or the first trigger signal is received within a preset time interval after the second trigger signal is received.

On the basis of the above technical solution, the operation signal generating unit includes:

a first operation signal generation subunit, configured to determine, if a combined operation signal of the first trigger signal and the second trigger signal is matched in the preset operation signal library, the combined operation signal as a corresponding operation signal;

a second operation signal generation subunit, configured to determine, if a combined operation signal of the first trigger signal and the second trigger signal is not matched in the preset operation signal library, the first trigger signal and the second trigger signal as operation signals corresponding to the first operation area and the second operation area, respectively.

On the basis of the technical scheme, the operation equipment comprises at least two pieces of sub-equipment, a plurality of operation parts are respectively arranged on each piece of sub-equipment, and the sub-equipment generates operation signals based on trigger signals sent by the operation parts and respectively and independently sends the operation signals to the corresponding equipment. On the basis of the technical scheme, the method comprises the following steps:

on the basis of the technical scheme, the operating device comprises a master device and at least one slave device, wherein a plurality of operating components are respectively arranged on each master device and each slave device, the slave devices transmit trigger signals to the master devices, and the master devices are used for generating operating signals based on the trigger signals transmitted by the slave devices and the trigger signals transmitted by the operating components of the master devices.

On the basis of the technical scheme, the information display interface comprises a 2D display interface and a 3D display interface, wherein the operation object corresponding to the 2D display interface comprises each content object in the display interface, and the operation object corresponding to the 3D display interface is task execution equipment.

On the basis of the technical scheme, the 2D display interface comprises a configuration interface.

The control device provided by the embodiment of the invention can execute the control method applied to the operating equipment provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

EXAMPLE five

Fig. 6 is a schematic structural diagram of a control device according to a fifth embodiment of the present invention, which is applied to a display device. The present embodiment is applicable to a case where an operation device is operated, and more specifically, to a case where a received operation signal is displayed. The specific structure of the control device comprises: an operation signal receiving module 510, a target operation signal generating module 520, and a target operation signal executing module 530; wherein,

an operation signal receiving module 510, configured to receive an operation signal sent by each operation device;

a target operation signal generation module 520, configured to determine the operation signal sent by the device as a target operation signal, or determine the target operation signal based on the operation signals sent by at least two operation devices;

a target operation signal execution module 530, configured to execute the target operation signal; the target operation signal comprises an operation object control signal and an object switching signal, the object switching signal is used for controlling and switching an information display interface and triggering an operation object corresponding to the information display interface, and the operation object control signal is used for controlling the state of the operation object in the information display interface.

The technical scheme of the embodiment of the invention specifically comprises the following steps: receiving operation signals of each operation device, determining the operation signals sent by any operation device as target operation signals, acquiring the operation signals sent by at least two operation devices, determining the operation signals as the target signals, and further executing the target operation signals; the target operation signal comprises an operation object control signal and an object switching signal, the object switching signal is used for controlling and switching an information display interface and triggering an operation object corresponding to the information display interface, and the operation object control signal is used for controlling the state of the operation object in the information display interface. According to the technical scheme of the embodiment of the invention, the interface switching of the information display interface and the control of the operation object corresponding to the switched interface are realized by receiving the operation signals and determining the signal type of the operation signals, namely, the target operation signal is executed, so that the interface switching and the operation of the operation object under different interfaces are realized based on the operation equipment, the operation convenience of the operation equipment is improved, and the task operation efficiency is improved.

On the basis of the technical scheme, the number of the operating devices is at least two;

the target operation signal generation module 620 includes:

the target operation signal generation unit is configured to determine a target operation signal based on the third operation signal, the fourth operation signal, and a preset operation signal library if the fourth operation signal sent by the second operation device is received within a preset time interval after the third operation signal sent by the first operation device is received, or the third operation signal is received within a preset time interval after the fourth operation signal is received.

On the basis of the above technical solution, the target operation signal generating unit includes:

a first target operation signal generation subunit, configured to determine, if a combined operation signal of the third operation signal and the fourth operation signal is matched in the preset operation signal library, the combined operation signal as a target operation signal;

a second target operation signal generation subunit, configured to determine, if a combined operation signal of the third operation signal and the fourth operation signal is not matched in the preset operation signal library, that the third operation signal and the fourth operation signal are respectively determined as a target operation signal corresponding to the first operation area or the first device, and that the second operation area or the second device is corresponding to the first operation area or the first device.

On the basis of the technical scheme, if the target operation signal comprises an object switching signal;

accordingly, the target operation signal executing module 630 includes:

a first target operation signal execution unit for switching the information display interface between a 2D display interface and a 3D display interface based on the object switching signal; or,

and the second target operation signal execution unit is used for additionally displaying the 2D display interface on the basis of the 3D display interface based on the object switching signal, or switching the mixed interface of the 2D display interface and the 3D display interface to the 3D display interface.

On the basis of the above technical solution, the target operation signal executing module 630 includes:

and the channel switching state control unit is used for controlling the switching states of at least two channels based on the object switching signal, wherein the at least two channels are respectively used for transmitting the 2D display interface and the 3D display interface.

On the basis of the technical scheme, the mixed interface of the 2D display interface and the 3D display interface comprises the following steps: and the 2D display interface and the 3D display interface are superposed to display a formed mixed interface, or the 2D display interface and the 3D display interface are combined to display the formed mixed interface in the display interface.

The control device provided by the embodiment of the invention can execute the control method applied to the display equipment provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

It should be noted that, in the embodiment of the control device, the included units and modules are merely divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.

EXAMPLE six

Fig. 7 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention. FIG. 7 illustrates a block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention. The electronic device 12 shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiment of the present invention.

As shown in fig. 7, the electronic device 12 is in the form of a general purpose computing electronic device. The components of electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.

Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

Electronic device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by electronic device 12 and includes both volatile and nonvolatile media, removable and non-removable media.

The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and/or cache memory 32. The electronic device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 7 and commonly referred to as a "hard drive"). Although not shown in FIG. 7, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.

Electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with electronic device 12, and/or with any devices (e.g., network card, modem, etc.) that enable electronic device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, the electronic device 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) via the network adapter 20. As shown in FIG. 7, the network adapter 20 communicates with the other modules of the electronic device 12 via the bus 18. It should be appreciated that although not shown in FIG. 7, other hardware and/or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

The processing unit 16 executes various functional applications and sample data acquisition by running the program stored in the system memory 28, for example, to implement the steps of a control method provided in this embodiment, specifically:

the control method applied to the operating device includes:

The method applied to the display device comprises the following steps:

receiving operation signals sent by each operation device;

Of course, those skilled in the art can understand that the processor may also implement the technical solution of the sample data obtaining method provided in any embodiment of the present invention.

EXAMPLE seven

The seventh embodiment provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements, for example, the steps of the control method provided in this embodiment, and specifically:

the control method applied to the operating device includes:

The method applied to the display device comprises the following steps:

receiving operation signals sent by each operation device;

Computer storage media for embodiments of the present invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer-readable storage medium may be, for example but not limited to: an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

It will be understood by those skilled in the art that the modules or steps of the invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of computing devices, and optionally they may be implemented by program code executable by a computing device, such that it may be stored in a memory device and executed by a computing device, or it may be separately fabricated into various integrated circuit modules, or it may be fabricated by fabricating a plurality of modules or steps thereof into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A control method applied to an operating device, comprising:

2. The method of claim 1, wherein the operating device comprises two operating regions, each operating region having a plurality of operating components disposed therein,

the generating of the corresponding operation signal based on the trigger type and the trigger parameter in the trigger signal includes:

3. The method according to claim 2, wherein generating the corresponding operation signal based on the first trigger signal sent by the operation component in the first operation region and/or the second trigger signal sent by the operation component in the second operation region comprises:

and if the second trigger signal is received within a preset time interval after the first trigger signal is received, or the first trigger signal is received within a preset time interval after the second trigger signal is received, determining the corresponding operation signal based on the first trigger signal, the second trigger signal and a preset operation signal library.

4. The method of claim 3, wherein determining the corresponding operation signal based on the first trigger signal, the second trigger signal, and a library of preset operation signals comprises:

5. The method according to claim 1, wherein the operating device comprises at least two sub-devices, each sub-device is provided with a plurality of operating components, and the sub-devices generate operating signals based on trigger signals sent by the operating components and respectively and independently send the operating signals to corresponding devices.

6. The method according to claim 1, wherein the operation device comprises a master device and at least one slave device, each of the master device and the slave device is provided with a plurality of operation components, the slave device transmits a trigger signal to the master device, and the master device is configured to generate an operation signal based on the trigger signal transmitted by each slave device and the trigger signal transmitted by its own operation component.

7. The method according to claim 1, wherein the information display interface comprises a 2D display interface and a 3D display interface, wherein the operation object corresponding to the 2D display interface comprises each content object in the display interface, and the operation object corresponding to the 3D display interface is task execution equipment.

8. The method of claim 7, wherein the 2D display interface comprises a configuration interface.

9. A control method of a display device is applied to the display device and is characterized by comprising the following steps:

receiving operation signals sent by each operation device;

10. The method of claim 9, wherein the operating devices are at least two;

the determining of the target operation signal based on the operation signals sent by the at least two operation devices comprises:

11. The method of claim 10, wherein determining a target operation signal based on the third operation signal, the fourth operation signal, and a library of preset operation signals comprises:

12. The method of claim 9, wherein if the target operation signal comprises an object switch signal;

correspondingly, the executing the target operation signal includes:

13. The method of claim 12, wherein said executing the target operation signal further comprises:

14. The method of claim 13, wherein the hybrid interface of the 2D display interface and the 3D display interface comprises: and the 2D display interface and the 3D display interface are superposed to display a formed mixed interface, or the 2D display interface and the 3D display interface are combined to display the formed mixed interface in the display interface.

15. An operating device control apparatus characterized by comprising:

16. A display device control apparatus, characterized by comprising:

17. An electronic device, comprising:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the control method of any one of claims 1-14.

18. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the control method according to any one of claims 1 to 14.