CN109783152B

CN109783152B - Physical hardware control method, device and computer readable storage medium

Info

Publication number: CN109783152B
Application number: CN201711115730.6A
Authority: CN
Inventors: 邵晓康; 胡锦宏
Original assignee: Shenzhen Maker Works Technology Co ltd
Current assignee: Shenzhen Maker Works Technology Co ltd
Priority date: 2017-11-13
Filing date: 2017-11-13
Publication date: 2021-08-13
Anticipated expiration: 2037-11-13
Also published as: CN109783152A

Abstract

The invention discloses a control method and a control device of physical hardware and a computer readable storage medium. The method comprises the following steps: carrying out extended loading of hardware roles through configuration files of physical hardware; performing hardware role rendering updating through a configuration file which is expanded and loaded for physical hardware, and obtaining action blocks which can be configured by hardware roles and hardware roles; according to the hardware role and the action block configuration operation of the executed action, obtaining a role interaction scene which is integrated with the hardware role and the action block of which the hardware role is mapped; and monitoring the triggering operation of the action block mapped by the hardware role, and performing action control on the corresponding physical hardware according to the triggered operation action block. The introduction of physical hardware into the role interaction scene is realized, and then the physical world and the virtual world are combined without being limited by the interaction between the virtual roles, and the physical hardware is not limited by fixedly configured programs, so that the role interaction scene performance is enhanced, and the control of the physical hardware and the interaction of the hardware roles are freely performed.

Description

Physical hardware control method, device and computer readable storage medium

Technical Field

The present invention relates to the field of computer application technologies, and in particular, to a method and an apparatus for controlling physical hardware, and a computer-readable storage medium.

Background

With the development of computer application technology, users can freely control virtual characters through the realization of character interaction scenes, so that the interaction between the virtual characters is realized. For example, in a game of game play performed by a game application, a displayed game screen corresponds to a character interaction scene, and a game character available for manipulation in the game screen is a character indicated in the character interaction scene.

However, existing character interactions are limited to virtual characters. The control of physical hardware, for example, various sensors, is developed and released by developers around functions and functional components that the sensors need to implement, and the free control of the physical hardware cannot be realized, so that a role interaction scene is introduced.

With the development of various applications such as internet of things and artificial intelligence, physical hardware is in diversified development and is increasingly used as an information transmission interface and introduced into the function realization of an application program. However, even so, for character interaction scenarios, it is still limited to virtual character interaction.

In summary, it is urgently needed to implement the introduction of the role corresponding to the physical hardware in the role interaction scene, so as to implement the free control of the physical hardware, and the method is not limited to the fixedly configured program.

Disclosure of Invention

In order to solve the technical problems that a character interaction scene in the related art is limited to interaction between virtual characters, and physical hardware is limited to a fixedly configured program, the invention provides a control method and a control device of the physical hardware and a computer-readable storage medium.

A method of physical hardware control, the method comprising:

carrying out extended loading of hardware roles corresponding to physical hardware through a configuration file defined by the physical hardware;

performing rendering updating of a hardware role through the configuration file which is expanded and loaded for the physical hardware, and obtaining a hardware role and action blocks which can be configured by the hardware role;

according to the hardware role carried out in the role interaction scene and the action block configuration operation of the executed action, obtaining the role interaction scene which is integrated with the hardware role and the action block of which the hardware role is mapped;

monitoring the triggering operation of the action block mapped by the hardware role, and performing action control on corresponding physical hardware according to the action block which is triggered to operate.

An apparatus for controlling physical hardware, the apparatus comprising:

the extended loading module is used for carrying out extended loading of hardware roles corresponding to the physical hardware through a configuration file defined by the physical hardware;

the rendering updating module is used for performing rendering updating on a hardware role through the configuration file which is expanded and loaded for the physical hardware to obtain a hardware role and action blocks which can be configured by the hardware role;

the role configuration module is used for obtaining a role interaction scene which is integrated with the hardware role and is mapped with the action blocks according to the hardware role carried out in the role interaction scene and the action block configuration operation of the executed action;

and the action execution module is used for monitoring the triggering operation of the action block mapped by the hardware role and controlling the action of the corresponding physical hardware according to the action block which is triggered to operate.

A control apparatus for physical hardware, comprising:

a processor; and

a memory having computer readable instructions stored thereon which, when executed by the processor, implement a method of controlling physical hardware as described above.

A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the method of controlling physical hardware as described above.

The technical scheme provided by the embodiment of the invention can have the following beneficial effects:

for the owned physical hardware, firstly, the extended loading of the corresponding hardware role of the physical hardware is carried out through the configuration file defined by the physical hardware, then, the rendering updating of the hardware role is carried out through the configuration file extended and loaded for the physical hardware, action blocks which can be configured by the hardware role and the hardware role are obtained, finally, the role interaction scene which is fused with the hardware role and is mapped by the hardware role is obtained according to the configuration operation of the hardware role and the action blocks of the executed action in the role interaction scene, so that the triggering operation of the action blocks mapped by the hardware role is monitored, the action control is carried out on the corresponding physical hardware according to the action blocks which are triggered to operate, the introduction of the physical hardware to the role interaction scene is realized, the physical world and the virtual world are combined, the interaction is realized, and the role interaction scene is not limited by the interaction between the virtual roles, the physical hardware is not limited by a fixedly configured program any more, the application of the physical hardware is extended, the performance of a role interaction scene is enhanced, and the control of the physical hardware and the interaction of hardware roles in the role interaction scene are freely performed.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 is a schematic diagram illustrating an implementation environment in accordance with the present invention, according to an exemplary embodiment;

FIG. 2 is a block diagram illustrating an apparatus in accordance with an exemplary embodiment;

FIG. 3 is a flow chart illustrating a method of controlling physical hardware in accordance with an exemplary embodiment;

FIG. 4 is a flowchart illustrating details of step 310 according to the corresponding embodiment of FIG. 3;

FIG. 5 is a flowchart illustrating details of step 330 according to a corresponding embodiment of FIG. 1;

FIG. 6 is a flowchart illustrating details of step 333 according to the corresponding embodiment of FIG. 5;

FIG. 7 is a flowchart illustrating details of step 350 according to the corresponding embodiment of FIG. 3;

FIG. 8 is a flowchart illustrating details of step 370 according to the corresponding embodiment of FIG. 3;

FIG. 9 is a flow chart illustrating a method of controlling physical hardware in accordance with another exemplary embodiment;

FIG. 10 is a flow chart illustrating a method of controlling physical hardware in accordance with another exemplary embodiment;

FIG. 11 is a schematic diagram illustrating a loading implementation of a character in a character interaction scenario in accordance with an illustrative embodiment;

FIG. 12 is a configuration file diagram of physical hardware A, shown in accordance with an exemplary embodiment;

FIG. 13 is a flowchart illustrating an implementation of physical hardware A performing an extended load to inject a corresponding hardware role in accordance with an illustrative embodiment;

FIG. 14 is a diagram illustrating a role loading interface in accordance with an illustrative embodiment;

FIG. 15 is a diagram illustrating an implementation of code hinting and error grammar highlighting in a code mode, according to an illustrative embodiment;

FIG. 16 is an implementation diagram illustrating conversion between code and action blocks in accordance with an illustrative embodiment;

FIG. 17 is a schematic diagram illustrating interface partitioning in accordance with an illustrative embodiment;

FIG. 18 is a block diagram illustrating a control device of physical hardware in accordance with an exemplary embodiment;

FIG. 19 is a block diagram illustrating details of an extended load module according to the corresponding embodiment of FIG. 18.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.

FIG. 1 is a schematic diagram illustrating an implementation environment in accordance with the present invention, according to an exemplary embodiment. The implementation environment of the present invention includes a control end 110 and a physical hardware 130 that can access the constructed character interaction scenario via the implementation in the control end 110, thereby implementing free control.

The control terminal 110 may be at least one of a computer terminal, a smart phone, a tablet computer, and other portable mobile terminals. The physical hardware 130 is a hardware device existing in the physical world separately from the control terminal 110, and may be, for example, a sensor itself, a vehicle body equipped with various sensors and enclosed by a shaped housing, or any hardware device equipped with a communication connection interface (e.g., a WiFi communication connection interface, etc.).

The physical hardware 130 is a physical entity actually existing in the physical world, and under the action of the control end 110, a role interaction scene is introduced by executing the method shown in the present invention, so as to realize free control of the physical hardware 130.

In this implementation environment, the control end 110 interacts with the physical hardware 130, wherein the control end 110 may be unique, and the corresponding physical hardware 130 may be one or more, and in case of multiple physical hardware, may be a single kind or multiple kinds.

FIG. 2 is a block diagram illustrating an apparatus according to an example embodiment. For example, the apparatus 200 may be a smartphone in the implementation environment described above.

Referring to fig. 2, the apparatus 200 may include one or more of the following components: a processing component 202, a memory 204, a power component 206, a multimedia component 208, an audio component 210, a sensor component 214, and a communication component 216.

The processing component 202 generally controls overall operation of the device 200, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations, among others. The processing components 202 may include one or more processors 218 to execute instructions to perform all or a portion of the steps of the methods described below. Further, the processing component 202 can include one or more modules that facilitate interaction between the processing component 202 and other components. For example, the processing component 202 can include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

The memory 204 is configured to store various types of data to support operations at the apparatus 200. Examples of such data include instructions for any application or method operating on the apparatus 200. The Memory 204 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk or optical disk. Also stored in memory 204 are one or more modules configured to be executed by the one or more processors 218 to perform all or a portion of the steps of any of the methods of fig. 2, 3, 4, and 5, described below.

The power supply component 206 provides power to the various components of the device 200. The power components 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the apparatus 200.

The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a touch panel. If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. The screen may further include an Organic Light Emitting Display (OLED for short).

The audio component 210 is configured to output and/or input audio signals. For example, the audio component 210 includes a Microphone (MIC) configured to receive external audio signals when the device 200 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 204 or transmitted via the communication component 216. In some embodiments, audio component 210 also includes a speaker for outputting audio signals.

The sensor component 214 includes one or more sensors for providing various aspects of status assessment for the device 200. For example, the sensor assembly 214 may detect an open/closed state of the device 200, the relative positioning of the components, the sensor assembly 214 may also detect a change in position of the device 200 or a component of the device 200, and a change in temperature of the device 200. In some embodiments, the sensor assembly 214 may also include a magnetic sensor, a pressure sensor, a temperature sensor, or a visual sensor. It should be noted that the above listed sensors are only preferred examples, and the configurable sensor of the present invention is not limited thereto.

The communication component 216 is configured to facilitate wired or wireless communication between the apparatus 200 and other devices. In one example, the apparatus 200 may access a WIreless network based on a communication standard, such as WiFi (Wireless-Fidelity). In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the Communication component 216 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, bluetooth technology, and other technologies.

In an exemplary embodiment, the apparatus 200 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital signal processors, digital signal processing devices, programmable logic devices, field programmable gate arrays, controllers, microcontrollers, microprocessors or other electronic components for performing the methods described below.

FIG. 3 is a flow chart illustrating a method of controlling physical hardware in accordance with an exemplary embodiment. The control method of the physical hardware is suitable for the control terminal 110 indicated by the implementation environment shown in fig. 1, and the control terminal 110 is the device shown in fig. 2 in an exemplary embodiment. As shown in fig. 3, the method for controlling physical hardware at least includes the following steps.

In step 310, an extended load of the hardware role corresponding to the physical hardware is performed through the configuration file defined by the physical hardware.

The physical hardware is controlled by the instruction sent by the accessed application program, and executes the action corresponding to the instruction through the external components of the physical hardware, such as a sensor, a display screen, a motor and the combination of various parts. The external information, such as temperature, distance, light and the like, is sensed by the sensor and transmitted to the main control module of the sensor, and the main control module feeds back the external information to the accessed application program, so that the external information feedback executed by the physical hardware is realized.

For another example, the physical hardware has a motion function by a combination of the motor and various components, and the combination of the various components on the physical hardware executes various mechanical motions by driving the motor under the control of the command.

It should be added here that the application program is the control program implemented by the method of the present invention.

Different physical hardware is composed of unique external components and further has different functions, and for a user, the physical hardware owned by the user is accessed into an application program according to the need of the user. For example, when it is desired to run the function of a certain physical hardware, the method of the present invention may be triggered to be executed to access the application program, so as to freely control the physical hardware.

It is understood that different physical hardware has corresponding functions and actions required to implement the functions. That is, different physical hardware has its attribute information, which describes the execution of the action of the corresponding physical hardware and indicates the realized function or the like. The configuration file defined by the physical hardware corresponds to the attribute information, and defines how to drive the physical hardware to execute the functions on one hand, and on the other hand, defines a role of the physical hardware as the accessed role, namely the existence of the hardware role, so that the access of the role interaction scene can be realized subsequently, and the physical hardware is freely controlled under the role interaction scene.

Therefore, the extended loading of the role in the role interaction scene is carried out through the configuration file defined by the physical hardware, so that the role interaction scene can be extended out of the hardware role and is not limited to the interaction of the virtual role.

The hardware role is the only existing form of the physical hardware in the role interaction scene, so that the role interaction scene evolves from the original interaction process which is purely virtualized on the data level through the virtual role, namely the realization of the virtual scene, to the tight combination of the virtual world and the physical world, and the virtual world realizes the interaction with the physical hardware and the physical world where the physical hardware is located through a channel constructed by the application program and the physical hardware.

By loading the configuration file defined by the physical hardware, role extension loading is carried out on the physical hardware, namely, the existing role is extended out of the role of the hardware.

The extension is relative to the original existing virtual roles, so that the existing roles include the virtual roles and hardware roles corresponding to the physical hardware owned by the user.

With the extended loading of the corresponding hardware role performed by the physical hardware, the configuration file defined by the physical hardware is loaded into the memory, so that the relevant processing of the configuration file can be executed, the physical hardware is accessed, and the corresponding hardware role and the control execution are introduced.

In step 330, the rendering update of the hardware role is performed by expanding the loaded configuration file for the physical hardware, and the action blocks of the hardware role and the hardware role available for configuration are obtained.

It should be understood that, the extended loading of the configuration file indicates that, on one hand, role extension is initiated and a hardware role and a related action block are introduced in an application program of the control terminal 110 that originally only implements various virtual roles, and on the other hand, the configuration file is loaded to a memory, and the configuration file carries various data related to the physical hardware and the corresponding hardware role, so as to perform various operations on the configuration file.

Furthermore, the configuration file carries resource files of the corresponding hardware role, such as hardware role name information, role icons, and the like, and in addition, the configuration file also carries action block information that can be configured to the corresponding hardware role. The action block information indicates the action block currently configured for this hardware role, and the specific definition of the action block, and the like. The action block is used for realizing the control of the hardware role to the corresponding physical hardware.

Therefore, the loaded configuration file is expanded, so that the rendering updating of the corresponding hardware role can be performed, and the existence and addition of the action blocks defined by the hardware role and the hardware role or the existence and addition of the newly added action blocks of the expanded hardware role can be realized. The obtained hardware role and the action blocks of which the hardware role can be configured can be freely activated under the control of a user, and the control of the hardware role to the physical hardware is further carried out through the operation of the activated action blocks.

That is to say, the obtained hardware roles and action blocks for which the hardware roles are configurable can be freely configured in the role interaction scene according to whether to control the corresponding physical hardware and the actions executed by the required physical hardware, and the hardware roles corresponding to the physical hardware are introduced through the execution of step 310 and step 330, so as to obtain the hardware roles and action blocks which can be freely selected.

It should be further noted that each hardware role, implementing its control of the physical hardware, is implemented by a configured action block. Each introduced hardware role has various action blocks for configuration, and physical hardware control required to be realized by a user can be selected from the action blocks or can be used completely, which is not limited herein.

The hardware role obtained at this time exists as an optional role, and the action blocks for which the hardware role is configurable exist as optional interactive actions.

By executing step 310 and step 330, a hardware role and an action block that can be configured by the hardware role are provided for a role interaction scene to be realized subsequently, and physical hardware owned by a user can obtain a corresponding hardware role by performing extended loading and rendering updating, so that the extended performance of the role and the flexibility of role deployment are improved.

In step 350, a role interaction scene that is merged into a hardware role and to which action blocks are mapped is obtained according to configuration operations for the hardware role performed in the role interaction scene and the action blocks of the performed action.

As mentioned above, the role interaction scenario is constructed by the application program implemented by the present invention, and the interaction between roles, including the interaction between hardware roles, the interaction between virtual roles, and the interaction between hardware roles and virtual roles, is implemented, and has very high interaction performance.

And configuring operations for the role interaction scene, wherein the configuring operations comprise role configuring operations and action block configuring operations. In a specific implementation of an exemplary embodiment, the configuration operation may include only an action block configuration operation to be attached to a hardware role configured to the role interaction scenario, for which an action block is added to the hardware role; and the method also can comprise role configuration operation and action block configuration operation executed on the basis of the role configuration operation so as to configure the corresponding hardware role to the role interaction scene and then carry out the configuration of the action block.

In an exemplary embodiment, the configuration operation may be a drag operation triggered by the corresponding character icon or action block icon.

The hardware role and the action blocks which can be configured by the hardware role are obtained through the steps, or the action blocks which are added for the expanded hardware role are rendered and updated to a page, such as a toolbar, so that a user can select and configure the action blocks to a role interaction scene through a triggered dragging operation.

In step 370, the triggering operation of the action block mapped by the hardware role is monitored, and action control is performed on the corresponding physical hardware according to the triggered operation action block.

Wherein, it should be understood that, for a hardware role, through the configuration of action blocks in the role interaction scene, a plurality of action blocks configured by the hardware role are obtained, and the action blocks are spliced with each other.

An action block, which has the code description information defined by it, and then obtains the action control information of the action block executing the defined action, so as to achieve the purpose of controlling the corresponding physical hardware to execute the specified action. Further, the splicing relationship between the code information and the code collocation information will be described. The corresponding components of the action blocks in the physical hardware, such as sensors, etc., are assembled in the physical hardware for realizing the specific function, and the splicing relationship among the action blocks corresponding to the components forms the code logic for realizing the specific function.

The action blocks mapped by the hardware roles in the role interaction scene all have corresponding code information, and the code information can be each specific code, can also be parameters of codes corresponding to the spliced action blocks, and can also be the representation and expression of variables, logic expressions, loops and the like in the codes corresponding to the spliced action blocks.

Therefore, the integration of the hardware role in the role interaction and the action control of the hardware role to the physical hardware are realized, and the action controlled and executed by the physical hardware is matched with the configured action block, so that the action block configuration operation can be initiated according to the requirement, the free control of the physical hardware is further realized, and the method has very high flexibility and controllability and high accuracy of the physical hardware control.

Fig. 4 is a flowchart illustrating details of step 310 according to the corresponding embodiment of fig. 3. This step 310, as shown in FIG. 4, includes at least the following steps.

In step 311, a configuration file defined for the corresponding physical hardware is obtained according to the triggered addition of the hardware role or the addition of the configurable action blocks available for the hardware role, where the configuration file is used to describe the added hardware role or the added action blocks of the hardware role.

In step 313, a configuration file is loaded to extend the hardware role or add an action block to the extended hardware role.

The new hardware role adding is to point to a role which can be configured to a role interaction scene to carry out the new hardware role adding. For users, they can make new additions to the hardware role of the owned physical hardware. For example, when a user newly obtains a physical hardware, the corresponding hardware role can be obtained by extending and loading the physical hardware, so that the hardware role is newly added in an application program for realizing a role interaction scene.

In addition to adding the hardware role, the existing action blocks, that is, the expanded hardware role, can be added to the action blocks capable of being configured to the hardware role.

The realization of the expansion loading, namely the addition of the hardware role or the action block, is carried out through the configuration file. Here, it is necessary to describe the configuration file in detail.

A configuration file, uniquely corresponding to a hardware role, but capable of corresponding to an action block or a plurality of action blocks. The obtained configuration file initiates the establishment of the hardware role when the indicated hardware role does not exist; and when the indicated hardware role already exists, the method is used for initiating the creation of the action block which can be configured by the hardware role.

The data carried by a configuration file can be roughly divided into two types, one type is related to the implemented role, and the other type is related to the action defined for the implemented role.

Specifically, the configuration file includes resource file information, action block configuration information, and code description information corresponding to the action block. The resource file information is related to the realized hardware role, and the resource file information indicates the name of the realized hardware role, the display style of the realized hardware role in the application program and the role interaction scene realized by the application program, and even the display style, the audio content and the like which can be output by the realized hardware role in the role interaction scene. Under the action of resource file information in the configuration file, the existence of the hardware role in the role interaction process is ensured, and then the hardware role can be adapted to the virtual role, and the performance of the hardware role being merged into the role interaction scene is ensured.

Further, in the configuration file, the resource file information includes a role name, a role icon, and the like; the action block configuration information indicates the action block configured for the hardware role indicated by the role name, and indicates the classification of the action block and the specific definition, i.e. the code information, under the action of the code description information corresponding to the action block.

With the acquisition of the defined configuration file of the physical hardware, the configuration file is loaded into a memory to perform corresponding hardware roles, or add action blocks for expanded hardware roles, so that hardware roles introduced by role interaction scenes are continuously enriched, accessible physical hardware is also continuously enriched, the interaction between the virtual world and the physical world is enriched while the virtual world and the physical world are communicated, and the extensible performance is very high.

Fig. 5 is a flowchart illustrating details of step 330 according to a corresponding embodiment of fig. 1. This step 330, as shown in fig. 5, includes at least the following steps.

In step 331, the loaded configuration file is parsed to generate a hardware role or a role extension object with action blocks added to the extended hardware role.

The data carried by the configuration file, such as the name of the role, the storage path of the data, the task, the configured action block and the code description information corresponding to the action block, the physical hardware corresponding to the implemented role, etc., can be obtained by the role of the loaded configuration file. It should be understood that the tasks referred to, i.e. the corresponding hardware roles, are currently being extended and loaded, and various tasks need to be executed to ensure availability and controllability of the loaded hardware roles. For example, the tasks that need to trigger processing include transmitting role names, transmitting action block description information, and listening whether an action block is triggered to run. Through the deployment of tasks, the newly added or expanded hardware role newly added action blocks are effective, and the performability of the processing flow corresponding to the subsequent hardware roles and action blocks is ensured.

The data carried by the configuration file is obtained through analysis of the configuration file, a role extension object is generated for the data, the analyzed data is packaged on one hand, and on the other hand, extension loading related to a hardware role and integration in a role interaction scene can be smoothly completed based on the role extension object.

In step 333, the rendering of the hardware role or the expanded hardware role newly added action block is performed through the role expansion object.

As mentioned above, the data encapsulated by the role extension object defines a hardware role, or defines a display style of an action block added to the extended hardware role, such as a role icon, and therefore, a corresponding interface is rendered through the role extension object.

In step 335, the hardware role obtained by rendering update or the expanded hardware role newly added action block is updated to the toolbar corresponding to the role interaction scene.

And rendering and updating the hardware role or the expanded hardware role newly-added action block so as to display the hardware role or the expanded hardware role newly-added action block on an interface, so that a user can select and configure the hardware role or the expanded hardware role newly-added action block into a role interaction scene at any time.

The toolbar is an area on the interface carrying various roles and each role being capable of configuring an action block. The new action block of the rendered updated hardware role or the expanded hardware role will exist in the form of an icon, such as the aforementioned role icon, and will be added to the toolbar.

Therefore, the expansion loading of the corresponding hardware role is realized for the physical hardware, the hardware role of the physical hardware which can be fused into the role interaction scene is obtained, and in addition, the action blocks of the expanded hardware role can be increased.

That is to say, the hardware role capable of acting on the role interaction scene can be added to the physical hardware at will, and the action block can also be added to the existing hardware role at will, so that the interaction between the hardware role and other roles and the control to the physical hardware are richer and more accurate, the method can adapt to the user requirements, and has very high flexibility and expandability.

Fig. 6 is a flowchart illustrating details of step 333 according to the corresponding embodiment of fig. 5. This step 333, as shown in fig. 6, includes at least the following steps.

In step 3331, it is determined whether the hardware role to which the role extension object belongs conflicts with the extended hardware role in the toolbar, and if not, step 3333 is executed, and if so, step 3335 is executed.

In step 3333, the creation of the hardware role is triggered, and the rendering of the extended hardware role is performed through the role extension object.

In step 3335, a new action block is obtained from the role extension object and added to the corresponding extended hardware role.

As described above, the toolbar displays all characters that can be configured into the character interaction scenario by the application program, including the hardware characters and the virtual characters, and also displays configurable action blocks. Therefore, when the role extension object generated by loading the configuration file is subjected to extension loading, whether the hardware role indicated by the role extension object exists in the toolbar or not should be judged for the generated role extension object so as to confirm whether the hardware role indicated by the role extension object is created or not.

If the hardware role indicated by the role extension object conflicts with the role in the toolbar, the role extension object indicates that the hardware role indicated by the role extension object has been created, and only new action blocks are configured for the created hardware role.

If the hardware role indicated by the role extension object does not conflict with the role in the toolbar, the fact that the hardware role indicated by the role extension object does not exist in the toolbar means that the hardware role needs to be created.

Thus, the creation of the hardware role is triggered, and the rendering update is realized by executing step 335 shown in fig. 5.

Through the exemplary embodiment, dynamic updating of roles and action blocks by a user can be realized, further, the role interaction scene is fused, not limited to a hardware role, and not limited to the new increase of the hardware role, dynamic updating can also be performed on action blocks which can be configured by the hardware role, and the action blocks correspond to code information, so that dynamic updating of the code information is also realized, for graphical programming realized through the action blocks, content and function realization which can be covered by the graphical programming is greatly enriched, and more realization possibilities are provided while codes are intuitively created.

Fig. 7 is a flowchart illustrating details of step 350 according to the corresponding embodiment of fig. 3. This step 350, as shown in FIG. 7, includes at least the following steps.

In step 351, the hardware roles configured in the character interaction scene are positioned according to the character configuration operation triggered in the character interaction scene, and the hardware roles are rendered to the stage area displayed in the character interaction scene.

Firstly, it should be noted that the interfaces implemented by the application program include two types, one type is a role loading interface, and is used for adding roles and adding action blocks of existing roles; the other is the control and display interface associated with the character interaction scenario.

The control and display interfaces related to the role interaction scene comprise a toolbar, a work area, a code area and a stage area. Action blocks exist in the tool bar, the action blocks are displayed in a classified mode, and the selected action blocks are moved to the working area along with the dragging operation triggered by the user.

The working area is used for implementing the configuration of the role in the role interaction scene, particularly the configuration of the action executed by the hardware role, it should be noted that the action executed by the hardware role includes a control action implemented to the corresponding physical hardware, and then the physical hardware is controlled by the hardware role under the action of the implemented control action. For the configuration of the action executed by a hardware role, in the working area, the selected action blocks are spliced together, and the action execution of the hardware role is ensured by the code information mapped by the action blocks and the splicing relation among the action blocks.

And the code area is used for displaying the code information on the action blocks existing in the working area and the splicing relation among the action blocks in real time. The code information mapped by the action block and the code logic formed by the splicing relation are directly reflected in the code area, and the code area has an editing function, can edit the displayed code information and further directly acts on the action block and the character.

The stage area, which can also be called a stage character area, is an interface presentation of a character interaction scene. All the roles merged into the role interaction scene are displayed in the stage area. In addition, the stage area is provided with a button for jumping to the role loading interface, so that a user can trigger the execution of a new role when needed, for example, the addition of a new hardware role.

And the role configuration operation triggered by the role interaction scene is used for configuring the merged role for the role interaction. In a specific implementation of an exemplary embodiment, according to the triggered role configuration operation, it is determined that a hardware role is configured for the role interaction scene, and at this time, the hardware role is rendered to a stage area where the role interaction scene is displayed.

In step 353, according to the action block configuration operation triggered to the hardware role, the splicing relationship between the action blocks configured for the hardware role is obtained, and the hardware roles displayed in the stage area and the action blocks spliced and mapped to the hardware roles are merged into the role interaction scene.

For the implementation of the role interaction scenario, after the configuration of the role is completed, the configuration of the action block needs to be performed for the configured role, and therefore, an action block configuration operation is initiated to the hardware role configured to the role interaction scenario.

The action block configuration of the hardware role is carried out by initiating action block configuration operation, a plurality of configured action blocks are spliced together to complete the configuration of action required to be executed by the hardware role or action combination, so that the mapped action blocks are also fused into the role interaction scene while the hardware role is fused into the role interaction scene, and further the interaction of the hardware role in the role interaction scene is realized.

In another exemplary embodiment, step 350 further comprises: and creating a main process for the hardware role rendered to the stage area, and creating a corresponding behavior action thread for the action block when the action block of the hardware role is configured.

It should be understood that, a user performs extended loading of a hardware role on owned physical hardware to introduce the hardware role corresponding to the physical hardware into an implemented application program, and there is a need to immediately use the hardware role to configure the hardware role to a role interaction scene.

Therefore, as hardware roles are rendered in the stage area and are merged into the role interaction scene, on one hand, a main process is created for the hardware roles, and the main process is used for realizing interaction between the roles in the role interaction scene and communication between the hardware roles and the outside.

For the action blocks configured by the hardware role, a behavior action thread is created for the action blocks, so that the action configured by the hardware role is executed through the behavior action thread. That is, the behavior action thread executes the code information corresponding to the action block mapped by the hardware role, so that the hardware role can realize the execution of the specified action, including the execution of the physical hardware control action.

Therefore, the accurate control of the physical hardware is realized through the multithreading mode, and in the realization, due to the multithreading mode, various actions can be executed simultaneously, so that the control performance of the hardware role on the physical hardware is improved.

Correspondingly, fig. 8 is a flowchart illustrating details of step 370 according to the corresponding embodiment of fig. 3. This step 370, as shown in fig. 8, includes at least the following steps.

In step 371, when it is monitored that the hardware role mapping action block is triggered to run, the action control information is generated by the action thread corresponding to the triggered action block according to the mapped code description information, and is transmitted back to the host process.

In step 373, the host process transmits the action control information to the physical hardware corresponding to the hardware role mapped by the action block, and triggers the physical hardware to execute the corresponding action.

The realized role interaction scene comprises two parts, wherein one part is role display carried out in a stage area and dynamic display of interaction with the roles; the other part is the working area. The working area not only presents the action blocks configured by the roles and the splicing relation among the action blocks, but also controls the executed actions of the roles in the role interaction scene.

Specifically, the user can apply an operation to the action block displayed in the working area as required to trigger the operation of the action block, so that the user can control the action executed by the character in the character interaction scene.

As mentioned above, each action block configured for a role, i.e., the action block displayed in the work area, has a corresponding behavior action thread. And monitoring the triggered operation of the action block in the working area, and executing the code information corresponding to the action block by the action thread corresponding to the action block so as to enable the role mapped by the corresponding action block to execute the corresponding action.

When the role mapped by the action block is a hardware role, the executed action comprises a control action on physical hardware. The execution of the corresponding behavior action thread can generate action control information for the action block, and the action control information is transmitted to the physical hardware to control the physical hardware to execute the corresponding action, thereby realizing the control of the hardware role to the physical hardware.

Furthermore, the implementation of the control from the hardware role to the physical hardware action is realized under the coordination of the behavior action thread corresponding to the executed action block and the main process. The action control information generated by the behavioral action thread needs to be transferable to the physical hardware by a communication partner with the physical hardware. Therefore, after the behavior action thread generates the action control information, the action control information is returned to the host process through the interface, and the host process sends the action control information to the physical hardware through the communication connection interface.

Correspondingly, the physical hardware receiving the action control information can execute the corresponding action, and returns the action execution result to the hardware role along with the execution of the action. Specifically, the action execution result is returned to the protocol processor on the behavior action thread along the receiving path, and the specific result is obtained through the analysis of the protocol processor and is returned to the main thread, so that the execution of the processing process corresponding to an action block is completed.

Therefore, under the action of the configured working area and the stage area, the code information for controlling interaction between the roles and execution of the physical hardware can be rapidly deployed for the hardware roles by dragging the plurality of action blocks in the toolbar to the working area, so that the corresponding action execution is realized in real time, the interaction performance between the virtual world and the physical world is enhanced, and the threshold of imaging programming under the interaction scene of the realized roles is also reduced.

FIG. 9 is a flow chart illustrating a method of controlling physical hardware, according to another exemplary embodiment. The flow chart of the control method of the physical hardware, as shown in fig. 9, at least includes the following steps.

In step 510, mapped code description information is located for the action blocks merged into the character interaction scene.

As described above, each action block has corresponding code description information. The code description information indicates the specific definition of the corresponding action block, for example, the parameter, the return value and the corresponding code information, and further, the method for executing the action for the corresponding role can be encapsulated by the parameter, the return value and the corresponding code information.

In the role interaction scene, each action block configured for the role is positioned to the mapped code collocation information according to the corresponding relation.

In step 530, the conversion from the action block to the code information is performed according to the code description information, and the code information in the code mode is displayed for the character interaction scene.

And the code information in the mapped code description information is extracted for the action block, and the extracted code information is displayed in a code area for a user to view the currently edited code.

In an exemplary embodiment, an action block definition layer is constructed according to the code description information, and interconversion between the action block and the code is realized through the constructed action block definition layer.

Specifically, the code description information includes parameters, return values, and code information, and in addition, attributes of the action block such as an action block ID and an action block description, and therefore, an action block definition layer is constructed according to the contents included in the code description information.

Thus, the action block configured in the role interaction scene is quickly converted to the corresponding code from the constructed action block definition layer and returned, or the written code is quickly converted to the corresponding action block, and the action block is automatically dragged into the work area, so that the dynamic configuration of the action executed by the role in the role interaction scene is realized, and a plurality of paths are provided for obtaining code information.

Through the exemplary embodiment, the code display of the configured action block is realized, the displayed code information is matched with the configured action block, and the displayed code information is dynamically increased along with the continuous configuration of the action block to the role, so that the programming learning experience of a user is enhanced.

FIG. 10 is a flow chart illustrating a method of controlling physical hardware, according to another exemplary embodiment. As shown in fig. 10, the method for controlling physical hardware at least includes the following steps.

In step 610, code information of the newly added action of the designated character is obtained according to the code information input triggered for the designated character in the code mode of the character interaction scene.

The code mode is an operation mode for configuring action blocks for a designated role in a role interaction scene in a code information input mode. That is, the action block configuration of the character includes the action block implementation in the drag toolbar and also includes the input of code information in the code mode.

Even if the action block is configured on the designated role, the core of the action block lies in the corresponding code information, and the execution action is realized according to the configured code information. Therefore, no matter what operation mode is collected, the configuration of the action executed by the character in the character interaction scene can be carried out.

Specifically, as the user input operation is triggered, the code region performs code information input, and this process is performed for a role in the role interaction scene, which is the designated role.

In step 630, the code description information is mapped according to the code information of the specified role adding action.

As described above, the action block has corresponding code description information, and the code description information includes code information, so that the code information input for specifying the role adding action can be mapped to the corresponding code description information.

In step 650, the action blocks of the designated role mapped by the code description information are located, and the action blocks are dynamically spliced to the action blocks of the designated role which are already mapped in the role interaction scene.

And obtaining the corresponding action block according to the mapped code description information, wherein the action block is the action block configured to the specified role through the input operation of the user.

It should be understood that the code information in the work area includes code information input by the user, which is matched with the action blocks existing in the work area, and the logical relationship between the code information corresponds to the splicing relationship between the action blocks in the work area.

Therefore, after the action block mapped by the input code information is located, the action block is automatically dragged into the work area, and the dragged action block in the work area is spliced according to the logical relation between the code information in the code area.

In an exemplary embodiment, after obtaining the code information input by the user for the specified role new action in the code mode, as described above, since the action block definition layer is already constructed for the action block based on the code description information, the action block mapped by the input code information can be directly obtained by the action block definition layer, which is convenient and fast.

Further, in an exemplary embodiment, step 610 includes: the character interaction scene continues to receive the input characters in the code mode, and grammar checking and/or macro definition matching is carried out on the input characters through a predefined code segment mapping library.

It should be understood that, in the code mode, the code information input process triggered in the code area is a continuous character input process. Therefore, in the process, the continuously input characters are received, grammar check is carried out on the characters through a predefined code segment mapping library, and prompt and grammar error reporting are timely and accurately carried out.

In addition, macro definition matching is carried out based on a predefined code segment library, and possibly corresponding code segments are displayed in a code area for a user to select and insert so as to realize quick input of code information.

The pre-defined code segment mapping library is used for storing various code segments, on one hand, whether the input characters meet the grammar specification is judged according to the code segment mapping library, and on the other hand, a matched macro definition, namely encapsulation of the code segments, is provided for a user so as to improve the input efficiency.

By the embodiment of the invention, the access of physical hardware is realized, the role interaction of the hardware role and the action control of the hardware role to the physical hardware are realized, on the basis, the action block configuration of code information corresponding to the freely executed action and the code information input reversely performed are realized, and the dynamic change of the action block is brought, for a user, the action of the role caused by the action of the code information and the change of the code information is sensed, even the change of the action executed by the physical hardware, so that for the programming learning performed by the user, the code information and the grammar are expressed in a three-dimensional image, and the intuition and the visualization in the programming learning are enhanced.

Therefore, the embodiment of the invention provides more interactive performance for various physical hardware, can realize random free configuration of physical hardware interaction and action execution, and can obtain instant verification for the configuration process, such as the configuration of action blocks and the input of code information, through the display of the stage area and the action execution of the physical hardware.

Through the embodiment of the invention, a completely visual and touchable programming process can be realized for the user.

Taking a physical hardware a as an example, the control process of the physical hardware is described with a specific scenario.

FIG. 11 is a schematic diagram illustrating an implementation of loading of a character in a character interaction scenario, according to an example embodiment. It should be noted that, the introduction of a role in a role interaction scene refers to new physical hardware, for example, physical hardware a, so that the physical hardware a can have a corresponding hardware role and further participate in interaction in the role interaction scene, as shown in fig. 11, no matter a virtual role or the hardware role a executes loading, parsing and rendering processes under the action of a role loader 710, that is, Extension Manager, as in the execution of steps 730 to 750, and further respectively causes a work area, a stage area and a code area to obtain corresponding content display.

In step 730, the role loading is performed on the configuration file, and similarly, the introduction of the new action block is performed through the configuration file.

FIG. 12 is a configuration file diagram of physical hardware A, shown in accordance with an exemplary embodiment. For the physical hardware A, no matter the introduction of the hardware role corresponding to the physical hardware A is carried out, or the action block is newly added for the interaction of the physical hardware A in the role interaction scene, the method can be realized through the corresponding configuration file.

As shown in fig. 12, the configuration file of the physical hardware a includes a role name, a role icon, an action block category, and an action block specific definition. The action block specific definition corresponds to code information and the like.

FIG. 13 is a flowchart illustrating an implementation of physical hardware A performing an extended load to inject a corresponding hardware role in accordance with an illustrative embodiment. However, it should be noted that, the method is not limited to the physical hardware a, and various different physical hardware may be loaded in an extended manner through the implementation flow, and then injected into a role interaction scenario.

Specifically, step 810 is executed to load a defined configuration file, which can be implemented by executing a loadExtension statement, so as to generate a role extension object and a role extension object extension.

Wherein, for the role extension object extension, the defined name is the role name; the path is a storage path; worker is the task that needs to be performed, which will be the behavioral action thread corresponding to the action block; blocks are action blocks; device is physical hardware.

Therefore, on one hand, the new hardware role extension loading can be realized, such as the subsequent execution of the step 820, and on the other hand, the newly added configurable action blocks for the extended hardware role, such as the subsequent execution of the step 830, have the capability of dynamically adding the action blocks.

As shown in step 820, it is determined whether the generated role extension object is new physical hardware, and the implementation of the newdevice determination statement may be performed. If it is new, then a hardware role addition for this physical hardware is made, as shown in

steps

821 and 840.

As shown in step 830, it is also determined that the generated role extension object is a new action block, and if the role extension object is a new action block, the action block and the corresponding code information are created and updated, as shown in the sequence of step 831, step 833 and step 840.

Therefore, new physical hardware, such as the extended loading of the hardware role corresponding to the physical hardware A, can be completed, and the action blocks of the hardware role of the extended loading of the physical hardware A can be newly added as required.

Fig. 14 is a diagram illustrating a role loading interface, according to an exemplary embodiment, through the process described above, the hardware role B in the stage area 870 and the action block set of the toolbar 880 shown in fig. 14 are obtained.

It should be understood that the action block set displayed in the interface is displayed in a category according to the action block category defined in the configuration file.

For each loading process, a corresponding extended task is created for this purpose, as in step 910, on one hand, generation corresponding to role extension is implemented, and on the other hand, a listening task in which the action block is triggered to run is included, as shown in task 920, and execution and communication of each action, especially implementation of communication between the hardware role and the physical hardware, are guaranteed under the action of the corresponding action thread.

When it is monitored that the action block configured by the hardware role corresponding to the physical hardware is triggered to run, the action thread corresponding to the action block, that is, the action thread created for the newly added action block through step 850 after the action block addition is executed, will obtain the action control information by the protocol processor.

The behavior action thread transmits the obtained action control information to the main process, and the main process executes the sending and receiving with the physical hardware, thereby ensuring the action execution of the physical hardware.

On the basis, the real-time proceeding of code prompting and error grammar highlighting in the code mode is extended. In the code area, the action block configuration of the character can be performed through the input of the character corresponding to the code information, furthermore, the action executed by the character can be configured without directly configuring the action in the action block mode, and the action can be configured correspondingly through the input of the character.

FIG. 15 is a diagram illustrating an implementation of code hinting and error grammar highlighting in a code mode, according to an illustrative embodiment.

Under the action of the grammar checker, the input characters are subjected to the check of the grammar specification, such as the execution of

steps

1010 and 1020, on one hand, and the execution of

steps

1030 and 1040, on the other hand, the code segments which can be corresponded are obtained.

On the basis, as the code information is input in the code area, the corresponding action block is dragged into the work area to realize the conversion of the code into the action block.

FIG. 16 is an implementation diagram illustrating conversion between code and action blocks, according to an example embodiment. For the implementation of the present invention, the input of the action block and the code information are both two entries of the role for performing action configuration, so that the action block and the input code information are converted to each other.

Specifically, for an action block, there is a corresponding action block definition layer, which is the only description of the action block. The action block definition layer includes an action block ID, an action block description, an action block parameter, an action block return value, and an action block attribute such as corresponding code information, and is an existing form of code description information.

When the codes are written, each sentence of codes finds the corresponding action block mapping in the action block definition layer, and then the graphs of the action blocks are reversely rendered to the working area, so that the function of simultaneously showing the graphs of the corresponding action blocks when each sentence of codes is input is realized.

FIG. 17 is a diagram illustrating interface partitioning, according to an example embodiment. As shown in FIG. 17, code area 1110, in code mode, performs character entry in the selected computer language and obtains a corresponding representation of the action blocks in work area 1130.

Whether an action block configured in work area 1130 or code information contained in code area 1110, configures the action performed for hardware role B in stage area 870.

Through the realization of the invention, physical hardware freely obtained by a user, such as electronic components provided with various sensors, equipment with an activity function and the like, can be integrated into a role interaction scene, and the user can easily fully understand each sentence of codes and the logic relation between the codes through an action block dragged at will and the matching of the action block and the code information by taking the physical hardware as the assistance, thereby providing an effective assistant tool for the programming and learning of the user while realizing graphical programming.

The following is an embodiment of the apparatus of the present invention, which can be used to implement the above-mentioned embodiment of the control method of the physical hardware of the present invention. For details that are not disclosed in the embodiments of the apparatus of the present invention, refer to the embodiments of the control method of physical hardware of the present invention.

FIG. 18 is a block diagram illustrating a control device of physical hardware in accordance with an exemplary embodiment. The control device of the physical hardware at least comprises: an extension load module 1210, a render update module 1230, a role configuration module 1250, and an action execution module 1270.

The extended loading module 1210 is configured to perform extended loading of hardware roles corresponding to physical hardware through a configuration file defined by the physical hardware.

And the rendering updating module 1230 is configured to perform rendering updating on the hardware role through the configuration file that is loaded for physical hardware extension, and obtain action blocks for the hardware role and the hardware role to be configured.

The role configuration module 1250 is configured to obtain a role interaction scene in which a hardware role is merged and the hardware role is mapped to an action block according to the hardware role performed in the role interaction scene and the action block configuration operation of the performed action.

And the action execution module 1270 is used for monitoring the triggering operation of the action block mapped by the hardware role and controlling the action of the corresponding physical hardware according to the triggered operation action block.

FIG. 19 is a block diagram illustrating details of an extended load module according to the corresponding embodiment of FIG. 18. The extension loading module 1210, as shown in fig. 19, at least includes: a configuration file obtaining unit 1211 and a configuration loading unit 1213.

A configuration file obtaining unit 1211, configured to obtain, according to the hardware role addition or the addition of the configuration action blocks available for the hardware role, a configuration file defined corresponding to the physical hardware, where the configuration file is used to describe the hardware role added or the action blocks added by the hardware role.

A configuration loading unit 1213, configured to load a configuration file to extend a hardware role or add an action block to the extended hardware role.

Optionally, the present invention further provides a control device of physical hardware, which can execute all or part of the steps of the control method of physical hardware shown in any one of fig. 2, fig. 3, fig. 4 and fig. 5 in the foregoing implementation environment. The device comprises:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to perform:

performing rendering updating of a hardware role through a configuration file which is expanded and loaded for physical hardware, and obtaining the hardware role and an action block which can be configured by the hardware role;

The specific manner in which the processor of the apparatus in this embodiment performs operations has been described in detail in the embodiment of the control method with respect to the physical hardware, and will not be elaborated upon here.

In an exemplary embodiment, a storage medium is also provided that is a computer-readable storage medium, such as may be transitory and non-transitory computer-readable storage media, including instructions. The storage medium, for example, includes the memory 104 of instructions executable by the processor 118 of the device 100 to perform the method described above.

It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims

1. A method of controlling physical hardware, the method comprising:

performing rendering updating of the hardware role through the configuration file which is extended and loaded for the physical hardware, and obtaining the hardware role and action blocks which can be configured by the hardware role;

2. The method according to claim 1, wherein the performing, by the configuration file defined by the physical hardware, the extended loading of the hardware role corresponding to the physical hardware comprises:

acquiring a configuration file defined by corresponding physical hardware according to hardware role addition caused by triggering or configuration action block addition available for the hardware role, wherein the configuration file is used for describing the newly added hardware role or the newly added action block of the hardware role;

and loading the configuration file to expand the hardware role or add an action block for the expanded hardware role.

3. The method of claim 2, wherein the performing a rendering update of a hardware role by the configuration file loaded for the physical hardware extension to obtain a hardware role and an action block for which the hardware role is configurable comprises:

analyzing the loaded configuration file to generate a hardware role or a role expansion object of a newly added action block for the expanded hardware role;

rendering a hardware role or a newly added action block of the expanded hardware role through the role expansion object;

and updating the hardware role obtained by rendering updating or the expanded hardware role newly-added action block to a toolbar corresponding to the role interaction scene.

4. The method of claim 1, further comprising:

respectively positioning mapped code description information for the action blocks merged into the role interaction scene;

and converting the action block into code information according to the code description information, and displaying the code information in a code mode on the role interaction scene.

5. The method of claim 1, further comprising:

according to code information input triggered for the specified role in the code mode of the role interaction scene, code information of the newly added action of the specified role is obtained;

positioning the code description information according to the code information of the newly added action of the specified role;

positioning the action blocks of the designated role mapped by the code description information, and dynamically splicing the action blocks to the action blocks mapped by the designated role in the role interaction scene.

6. The method according to claim 5, wherein the obtaining code information of the newly added action of the designated character according to the code information input triggered by the character interaction scenario in the code mode for the designated character comprises:

the character interaction scene continues to receive the input characters in the code mode, and grammar checking and/or macro definition matching is carried out on the input characters through a predefined code segment mapping library.

7. An apparatus for controlling physical hardware, the apparatus comprising:

8. The apparatus of claim 7, wherein the extension loading module comprises:

a configuration file obtaining unit, configured to obtain a configuration file defined for corresponding physical hardware according to a hardware role newly added by triggering or an action block available for configuration for the hardware role, where the configuration file is used to describe a newly added hardware role or the action block newly added by the hardware role;

and the configuration loading unit is used for loading the configuration file so as to expand the hardware role or add an action block for the expanded hardware role.

9. A control apparatus for physical hardware, comprising:

a processor; and

a memory having stored thereon computer readable instructions which, when executed by the processor, implement a method of controlling physical hardware according to any of claims 1 to 6.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, implements a method of controlling physical hardware according to any one of claims 1 to 6.