CN113515272A - Visual programming method and system - Google Patents
Visual programming method and system Download PDFInfo
- Publication number
- CN113515272A CN113515272A CN202010273369.5A CN202010273369A CN113515272A CN 113515272 A CN113515272 A CN 113515272A CN 202010273369 A CN202010273369 A CN 202010273369A CN 113515272 A CN113515272 A CN 113515272A
- Authority
- CN
- China
- Prior art keywords
- instruction
- building block
- node
- behavior tree
- instruction building
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000000007 visual effect Effects 0.000 title claims abstract description 28
- 230000006870 function Effects 0.000 claims description 80
- 238000012800 visualization Methods 0.000 claims description 19
- 238000012300 Sequence Analysis Methods 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 4
- 239000002131 composite material Substances 0.000 claims description 3
- 230000033001 locomotion Effects 0.000 claims description 3
- 239000000463 material Substances 0.000 claims description 2
- 238000000547 structure data Methods 0.000 claims description 2
- 230000008034 disappearance Effects 0.000 claims 1
- 238000006073 displacement reaction Methods 0.000 claims 1
- 230000006399 behavior Effects 0.000 abstract description 108
- 238000010586 diagram Methods 0.000 description 8
- 230000009471 action Effects 0.000 description 5
- 239000000853 adhesive Substances 0.000 description 2
- 230000001070 adhesive effect Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000005034 decoration Methods 0.000 description 2
- 239000002245 particle Substances 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000004026 adhesive bonding Methods 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/34—Graphical or visual programming
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The embodiment of the invention aims to provide a visual programming method and a visual programming system, wherein a graphic element in a program object display view type is used as a program object by replacing a text programming language with the graphic element, and the functional meaning expressed by an instruction building block graphic sequence is realized by the operation of an object behavior tree node. By the system, children can quickly master programming thinking and programming methods on the premise of not learning languages, and the problems that the children are too high in threshold for learning programming and difficult to popularize are solved. The visual programming system provided by the embodiment of the application comprises an instruction building block sequence editing module, an instruction building block sequence analyzing module, an object behavior tree generating module, a global variable module, a behavior tree operating module, a behavior tree node operating result visual display module and the like.
Description
Technical Field
The invention relates to the field of electronic teaching, in particular to a method and a system for visual programming teaching.
Background
The state institute points out that artificial intelligence becomes a new focus of international competition, and people's intelligent education projects should be developed step by step, and relevant courses of artificial intelligence are set in the middle and primary schools, so that programming education is popularized step by step. However, in the primary school stage of the just-touch programming, if the traditional computer language is directly learned, the learning interest of children is lost due to the fact that the traditional computer language is difficult to access and the content is boring. The invention aims to solve the threshold problem of programming learning of children, provides a system which can program through the operation of visual instructions without learning the traditional computer language, and enables children to more easily master the programming concept and thinking through the system, so as to arouse the learning interest of programming and lay a foundation for deeper programming learning in the future.
The visual programming has the characteristics of low learning threshold and simple operation, and the core of the visual programming lies in converting a text programming language in a code form into a graphic programming language which is easy to understand and expressing functional meanings graphically through a series of instructions. At present, when a visual programming scheme is adopted, a user can firstly select a command building block required by the user in a building block list field of a programming interface, then the building block is placed in a programming operation field, and a command sequence is formed through bonding and nesting, so that the compiling of a graphical programming language is realized.
A behavior tree is a formal graph modeling language and is a tree structure formed by behavior nodes, and each node represents a behavior. In a behavioral tree, nodes are hierarchical, with children nodes controlled by their parents. Each node's execution has a result (success, failure or run) that is managed by its parent node, which determines what to do next, and the type of parent node determines the different control types. The nodes do not need to maintain conversion to other nodes, and the modularity of the nodes is greatly enhanced. In fact, in the action tree, since nodes no longer have transitions, they are no longer states, but actions.
Disclosure of Invention
The embodiment of the invention aims to provide a visual programming method and a visual programming system, wherein a graphic element in a view type shown by a programming object is used as the programming object by replacing a text programming language with the graphic element, and the functional meaning expressed by an instruction building block graphic sequence is realized by the operation of an object behavior tree node. By the system, children can quickly master programming thinking and programming methods on the premise of not learning languages, and the problems that the children are too high in threshold for learning programming and difficult to popularize are solved.
In order to achieve the above object, an embodiment of the present application provides the following technical solutions, including:
is a graphical object oriented visual programming.
The text programming language is replaced with instruction building block graphics.
And forming an instruction building block sequence by editing the instruction building blocks, and expressing the operation sequence of the instructions by using the arrangement of the instruction building block sequence.
And converting the arrangement data of the instruction building block sequence and the parameters of the instruction building blocks into a behavior tree combined by the instruction building block nodes.
And realizing the function and the operation sequence of the instruction building block sequence expression through the operation of the behavior tree nodes.
And visually displaying the running sequence and the function expressed by the programming instruction sequence through the sequence and the result of the programming object executing the behavior tree node function in the visual scene.
The visual programming system comprises an instruction building block sequence editing module, an instruction building block sequence analyzing module, an object behavior tree generating module, a global variable module, a behavior tree operating module, a behavior tree node operating result visual display module and the like.
In a first aspect, embodiments of the present application further provide a visualization programming method and system, where the method and system include:
the representation form of the instruction building blocks is graphical symbols, and the instruction building blocks with different functions are represented by different graphical symbols.
Each instruction building block graphical symbol object contains a set of parameter data for generating a behavior tree node.
The instruction building block behavior tree node combination modes of all instruction building blocks are contained in a frame in a unified instruction building block node combination structure, and the instruction building blocks use part or all nodes in the unified instruction building block node combination structure to form corresponding instruction building block behavior tree node combinations according to the function difference.
And different instruction building blocks set corresponding behavior tree node combinations and node parameter data according to the expressed functions.
The function expressed by the instruction building block is simulated through the operation sequence and the result of the nodes in the behavior tree node combination of the instruction building block.
In a second aspect, embodiments of the present application further provide a visualization programming method and system, where the method and system include:
the command building block behavior tree node combined structure framework and the programmable object behavior tree node combined structure framework.
Instruction building blocks action tree node integrated configuration frame:
in the instruction building block behavior tree node combined structure framework, an instruction building block father node can manage a plurality of parameter nodes and a plurality of function nodes.
And the parameter child points manage parameter value nodes and parent nodes of the instruction building block behavior tree node combination of the instruction building block which is embedded in the instruction as the parameter values.
The function node can manage a child node execution sequence management node, and the child node execution sequence management node manages a parent node of an instruction building block behavior tree node combination of an instruction building block which is a functional module embedded in an instruction.
The parameter value nodes can acquire and record various parameters and variables such as numerical parameters and variables, collision detection value parameters and variables, text parameter values and variables and the like through corresponding setting.
Nodes in the functional branch are used to perform functions of the instruction such as move, loop, determine, etc. The function node may set the dynamic parameters returned from the call parameter node in the global variable manager.
The function node can be set to execute different types of functions, such as operation addition, subtraction, multiplication and division results, judgment whether parameters are consistent with target values, object coordinate movement, object coordinate rotation, nested instruction loop execution and the like.
The programmable object behavior tree node composite structure framework comprises the following steps:
in the embodiment of the present application, a plurality of preset programmable objects may be programmed, and each programmable object generates a corresponding class programming object behavior tree node combination for managing execution of a corresponding instruction sequence.
The father node of the behavior tree node combination is a behavior tree root node of the programmable object and is used for managing the operation of the parameter branch father node and the function branch father node.
And the parameter nodes are used for acquiring and recording various parameter values related to the object instruction building block sequence, and adding or updating the parameter values serving as variables in the global variable manager, wherein the parameter values can be numerical values, texts and the like.
And the function node is used for managing the operation of the instruction sequence father node.
In the embodiment of the application, the instruction sequence may be triggered to execute by an event, a message, or the like, which allows a user to edit multiple instruction building block sequences for the same object, and after the instruction sequences generate the behavior tree node combination, a parent node of the instruction building block sequence will be used as a branch of a functional node in the object behavior tree node combination.
In a fourth aspect, embodiments of the present application further provide a visualization programming method and system, where the method and system include:
and the instruction building block sequence editing module can edit the instruction building block pattern elements into an instruction building block sequence consisting of the instruction building block pattern elements. Which comprises the following steps:
the instruction building block type display method is used for displaying the classification labels of various instruction building blocks, and the instruction building block types displayed in the instruction building block list can be switched by selecting the labels.
The instruction building blocks are used for displaying various instruction building block sequences which can be used for editing the instruction building block sequences, and the instruction building blocks can be selected from the instruction building block list to generate editable copies to the instruction building block sequence editing area.
The editing area is used for editing the instruction building block sequence, and the instruction building blocks can be moved, adhered, nested, deleted and the like in the instruction building block editing area; a plurality of instruction blocks can be edited into an instruction block sequence in the instruction block editing area.
Each instruction building block is provided with a bonding point, the instruction building blocks can be bonded up and down and embedded inside and outside through the bonding points, and an instruction building block sequence can be formed through bonding and embedding.
Some instruction blocks include editable parameters, and the editable parameters in the instruction blocks can be modified in the instruction block editing area.
The object instruction building block in the type of the instruction building block is used for specifying a visual programming object; a plurality of different object instruction building blocks can be arranged on the same object according to different event triggering modes, and a plurality of instruction sequences can be edited on the same object by using the different object instruction building blocks.
In a fifth aspect, embodiments of the present application further provide a visualization programming method and system, where the method and system include:
the instruction building block sequence analysis module is used for analyzing the arrangement data of the instruction building blocks into behavior tree node combined structure data, analyzing the parameters of the instruction building blocks into parameter data of the behavior tree nodes of the instruction building blocks, and analyzing the variable data of the instruction building blocks into behavior tree global variable data.
In a sixth aspect, embodiments of the present application further provide a visualization programming method and system, where the method and system include:
and the object behavior tree generation module can generate the behavior tree of the object according to the behavior tree node combination data output by the instruction building block sequence analysis module.
And the behavior tree node parameter data is given to the corresponding behavior tree node according to the behavior tree node parameter data output by the instruction building block sequence analysis function.
In a seventh aspect, an embodiment of the present application further provides a visualization programming method and system, where the method and system include:
and the global variable module is used for generating a corresponding behavior tree global variable according to the variable data output by the instruction building block sequence analysis module.
In one possible implementation, nodes in the behavior tree may make changes and reads to behavior tree global variables.
In a sixth aspect, embodiments of the present application further provide a visualization programming method and system, where the method and system include:
and the behavior tree operation module is used for managing the operation of the behavior tree.
The operation rule of the behavior tree is that the operation is started by the root node of the object; the operation of the child nodes is managed by the parent node.
And simulating the functions and the operation sequence expressed by the instruction building block sequence by using the operation sequence and the operation result of the nodes in the behavior tree through setting the operation rule of the behavior tree and setting the combination of the instruction building block nodes.
In an eighth aspect, embodiments of the present application further provide a visualization programming method and system, where the method and system include:
and the behavior tree node operation result visualization display module is used for displaying the operation sequence and the structure of the object behavior tree converted from the programming instruction building block sequence in a visualization mode.
The presentation view content is composed of various graphic element objects which can be set into programmable objects and non-programmable objects.
There may be any number of graphic element objects in the behavior tree node operation result visualization presentation module.
The graphic object is not limited to a specific graphic, and may be any shape such as a character image, an animal image, an environment, a particle, a plant, a machine, a building, a decoration, a UI interface, an icon, and the like.
In one implementation, the graphic object may be a 2d graphic or a 3d graphic.
In an embodiment of the application, a visualization programming method and system are provided. According to the embodiment of the invention, the combination structure and the operation mechanism of the behavior tree nodes after the behavior tree nodes are generated by the instruction building blocks are unified through the instruction building block behavior tree general node combination framework. In the instruction building block universal node combination frame, functions of various instruction building blocks can be accurately simulated through unified node level design, and in the action tree node combination generated by the instruction building block sequence formed by the various instruction building blocks, the functions and the operation sequence expressed by the instruction building block sequence can be accurately simulated through the action tree node combination generated by the unified frame, so that the system can contain more types of instructions and perform complex combination on the instructions without generating wrong simulation.
Drawings
FIG. 1 is an exemplary diagram of an instruction sequence editing interface layout of an instruction building block sequence editing module.
FIG. 2 is an exemplary illustration of an instruction block being spliced into an instruction block sequence.
FIG. 3 is an exemplary diagram of instructions that may be embedded in an instruction building block in which instructions may be embedded up and down.
FIG. 4 is an exemplary diagram of instructions that may be embedded in an embeddable instruction block in which instructions may be embedded in and out.
FIG. 5 is an exemplary diagram of an instruction block behavior node combination frame corresponding to an instruction block.
Fig. 6 is an exemplary diagram of a combination manner of instructing a parent node of a sequence behavior tree as a child node of a combination of nodes of an object behavior tree.
FIG. 7 is an exemplary diagram of a visualization scenario when an object runs a behavior tree.
FIG. 8 is a flowchart illustrating the operation of the visual programming method and system of the present invention.
FIG. 9 is an exemplary diagram of a node combination structure after a building block sequence is instructed to generate a behavior tree.
Detailed Description
In order to make the objects, features and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the embodiments described below are only a part of the embodiments of the present application, but not all the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present application belong to the scope of protection of the embodiments in the present application.
The terms "comprises" and "comprising," and any variations thereof, in the description and claims of the embodiments of the present application and the drawings described above, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Fig. 1 is a schematic view of a visual instruction sequence editing interface of an instruction editing module.
As shown in 101 in fig. 1, in the embodiment of the present application, instruction building block type selection tags are provided, the instruction building block types include objects, motions, operations, controls, detections, appearances, events, functions, and the like, and by clicking different tags, instruction building blocks of corresponding types can be displayed in an instruction building block display list 102.
As shown at 102 in fig. 1, in the embodiment of the present application, an instruction block list is provided for displaying various types of instruction blocks. Each type of instruction building block corresponds to a group of color matching. The instruction blocks in the instruction list can create instruction blocks which can be used for editing the instruction sequence in the 103 instruction sequence editing area in a dragging mode.
As shown in fig. 1 at 103, in the embodiment of the present application, an instruction sequence editing area is provided, in which editing of an instruction sequence can be completed by operations of gluing, nesting, moving, deleting, copying, and the like, on an instruction block.
Some instruction blocks can be embedded with other instruction blocks, which means that the operation of the embedded instruction is controlled by the instruction.
Each instruction building block is provided with a bonding point, and the instruction building blocks are bonded and nested through the bonding points.
Fig. 2 and 2 are examples of implementing the bonding of the instruction blocks through the bonding points of the instruction blocks.
As shown in 201 of fig. 2, all instruction blocks need to have sticky points, which are invisible virtual objects.
Instruction building blocks a has set up two upper and lower bonding point in this embodiment, and the effect of bonding point is whether to detect and bump in other bonding point, if bump, judges whether can bond, if can bond then the bonding function of execution instruction building blocks.
As shown in fig. 2, 202 is an upper bonding point of the instruction block a, and 203 is a lower bonding point of the instruction block a. The upper bonding point of any instruction building block can only be bonded with the lower bonding point of other instructions, and the lower bonding point of any instruction can only be bonded with the upper bonding point of other instructions.
The positions and the number of the bonding points of all types of instructions are not consistent, and the positions and the number of the bonding points can be correspondingly configured according to the expressed functions of the instruction blocks, for example, some instruction blocks only have one upper bonding point, some instruction blocks only have one lower bonding point, some instruction blocks have two left and right bonding points, and the like.
When the position attribute of the bonding point of the instruction building block is left and right, the position bonding rule is that the left bonding point can only bond the right bonding point, and the right bonding point can only bond the left bonding point.
In this embodiment, the sticky point further has an instruction type attribute, and the sticky instruction type attribute is set to prevent any instruction from being sticky, thereby eliminating invalid sticking or nesting between instructions, for example, a numerical instruction may be nested in an operation instruction, but may not be nested as a detection instruction.
As shown at 204 in fig. 2, the white dotted circles are the bonding points of the instruction blocks in the edited instruction block series, and when a new instruction block is bonded to them, they will automatically disconnect the previous bonding points and automatically bond the bonding points of the new instruction block up and down.
As shown at 205 in fig. 2, instruction block a may be glued anywhere in the edited sequence of instruction blocks.
Fig. 3, 3 are examples of an instruction block that can be embedded in an instruction block in which the instruction block can be embedded up and down.
In this embodiment, there is a class of special instruction blocks, referred to herein as embeddable instruction blocks, that embed other instruction blocks in the instruction graph structure.
The embeddable instruction block can have any number of embeddable slots according to the functional requirements of the instruction block. Meanwhile, the number of the instruction blocks which can be embedded or can be embedded by which type of instruction blocks can be set according to the requirement of the instruction function.
The instruction block a at 301 in fig. 3 is a form that can be inserted into an instruction block, and upper and lower two adhesion points are provided in the insertable groove, and the inserted instruction block can be adhered by the two adhesion points.
The insertable slot of the command block a is shown at 302 in fig. 3 to be retractable according to the total height of the command block inserted therein.
303 instruction building block b in fig. 3 nests in instruction building block a's the structure that contains through dragging the mode of bonding, and when instruction building block b's last adhesive linkage and instruction building block b contain the last adhesive linkage of structure collision, instruction building block b alright nestification is in instruction building block a.
In this embodiment 304 is a command block sequence, which can be inserted into the slot of the command block a by means of dragging and bonding, and when the upper bonding point of the 1 st command block in the command block sequence 304 collides with the upper bonding point of the slot of the command block b, the command block sequence 304 can be nested into the slot of the command block a.
When the instruction building blocks need to be deleted, only the instruction building blocks needing to be deleted are selected and are popped out of the instruction building block sequence, the containing structure of the instruction building block a automatically recalculates the height of the contained instruction building block sequence and shortens the height to be equal to the contained instruction building block sequence.
Fig. 4, 4 are examples of an embeddable instruction block in which an instruction block can be embedded right and left.
As shown in fig. 4, in this embodiment, there is a type of command block that can be inserted into a slot that can be inserted into 1 command block, and multiple insertion is possible.
The 401 instruction block a in fig. 4 is a form that can be embedded in an instruction block.
In fig. 4, 402 is an embeddable slot of an embeddable instruction block a.
As shown at 403 in fig. 4, the embedded slot of the command block a can be extended or contracted according to the width of the command block embedded therein, and the width of the command block a is also extended or contracted together.
As shown at 404 in fig. 4, the embedded slot of the command block a can be extended or contracted according to the height of the command block embedded therein, and the height of the command block a is also extended or contracted together.
As shown in 405 in fig. 4, the instruction block b is an embeddable instruction, and can be dragged to the instruction block a by dragging, where the instruction block b can be embedded into the instruction block groove, and can be automatically embedded into the instruction block a after the bonding point of the instruction block b collides with the bonding point of the instruction block a.
As shown at 406 in fig. 4, instruction block c may be embedded and may be embedded into an instruction block, two instruction blocks may be embedded, and it may also be embedded into instruction block a by dragging.
Fig. 5 and 5 are examples of object behavior tree node combination structure frameworks.
In the present embodiment, each instruction block corresponds to a group of behavior tree node combinations, and such behavior tree node combinations are hereinafter referred to as instruction block behavior tree node combinations.
The instruction building block behavior tree node combination structures of the instruction building blocks are all contained in the behavior tree node combination structure frame shown in fig. 5, and different instruction building blocks use partial nodes thereof as the instruction building block behavior tree node combination structures according to the difference of functions.
As shown at 501 in fig. 5, an instruction building block parent node manages the operation of parameter branches and function branches, and an instruction building block parent node may manage a plurality of parameter branches and function branches.
502. The parameter 1 child node is used to manage the operation of its child nodes.
As shown in 503 of fig. 5, in the instruction block behavior tree node combination structure framework, any number of parameter branch nodes may be set in the instruction block node combination. For example, the instruction building block moves to an X coordinate, a Y coordinate, and a Z coordinate, which include 3 coordinate value parameters, and the corresponding instruction building block node combination may set 3 parameter branches to respectively correspond to 3 coordinates.
As shown at 504 in fig. 5, the function node may be configured to perform various functions, such as adding, subtracting, multiplying, dividing, determining whether the parameters are consistent with the target values, moving object coordinates, rotating object coordinates, looping through child nodes, invoking corresponding variables from global variables, and the like.
As shown in 505 of fig. 5, in the instruction building block behavior tree node combination structure framework, any number of function nodes may be set in the instruction building block node combination. For example, if the execution … is positive, otherwise the execution … instructs the building blocks to execute one of two instruction building sequences nested therein according to conditions, such instruction building block node combinations require a plurality of function nodes to be set.
As shown in 506 in fig. 5, parameters of some instruction blocks may be embedded in the instruction block as parameters by using variable instruction blocks, judgment condition instruction blocks, addition operation instruction blocks, and the like. After the generated behavior tree, the position of the parent node of the instruction block node combination of the instruction block nested in the behavior tree is generated in the branch of the parameter node.
As shown in 507 in fig. 5, the parameter value node is configured to obtain and record various parameter values, and add or update the parameter values as variables in the global variable manager, where the parameter values may be numerical values, texts, and the like.
As shown at 508 in fig. 5, some instruction blocks may have other instruction blocks or instruction block sequences nested in their instruction blocks, representing that this instruction may control the operation of the instruction or instruction sequence nested therein, and in such instruction block behavior tree node combinations of instruction blocks, the 508 child node operation management node is used to control the operation of the parent node of the instruction block node combination of the instruction block nested therein.
Fig. 6 is an example of a combination manner of instructing a parent node of the sequence behavior tree as a child node of the combination of nodes of the object behavior tree.
In the embodiment of the present application, a plurality of preset programmable objects may be programmed, and each programmable object generates a corresponding class programming object behavior tree node combination for managing execution of a corresponding instruction sequence.
In the embodiment of the application, an instruction sequence triggered by a plurality of different events can be edited for one object, and the parent node of the behavior tree node combination of the instruction sequence performs operation management through 603 functional nodes in the object behavior tree node combination.
As shown at 601 in fig. 6, the behavior tree node 601 is a behavior tree root node of the programmable object, and is used for managing the operation of the parameter branch parent node and the functional branch parent node.
As shown in 602 in fig. 6, the behavior tree node 602 is a parameter node, and is configured to obtain and record various parameter values related to the object instruction building block sequence, and add or update the parameter values as variables in the global variable manager, where the parameter values may be values, texts, and the like.
As shown at 603 in fig. 6, the behavior tree node 603 is a function node for managing the operation of the parent node of the instruction sequence.
As shown in 604 of FIG. 6, the method and system can trigger the execution of the instruction sequence by means of event, message, etc., which allows the user to edit multiple instruction building block sequences for the same object, and after these instruction sequences generate the behavior tree node combination, the parent node of the instruction building block sequence will be used as the branch of the functional node in the object behavior tree node combination.
FIG. 7 is an example of a visualization scenario showing the operation results of nodes of a behavior tree when the object operates the behavior tree.
As shown at 701 in fig. 7, a visualization scene is composed of graphic elements, and the representation form of the scene can be a 2-dimensional scene or a 3-dimensional scene, and which representation form is used depends on the development software and the representation requirements.
In the embodiment of the present application, the graphic object is not limited to a specific graphic, and may be any graphic such as a human figure, an animal figure, an environment, a particle, a plant, a machine, a building, a decoration, a UI interface, an icon, and the like. These graphics can be set into programmable objects and non-programmable objects.
As shown in 701 of fig. 7, a programmable graphic object, in this embodiment, an object programmed by using an instruction block is a graphic object, and all attributes of the graphic object can be used as parameters of the instruction block and parameters of nodes in a corresponding instruction block node combination. When the behavior tree is operated, the attributes of the graphic object execute the change of various attributes according to the operation result of the behavior tree nodes, so that the operation sequence and the function expressed by the instruction building block sequence are visually displayed in a visual mode.
In the embodiment of the present application, the attributes of the graphic object include, but are not limited to, size, coordinates, rotation value, material, model, brightness, color, display, animation, etc., and sub-attributes such as audio play and volume control may be given to the graphic object.
In the embodiment of the application, a plurality of programming objects can be simultaneously displayed in the visualization scene of the object operation behavior tree, and each programming object executes corresponding change in the visualization scene of the object operation behavior tree according to the operation sequence and the result of the behavior tree.
In the embodiment of the present application, in the example diagram of the visualization scene of the object operation behavior tree, non-programming object graphic elements and virtual objects may be displayed, and these graphic elements and virtual objects may be used as a background and for collision detection, etc.
FIG. 8 is an example of the operational flow of the visual programming method and system provided by the present invention.
As shown in fig. 8, in the present embodiment, the following flow is used to implement the editing of the instruction building block sequence to the final operation presentation.
801. And editing the programming object programming instruction by using the instruction building block through the instruction building block sequence editing module, thereby realizing the programming of the programming object.
802. And traversing the edited instruction building block sequence through the instruction building block sequence analysis module to generate the behavior tree node combination data.
Each instruction building block graphic object comprises instruction building block behavior tree node combination parameters, instruction functional parameters and the like; the arrangement nesting mode of the instruction building block sequence corresponds to a node branch forming mode in the object behavior tree, and node combination data and node data of the behavior tree can be generated by analyzing the instruction building block sequence.
803. And after the generation of the behavior tree node combination data is finished, operating an object behavior tree generation module to generate a behavior tree of the object from the behavior tree node combination data. And executing the function of the global variable manager to generate the corresponding global variable.
804. And executing the operation function of the behavior tree, simultaneously, visually expressing the programming instruction according to the result executed by the behavior tree node through the programmed graphic object in the behavior tree node operation result display module, and correspondingly changing the programmed graphic object according to the operation sequence and the function execution of the behavior tree node, thereby expressing the function and the execution sequence expressed by the instruction building block sequence in a visual form.
FIG. 9 illustrates an example of implementing the operation sequence and function expressed by an instruction building block sequence after the instruction building block sequence generates an object instruction building block sequence behavior tree node combination.
As shown in fig. 9, 901 is an example of a sequence of instruction blocks composed of a plurality of instruction blocks, where the instruction blocks include two combinations of top-bottom splicing and nesting. 902 is an example of a combination of instruction building block sequence behavior tree nodes generated by 901.
The function and operation sequence expressed by the instruction building block sequence are as follows:
firstly, an edited instruction building block sequence is assigned to an object A through an instruction building block a 101;
when the instruction building block runs, executing a102 instruction building block, and controlling the programming object A to rotate 90 degrees leftwards;
then, executing a circulation function of an instruction building block a103, operating the instruction building block a104 embedded in the circulation content area of the instruction building block a103 for 3 times, and executing and controlling the programming object to move the object A forward for 3 times in a unit;
and finally, executing the function of the instruction building block a105, and controlling the programming object A to rotate 90 degrees to the right.
The parent node combination correspondence relationship of the instruction block and the instruction block behavior tree node in fig. 9 is a101 corresponding to b101, a102 corresponding to b102, a103 corresponding to b103, a104 corresponding to b104, and a105 corresponding to b 105.
As shown in FIG. 9, the behavior tree node combinations in 902 illustrate the behavior tree node combinations generated by the 901 instruction building block sequence.
According to the operation rule of the behavior tree set in this embodiment, the operation flow is as follows:
and executing the step 1, wherein the operation object A instructs the parent node b101 of the building block sequence, and controls the operation of the parameter branch node b101a and the function node b101 c.
And executing the step 2, operating the parameter branch node b101a and controlling the operation of the parameter value node b101 b.
And step 3 is executed, the parameter value node b101b is operated, the instruction series operation parameters are obtained, and the corresponding global variables are updated.
And step 4 is executed, the parameter value section b101b returns the completion value to the object A instruction building block sequence parent node b101 step by step upwards, and the object A instruction building block sequence parent node b101 is executed.
And step 5 is executed, the functional node b101c is operated, the variable of the instruction sequence is obtained, and the node function is executed. And when the variable is not equal to the judgment value, the child node is not operated, and a failure value is returned upwards. The variable value is set to meet the judgment condition.
Step 6 is executed to operate the child node operation management node b101d to execute the management child node operation function.
And 7, executing the step 7, operating the father node b102 in the building block node combination instructed to rotate left by 90 degrees, and returning a completion value to the node operation management node b101d upwards step by step after the operation of the last node in the building block node combination instructed to rotate left by 90 degrees is completed.
And 8, executing the step 8, executing the loop 3 times to instruct the building block father node b103, and executing the operation functions of the management parameter branch node b103b and the loop function node b103 c.
Step 9 is executed to operate the parameter branch node b103a and perform the child node operation management function.
Step 10 is executed, the loop parameter value node b103b is operated, the loop time variable of the instruction block in the variable manager is updated, in this example, the variable value is 3, and the corresponding global variable is set to 3.
And step 11, returning the completion value to the loop 3-time instruction parent node b103 step by step upwards, executing the child node operation management function, and operating the loop function node b103 c.
And step 12 is executed, the loop function node b103c is operated, and after the loop variable is acquired for 3 times, the function of repeatedly operating the child nodes for 3 times is executed.
Step 13 is executed to operate the child node operation management node b103d, and the child node operation management function is executed.
The b103d node can manage a plurality of instruction building block father nodes, when a plurality of instruction building blocks are nested in the loop content part of the loop instruction building block, the father nodes of the instruction building blocks have b103d nodes for management operation, and the arrangement can realize the loop operation of a plurality of instructions nested in the father nodes according to the functions and the execution sequence expressed by the instruction sequence.
And step 14, executing the parent node b105 in the unit instruction building block node combination to move forward, and after the last operated child node in the unit instruction building block is moved forward to finish operation, gradually returning the success value to the child node operation management node b103c upwards.
And step 15 is executed, after the cycle function node receives the completion value returned by the child node, the cycle number is reduced by 1, and the child node is continuously operated. The value is repeated until the number of cycles remains 0.
And executing the step 16, and when the sub-node circulation value of the circulation function node b103c is 0, returning the completion value to the sub-node operation management node b101d step by step upwards.
In step 17, the child node operation management node b101d executes a child node operation management function, and starts operation of the parent node b104 in the node combination of the instruction rotated to the right by 90 degrees.
And step 18 is executed, the father node b104 in the instruction node combination which rotates rightwards by 90 degrees is operated, and when the operation of the last child node in the instruction node combination which rotates rightwards by 90 degrees is finished, the success value is returned upwards step by step to the object A instruction sequence building block father node b 101.
Therefore, according to the function and the execution sequence expressed by the instruction sequence, the operation of the behavior tree and the execution of the operation result of the programming object on the node are completely realized.
Through the above description of the embodiments, those skilled in the art will clearly understand that the embodiments of the present application can be implemented by software plus necessary general-purpose hardware, and certainly can also be implemented by special-purpose hardware including an application-specific integrated circuit, a special-purpose CPU, a special-purpose memory, special-purpose components, and the like. Generally, functions performed by computer programs can be easily implemented by corresponding hardware, and specific hardware structures for implementing the same functions may be various, such as analog circuits, digital circuits, or dedicated circuits. However, for the embodiments of the present application, the implementation of the software program is more preferable. Based on such understanding, the technical solutions of the embodiments of the present application may be embodied in the form of a software product, where the computer software product is stored in a readable storage medium, such as a floppy disk, a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random-Access Memory (RAM), a magnetic disk or an optical disk of a computer, and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) to execute the methods described in the embodiments of the present application.
In summary, the above embodiments are only used for illustrating the technical solutions of the embodiments of the present application, and not for limiting the same; although the embodiments of the present application have been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that: the technical solutions described in the above embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (11)
1. A method and system for visual programming, comprising:
is a visual programming oriented to graphic objects; replacing the text programming language with the instruction building block graph; forming an instruction building block sequence by editing the instruction building blocks, and expressing the operation sequence of the instructions by using the arrangement of the instruction building block sequence; converting the arrangement data of the instruction building block sequence and the parameters of the instruction building blocks into a behavior tree combined by the instruction building block nodes; the function and the operation sequence of the instruction building block sequence expression are realized through the operation of the behavior tree nodes; visually displaying the running sequence and the function expressed by the programming instruction sequence through the sequence and the result of the programming object executing the behavior tree node function in the visual scene;
the visual programming system comprises modules such as an instruction building block sequence editing module, an instruction building block sequence analyzing module, an object behavior tree generating module, a global variable module, a behavior tree operating module, a behavior tree node operating result visual display module and the like.
2. The method of claim 1, wherein the step of programming the graphic element comprises:
using the graphic elements as programming objects, wherein all attributes of the programming objects can be used as parameters of the instruction building blocks and parameters of nodes in corresponding instruction building block node combinations;
the attribute of the programming object can be modified through the operation result of the object behavior tree node, so that the operation sequence and the function expressed by the instruction building block sequence are displayed through the programming object; attributes of a graphical object include, but are not limited to, size, coordinates, rotation values, material, model, brightness, color, display, audio, volume, and the like.
3. The instruction building block sequence editing module of claim 1, comprising:
the instruction building blocks are characterized in that the expression form of the instruction building blocks is graphic symbols, and the instruction building blocks with different functions are expressed by different graphic symbols; each instruction building block comprises a group of instruction parameter data; each instruction building block corresponds to one instruction building block behavior tree node combination, and all the instruction building block behavior tree node combinations are contained in a unified instruction building block behavior tree node combination structure frame; each instruction building block is provided with a bonding point, the instruction building blocks can be bonded up and down and nested inside and outside through the bonding points, and an instruction building block sequence can be formed through bonding and nesting; some instruction blocks comprise editable parameters, and the editable parameters in the instruction blocks can be input and modified in the instruction block editing area;
the instruction building block type tag list is characterized in that the instruction building block type tag list is used for displaying classification tags of various instruction building blocks, and the instruction building block types displayed in the instruction building block list can be switched by selecting the tags;
the instruction building block list is used for displaying various instruction building blocks which can be used for editing the instruction building block sequence, and the instruction building blocks can be selected from the instruction building block list to generate editable copies to the instruction building block sequence editing area;
the instruction building block sequence editing area is characterized in that the instruction building blocks can be moved, bonded, nested, deleted and the like in the instruction building block editing area; a plurality of instruction building blocks can be edited into an instruction building block sequence in the instruction building block editing area;
an object instruction building block for specifying a visually programmed object; a plurality of different object instruction building blocks can be arranged on the same object according to different event triggering modes, and a plurality of instruction sequences can be edited on the same object by using the different object instruction building blocks.
4. The instruction building block sequence parsing module of claim 1, comprising:
the arrangement data of the instruction building blocks can be analyzed into behavior tree node combined structure data, the parameters of the instruction building blocks can be analyzed into parameter data of the behavior tree nodes of the instruction building blocks, and the variable data of the instruction building blocks can be analyzed into behavior tree global variable data.
5. The object behavior tree generation module of claim 1, comprising:
generating a behavior tree of an object according to the behavior tree node combination data output by the instruction building block sequence analysis module; and the behavior tree node parameter data output by the instruction building block sequence analysis function can be endowed to the corresponding behavior tree node.
6. The global variable module of claim 1, comprising:
generating a corresponding behavior tree global variable according to variable data output by the instruction building block sequence analysis module; nodes in the behavior tree can change and read global variables of the behavior tree.
7. The behavior tree execution module of claim 1, comprising:
for managing the operation of the behavior tree; starting running by a root node of the object; managing, by the parent node, operation of the child node; by setting the operation rule of the behavior tree and the functions of the nodes, the operation sequence and the operation result of the nodes in the behavior tree can simulate the functions and the operation sequence expressed by the instruction building block sequence.
8. The visualization programming run result presentation module of claim 1, comprising:
the behavior tree node operation result shows a view, the view content can be composed of various graphic element objects, and any plurality of programmable graphic element objects and any plurality of non-programmable graphic element objects can be contained;
the programmable graphic element object can be any form of graphic element such as 2D graphics, 3D graphics, video and the like; the functions of displacement, rotation, scaling, color change, display, disappearance and the like can be executed in the behavior tree node operation result display view according to the operation result of the behavior tree node.
9. The method of claim 2, wherein each instruction block has a corresponding instruction block behavior tree node composite structure contained in a unified instruction block behavior tree node composite structure frame, comprising:
setting corresponding behavior tree node combinations according to functions expressed by different instruction building blocks, and realizing the functions expressed by the instruction building blocks through the operation sequence and results of the nodes in the behavior tree node combinations; all the instruction building block behavior tree node combination modes of the instruction building blocks are contained in a frame in a unified instruction building block node combination structure; and the instruction building block uses part or all of the nodes in the unified instruction building block node combination structure to form corresponding instruction building block behavior tree node combinations according to the function difference.
10. The unified instruction building block behavior tree node building block structural framework of claim 2, comprising:
in the behavior tree node combined structure framework, instructing a building block father node to manage one or more parameter nodes and one or more function nodes;
parameter node management parameter value nodes and parent nodes of instruction building blocks as the instruction building blocks with parameter values nested in the instructions are combined;
the parameter value nodes can acquire and record various parameters and variables such as numerical parameters and variables, collision detection value parameters and variables, text parameter values and variables and the like through corresponding setting;
the function node can manage a child node execution sequence management node, and the child node execution sequence management node manages a parent node of an instruction building block behavior tree node combination of an instruction building block of a function module nested in an instruction;
the function nodes are used for executing the function of the instruction, and can be set to execute different types of functions, such as operation addition, subtraction, multiplication and division results, judgment of whether parameters are consistent with target values, object coordinate movement, object coordinate rotation, nested instruction loop execution and the like; the function node may set the dynamic variable returned from the calling parameter node in the global variable manager.
11. The behavior tree generation function of claim 3, comprising:
in the generated behavior tree, an object corresponds to an object behavior tree node combination, a father node of the object behavior tree node combination is a root node of the object behavior tree, and the father node of the object behavior tree manages a parameter node and a function node;
the parameter node is used for managing parameters and variables related to the object instruction sequence;
the function node is used for managing the operation of the father node of the instruction sequence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010273369.5A CN113515272A (en) | 2020-04-09 | 2020-04-09 | Visual programming method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010273369.5A CN113515272A (en) | 2020-04-09 | 2020-04-09 | Visual programming method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113515272A true CN113515272A (en) | 2021-10-19 |
Family
ID=78061005
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010273369.5A Pending CN113515272A (en) | 2020-04-09 | 2020-04-09 | Visual programming method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113515272A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115543290A (en) * | 2022-10-12 | 2022-12-30 | 睿珀智能科技有限公司 | Visual programming method and system |
| WO2023159571A1 (en) * | 2022-02-28 | 2023-08-31 | 西门子（中国）有限公司 | Workflow creation method and apparatus, and platform and computer-readable medium |
| WO2023164841A1 (en) * | 2022-03-02 | 2023-09-07 | 西门子股份公司 | Workflow execution method and apparatus, and storage medium and program product |
| CN117389569A (en) * | 2023-10-26 | 2024-01-12 | 重庆猪哥亮科技有限责任公司 | Program interpretation execution method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106155658A (en) * | 2015-04-08 | 2016-11-23 | 广州四三九九信息科技有限公司 | The behavior tree editing machine realized based on U3D Plugin Mechanism |
| EP3214510A1 (en) * | 2016-03-03 | 2017-09-06 | Magazino GmbH | Controlling process of robots having a behavior tree architecture |
| CN109634592A (en) * | 2018-12-29 | 2019-04-16 | 深圳点猫科技有限公司 | Graphical Python programming exchange method, system and electronic equipment |
| CN110147883A (en) * | 2019-05-28 | 2019-08-20 | 航天科工***仿真科技（北京）有限公司 | Training method, device, equipment and the storage medium of model for emulation of fighting |
| CN110377280A (en) * | 2019-07-29 | 2019-10-25 | 郑州幻视科技有限公司 | A kind of visualization system for teaching programming |
-
2020
- 2020-04-09 CN CN202010273369.5A patent/CN113515272A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106155658A (en) * | 2015-04-08 | 2016-11-23 | 广州四三九九信息科技有限公司 | The behavior tree editing machine realized based on U3D Plugin Mechanism |
| EP3214510A1 (en) * | 2016-03-03 | 2017-09-06 | Magazino GmbH | Controlling process of robots having a behavior tree architecture |
| CN109634592A (en) * | 2018-12-29 | 2019-04-16 | 深圳点猫科技有限公司 | Graphical Python programming exchange method, system and electronic equipment |
| CN110147883A (en) * | 2019-05-28 | 2019-08-20 | 航天科工***仿真科技（北京）有限公司 | Training method, device, equipment and the storage medium of model for emulation of fighting |
| CN110377280A (en) * | 2019-07-29 | 2019-10-25 | 郑州幻视科技有限公司 | A kind of visualization system for teaching programming |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023159571A1 (en) * | 2022-02-28 | 2023-08-31 | 西门子（中国）有限公司 | Workflow creation method and apparatus, and platform and computer-readable medium |
| WO2023164841A1 (en) * | 2022-03-02 | 2023-09-07 | 西门子股份公司 | Workflow execution method and apparatus, and storage medium and program product |
| CN115543290A (en) * | 2022-10-12 | 2022-12-30 | 睿珀智能科技有限公司 | Visual programming method and system |
| CN115543290B (en) * | 2022-10-12 | 2024-04-19 | 睿珀智能科技有限公司 | Visual programming method and system |
| CN117389569A (en) * | 2023-10-26 | 2024-01-12 | 重庆猪哥亮科技有限责任公司 | Program interpretation execution method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113515272A (en) | Visual programming method and system | |
| US6011562A (en) | Method and system employing an NLE to create and modify 3D animations by mixing and compositing animation data | |
| TWI413017B (en) | Method and computer system for blended object attribute keyframing model | |
| JP2006528381A (en) | Virtual environment controller | |
| KR101323148B1 (en) | Ucr contents editor device | |
| CN105264474A (en) | Machine vision system program editing environment including operating context aware copy and paste feature | |
| CN106030461A (en) | Platform for developing immersive reality-virtuality continuum-based environment and methods thereof | |
| CN109191590B (en) | Processing system and processing method for manufacturing virtual reality application | |
| Satheesh | Unreal Engine 4 Game Development Essentials | |
| Singh et al. | Game development using unity game engine | |
| US20110209117A1 (en) | Methods and systems related to creation of interactive multimdedia applications | |
| Skorupski et al. | Novice-friendly authoring of plan-based interactive storyboards | |
| US11625900B2 (en) | Broker for instancing | |
| US20160171738A1 (en) | Heirarchy-based character rigging | |
| CN116342763A (en) | Intelligent multi-mode animation creation system and creation method | |
| Travers | Recursive interfaces for reactive objects | |
| EP4097607B1 (en) | Applying non-destructive edits to nested instances for efficient rendering | |
| Hashim et al. | Drawing Transforms: A Unifying Interaction Primitive to Procedurally Manipulate Graphics across Style, Space, and Time | |
| Zhang et al. | PoseVEC: Authoring Adaptive Pose-aware Effects using Visual Programming and Demonstrations | |
| Hopkins et al. | The rube framework for personalized 3-d software visualization | |
| CN109326005B (en) | Editing system and editing method for virtual reality application software actions | |
| Yuan et al. | Virtual reality platform based on open sourced graphics toolkit openscenegraph | |
| KR100817506B1 (en) | Method for producing intellectual contents | |
| Carra et al. | gMotion: A spatio-temporal grammar for the procedural generation of motion graphics | |
| Evers et al. | Building artificial memory to autonomous agents using dynamic and hierarchical finite state machine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |