CN114064010A - Front-end code generation method, device, system and storage medium - Google Patents

Front-end code generation method, device, system and storage medium Download PDF

Info

Publication number
CN114064010A
CN114064010A CN202111390376.4A CN202111390376A CN114064010A CN 114064010 A CN114064010 A CN 114064010A CN 202111390376 A CN202111390376 A CN 202111390376A CN 114064010 A CN114064010 A CN 114064010A
Authority
CN
China
Prior art keywords
user interface
graphical user
dsl
code
visual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111390376.4A
Other languages
Chinese (zh)
Inventor
杨瑞光
罗涛
施佳子
于海燕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202111390376.4A priority Critical patent/CN114064010A/en
Publication of CN114064010A publication Critical patent/CN114064010A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/30—Creation or generation of source code
    • G06F8/31—Programming languages or programming paradigms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/30—Creation or generation of source code
    • G06F8/35—Creation or generation of source code model driven
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/30—Creation or generation of source code
    • G06F8/38—Creation or generation of source code for implementing user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformation of program code
    • G06F8/41—Compilation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The present disclosure provides a method, an apparatus, a system and a storage medium for generating a front-end code, which are applied to a computer technology in the financial field, and include: the method comprises the steps of obtaining a visual draft of a graphical user interface at the current moment, a visual draft of the graphical user interface at the previous moment at the current moment and a DSL code corresponding to the visual draft of the graphical user interface at the previous moment, taking the visual draft of the graphical user interface at the current moment and the DSL code corresponding to the visual draft of the graphical user interface at the previous moment as input, taking the visual draft of the graphical user interface at the current moment as a target label to establish a DSL code prediction model, inputting the visual draft of the graphical user interface at the current moment into the DSL code prediction model to obtain the DSL code corresponding to the graphical user interface at the current moment, generating a front end code corresponding to the graphical user interface at the current moment according to the DSL code, and realizing automatic generation of the front end code.

Description

Front-end code generation method, device, system and storage medium
Technical Field
The present disclosure relates to computer technologies in the field of finance, and more particularly, to a front-end code generation method, apparatus, system, and storage medium.
Background
In the existing front-end development process, a front-end development engineer needs to convert a UI given by a designer into corresponding front-end code. Because the user interface is very complicated, a front-end development engineer not only needs to write a UI interface, but also needs to write corresponding interactive logic, so that the corresponding front-end code amount is huge. And a large number of repeated label elements exist in the work of writing the UI, so that the front-end development work is complicated and the work repeatability is large. In addition, the customized development website and the mobile terminal APP interface have different computer languages, and thus, there is a cumbersome and repetitive work in developing software for a plurality of platforms.
Disclosure of Invention
In view of the above, the present disclosure provides a front-end code generation method, apparatus, system and storage medium.
One aspect of the present disclosure provides a front-end code generation method, including:
acquiring a visual draft of a graphical user interface at the current moment, a visual draft of the graphical user interface at the previous moment at the current moment and a DSL code of a domain specific language corresponding to the visual draft of the graphical user interface at the previous moment;
based on a long-term and short-term memory network, taking the visual draft of the graphical user interface at the previous moment and a DSL code corresponding to the visual draft of the graphical user interface at the previous moment as input, taking the visual draft of the graphical user interface at the current moment as a target label to establish a DSL code prediction model, wherein the DSL code prediction model is used for predicting the DSL code corresponding to the graphical user interface at the current moment;
inputting the visual draft of the graphical user interface at the current moment into the DSL code prediction model to obtain a DSL code corresponding to the graphical user interface at the current moment;
and generating a front-end code corresponding to the graphical user interface at the current moment according to the DSL code.
In the embodiment of the present disclosure, the graphical user interface elements in the visual draft of the graphical user interface at the previous moment are extracted;
identifying the graphical user interface element type;
determining code characteristics of a DSL code corresponding to the graphical user interface element type;
and generating the DSL code corresponding to the visual draft of the graphical user interface at the previous moment according to the code characteristics of the DSL code.
In an embodiment of the present disclosure, the extracting graphical user interface elements in the visual draft of the graphical user interface at the previous time includes:
and extracting image features in the visual manuscript through a convolutional neural network to obtain a feature vector with fixed dimensionality.
In an embodiment of the present disclosure, the identifying the gui element type includes:
inputting the feature vector into a preset classifier, and outputting the classification to which the visual manuscript belongs;
and marking the position of the object contained in the visual manuscript and the category of the object.
In an embodiment of the present disclosure, the determining a code feature of the DSL code corresponding to the graphical user interface element type includes:
acquiring the corresponding relation between the graphical user interface element type and the code characteristics of the DSL code;
and determining the code characteristics of the DSL code corresponding to the graphical user interface element type according to the corresponding relation.
In the disclosed embodiment, when training the DSL code prediction model, 48 steps are used to develop the long-short term memory network.
Another aspect of the present disclosure provides a front-end code generating apparatus including:
the acquisition module is used for acquiring a visual draft of a graphical user interface at the current moment, a visual draft of the graphical user interface at the previous moment at the current moment and a DSL code of a domain specific language corresponding to the visual draft of the graphical user interface at the previous moment;
the establishing module is used for taking the visual draft of the graphical user interface at the previous moment and the DSL code corresponding to the visual draft of the graphical user interface at the previous moment as input, taking the visual draft of the graphical user interface at the current moment as a target label to establish a DSL code prediction model, and the DSL code prediction model is used for predicting the DSL code corresponding to the graphical user interface at the current moment;
an input module, configured to input the visual draft of the graphical user interface at the current time into the DSL code prediction model, so as to obtain a DSL code corresponding to the graphical user interface at the current time;
and the generating module is used for generating a front-end code corresponding to the graphical user interface at the current moment according to the DSL code.
In the embodiment of the present disclosure, the method further includes:
the extraction module is used for extracting the graphical user interface elements in the visual draft of the graphical user interface at the previous moment;
an identification module for identifying the graphical user interface element type;
a determining module, configured to determine a code feature of the DSL code corresponding to the graphical user interface element type;
and the DSL code generating module is used for generating the DSL code corresponding to the visual draft of the graphical user interface at the previous moment according to the code characteristics of the DSL code.
In the embodiment of the present disclosure, the extraction module is specifically configured to extract the image features in the visual manuscript through a convolutional neural network to obtain a feature vector with a fixed dimension.
In an embodiment of the present disclosure, the identification module includes:
the vector input module is used for inputting the feature vectors into a preset classifier and outputting the classification to which the visual manuscript belongs;
and the marking module is used for marking the position of the object contained in the visual manuscript and the category of the object.
In an embodiment of the present disclosure, the determining module includes:
a relationship obtaining module, configured to obtain a correspondence between the gui element type and a code feature of the DSL code;
and the characteristic determining module is used for determining the code characteristics of the DSL codes corresponding to the graphical user interface element types according to the corresponding relation.
In the disclosed embodiment, when training the DSL code prediction model, 48 steps are used to develop the long-short term memory network.
Another aspect of the present disclosure provides a computer system comprising:
one or more processors;
a memory for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method as described above.
Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions for implementing the method as described above when executed.
Another aspect of the disclosure provides a computer program comprising computer executable instructions for implementing the method as described above when executed.
According to the embodiment of the disclosure, a visual draft of a graphical user interface at a current moment, a visual draft of the graphical user interface at a previous moment at the current moment and a DSL code corresponding to the visual draft of the graphical user interface at the previous moment are acquired, based on a long-term and short-term memory network, the visual draft of the graphical user interface at the current moment and the DSL code corresponding to the visual draft of the graphical user interface at the current moment are used as input, the visual draft of the graphical user interface at the current moment is used as a target label to establish a DSL code prediction model, the visual draft of the graphical user interface at the current moment is input into the DSL code prediction model to obtain the DSL code corresponding to the graphical user interface at the current moment, and a front-end code corresponding to the graphical user interface at the current moment is generated according to the DSL code, so that automatic generation of the front-end code is realized.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:
FIG. 1 schematically shows a flow diagram of a front-end code generation method according to an embodiment of the present disclosure;
figure 2 schematically illustrates a flow diagram of DSL code corresponding to a visual draft of a graphical user interface at a previous time according to an embodiment of the present disclosure;
figure 3 schematically illustrates a flow chart of determining code characteristics of DSL codes corresponding to graphical user interface element types according to an embodiment of the present disclosure;
FIG. 4 schematically shows a block diagram of a front-end code generation apparatus according to an embodiment of the present disclosure;
FIG. 5 schematically shows a block diagram of a front-end code generation apparatus according to an embodiment of the present disclosure; and
FIG. 6 schematically shows a block diagram of a computer system according to an embodiment of the disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a convention analogous to "A, B or at least one of C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B or C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
The embodiment of the disclosure provides a front-end code generation method, which includes acquiring a visual draft of a graphical user interface at the current moment, a visual draft of the graphical user interface at the previous moment at the current moment, and a DSL code of a domain specific language corresponding to the visual draft of the graphical user interface at the previous moment; based on a long-term and short-term memory network, taking the visual draft of the graphical user interface at the previous moment and a DSL code corresponding to the visual draft of the graphical user interface at the previous moment as input, taking the visual draft of the graphical user interface at the current moment as a target label to establish a DSL code prediction model, wherein the DSL code prediction model is used for predicting the DSL code corresponding to the graphical user interface at the current moment; inputting the visual draft of the graphical user interface at the current moment into the DSL code prediction model to obtain a DSL code corresponding to the graphical user interface at the current moment; and generating a front-end code corresponding to the graphical user interface at the current moment according to the DSL code.
In the disclosed embodiments, the visual drafts of the graphical user interface may be originally stored in the terminal device, or stored on an external storage device and may be imported into the terminal device. Then, the terminal device may locally execute the front-end code generation method provided by the embodiment of the present disclosure, or send the visual draft of the graphical user interface to another terminal device, a server, or a server cluster, and execute the front-end code generation method provided by the embodiment of the present disclosure by another terminal device, a server, or a server cluster that receives the visual draft of the graphical user interface.
The front-end code generation method provided by the embodiment of the disclosure can be applied to GUI development of various devices, such as a mobile terminal device, a personal computer, a game terminal device, and the like, and the disclosure does not limit this.
Fig. 1 schematically shows a flow chart of a front-end code generation method according to an embodiment of the present disclosure.
As shown in fig. 1, the method includes operations S101 to S104.
In operation S101, a visual draft of a current-time graphical user interface, a visual draft of a previous-time graphical user interface of the current time, and a domain-specific language DSL code corresponding to the visual draft of the previous-time graphical user interface are obtained.
In operation S102, based on the long-term and short-term memory network, the DSL code corresponding to the visual draft of the graphical user interface at the previous time and the visual draft of the graphical user interface at the previous time are used as input, and the DSL code prediction model is established with the visual draft of the graphical user interface at the current time as a target label.
In operation S103, the visual draft of the current-time gui is input into the DSL code prediction model, so as to obtain a DSL code corresponding to the current-time gui.
In operation S104, a front-end code corresponding to the current-time graphical user interface is generated according to the DSL code.
In this embodiment, the graphical user interface is a form of user interface that allows a user to interact with the electronic device not through a text interface, typed command tags, or words, but through icons and visual indicators (e.g., secondary symbol prompts). The visual draft of the graphical user interface may be obtained by capturing a screen of the electronic device, or by capturing a video, which is not limited by the present disclosure.
In this embodiment, the DSL code prediction model is configured to predict a DSL code corresponding to the graphical user interface at the current time, and input the visual draft of the graphical user interface at the current time into the DSL code prediction model, so as to obtain the DSL code corresponding to the graphical user interface at the current time.
In this embodiment, generating the front-end code corresponding to the graphical user interface at the current time according to the DSL code may be compiling the DSL code into an actually executable front-end code, so as to generate the front-end code.
According to the disclosed embodiment, a visual draft of a graphical user interface at the current moment, a visual draft of the graphical user interface at the previous moment at the current moment and a DSL code corresponding to the visual draft of the graphical user interface at the previous moment are obtained, the visual draft of the graphical user interface at the current moment and the DSL code corresponding to the visual draft of the graphical user interface at the current moment are used as input, the visual draft of the graphical user interface at the current moment is used as a target label to establish a DSL code prediction model, the visual draft of the graphical user interface at the current moment is input into the DSL code prediction model to obtain the DSL code corresponding to the graphical user interface at the current moment, a front-end code corresponding to the graphical user interface at the current moment is generated according to the DSL code, and automatic generation of the front-end code is realized through the established DSL code prediction model, therefore, the workload of a front-end engineer is reduced, the software development efficiency is improved, meanwhile, a front-end code is automatically generated, a unified standard can be provided for design and development, and information loss in the process of demand, design, development and delivery is reduced.
Fig. 2 schematically illustrates a flow diagram of DSL code corresponding to a visual rendition of a graphical user interface at a previous time in accordance with an embodiment of the disclosure.
As shown in fig. 2, the method includes operations S201 to S204.
In operation S201, a graphical user interface element in the visual draft of the graphical user interface at the previous time is extracted.
In operation S202, the graphical user interface element type is identified.
In operation S203, a code characteristic of the DSL code corresponding to the graphical user interface element type is determined.
In operation S204, a DSL code corresponding to the visual draft of the graphical user interface at the previous time is generated according to the code characteristics of the DSL code.
In this embodiment of the present disclosure, extracting the graphical user interface element in the visual draft of the graphical user interface at the previous time may be extracting an image feature in the visual draft through a convolutional neural network, so as to obtain a feature vector of a fixed dimension.
In an embodiment of the disclosure, identifying the graphical user interface element type includes: inputting the feature vector into a preset classifier, outputting the classification to which the visual draft belongs, and marking the position of the object contained in the visual draft and the category to which the object belongs. The objects included in the visual drafts are, for example, buttons, tabs, controls, and the like in the visual drafts.
In the disclosure, a computer visual target detection algorithm may be adopted to perform feature extraction on a visual draft of a graphical user interface by using a convolutional neural network, obtain a feature vector of a fixed dimension, connect a classifier behind the feature vector, accurately output a classification to which the visual draft belongs and mark a position of an object included in the visual draft and a category to which the object belongs, and extract an input picture into a vector of a fixed dimension, so that the vector of a fixed dimension includes important information of the whole picture.
Fig. 3 schematically illustrates a flow chart for determining code characteristics of DSL codes corresponding to graphical user interface element types according to an embodiment of the present disclosure.
As shown in fig. 3, the method includes operations S301 to S302.
In operation S301, a correspondence between the gui element type and the code characteristic of the DSL code is obtained.
In operation S302, a code characteristic of the DSL code corresponding to the gui element type is determined according to the correspondence.
In the present disclosure, in a first step, the DSL language is designed to describe the graphical user interface. DSL is pseudo-code of the front-end code that abstractly defines the structural semantics and style of the front-end code. In the machine learning process, only the layout of the graphical user interface, different graphical controls and the relationship between the graphical controls need to be concerned, and the text value of the label control is ignored, so that the search space and the total vocabulary of the DSL can be reduced. Secondly, the DSL program is coded into numbers that can be understood by a machine, and the DSL file is output as a language model and finally presented in the form of pseudo codes, but the algorithm and the model do not recognize character strings, so that a number needs to be defined for each DSL element in advance, the language model outputs the number, and finally the number is mapped back to the DSL element name mapped by the language model, and finally the DSL file is generated. Thirdly, extracting the characteristics of the input DSL codes through a language characteristic extractor, and extracting the DSL codes into a vector with fixed dimensionality so as to enable the vector to contain important information in the whole DSL codes.
In the disclosed embodiment, 48 steps are used to develop the long-short term memory network when training the DSL code prediction model. In the training phase, the length T of the DSL code used for training is important for modeling long-term correlation, because the preceding DSL code may affect the subsequent DSL code, for example, a brace appears in front of the DSL code, and a reversed brace is necessarily corresponding to the preceding DSL code, so that the length T needs to be set to be large to make the field of view of each feature extraction longer, and the extracted features can contain the most semantics. The trained DSL code input file is divided by a sliding window with the size of 48, namely a long-term and short-term memory network is expanded by 48 steps, so that the phenomenon that the performance of a model is reduced because useless codes before are too early cause influence on the current prediction because the length T is set to be too large or too small can be avoided.
Fig. 4 schematically shows a block diagram of a front-end code generation apparatus according to an embodiment of the present disclosure.
As shown in fig. 4, the front-end code generating apparatus 400 includes an obtaining module 410, a building module 420, an inputting module 430, and a generating module 440.
The obtaining module 410 is configured to obtain a visual draft of the graphical user interface at the current time, a visual draft of the graphical user interface at a time before the current time, and a DSL code in the domain-specific language corresponding to the visual draft of the graphical user interface at the time before the current time.
A building module 420, configured to use, based on the long-term and short-term memory network, the visual draft of the graphical user interface at the previous time and the DSL code corresponding to the visual draft of the graphical user interface at the previous time as input, use the visual draft of the graphical user interface at the current time as a target label, and build a DSL code prediction model, where the DSL code prediction model is used to predict the DSL code corresponding to the graphical user interface at the current time.
An input module 430, configured to input the visual draft of the current-time graphical user interface into the DSL code prediction model, so as to obtain a DSL code corresponding to the current-time graphical user interface.
A generating module 440, configured to generate a front-end code corresponding to the current-time graphical user interface according to the DSL code.
Fig. 5 schematically shows a block diagram of a front-end code generation apparatus according to an embodiment of the present disclosure.
As shown in fig. 5, the front-end code generating apparatus 400 includes, in addition to the obtaining module 410, the establishing module 420, the inputting module 430 and the generating module 440 shown in fig. 4: an extraction module 510, an identification module 520, a determination module 530, and a DSL code generation module 540.
An extracting module 510, configured to extract a graphical user interface element in the visual draft of the graphical user interface at the previous time.
An identifying module 520 for identifying the graphical user interface element type.
A determining module 530, configured to determine a code characteristic of the DSL code corresponding to the graphical user interface element type.
The DSL code generating module 540 is configured to generate, according to the code feature of the DSL code, a DSL code corresponding to the visual draft of the graphical user interface at the previous time.
In the disclosure, a computer visual target detection algorithm may be adopted to perform feature extraction on a visual draft of a graphical user interface by using a convolutional neural network, obtain a feature vector of a fixed dimension, connect a classifier behind the feature vector, accurately output a classification to which the visual draft belongs and mark a position of an object included in the visual draft and a category to which the object belongs, and extract an input picture into a vector of a fixed dimension, so that the vector of a fixed dimension includes important information of the whole picture.
In an embodiment of the present disclosure, the extracting module 510 is specifically configured to extract image features in the visual manuscript through a convolutional neural network, so as to obtain a feature vector with a fixed dimension.
In an embodiment of the present disclosure, the identifying module 520 includes: the vector input module is used for inputting the feature vector into a preset classifier and outputting the classification to which the visual manuscript belongs; and the marking module is used for marking the position of the object contained in the visual manuscript and the category of the object.
In an embodiment of the present disclosure, the determining module 530 includes: the relation acquisition module is used for acquiring the corresponding relation between the graphical user interface element type and the code characteristics of the DSL code; and the characteristic determining module is used for determining the code characteristics of the DSL codes corresponding to the graphical user interface element types according to the corresponding relation.
In the present disclosure, in a first step, the DSL language is designed to describe the graphical user interface. DSL is pseudo-code of the front-end code that abstractly defines the structural semantics and style of the front-end code. In the machine learning process, only the layout of the graphical user interface, different graphical controls and the relationship between the graphical controls need to be concerned, and the text value of the label control is ignored, so that the search space and the total vocabulary of the DSL can be reduced. Secondly, the DSL program is coded into numbers that can be understood by a machine, and the DSL file is output as a language model and finally presented in the form of pseudo codes, but the algorithm and the model do not recognize character strings, so that a number needs to be defined for each DSL element in advance, the language model outputs the number, and finally the number is mapped back to the DSL element name mapped by the language model, and finally the DSL file is generated. Thirdly, extracting the characteristics of the input DSL codes through a language characteristic extractor, and extracting the DSL codes into a vector with fixed dimensionality so as to enable the vector to contain important information in the whole DSL codes.
In one embodiment of the present disclosure, 48 steps are used to develop the long-short term memory network when training the DSL code prediction model.
In the disclosed embodiment, 48 steps are used to develop the long-short term memory network when training the DSL code prediction model. In the training phase, the length T of the DSL code used for training is important for modeling long-term correlation, because the preceding DSL code may affect the subsequent DSL code, for example, a brace appears in front of the DSL code, and a reversed brace is necessarily corresponding to the preceding DSL code, so that the length T needs to be set to be large to make the field of view of each feature extraction longer, and the extracted features can contain the most semantics. However, setting the length T too large will cause too early useless codes to affect the current prediction, and the model performance will be degraded. Thus, the trained DSL code input file is partitioned by a sliding window of size 48, i.e. a 48-step expansion of the long-short term memory network.
According to the disclosed embodiment, a visual draft of a graphical user interface at the current moment, a visual draft of the graphical user interface at the previous moment at the current moment and a DSL code corresponding to the visual draft of the graphical user interface at the previous moment are obtained, the visual draft of the graphical user interface at the current moment and the DSL code corresponding to the visual draft of the graphical user interface at the current moment are used as input, the visual draft of the graphical user interface at the current moment is used as a target label to establish a DSL code prediction model, the visual draft of the graphical user interface at the current moment is input into the DSL code prediction model to obtain the DSL code corresponding to the graphical user interface at the current moment, a front-end code corresponding to the graphical user interface at the current moment is generated according to the DSL code, and automatic generation of the front-end code is realized through the established DSL code prediction model, therefore, the workload of a front-end engineer is reduced, the software development efficiency is improved, meanwhile, a front-end code is automatically generated, a unified standard can be provided for design and development, and information loss in the process of demand, design, development and delivery is reduced.
Any number of modules, sub-modules, units, sub-units, or at least part of the functionality of any number thereof according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be implemented by being split into a plurality of modules. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in any other reasonable manner of hardware or firmware by integrating or packaging a circuit, or in any one of or a suitable combination of software, hardware, and firmware implementations. Alternatively, one or more of the modules, sub-modules, units, sub-units according to embodiments of the disclosure may be at least partially implemented as a computer program module, which when executed may perform the corresponding functions.
For example, any plurality of the obtaining module 410, the establishing module 420, the inputting module 430 and the generating module 440 may be combined and implemented in one module/unit/sub-unit, or any one of the modules/units/sub-units may be split into a plurality of modules/units/sub-units. Alternatively, at least part of the functionality of one or more of these modules/units/sub-units may be combined with at least part of the functionality of other modules/units/sub-units and implemented in one module/unit/sub-unit. According to an embodiment of the present disclosure, at least one of the obtaining module 410, the establishing module 420, the inputting module 430 and the generating module 440 may be implemented at least partially as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or may be implemented by any one of three implementations of software, hardware and firmware, or any suitable combination of any of the three. Alternatively, at least one of the obtaining module 410, the establishing module 420, the inputting module 430 and the generating module 440 may be at least partially implemented as a computer program module, which when executed may perform the respective functions.
It should be noted that the front-end code generation apparatus part in the embodiment of the present disclosure corresponds to the front-end code generation method part in the embodiment of the present disclosure, and the description of the front-end code generation apparatus part specifically refers to the front-end code generation method part, which is not described herein again.
Fig. 6 schematically shows a block diagram of a computer system suitable for implementing the above described method according to an embodiment of the present disclosure. The computer system illustrated in FIG. 6 is only one example and should not impose any limitations on the scope of use or functionality of embodiments of the disclosure.
As shown in fig. 6, a computer system 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. Processor 601 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 601 may also include onboard memory for caching purposes. Processor 601 may include a single processing unit or multiple processing units for performing different actions of a method flow according to embodiments of the disclosure.
In the RAM 603, various programs and data necessary for the operation of the system 600 are stored. The processor 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. The processor 601 performs various operations of the method flows according to the embodiments of the present disclosure by executing programs in the ROM 602 and/or RAM 603. It is to be noted that the programs may also be stored in one or more memories other than the ROM 602 and RAM 603. The processor 601 may also perform various operations of the method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the present disclosure, system 600 may also include an input/output (I/O) interface 605, input/output (I/O) interface 605 also connected to bus 604. The system 600 may also include one or more of the following components connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
According to embodiments of the present disclosure, method flows according to embodiments of the present disclosure may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program, when executed by the processor 601, performs the above-described functions defined in the system of the embodiments of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples may include, but are not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM 602 and/or RAM 603 described above and/or one or more memories other than the ROM 602 and RAM 603.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.
The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.

Claims (11)

1. A front-end code generation method, comprising:
acquiring a visual draft of a graphical user interface at the current moment, a visual draft of the graphical user interface at the previous moment at the current moment and a DSL code of a domain specific language corresponding to the visual draft of the graphical user interface at the previous moment;
based on a long-term and short-term memory network, taking the visual draft of the graphical user interface at the previous moment and a DSL code corresponding to the visual draft of the graphical user interface at the previous moment as input, taking the visual draft of the graphical user interface at the current moment as a target label to establish a DSL code prediction model, wherein the DSL code prediction model is used for predicting the DSL code corresponding to the graphical user interface at the current moment;
inputting the visual draft of the graphical user interface at the current moment into the DSL code prediction model to obtain a DSL code corresponding to the graphical user interface at the current moment;
and generating a front-end code corresponding to the graphical user interface at the current moment according to the DSL code.
2. The method of claim 1, wherein:
extracting graphical user interface elements in the visual draft of the graphical user interface at the previous moment;
identifying the graphical user interface element type;
determining code characteristics of a DSL code corresponding to the graphical user interface element type;
and generating the DSL code corresponding to the visual draft of the graphical user interface at the previous moment according to the code characteristics of the DSL code.
3. The method of claim 2, wherein the extracting graphical user interface elements in the visual rendition of the graphical user interface at the previous time comprises:
and extracting image features in the visual manuscript through a convolutional neural network to obtain a feature vector with fixed dimensionality.
4. The method of claim 3, wherein the identifying the graphical user interface element type comprises:
inputting the feature vector into a preset classifier, and outputting the classification to which the visual manuscript belongs;
and marking the position of the object contained in the visual manuscript and the category of the object.
5. The method of claim 2, the determining code characteristics of the DSL code corresponding to the graphical user interface element type comprising:
acquiring the corresponding relation between the graphical user interface element type and the code characteristics of the DSL code;
and determining the code characteristics of the DSL code corresponding to the graphical user interface element type according to the corresponding relation.
6. The method of claim 1 wherein the long-short term memory network is expanded in 48 steps when training the DSL code prediction model.
7. A front-end code generation apparatus, comprising:
the acquisition module is used for acquiring a visual draft of a graphical user interface at the current moment, a visual draft of the graphical user interface at the previous moment at the current moment and a DSL code of a domain specific language corresponding to the visual draft of the graphical user interface at the previous moment;
the establishing module is used for taking the visual draft of the graphical user interface at the previous moment and the DSL code corresponding to the visual draft of the graphical user interface at the previous moment as input, taking the visual draft of the graphical user interface at the current moment as a target label to establish a DSL code prediction model, and the DSL code prediction model is used for predicting the DSL code corresponding to the graphical user interface at the current moment;
an input module, configured to input the visual draft of the graphical user interface at the current time into the DSL code prediction model, so as to obtain a DSL code corresponding to the graphical user interface at the current time;
and the generating module is used for generating a front-end code corresponding to the graphical user interface at the current moment according to the DSL code.
8. The apparatus of claim 7, wherein:
the extraction module is used for extracting the graphical user interface elements in the visual draft of the graphical user interface at the previous moment;
an identification module for identifying the graphical user interface element type;
a determining module, configured to determine a code feature of the DSL code corresponding to the graphical user interface element type;
and the DSL code generating module is used for generating the DSL code corresponding to the visual draft of the graphical user interface at the previous moment according to the code characteristics of the DSL code.
9. The apparatus of claim 7, wherein the long-short term memory network is deployed in 48 steps when training the DSL code prediction model.
10. A computer system, comprising:
one or more processors;
a memory for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-6.
11. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to carry out the method of any one of claims 1 to 6.
CN202111390376.4A 2021-11-22 2021-11-22 Front-end code generation method, device, system and storage medium Pending CN114064010A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111390376.4A CN114064010A (en) 2021-11-22 2021-11-22 Front-end code generation method, device, system and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111390376.4A CN114064010A (en) 2021-11-22 2021-11-22 Front-end code generation method, device, system and storage medium

Publications (1)

Publication Number Publication Date
CN114064010A true CN114064010A (en) 2022-02-18

Family

ID=80279119

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111390376.4A Pending CN114064010A (en) 2021-11-22 2021-11-22 Front-end code generation method, device, system and storage medium

Country Status (1)

Country Link
CN (1) CN114064010A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115543324A (en) * 2022-11-30 2022-12-30 网易（杭州）网络有限公司 Method, device, equipment and medium for generating interactive interface

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115543324A (en) * 2022-11-30 2022-12-30 网易（杭州）网络有限公司 Method, device, equipment and medium for generating interactive interface
CN115543324B (en) * 2022-11-30 2023-03-10 网易（杭州）网络有限公司 Method, device, equipment and medium for generating interactive interface

Similar Documents

Publication Publication Date Title
CN109002510B (en) Dialogue processing method, device, equipment and medium
US9619735B1 (en) Pure convolutional neural network localization
US9766868B2 (en) Dynamic source code generation
US9619209B1 (en) Dynamic source code generation
TW202020691A (en) Feature word determination method and device and server
KR20220122566A (en) Text recognition model training method, text recognition method, and apparatus
CN107729246B (en) Auxiliary test method, device and equipment for target application and storage medium
CN109947924B (en) Dialogue system training data construction method and device, electronic equipment and storage medium
CN111291882A (en) Model conversion method, device, equipment and computer storage medium
CN112541332B (en) Form information extraction method and device, electronic equipment and storage medium
KR20210065076A (en) Method, apparatus, device, and storage medium for obtaining document layout
CN104881673A (en) Mode identification method based on information integration and system thereof
CN114330588A (en) Picture classification method, picture classification model training method and related device
CN110018827B (en) Method and device for automatically generating code, electronic equipment and readable storage medium
CN114373460A (en) Instruction determination method, device, equipment and medium for vehicle-mounted voice assistant
CN114064010A (en) Front-end code generation method, device, system and storage medium
CN113762303B (en) Image classification method, device, electronic equipment and storage medium
CN114495101A (en) Text detection method, and training method and device of text detection network
CN112270169B (en) Method and device for predicting dialogue roles, electronic equipment and storage medium
CN112542163B (en) Intelligent voice interaction method, device and storage medium
CN113761923A (en) Named entity recognition method and device, electronic equipment and storage medium
CN115687146A (en) BIOS (basic input output System) test method and device, computer equipment and storage medium
KR20230044914A (en) Korean relation extraciton model and method with entity positon information
CN115017922A (en) Method and device for translating picture, electronic equipment and readable storage medium
US11099977B1 (en) Method, device and computer-readable storage medium for testing bios using menu map obtained by performing image identification on bios interface

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination