CN113946615A - Data structuring processing method, device, equipment and storage medium - Google Patents
Data structuring processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113946615A CN113946615A CN202111217653.1A CN202111217653A CN113946615A CN 113946615 A CN113946615 A CN 113946615A CN 202111217653 A CN202111217653 A CN 202111217653A CN 113946615 A CN113946615 A CN 113946615A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- fragment
- syntax tree
- fragment data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 7
- 239000012634 fragment Substances 0.000 claims abstract description 108
- 238000000034 method Methods 0.000 claims abstract description 48
- 238000012545 processing Methods 0.000 claims abstract description 32
- 238000013507 mapping Methods 0.000 claims abstract description 23
- 230000006870 function Effects 0.000 claims description 26
- 238000003780 insertion Methods 0.000 claims description 15
- 230000037431 insertion Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 12
- 230000006835 compression Effects 0.000 claims description 9
- 238000007906 compression Methods 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 238000002372 labelling Methods 0.000 claims description 6
- 238000013473 artificial intelligence Methods 0.000 abstract description 4
- 230000005540 biological transmission Effects 0.000 description 46
- 230000008569 process Effects 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 6
- 238000000547 structure data Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000010276 construction Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 239000003550 marker Substances 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to the technical field of artificial intelligence, and discloses a data structured processing method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring target data, and cutting the target data to obtain at least two fragment data; creating and configuring a thread pool, and transmitting the fragment data to a parameter value taking function through different storage threads; analyzing context information of the acquired fragment data, and carrying out value taking on the fragment data through a parameter value taking function based on the context information to obtain an expression parameter; acquiring a display structure of a target object, and establishing a target syntax tree corresponding to the display structure; extracting variable marks and data variables corresponding to the variable marks from the expression parameters through a mapping model, and mapping the data variables to each object node of a target syntax tree according to the variable marks to obtain structural data corresponding to a target object; thereby improving the efficiency of the structured processing of the data.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a method, an apparatus, a device, and a storage medium for structured processing of data.
Background
The BFF (Bank For front) system is a layer between App/Web and back-end micro-service, and plays a role in data aggregation, cutting and editing. In order to meet the requirements of different mobile terminals and desktop terminals on different display formats, data provided by a back end needs to be customized through a BFF system.
However, if the BFF system is used to acquire data in separate fields for each customization process, the efficiency of the customization process for converting data into a target structure is reduced.
Disclosure of Invention
The present application mainly aims to provide a method, an apparatus, a device, and a storage medium for structured processing of data, and aims to solve the problem in the prior art that the efficiency of structured processing of data is low.
In order to achieve the above object, the present application provides a method for structured processing of data, the method comprising:
acquiring target data, and cutting the target data to obtain at least two fragment data;
creating and configuring a thread pool, and transmitting the fragment data to a parameter value taking function through different storage threads based on at least two storage threads in the thread pool;
analyzing context information of the fragment data in the parameter value taking function, and taking values of the fragment data through the parameter value taking function based on the context information to obtain expression parameters;
acquiring a display structure of a target object, and establishing a target syntax tree corresponding to the display structure, wherein the target syntax tree comprises at least two editable object nodes;
extracting variable marks and data variables corresponding to the variable marks from the expression parameters through a mapping model, and mapping the data variables to each object node of the target syntax tree according to the variable marks to obtain structural data corresponding to the target object.
Further, the establishing of the target syntax tree corresponding to the display structure includes:
identifying a variable insertion bit in the display structure;
and taking the variable insertion bit as the object node, inserting an identification anchor point into the object node, and generating the target syntax tree according to the identification anchor point.
Further, after the clipping is performed on the target data, the method further includes:
performing context labeling according to the sequence of the fragment data in the target data;
the analyzing the context information of the acquired fragment data comprises:
and analyzing the context information of the fragment data according to the context label.
Further, the clipping the target data to obtain at least two fragment data includes:
judging whether the target data comprises picture data or not, and if so, cutting the picture data from the target data;
and performing data stream conversion on the picture data, performing equal proportion compression on the obtained data stream of the picture data, and taking the compressed data stream as fragment data corresponding to the picture data.
Further, the determining whether the target data includes picture data includes:
and identifying the data format of each data contained in the target data, and identifying whether the target data comprises picture data according to the data format.
Further, after the target syntax tree corresponding to the display structure is established, the method further includes:
generating a tree index table according to the target syntax tree, wherein the tree index table comprises index codes associated with the object nodes;
establishing a link between each index code and the corresponding object node;
and responding to an index query instruction, and calling the object node through the link according to the index code corresponding to the index query instruction.
The present application further provides a structured processing apparatus for data, including:
the data cutting module is used for acquiring target data and cutting the target data to obtain at least two fragment data;
the multithreading acquisition module is used for creating and configuring a thread pool, and transmitting the fragment data to a parameter value taking function through different storage threads based on at least two storage threads in the thread pool;
the expression parameter acquisition module is used for analyzing the context information of the fragment data in the parameter value taking function and taking values of the fragment data through the parameter value taking function based on the context information to obtain expression parameters;
the syntax tree generating module is used for acquiring a display structure of a target object and establishing a target syntax tree corresponding to the display structure, wherein the target syntax tree comprises at least two editable object nodes;
and the data structuring module is used for extracting variable marks and data variables corresponding to the variable marks from the expression parameters through a mapping model, and mapping the data variables to each object node of the target syntax tree according to the variable marks to obtain structural data corresponding to the target object.
The present application further proposes a computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the steps of any of the above methods when executing the computer program.
The present application also proposes a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method of any of the above.
According to the data structuring processing method, the data structuring processing device, the data structuring processing equipment and the data structuring processing storage medium, the target data are cut to obtain the plurality of fragment data, so that the problem that the transmission efficiency and the processing efficiency are low due to the fact that a transmission object is large is solved; the method has the advantages that the thread pool comprising a plurality of storage threads is established, and multi-thread transmission is carried out on a large amount of fragment data, so that the data transmission efficiency is improved, and the transmission resource pressure is reduced; by analyzing the context information of the fragment data and taking values based on the context information, the correctness of the expression parameters can be ensured, and the problem of data confusion caused by multi-thread transmission is avoided; by establishing the target syntax tree for the target object structure, the corresponding display format can be converted into an editable syntax tree state so as to facilitate the insertion of subsequent data into the target syntax tree, and after the target syntax tree is generated according to the target object, the original structure in the target syntax tree is replaced by the data variable through the editable object node to generate the structure data.
Drawings
Fig. 1 is a schematic flowchart of a method for structured processing of data according to an embodiment of the present application;
FIG. 2 is a flow chart illustrating a method for structured processing of data according to an embodiment of the present disclosure;
FIG. 3 is a block diagram illustrating an exemplary structure of a device for structured processing of data according to an embodiment of the present disclosure;
fig. 4 is a block diagram illustrating a structure of a computer device according to an embodiment of the present application.
The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
With reference to fig. 1, in order to achieve the above object, the present application proposes a method for structured processing of data, the method comprising:
s1: acquiring target data, and cutting the target data to obtain at least two fragment data;
s2: creating and configuring a thread pool, and respectively inputting the fragment data into different storage threads of the thread pool;
s3: respectively analyzing context information of the fragment data in different storage threads, and carrying out value taking on the fragment data through a parameter value taking function based on the context information to obtain expression parameters;
s4: acquiring a display structure of a target object, and establishing a target syntax tree corresponding to the display structure, wherein the target syntax tree comprises at least two editable object nodes;
s5: extracting variable marks and data variables corresponding to the variable marks from the expression parameters through a mapping model, and mapping the data variables to each object node of the target syntax tree according to the variable marks to obtain structural data corresponding to the target object.
In the embodiment, the target data is cut to obtain a plurality of fragment data, so that the problems of low transmission efficiency and low processing efficiency caused by large transmission objects are solved; the method comprises the steps that a thread pool comprising a plurality of storage threads is established, and multi-thread acquisition is carried out on a large amount of fragment data, so that the data acquisition efficiency is improved, and the pressure of transmission resources is reduced; by analyzing the context information of the fragment data and taking values based on the context information, the correctness of the expression parameters can be ensured, and the problem of data confusion caused by multi-thread transmission is avoided; by establishing the target syntax tree for the target object structure, the corresponding display format can be converted into an editable syntax tree state so as to facilitate the insertion of subsequent data into the target syntax tree, and after the target syntax tree is generated according to the target object, the original structure in the target syntax tree is replaced by the data variable through the editable object node to generate the structure data.
With respect to step S1, the present embodiment is applied to data processing, and in particular, in the data customization processing of the BFF layer, the data of the BFF layer may be subjected to customized structuring processing based on artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result. In a specific embodiment, the data clipping may be performed by a clipping model, which may be a convolutional neural network model including a data clipping algorithm, where the data clipping algorithm may be to clip the target data according to a preset segment length, a preset segment identifier, and the like, so as to obtain at least two segment data with a smaller data length than the total target data. In this embodiment, at least two fragment data are obtained by cutting the target data, so that the problem of low transmission efficiency and processing efficiency caused by a large transmission object is solved.
For step S2, in order to avoid the problem that the existing thread pool will cause additional occupation of system resources in the time that data customization is not needed, a new thread pool is created before data customization is performed each time; specifically, a Thread pool may be created by the initial Thread Local, that is, the value of threadthalls of the parent Thread is copied to threadthalls of the new Thread by the initial Thread Local, and the fragment data may be input to the parameter value taking function by each Thread by threadthals in the hreadlocal. On the basis, a thread pool comprising at least two storage threads is established, and fragment data can be transmitted in a multi-thread mode, namely, the fragment data is divided into at least two parts and transmitted and stored through the plurality of storage threads respectively. In the embodiment, the thread pool comprising a plurality of storage threads is established, and multi-thread transmission is performed on a large amount of fragment data, so that the data transmission efficiency is improved, and the transmission resource pressure is reduced.
For step S3, in order to obtain a parameter to be customized and displayed according to the clipped fragment data, a preset context object may be injected in a Groovy Shell manner based on a parameter value taking function, and a parameter attribute and a class path are configured, and then a value based on the context object is taken through a Groovy Shell () parameter value taking function, so as to obtain the expression parameter. In addition, in Java, different types of value taking modes cannot be obtained through a point mode, for example, a map type is realized through a get method, a value taking expression cannot be written in a unified and concise manner, a Groovy core package can be introduced through a value taking mode of Groovy Shell (), values can be taken from fragment data through a point mode such as result.
For the context object, specifically, because the original target data is often data with a description sequence, and the problem that the description sequence between the fragment data is disturbed due to different transmission sequences may occur after the original target data is cut into the fragment data, it is necessary to perform context analysis on the cut point of each fragment data through a context model, identify correct context information of each fragment data, and then perform value taking on the fragment data through Groovy Shell (), so as to obtain an expression parameter corresponding to the source object of the target data. In this embodiment, by analyzing the context information of the fragment data and taking values based on the context information, the correctness of the expression parameters can be ensured, and the problem of data confusion caused by multi-thread transmission is avoided.
For step S4, the target object is usually an object such as an application or a web page that needs to display the target data according to a certain display format, and since different applications and web pages often have different display format requirements, a display structure corresponding to the display format needs to be obtained, and a Json tree target syntax tree based on jackson is generated according to the structure, specifically, jackson can be used to read count node values in the target object structure and traverse all nodes under list, and various Json node nodes, such as object node ObjectNode, array node, and the like, are constructed through Json node factory classes; representing a Json node through JsonNode classes, and adding a Json value into the Json node to construct a Json tree; the JsonNode node is converted to the final Json string by the ObjectMapper class. And the object node ObjectNode is an editable node capable of inserting various variable values. In this embodiment, by establishing the target syntax tree for the target object structure, the corresponding presentation format can be converted into an editable syntax tree state, so that subsequent data can be inserted into the target syntax tree.
For step S5, the mapping model may be a convolutional neural network model including mapping and a parameter clipping algorithm, the parameter clipping algorithm may be to obtain a variable flag in an expression parameter and extract a data variable corresponding to the variable flag, the mapping algorithm may be to insert the data variable into a corresponding node in a target syntax tree according to the variable flag, and the expression parameter may further include a variable name so that a user may recognize a function of the variable that is not used. After the data variable is inserted, format conversion is carried out according to the target syntax tree after the data variable is inserted, so that the target data is converted into customized data, namely the structural data, in the display format corresponding to the target object, an application program or a webpage displays the structural data on a display interface, and the customization of the display format of the target data is realized. After the primary structure data is obtained, the Apollo can be used for managing the metadata of the mapping model and the display structure, and the metadata can be updated at the operation period, so that the display structure is dynamically obtained, the structure of the target syntax tree is dynamically modified, and the conversion of various display formats is realized. In this embodiment, the display format conversion of the target data is realized by mapping the data variable into the target syntax tree.
In one embodiment, referring to fig. 2, the establishing of the target syntax tree corresponding to the presentation structure includes:
s41: identifying a variable insertion bit in the display structure;
s42: and taking the variable insertion bit as the object node, inserting an identification anchor point into the object node, and generating the target syntax tree according to the identification anchor point.
In the embodiment, the variable insertion bit in the display structure is identified, the variable insertion bit is used as the object node, and the target syntax tree is generated according to the identification anchor point in the object node, so that the variable position is accurately positioned, and the construction efficiency and accuracy of the structural data are improved.
For step S42, the special character such as "#" may be used as the identification anchor, and the identification anchor is inserted into the object node position in the target syntax tree, so that the mapping model inserts the data variable corresponding to the # into the object node of the target syntax tree after identifying the identification anchor such as "#", thereby accurately positioning the variable position and improving the construction efficiency and accuracy of the structural data.
Preferably, in order to reduce the probability that the identification anchor is mistakenly identified, a combination of unusual special characters in the plurality of target data, such as "# # # #", "# # # # # # # # # #, and the like, may be set as the identification anchor to reduce the coincidence probability of the identification anchor and the target data and improve the accuracy of identification and object insertion.
In one embodiment, after the clipping the target data, the method further includes:
s111: performing context labeling according to the sequence of the fragment data in the target data;
said performing context information analysis on said acquired fragment data S3, comprising:
s31: and analyzing the context information of the fragment data according to the context label.
The embodiment performs context labeling on the fragment data after target data is cut and before data multithread transmission, so as to facilitate the subsequent and rapid context information analysis.
For step S111, since the actually obtained sequence of the fragment data is often difficult to be different from the sequence of the fragment data in the original target data in the multi-threaded data transmission, if the fragment data with disordered sequence is directly evaluated, the structural data in the finally generated presentation format may not be consistent with the content meaning of the original target data; illustratively, taking the user registry as an example, if the target data contains information such as (1) name: zhangqi, (2) register the mobile phone number: 139xxxxxxxx, (3) registration time: 2019-06-01, when the fragment data is out of order, the resulting structure data may be (1) registration time: 2019-06-01, (2) registering mobile phone number: 139xxxxxxxx, (3) name: zhangao, etc. Therefore, the embodiment performs context labeling on the fragment data after target data is clipped and before data multithread transmission, so as to facilitate rapid subsequent context information analysis. Preferably, the sequence numbers may be sequentially inserted into all the fragment data in the order of the target data, or a right marker corresponding to the above may be added to a portion of the fragment data connected to the above, and a left marker corresponding to the below may be added to a portion of the fragment data connected to the below, so as to facilitate the subsequent data connection.
With step S31, based on the context labels formed as described above, context information analysis is performed on fragment data in the label order.
In one embodiment, the creating and configuring a thread pool includes:
s21: acquiring the number of fragments of the fragment data;
s22: and generating the thread quantity corresponding to the fragment quantity according to a preset proportion, and creating and configuring the thread pool according to the thread quantity.
In the embodiment, the corresponding number of threads are generated according to the number of the fragments, and the thread pool is configured, so that the data transmission balance is improved, and the overlarge time difference of different fragment data reaching the storage system is avoided.
For step S22, in order to ensure the data transmission efficiency, the present embodiment uses more threads for transferring larger target data, specifically, a certain calculation ratio may be preset, and the total number of storage threads that should be set is obtained according to the number of fragments of fragment data. Preferably, different fragment data can be allocated to appropriate storage threads according to the data sizes of the fragment data, so that the sizes of the fragment data transmitted in the storage threads are basically consistent, the data transmission balance is improved, and the time difference of the different fragment data reaching a storage system is avoided.
In one embodiment, the cropping the target data to obtain at least two fragment data includes:
s23: judging whether the target data comprises picture data or not, and if so, cutting the picture data from the target data;
s24: and performing data stream conversion on the picture data, performing equal proportion compression on the obtained data stream of the picture data, and taking the compressed data stream as fragment data corresponding to the picture data.
The embodiment extracts the picture data and performs equal-proportion compression on the picture data, thereby reducing the transmission flow occupied by the picture data and improving the transmission efficiency of the fragment data.
In step S23, the picture data has a large data size and a high bandwidth requirement for transmission.
In step S24, in order to improve the transmission efficiency of the picture data, in this embodiment, the picture data is first subjected to data stream conversion, then the data stream is subjected to equal-proportion compression by a Canvas picture compression technique to obtain a compressed data stream with a small data size, the compressed data stream is transmitted as a new fragment data, which can reduce the bandwidth required for transmitting the picture data and improve the transmission efficiency, and after the picture data subjected to multi-thread transmission is acquired, the data stream is exported in an image format to obtain a required expression parameter, thereby completing the compression and transmission of the picture data.
In one embodiment, the determining whether the target data includes picture data includes:
s231: and identifying the data format of each data contained in the target data, and identifying whether the target data comprises picture data according to the data format.
According to the embodiment, whether the target data contains the picture data or not is judged through the data format, so that the accuracy of data extraction is improved.
For step S231, in a specific implementation manner, the data format of the picture data is often BMP, JPEG, TIFF, and the like, and for common picture data, the size of the smaller picture data may be 20KB and the like, and the size of the larger picture data may be 10M or even hundreds of M, so as to avoid that the larger picture data occupies more transmission bandwidth and causes a decrease in data transmission efficiency, the data format recognition is performed on the larger picture data in this embodiment, thereby completing extraction and compression, and decompressing and restoring after completing transmission, and improving the data transmission efficiency.
In one embodiment, after the establishing the target syntax tree corresponding to the presentation structure, the method further includes:
s411: generating a tree index table according to the target syntax tree, wherein the tree index table comprises index codes associated with the object nodes;
s412: establishing a link between each index code and the corresponding object node;
s413: and responding to an index query instruction, and calling the object node through the link according to the index code corresponding to the index query instruction.
In the embodiment, the index codes associated with the object nodes are set, and the corresponding object nodes are called according to the index query instruction, so that a user can conveniently check data abnormity, or data of the individual nodes can be modified in a targeted manner, and the flexibility of checking and modifying the data abnormity is improved.
For step S411, the tree index table may be one or more sets of at least two index codes, where the number of index codes is the same as the number of object nodes, and the index codes and the object nodes are in one-to-one correspondence. In a specific embodiment, the index code may be one of a number or a letter, or a combination of the number and the letter, so as to facilitate the user's query.
For step S412, after the tree index table is generated, a corresponding link is established for each index code, where the link may be an address of a jump to a corresponding object node or a call path of the object node.
In step S413, when the index query instruction sent by the user is received, the corresponding object node may be called according to the link, so that the user may check the data exception or modify the data of the individual node in a targeted manner.
Referring to fig. 3, the present application further proposes a data structured processing apparatus, including:
the data clipping module 100 is configured to obtain target data and clip the target data to obtain at least two fragment data;
a multithreading obtaining module 200, configured to create and configure a thread pool, and transmit the fragment data to a parameter value taking function through different storage threads based on at least two storage threads in the thread pool;
an expression parameter obtaining module 300, configured to perform context information analysis on the fragment data in the parameter value taking function, and perform value taking on the fragment data through the parameter value taking function based on the context information to obtain an expression parameter;
a syntax tree generating module 400, configured to obtain a display structure of a target object, and establish a target syntax tree corresponding to the display structure, where the target syntax tree includes at least two editable object nodes;
a data structuring module 500, configured to extract variable tags and data variables corresponding to the variable tags from the expression parameters through a mapping model, and map the data variables to the object nodes of the target syntax tree according to the variable tags, so as to obtain structural data corresponding to the target object.
In the embodiment, the target data is cut to obtain a plurality of fragment data, so that the problems of low transmission efficiency and low processing efficiency caused by large transmission objects are solved; the method has the advantages that the thread pool comprising a plurality of storage threads is established, and multi-thread transmission is carried out on a large amount of fragment data, so that the data transmission efficiency is improved, and the transmission resource pressure is reduced; by analyzing the context information of the fragment data and taking values based on the context information, the correctness of the expression parameters can be ensured, and the problem of data confusion caused by multi-thread transmission is avoided; by establishing the target syntax tree for the target object structure, the corresponding display format can be converted into an editable syntax tree state so as to facilitate the insertion of subsequent data into the target syntax tree, and after the target syntax tree is generated according to the target object, the original structure in the target syntax tree is replaced by the data variable through the editable object node to generate the structure data.
In one embodiment, the syntax tree generation module 400 is configured to:
identifying a variable insertion bit in the display structure;
and taking the variable insertion bit as the object node, inserting an identification anchor point into the object node, and generating the target syntax tree according to the identification anchor point.
In one embodiment, the data cropping module 100 is further configured to:
performing context labeling according to the sequence of the fragment data in the target data;
the expression parameter obtaining module 300 is further configured to:
and analyzing the context information of the fragment data according to the context label.
In one embodiment, the multithreaded fetch module 200 is further configured to:
acquiring the number of fragments of the fragment data;
and generating the thread quantity corresponding to the fragment quantity according to a preset proportion, and creating and configuring the thread pool according to the thread quantity.
In one embodiment, the data cropping module 100 is further configured to:
judging whether the target data comprises picture data or not, and if so, cutting the picture data from the target data;
and performing data stream conversion on the picture data, performing equal proportion compression on the obtained data stream of the picture data, and taking the compressed data stream as fragment data corresponding to the picture data.
In one embodiment, the data cropping module 100 is further configured to:
and identifying the data format of each data contained in the target data, and identifying the picture data in the target data according to the data format.
In one embodiment, the index table creating module 600 is further included for:
generating a tree index table according to the target syntax tree, wherein the tree index table comprises index codes associated with the object nodes;
establishing a link between each index code and the corresponding object node;
and responding to an index query instruction, and calling the object node through the link according to the index code corresponding to the index query instruction.
Referring to fig. 4, a computer device, which may be a server and whose internal structure may be as shown in fig. 4, is also provided in the embodiment of the present application. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the computer designed processor is used to provide computational and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used for storing data such as a structured processing method of the data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of structured processing of data. The data structuring processing method comprises the following steps: acquiring target data, and cutting the target data to obtain at least two fragment data; creating and configuring a thread pool, and transmitting the fragment data to a parameter value taking function through different storage threads based on at least two storage threads in the thread pool; analyzing context information of the fragment data in the parameter value taking function, and taking values of the fragment data through the parameter value taking function based on the context information to obtain expression parameters; acquiring a display structure of a target object, and establishing a target syntax tree corresponding to the display structure, wherein the target syntax tree comprises at least two editable object nodes; extracting variable marks and data variables corresponding to the variable marks from the expression parameters through a mapping model, and mapping the data variables to each object node of the target syntax tree according to the variable marks to obtain structural data corresponding to the target object.
An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements a method for structured processing of data, including the steps of: acquiring target data, and cutting the target data to obtain at least two fragment data; creating and configuring a thread pool, and transmitting the fragment data to a parameter value taking function through different storage threads based on at least two storage threads in the thread pool; analyzing context information of the fragment data in the parameter value taking function, and taking values of the fragment data through the parameter value taking function based on the context information to obtain expression parameters; acquiring a display structure of a target object, and establishing a target syntax tree corresponding to the display structure, wherein the target syntax tree comprises at least two editable object nodes; extracting variable marks and data variables corresponding to the variable marks from the expression parameters through a mapping model, and mapping the data variables to each object node of the target syntax tree according to the variable marks to obtain structural data corresponding to the target object.
In the above-described executed data structuring processing method, in this embodiment, a plurality of fragment data are obtained by cutting target data, so that the problem of low transmission efficiency and processing efficiency due to a large transmission object is solved; the method has the advantages that the thread pool comprising a plurality of storage threads is established, and multi-thread transmission is carried out on a large amount of fragment data, so that the data transmission efficiency is improved, and the transmission resource pressure is reduced; by analyzing the context information of the fragment data and taking values based on the context information, the correctness of the expression parameters can be ensured, and the problem of data confusion caused by multi-thread transmission is avoided; by establishing the target syntax tree for the target object structure, the corresponding display format can be converted into an editable syntax tree state so as to facilitate the insertion of subsequent data into the target syntax tree, and after the target syntax tree is generated according to the target object, the original structure in the target syntax tree is replaced by the data variable through the editable object node to generate the structure data.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium provided herein and used in the examples may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double-rate SDRAM (SSRSDRAM), Enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and bus dynamic RAM (RDRAM).
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.
The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.
Claims (10)
1. A method for structured processing of data, the method comprising:
acquiring target data, and cutting the target data to obtain at least two fragment data;
creating and configuring a thread pool, and transmitting the fragment data to a parameter value taking function through different storage threads based on at least two storage threads in the thread pool;
analyzing context information of the fragment data in the parameter value taking function, and taking values of the fragment data through the parameter value taking function based on the context information to obtain expression parameters;
acquiring a display structure of a target object, and establishing a target syntax tree corresponding to the display structure, wherein the target syntax tree comprises at least two editable object nodes;
extracting variable marks and data variables corresponding to the variable marks from the expression parameters through a mapping model, and mapping the data variables to each object node of the target syntax tree according to the variable marks to obtain structural data corresponding to the target object.
2. The method for structured processing of data according to claim 1, wherein the establishing of the target syntax tree corresponding to the presentation structure comprises:
identifying a variable insertion bit in the display structure;
and taking the variable insertion bit as the object node, inserting an identification anchor point into the object node, and generating the target syntax tree according to the identification anchor point.
3. The method for structured processing of data according to claim 1, wherein after the clipping the target data, further comprising:
performing context labeling according to the sequence of the fragment data in the target data;
the analyzing the context information of the acquired fragment data comprises:
and analyzing the context information of the fragment data according to the context label.
4. The method for structured processing of data according to claim 1, wherein said creating and configuring a thread pool comprises:
acquiring the number of fragments of the fragment data;
and generating the thread quantity corresponding to the fragment quantity according to a preset proportion, and creating and configuring the thread pool according to the thread quantity.
5. The method for structured processing of data according to claim 1, wherein the clipping the target data to obtain at least two fragment data includes:
judging whether the target data comprises picture data or not, and if so, cutting the picture data from the target data;
and performing data stream conversion on the picture data, performing equal proportion compression on the obtained data stream of the picture data, and taking the compressed data stream as fragment data corresponding to the picture data.
6. The method for processing the data according to claim 5, wherein the determining whether the target data includes the picture data includes:
and identifying the data format of each data contained in the target data, and identifying whether the target data comprises picture data according to the data format.
7. The method for structured processing of data according to claim 1, wherein after the creating of the target syntax tree corresponding to the presentation structure, further comprising:
generating a tree index table according to the target syntax tree, wherein the tree index table comprises index codes associated with the object nodes;
establishing a link between each index code and the corresponding object node;
and responding to an index query instruction, and calling the object node through the link according to the index code corresponding to the index query instruction.
8. An apparatus for structured processing of data, comprising:
the data cutting module is used for acquiring target data and cutting the target data to obtain at least two fragment data;
the multithreading acquisition module is used for creating and configuring a thread pool, and transmitting the fragment data to a parameter value taking function through different storage threads based on at least two storage threads in the thread pool;
the expression parameter acquisition module is used for analyzing the context information of the fragment data in the parameter value taking function and taking values of the fragment data through the parameter value taking function based on the context information to obtain expression parameters;
the syntax tree generating module is used for acquiring a display structure of a target object and establishing a target syntax tree corresponding to the display structure, wherein the target syntax tree comprises at least two editable object nodes;
and the data structuring module is used for extracting variable marks and data variables corresponding to the variable marks from the expression parameters through a mapping model, and mapping the data variables to each object node of the target syntax tree according to the variable marks to obtain structural data corresponding to the target object.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111217653.1A CN113946615A (en) | 2021-10-19 | 2021-10-19 | Data structuring processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111217653.1A CN113946615A (en) | 2021-10-19 | 2021-10-19 | Data structuring processing method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113946615A true CN113946615A (en) | 2022-01-18 |
Family
ID=79331474
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111217653.1A Pending CN113946615A (en) | 2021-10-19 | 2021-10-19 | Data structuring processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113946615A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115576603A (en) * | 2022-11-23 | 2023-01-06 | 云账户技术(天津)有限公司 | Method and device for acquiring variable values in code segments |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109657103A (en) * | 2018-12-19 | 2019-04-19 | 广州天鹏计算机科技有限公司 | Conversion method, device, computer equipment and the storage medium of data structure |
| CN110209766A (en) * | 2019-05-23 | 2019-09-06 | 招商局金融科技有限公司 | Method for exhibiting data, electronic device and storage medium |
-
2021
- 2021-10-19 CN CN202111217653.1A patent/CN113946615A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109657103A (en) * | 2018-12-19 | 2019-04-19 | 广州天鹏计算机科技有限公司 | Conversion method, device, computer equipment and the storage medium of data structure |
| CN110209766A (en) * | 2019-05-23 | 2019-09-06 | 招商局金融科技有限公司 | Method for exhibiting data, electronic device and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 陆晨等: "GraphQL及元数据驱动架构在后端BFF中的实践", pages 1 - 15, Retrieved from the Internet <URL:https://tech.meituan.com/2021/05/06/bff-graphql.html> * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115576603A (en) * | 2022-11-23 | 2023-01-06 | 云账户技术(天津)有限公司 | Method and device for acquiring variable values in code segments |
| CN115576603B (en) * | 2022-11-23 | 2023-03-10 | 云账户技术(天津)有限公司 | Method and device for acquiring variable values in code segment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106598574B (en) | Page rendering method and device | |
| CN110889270B (en) | Form page document generation method, form page rendering method and device | |
| CN109254773B (en) | Skeleton page generation method, device, equipment and storage medium | |
| CN109840083B (en) | Webpage component template construction method and device, computer equipment and storage medium | |
| CN109684607B (en) | JSON data analysis method and device, computer equipment and storage medium | |
| CN112651238A (en) | Training corpus expansion method and device and intention recognition model training method and device | |
| CN112882974B (en) | JSON data conversion method and device, computer equipment and storage medium | |
| CN107729014B (en) | Page jump method and structure based on Android | |
| CN113536735B (en) | Text marking method, system and storage medium based on keywords | |
| CN108664546B (en) | XML data structure conversion method and device | |
| CN112187713B (en) | Message conversion method, device, computer equipment and storage medium | |
| CN111147354B (en) | Message processing method, device, equipment and storage medium | |
| CN111191079A (en) | Document content acquisition method, device, equipment and storage medium | |
| CN111831920A (en) | User demand analysis method and device, computer equipment and storage medium | |
| CN113946615A (en) | Data structuring processing method, device, equipment and storage medium | |
| CN108897772B (en) | Data source supplementing method and device, computer equipment and storage medium | |
| CN112650600B (en) | Method, device and computer equipment for pushing message content | |
| CN114021175B (en) | User portrait configuration method and device, computer equipment and medium | |
| CN112101023B (en) | Text processing method and device and electronic equipment | |
| CN111768767B (en) | User tag extraction method and device, server and computer readable storage medium | |
| CN112714148A (en) | Interface configuration method, device, equipment and medium | |
| CN111258586B (en) | Fast application running and compiling method and device, electronic equipment and storage medium | |
| CN112947938B (en) | File conversion method and device, electronic equipment and storage medium | |
| CN114760365B (en) | Data extraction method and device and electronic equipment | |
| CN114827309B (en) | Equipment fingerprint generation method, device, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |