CN117076661B - Legislation planning intention recognition method for tuning of pre-training large language model - Google Patents
Legislation planning intention recognition method for tuning of pre-training large language model Download PDFInfo
- Publication number
- CN117076661B CN117076661B CN202311052216.8A CN202311052216A CN117076661B CN 117076661 B CN117076661 B CN 117076661B CN 202311052216 A CN202311052216 A CN 202311052216A CN 117076661 B CN117076661 B CN 117076661B
- Authority
- CN
- China
- Prior art keywords
- legal
- language model
- tuning
- opinion
- planning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000012549 training Methods 0.000 title claims abstract description 24
- 238000000605 extraction Methods 0.000 claims abstract description 58
- 230000006870 function Effects 0.000 claims abstract description 26
- 230000006399 behavior Effects 0.000 claims description 17
- 230000008439 repair process Effects 0.000 claims description 5
- 230000008676 import Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000012916 structural analysis Methods 0.000 claims description 2
- 238000013461 design Methods 0.000 abstract description 7
- 230000008569 process Effects 0.000 abstract description 3
- 238000012545 processing Methods 0.000 description 7
- 230000009471 action Effects 0.000 description 6
- 230000008520 organization Effects 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000003542 behavioural effect Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000014509 gene expression Effects 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000003058 natural language processing Methods 0.000 description 3
- 238000012706 support-vector machine Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 239000002131 composite material Substances 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 238000007637 random forest analysis Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/34—Browsing; Visualisation therefor
- G06F16/345—Summarisation for human users
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/18—Legal services
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Tourism & Hospitality (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Technology Law (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a method for identifying legal planning intentions for tuning a pre-training large language model, which is used for obtaining legal planning opinion text information and obtaining a legal planning opinion ontology; generating a pre-training large language model tuning instruction generating function in a design natural language form by traversing the legislation planning opinion knowledge body; and calling a big language model tuning prompt instruction generating function (S, C, T) to generate a natural language form legislation intention recognition prompt instruction, thereby realizing tuning recognition of the legislation intention and abstract generation. Compared with the prior art, the invention realizes the tuning and optimizing identification of the legislation intention by generating the tuning and optimizing instruction in the form of natural language in batches, and combines the more convenient and efficient information extraction method and the improvement of the small sample training method in the process.
Description
Technical Field
The invention belongs to the technical field of artificial intelligence and natural language processing, and particularly relates to a knowledge-enhancement-oriented legislation planning/planning intention recognition method for extracting information of a pre-trained large language model.
Background
In order to better realize legislation planning and planning, comprehensive classification is required for massive various opinion texts, and various opinion texts are intelligently identified through an information extraction technology. Opinion text often includes various opinion-related text data of large data types. In the face of such a wide variety of opinion text data, it becomes particularly important how to perform efficient extraction and evaluation. Conventional approaches often require significant time and effort to manually comb and sort. With the gradual application of artificial intelligence technology, information extraction can be used for intelligently identifying elements of various opinions through natural language processing technology. The extraction of information and the identification of intent associated with the present invention in the prior art is described in detail below:
The information extraction method comprises an entity and relation extraction method, an event extraction method and a joint information extraction method. The extraction method of the entity and the relation comprises the following steps: ① Rule-Based Methods (Rule-Based Methods) identify entities and their relationships by manually formulating various rules and templates, rely on manually formulated rules and features, are difficult to expand into a wider field, and cannot efficiently handle Rule conflicts and complex language structures, and are difficult to accommodate large-scale and complex corpus datasets. ② Feature-Based Methods (Feature-Based Methods) assist in entity relationship extraction by manually designing and extracting various features of text, and classification is performed on the basis of Feature extraction using conventional machine learning algorithms such as Support Vector Machines (SVMs) and Random Forest (Random Forest) and the like. It has the disadvantage that it is difficult to properly design features that cover all entity types and language constructs. In addition, the method requires a lot of time and resources for feature engineering, and is not highly interpretative. ③ Neural network-Based Methods (Neural Network-Based Methods): the tag sequence is converted into a relationship between entity pairs using Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) based methods. In addition, text and structural extraction of representations of entities and relationships is performed using methods such as graph rolling networks (GCNs), graph Neural Networks (GNNs), and the like. At the same time, there are joint extraction techniques for extracting entities and relationships simultaneously. ④ The extraction of the pre-training model can be regarded as a knowledge-enhanced joint extraction method. It uses large deep learning models containing pre-trained language representations, such as BERT (excluded text) and XLNet (scalable self-contained Wen Yuyi) to handle entities and relationships in text. By encoding the context information as words or vectors of words, the pre-trained language representation model can better understand the context information, thereby improving performance of entity recognition and relationship extraction. Extraction methods of pre-trained models often use remote supervision to automatically construct large scale annotation data to train the model. In addition, external vocabulary and syntactic knowledge may be integrated into the pre-trained model, for example, using external knowledge in common attention mechanisms between entities and relationships to enhance the expression and semantic understanding capabilities of the model. (2) The event extraction method is divided into two main parts, namely trigger word recognition and event element recognition. Trigger word recognition refers to recognition of words or phrases in text that represent the occurrence of an event, while event elements refer to various elements, such as entities, times, places, etc., associated with the event. Some methods may implement event extraction by a combination of natural language processing and machine learning methods, such as support vector machines, conditional random fields, deep learning, and attention mechanisms, etc. For example, the CNN-BiLSTM-CRF model combines a convolutional neural network as an initial feature learning model, a bidirectional long-short-time memory network as an event sentence modeling model and a conditional random field as a classification model, so that the performance of event extraction is further improved. In addition, pre-training models, such as BERT and XLNet, are also used in event extraction tasks, achieving better results. The method identifies trigger words, entities and event types from unstructured text and classifies and organizes them for further understanding and analysis of events in the text. The event extraction can be applied to the fields of news reports, social media, financial analysis and the like, and is helpful for deeper mining and understanding of event development and relationship. (3) Joint information extraction (Unified Information Extraction): the current technology is the combined information extraction technology, and compared with the traditional information extraction method, the combined information extraction technology can acquire important information in the text more comprehensively and accurately. In federated information extraction, entity extraction, relationship extraction, and event extraction are fundamental subtasks. Entity extraction refers to finding named entities from text, such as person names, place names, organization names, etc. Relationship extraction refers to finding semantic relationships between entities, such as work, friends, family relationships, etc. Event extraction refers to extracting an event, including event type, trigger words, participants, and the like. These subtasks are not independent, but are interdependent and related. The joint information extraction is an information extraction method comprehensively utilizing a plurality of subtasks, and can simultaneously extract a plurality of semantic information such as entities, relations, events and the like and perform joint optimization in the same model.
The intention recognition (ID) is a key to the constitution of the man-machine conversation system. The intention is the intention of the user, i.e. what the user wants to do. An intent is sometimes also referred to as a "Dialog behavior" (Dialog Act), i.e., a behavior in which the user's state of information or context that is shared in a Dialog changes and is continually updated. Intent is generally named "verb+noun," such as querying weather, booking hotels, etc. While intent recognition is also referred to as intent classification, i.e., classifying user utterances into previously defined intent categories according to the domain and intent to which they relate.
Disclosure of Invention
The invention aims to provide a pre-training large language model tuning-oriented legal planning intent recognition method, which combines the extraction of legal planning opinions and intent recognition based on knowledge enhancement, and realizes tuning recognition and abstract generation of the legal intent by utilizing a legal planning opinion ontology with a prompt instruction and a small sample example attribute.
The invention is realized by the following technical scheme:
A legislated planning intention recognition method for tuning of a pre-training large language model comprises the following steps:
step 1, obtaining text information of the opinion of the legal planning plan, and classifying and grading according to concepts and relations thereof formed by entities, relations and events in the text information to obtain an opinion ontology of the legal planning plan;
Step 2, traversing the knowledge body S of the opinion of the legal planning plan obtained in the analysis step 1, analyzing to obtain a set C of classes and a text T to be processed, and setting LLM tuning prompt instruction generating functions (S, C, T) in a natural language form; the large language model tuning prompt instruction generating function (S, C, T) is shown in the following formula:
LLM tuning instruction generating function (S, C, T)) = [ LLM capability setting hint ] + [ class and attribute traversal template (S, C) ]++ [ small sample example (C) ]+ [ import text hint ] +T+ [ LLM output start hint ]
Wherein, the LLM capability setting prompt is a prompt instruction of extracting capability setting for the elements of the large language model, and the LLM output starting prompt is a prompt instruction of output starting of the large language model;
And 3, cutting the long text of the vertical planning opinion, inputting the cut documents into a large language model, pre-training each cut document by using the large language model, respectively abstracting each cut document by using the large language model, extracting various elements of the vertical planning opinion, generating abstract documents corresponding to the cut documents in batches, splicing the abstract documents in sequence to form a vertical planning opinion sketch text, inputting the vertical planning opinion sketch text into a next-stage large language model, calling the large language model tuning prompt instruction generating function (S, C and T) for executing the step 2, and outputting tuning recognition and abstract generation of the vertical intention of the vertical planning opinion text.
Furthermore, the step 3 supports multitasking pre-training big language model tuning instruction generating functions, and the functions realize batch generation of big language model fine tuning instructions aiming at different tasks based on a guide word rule template.
Further, the method comprises the step of inquiring and traversing the prompt instructions in the arbitrary class attribute C iaj, and generating a multitasking-oriented large language model tuning prompt instruction generating function (S, C, T) in batches aiming at the extraction tasks of specific entities, relations and events.
Further, each class c i includes an ordered list of attributes, as shown in the following formula:
Attributes(ci)＝{cia1,...,ciam}
Where c i is any one of all classes { c 1,...,cn }, and a j is any one of all attributes { a 1,...,am }.
Further, for any one of the attributes a j, a plurality of secondary attributes are further included:
Name (a j): a name representing the attribute;
Multivalued (a j) = { True, false }: whether the value representing a j is a list or a single value;
range (a j): representing the value allowed by the value range of the attribute a j;
Prompt (a j): large language model hint instructions representing classes or attributes;
FewShotExamples (a j) =: a small sample representing a class or attribute.
Further, the step 3 further includes the following processing:
dividing legal and legal long texts to be processed into a plurality of cutting documents, inputting the cutting documents into a large language model for pre-training, generating abstract documents of the cutting documents in batches, and splicing the abstract documents in sequence to form the legal planning opinion shorthand text.
Compared with the prior art, the invention can achieve the following beneficial technical effects:
1) The method has the advantages that more convenient and efficient information extraction is provided, the performance optimization of extracting various types of information facing the opinion of the legislated planning plan is realized, the design reduces the requirement of a large amount of annotation data of the text to be processed, and meanwhile, the extracted information has strong generalization capability and interpretability;
2) Tuning identification of legislation intention is realized by generating tuning instructions in a natural language form in batches;
3) The complexity of training the large language model in the whole process is reduced by using the small sample, and the problem that the length of the large language model processing legislative planning opinion long text is limited is solved.
Drawings
FIG. 1 is a flow chart of the whole intelligent generation method of the legislation opinion aiming at large language model and intention recognition;
FIG. 2 is an exemplary diagram of an ontology of legal planning ideas;
FIG. 3 is an exemplary graph of extraction results of a pre-trained large language model output;
FIG. 4 is a flow chart of the Large Language Model (LLM) tuning hint instruction generation for pre-training of step 2 of the present invention;
Fig. 5 is a flow chart of the legislative intent recognition of the legislative plan intent text of step 3 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly described below with reference to the accompanying drawings in the embodiments of the present invention.
The technical scheme of the invention is as follows:
As shown in fig. 1, the method for intelligently generating the legal planning opinions for large language model and intention recognition of the present invention comprises the following specific steps:
And step 1, classifying and grading key concepts and relations of the text information of the vertical planning opinion, and obtaining an ontology of the legal planning opinion, as shown in fig. 2. The class is defined as follows:
Classes(S)＝{c1,...,cn}
Wherein c 1,...,cn represents all classes 1 to n;
The class corresponds to entity types existing in the ontology of the legal planning plan opinion, and particularly relates to the ontology of the legal planning plan opinion, and the structure of the class comprises a first hierarchy { legal subject, action object, legal (repair) opinion, law regulation, action, rule hypothesis conditions, legal consequences } and a third hierarchy { legal subject, legal purpose, legal program, legal person, legal program, legal principle, legal unit, legal subject, legal content, legal purpose } and a fourth hierarchy { natural person and organization }, which are classified from top to bottom. Relationships between classes include composition, change, correlation, release, compliance and presentation.
Each class c i has an ordered list of attributes, as shown in the following formula:
Attributes(ci)＝{cia1,...,ciam}
Where c i is any one of all classes { c 1,...,cn }, and a j is any one of all attributes { a 1,...,am }. More secondary attributes are included for any one attribute a j. The method specifically comprises the following steps:
Name (a j): a name representing the attribute;
Multivalued (a j) = { True, false }: whether the value representing a j is a list or a single value;
Range (a j): representing the value allowed by the value range of the attribute a j;
Prompt (a j): large language model hint instructions representing classes or attributes; the setting is different from the setting of the traditional knowledge graph body, and aims to provide a convenient prompt for the invention;
FewShotExamples (a j) =: small sample instances representing classes or attributes; the setting is different from the traditional setting of the knowledge graph body with natural language prompt (prompt), and aims to utilize the instruction learning and the small sample learning capacity of the large language model, thereby reducing the required amount of model training data, even only using the reasoning of the large language model without training.
For example: attributes of legal principals include name, principal category. The attributes of the behavior object include the object name. The attributes of the legal (repair) opinion include opinion types. The attributes of the laws and regulations include the name of the regulations, whether or not it is currently valid. The attributes of the behavior include a behavior name and a behavior attribute. The attributes of the treaty assumption conditions include time, place, amount, etc. The legal object is the next level of legal subject, behavioral object. Providing opinion individuals, legislative programs, legislative principles, and providing opinion units as a ranking in the form of an ordered list of legislative (repair) opinions;
In the invention, the legal planning opinion ontology is a knowledge enhancement language model.
Step 2, as shown in fig. 4, analyzing the ontology S of the opinion of the legal planning plan obtained in the step 1, respectively obtaining a set C of classes and a text T to be processed, and setting a Large Language Model (LLM) tuning prompt (prompt) instruction generating function (S, C, T) which is to be used as the pre-training input of the step 3;
The Large Language Model (LLM) tuning prompt instruction generating function (S, C, T) is shown in the following formula:
LLM tuning instruction generating function (S, C, T)) = [ LLM capability setting hint ] + [ class and attribute traversal template (S, C) ]++ [ small sample example (C) ]+ [ import text hint ] +T+ [ LLM output start hint ]
The LLM capability setting prompt is a natural language prompt (prompt) instruction of the element extraction capability setting of a Large Language Model (LLM), and the LLM output starting prompt is a prompt (prompt) instruction of the output starting of the Large Language Model (LLM);
After generating a tuning prompt instruction, inputting the prompt instruction generated by the LLM tuning prompt instruction generating function into a large language model (such as GPT3.5, GPT4, GLM 6B..) to output a structural analysis result, carrying out result comparison and verification according to a legislative planning opinion ontology S, and storing data (comprising structural data and graph data);
The following is a specific embodiment of the law and regulation multitask information extraction process, and extraction of factors such as main bodies, behaviors, behavior objects, legal consequences, valid conditions of regulations, legal abstract, legal attributes and the like in the law and regulation is completed.
Examples of natural language descriptions about [ LLM capability settings hints ] are: "LLM is set as a legal and legal element extraction tool, and element extraction is carried out on the provided text to be extracted according to different definitions of each element. As shown in FIG. 3, the pre-trained large language model outputs csv data, similar to a tabular structured text.
The first row is the name of each element and the first column is the French strip number. The elements to be extracted include: legal subject, behavior, behavioral object, legal outcome, legal condition, legal abstract, legal attribute. "examples of natural language descriptions for [ class and attribute traversal templates (ontologies of opinion of the legal planning scheme) ] are:
Examples of natural language descriptions of [ class and attribute traversal templates (ontologies of legal planning ideas, subjects) ] are: the subject refers to the target to which the law specification is directed, such as individuals, organizations, countries, and the like. The legal provision abstract is to summarize the content to be expressed by one legal provision through a concise and condensed language, and the expressions are unified. Usually in the form of verbs + nouns, and no more than 10 words.
Examples of natural language descriptions of [ class and attribute traversal templates (legal planning plan opinion ontology, behavior) ] are: behavior refers to actions that can produce certain legal consequences. Generally verb, [ small sample example (behavior) ]: such as formulation, approval, implementation, security, etc.
Examples of natural language descriptions of [ class and attribute traversal templates (legal planning plan opinion ontology, behavioral objects) ] are: behavior objects refer to objects, usually nouns, to which behaviors are directed in a legal strip, [ small sample examples (behavior objects) ]: such as policies, regulations, rights, etc.
Examples of natural language descriptions of [ class and attribute traversal templates (legal plan opinion ontology, legal consequences) ] are: legal consequences include affirmative and negative consequences, [ small sample examples (legal consequences) ]: negative consequences typically include abstract expressions such as "blame for correction", "fines", or "legal liabilities" followed. "
Examples of natural language descriptions of [ class and attribute traversal templates (rule planning plan opinion ontology, treaty assumption conditions) ] are: : the treaty assumption condition refers to that the legal provision is in effect when certain conditions (e.g., time, place, amount.) are met.
Examples of natural language descriptions of [ class and attribute traversal templates (legal planning plan opinion ontology, legal attributes) ] are: the legal provision attribute refers to whether the legal provision belongs to imperative terms, forbidden terms, authorized terms, or rights obligation composite terms. Examples of small samples (french strip properties) ]: imperative terms refer to terms under which a person must do some kind of action, which are imperative terms when "must", "should" or similar words appear in the text. Prohibited terms refer to terms that prohibit certain actions from being taken, which are prohibited when "must not," "prohibited," "strictly prohibited," or similar words appear in the text. The terms of authority refer to terms that a person may make or require someone else to do a certain action, which are attributed to when "can", "have rights", "have … …'s freedom", "not violated by … …" or similar words appear in the text. The obligation composite terms refer to terms having both properties of granting rights and setting obligations, mostly the organization and activity rules of the relevant national authorities and their staff.
Examples of natural language descriptions of [ introduce text prompts ] are: "the text to be extracted is as follows: "
Examples of [ text to be processed ] are: "eighth xxxxxxxxxx organization sets up dispatch organization as needed to fulfill responsibilities. xxxxxxxxxx institutions perform unified leadership and management on dispatch institutions. The derivative of xxxxxxxxxx is within the authority of xxxxxxxxxx to fulfill supervisory management responsibilities. "
Examples of natural language descriptions of [ LLM output start prompt ] are: "output: "
In order to improve efficiency, the step also designs a Prompt (Prompt) instruction in the query traversal arbitrary class attribute C iaj, and a processing flow of generating a multitasking-oriented Large Language Model (LLM) tuning Prompt instruction generating function (S, C, T) in batches aiming at the extraction tasks of specific entities, relations and events;
Step 3, as shown in fig. 5, dividing the rule planning project opinion text with length=n into a plurality of divided parts, such as a cutting document 1 and a cutting document 2, wherein m is the maximum processing byte length of a large language model, each cutting document is a smaller text block, inputting the cutting document into a pre-training Large Language Model (LLM), respectively abstracting each cutting document by using the Large Language Model (LLM), extracting various elements of rule planning opinion, generating abstract documents (small texts) corresponding to the cutting document in batches, such as abstract document 1 and abstract document 1, and performing abstract document L in sequence to form rule planning opinion abbreviation text meeting the processing length requirement of the Large Language Model (LLM), inputting the large language model (such as GPT3.5, GPT4 and glm6 b) of the next stage, simultaneously calling the Large Language Model (LLM) tuning prompt generating function (S), and outputting rule planning opinion generating function (S) of the rule planning opinion of step 2;
traversing 'class or attribute large language model Prompt instructions' and 'small sample examples' in the legislation planning plan opinion ontology to generate legislation intention recognition Prompt instructions (promts), wherein the generation mode of the legislation intention recognition Prompt instructions is consistent with the description mode in the step 2; the legislation intention identifies a prompt instruction, which is similar to the tuning instruction extracted by the elements of the prosecution, and is specially introduced with SummaryPrompt, such as ' setting you as an article abstracter, abstracting the intention of the text ', and abstracting the short text after the long text is cut '; and then identifying Prompt instructions Prompt through legislation intention, which is mainly used for classifying the legislation intention and extracting the abstract again for the text spliced by the abstracts of the prosecution, and the form approaches to the extraction of the elements.
As shown in fig. 4, the algorithm framework of this step is shown. The design of the step is that the length of the text of the opinion of the legislation plan is long and is usually larger than the maximum length of the text which can be processed by the prior pre-trained large language model, so that the processing of the long text is difficult to be completed by most large language models at one time. In order to solve the problem, the invention uses a long text mapping simplification method to pretrain with the knowledge body of the legal planning opinion with a prompt instruction and a small sample example attribute as input on the basis of completing the steps 1 and 2.
The related concepts mentioned in the above schemes are described as follows:
An "legislative planning plan opinion ontology" is a tool used to describe the concept and relationship hierarchy of a domain term set, and is intended to help understand the nature and inherent relevance of domain knowledge. In the legislation planning opinion, the purpose of the ontology is to classify and layer the core knowledge of the field to form a comprehensive knowledge system, including opinion categories in terms of legislation planning opinion sources, legislation (repair) intentions, legal standards, legal programs (fact identification, punishment modes, and judgment standards) and the like. The elements of the ontology comprise a plurality of aspects of entities, attributes, relationships, categories and the like, and through the description and grouping of the elements, various information of the planning opinion of the regulations can be effectively and comprehensively represented, so that powerful assistance is provided for the extraction of the entities and the extraction of the relationships.
The intent recognition based on text summaries is a text classification method that can extract information from the summaries of an article to determine the subject and intent of the article. The intention recognition of the article abstract can extract corresponding characteristic information from the abstract of the article by means of a language model, a machine learning algorithm and the like under the condition of not processing a large amount of texts, so that the intention of the article is judged. The method is suitable for application scenes needing to quickly search, screen and induce a large amount of documents or information, such as information search, text classification and other fields. Meanwhile, the method has the advantages of high efficiency, high speed, low cost and the like, and is widely applied to various text analysis and information processing tasks.
Small sample learning (few-shot learning) is an algorithm that applies meta-learning in the field of supervision learning. In few-shot training, the training set only contains a small amount of data, and in the training stage, several categories are randomly extracted in the training set, each category only has several samples, and a meta-task (meta-task) is constructed and used as a support set (support set) input of a model; samples are extracted from the remaining data in these categories as prediction objects (batch sets) of the model. The purpose of the task, called the N-way K-shot problem, is to allow the model to learn how to distinguish these categories from a small amount of data. Few-shot learning aims to enable the model to learn "learning" and to be able to handle similar types of tasks, not just single classification tasks.
The whole flow of the invention has the following innovation points:
(1) The method comprises the steps of creatively providing an ontology description method with Large Language Model (LLM) prompt instruction attributes and small sample example attributes, planning various elements of the opposite laws and regulations, such as { laws and regulations, laws, legal subjects, behaviors, object objects, treaty assumption conditions, legal consequences, legislation (repairing) opinions, legislation purposes, legislation programs and legislation objects @ and relations between the two, and adding specific large language model adjustment prompt instruction attributes and small sample example attributes to form a legislation planning opinion ontology oriented to large language model knowledge enhancement;
(2) The invention innovatively designs a pretrained large language model tuning instruction generating function supporting multitasking, and the function is based on a guide word rule template, so that batch generation of fine tuning instructions of a large language model for different tasks is realized, and additional labeling data and complex training of the large language model are not needed. The natural language instruction is utilized to guide a pre-trained large language model (such as the ChatGPT, GPT4 and LLama-13B of the Qinghai ChatGLM-130B, meta of openAI), so that the performance of the large language model in the task of extracting the legislative planning suggestion information is improved;
(3) The method comprises the steps of creatively utilizing a long text mapping simplification method, firstly cutting off and splitting a long text of an illegal planning suggestion, then utilizing a large language model to carry out abstract abbreviation for limiting word length on the cut text, splicing a plurality of sections of abstract text to form a new full text abstract text, combining a forensic illegal planning suggestion ontology oriented to large language model knowledge enhancement, and identifying and classifying illegal intention and abstracting intention;
(4) In the whole, the invention designs a set of pretrained large language model information extraction and intention recognition technology with enhanced knowledge of the legislation planning plans, and compared with the traditional single-task information extraction, the invention is a complex multi-task-oriented domain information extraction method, is suitable for the extraction of entities, relations and events of the legislation planning plans and the recognition of the legislation intention, and realizes the full coverage of the type of the legislation knowledge.
In summary, the advantages of the present invention are described in detail as follows:
1) The interpretability is strong: the method of the invention generates the large language model tuning instruction in the form of the natural language instruction based on the large language model prompting instruction (Prompt (a)) with class or attribute and the legislative planning opinion ontology of the small sample example (FewShotExamples (a)), which enables people to know the tuning direction of the model, and has better interpretation compared with the original traditional information extraction method.
2) The data requirement is low, and the generalization performance is strong: because the method is based on natural language instructions, the method can process various information extraction tasks without a large amount of annotation data. The ability to support zero-shot learning (zero-shot learning) and small-shot learning (few-shot learning) allows it to be applied to multiple information extraction tasks.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, the present invention is not limited to the description of the above-described technical solutions, and various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the present invention, and such modifications and variations fall within the scope defined by the appended claims.
Claims (6)
1. The legislated planning intention recognition method for optimizing a pre-training large language model is characterized by comprising the following steps of:
Step 1, obtaining text information of the opinion of the legal planning plan, and obtaining hierarchical structured classes and attributes according to concepts and relations in the text information to form an opinion ontology S of the legal planning plan;
Step 2, traversing the knowledge body S of the opinion of the legal planning plan obtained in the step 1, analyzing a set C of classes and a text T to be processed, setting a LLM tuning prompt instruction generating function (S, C, T) in a natural language form, wherein the large language model tuning prompt instruction generating function (S, C, T) is shown in the following formula:
LLM tuning instruction generation function (S, C, T) = [ LLM Capacity setting hint ]
The + [ class and attribute traversal templates (S, C) ] + [ small sample example (C) ] + [ import text prompt ]
+T+ [ LLM output start prompt ]
Wherein, the LLM capability setting prompt is a prompt instruction of extracting capability setting for the elements of the large language model, and the LLM output starting prompt is a prompt instruction of output starting of the large language model;
And 3, dividing the legal and legal long text to be processed into a plurality of cutting documents, inputting the cutting documents into a large language model, respectively abstracting each cutting document by utilizing the large language model, extracting various elements of the legal and legal plan opinions, generating abstract documents corresponding to the cutting documents in batches, splicing the abstract documents in sequence to form the legal and legal plan opinion shorthand text, inputting the legal and legal plan opinion shorthand text into the next-stage large language model, and simultaneously calling the large language model tuning prompt instruction generating function (S, C, T) for executing the step 2, and outputting tuning identification and abstract generation of the legal intention of the legal and legal plan opinion text.
2. The method for identifying the legal planning intent for tuning a pre-trained large language model according to claim 1, wherein the step 3 supports a multitasking pre-trained large language model tuning instruction generating function, and the function realizes batch generation of fine tuning instructions of a large language model for different tasks based on a guide word rule template.
3. The method for identifying the legislated programming intent for tuning a pre-trained large language model according to claim 1, comprising the step of searching through prompt instructions in any type of ci attributes aj, and generating a multi-task-oriented large language model tuning prompt instruction generating function (S, C, T) in batches for extraction tasks of specific entities, relations and events.
4. The method for identifying the intent of the legislated plan for optimizing a pre-trained large language model of claim 1,
Formalized definition is carried out:
Classes(S)＝{c1,...,cn}
The class corresponds to entity class existing in the ontology, and specifically to the legislation planning opinion ontology, and at least comprises laws and regulations, legal strips, legal subjects, behaviors, object objects, legal assumption conditions, legal consequences, legislation/repair opinions, legislation purposes, legislation programs and legislation objects;
Each class ci includes an ordered list of attributes, as shown in the following formula:
Attributes(ci)＝{cia1,...,ciam}
Where ci is any one of all classes { c 1,..cn } and aj is any one of all attributes { a 1,..am }.
5. The method for legislated programming intent recognition for optimization of a pre-trained large language model of claim 4, wherein for any one attribute aj comprises a plurality of secondary attributes:
Name (a j): a name representing the attribute;
Multivalued (a j) = { True, false }: whether the value representing a j is a list or a single value;
range (a j): representing the value allowed by the value range of the attribute a j;
Prompt (a j): large language model hint instructions representing classes or attributes;
FewShotExamples (a j) =: a small sample representing a class or attribute.
6. The method for identifying the intent of the legislation for tuning the pretrained large language model according to claim 1, wherein the step 2 further comprises the steps of inputting the tuning prompt instruction generated by the LLM tuning prompt instruction generating function into the large language model after generating the tuning prompt instruction, outputting the structural analysis result, comparing and checking the result according to the ontology S of the opinion of the legislation planning, and storing the structural data and the graph data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311052216.8A CN117076661B (en) | 2023-08-17 | 2023-08-17 | Legislation planning intention recognition method for tuning of pre-training large language model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311052216.8A CN117076661B (en) | 2023-08-17 | 2023-08-17 | Legislation planning intention recognition method for tuning of pre-training large language model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117076661A CN117076661A (en) | 2023-11-17 |
| CN117076661B true CN117076661B (en) | 2024-05-31 |
Family
ID=88714792
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311052216.8A Active CN117076661B (en) | 2023-08-17 | 2023-08-17 | Legislation planning intention recognition method for tuning of pre-training large language model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117076661B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111444721A (en) * | 2020-05-27 | 2020-07-24 | 南京大学 | Chinese text key information extraction method based on pre-training language model |
| CN114217766A (en) * | 2021-12-15 | 2022-03-22 | 北京工业大学 | Semi-automatic demand extraction method based on pre-training language fine-tuning and dependency characteristics |
| CN116450867A (en) * | 2023-06-15 | 2023-07-18 | 北京枫清科技有限公司 | Graph data semantic search method based on contrast learning and large language model |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113705187B (en) * | 2021-08-13 | 2023-08-01 | 北京百度网讯科技有限公司 | Method and device for generating pre-training language model, electronic equipment and storage medium |
-
2023
- 2023-08-17 CN CN202311052216.8A patent/CN117076661B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111444721A (en) * | 2020-05-27 | 2020-07-24 | 南京大学 | Chinese text key information extraction method based on pre-training language model |
| CN114217766A (en) * | 2021-12-15 | 2022-03-22 | 北京工业大学 | Semi-automatic demand extraction method based on pre-training language fine-tuning and dependency characteristics |
| CN116450867A (en) * | 2023-06-15 | 2023-07-18 | 北京枫清科技有限公司 | Graph data semantic search method based on contrast learning and large language model |
Non-Patent Citations (2)
| Title |
|---|
| A Robust Least Mean M-estimate Adaptive Filtering Algorithm Based on Geometric Algebra for System Identification;Shaohui Lv 等;TWELFTH INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING SYSTEMS;20210714;1-8 * |
| 大语言模型嵌入图书馆知识服务的风险范式与管控策略;罗飞 等;图书与情报;20230625(第3期);99-106 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117076661A (en) | 2023-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108897857B (en) | Chinese text subject sentence generating method facing field | |
| CN111967761A (en) | Monitoring and early warning method and device based on knowledge graph and electronic equipment | |
| CN112183059B (en) | Chinese structured event extraction method | |
| Fan et al. | How to construct a power knowledge graph with dispatching data? | |
| CN113157859A (en) | Event detection method based on upper concept information | |
| CN115310551A (en) | Text analysis model training method and device, electronic equipment and storage medium | |
| Sarwar et al. | StyloThai: A scalable framework for stylometric authorship identification of thai documents | |
| Dias et al. | State of the Art in Artificial Intelligence applied to the Legal Domain | |
| Liu et al. | The extension of domain ontology based on text clustering | |
| CN117076661B (en) | Legislation planning intention recognition method for tuning of pre-training large language model | |
| WO2019043380A1 (en) | Semantic parsing | |
| Santosh et al. | PROO ontology development for learning feature specific sentiment relationship rules on reviews categorisation: a semantic data mining approach | |
| CN115391522A (en) | Text topic modeling method and system based on social platform metadata | |
| CN115129859A (en) | Intention recognition method, intention recognition device, electronic device and storage medium | |
| CN114417008A (en) | Construction engineering field-oriented knowledge graph construction method and system | |
| Rosso-Pelayo et al. | Business process mining and rules detection for unstructured information | |
| Chen | English translation template retrieval based on semantic distance ontology knowledge recognition algorithm | |
| Zhang et al. | A dynamic window split-based approach for extracting professional terms from Chinese courses | |
| Rybak et al. | Machine learning-enhanced text mining as a support tool for research on climate change: theoretical and technical considerations | |
| Gao et al. | Few-shot fake news detection via prompt-based tuning | |
| Wang et al. | The big data analysis and visualization of mass messages under “smart government affairs” based on text mining | |
| Wang et al. | Question answering system of discipline inspection laws and regulations based on knowledge graph | |
| Zhu | A Study on the Method of Press Freedom Value Assessment Based on Artificial Neural Network | |
| Khan | Processing big data with natural semantics and natural language understanding using brain-like approach | |
| Cook et al. | SHAMROQ: Towards semantic models of regulations. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |