CN117575829A - Public opinion propagation modeling simulation and risk early warning method based on large language model - Google Patents

Public opinion propagation modeling simulation and risk early warning method based on large language model Download PDF

Info

Publication number
CN117575829A
CN117575829A CN202311591174.5A CN202311591174A CN117575829A CN 117575829 A CN117575829 A CN 117575829A CN 202311591174 A CN202311591174 A CN 202311591174A CN 117575829 A CN117575829 A CN 117575829A
Authority
CN
China
Prior art keywords
information
self
public opinion
iteration
language model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311591174.5A
Other languages
Chinese (zh)
Inventor
张杨
王超
王永恒
晁玉珊
董世海
李磊
邵彬
肖恒进
董子铭
杨亚飞
韩珺婷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Lab
Original Assignee
Zhejiang Lab
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Lab filed Critical Zhejiang Lab
Priority to CN202311591174.5A priority Critical patent/CN117575829A/en
Publication of CN117575829A publication Critical patent/CN117575829A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/01Social networking
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/004Artificial life, i.e. computing arrangements simulating life
    • G06N3/006Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/04Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Economics (AREA)
  • Human Resources & Organizations (AREA)
  • Software Systems (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • Artificial Intelligence (AREA)
  • General Business, Economics & Management (AREA)
  • Marketing (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Game Theory and Decision Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Quality & Reliability (AREA)
  • Operations Research (AREA)
  • Molecular Biology (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Development Economics (AREA)
  • Primary Health Care (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The present specification discloses a public opinion propagation modeling simulation and risk early warning method based on a large language model, which can acquire event public opinion information, perform fine tuning training on a preset large language model according to the event public opinion information, then initialize a topology network, initialize self characteristics of an agent, determine memory flow information of the agent in each iteration, screen target information from the memory flow information, input self characteristics updated in the previous iteration and the target information into the large language model, obtain self characteristics updated in the current iteration, input self characteristics updated in the current iteration into the large language model, obtain predicted behavior information of the agent aiming at a target event, and update the topology network to perform the next iteration. Finally, risk early warning can be carried out according to the behavior information of each intelligent agent predicted in each round of iteration.

Description

Public opinion propagation modeling simulation and risk early warning method based on large language model
Technical Field
The specification relates to the field of deep learning, in particular to a public opinion propagation modeling simulation and risk early warning method based on a large language model.
Background
With the continuous development of information technology, a large number of users release personal views of events on an online platform, and in practical application, the propagation process of the event public opinion can be analyzed to perform risk early warning.
At present, the analysis method of the public opinion propagation process comprises a big data analysis method and a simulation method, wherein the big data analysis method utilizes huge data volume, and uses technologies such as machine learning and the like to explain the public opinion evolution rule, but mainly focuses on the surface characteristics of the data, and is difficult to mine a deep dynamic process. Compared with a big data analysis method, the simulation method is more suitable for researching the emerging mechanism from microscopic behavior to macroscopic phenomenon in public opinion propagation. The common simulation methods comprise an equation-based simulation method and an agent-based simulation method, however, the existing simulation methods still have certain limitations, namely that the simulation results depend on the initial parameter setting and the rule formulation too much, and the simulation results cannot simulate the diversity behaviors of users on social media.
In order to make up for the defects in the prior simulation method, the discovery provides a public opinion propagation modeling simulation and risk early warning method based on a large language model, the intelligent body is modeled by applying the large language model, and the public opinion propagation process is deduced on the basis, so that a new path is provided for researching the public opinion propagation.
Disclosure of Invention
The specification provides a public opinion propagation modeling simulation and risk early warning method based on a large language model to partially solve the problems existing in the prior art.
The technical scheme adopted in the specification is as follows:
the specification provides a public opinion propagation modeling simulation and risk early warning method based on a large language model, which comprises the following steps:
acquiring event public opinion information, and performing fine tuning training on a preset large language model according to the event public opinion information;
initializing a topology network and initializing self-characteristics of the agents, wherein the topology network is used for representing social relations among the agents, nodes in the topology network are used for representing the agents, one agent is used for simulating one user, and the self-characteristics comprise: at least one of a self-confidence score, a self-constraining willingness score;
in each iteration, determining memory flow information of an intelligent agent, screening target information from the memory flow information, and inputting self characteristics obtained by updating in the previous iteration and the target information into the large language model to obtain self characteristics obtained by updating in the current iteration, wherein the target information comprises: memory information associated with a target event and memory information corresponding to neighbor agents with influence on the agents determined through the topology network;
Inputting self characteristics obtained by updating in the round of iteration into the large language model to obtain predicted behavior information of the intelligent agent aiming at the target event, and updating the topology network according to the behavior information to perform the next round of iteration;
and carrying out risk early warning according to the behavior information of each intelligent agent predicted in each round of iteration.
Optionally, obtaining event public opinion information specifically includes:
acquiring basic information of a user, participated public opinion events and actual comments published aiming at the public opinion events as event public opinion information;
performing fine tuning training on a preset large language model according to the event public opinion information, wherein the fine tuning training comprises the following steps of:
inputting basic information of a user and participated public opinion events into a preset large language model, inputting actual comments of the user aiming at the public opinion events into the large language model as an optimization target, and performing fine tuning training on the large language model.
Optionally, initializing the topology network specifically includes:
and initializing a topological network according to the acquired actual user relationship and the attraction score corresponding to each agent.
Optionally, after initializing the topology network, the method further comprises:
Determining an influence value corresponding to each intelligent agent according to the topology network;
through the topological network, determining memory information of neighbor agents with influence on the agents specifically comprises the following steps:
and determining memory information corresponding to neighbor agents with influence on the agents according to the influence value corresponding to each agent determined by the topology network.
Optionally, determining, according to the topology network, an influence value corresponding to each agent specifically includes:
determining an adjacent matrix of the topological network, and normalizing the adjacent matrix to obtain a normalized adjacent matrix;
and according to the standardized adjacency matrix, the influence value corresponding to each agent.
Optionally, determining memory flow information of the agent specifically includes:
build set B i The set B i Representing a set of perspectives of other agents perceived by agent i on a target event;
according to public opinion event set M and set B i And determining memory flow information of the intelligent agent i, wherein the public opinion event set M comprises time, public opinion event content and action activities adopted by each user.
Optionally, inputting the self-feature updated in the previous iteration and the target information into the large language model to obtain the self-feature updated in the present iteration, which specifically includes:
Inputting the self-constraint willingness score updated in the previous iteration and the memory information associated with the target event into the large language model to obtain the self-constraint willingness score updated in the current iteration;
and inputting the self-confidence score updated in the previous iteration and the memory information corresponding to the neighbor agent with influence on the agent determined by the topology network into the large language model to obtain the self-confidence score updated in the current iteration.
The specification provides a public opinion propagation modeling simulation and risk early warning device based on a large language model, which comprises the following components:
the fine tuning module is used for acquiring event public opinion information and carrying out fine tuning training on a preset large language model according to the event public opinion information;
the system comprises an initialization module, a topology network and a self-feature generation module, wherein the initialization module is used for initializing a topology network and initializing the self-feature of an agent, the topology network is used for representing social relations among agents, nodes in the topology network are used for representing the agents, one agent is used for simulating one user, and the self-feature comprises: at least one of a self-confidence score, a self-constraining willingness score;
The screening module is used for determining memory flow information of the intelligent agent in each round of iteration, screening target information from the memory flow information, inputting self characteristics obtained by updating in the previous round of iteration and the target information into the large language model, and obtaining self characteristics obtained by updating in the current round of iteration, wherein the target information comprises: memory information associated with a target event and memory information corresponding to neighbor agents with influence on the agents determined through the topology network;
the updating module is used for inputting self characteristics obtained by updating in the round of iteration into the large language model to obtain predicted behavior information of the intelligent agent aiming at the target event, and updating the topology network according to the behavior information so as to carry out the next round of iteration;
and the early warning module is used for carrying out risk early warning according to the behavior information of each intelligent agent predicted in each round of iteration.
The present specification provides a computer readable storage medium storing a computer program which when executed by a processor implements the above-described large language model-based public opinion propagation modeling simulation and risk early warning method.
The present specification provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the above-described large language model-based public opinion propagation modeling simulation and risk early warning method when executing the program.
The above-mentioned at least one technical scheme that this specification adopted can reach following beneficial effect:
from the above-mentioned public opinion propagation modeling simulation and risk early warning based on a big language model, it can be seen that event public opinion information is obtained, and a preset big language model is subjected to fine tuning training according to the event public opinion information, then a topology network can be initialized, and self features of the agents are initialized, where the topology network is used for representing social relations between the agents, a node in the topology network is used for representing the agents, and an agent is used for simulating a user, and the self features include: at least one of self-confidence score and self-constraint willingness score, in each iteration, determining memory flow information of an agent, screening target information from the memory flow information, and inputting self-characteristics updated in the previous iteration and the target information into the large language model to obtain self-characteristics updated in the present iteration, wherein the target information comprises: and the memory information associated with the target event and the memory information corresponding to the neighbor intelligent agent with influence on the intelligent agent, which is determined by the topology network, are input into a large language model by self characteristics updated in the round of iteration to obtain predicted behavior information of the intelligent agent for the target event, and the topology network is updated according to the behavior information so as to carry out the next round of iteration. Finally, risk early warning can be carried out according to the behavior information of each intelligent agent predicted in each round of iteration.
Compared with the prior art, the invention has the advantages and positive effects that:
1. by constructing a scaleless network, the method can better capture the relationship between social media users, and the reality condition of social relationship change can be reflected by utilizing the broken edge reconnection strategy, so that the influence degree of the opinion of the intelligent agent on the network public opinion atmosphere can be better studied.
2. Modeling the agent by using a large language model to deduce the behavior of the user on social media, such as: praise, forwarding, commenting and the like, and realizing the true restoration of the evolution process of public opinion.
3. By fine tuning the large language model by real data of social media users and public opinion events, more reasonable agent response can be generated, and the simulation fidelity is improved.
4. After each intelligent agent reacts, the network structure is updated and iterated, so that the dynamic process of public opinion propagation is better simulated and analyzed, and the timeliness, accuracy and authenticity of research are ensured.
Drawings
The accompanying drawings, which are included to provide a further understanding of the specification, illustrate and explain the exemplary embodiments of the present specification and their description, are not intended to limit the specification unduly. In the drawings:
FIG. 1 is a schematic flow chart of a large language model-based public opinion propagation modeling simulation and risk early warning method provided in the present specification;
FIG. 2 is a schematic flow chart of determining an event with highest correlation to a target event provided in the present specification;
FIG. 3 is a schematic flow chart of determining an influence value provided in the present specification;
FIG. 4 is a schematic diagram of a large language model-based public opinion propagation modeling simulation and risk early warning device provided in the present specification;
fig. 5 is a schematic view of the electronic device corresponding to fig. 1 provided in the present specification.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the present specification more apparent, the technical solutions of the present specification will be clearly and completely described below with reference to specific embodiments of the present specification and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present specification. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.
The following describes in detail the technical solutions provided by the embodiments of the present specification with reference to the accompanying drawings.
Fig. 1 is a schematic flow chart of a large language model-based public opinion propagation modeling simulation and risk early warning method provided in the present specification, specifically including the following steps:
s100: and acquiring event public opinion information, and performing fine tuning training on a preset large language model according to the event public opinion information.
S102: initializing a topology network and initializing self-characteristics of the agents, wherein the topology network is used for representing social relations among the agents, nodes in the topology network are used for representing the agents, one agent is used for simulating one user, and the self-characteristics comprise: at least one of self-confidence score and self-constraining willingness score.
In the specification, the transmission process of the public opinion of the event can be simulated, so that after a new event arrives, risk early warning can be carried out on the event by simulating the transmission process of the public opinion of the event.
Firstly, event public opinion information can be obtained, and fine tuning training is carried out on a preset large language model according to the event public opinion information.
Wherein, in fine tuning training: basic information of a user, participated public opinion events and actual comments published aiming at the public opinion events can be obtained, the basic information of the user and the participated public opinion events are input into a preset large language model, the large language model is used for inputting the actual comments published by the user aiming at the public opinion events as an optimization target, and fine tuning training is carried out on the large language model.
It should be noted that, during fine tuning, the collected data may be preprocessed, including operations such as deduplication, cleaning, word segmentation, and the like. After finishing the data, storing the data as a json file, thereby facilitating the subsequent use.
And fine tuning by a LoRA method, firstly freezing pre-trained model weights, then injecting a trainable rank decomposition matrix into each layer of a transducer architecture, training, integrating the model weights after LoRA training into the original model weights of Baichuan-13B, and obtaining the Baichuan-13B model after fine tuning.
The server may then initialize a topology network for representing social relationships between agents, and initialize self-characteristics of the agents, wherein nodes in the topology network are used for representing agents, one agent is used for simulating one user, the self-characteristics comprising: self confidence score, self-constraining willingness score, etc.
When initializing self-characteristics of the agent, 30 public opinion events which participate in discussion of ten thousands of people can be selected from social media, 100 users (corresponding to the agent) are selected from the participants, the 100 users are required to participate in at least 15 or more selected public opinion events, and basic information (such as account name, filled age and the like) of the users is obtained.
The self-confidence score is initialized, the self-confidence measures the self-certainty degree of an agent to the opinion, the confidence of the agent is influenced by most of the opinion observed by the agent, and a value between [0,1] is randomly generated as the self-confidence score of the agent by using Gaussian distribution.
Initializing self-constraint willingness scores, wherein self-constraint definition in the public opinion transmission process is to hide true views from audiences who do not agree with the self-constraint willingness, and measuring the degree of self-constraint.
Initializing activity: calculating the activity degree of the intelligent agent by using the participation degree of the intelligent agent to the public opinion event, constructing a matrix, representing the intelligent agent by using rows, representing the public opinion event by using columns, wherein the value in the matrix represents the participation degree (sum of praise number, forwarding number and comment number) of the intelligent agent in the corresponding event, if the intelligent agent does not participate, the participation degree is recorded as 0, the participation degree of each column corresponding to each intelligent agent is added to obtain a comprehensive score, and then normalization is carried out to obtain scores between 100 [0,1] as the activity degree of each intelligent agent.
The initialized self-confidence score, the self-constraint willingness score and the liveness mentioned above can be all used as initial characteristic values corresponding to the intelligent agent.
And determining a public opinion event set M in which each agent participates, wherein each event M in the public opinion event set M consists of time, public opinion event content and action activities (including praise, forwarding, comment content or silence and the like) taken by each user.
The topology network mentioned above may be a scaleless network, and when initializing the topology network, the topology network may be initialized according to the obtained actual user relationship (such as user's attention to other users, praise, comment, etc.) and the corresponding attraction score of each agent.
Here we define several mathematical symbols: the topology network consisting of 100 agents is denoted by G. G= (V, E), v= {1,2, … } is a set of all agents, e= { { i, j }:i, j E V } represents a connection between agents, N (θ) represents a set of all critical points of one agent, i.e., N (i) = { j E V (i, j) E }, the assumption that the topology network needs to satisfy is as follows:
(1) The network remains static in the process of forming the opinion of the intelligent agent, namely, each time iteration, and the network structure is not updated until one time iteration is completed under the assumption that the opinion formation speed is faster than the change of the social network in the real world;
(2) The network model is able to study the influence of different network topologies, in particular network densities and participant positions, on public opinion by changing parameters (which may include the number of network nodes, network densities, etc.).
The topology network structure is constructed as follows:
s2.1 connection node: in a scaleless network, the connections between nodes are not established randomly, but are established based on the attractive force scores between nodes, which are formulated as:
A i =β(γ-2)+q i
where β refers to the number of edges connected by each new node. Every time a new node joins the network, it will connect beta edges with the existing node, gamma is the power law index, q i Is an attraction score for agent initialization;
s2.2, reconnecting broken edges: the social network structure is not invariable, so that the network structure is disconnected and reconnected along with the iteration of an event in the modeling process, the actions of getting a relationship on social media and the like are simulated, the basic idea of disconnected and reconnected is that the connection mode of a network is changed on the basis of not changing network nodes, and the propagation process of public opinion on the social media can be better simulated by observing the influence of disconnected and reconnected on information propagation, range and path.
S104: in each iteration, determining memory flow information of an intelligent agent, screening target information from the memory flow information, and inputting self characteristics obtained by updating in the previous iteration and the target information into the large language model to obtain self characteristics obtained by updating in the current iteration, wherein the target information comprises: and the memory information is associated with the target event, and the memory information corresponding to the neighbor intelligent agent with influence on the intelligent agent is determined through the topology network.
S106: and inputting the self characteristics updated in the round of iteration into the large language model to obtain predicted behavior information of the intelligent agent aiming at the target event, and updating the topology network according to the behavior information so as to perform the next round of iteration.
In the subsequent process, the simulation of the event public opinion development process needs to be carried out in an iterative mode.
Specifically, in each iteration, memory flow information of the agent is determined, target information is screened out from the memory flow information, and self characteristics updated in the previous iteration and the target information are input into the large language model to obtain self characteristics updated in the iteration.
It should be noted that the memory flow information may represent the event that the agent experiences before the iteration and the memory of the view related to the event.
Specifically, set B can be constructed i ={b 1 ,b 2 ,…b i-1 ,b i+1 ,…b 100 Set B i For representing a set of perspectives of other agents perceived by agent i on a target event H, wherein b i The perspective of agent i on target event H is shown. Wherein the set Bi needs to be initialized at the first iteration, and each subsequent iteration can be generated by a large language model at the previous moment.
Then, according to the public opinion event set M and set B i And determining memory flow information of the intelligent agent i.
The format of each piece of memory stream information is as follows: agent, public opinion event content, time of participation, behavioral activity.
After the memory flow information of each agent is determined, for one agent, the target information can be screened out from the memory flow information, the target information and the self characteristics updated in the previous iteration are input into a large language model, the self characteristics updated in the current iteration are obtained, and then the self characteristics updated in the current iteration are input into the large language model, so that the predicted behavior information of the agent aiming at the target event is obtained.
The behavior information may include information on whether the agent expresses a viewpoint on the target event, a specific content of the viewpoint if the agent expresses the viewpoint on the target event, and the like.
Then, the topology network can be updated according to the behavior information, and the next iteration is carried out.
It should be noted that, the above-mentioned screened target information may include memory information associated with the target event, memory information corresponding to a neighbor agent having an influence on the agent determined through the topology network, and the like.
Wherein the screening can be performed by:
1. the information of the event with the highest correlation with the target event can be screened out from the public opinion event set M and used as the memory information associated with the target event.
Fig. 2 is a schematic flow chart of determining an event with highest correlation with a target event provided in the present specification.
As shown in fig. 2, the embedded vector of each public opinion event in the public opinion event set M and the embedded vector of the target event can be determined through the large language model, and then the similarity of the embedded vectors between the public opinion event and the target event is determined.
It should be noted that the influence of the public opinion events decays with time, so that the influence value of the public opinion events in time can be determined according to the occurrence time of each public opinion event.
The similarity of the embedded vector between the public opinion event and the target event and the influence value of the public opinion event in time can be combined, so that top-k public opinion events are selected and used as the event with the highest correlation with the target event, and corresponding memory information can be screened from the public opinion event set M.
In fig. 2, there are 30 public opinion events, and 20 events are selected from the 30 public opinion events for illustration, it can be seen that the similarity between each public opinion event and the target event can be calculated first, then the influence value (the more recent the time is, the larger the influence value) of the 30 public opinion events in time is calculated, wherein the influence value can be determined by an exponential retrieval function (which can be a time decay function), then the weighted summation (the weights can be set to be the same or set according to the requirement) can be performed on the influence value of the similarity and the public opinion event in time, so that the 20 events with the highest correlation can be selected according to the obtained summation result.
2. The influence value corresponding to each intelligent agent can be determined according to the topology network, and the influence value corresponding to each intelligent agent determined by the topology network is selected from the set B i And determining the memory information corresponding to the neighbor agent with influence on the agent.
When the influence value corresponding to each agent is determined through the topology network, an adjacent matrix of the topology network can be determined, the adjacent matrix is standardized, a standardized adjacent matrix is obtained, and the influence value corresponding to each agent is obtained according to the standardized adjacent matrix.
Specifically, as shown in fig. 3, the following steps may be performed:
fig. 3 is a schematic flow chart of determining an influence value provided in the present specification.
S3.1 determines an adjacency matrix, representing the scaleless network as adjacency matrix L, wherein L [ i, j ] represents whether there is a connection between nodes i and j, and if there is a value of 1, otherwise 0.
S3.2, a degree matrix D is constructed, wherein D [ i, i ] represents the degree of the node i (the number of edges connected with the node i), and then a standardized adjacency matrix P is constructed, wherein P [ i ] [ j ] = L [ i ] [ j ]/D [ i ] [ i ].
S3.3, initializing a feature vector x, wherein x [ i ] represents an initial feature value of the node i, the initial feature value can comprise the activity degree, the self-constraint willingness score, the self-confidence score and the like, and the initial feature value can be randomly initialized or can be obtained through the real behavior of a user in a social network site.
S3.4 iterates, updating the feature vector, and in each iteration, x=p×x.
S3.5, obtaining a result: after each iteration, calculating the change result of the feature vector x, and when the Euclidean distance between two iterations is smaller than 1e-6, considering the feature vector as the feature vector centrality of the node, wherein x [ i ] represents the influence of the intelligent agent i in the network at the moment and can be used as the influence value of the intelligent agent i.
The self-confidence score and self-constraining willingness score in the agent's own features can then be updated through a large language model.
Specifically, the self-constraint willingness score updated in the previous iteration and the memory information related to the target event can be input into a large language model to obtain the self-constraint willingness score updated in the previous iteration, and the self-confidence score updated in the previous iteration and the memory information corresponding to the neighbor agent with influence on the agent determined by the topology network are input into the large language model to obtain the self-confidence score updated in the previous iteration.
When the updated self-constraint willingness score and self-confidence score are obtained, the self-characteristics obtained by updating in the round of iteration can be input into a large language model to obtain predicted behavior information of the intelligent agent aiming at the target event, and the topology network is updated according to the behavior information so as to carry out the next round of iteration.
Specific examples of the hint statement input to the large language model are given herein (but the form of the hint statement input to the large language model in practice is not limited).
For self-confidence scores: given that the self-confidence before the small eyesight is 8 points, for the thing that the small child falls into the water (the target event), he considers that the small red is not the person who pays attention to the list, considers that the small white is the vermicelli, and also considers that the small red is the person who pays attention to the small child, and please help me correct the self-confidence score of the small eyesight at the next moment.
Constraint score for self willingness: knowing that the self-constrained willingness initialization score at this time step is 3 points, he previously published a perspective for both event a and event B, and that for the target event, please correct his self-willingness expression score.
Behavior information for a predicted agent for a target event: knowing the self-characteristics M of the mins, it is known that a public opinion event H (target event) occurred at this time, please you speculate that the mins' attitudes to the event in the social platform are praise, forward, silence, or comment. The comment can adopt behavior attitudes of praise, forwarding or comment at the same time, but if silence is selected, other behavior attitudes cannot be selected, and if comment is selected, comment content of the comment is output.
S108: and carrying out risk early warning according to the behavior information of each intelligent agent predicted in each round of iteration.
After the behavior information of each agent predicted in each iteration is determined, risk early warning can be performed according to the behavior information of each agent predicted in each iteration.
Specifically, the behavior information of each agent predicted in each round of iteration can be used as a simulated public opinion propagation process, and through analysis, the change trend of the response of the agent to the public opinion event on social media along with time and the propagation and evolution process of the public opinion in the network along with the behavior activity and the change of the social relationship of different agents can be observed.
Therefore, the process of the public opinion from occurrence to climax to decay can be simulated through the method, so that whether the risk early warning needs to be carried out on the target event or not is determined, and the risk control of the related mechanism on the target event is facilitated.
Specifically, whether the target event has a risk causing public opinion crisis or not can be determined, risk which is easy to cause adverse effects is easily caused, risk early warning is performed, and whether the target event has a risk can be measured through the negative evaluation number or the negative evaluation proportion of the simulated target event in a period of time.
For convenience of description, the execution subject for executing the method is described as a server, and the execution subject of the method may be a computer, a large-scale service platform, or the like, which is not limited herein. The features of the following examples and embodiments may be combined with each other without any conflict.
In addition, all the actions for acquiring signals, information or data in the present specification are performed under the condition of conforming to the corresponding data protection rule policy of the place and obtaining the authorization given by the corresponding device owner.
The above method for modeling and simulating public opinion propagation and risk early warning based on a large language model provided in one or more embodiments of the present specification further provides a device for modeling and simulating public opinion propagation and risk early warning based on a large language model based on the same thought, as shown in fig. 4.
Fig. 4 is a schematic diagram of a public opinion propagation modeling simulation and risk early warning device based on a large language model provided in the present specification, including:
the fine tuning module 401 is configured to obtain event public opinion information, and perform fine tuning training on a preset large language model according to the event public opinion information;
An initialization module 402, configured to initialize a topology network, where nodes in the topology network are used to represent agents, and initialize self-characteristics of agents, where the self-characteristics include: at least one of a self-confidence score, a self-constraining willingness score;
the screening module 403 is configured to determine memory flow information of an agent in each iteration, screen target information from the memory flow information, and input self features updated in a previous iteration and the target information into the large language model to obtain self features updated in a current iteration, where the target information includes: memory information associated with a target event and memory information corresponding to neighbor agents with influence on the agents determined through the topology network;
the updating module 404 is configured to input the self-feature updated in the current iteration to the large language model, obtain predicted behavior information of the agent for the target event, and update the topology network according to the behavior information, so as to perform the next iteration;
And the early warning module 405 is configured to perform risk early warning according to the behavior information of each agent predicted in each iteration.
Optionally, the fine tuning module 401 is configured to obtain basic information of a user, a participating public opinion event, and an actual comment published for the public opinion event as event public opinion information; inputting basic information of a user and participated public opinion events into a preset large language model, inputting actual comments of the user aiming at the public opinion events into the large language model as an optimization target, and performing fine tuning training on the large language model.
Optionally, the initializing module 402 is configured to initialize the topology network according to the obtained actual user relationship and the attraction score corresponding to each agent.
Optionally, the screening module 403 is configured to determine, according to the topology network, an impact value corresponding to each agent; and determining memory information corresponding to neighbor agents with influence on the agents according to the influence value corresponding to each agent determined by the topology network.
Optionally, the screening module 403 is configured to determine an adjacency matrix of the topology network, and normalize the adjacency matrix to obtain a normalized adjacency matrix; and according to the standardized adjacency matrix, the influence value corresponding to each agent.
Optionally, the filtering module 403 is configured to construct a set B i The set B i Representing a set of perspectives of other agents perceived by agent i on a target event; according to public opinion event set M and set B i And determining memory flow information of the intelligent agent i, wherein the public opinion event set M comprises time, public opinion event content and action activities adopted by each user.
Optionally, the updating module 404 is configured to input the self-constraint willingness score updated in the previous iteration and the memory information associated with the target event into the large language model, so as to obtain the self-constraint willingness score updated in the current iteration; and inputting the self-confidence score updated in the previous iteration and the memory information corresponding to the neighbor agent with influence on the agent determined by the topology network into the large language model to obtain the self-confidence score updated in the current iteration.
The present specification also provides a computer-readable storage medium storing a computer program operable to perform the above-described large language model-based public opinion propagation modeling simulation and risk early warning method.
The present specification also provides a schematic structural diagram of the electronic device shown in fig. 5. At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile storage, as illustrated in fig. 5, although other hardware required by other services may be included. The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to realize the public opinion propagation modeling simulation and risk early warning method based on the large language model.
Of course, other implementations, such as logic devices or combinations of hardware and software, are not excluded from the present description, that is, the execution subject of the following processing flows is not limited to each logic unit, but may be hardware or logic devices.
In the 90 s of the 20 th century, improvements to one technology could clearly be distinguished as improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable Gate Array, FPGA)) is an integrated circuit whose logic function is determined by the programming of the device by a user. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented by using "logic compiler" software, which is similar to the software compiler used in program development and writing, and the original code before the compiling is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but not just one of the hdds, but a plurality of kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), lava, lola, myHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic controllers, and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in one or more software and/or hardware elements when implemented in the present specification.
It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing is merely exemplary of the present disclosure and is not intended to limit the disclosure. Various modifications and alterations to this specification will become apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, or the like, which are within the spirit and principles of the present description, are intended to be included within the scope of the claims of the present description.

Claims (10)

1. A public opinion propagation modeling simulation and risk early warning method based on a large language model is characterized by comprising the following steps:
acquiring event public opinion information, and performing fine tuning training on a preset large language model according to the event public opinion information;
initializing a topology network and initializing self-characteristics of the agents, wherein the topology network is used for representing social relations among the agents, nodes in the topology network are used for representing the agents, one agent is used for simulating one user, and the self-characteristics comprise: at least one of a self-confidence score, a self-constraining willingness score;
In each iteration, determining memory flow information of an intelligent agent, screening target information from the memory flow information, and inputting self characteristics obtained by updating in the previous iteration and the target information into the large language model to obtain self characteristics obtained by updating in the current iteration, wherein the target information comprises: memory information associated with a target event and memory information corresponding to neighbor agents with influence on the agents determined through the topology network;
inputting self characteristics obtained by updating in the round of iteration into the large language model to obtain predicted behavior information of the intelligent agent aiming at the target event, and updating the topology network according to the behavior information to perform the next round of iteration;
and carrying out risk early warning according to the behavior information of each intelligent agent predicted in each round of iteration.
2. The method of claim 1, wherein obtaining event public opinion information comprises:
acquiring basic information of a user, participated public opinion events and actual comments published aiming at the public opinion events as event public opinion information;
performing fine tuning training on a preset large language model according to the event public opinion information, wherein the fine tuning training comprises the following steps of:
Inputting basic information of a user and participated public opinion events into a preset large language model, inputting actual comments of the user aiming at the public opinion events into the large language model as an optimization target, and performing fine tuning training on the large language model.
3. The method according to claim 1, wherein initializing the topology network comprises:
and initializing a topological network according to the acquired actual user relationship and the attraction score corresponding to each agent.
4. The method of claim 1, wherein after initializing the topology network, the method further comprises:
determining an influence value corresponding to each intelligent agent according to the topology network;
through the topological network, determining memory information of neighbor agents with influence on the agents specifically comprises the following steps:
and determining memory information corresponding to neighbor agents with influence on the agents according to the influence value corresponding to each agent determined by the topology network.
5. The method of claim 4, wherein determining the impact value corresponding to each agent according to the topology network, specifically comprises:
Determining an adjacent matrix of the topological network, and normalizing the adjacent matrix to obtain a normalized adjacent matrix;
and according to the standardized adjacency matrix, the influence value corresponding to each agent.
6. The method of claim 1, wherein determining memory flow information for an agent, in particular, comprises:
build set B i The set B i Representing a set of perspectives of other agents perceived by agent i on a target event;
according to public opinion event set M and set B i And determining memory flow information of the intelligent agent i, wherein the public opinion event set M comprises time, public opinion event content and action activities adopted by each user.
7. The method of claim 1, wherein inputting the self-feature updated in the previous iteration and the target information into the large language model to obtain the self-feature updated in the current iteration, specifically comprises:
inputting the self-constraint willingness score updated in the previous iteration and the memory information associated with the target event into the large language model to obtain the self-constraint willingness score updated in the current iteration;
and inputting the self-confidence score updated in the previous iteration and the memory information corresponding to the neighbor agent with influence on the agent determined by the topology network into the large language model to obtain the self-confidence score updated in the current iteration.
8. Public opinion propagation modeling simulation and risk early warning device based on big language model, characterized by comprising:
the fine tuning module is used for acquiring event public opinion information and carrying out fine tuning training on a preset large language model according to the event public opinion information;
the system comprises an initialization module, a topology network and a self-feature generation module, wherein the initialization module is used for initializing a topology network and initializing the self-feature of an agent, the topology network is used for representing social relations among agents, nodes in the topology network are used for representing the agents, one agent is used for simulating one user, and the self-feature comprises: at least one of a self-confidence score, a self-constraining willingness score;
the screening module is used for determining memory flow information of the intelligent agent in each round of iteration, screening target information from the memory flow information, inputting self characteristics obtained by updating in the previous round of iteration and the target information into the large language model, and obtaining self characteristics obtained by updating in the current round of iteration, wherein the target information comprises: memory information associated with a target event and memory information corresponding to neighbor agents with influence on the agents determined through the topology network;
The updating module is used for inputting self characteristics obtained by updating in the round of iteration into the large language model to obtain predicted behavior information of the intelligent agent aiming at the target event, and updating the topology network according to the behavior information so as to carry out the next round of iteration;
and the early warning module is used for carrying out risk early warning according to the behavior information of each intelligent agent predicted in each round of iteration.
9. A computer-readable storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the method of any of the preceding claims 1-7.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of the preceding claims 1-7 when executing the program.
CN202311591174.5A 2023-11-24 2023-11-24 Public opinion propagation modeling simulation and risk early warning method based on large language model Pending CN117575829A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311591174.5A CN117575829A (en) 2023-11-24 2023-11-24 Public opinion propagation modeling simulation and risk early warning method based on large language model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311591174.5A CN117575829A (en) 2023-11-24 2023-11-24 Public opinion propagation modeling simulation and risk early warning method based on large language model

Publications (1)

Publication Number Publication Date
CN117575829A true CN117575829A (en) 2024-02-20

Family

ID=89887883

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311591174.5A Pending CN117575829A (en) 2023-11-24 2023-11-24 Public opinion propagation modeling simulation and risk early warning method based on large language model

Country Status (1)

Country Link
CN (1) CN117575829A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107038178A (en) * 2016-08-03 2017-08-11 平安科技(深圳)有限公司 The analysis of public opinion method and apparatus
CN109064347A (en) * 2017-06-11 2018-12-21 南京理工大学 Information based on multiple agent is propagated and public sentiment evolution simulation method
CN115269950A (en) * 2022-06-07 2022-11-01 青岛理工大学 Public opinion information content mining and propagation monitoring analysis method
US20220358369A1 (en) * 2019-07-08 2022-11-10 Soul Machines Limited Memory in embodied agents

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107038178A (en) * 2016-08-03 2017-08-11 平安科技(深圳)有限公司 The analysis of public opinion method and apparatus
CN109064347A (en) * 2017-06-11 2018-12-21 南京理工大学 Information based on multiple agent is propagated and public sentiment evolution simulation method
US20220358369A1 (en) * 2019-07-08 2022-11-10 Soul Machines Limited Memory in embodied agents
CN115269950A (en) * 2022-06-07 2022-11-01 青岛理工大学 Public opinion information content mining and propagation monitoring analysis method

Similar Documents

Publication Publication Date Title
Denning et al. The profession of IT computing's paradigm
CN111176758B (en) Configuration parameter recommendation method and device, terminal and storage medium
Anthes Deep learning comes of age
CN113792937A (en) Social network influence prediction method and device based on graph neural network
CN113158608A (en) Processing method, device and equipment for determining parameters of analog circuit and storage medium
CN111401557B (en) Agent decision making method, AI model training method, server and medium
CN109903100A (en) A kind of customer churn prediction technique, device and readable storage medium storing program for executing
CN115860107B (en) Multi-machine searching method and system based on multi-agent deep reinforcement learning
Sprevak Three challenges to Chalmers on computational implementation
CN111611703B (en) Sand table deduction method, device and equipment based on digital twin and storage medium
CN111282272A (en) Information processing method, computer readable medium and electronic device
CN113962390A (en) Method for constructing diversified search strategy model based on deep reinforcement learning network
Hölldobler et al. Lessons Learned from AlphaGo.
CN116151363A (en) Distributed reinforcement learning system
US9224088B2 (en) Methods, systems, and computer program products for simulating a scenario by updating events over a time window including the past, present, and future
CN114404977A (en) Training method of behavior model and training method of structure expansion model
Whitworth The emergence of the physical world from information processing
CN117575829A (en) Public opinion propagation modeling simulation and risk early warning method based on large language model
Jaschek et al. Mysterious murder-mcts-driven murder mystery generation
CN112274935A (en) AI model training method, use method, computer device and storage medium
CN116977661A (en) Data processing method, device, equipment, storage medium and program product
US8280707B2 (en) Methods, systems, and computer program products for simulating a scenario by updating events over a time window including the past, present, and future
CN116943220A (en) Game artificial intelligence control method, device, equipment and storage medium
Snodgrass Markov models for procedural content generation
Sequeira et al. Simplifying crowd automation in the virtual laboratory of archaeology

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination