CN118297133A

CN118297133A - User service method and device based on LLM model

Info

Publication number: CN118297133A
Application number: CN202410426730.1A
Authority: CN
Inventors: 宋宜轩; 邹银超
Original assignee: Alipay Hangzhou Information Technology Co Ltd
Current assignee: Alipay Hangzhou Information Technology Co Ltd
Priority date: 2024-04-09
Filing date: 2024-04-09
Publication date: 2024-07-05

Abstract

A user service method based on LLM model, the said method is applied to the intelligent service terminal; the intelligent service terminal is provided with at least one LLM service model; the LLM service model is obtained by performing fine tuning training on a LLM basic model based on user service data corresponding to a service scene; comprising the following steps: acquiring a target LLM service model corresponding to a target service scene designated by a user from at least one LLM service model; responding to a service request of a user, operating a target LLM service model to provide user service corresponding to a target service scene for the user, and collecting a user service data set generated in the process of providing the user service for the user; and further performing fine tuning training on the target LLM service model based on the user service data set to obtain a personalized LLM service model corresponding to the user.

Description

User service method and device based on LLM model

Technical Field

The embodiment of the specification relates to the technical field of artificial intelligence, in particular to a user service method and device based on an LLM model.

Background

LLM model (Large Language Model ) is a machine learning model with billions or trillion parameter scale built by using deep learning technology and adopting massive data for pre-training, and is a basic technology of new generation artificial intelligence application. Because the LLM model is trained based on large-scale data in a pre-training mode, the LLM model has the characteristics of large scale, strong universality, good emergence and the like.

With the continuous updating of technology, the LLM model is also an artificial intelligence technology with the greatest attention and influence scope worldwide in recent years, and more companies and institutions also push out products related to the LLM model. On the basis, how to better utilize the LLM model and explore the application scene of the LLM model has a huge application prospect. Moreover, a huge amount of application scenes also create huge future imagination space for the development of LLM model technology.

Disclosure of Invention

The specification provides a user service method based on an LLM model, which is applied to an intelligent service terminal; the intelligent service terminal is provided with at least one LLM service model; the LLM service model is obtained by performing fine tuning training on a LLM basic model based on user service data corresponding to a service scene; the method comprises the following steps:

acquiring a target LLM service model corresponding to a target service scene designated by a user from the at least one LLM service model;

Responding to a service request of a user, operating the target LLM service model to provide user service corresponding to the target service scene for the user, and collecting a user service data set generated in the process of providing the user service for the user; wherein the user service data set contains service data generated in the process of providing the user service to the user; and, service feedback data for the user service by the user;

and performing fine tuning training on the target LLM service model based on the user service data set to obtain a personalized LLM service model corresponding to the user, and continuously providing user service corresponding to the target service scene for the user based on the personalized LLM service model.

Optionally, the LLM basic model with the pre-trained performance is provided on a service platform accessed by the intelligent service terminal; the at least one LLM service model is obtained by performing fine tuning training on the LLM basic model based on user service data corresponding to at least one service scene;

Obtaining a target LLM service model corresponding to a target service scene specified by a user from the at least one LLM service model, including:

Responding to a downloading operation triggered by a user and aiming at the LLM service model, and determining a target service scene appointed by the user;

And downloading a target LLM service model corresponding to the target service scene from the service platform.

Optionally, before the target LLM service model is operated to provide the user service corresponding to the target service scene for the user in response to the service request of the user, the method further includes:

Determining the resource quantity of computing power resources allocated by the intelligent service terminal for the target LLM service model;

And performing model compression on the target LLM service model at least with reference to the resource quantity, and deploying the compressed target LLM service model on the intelligent service terminal.

Optionally, the service platform is a cloud service platform; the LLM basic model and the LLM service model are LLM cloud service models which are obtained by training based on cloud service data maintained on the cloud service platform.

Optionally, collecting the service feedback data generated in the process of providing the user service to the user includes:

outputting a service feedback prompt to the user for the user service provided to the user;

And acquiring service feedback data submitted by the user according to the service feedback prompt.

Optionally, the service feedback prompt is a voice service feedback prompt; correspondingly, the service feedback data is submitted by the user in the form of voice.

Optionally, performing fine tuning training on the target LLM service model based on the user service data set includes:

Performing fine tuning training on the target LLM service model locally based on the user service data set; or alternatively

And carrying out security privacy calculation on the user service data set to obtain a ciphertext data set corresponding to the user service data set, uploading the ciphertext data set to the service platform, and carrying out fine tuning training on the target LLM service model by the service platform based on the ciphertext data set.

Optionally, the security privacy calculation comprises homomorphic encryption calculation;

performing security privacy calculation on the user service data set to obtain a ciphertext data set corresponding to the user service data set, uploading the ciphertext data set to the service platform, and performing fine tuning training on the target LLM service model by the service platform based on the ciphertext data set, wherein the method comprises the following steps:

And carrying out homomorphic encryption calculation on the user service data set to obtain a ciphertext data set corresponding to the user service data set, uploading the ciphertext data set to the service platform, carrying out fine tuning training calculation on the target LLM service model by the service platform based on the ciphertext data set, carrying out homomorphic decryption on a calculation result, and updating model parameters of the target LLM service model based on the decrypted calculation result to finish fine tuning training on the target LLM service model.

constructing a data tag based on the service feedback data contained in the user service data set;

And carrying out data marking on the service data contained in the user service data set based on the constructed data tag, and carrying out supervised fine tuning training on the target LLM service model by taking the service data with the data marking completed as a training sample.

And taking the service data contained in the user service data set as a reinforcement learning action and taking the service feedback data contained in the user service data set as a reinforcement learning state, and performing reinforcement learning-based fine tuning training on the target LLM service model.

Optionally, the LLM model is a multi-modal LLM model.

Optionally, the intelligent service terminal comprises an intelligent robot.

The specification also provides a user service device based on the LLM model, which is applied to the intelligent service terminal; the intelligent service terminal is provided with at least one LLM service model; the LLM service model is obtained by performing fine tuning training on a LLM basic model based on user service data corresponding to a service scene; the device comprises:

the acquisition module acquires a target LLM service model corresponding to a target service scene designated by a user from the at least one LLM service model;

the collection module is used for responding to a service request of a user, running the target LLM service model to provide user service corresponding to the target service scene for the user, and collecting a user service data set generated in the process of providing the user service for the user; wherein the user service data set contains service data generated in the process of providing the user service to the user; and, service feedback data for the user service by the user;

And the training module is used for further carrying out fine-tuning training on the target LLM service model based on the user service data set so as to obtain a personalized LLM service model corresponding to the user, and continuously providing user service corresponding to the target service scene for the user based on the personalized LLM service model.

In the above embodiment, by providing the intelligent service terminal with the LLM service model obtained by performing fine-tuning training on the LLM base model based on the user service data corresponding to the service scene, the user can select a suitable LLM service model from the at least one LLM service model based on the needs of the user, so that the user can be provided with the service required by the user, and the flexibility of using the LLM model can be improved. Moreover, when the user needs to migrate to a new intelligent service terminal, only the LLM service model is needed to be selected from the at least one LLM service model again, the LLM service model can be continuously used for providing services for the user, and therefore migration cost of the user can be reduced.

On the one hand, fine tuning training is performed on the LLM basic model based on user service data corresponding to the service scene on the intelligent service terminal to obtain at least one LLM service model, so that a user can select a proper LLM service model from the at least one LLM service model based on the self requirement, and services required by the user can be provided for the user, and the flexibility of using the LLM model can be improved. Moreover, when the user needs to migrate to a new intelligent service terminal, only the LLM service model is needed to be selected from the at least one LLM service model again, the LLM service model can be continuously used for providing services for the user, and therefore migration cost of the user can be reduced.

On the other hand, by collecting the personalized service data set of the user generated in the process that the LLM service model selected by the user provides service for the user and further carrying out fine tuning training on the LLM service model based on the collected user service data set, the personalized LLM service model corresponding to the user can be obtained, thereby realizing the deep service customization of the intelligent service terminal of the user, and enabling the intelligent service terminal of the user to provide personalized user service for the user based on the LLM service model.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings that are needed in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present disclosure, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a user service method based on the LLM model, as shown in one embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a service framework for providing services to users based on the LLM model in one embodiment of the present disclosure;

FIG. 3 is a schematic diagram illustrating two-level fine-tuning training of LLM service models according to one embodiment of the present disclosure;

FIG. 4 is a schematic block diagram of an electronic device shown in an embodiment of the present disclosure;

fig. 5 is a block diagram of a user service apparatus based on LLM model according to an embodiment of the present specification.

Detailed Description

In order to make the technical solutions in the present specification better understood by those skilled in the art, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present specification, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.

In the related art, training for the LLM model may generally include two training phases, pre-training and fine-tuning training.

The pre-training is a training method for performing unsupervised training on a language model through a large number of unlabeled language texts, so that the model has a plurality of general language processing capacities.

For example, the mainstream pre-training scheme adopted by the LLM model is a training method called Mask mechanism. By adopting a Mask mechanism, part of vocabularies in the unmarked language text can be masked in advance, and the ability of the model to predict the masked vocabularies is further trained on the basis.

The fine tuning training is a training method in which parameters of a model are adjusted based on a data set in a specific application scene on the basis of pre-training, for example, a part or all of the parameters of the model can be adjusted and updated, or the existing parameters of the model can be frozen, and the parameters are newly added on the basis of the frozen parameters, so that the model after fine tuning training can be applied to the application scene, and the model capability corresponding to the application scene is provided.

For example, taking an application scenario of intelligent question and answer as an example, in practical application, on the basis of a model which is pre-trained by adopting the Mask mechanism, fine tuning training can be further performed on the model based on a pre-constructed data sample related to the intelligent question and answer, so that the pre-trained model is further adjusted to be an application scenario which can be suitable for the intelligent question and answer, and a model of multi-round dialogue of a required answer can be generated for a user by performing common sense reasoning based on a problem input by the user.

Along with the continuous progress of technology, application scenes derived based on the LLM model are more and more abundant, so how to utilize the derived application scenes to provide better service for users is a problem that the application scenes in the field of the LLM model are explored.

In view of this, a user service mode is proposed in this specification that combines the LLM model with the intelligent service terminal in depth.

In the service mode, on one hand, fine-tuning training for the LLM basic model based on user service data corresponding to the service scene can be provided on the intelligent service terminal to obtain at least one LLM service model, so that a user can select a proper LLM service model from the at least one LLM service model based on the self requirement, and the user can be provided with the service required by the user, and the flexibility of using the LLM model can be improved. Moreover, when the user needs to migrate to a new intelligent service terminal, only the LLM service model is needed to be selected from the at least one LLM service model again, the LLM service model can be continuously used for providing services for the user, and therefore migration cost of the user can be reduced.

Referring to fig. 1, fig. 1 is a flowchart of a user service method based on LLM model shown in the present specification, wherein the intelligent service has at least one LLM service model; the LLM service model is obtained by performing fine adjustment training on a LLM basic model based on user service data corresponding to a service scene; the method comprises the following implementation processes:

step 102: acquiring a target LLM service model corresponding to a target service scene designated by a user from the at least one LLM service model;

The intelligent service terminal of the user can comprise intelligent equipment for providing service for the user in the form of tasks.

For example, in one example, the smart device may be a smart robot, and the user may customize the service of the smart robot for the user service provided by the user by deploying a required service scenario corresponding to the LLM service model on the smart robot.

Of course, in practical application, the intelligent device may be other intelligent devices besides the intelligent robot.

For example, the smart devices may be classified based on the types of services that they may provide to users, and may also include electronic households, electronic pets, smart coaches, etc., which are not listed in this specification. In one embodiment, please refer to fig. 2, fig. 2 is a schematic diagram of a service framework for providing services to users based on LLM model shown in the present specification.

As shown in fig. 2, in practical application, an intelligent service terminal of a user may access a service platform as a background. Wherein, on the service platform, a LLM basic model can be pre-trained in a pre-training mode based on user data maintained on the platform.

In one embodiment shown, the LLM base model may be a multimodal LLM model. That is, the LLM base model may be a LLM model that can process a plurality of different types of data such as images, sounds, videos, etc. simultaneously, in addition to text.

For example, in practical application, in the process of training the LLM base model in a pre-training manner based on user data maintained on the platform, the service platform may introduce a plurality of different types of data such as images, sounds, videos, etc. as training samples based on the training samples in text form to train the LLM base model, so that the LLM base model has multi-modal data processing capability.

In addition, the user data maintained on the service platform may further include user service data corresponding to at least one preset service scenario. The service platform may further perform fine-tuning training on the LLM base model based on the user service data corresponding to the at least one service scenario on the basis of the LLM base model, to obtain at least one LLM service model. The at least one LLM service model may correspond to different service scenarios, and have different service capabilities.

For example, in one example, the at least one LLM service model may specifically include a first LLM service model with learning coaching capability, a second LLM service model with medical capability, a third LLM service model with communication session capability, and so on, which are not listed in the present specification.

The service platform may be a centralized service platform or a decentralized service platform, and is not particularly limited in this specification.

For example, in one embodiment, the service platform may be a decentralized cloud service platform, and the LLM base model and the LLM service model may be LLM cloud service models that are obtained by training with computing power of a service cluster in the cloud service platform based on cloud service data maintained on the cloud service platform. Of course, in practical application, the service platform may be a centralized service platform, and is not particularly limited in this specification.

At least one LLM service model obtained by performing fine tuning training on the LLM basic model on the service platform can be opened to a user, so that the user can select a target LLM service model corresponding to a target service scene required by the user from the at least one LLM service model based on specific requirements. And the intelligent service terminal of the user can respond to the selection operation of the user to acquire a target LLM service model corresponding to the target service scene designated by the user from the at least one LLM service model, and then deploy the acquired target LLM service model locally.

In one embodiment shown, with continued reference to fig. 2, the service platform may provide a service download interface to the user and open at least one LLM service model from the fine-tuning training to the user via the service download interface. And the user can perform downloading operation in the service downloading interface based on the self requirement to specify a required target service scene and trigger downloading of a target LLM service model corresponding to the target service scene.

The intelligent service terminal can determine a target service scene designated by the user in response to the downloading operation of the LLM service model triggered by the user in the service downloading interface, and send a downloading request containing a scene identifier corresponding to the target service scene to the service platform. After receiving the download request, the service platform can acquire a scene identifier contained in the download request, determine a target LLM service model corresponding to a target service scene indicated by the scene identifier from at least one provided LLM service model, and return a download result of the target LLM service model to the intelligent service terminal. After the intelligent service terminal acquires the target LLM service model, the intelligent service terminal can locally deploy the target LLM service model.

It should be noted that, in practical application, the computing power resource that the intelligent service terminal can provide may be far lower than that of the service platform, so if the user selects to download the target LLM service model to the intelligent service terminal for running, the situation may occur that the resource amount of the computing power resource of the intelligent service terminal is insufficient for normal running of the target LLM service model.

In the illustrated embodiment, if the user selects to download the target LLM service model to the intelligent service terminal for running, in order to ensure that the computing power resources allocated to the target LLM service model on the intelligent service terminal are enough to run the target LLM service model, after the intelligent service terminal obtains the target LLM service model, the intelligent service terminal may further determine the resource amount of the computing power resources allocated to the target LLM service model by the intelligent service terminal, and then at least refer to the resource amount to perform model compression on the target LLM service model, so that the compressed target LLM service model can run normally on the intelligent service terminal, and then deploy the compressed target LLM service model on the intelligent service terminal.

It should be noted that, performing model compression on the LLM model generally refers to reducing the scale of parameters of the model on the premise of maintaining the performance of the LLM model, so as to enable deployment and operation of the LLM model in a resource-constrained environment. In practical application, a specific method for compressing the LLM model to reduce the scale of the parameters of the model can be flexibly adjusted and selected according to specific requirements and the scale of the computing power resource of the intelligent service terminal, and is not particularly limited in the present specification.

For example, common methods for reducing the size of parameters of the model may include such methods as sparsifying parameters of the LLM model, pruning parameters of the LLM model, quantifying parameters of the LLM model, and the like, which may be used alone or in combination in practical applications.

It should be explained that, the sparse processing of the parameters of the LLM model generally means that some parameters with low contribution to the model in the LLM model are set to zero or close to zero, so as to achieve the purpose of reducing the number of parameters. This can reduce the parameter scale of the model to some extent while preserving the performance of the model on a particular task.

Pruning the parameters of the LLM model generally refers to removing some parameters of the LLM model that have a low degree of contribution to the model, thereby reducing the parameter size.

Parameter quantization of parameters of the LLM model generally refers to converting parameters in the LLM model from floating point numbers to lower bit width integers or fixed point numbers to reduce the storage requirements of the model. For example, FP8 (Fixed-Point 8-bit) quantization is a commonly used method of parameter quantization, and the FP8 quantization is used to transform parameters from a floating-Point representation to a Fixed-Point representation.

It should be noted that, in both the sparsification process and pruning process of the parameters of the LLM model, the parameters having low contribution to the model need to be carefully selected to ensure that the influence on the performance of the model is as small as possible. In practical application, a proper method can be selected based on practical requirements, contribution degree of parameters in the LLM model to the LLM model is calculated, then parameters with smaller influence on the LLM model are screened out according to a calculation result, and then thinning or pruning treatment is carried out on the parameters. The specific process of calculating the contribution degree of the parameters in the LLM model to the LLM model is not described in detail in the present specification.

Of course, in practical application, the target LLM service model selected by the user from the at least one LLM service model may be specifically and remotely deployed on the service platform only, without being downloaded to the intelligent service terminal of the user. In this case, after the user designates the target service scenario on the intelligent service terminal, the intelligent service terminal may provide the corresponding service to the user by remotely calling the target LLM service model corresponding to the target service scenario deployed on the service platform. Step 104: responding to a service request of a user, operating the target LLM service model to provide user service corresponding to the target service scene for the user, and collecting a user service data set generated in the process of providing the user service for the user; wherein the user service data set comprises service data generated in the process of providing the user service for the user and service feedback data of the user for the user service;

After the user selects a target LLM service model corresponding to a target service scenario designated by the user from the at least one LLM service model, the intelligent service terminal may operate the target LLM service model in response to a service request of the user to provide the user with a user service corresponding to the target service scenario.

In the illustrated embodiment, referring to fig. 2, if a user selects to download a target LLM service model corresponding to a target service scenario designated by the user from the at least one LLM service model to a local location of an intelligent service terminal for operation, the intelligent service terminal may execute the target LLM service model in response to a service request of the user after deploying the target LLM service model to the local location, and provide a user service corresponding to the target service scenario to the user.

For example, taking the target LLM service model as a question-answer model with interactive dialogue capability as an example, the service request may specifically be a question text which is input by the user and can embody the needs or intentions of the user, and after receiving the question text, the target LLM service model may perform general knowledge reasoning based on the question input by the user, generate and output an answer for the user, which can solve the question of the user, and develop the dialogue with the user.

With continued reference to fig. 2, in the process of running the target LLM service model to provide the user with the user service corresponding to the target service scenario, some personalized user service data is usually generated. The intelligent service terminal can also collect the generated user service data to form a user service data set, and the collected user service data set can be used for further fine tuning training on the target LLM service model.

Wherein the personalized user service data generated by running the target LLM service model may generally include service data generated in the process of providing user services to users.

For example, taking the target LLM service model as a question-answer model with interactive dialogue capability as an example, the service data may be answer text generated for the user based on the questions input by the user during the process of developing dialogue between the target LLM service model and the user. In addition, the personalized user service data may further include service feedback data of the user for the user service provided by the target LLM service model.

It should be noted that, the user service data (including, but not limited to, user equipment information, user personal information, data for analysis, stored data, presented data, etc.) in the above embodiments are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the data are required to comply with relevant laws and regulations and standards of relevant countries and regions, and are provided with corresponding operation entries for the user to select authorization or rejection.

In one embodiment shown, the intelligent service terminal may provide an interactive mechanism for the user to conduct service feedback for the user's service. When collecting service feedback data, the intelligent service terminal can specifically output a service feedback prompt for the user service to the user based on the interaction mechanism, and acquire service feedback data further submitted by the user according to the service feedback prompt.

For example, in one example, the interaction mechanism of the user service feedback may be a mechanism for interaction with the user through voice. When collecting service feedback data, the intelligent service terminal can specifically output a service feedback prompt in a voice form to the user, and acquire the service feedback data in the voice form which is further submitted by the user according to the service feedback prompt.

Step 106: and performing fine tuning training on the target LLM service model based on the user service data set to obtain a personalized LLM service model corresponding to the user, and continuously providing user service corresponding to the target service scene for the user based on the personalized LLM service model.

With continued reference to fig. 2, after collecting the user service data generated by running the target LLM service model to form a user service data set, the intelligent service terminal may further perform fine-tuning training on the target LLM service model based on the collected user service data set.

It should be noted that, since the user service data generated by running the target LLM service model on the intelligent service terminal is generally personalized service data of some users, after performing fine-tuning training on the target LLM service model based on the user service data set, a personalized LLM service model corresponding to the users can be obtained.

In one embodiment, please refer to fig. 3, fig. 3 is a schematic diagram illustrating two-stage fine-tuning training of LLM service models according to the present disclosure.

As shown in fig. 3, the two-stage fine tuning training may specifically include a process of performing a first-stage fine tuning training on the LLM basic service model on the service platform to obtain the target LLM service model, and a process of performing a second-stage fine tuning training on the target LLM service model locally at the intelligent service terminal based on the collected user service data set to obtain the personalized LLM service model corresponding to the user. In this case, after the intelligent service terminal collects the user service data set, the intelligent service terminal may further perform fine-tuning training on the deployed target LLM service model locally on the intelligent service terminal, so as to obtain a personalized LLM service model corresponding to the user.

For example, in one example, a TEE (Trusted execution environment ) running environment may be built in the local running environment of the intelligent service terminal, and in the TEE running environment, fine-tuning training is performed on the target LLM service model based on the collected user service data set.

It should be noted that, because the user service data generated by the intelligent service terminal running the target LLM service model locally is usually some data related to user privacy, the second-level fine tuning training is completed locally at the intelligent service terminal, so that the problem of user privacy data leakage caused by that the user service data is led out of the intelligent service terminal can be avoided.

Of course, in practical application, if the data size of the user service data set collected by the intelligent service terminal is larger, the second-stage fine-tuning training shown in fig. 3 may specifically be performed by migrating to the service platform in a form of security privacy calculation.

In another embodiment, the intelligent service terminal may perform security privacy calculation on the collected user service data set to obtain a ciphertext data set corresponding to the user service data set, and then upload the ciphertext data set to the service platform, where the service platform further performs fine tuning training on the target LLM service model based on the ciphertext data set on the premise of not revealing the privacy data of the user.

The type of the algorithm for the security privacy calculation used by the intelligent service terminal is not particularly limited in the present specification.

For example, in one example, the security privacy calculation may specifically be a homomorphic encryption algorithm. Based on homomorphic encryption algorithm, the data can be homomorphic encrypted and then calculated in the form of ciphertext, and the calculation result obtained after corresponding homomorphic decryption of the obtained ciphertext calculation result is equivalent to the calculation result obtained by directly carrying out the same calculation on the original plaintext data, so that the data can be "invisible". The intelligent service terminal can perform homomorphic encryption calculation on the user service data set to obtain a ciphertext data set corresponding to the user service data set, the ciphertext data set is uploaded to the service platform, and the service platform performs fine tuning training calculation on the target LLM service model based on the ciphertext data set. After finishing the fine-tuning training calculation for the target LLM service model based on the ciphertext data set, the service platform can homomorphically decrypt the calculation result, and then adjust and update the model parameters of the target LLM service model based on the decrypted calculation result so as to finish the fine-tuning training for the target LLM service model.

The process of homomorphic encryption and decryption calculation for the user service data set and adjustment and update for the model parameters of the target LLM service model is not described in detail in the present specification, and those skilled in the art may refer to the description in the related art when the technical solution disclosed in the present specification is put into practice.

Whether the target LLM service model is subjected to fine tuning training based on the user service data set locally at the intelligent service terminal or the user service data set is uploaded to the service platform, the service platform adopts a secure privacy calculation mode to perform fine tuning training on the target LLM service model based on the user service data set, and the specific training method is not particularly limited in the specification.

In one embodiment shown, the targeted LLM service model described above may be fine-tuned using a supervised training approach. In this case, when performing fine tuning training on the target LLM service model based on the user service data set, a data tag may be constructed based on service feedback data included in the user service data set, then data marking is performed on service data included in the user service data set based on the constructed data tag, and then the service data with the data marked is used as a training sample, and the supervised fine tuning training is performed on the target LLM service model.

The specific process of performing the supervised fine tuning training on the target LLM service model based on the user service data set is not described in detail in the present specification, and a person skilled in the art may refer to the description in the related art when the technical solution disclosed in the present specification is put into practice.

In another embodiment shown, the target LLM service model may also be fine-tuned by a training method of reinforcement learning. In this case, when performing the fine-tuning training on the target LLM service model based on the user service data set, the fine-tuning training based on the reinforcement learning may be performed on the target LLM service model with the service data included in the user service data set as the reinforcement learning operation and the service feedback data included in the user service data set as the reinforcement learning state.

The specific process of performing reinforcement learning-based fine-tuning training on the target LLM service model based on the user service data set is not described in detail in the present specification, and a person skilled in the art may refer to the description in the related art when the technical solution disclosed in the present specification is put into practice.

In the present specification, after fine tuning training is further performed on the target LLM service model based on the user service data set to obtain a personalized LLM service model corresponding to the user, the intelligent service terminal may continue to provide, for the user, user services corresponding to the target service scenario based on the personalized LLM service model, and continue to collect service data generated in the process of running the personalized LLM service model in the process of providing services, and then continue to perform fine tuning training on the LLM service model based on the service data, so that the target LLM service model locally deployed at the intelligent service terminal may be optimized continuously, and more personalized user services may be provided for the user.

In the above technical solution, on one hand, by deploying the pre-trained LLM base model on the service platform and performing fine-tuning training on the LLM base model based on the user service data corresponding to at least one service scene to obtain at least one LLM service model, a user can select a suitable LLM service model from the at least one LLM service model based on the needs of the user, and deploy the LLM model to the intelligent service terminal, services required by the user can be provided for the user, so that the flexibility of using the LLM model can be improved. Moreover, when the user needs to migrate to a new intelligent service terminal, the user only needs to select the LLM service model from the at least one LLM service model again and deploy the LLM service model to the new intelligent service terminal, so that the migration cost of the user can be reduced.

On the other hand, by collecting the personalized service data set of the user generated in the process that the LLM model deployed on the intelligent service terminal of the user provides services for the user and further carrying out fine adjustment training on the LLM model based on the collected user service data set, the personalized LLM service model corresponding to the user can be obtained, thereby realizing deep service customization of the intelligent service terminal of the user, and enabling the intelligent service terminal of the user to provide personalized user services for the user based on the deployed LLM service model.

Corresponding to the embodiments of the aforementioned method, the present specification also provides embodiments of an apparatus, an electronic device, and a storage medium.

Fig. 4 is a schematic structural diagram of an electronic device according to an exemplary embodiment. Referring to fig. 4, at the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, a memory 408, and a nonvolatile memory 410, although other hardware required by other services is possible. One or more embodiments of the present description may be implemented in a software-based manner, such as by the processor 402 reading a corresponding computer program from the non-volatile memory 410 into the memory 408 and then running. Of course, in addition to software implementation, one or more embodiments of the present disclosure do not exclude other implementation manners, such as a logic device or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but may also be hardware or a logic device.

As shown in fig. 5, fig. 5 is a block diagram of a user service apparatus based on LLM model according to an exemplary embodiment of the present specification, which may be applied to an electronic device as shown in fig. 4 to implement the technical solution of the present specification. The electronic equipment is connected with a service platform, wherein a pre-trained LLM basic model is provided on the service platform; and performing fine tuning training on the LLM basic model based on user service data corresponding to at least one service scene to obtain at least one LLM service model; the apparatus 500 includes:

The obtaining module 501 obtains a target LLM service model corresponding to a target service scene specified by a user from the at least one LLM service model, and deploys the target LLM service model on the intelligent service terminal;

The collection module 502 is used for responding to a service request of a user, running the target LLM service model to provide user service corresponding to the target service scene for the user, and collecting a user service data set generated in the process of providing the user service for the user; wherein the user service data set contains service data generated in the process of providing the user service to the user; and, service feedback data for the user service by the user;

The training module 503 further performs fine-tuning training on the target LLM service model based on the user service data set, so as to obtain a personalized LLM service model corresponding to the user, and continues to provide the user service corresponding to the target service scene for the user based on the personalized LLM service model.

Correspondingly, the specification also provides electronic equipment, which comprises a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the steps of all of the method flows described previously.

Accordingly, the present specification also provides a computer readable storage medium having stored thereon executable computer program instructions; wherein the instructions, when executed by the processor, implement the steps of the overall method flow described previously.

Accordingly, the present specification also provides a computer program product having executable computer program instructions stored thereon; wherein the computer program instructions, when executed by the processor, implement the steps of the overall method flow described previously.

In the 90 s of the 20 th century, improvements to one technology could clearly be distinguished as improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable GATE ARRAY, FPGA)) is an integrated circuit whose logic functions are determined by user programming of the device. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented with "logic compiler (logic compiler)" software, which is similar to the software compiler used in program development and writing, and the original code before being compiled is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but HDL is not just one, but a plurality of kinds, such as ABEL(Advanced Boolean Expression Language)、AHDL(Altera Hardware Description Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby Hardware Description Language), and VHDL (Very-High-SPEED INTEGRATED Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.

The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application SPECIFIC INTEGRATED Circuits (ASICs), programmable logic controllers, and embedded microcontrollers, examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C2051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.

The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation device is a server system. Of course, it is not excluded that with the development of future computer technology, the computer implementing the functions of the above-described embodiments may be, for example, a personal computer, a laptop computer, a car-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

Although one or more embodiments of the present description provide method operational steps as described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps recited in the embodiments is merely one way of performing the order of steps and does not represent a unique order of execution. When implemented in an actual device or end product, the instructions may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even in a distributed data processing environment) as illustrated by the embodiments or by the figures. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, it is not excluded that additional identical or equivalent elements may be present in a process, method, article, or apparatus that comprises a described element. For example, if first, second, etc. words are used to indicate a name, but not any particular order.

For convenience of description, the above devices are described as being functionally divided into various modules, respectively. Of course, when one or more of the present description is implemented, the functions of each module may be implemented in the same piece or pieces of software and/or hardware, or a module that implements the same function may be implemented by a plurality of sub-modules or a combination of sub-units, or the like. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, read only compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

One skilled in the relevant art will recognize that one or more embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, one or more embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, one or more embodiments of the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

One or more embodiments of the present specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments. In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present specification. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.

The foregoing is merely an example of one or more embodiments of the present specification and is not intended to limit the one or more embodiments of the present specification. Various modifications and alterations to one or more embodiments of this description will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, or the like, which is within the spirit and principles of the present specification, should be included in the scope of the claims.

Claims

1.A user service method based on LLM model, the said method is applied to the intelligent service terminal; the intelligent service terminal is provided with at least one LLM service model; the LLM service model is obtained by performing fine tuning training on a LLM basic model based on user service data corresponding to a service scene; the method comprises the following steps:

2. The method of claim 1, wherein the pre-trained LLM base model is provided on a service platform to which the intelligent service terminal is connected; the at least one LLM service model is obtained by performing fine tuning training on the LLM basic model based on user service data corresponding to at least one service scene;

3. The method of claim 2, in response to a service request of a user, prior to running the target LLM service model to provide a user service corresponding to the target service scenario to the user, further comprising:

4. The method of claim 1, the service platform being a cloud service platform; the LLM basic model and the LLM service model are LLM cloud service models which are obtained by training based on cloud service data maintained on the cloud service platform.

5. The method of claim 1, collecting the service feedback data generated in providing the user service to the user, comprising:

6. The method of claim 1, the service feedback cues are voice-form service feedback cues; correspondingly, the service feedback data is submitted by the user in the form of voice.

7. The method of claim 1, fine-tuning the target LLM service model based on the user service data set, comprising:

8. The method of claim 7, the secure privacy computation comprising homomorphic encryption computation;

9. The method of claim 7, fine-tuning the target LLM service model based on the user service data set, comprising:

10. The method of claim 9, fine-tuning the target LLM service model based on the user service data set, comprising:

11. The method of claim 1, wherein the LLM model is a multi-modal LLM model.

12. The method of claim 1, the intelligent service terminal comprising an intelligent robot.

13. A user service device based on a LLM model, the device being applied to an intelligent service terminal; the intelligent service terminal is provided with at least one LLM service model; the LLM service model is obtained by performing fine tuning training on a LLM basic model based on user service data corresponding to a service scene; the device comprises:

14. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method of any of claims 1 to 12 when the computer program is executed.

15. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of any of claims 1 to 12.

16. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the method of any of claims 1 to 12.