CN114676786A

CN114676786A - Marking template generation method, data marking method and device

Info

Publication number: CN114676786A
Application number: CN202210368962.7A
Authority: CN
Inventors: 胡鸣人; 谢爽; 张宇; 沈培培; 周志远; 刘魏光; 华蔚颖; 施恩
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2022-04-08
Filing date: 2022-04-08
Publication date: 2022-06-28

Abstract

The disclosure provides a label template generation method, a data labeling method and a data labeling device, relates to the technical field of computers, and further relates to the technical field of deep learning and artificial intelligence. The specific implementation scheme is as follows: the method comprises the steps of firstly responding to a received development instruction of a development user for a target labeling template, obtaining an interactive plug-in corresponding to the target labeling template, wherein the interactive plug-in is used for representing front-end processing logic and display logic of the target labeling template, then generating a service plug-in corresponding to the target labeling template based on the development operation of the development user for the back-end service of the target labeling template, the service plug-in is used for representing the back-end processing logic and storage service of the target labeling template, and finally responding to a received generation instruction for the target labeling template, and generating the target labeling template based on the interactive plug-in and the service plug-in, so that the development user can create a required labeling template according to self requirements, and the labeling template can adapt to user requirements.

Description

Marking template generation method, data marking method and device

Technical Field

The disclosure relates to the technical field of computers, in particular to a method for generating a labeling template, a method for labeling data and a device for labeling data.

Background

With the continuous development and progress of artificial intelligence, the development of the artificial intelligence becomes more and more important, and the work of data annotation occupies more than 50% of the work load of the whole artificial intelligence development process, so that an annotation tool is an important ring. In the process of using the artificial intelligence development platform, the first experienced function of a client is often data annotation, so the data annotation function is very important for the artificial intelligence development platform.

The range of the data labeling template is very wide, for example, image classification data labeling, data labeling of object detection, data labeling of entity relationship extraction and the like, under the background of the vigorous development of the whole artificial intelligence industry, the fields related to the data to be labeled are more and more, and new labeling templates need to be continuously researched and developed to adapt to various data to be labeled.

Disclosure of Invention

The disclosure provides a labeling template generation method, a data labeling device, electronic equipment, a storage medium and a computer program product.

According to an aspect of the present disclosure, there is provided an annotation template generation method, including: in response to receiving a development instruction of a development user for a target labeling template, acquiring an interactive plug-in corresponding to the target labeling template, wherein the interactive plug-in is used for representing front-end processing logic and display logic of the target labeling template; generating a service plug-in corresponding to the target labeling template based on the development operation of a development user on the back-end service of the target labeling template, wherein the service plug-in is used for representing the back-end processing logic and the storage service of the target labeling template; and in response to receiving a generation instruction for the target annotation template, generating the target annotation template based on the interactive plug-in and the service plug-in.

According to another aspect of the present disclosure, there is provided an annotation template generation method, including: responding to an interactive plugin and a service plugin of a target labeling template sent by a receiving terminal, and carrying out front-end loading on the target labeling template based on the interactive plugin and the service plugin of the target labeling template to obtain a front-end service corresponding to the target labeling template, wherein the front-end service comprises a front-end processing service and a display service of the target labeling template, the interactive plugin is used for representing the front-end processing logic and the display logic of the target labeling template, and the service plugin is used for representing the rear-end processing logic and the storage service of the target labeling template; based on a service plug-in of the target labeling template, establishing a back-end service of the target labeling template, wherein the back-end service comprises a back-end processing service and a storage service of the target labeling template; and associating the front-end service with the back-end service to generate a target labeling template.

According to another aspect of the present disclosure, there is provided a data annotation method, including: receiving data to be marked corresponding to a target marking template uploaded by a marking user; the authentication information and the data to be labeled of the target labeling template are sent to the server side, so that the server side judges the data to be labeled based on the authentication information and the service plug-in of the target labeling template, and the service plug-in is used for representing the back-end processing logic and the storage service of the target labeling template; responding to the received data to be marked, which is returned by the server and judged by the data, and displaying the data to be marked, which is judged by the data, by the interactive plug-in based on the target marking template; and responding to the received marking instruction of the marking user, and generating a marking result corresponding to the data to be marked, which is judged by the data.

According to another aspect of the present disclosure, there is provided an annotation template generation apparatus, including: the acquisition module is configured to respond to a received development instruction of a development user for the target marking template, and acquire an interactive plug-in corresponding to the target marking template, wherein the interactive plug-in is used for representing front-end processing logic and display logic of the target marking template; the first generation module is configured to generate a service plug-in corresponding to the target labeling template based on the back-end service development operation of a development user on the target labeling template, and the service plug-in is used for representing back-end processing logic and storage service of the target labeling template; and the second generation module is configured to respond to the received generation instruction for the target annotation template, and generate the target annotation template based on the interactive plug-in and the service plug-in.

According to another aspect of the present disclosure, there is provided an annotation template generation apparatus, including: the front-end loading module is configured to respond to an interactive plugin and a service plugin of a target marking template sent by a receiving terminal, and carry out front-end loading on the target marking template based on the interactive plugin and the service plugin of the target marking template to obtain a front-end service corresponding to the target marking template, wherein the front-end service comprises a front-end processing service and a display service of the target marking template, the interactive plugin is used for representing the front-end processing logic and the display logic of the target marking template, and the service plugin is used for representing the rear-end processing logic and the storage service of the target marking template; the creating module is configured to create a back-end service of the target labeling template based on the service plug-in of the target labeling template, wherein the back-end service comprises a back-end processing service and a storage service of the target labeling template; and the third generation module is configured to associate the front-end service and the back-end service to generate a target annotation template.

According to another aspect of the present disclosure, there is provided a data annotation apparatus, including: the receiving module is configured to receive data to be annotated, which is uploaded by an annotation user and corresponds to the target annotation template; the sending module is configured to send the authentication information and the data to be labeled of the target labeling template to the server side so that the server side can judge the data to be labeled based on the authentication information and the service plug-in of the target labeling template, and the service plug-in is used for representing the back-end processing logic and the storage service of the target labeling template; the display module is configured to respond to the received data to be annotated, which is returned by the server and judged by the data, and display the data to be annotated, which is judged by the data, on the basis of the interactive plug-in of the target annotation template; and the fourth generation module is configured to generate a labeling result corresponding to the data to be labeled, which is judged by the data, in response to receiving a labeling instruction of the labeling user.

According to another aspect of the present disclosure, there is provided an electronic device comprising at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the annotation template generation method or the data annotation method.

According to another aspect of the present disclosure, there is provided a computer-readable medium having stored thereon computer instructions for enabling a computer to execute the above annotation template generation method or the above data annotation method.

According to another aspect of the present disclosure, a computer program product is provided, which includes computer programs/instructions, and when the computer programs/instructions are executed by a processor, the annotation template generation method or the data annotation method is implemented.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a flow diagram of one embodiment of an annotation template generation method according to the present disclosure;

FIG. 2 is a flow diagram for one embodiment of obtaining a presentation plug-in according to the present disclosure;

FIG. 3 is a flow diagram for one embodiment of obtaining a front-end logic plug-in, according to the present disclosure;

FIG. 4 is a flow diagram for one embodiment of obtaining a service plug-in, according to the present disclosure;

FIG. 5A is a flow diagram of another embodiment of an annotation template generation method according to the present disclosure;

FIG. 5B is a block diagram of the front-end service and the back-end service of the annotation template generated according to the annotation template generation method of the present disclosure;

FIG. 6 is a flow diagram for one embodiment of a backend service creating target annotation templates, according to the present disclosure;

FIG. 7 is a flow diagram of yet another embodiment of an annotation template generation method according to the present disclosure;

FIG. 8 is a flow diagram of one embodiment of a method of data annotation according to the present disclosure;

FIG. 9 is a flow diagram of another embodiment of a method of data annotation according to the present disclosure;

FIG. 10 is a schematic diagram illustrating an embodiment of an annotation template generation apparatus according to the present disclosure;

FIG. 11 is a schematic structural diagram of another embodiment of an annotation template generation apparatus according to the present disclosure;

FIG. 12 is a schematic block diagram illustrating one embodiment of a data annotation device according to the present disclosure;

fig. 13 is a block diagram of an electronic device for implementing the annotation template generation method and the data annotation method of the embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Referring to fig. 1, fig. 1 shows a flow diagram 100 of an embodiment of an annotation template generation method that can be applied to the present disclosure. The method for generating the labeling template comprises the following steps:

and step 110, in response to receiving a development instruction of a development user for the target labeling template, acquiring an interactive plug-in corresponding to the target labeling template.

In this embodiment, an execution subject (e.g., a terminal device) of the annotation template generation method may present a development interface of the annotation template to a development user through a screen, where the development interface may include information such as a template name filling column of the annotation template, a development control for the development user to click on. The development user can fill the template name of the target labeling template in the development interface, the template name can correspond to information forms such as character fields, after the template name of the target labeling template is filled, the development user can click the development control in the development interface to generate a development instruction for the target labeling template, and the development instruction is generated according to the click operation of the development user on the development control in the development interface and is used for representing the development operation of the development user on the target labeling template.

After receiving a development instruction of a development user for a target annotation template, the execution main body can present a configuration interface of an interactive plug-in to the development user through a screen, wherein the interactive plug-in can correspond to the target annotation template, namely each annotation template corresponds to one interactive plug-in, and the interactive plug-in is used for representing front-end processing logic and display logic of the target annotation template, wherein the front-end processing logic can represent processing logic of the target annotation template in the annotation process and can comprise the processing logic of the annotation instruction, the processing logic of data cache, the processing logic of a specific annotation scene and the like; the presentation logic may characterize the presentation manner of the target annotation template, and may include rendering of the target annotation template, and so on. The development user can configure the interactive plug-in corresponding to the target labeling template in the configuration interface and input the configuration file required by generating the interactive plug-in, so that the execution main body can pack the configuration file input by the development user into the interactive plug-in corresponding to the target labeling template.

As an example, the execution subject may present a general template of the interactive plug-in to a development user in a configuration interface of the interactive plug-in, where the general template may include general files of a plurality of plug-ins and a custom portion corresponding to the interactive plug-in of the target annotation template, and the development user may write a configuration file for the interactive plug-in of the target annotation template in the custom portion, so that the configuration file written by the development user may be packaged into the interactive plug-in corresponding to the target annotation template.

And 120, generating a service plug-in corresponding to the target labeling template based on the back-end service development operation of the development user on the target labeling template.

In this embodiment, the execution main body obtains the interactive plug-in according to the configuration file of the development user in the configuration interface of the interactive plug-in, and may present, to the development user, a back-end service development interface of the target annotation template, where the back-end service development interface is used to configure a back-end service of the target annotation template, and the back-end service development interface may include a back-end processing logic compiling module and a storage service compiling module. The development user can compile the back-end processing logic of the target labeling template in a compiling module of the back-end processing logic, the back-end processing logic can represent the processing logic for processing the data to be labeled and the corresponding labeling result, the back-end storage service of the target labeling template can also be compiled in a compiling module of the storage service, and the back-end storage service can represent a database for storing the data to be labeled and the corresponding labeling result.

And the execution main body packs the configuration files corresponding to the back-end processing logic and the storage service written in the back-end service development interface by the development user into the service plug-in corresponding to the target labeling template, so that the service plug-in can represent the back-end processing logic and the storage service of the target labeling template.

And step 130, in response to receiving a generation instruction for the target annotation template, generating the target annotation template based on the interactive plug-in and the service plug-in.

In this embodiment, after the development user writes the service plug-in of the target annotation template, the execution main body may present a registration interface of the target annotation template to the development user through the screen, where the registration interface may include information such as a template name and a registration control of the target annotation template. The development user can click the registration control in the registration interface to generate a generation instruction for the target annotation template, the generation instruction is generated according to the click operation of the development user on the registration control in the registration interface and is used for representing the generation operation of the development user on the target annotation template after the development operation of the development user on the target annotation template is completed, and then the execution main body can package the interactive plug-in and the service plug-in after receiving the generation instruction for the target annotation template so as to send the packaged content to the server, so that the server generates the target annotation template according to the interactive plug-in and the service plug-in of the target annotation template.

The marking template generation method provided by the embodiment of the disclosure can enable a development user to create a required marking template according to self requirements by responding to the received development instruction of the development user for a target marking template, acquiring an interactive plug-in corresponding to the target marking template, wherein the interactive plug-in is used for representing the front-end processing logic and the display logic of the target marking template, then generating a service plug-in corresponding to the target marking template based on the development operation of the development user for the back-end service of the target marking template, the service plug-in is used for representing the back-end processing logic and the storage service of the target marking template, and finally generating the target marking template based on the interactive plug-in and the service plug-in response to the received generation instruction for the target marking template, so that the marking template can adapt to the user requirements, the quantity and the type of the marking template can be enriched rapidly, and the richness of the marking template is improved, and the generation process of the annotation template is not limited to the existing developers, more users can participate in the generation and the updating of the annotation template, the updating efficiency of the annotation template is improved, and therefore the data annotation requirements of multiple parties such as a cloud platform and privatized clients can be met.

With continued reference to fig. 2, fig. 2 shows a flowchart of an embodiment of obtaining a presentation plug-in, that is, the step 110, in response to receiving a development instruction of a development user for a target annotation template, obtaining an interactive plug-in corresponding to the target annotation template, including the following steps:

the interactive plug-in may include a display plug-in for representing display logic of the target annotation template, where the display logic of the target annotation template may include a rendering manner, a labeling manner, and the like of the target annotation template, and the display logic of each annotation template may be the same or different, that is, each annotation template may correspond to one display logic, and may determine one display plug-in.

Step 210, in response to receiving a development instruction of a development user for a target annotation template, presenting a first configuration interface for a display plug-in.

In this embodiment, the execution subject may present a development interface of the annotated template to the development user through the screen, where the development interface may include information such as a template name filling column of the annotated template, a development control for the development user to click on. The development user can fill the template name of the target labeling template in the development interface, the template name can correspond to information forms such as character fields, and the development user can click the development control in the development interface after filling the template name of the target labeling template so as to generate a development instruction for the target labeling template.

After receiving a development instruction of a development user for a target labeling template, the execution main body can present a first configuration interface for a display plug-in to the development user through a screen, the first configuration interface can comprise a universal template for the display plug-in, the universal template can comprise a plurality of universal files for the display plug-in and a custom part corresponding to the display plug-in of the target labeling template, and the development user can write the configuration file for the display plug-in of the target labeling template by the custom part.

Step 220, receiving a plurality of annotation category information and/or corresponding annotation mode selection information input by the development user in the first configuration interface.

In this embodiment, the execution main body may receive a plurality of annotation category information and/or corresponding annotation mode selection information, which are input by a development user in a user-defined portion of the first configuration interface for the target annotation template, where the annotation category information may be determined according to an annotation task of the target annotation template.

The multiple pieces of labeling category information customized by the development user may be different for different labeling tasks, for example, when a character sample is labeled, the multiple pieces of labeling category information customized by the development user may include a face position, an expression category, a gender category, a glasses position, a glasses category, and the like. When the vehicle sample is labeled, the user-defined multiple labeling category information may include a vehicle category, a vehicle color, and the like, and the labeling category information is not specifically limited in the present disclosure, and can be determined by those skilled in the art as needed. And, the marking mode can include at least one in frame, polygon, key point, choice question, question and answer question, and different mark categories can correspond different marking modes, for example, when marking the expression, the marking mode can be the choice question, when marking whole face position, the marking mode can be the frame, and this disclosure does not do specific restriction to the marking mode, and technical staff in the art can set up as required.

And step 230, obtaining the display plug-in corresponding to the target labeling template according to the plurality of labeling category information and/or the corresponding labeling mode selection information.

In this embodiment, the execution main body obtains a plurality of label category information customized by a development user and label mode selection information corresponding to the label category information, and may package a universal template into which the label category information and/or the label mode selection information is input, to form a display plug-in corresponding to the target label template.

In the embodiment, the display plug-in of the target labeling template can be written by developing a plurality of labeling category information and/or corresponding labeling mode selection information of the user-defined target labeling template, so that the writing mode of the display plug-in is simplified, the writing efficiency of the display plug-in of the target labeling template is improved, and more developing users can participate in the developing and generating process of the target labeling template.

Referring to fig. 3, fig. 3 is a flowchart illustrating an embodiment of obtaining a front-end logic plug-in, that is, the step 110, in response to receiving a development request of a development user for a target annotation template, obtaining an interactive plug-in corresponding to the target annotation template, including the following steps:

the interactive plug-in may comprise a front-end logic plug-in for characterizing the front-end processing logic of the target annotation template, wherein the front-end processing logic of the target annotation template may characterize the processing logic of the target annotation template during the annotation process.

Step 310, in response to receiving a development instruction of a development user for the target annotation template, presenting a second configuration interface for the front-end logic plug-in.

After receiving a development instruction of a development user for a target labeling template, the execution main body can present a second configuration interface for a front-end logic plug-in to the development user through a screen, wherein the second configuration interface can comprise a common template of the front-end logic plug-in, the common template can comprise common files of a plurality of front-end logic plug-ins and a custom part corresponding to the front-end logic plug-in of the target labeling template, and the development user can write a configuration file for the front-end logic plug-in of the target labeling template by the custom part.

And step 320, receiving the front-end processing logic configuration file input by the developer in the second configuration interface.

In this embodiment, the execution subject may receive a front-end processing logic configuration file input by a developer in a custom part of the second configuration interface, where the front-end processing logic configuration file may be used to characterize processing logic of the target annotation template in the annotation process, and may include processing logic of an annotation instruction, processing logic of a data cache, processing logic of a specific annotation scene, and the like, and the annotation scene may include, but is not limited to, a plurality of scenes, such as a single annotation, a multi-person annotation, data import, data export, single data presentation, multiple data page turning presentation, data cleansing, and data publishing.

And 330, acquiring a front-end logic plug-in corresponding to the target labeling template according to the processing logic configuration file.

In this embodiment, the execution subject obtains a front-end processing logic configuration file customized by a development user, and may package a general template input into the front-end processing logic configuration file into a front-end logic plug-in corresponding to the target annotation template.

In the embodiment, the front-end logic plug-in of the target labeling template can be written by the front-end processing logic configuration file of the target labeling template customized by the development user, so that the writing mode of the front-end logic plug-in is simplified, the writing efficiency of the front-end logic plug-in of the target labeling template is improved, and more development users can participate in the development and generation processes of the target labeling template.

Referring to fig. 4, fig. 4 is a flowchart illustrating an embodiment of obtaining a service plug-in, that is, the step 120, of generating the service plug-in corresponding to the target annotation template based on the backend service development operation of the development user on the target annotation template, including the following steps:

step 410, in response to receiving a back-end service development operation of a development user for a target annotation template, presenting a third configuration interface for a service plug-in.

In this embodiment, after acquiring the interactive plug-in according to the configuration file of the development user in the configuration interface of the interactive plug-in, the execution main body may present the back-end service development interface of the target annotation template to the development user, and after receiving the back-end service development operation for the target annotation template in the back-end service development interface of the development user, the execution main body may present the third configuration interface of the service plug-in to the development user through the screen. The third configuration interface may include a back-end processing logic compiling module and a storage service compiling module, and the development user may compile the back-end processing logic of the target annotation template in the back-end processing logic compiling module and may also compile the back-end storage service of the target annotation template in the storage service compiling module.

And step 420, receiving a back-end processing logic configuration file and a database for storing data, which are input by a developer in the third configuration interface.

In this embodiment, a development user may write the back-end processing logic in the third configuration interface to generate a back-end processing logic configuration file, where the back-end processing logic configuration file may represent processing logic of the data to be annotated and the annotation result, and may include performing data judgment on a data type and/or a data format of the data to be annotated corresponding to the target annotation template, judging whether the data type and/or the data format of the data to be annotated is a specific data type and/or data format corresponding to the target annotation template, and further including performing operation such as result judgment processing on the annotation result. For example, the image annotation template may correspond to data of an image type, the text annotation template may correspond to data of a text type, the video annotation template may correspond to data of a video type, and the like, and then the target annotation template may correspond to annotation data of a specific data type and/or data format.

The development user can also write the storage service in the third configuration interface, select a database corresponding to the target annotation template from a plurality of databases, where the database can be used to store data, and the data may be data to be annotated and an annotation result. The execution subject can receive a back-end processing logic configuration file and a database for storing data, which are input by a development user in the third configuration interface.

And 430, acquiring the service plug-in corresponding to the target labeling template according to the back-end processing logic configuration file and the database for storing data.

In this embodiment, the execution subject may be packaged into a service plug-in corresponding to the target annotation template according to the back-end processing logic configuration file and the database for storing data, which are input by the development user in the third configuration interface.

In the embodiment, the development user can customize the back-end processing logic configuration file of the target labeling template and the database for storing data, so that the service plug-in of the target labeling template can be written, the writing mode of the service plug-in is simplified, the writing efficiency of the service plug-in of the target labeling template is improved, and more development users can participate in the development and generation process of the target labeling template.

As an optional implementation manner, the step 130, in response to receiving the generation instruction for the target annotation template, of generating the target annotation template based on the interactive plug-in and the service plug-in, may include the following steps: in response to receiving a generation instruction for the target labeling template, acquiring authentication information corresponding to the target labeling template; and generating the target labeling template based on the interactive plug-in, the service plug-in and the authentication information of the target labeling template.

Specifically, after the development user writes the interactive plug-in and the service plug-in of the target annotation template, the execution main body may present a registration interface of the target annotation template to the development user through the screen, where the registration interface may include information such as a template name and a registration control of the target annotation template. The development user can click the registration control in the registration interface to generate a generation instruction for the target labeling template, the execution main body can acquire corresponding authentication information from the server according to the target labeling template, the authentication information can represent authentication information of the target labeling template, such as ak (Access Key id)/sk (secret Access Key) information, and each labeling template corresponds to one authentication information. And then the execution main body packages the interactive plug-in, the service plug-in and the authentication information of the target labeling template so as to send the packaged content to the server, so that the server generates the target labeling template according to the authentication information, the interactive plug-in and the service plug-in of the target labeling template.

In the implementation mode, the authentication information is acquired for the target labeling template to distinguish the labeling templates, so that different labeling template access requests can be distributed to the corresponding labeling templates, and the pertinence of the labeling templates is improved.

Referring to FIG. 5A, FIG. 5A illustrates a flow diagram 500 of another embodiment of an annotation template generation method. The label template generation method can comprise the following steps:

and 510, responding to the interactive plug-in and the service plug-in of the target labeling template sent by the receiving terminal, and carrying out front-end loading on the target labeling template based on the interactive plug-in of the target labeling template to obtain front-end service corresponding to the target labeling template.

In this embodiment, an execution subject (for example, a server) of the annotation template generation method may receive an interactive plug-in and a service plug-in of a target annotation template sent by a terminal, distinguish the interactive plug-in and the service plug-in of the target annotation template, and determine that the interactive plug-in should operate at a front end of the terminal and the service plug-in should operate at a back end, where the interactive plug-in is used to represent front-end processing logic and display logic of the target annotation template, and the service plug-in is used to represent back-end processing logic and storage service of the target annotation template.

The execution main body can perform front-end loading of the target labeling template according to the interactive plug-in of the target labeling template, and complete the front-end loading of the interactive plug-in of the target labeling template so as to obtain front-end services corresponding to the target labeling template, wherein the front-end services can comprise front-end processing services and display services of the target labeling template and can correspond to front-end logic plug-ins and display plug-ins of the target labeling template. The front-end service may further include a basic component corresponding to the target annotation template, and the basic component may include services such as a communication protocol sdk and a template UI library.

And 520, creating a back-end service of the target labeling template based on the service plug-in of the target labeling template.

In this embodiment, the execution main body may create a back-end service of the target annotation template according to the service plug-in of the target annotation template, analyze the service plug-in, and publish a configuration file corresponding to the analyzed service plug-in as a container to obtain a plurality of container clusters corresponding to the target annotation template, that is, create a back-end service corresponding to the target annotation template, where the back-end service may include a back-end processing service and a storage service of the target annotation template, and may correspond to the back-end logic plug-in of the target annotation template and a database corresponding to the target annotation template.

And 530, associating the front-end service with the back-end service to generate a target labeling template.

In this embodiment, after the execution main body obtains the front-end service and the back-end service of the target annotation template, the execution main body may associate the front-end service and the back-end service corresponding to the target annotation template, create an access channel for the front-end service and the back-end service corresponding to the target annotation template, and generate the target annotation template.

And the execution main body can generate a successful information notification generated by the target labeling template and also can generate a message notification of the target labeling template on line, so that the target labeling template can be accessed and used.

With continued reference to FIG. 5B, FIG. 5B shows a block diagram of the front-end service and the back-end service of the annotation template, different annotation templates may include the front-end service and the back-end service, for example, annotation template A may include a corresponding presentation plug-in and front-end logic plug-in, and a corresponding back-end logic plug-in, general service plug-in and database 1, and annotation template B may include a corresponding presentation plug-in and front-end logic plug-in, and a corresponding back-end logic plug-in, general service plug-in and database 2, etc.

The method for generating the labeling template comprises the steps of responding to an interactive plugin and a service plugin of a target labeling template sent by a receiving terminal, carrying out front-end loading on the target labeling template based on the interactive plugin and the service plugin of the target labeling template to obtain a front-end service corresponding to the target labeling template, wherein the front-end service comprises the front-end processing service and the display service of the target labeling template, the interactive plugin is used for representing the front-end processing logic and the display logic of the target labeling template, the service plugin is used for representing the rear-end processing logic and the storage service of the target labeling template, then the rear-end service of the target labeling template is created based on the service plugin of the target labeling template, the rear-end service comprises the rear-end processing service and the storage service of the target labeling template, and finally the front-end service and the rear-end service are associated to generate the target labeling template, the method can automatically generate the label template required by the development user based on the interactive plug-in and the service plug-in created by the development user, so that the label template can adapt to the user requirement, the number and the types of the label template can be enriched rapidly, the richness of the label template is improved, the generation process of the label template is not limited to the existing developers, more users can participate in the generation and the update of the label template, the update efficiency of the label template is improved, and the data label requirements of multiple parties such as a cloud platform and a privatized client can be met.

Referring to FIG. 6, FIG. 6 is a flowchart of an embodiment of creating a backend service of a target annotation template, namely the above step 520, and creating the backend service of the target annotation template based on the service plug-in of the target annotation template, including the following steps:

and 610, generating a back-end logic plug-in corresponding to the target labeling template based on the service plug-in of the target labeling template, and determining a database corresponding to the target labeling template.

In this embodiment, the execution main body parses the service plug-in of the target annotation template to obtain a backend service configuration file corresponding to the target annotation template, where the backend service configuration file includes a backend processing logic file and a selected database.

The execution main body can generate a back-end logic plug-in corresponding to the target marking template according to the back-end processing logic file in the back-end service configuration file, and determine a database corresponding to the target marking template according to the selected database, wherein the back-end logic plug-in can be used for representing the processing logic of the data to be marked and the marking result.

And step 620, acquiring the universal service plug-in corresponding to the target labeling template.

In this embodiment, the execution main body may configure a corresponding generic service plug-in for the target annotation template, where the generic service plug-in may be used to represent a preset service of the target annotation template, and may include a service provided by the execution main body for the annotation template, and may include service on/off lines, scaling, and related operation and maintenance support, for example, preset services such as application monitoring, problem location, initialization, status reporting, log reporting, observability, and the like.

And 630, acquiring the back-end service corresponding to the target labeling template based on the back-end logic plug-in, the database and the general service plug-in.

In this embodiment, after the execution main body obtains the backend logic plug-in, the database, and the universal service plug-in of the target annotation template, the execution main body may publish the backend logic plug-in as a container, add the universal service plug-in to the container, and associate the container with the database of the target annotation template, thereby obtaining the backend service corresponding to the target annotation template.

In the embodiment, the back-end service corresponding to the target labeling template is obtained based on the back-end logic plug-in, the database and the general service plug-in, so that the back-end service corresponding to the target labeling template can include more service contents, and the service contents and the using effect of the target labeling template are improved.

Referring to FIG. 7, FIG. 7 shows a flow diagram of yet another embodiment of an annotation template generation method, which may include the steps of:

step 710, in response to receiving the authentication information, the interactive plugin and the service plugin of the target labeling template sent by the terminal, performing front-end loading on the target labeling template based on the interactive plugin and the authentication information of the target labeling template to obtain front-end service corresponding to the target labeling template.

In this embodiment, the execution main body may receive authentication information, an interactive plugin and a service plugin of the target annotation template sent by the terminal, distinguish the interactive plugin and the service plugin of the target annotation template, and determine that the interactive plugin should operate at the front end of the terminal and the service plugin should operate at the back end.

The execution main body can load the front end of the target labeling template according to the interactive plug-in and the authentication information of the target labeling template, and complete the loading of the interactive plug-in of the target labeling template at the front end so as to obtain the front end service corresponding to the target labeling template, wherein the front end service can be used for representing the front end processing logic and the display logic of the target labeling template and can correspond to the front end logic plug-in and the display plug-in of the target labeling template. The execution main body can associate the front-end service of the target labeling template with the authentication information to obtain the front-end service corresponding to the associated target labeling template.

And 720, establishing the back-end service of the target labeling template based on the service plug-in and the authentication information of the target labeling template.

In this embodiment, the execution main body may create a backend service of the target annotation template according to the service plug-in and the authentication information of the target annotation template, analyze the service plug-in, and publish the configuration file corresponding to the analyzed service plug-in as a container to obtain a plurality of container clusters corresponding to the target annotation template, that is, create the backend service corresponding to the target annotation template. The execution main body can associate the back-end service of the target labeling template with the authentication information to obtain the back-end service corresponding to the associated target labeling template.

And 730, associating the front-end service and the back-end service corresponding to the target labeling template to generate the target labeling template.

In the embodiment, the target labeling template is generated by the interaction plug-in based on the target labeling template, the authentication information and the service plug-in, and the labeling templates can be distinguished, so that different labeling template access requests can be distributed to the corresponding labeling templates, and the pertinence of the labeling templates is improved.

As an optional implementation manner, the method for generating a labeling template may further include the following steps: responding to the generated target labeling template, and acquiring running state information of the target labeling template; and generating operation and maintenance information of the target labeling template based on the operation state information of the target labeling template.

Specifically, the execution main body may obtain the running state information of the target annotation template in real time after the target annotation template is generated and when a back-end service of the target annotation template is started and run. Then, the execution main body generates operation and maintenance information of the target labeling template according to the operation state information of the target labeling template, where the operation and maintenance information may include an operation state of the target labeling template, and the operation state may include normal operation or operation failure.

In the implementation mode, the running state information of the target labeling template is acquired in real time, so that the target labeling template is monitored in real time, and the running problem of the target labeling template can be found more quickly.

Referring to FIG. 8, FIG. 8 illustrates a flow diagram 800 of one embodiment of a data annotation process. The data annotation method can comprise the following steps:

and 810, receiving the data to be annotated corresponding to the target annotation template uploaded by the annotation user.

In this embodiment, an execution subject (for example, a terminal device) of the data annotation method may receive an access request of an annotation user to a target annotation template, and present a data upload interface to the annotation user, where the annotation user may upload data to be annotated corresponding to the target annotation template in the data upload interface. Therefore, the execution main body can receive the data to be labeled, which is uploaded by the labeling user and corresponds to the target labeling template.

And step 820, sending the authentication information of the target labeling template and the data to be labeled to the server, so that the server performs data judgment on the data to be labeled based on the authentication information and the service plug-in of the target labeling template.

In this embodiment, after receiving the to-be-labeled data corresponding to the target labeling template uploaded by the labeling user, the executing body may send the authentication information of the target labeling template and the to-be-labeled data to the server.

And the server determines a database corresponding to the target marking template according to the authentication information of the target marking template and stores the data to be marked into the corresponding database. And then the server judges the data to be marked according to the service plug-in of the target marking template, can judge the data type and/or data format and the like of the data to be marked, judges whether the data type and/or data format of the data to be marked is the specific data type and/or data format corresponding to the target marking template, if the data type and/or data format of the data to be marked is determined to be the specific data type and/or data format corresponding to the target marking template, the data to be marked is determined to pass the data judgment, and the data to be marked which passes the data judgment is returned to the execution main body. If the data type and/or the data format of the data to be marked are determined not to be the specific data type and/or the data format corresponding to the target marking template, determining that the data to be marked do not meet the requirement of the target marking template, determining that the data to be marked do not pass data judgment, not returning the data to be marked to the execution main body, sending prompt information to the execution main body, and prompting a marking user that the data to be marked do not meet the requirement of the target marking template and are not the specific data type and/or the data format corresponding to the target marking template.

Step 830, in response to receiving the data to be annotated, which is returned by the server and is judged by the data, displaying the data to be annotated, which is judged by the data, based on the interactive plug-in of the target annotation template.

In this embodiment, after receiving the data to be annotated, which is judged by the data and returned by the server, the execution main body may call an interactive plug-in of the target annotation template, display the data to be annotated, which is judged by the data, and display the data to be annotated to the annotation user through a screen.

Step 840, responding to the received marking instruction of the marking user, and generating a marking result corresponding to the data to be marked judged by the data.

In this embodiment, the execution main body may receive a labeling instruction of a labeling user for the displayed data to be labeled, where the labeling instruction may be labeling category information corresponding to the target labeling template and/or a corresponding labeling manner. The execution main body can perform data annotation on the data to be annotated judged by the data according to the annotation instruction of the annotation user so as to generate a corresponding annotation result.

The data annotation method provided by the embodiment of the disclosure receives data to be annotated corresponding to a target annotation template uploaded by an annotation user, then sends authentication information and the data to be annotated of the target annotation template to a server, so that the server performs data judgment on the data to be annotated based on the authentication information and a service plug-in of the target annotation template, the service plug-in is used for representing back-end processing logic and storage service of the target annotation template, then responds to the data to be annotated judged by the data returned by the server, displays the data to be annotated judged by the data based on an interactive plug-in of the target annotation template, and finally responds to an annotation instruction received from the annotation user to generate an annotation result corresponding to the data to be annotated judged by the data, so that the annotation user can select the target annotation template according to own requirements, and the target labeling template can be utilized to label the data to be labeled, so that the labeling user can use the required labeling template, and the data labeling efficiency and the data labeling effect are improved.

Referring to FIG. 9, FIG. 9 shows a flow chart 900 of another embodiment of a data annotation process. The data annotation method can comprise the following steps:

step 910, receiving data to be annotated corresponding to the target annotation template uploaded by the annotation user.

Step 910 of this embodiment may be performed in a manner similar to step 810 in the embodiment shown in fig. 8, which is not described herein again.

And 920, sending the authentication information of the target labeling template and the data to be labeled to the server side so that the server side judges the data to be labeled based on the authentication information and the service plug-in of the target labeling template.

Step 920 of this embodiment may be performed in a manner similar to step 820 of the embodiment shown in fig. 8, which is not described herein again.

Step 930, in response to receiving the data to be annotated, which is returned by the server and is judged by the data, displaying the data to be annotated, which is judged by the data, based on the display plug-in of the target annotation template.

The interactive plug-ins may include a presentation plug-in for presenting presentation logic of the target annotation template and a front-end logic plug-in for presenting front-end processing logic of the target annotation template.

In this embodiment, after receiving the data to be annotated, which is judged by the data and returned by the server, the execution main body may call a display plug-in of the target annotation template, display the data to be annotated, which is judged by the data, and display the data to be annotated to the annotation user through a screen.

And 940, responding to the received marking instruction of the marking user, performing logic processing on the marking instruction based on the front-end logic plug-in of the target marking template, and determining the processed marking instruction.

In this embodiment, the execution main body may receive a labeling instruction of a labeling user for the displayed data to be labeled, where the labeling instruction may be the labeling category information corresponding to the target labeling template and the labeling mode corresponding to the labeling category information. The execution main body can call the front-end logic plug-in of the target labeling template to perform logic processing on the labeling instruction, judge whether the labeling instruction accords with the processing logic of the front-end logic plug-in, and directly determine the labeling instruction as the processed labeling instruction if the labeling instruction accords with the processing logic of the front-end logic plug-in. If the marking instruction does not accord with the processing logic of the front-end logic plug-in, the front-end logic plug-in is called to carry out logic processing on the marking instruction, so that the marking instruction accords with the processing logic of the front-end logic plug-in, and the processed marking instruction is determined.

And step 950, generating a labeling result corresponding to the data to be labeled, which is judged by the data, based on the processed labeling instruction.

In this embodiment, after the execution main body obtains the processed annotation instruction, the execution main body may perform data annotation on the data to be annotated, which is determined by the data, according to the processed annotation instruction, so as to generate a corresponding annotation result.

In this embodiment, the data to be labeled is displayed and the labeling instruction is judged according to the display plug-in and the front-end logic plug-in, so that the labeling instruction is more attached to the target labeling template, and the accuracy and the labeling effect of data labeling are improved.

As an optional implementation manner, the data annotation method may further include the following steps: and responding to the generated marking result of the data to be marked, and sending the marking result to the server side so that the server side performs data processing on the marking result on the basis of the authentication information and the service plug-in and stores the processed marking result into a corresponding database.

Specifically, after the execution main body generates a labeling result of the data to be labeled according to the labeling instruction of the labeling user, the execution main body may send the labeling result to the server, where the labeling result may be labeled data generated based on the labeling instruction or a labeling instruction of the labeling user.

After receiving the labeling result of the data to be labeled, the server may call a service plug-in corresponding to the target labeling template to perform data processing on the labeling result, where the data processing may include processing operations such as further verification or further calculation on the labeling result, so as to obtain a processed labeling result. The server can store the processed labeling result into a corresponding database according to the authentication information of the target labeling template.

In the implementation mode, the marking result is further processed by the service plug-in, so that the marking result is more accurate, the processed marking result is stored in the corresponding database based on the authentication information, so that the marking result can be stored in the database corresponding to the target marking template, and the pertinence of data storage is realized.

With further reference to fig. 10, as an implementation of the method shown in the above-mentioned figures, the present disclosure provides an embodiment of an annotation template generation apparatus, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 1, and the apparatus may be specifically applied to various electronic devices.

As shown in fig. 10, the annotation template generation apparatus 1000 of the present embodiment includes: an acquisition module 1010, a first generation module 1020, and a second generation module 1030.

The obtaining module 1010 is configured to, in response to receiving a development instruction of a development user for a target annotation template, obtain an interactive plug-in corresponding to the target annotation template, where the interactive plug-in is used to represent front-end processing logic and presentation logic of the target annotation template;

a first generating module 1020 configured to generate a service plug-in corresponding to the target annotation template based on a back-end service development operation of a development user on the target annotation template, where the service plug-in is used to represent back-end processing logic and storage service of the target annotation template;

a second generating module 1030 configured to generate the target annotation template based on the interaction plug-in and the service plug-in response to receiving a generation instruction for the target annotation template.

In some optional manners of the embodiment, the interactive plug-ins include a presentation plug-in for presenting presentation logic of the target annotation template; and an obtaining module 1010, further configured to: presenting a first configuration interface aiming at the display plug-in response to receiving a development instruction of a development user aiming at a target annotation template; receiving a plurality of label category information and/or corresponding label mode selection information input by a development user in a first configuration interface; and acquiring a display plug-in corresponding to the target labeling template according to the plurality of labeling category information and/or the corresponding labeling mode selection information.

In some optional modes of the embodiment, the interactive plug-in includes a front-end logic plug-in for characterizing front-end processing logic of the target annotation template; and, an obtaining module 1010, further configured to: presenting a second configuration interface aiming at the front-end logic plug-in response to receiving a development instruction of a development user aiming at the target labeling template; receiving a front-end processing logic configuration file input by a developer in a second configuration interface, wherein the front-end processing logic configuration file is used for representing the processing logic of the target annotation template in the annotation process; and acquiring a front-end logic plug-in corresponding to the target marking template according to the processing logic configuration file.

In some alternatives of the present embodiment, the first generating module 1020 is further configured to: presenting a third configuration interface aiming at the service plug-in response to receiving the back-end service development operation of a development user aiming at the target annotation template; receiving a back-end processing logic configuration file and a database for storing data, wherein the back-end processing logic configuration file is input by a developer in a third configuration interface and is used for representing processing logic of data to be labeled and a labeling result; and acquiring the service plug-in corresponding to the target labeling template according to the back-end processing logic configuration file and the database for storing data.

In some optional aspects of this embodiment, the second generating module 1030 is further configured to: in response to receiving a generation instruction for the target labeling template, acquiring authentication information corresponding to the target labeling template; and generating the target labeling template based on the interactive plug-in, the service plug-in and the authentication information of the target labeling template.

The annotation template generation device provided by the embodiment of the disclosure can enable a development user to create a required annotation template according to self requirements by responding to the received development instruction of the development user for a target annotation template, acquiring an interactive plug-in corresponding to the target annotation template, wherein the interactive plug-in is used for representing the front-end processing logic and the display logic of the target annotation template, then generating a service plug-in corresponding to the target annotation template based on the development operation of the development user for the back-end service of the target annotation template, the service plug-in is used for representing the back-end processing logic and the storage service of the target annotation template, and finally generating the target annotation template based on the interactive plug-in and the service plug-in response to the received generation instruction for the target annotation template, so that the annotation template can adapt to the user requirements, the number and the types of the annotation templates can be enriched rapidly, and the richness of the annotation template is improved, and the generation process of the annotation template is not limited to the existing developers, more users can participate in the generation and the updating of the annotation template, the updating efficiency of the annotation template is improved, and therefore the data annotation requirements of multiple parties such as a cloud platform and privatized clients can be met.

Those skilled in the art will appreciate that the above-described apparatus may also include some other well-known structure, such as a processor, memory, etc., which is not shown in fig. 10 in order not to unnecessarily obscure embodiments of the present disclosure.

With further reference to fig. 11, as an implementation of the method shown in the above-mentioned figures, the present disclosure provides an embodiment of an annotation template generation apparatus, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 5, and the apparatus may be specifically applied to various electronic devices.

As shown in fig. 11, the annotation template generation apparatus 1100 of the present embodiment includes: a front-end load module 1110, a create module 1120, and a third generate module 1130.

The front-end loading module 1110 is configured to respond to an interactive plugin and a service plugin of a target labeling template sent by a receiving terminal, perform front-end loading of the target labeling template based on the interactive plugin and the service plugin of the target labeling template to obtain a front-end service corresponding to the target labeling template, wherein the front-end service comprises a front-end processing service and a display service of the target labeling template, the interactive plugin is used for representing the front-end processing logic and the display logic of the target labeling template, and the service plugin is used for representing the back-end processing logic and the storage service of the target labeling template;

a creating module 1120 configured to create a backend service of the target annotation template based on the service plug-in of the target annotation template, wherein the backend service comprises a backend processing service and a storage service of the target annotation template;

a third generating module 1130, configured to associate the front-end service and the back-end service, generates a target annotation template.

In some optional aspects of this embodiment, the creating module 1120 is further configured to: generating a back-end logic plug-in corresponding to the target labeling template based on the service plug-in of the target labeling template, and determining a database corresponding to the target labeling template, wherein the back-end logic plug-in is used for representing the processing logic of the data to be labeled and the labeling result; acquiring a general service plug-in corresponding to the target labeling template, wherein the general service plug-in is used for representing a preset service of the target labeling template; and acquiring the back-end service corresponding to the target marking template based on the back-end logic plug-in, the database and the general service plug-in.

In some optional manners of this embodiment, the apparatus further includes a receiving module, where the receiving module is configured to receive authentication information corresponding to the target annotation template sent by the terminal; and, a front-end load module 1110, further configured to: performing front-end loading of the target labeling template based on the interactive plug-in and the authentication information of the target labeling template to obtain front-end service corresponding to the target labeling template; and a creation module 1120, further configured to: and establishing the back-end service of the target labeling template based on the service plug-in and the authentication information of the target labeling template.

In some optional manners of this embodiment, the apparatus further includes: an acquisition module; the acquisition module is configured to respond to the generation of the target labeling template and acquire the running state information of the target labeling template; the third generating module 1130 is configured to generate the operation and maintenance information of the target annotation template based on the operation state information of the target annotation template.

The annotation template generation device provided by the embodiment of the disclosure obtains the front-end service corresponding to the target annotation template by responding to the interactive plugin and the service plugin of the target annotation template sent by the receiving terminal and loading the front end of the target annotation template based on the interactive plugin and the service plugin of the target annotation template, wherein the front-end service comprises the front-end processing service and the display service of the target annotation template, the interactive plugin is used for representing the front-end processing logic and the display logic of the target annotation template, the service plugin is used for representing the rear-end processing logic and the storage service of the target annotation template, then the rear-end service of the target annotation template is created based on the service plugin of the target annotation template, wherein the rear-end service comprises the rear-end processing service and the storage service of the target annotation template, and finally the front-end service and the rear-end service are associated to generate the target annotation template, the method can automatically generate the label template required by the development user based on the interactive plug-in and the service plug-in created by the development user, so that the label template can adapt to the user requirement, the number and the types of the label template can be enriched rapidly, the richness of the label template is improved, the generation process of the label template is not limited to the existing developers, more users can participate in the generation and the update of the label template, the update efficiency of the label template is improved, and the data label requirements of multiple parties such as a cloud platform and a privatized client can be met.

Those skilled in the art will appreciate that the above-described apparatus may also include some other well-known structure, such as a processor, memory, etc., which is not shown in fig. 11 in order to not unnecessarily obscure embodiments of the present disclosure.

With further reference to fig. 12, as an implementation of the method shown in the above-mentioned figures, the present disclosure provides an embodiment of a data annotation device, which corresponds to the embodiment of the method shown in fig. 8, and which can be applied in various electronic devices.

As shown in fig. 12, the data annotation device 1200 of the present embodiment includes: a receiving module 1210, a sending module 1220, a presentation module 1230 and a fourth generating module 1240.

The receiving module 1210 is configured to receive data to be annotated, which is uploaded by an annotation user and corresponds to a target annotation template;

the sending module 1220 is configured to send the authentication information of the target annotation template and the data to be annotated to the server, so that the server performs data judgment on the data to be annotated based on the authentication information and a service plug-in of the target annotation template, where the service plug-in is used for representing back-end processing logic and storage service of the target annotation template;

the display module 1230 is configured to display the data to be annotated judged by the data based on the interactive plug-in of the target annotation template in response to receiving the data to be annotated judged by the data returned by the server;

the fourth generating module 1240 is configured to generate, in response to receiving the annotation instruction of the annotation user, an annotation result corresponding to the data to be annotated, which is determined by the data.

In some optional manners of this embodiment, the interactive plug-ins include a presentation plug-in for characterizing presentation logic of the target annotation template and a front-end logic plug-in for characterizing front-end processing logic of the target annotation template; and, a presentation module 1230, further configured to: responding to the received data to be marked, which is returned by the server and is judged by the data, and displaying the data to be marked, which is judged by the data, by a display plug-in based on the target marking template; and a fourth generating module 1240 further configured to: responding to a received marking instruction of a marking user, carrying out logic processing on the marking instruction based on a front-end logic plug-in of a target marking template, and determining the processed marking instruction; and generating a labeling result corresponding to the data to be labeled, which is judged by the data, based on the processed labeling instruction.

In some optional aspects of this embodiment, the sending module 1220 is further configured to: and responding to the generated labeling result of the data to be labeled, and sending the labeling result to the server side so that the server side performs data processing on the labeling result on the basis of the authentication information and the service plug-in and stores the processed labeling result in a corresponding database.

Those skilled in the art will appreciate that the above-described apparatus may also include some other well-known structures, such as processors, memories, etc., which are not shown in fig. 12 in order to not unnecessarily obscure embodiments of the present disclosure.

In the technical scheme of the disclosure, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the personal information of the related user are all in accordance with the regulations of related laws and regulations and do not violate the good customs of the public order.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

Fig. 13 illustrates a schematic block diagram of an example electronic device 1300 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 13, the electronic device 1300 includes a computing unit 1301 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)1302 or a computer program loaded from a storage unit 1308 into a Random Access Memory (RAM) 1303. In the RAM 1303, various programs and data necessary for the operation of the device 1300 can also be stored. The calculation unit 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input/output (I/O) interface 1305 is also connected to bus 1304.

A number of components in the electronic device 1300 are connected to the I/O interface 1305, including: an input unit 1306 such as a keyboard, a mouse, or the like; an output unit 1307 such as various types of displays, speakers, and the like; storage unit 1308, such as a magnetic disk, optical disk, or the like; and a communication unit 1309 such as a network card, modem, wireless communication transceiver, etc. The communication unit 1309 allows the device 1300 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

Computing unit 1301 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of computing unit 1301 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 1301 executes the respective methods and processes described above, such as the annotation template generation method and the data annotation method. For example, in some embodiments, the annotation template generation method and the data annotation method can be implemented as computer software programs that are tangibly embodied on a machine-readable medium, such as the storage unit 1308. In some embodiments, some or all of the computer program may be loaded onto and/or installed onto device 1300 via ROM 1302 and/or communications unit 1309. When the computer program is loaded into the RAM 1303 and executed by the computing unit 1301, one or more steps of the annotation template generation method and the data annotation method described above may be performed. Alternatively, in other embodiments, the computing unit 1301 may be configured to perform the annotation template generation method and the data annotation method in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A method for generating an annotation template comprises the following steps:

responding to a received development instruction of a development user for a target labeling template, and acquiring an interactive plug-in corresponding to the target labeling template, wherein the interactive plug-in is used for representing front-end processing logic and display logic of the target labeling template;

generating a service plug-in corresponding to the target labeling template based on the development operation of the development user on the back-end service of the target labeling template, wherein the service plug-in is used for representing the back-end processing logic and the storage service of the target labeling template;

in response to receiving a generation instruction for the target annotation template, generating the target annotation template based on the interaction plug-in and the service plug-in.

2. The method of claim 1, wherein the interactive plug-in comprises a presentation plug-in for presenting presentation logic of the target annotation template; and responding to a received development instruction of a development user for the target labeling template, and acquiring an interactive plug-in corresponding to the target labeling template, wherein the interactive plug-in comprises the following steps:

presenting a first configuration interface for the display plug-in response to receiving a development instruction of a development user for a target annotation template;

receiving a plurality of annotation category information and/or corresponding annotation mode selection information input by the development user in the first configuration interface;

and acquiring a display plug-in corresponding to the target labeling template according to the plurality of labeling category information and/or the corresponding labeling mode selection information.

3. The method of claim 1 or 2, wherein the interaction plug-in comprises a front-end logic plug-in for characterizing front-end processing logic of the target annotation template; and responding to a received development instruction of a development user for the target labeling template, and acquiring an interactive plug-in corresponding to the target labeling template, wherein the interactive plug-in comprises the following steps:

presenting a second configuration interface aiming at the front-end logic plug-in response to receiving a development instruction of a development user aiming at a target annotation template;

receiving a front-end processing logic configuration file input by a developer in the second configuration interface, wherein the front-end processing logic configuration file is used for representing the processing logic of the target annotation template in the annotation process;

and acquiring a front-end logic plug-in corresponding to the target marking template according to the processing logic configuration file.

4. The method of claim 1, wherein the generating a service plug-in corresponding to the target annotation template based on a backend service development operation of the development user on the target annotation template comprises:

presenting a third configuration interface for the service plug-in response to receiving a back-end service development operation of the development user for the target annotation template;

receiving a back-end processing logic configuration file and a database for storing data, which are input by the developer in the third configuration interface, wherein the back-end processing logic configuration file is used for representing processing logic of data to be labeled and a labeling result;

and acquiring the service plug-in corresponding to the target marking template according to the back-end processing logic configuration file and a database for storing data.

5. The method of any of claims 1-4, wherein the generating the target annotation template based on the interaction plug-in and the service plug-in response to receiving a generation instruction for the target annotation template comprises:

responding to a received generation instruction aiming at the target labeling template, and acquiring authentication information corresponding to the target labeling template;

and generating the target labeling template based on the interactive plug-in of the target labeling template, the service plug-in and the authentication information.

6. A method for generating an annotation template comprises the following steps:

responding to an interactive plugin and a service plugin of a target labeling template sent by a receiving terminal, and carrying out front-end loading on the target labeling template based on the interactive plugin and the service plugin of the target labeling template to obtain a front-end service corresponding to the target labeling template, wherein the front-end service comprises a front-end processing service and a display service of the target labeling template, the interactive plugin is used for representing the front-end processing logic and the display logic of the target labeling template, and the service plugin is used for representing the back-end processing logic and the storage service of the target labeling template;

creating a back-end service of the target labeling template based on a service plug-in of the target labeling template, wherein the back-end service comprises a back-end processing service and a storage service of the target labeling template;

and associating the front-end service with the back-end service to generate the target labeling template.

7. The method of claim 6, wherein the creating a backend service of the target annotation template based on the service plug-in of the target annotation template comprises:

generating a back-end logic plug-in corresponding to the target labeling template based on the service plug-in of the target labeling template, and determining a database corresponding to the target labeling template, wherein the back-end logic plug-in is used for representing the processing logic of the data to be labeled and the labeling result;

acquiring a general service plug-in corresponding to the target labeling template, wherein the general service plug-in is used for representing a preset service of the target labeling template;

and acquiring the back-end service corresponding to the target marking template based on the back-end logic plug-in, the database and the general service plug-in.

8. The method of claim 6, further comprising: receiving authentication information corresponding to the target marking template sent by the terminal; and the number of the first and second groups,

the interactive plug-in based on the target labeling template carries out front-end loading of the target labeling template to obtain front-end services corresponding to the target labeling template, and the method comprises the following steps:

based on the interactive plug-in of the target labeling template and the authentication information, carrying out front-end loading on the target labeling template to obtain front-end service corresponding to the target labeling template; and

the creating of the back-end service of the target labeling template based on the service plug-in of the target labeling template comprises the following steps:

and creating the back-end service of the target labeling template based on the service plug-in of the target labeling template and the authentication information.

9. The method of any of claims 6-8, further comprising:

responding to the generated target labeling template, and acquiring running state information of the target labeling template;

and generating operation and maintenance information of the target labeling template based on the operation state information of the target labeling template.

10. A method of data annotation, comprising:

receiving data to be marked corresponding to a target marking template uploaded by a marking user;

sending the authentication information of the target labeling template and the data to be labeled to a server so that the server performs data judgment on the data to be labeled based on the authentication information and a service plug-in of the target labeling template, wherein the service plug-in is used for representing the back-end processing logic and the storage service of the target labeling template;

responding to the received data to be marked, which is judged by data and returned by the server, and displaying the data to be marked judged by the data on the basis of the interactive plug-in of the target marking template;

and generating a labeling result corresponding to the data to be labeled judged by the data in response to the received labeling instruction of the labeling user.

11. The method of claim 10, wherein the interaction plug-ins include a presentation plug-in for characterizing presentation logic of the target annotation template and a front-end logic plug-in for characterizing front-end processing logic of the target annotation template; and the number of the first and second groups,

the step of displaying the data to be marked judged by the data based on the interactive plug-in of the target marking template in response to the received data to be marked judged by the data returned by the server side comprises the following steps:

responding to the received data to be marked, which is returned by the server and is judged by data, and displaying the data to be marked, which is judged by the data, on the basis of the display plug-in of the target marking template; and

the generating of the labeling result corresponding to the data to be labeled judged by the data in response to the receiving of the labeling instruction of the labeling user comprises:

responding to the received marking instruction of the marking user, carrying out logic processing on the marking instruction based on a front-end logic plug-in of the target marking template, and determining the processed marking instruction;

and generating a labeling result corresponding to the data to be labeled, which is judged by the data, based on the processed labeling instruction.

12. The method of any of claims 10-11, further comprising:

and responding to the generated marking result of the data to be marked, and sending the marking result to the server, so that the server performs data processing on the marking result on the basis of the authentication information and the service plug-in and stores the processed marking result in a corresponding database.

13. An annotation template generation apparatus comprising:

the acquisition module is configured to respond to a received development instruction of a development user for a target labeling template, and acquire an interactive plug-in corresponding to the target labeling template, wherein the interactive plug-in is used for representing front-end processing logic and display logic of the target labeling template;

the first generation module is configured to generate a service plug-in corresponding to the target labeling template based on a back-end service development operation of the development user on the target labeling template, and the service plug-in is used for representing back-end processing logic and storage service of the target labeling template;

a second generation module configured to generate the target annotation template based on the interaction plug-in and the service plug-in response to receiving a generation instruction for the target annotation template.

14. An annotation template generation apparatus comprising:

the front-end loading module is configured to respond to an interactive plugin and a service plugin of a target labeling template sent by a receiving terminal, and perform front-end loading of the target labeling template based on the interactive plugin and obtain a front-end service corresponding to the target labeling template, wherein the front-end service comprises a front-end processing service and a display service of the target labeling template, the interactive plugin is used for representing the front-end processing logic and the display logic of the target labeling template, and the service plugin is used for representing the back-end processing logic and the storage service of the target labeling template;

a creating module configured to create a backend service of the target annotation template based on a service plug-in of the target annotation template, wherein the backend service comprises a backend processing service and a storage service of the target annotation template;

a third generation module configured to associate the front-end service and the back-end service to generate the target annotation template.

15. A data annotation device, comprising:

the receiving module is configured to receive data to be labeled, corresponding to the target labeling template, uploaded by a labeling user;

the sending module is configured to send the authentication information and the data to be labeled of the target labeling template to the server side so that the server side can judge the data of the data to be labeled based on the authentication information and a service plug-in of the target labeling template, wherein the service plug-in is used for representing the back-end processing logic and the storage service of the target labeling template;

the display module is configured to respond to the received data to be annotated, which is returned by the server and judged by data, and display the data to be annotated judged by the data based on the interactive plug-in of the target annotation template;

and the fourth generation module is configured to generate a labeling result corresponding to the data to be labeled, which is judged by the data, in response to receiving a labeling instruction of the labeling user.

16. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.

17. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-12.

18. A computer program product comprising computer program/instructions, characterized in that the computer program/instructions, when executed by a processor, implement the steps of the method of any of claims 1-12.