CN114442876A - Management method, device and system of marking tool - Google Patents
Management method, device and system of marking tool Download PDFInfo
- Publication number
- CN114442876A CN114442876A CN202011192361.2A CN202011192361A CN114442876A CN 114442876 A CN114442876 A CN 114442876A CN 202011192361 A CN202011192361 A CN 202011192361A CN 114442876 A CN114442876 A CN 114442876A
- Authority
- CN
- China
- Prior art keywords
- labeling
- control
- marking
- label
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 title description 23
- 238000002372 labelling Methods 0.000 claims abstract description 324
- 238000000034 method Methods 0.000 claims abstract description 45
- 230000000694 effects Effects 0.000 claims description 47
- 238000012549 training Methods 0.000 description 82
- 238000013528 artificial neural network Methods 0.000 description 40
- 239000003550 marker Substances 0.000 description 19
- 238000004891 communication Methods 0.000 description 15
- 230000006870 function Effects 0.000 description 13
- 230000008569 process Effects 0.000 description 12
- 238000012545 processing Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 11
- 238000004590 computer program Methods 0.000 description 9
- 230000036544 posture Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 5
- 230000002452 interceptive effect Effects 0.000 description 5
- 230000005284 excitation Effects 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 210000002569 neuron Anatomy 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000011478 gradient descent method Methods 0.000 description 2
- 230000009191 jumping Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 210000000216 zygoma Anatomy 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013529 biological neural network Methods 0.000 description 1
- 230000003925 brain function Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/0482—Interaction with lists of selectable items, e.g. menus
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The embodiment of the application discloses a method, a device and a system for managing a marking tool, which are used in the field of labor. The method comprises the following steps: establishing a labeling project according to the labeling task; binding a plurality of labeling controls and/or a plurality of label sets for the labeling items; establishing an operation interface corresponding to the labeling item; the operation interface comprises a plurality of labeling controls and/or a plurality of label sets, and the operation interface is used for labeling the operation object according to the plurality of labeling controls and/or the label sets; therefore, a marking operator can utilize the plurality of marking controls to carry out multi-task marking on the same operation object under the same marking item, and marking efficiency is improved.
Description
Technical Field
The embodiment of the application relates to the field of artificial intelligence, in particular to a method, a device and a system for managing a labeling tool.
Background
The neural network is a complex network formed by connecting a large number of neurons, reflects many basic characteristics of human brain functions, and is a highly complex nonlinear dynamical learning system; as an operational model, each node of the neural network represents a specific output function, called activation function (activation function), and the connection between each two nodes represents a weighted value of the connection signal to the next node, called weighted value; the output of the neural network therefore differs depending on the connection mode of the network, the weight value, and the excitation function.
The training process of the neural network refers to a process of continuously optimizing neural network parameters, namely a process of continuously approaching the neural network parameters to correct parameters by performing iterative operation on a large number of training samples; specifically, each training sample comprises an input value and label information (correct output value), the input value of the training sample is input into a neural network, a model output value corresponding to the input value is obtained through neural network operation, then the neural network parameters are updated through a gradient descent method through the difference between the model output value and the label information, and finally the correct parameter value is approached through a large number of iteration processes; therefore, the method can be used for correctly labeling the sample data and acquiring the labeling information of the sample data, and becomes a most basic link for neural network training.
The labeling system provides a large number of high-quality training samples for the training process of the neural network, wherein the training samples comprise a plurality of labeling controls, and different labeling controls are used for different labeling tasks; in the existing labeling system, because each labeling control is independent, when labeling different tasks for the same sample, a labeler needs to continuously switch pages corresponding to the labeling controls, and needs to create labeling items under operation pages of a plurality of labeling controls to finish labeling in sequence; therefore, a lot of switching and creating operations are required for a label operator, and the labeling efficiency is low; therefore, how to optimize the annotation system and provide a more efficient and more convenient annotation control operation interface becomes a problem to be solved urgently.
Disclosure of Invention
The embodiment of the application provides a method, a device and a system for managing a labeling tool, which are used for combining a plurality of labeling controls, so that a marker can label a training sample by using the plurality of labeling controls under the same labeling project, and the labeling efficiency of the training sample is improved.
A first aspect of an embodiment of the present application provides a method for managing a labeling tool, including:
when a marker carries out multi-task marking on a training sample, a marking project can be established in a marking tool management system aiming at the same training sample, and multi-marking operation is carried out on the training sample; then binding a plurality of labeling controls and a plurality of label sets under the labeling project, and finally establishing a corresponding interactive interface for the labeling project; optionally, a plurality of labeling controls and label sets may be displayed on the interactive interface, so that a labeler may perform a plurality of labeling operations on the training sample on the same interface.
The marking system provides an interactive interface comprising a plurality of marking controls for a marker, when a user carries out multitask marking on a training sample, marking operation can be carried out on the same interface, and switching back and forth in interfaces corresponding to the plurality of marking controls is not needed, so that marking efficiency of the marker is improved, and user experience is improved.
In an optional embodiment, according to the labeling requirement of the multitask labeling, multiple labeling activities can be created under a labeling project, then, whether a labeling control needs to be bound to the labeling activities or not is judged, if yes, a labeling control is bound to the labeling activities, and a label set is bound to the labeling controls; and if the labeling control is not needed, directly binding a label set for the labeling activity for integrally describing the training sample, wherein the labeling control is generally used for determining the labeled part of the training sample, the label set is used for describing the label set, and the label set jointly form the labeling information of the training sample.
A plurality of labeling activities of the same training sample are bound under the same labeling item, so that a labeler can intensively perform a plurality of labeling activities on the training sample, and the labeling efficiency of the labeler is improved.
In an optional embodiment, when the annotation system renders an operation interface corresponding to the annotation item according to the binding information, the annotator can input related annotation information through the operation interface; specifically, a annotator can select a labeling control through an operation interface, the system determines a target labeling control according to the selection operation of a user, and then displays an operation interface corresponding to the target labeling control on the operation interface, so that the user can operate a training sample under the target labeling control, input an instruction, the operation interface senses the input of the user, and determines labeling information corresponding to the training sample according to the related operation of the user.
In an optional implementation manner, after a marker operates a training sample under a certain marking control, an operation interface needs to pop up a label set under the marking control, the marker selects labels in the label set through the operation interface to generate labels of the training sample, and finally, the labels and a labeling area under the marking control form labeling information of the training sample.
In an alternative embodiment, the tags in the tagset may include preset tags. The system also comprises a custom tag, wherein the form of the custom tag can be an input box; the annotator can select the preset label as the label of the training sample, can input the expression of the training sample through the input box, and finally generates the annotation information of the training sample according to the content input by the annotator.
In an optional embodiment, a network model may be further bound to each labeling control, and the network model may generate a pre-label of the training sample according to the labeling operation of the labeling operator, and then finally generate labeling information of the training sample through the pre-label.
A second aspect of the embodiments of the present application provides a device for managing a labeling tool, including:
the establishing unit is used for establishing an annotation item according to the annotation task;
the processing unit is used for binding a plurality of labeling controls and/or a plurality of label sets for the labeling items;
the establishing unit is further used for establishing an operation interface corresponding to the labeling item; the operation interface comprises the plurality of labeling controls and/or the plurality of label sets, and the operation interface is used for labeling the operation object according to the plurality of labeling controls and/or the plurality of label sets.
In one optional embodiment, the annotation item comprises a plurality of annotation activities; the processing unit is specifically configured to bind a plurality of labeling controls to the plurality of labeling activities; binding a label set for each labeling control; the labeling control is used for determining a marking position of the operation object, and the label set is used for describing the marking position.
In an optional embodiment, the management apparatus of the annotation tool further includes a receiving unit and a determining unit, where the receiving unit is configured to receive, on the operation interface, a first selection operation of the annotation control by a user;
the determining unit is used for determining a target labeling control according to the first selection operation;
the receiving unit is further configured to receive, under the target labeling control, a labeling operation input for the operation object, where the labeling operation corresponds to the target labeling control;
and the processing unit is further used for labeling the operation object according to the labeling operation.
In an optional embodiment, the processing unit is specifically configured to display, according to the tagging operation, a tab set corresponding to the target tagging control;
the receiving unit is further configured to receive a second selection operation for the tab set corresponding to the target annotation control;
the determining unit is further configured to determine a target tag according to the second selecting operation, and determine the labeling information of the operation object according to the target tag.
In an optional embodiment, the tab set includes a custom tab, and the receiving unit is further configured to receive an input operation on the custom tab;
the determining unit is further configured to determine the content of the custom tag according to the input operation; and determining the labeling information of the operation object according to the content of the custom tag.
In an optional embodiment, the processing unit is further configured to bind a network model for the annotation control; the network model is used for determining the labeling information of the operation object; inputting the operation object and the labeling operation to the network model; obtaining a target label corresponding to the operation object through the network model; and determining the labeling information of the operation object according to the target label.
The third aspect of the present application provides a management system of a labeling tool, comprising an operation interface and a server; the server is configured to: establishing a labeling project according to the labeling task; binding a plurality of labeling controls and/or a plurality of label sets for the labeling items; establishing an operation interface corresponding to the labeling item; the operation interface comprises the plurality of labeling controls and/or the plurality of label sets, and the operation interface is used for labeling the operation object according to the plurality of labeling controls and/or the plurality of label sets.
In one optional embodiment, the annotation item comprises a plurality of annotation activities; the server is specifically configured to: binding a plurality of labeling controls for the plurality of labeling activities; binding a label set for each labeling control; the labeling control is used for determining a marking position of the operation object, and the label set is used for describing the marking position.
In an optional embodiment, the operation interface is configured to receive a first selection operation of the annotation control by a user;
the server is further used for determining a target labeling control according to the first selection operation;
the operation interface is further used for receiving a marking operation input aiming at the operation object under the target marking control, and the marking operation corresponds to the target marking control;
and the server is also used for labeling the operation object according to the labeling operation.
In an optional embodiment, the operation interface is specifically configured to display, according to the tagging operation, a tab set corresponding to the target tagging control; receiving a second selection operation aiming at the label set corresponding to the target labeling control;
the server is specifically configured to determine a target tag according to the second selection operation, and determine the labeling information of the operation object according to the target tag.
In an optional embodiment, the tab set includes a custom tab, and the operation interface is further configured to receive an input operation on the custom tab;
the server is further used for determining the content of the custom tag according to the input operation; and determining the labeling information of the operation object according to the content of the custom tag.
In an optional embodiment, the server is further configured to: binding a network model for the labeling control; the network model is used for determining the labeling information of the operation object; inputting the operation object and the labeling operation to the network model; obtaining a target label corresponding to the operation object through the network model; and determining the labeling information of the operation object according to the target label.
A fourth aspect of the present application provides a chip or a chip system, where the chip or the chip system includes at least one processor and a communication interface, where the communication interface and the at least one processor are interconnected by a line, and the at least one processor is configured to run a computer program or instructions to perform the method for managing an annotation tool described in any one of the possible implementation manners of the first aspect to the first aspect;
the communication interface in the chip may be an input/output interface, a pin, a circuit, or the like.
In one possible implementation, the chip or chip system described above in this application further comprises at least one memory having instructions stored therein. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or may be a storage unit of the chip (e.g., a read-only memory, a random access memory, etc.).
A fifth aspect of the embodiments of the present application provides a computer-readable storage medium, in which a computer program is stored, and when the computer program runs on a computer, the computer program causes the computer to execute a method for managing an annotation tool according to the first aspect.
According to the technical scheme, the embodiment of the application has the following advantages:
in the embodiment of the application, a marking tool management system firstly establishes a marking project according to a marking task; and then binding a plurality of labeling controls and/or a plurality of label sets for the labeling items, and finally rendering an operation interface corresponding to the labeling items, wherein a labeler can complete a plurality of labeling activities under the same labeling item interface without repeated item creating actions or switching back and forth in the interfaces of a plurality of labeling controls, and labels the operation objects under one labeling item, so that each operation object generates a plurality of labels, and the labeling efficiency of the training samples is improved.
Drawings
FIG. 1 is a system architecture diagram of a management system of a marking tool provided in an embodiment of the present application;
fig. 2 is a schematic flowchart illustrating a method for managing a marking tool according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of an operation interface according to an embodiment of the present application;
fig. 4 is a schematic flowchart of a display method of an operation interface according to an embodiment of the present disclosure;
FIG. 5 is a schematic structural diagram of a management apparatus of a marking tool provided in the present application;
FIG. 6 is a system architecture diagram of a management system for a tagging tool provided herein;
fig. 7 is a schematic structural diagram of a management device of another annotation tool provided in the present application.
Detailed Description
The embodiment of the application provides a method, a device and a system for managing a labeling tool, which are used for combining a plurality of labeling controls, so that a marker can label a training sample by using the plurality of labeling controls under the same labeling project, and the labeling efficiency of the training sample is improved.
The technical solutions in the present application will be described in detail below with reference to the drawings in the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all embodiments.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
An Artificial Neural Network (ANN), referred to as Neural Network (NN), is a mathematical model or computational model that mimics the structure and function of a biological neural network. Neural networks are computed from a large number of artificial neuron connections. Generally, an artificial neural network can change an internal structure based on external information, and is an adaptive system. Modern neural networks are a non-linear statistical data modeling tool, a model for modeling complex relationships between inputs and outputs, or for exploring data.
As an operational model, a neural network is composed of a large number of nodes (neurons) and interconnections between them. Each node represents a particular output function, called an excitation function or an activation function. And the connection between every two nodes represents a weighted value, which is called weight and is equivalent to the memory of the artificial neural network. It can be understood that the output of the neural network is different according to the connection mode of the network, the weight value and the excitation function; the network itself is usually an approximation to some algorithm or function in nature, and may also be an expression of a logic strategy.
The training process of the neural network refers to a process of continuously optimizing neural network parameters, including parameters, weight values and the like of an excitation function; performing iterative operation on a large number of training samples to enable the neural network parameters to continuously approach to correct parameters, specifically, each training sample comprises labeling information (correct output value), inputting the training samples to the neural network, obtaining model output values corresponding to the training samples through the neural network operation, and updating the neural network parameters through a gradient descent method through the difference between the model output values and the labeling information; through a large number of iterative processes, all the parameters are finally enabled to approach to correct parameter values, and the calculation accuracy of the neural network is continuously improved. Therefore, the method can be used for correctly labeling the sample data and acquiring the labeling information of the sample data, and becomes a most basic link for neural network training.
The existing labeling system provides a plurality of labeling tools, so that various labeling behaviors can be met, a labeler can add various labels (labeling information) to a training sample by using the plurality of labeling tools, and the labeling tools can exemplarily comprise a labeling control and a label set; wherein, the marking control can be a drawing or marking tool to be used in the marking process; and the label set can represent the information description of the training sample generated in the labeling activity.
Illustratively, a certain neural network is a portrait classifier, which aims to distinguish whether a portrait is smiling, and then corresponding training samples are a plurality of groups of portrait pictures; before training, marking a plurality of groups of portrait pictures serving as training samples, wherein key points of the plurality of groups of portrait pictures can be marked by using a graphic marking control, for example, the key points may include canthus, cheilox, cheekbone, and the like; then, a label set is utilized to describe a plurality of groups of portrait pictures, including smiling and non-smiling; finally, the training sample is input into the neural network, the output value of the neural network corresponding to the training sample and the error of the label are reversely transmitted, and the parameters of the neural network are continuously adjusted, so that the output value of the neural network is more and more approximate to the correct classification (label), and the classification performance of the neural network is optimized.
The marking control can be divided into an image control, a character control, an audio control, a video control and the like according to different operation objects; the image controls can comprise a key point control, a rectangular frame control, a broken line control and the like, wherein the key point control is used for marking key points of the image, the rectangular frame control marks a local area of the image, and the broken line control is used for delineating the key area of the image; the text control can comprise an entity marking control or a relation marking control, wherein the entity marking control is used for selecting part of texts in a section of texts, and the relation marking control is used for recording the relation between text entities; the audio controls can include a breakpoint control or an interception control, the breakpoint control is used for splitting a section of audio, and the interception control is used for intercepting part of the audio in the section of audio; the video controls include a key point control, a rectangular frame control, a polygonal line control, and the like, and the specific functions thereof are similar to those of the image controls, and are not specifically described in detail. The labeling control can meet various labeling requirements, and a labeling operator labels each training sample by utilizing the labeling control.
In the existing labeling system, all labeling controls are independent from each other, and when a training sample is to be labeled by using a certain labeling control, a labeling item needs to be created in an interface corresponding to the labeling control, and then the labeling is completed by using the labeling control; however, for the same training sample, there are often multiple labeling tasks that appear together in combination; experiments prove that more labeling information is contained in the same training sample, which is more beneficial to training of a neural network, for example, in the above example, to train the capability of the neural network to judge the smile of the portrait, the key points such as the canthus, the labial angle, the cheekbone and the like of a plurality of groups of portrait photos can be marked, and the postures of people in the groups of portrait photos can be marked to assist the neural network in judging whether the people in the image smile or not; therefore, when a plurality of groups of portrait photos are marked, not only key points need to be marked, but also body parts need to be selected by the rectangular frame, namely, the key point control and the rectangular frame control are needed to finish the marking activities of the portrait photos. In the prior art, a marker needs to open a key point control page to create a marking item, mark and store key points of a portrait picture, open a rectangular frame control page to create the marking item, and introduce the portrait picture into a selected area by using a rectangular frame; therefore, a lot of redundant work is generated by a marker, a plurality of projects are required to be created for the same training sample, and switching among different projects is required, so that the marking efficiency is seriously influenced. Therefore, a more efficient labeling system is determined, and a labeling operator can conveniently perform multiple labeling activities on the same training sample, which becomes an urgent problem to be solved.
Based on the problems, the application provides a management system of a labeling tool, which is used for combining a plurality of labeling controls and improving the labeling efficiency of a labeler for performing multi-task labeling on the same training sample. Fig. 1 is a system architecture diagram of a management system of a labeling tool according to an embodiment of the present application, where the system architecture diagram includes a tool configuration module, a labeling interface display module, and a labeling interface storage module.
The tool configuration module is a key module in the application and is used for combining a plurality of labeling controls, a user configures a plurality of different labeling tools in the tool configuration module, and then a labeling interface is rendered according to configuration information of the tools for a annotator to use a plurality of labeling tools on one labeling interface.
The marking interface display module renders the display modes and the use forms of a plurality of marking controls according to the configuration information in the tool configuration module, and renders the display modes and the use forms of a plurality of types of label sets according to the relationship between the label sets and the marking controls to be provided for man-machine interaction interfaces of marking personnel.
The marking interface storage module receives the content marked by the marker through the marking interface and stores the mark (marking control) and the description (label information) corresponding to the training sample; illustratively, it can record the mark of the control on the sample by means of key-value pair, and hang under its control; the label set records labels in a key-value pair mode, the control and the label set are hung under the related labeling activities, and a plurality of labeling activities are hung under one labeling item, so that the labeling result is stored by using structured data.
Fig. 2 is a schematic flowchart illustrating a method for managing a marking tool according to an embodiment of the present disclosure; as shown in fig. 2, the management method includes the steps of:
201. and establishing a labeling project according to the labeling task.
When a marker is required to mark a training sample, a marking item is established first, illustratively, if a neural network trained by the training sample is a human image classifier and aims to analyze whether a human face in a human image picture is smiling, a marking task is to mark the training sample with a smile, so that the marking item is required to be established first, and it can be understood that marking actions under the marking item are all performed around whether the human face in the training sample is smiling.
The marking system can provide a front-end interface, and a marker can input a related instruction in the front-end interface, so that a server corresponding to the marking system establishes a marked item according to the instruction, and training samples are hung under the marked item.
202. And creating a plurality of annotation activities under the annotation item.
After the annotation item is established, a plurality of annotation activities can be established under the annotation item, for example, in the above example, to enable the neural network to more accurately judge whether the face in the portrait picture smiles, the key points such as the canthus and the lip angle can be marked, and the age, the body posture and the like corresponding to the face can be identified; it can be understood that these information can assist in judging whether a person smiles, for example, by judging the age corresponding to the face of a person, the radian of the mouth corner of persons of different ages when smiling can be determined, and by the posture of the person, it can be known that the person in the jumping posture is more likely to be happy and the probability of smiling is high; therefore, under the annotation item, a plurality of annotation activities may need to be established, and for example, three annotation activities may be included: marking key points such as canthus, mouth corner and the like; marking the face area; and marking the portrait posture in the portrait picture.
203. Sequentially judging whether each marking activity needs to bind the control; if yes, go to step 204, otherwise go to step 205.
When a plurality of labeling activities are established, corresponding tools need to be bound for each labeling activity, firstly, whether a labeling control needs to be bound or not can be judged according to the type of each labeling activity, if so, the corresponding control needs to be bound for the labeling activity, and if not, a label set can be directly added to label a training sample.
Illustratively, when the labeling activity is the action of labeling the portrait in the portrait picture, a control needs to be labeled to check the body part of the portrait, and thus the labeling control needs to be bound; and when the marking activity is used for marking the definition of the portrait picture, the marking control can be not bound, and the label set corresponding to the definition can be directly bound, so that a marker can select corresponding numbers in the label set to directly describe the definition of the portrait picture.
204. And binding a labeling control for the labeling activity according to the type of the labeling activity.
When the labeling activity needs to bind the labeling control, selecting the corresponding labeling control according to the type of the labeling activity to bind; illustratively, in the example of step 202, three labeling controls may be bound for the three labeling activities, that is, the key point controls are bound for the labeling activities that mark key points such as the canthus and the mouth corner; a rectangular control is bound in a labeling activity for marking the face area, and the face area is locked; marking the portrait posture, movably binding a broken line control, tracing the portrait body, and the like; it will be appreciated that one annotation item will bind multiple annotation controls.
205. And binding the label set.
For example, in the example shown in step 204, the key point control may correspond to one label set, and the label set may include two labels of "smile" and "not smile"; the rectangular control can correspond to a tab set about age, and the tab set can include a plurality of tabs of "0-10 years", "10-20 years", "20-30 years", and so on; the polyline control can correspond to a tab set related to a gesture, and can comprise a plurality of tabs of 'jumping', 'standing', 'sitting' and the like; it can be understood that the above-mentioned control may also correspond to other types of tab sets, and the setting of the tab set corresponding to each labeling control is related to the labeling activity, which is not limited specifically.
The label set can also be set independently, the label set which is generally set independently is an overall description of the training sample, including the subject, style, summary, etc. of the training sample, and the specific form is not limited.
206. And judging whether the intelligent model is bound, if so, executing the step 207, and if not, returning to the step 203.
Illustratively, an intelligent model may also be bound to each labeling control, the step 205 of binding a label set to the labeling control is generally preset manually, and a labeler needs to select a label in the label set to generate a label of a certain training sample, and the intelligent model may automatically generate a label corresponding to the training sample according to the labeling control, so that the labeling work of the labeler is reduced, and the efficiency of labeling the training sample can be improved.
207. And binding an intelligent model for the labeling activity.
It can be understood that after the corresponding intelligent model is bound for the labeling activity, the labeled contents of the training sample and the labeling control can be input into the intelligent model, and the intelligent model obtains the label of the training sample through calculation. For example, an intelligent model may be bound to the labeling activities, and a plurality of labels of the training sample are obtained through the intelligent model, which is not limited specifically.
208. And establishing an operation interface corresponding to the labeling item.
After a plurality of labeling controls, label sets and intelligent models are bound for the labeling project, the plurality of labeling controls and label sets need to be combined and provided for the corresponding interactive interface of a labeling operator.
Under the interactive interface, a marker can operate a plurality of marking controls, namely the marker can complete a plurality of marking activities under the interface of the same marking item without repeated action of creating the item and switching back and forth in the interfaces of the plurality of marking controls, and marks training samples under one marking item, so that each training sample generates a plurality of labels, and the marking efficiency of the training samples is improved.
Fig. 3 is a schematic structural diagram of an operation interface provided in an embodiment of the present application, and as shown in fig. 3, the operation interface includes a control bar, an operation panel (display sample), a sample list, and a tab set bar. After the labeling item is bound with a plurality of labeling controls, the server can establish an operation interface for the labeling item, and a labeler can perform multi-task labeling on the training sample on the operation interface; fig. 4 is a schematic flowchart of a display method of an operation interface provided in an embodiment of the present application, and as shown in fig. 4, the display method includes:
401. and displaying a control bar on the operation interface.
Firstly, the operation interface of the labeling item can display a control bar firstly, the control bar comprises a plurality of labeling controls bound by the labeling item, specifically, each labeling control can correspond to a button, and a labeler can select the labeling controls by clicking or double clicking the buttons.
402. Receiving a first selection operation of a labeling control in the control bar.
The server determines an instruction input by a user through the received first selection operation, and determines the annotation control selected by the user according to the instruction.
403. And determining a target labeling control according to the first selection operation.
404. And receiving a labeling operation aiming at the training sample under the target labeling control.
After the target labeling control is selected, the operation panel correspondingly provides the function of the target labeling control, illustratively, when the rectangular control is selected, the operation panel can select a rectangular frame of a training sample, illustratively, a marker selects a training sample from a sample list, the training sample is guided into the operation panel, then the rectangular control is selected in a control bar, the rectangular frame is determined by inputting actions such as sliding operation and the like on the operation panel, and a labeling area on the training sample is selected.
405. And displaying the label set corresponding to the target labeling control in a label set bar according to the labeling operation.
After the marker marks the training sample by using the selected target marking control, a label set corresponding to the target marking control can be popped up in a label set column, and it can be understood that the display mode of labels in the label set can include a radio button mode, that is, the marker can only select one label from a plurality of labels, or can select more labels, that is, the marker can select a plurality of labels in the label set; illustratively, the tab set can also provide a custom tab, after selecting the custom tab, a annotator can input corresponding content to generate a tab of the custom content, and the custom content is used as the labeling information of the training sample, and specifically, the display mode of the tab set is not limited.
It can be understood that the display modes of the labeling control in the control bar and the corresponding tab set are associated, for example, when the button of the labeling control is lighted, the corresponding tab set button can be lighted, so that the association relationship between the labeling control and the tab set can be displayed more easily, and the user experience is provided.
406. And receiving a second selection operation aiming at the label set corresponding to the target labeling control.
The label operator can select the labels in the label set, so as to generate the labels of the training sample.
407. And determining a target label according to the second selection operation, and determining the labeling information of the operation object according to the target label.
And generating the labeling information of the training sample according to the content of the selected label.
Fig. 5 is a schematic structural diagram of a management device of a marking tool according to the present application, and as shown in fig. 5, the management device of a marking tool includes:
the establishing unit 501 is used for establishing an annotation item according to the annotation task;
a processing unit 502, configured to bind, for the tagged item, multiple tagged controls and/or multiple tag sets;
the establishing unit 501 is further configured to establish an operation interface corresponding to the annotation item; the operation interface comprises the plurality of labeling controls and/or the plurality of label sets, and the operation interface is used for labeling the operation object according to the plurality of labeling controls and/or the plurality of label sets.
In one optional embodiment, the annotation item comprises a plurality of annotation activities; the processing unit 502 is specifically configured to bind a plurality of labeled controls to the plurality of labeled activities; binding a label set for each labeling control; the labeling control is used for determining a marking position of the operation object, and the label set is used for describing the marking position.
In an optional embodiment, the management apparatus of the annotation tool further includes a receiving unit 503 and a determining unit 504, where the receiving unit 503 is configured to receive a first selection operation of the annotation control by a user on the operation interface;
the determining unit 504 is configured to determine a target annotation control according to the first selection operation;
the receiving unit 503 is further configured to receive, under the target labeling control, a labeling operation input for the operation object, where the labeling operation corresponds to the target labeling control;
the processing unit 502 is further configured to label the operation object according to the labeling operation.
In an optional embodiment, the processing unit 502 is specifically configured to display, according to the tagging operation, a tab set corresponding to the target tagging control;
the receiving unit 503 is further configured to receive a second selection operation for the tab set corresponding to the target annotation control;
the determining unit 504 is further configured to determine a target tag according to the second selecting operation, and determine the labeling information of the operation object according to the target tag.
In an optional embodiment, the tab set includes a custom tab, and the receiving unit 503 is further configured to receive an input operation on the custom tab;
the determining unit 504 is further configured to determine the content of the custom tag according to the input operation; and determining the labeling information of the operation object according to the content of the custom tag.
In an optional embodiment, the processing unit 502 is further configured to bind a network model for the annotation control; the network model is used for determining the labeling information of the operation object; inputting the operation object and the labeling operation to the network model; obtaining a target label corresponding to the operation object through the network model; and determining the labeling information of the operation object according to the target label.
Fig. 6 is a system architecture diagram of a management system of an annotation tool provided in the present application, and as shown in fig. 6, the management system of the annotation tool includes an operation interface 601 and a server 602.
The server 602 is configured to: establishing a labeling project according to the labeling task; binding a plurality of labeling controls and/or a plurality of label sets for the labeling items; establishing an operation interface corresponding to the labeling item; the operation interface comprises the plurality of labeling controls and/or the plurality of label sets, and the operation interface is used for labeling the operation object according to the plurality of labeling controls and/or the plurality of label sets.
In one optional embodiment, the annotation item comprises a plurality of annotation activities; the server 602 is specifically configured to: binding a plurality of labeling controls for the plurality of labeling activities; binding a label set for each labeling control; the labeling control is used for determining a marking position of the operation object, and the label set is used for describing the marking position.
In an optional embodiment, the operation interface 601 is configured to receive a first selection operation of the annotation control by a user;
the server 602 is further configured to determine a target annotation control according to the first selection operation;
the operation interface 601 is further configured to receive, under the target labeling control, a labeling operation input for the operation object, where the labeling operation corresponds to the target labeling control;
the server 602 is further configured to label the operation object according to the labeling operation.
In an optional embodiment, the operation interface 601 is specifically configured to display a tab set corresponding to the target labeling control according to the labeling operation; receiving a second selection operation aiming at the label set corresponding to the target labeling control;
the server 602 is specifically configured to determine a target tag according to the second selection operation, and determine the labeling information of the operation object according to the target tag.
In an optional embodiment, the tab set includes a custom tab, and the operation interface 601 is further configured to receive an input operation on the custom tab;
the server 602 is further configured to determine the content of the custom tag according to the input operation; and determining the labeling information of the operation object according to the content of the custom tag.
In an optional embodiment, the server 602 is further configured to: binding a network model for the labeling control; the network model is used for determining the labeling information of the operation object; inputting the operation object and the labeling operation to the network model; obtaining a target label corresponding to the operation object through the network model; and determining the labeling information of the operation object according to the target label.
Referring to fig. 7, a schematic structural diagram of another annotation tool management apparatus 700 according to an embodiment of the present application is shown, where the annotation tool management apparatus 700 includes: a processor 701, a memory 702, and a communication interface 703.
The processor 701, the memory 702, and the communication interface 703 are connected to each other by a bus; the bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 7, but this is not intended to represent only one bus or type of bus.
The memory 702 may include volatile memory (volatile memory), such as random-access memory (RAM); the memory may also include a non-volatile memory (non-volatile memory), such as a flash memory (flash memory), a Hard Disk Drive (HDD) or a solid-state drive (SSD); the memory 702 may also comprise a combination of the above types of memory.
The processor 701 may be a Central Processing Unit (CPU), a Network Processor (NP), or a combination of a CPU and an NP. The processor 701 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a Programmable Logic Device (PLD), or a combination thereof. The PLD may be a Complex Programmable Logic Device (CPLD), a field-programmable gate array (FPGA), a General Array Logic (GAL), or any combination thereof.
The communication interface 703 may be a wired communication interface, such as an ethernet interface, a wireless communication interface, or a combination thereof. The ethernet interface may be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface may be a WLAN interface, a cellular network communication interface, a combination thereof, or the like.
The processor 701 is configured to execute the computer program or the instructions in the memory 702 to perform the method for managing an annotation tool described in any one of the possible implementations of the embodiment shown in fig. 2 and the method for displaying an operation interface described in any one of the possible implementations of the embodiment shown in fig. 4;
the embodiment of the present application further provides a chip or a chip system, where the chip or the chip system includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is configured to run a computer program or an instruction to perform a method for managing a labeling tool described in any one of possible implementations of the embodiment shown in fig. 2 and a method for displaying an operation interface described in any one of possible implementations of the embodiment shown in fig. 4;
the communication interface in the chip may be an input/output interface, a pin, a circuit, or the like.
In one possible implementation, the chip or chip system described above in this application further comprises at least one memory having instructions stored therein. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or may be a storage unit of the chip (e.g., a read-only memory, a random access memory, etc.).
An embodiment of the present application further provides a computer storage medium for storing computer software instructions for the management apparatus based on a marking tool, which includes a program designed for executing the management apparatus based on a marking tool.
The embodiment of the present application further provides a computer program product, where the computer program product includes computer software instructions, and the computer software instructions may be loaded by a processor to implement the flow in the management method of the annotation tool.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one type of logical functional division, and other divisions may be realized in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like.
Claims (13)
1. A method for managing a marking tool, the method comprising:
establishing a labeling project according to the labeling task;
binding a plurality of labeling controls and/or a plurality of label sets for the labeling items;
establishing an operation interface corresponding to the labeling item; the operation interface comprises the plurality of labeling controls and/or the plurality of label sets, and the operation interface is used for labeling the operation object according to the plurality of labeling controls and/or the plurality of label sets.
2. The method of claim 1, wherein the annotation item comprises a plurality of annotation activities; the binding of the plurality of annotation controls and/or the plurality of tab sets for the annotation item includes:
binding a plurality of labeling controls for the plurality of labeling activities;
binding a label set for each labeling control; the labeling control is used for determining a marking position of the operation object, and the label set is used for describing the marking position.
3. The method according to any one of claims 1 to 2, further comprising:
receiving a first selection operation of a user on the labeling control on the operation interface;
determining a target labeling control according to the first selection operation;
receiving a labeling operation input for the operation object under the target labeling control, wherein the labeling operation corresponds to the target labeling control;
and marking the operation object according to the marking operation.
4. The method according to claim 3, wherein the labeling the operation object according to the labeling operation comprises:
displaying a label set corresponding to the target labeling control according to the labeling operation;
receiving a second selection operation aiming at the label set corresponding to the target labeling control;
and determining a target label according to the second selection operation, and determining the labeling information of the operation object according to the target label.
5. The method of claim 4, wherein the set of tags includes custom tags, the method further comprising:
receiving input operation of the custom tag;
determining the content of the custom tag according to the input operation;
and determining the labeling information of the operation object according to the content of the custom tag.
6. The method according to any one of claims 3 to 5, further comprising:
binding a network model for the labeling control; the network model is used for determining the labeling information of the operation object;
the labeling the operation object according to the labeling operation comprises the following steps:
inputting the operation object and the labeling operation to the network model;
obtaining a target label corresponding to the operation object through the network model;
and determining the labeling information of the operation object according to the target label.
7. The system for managing the marking tool is characterized by comprising an operation interface and a server; the server is configured to:
establishing a labeling project according to the labeling task;
binding a plurality of labeling controls and/or a plurality of label sets for the labeling items;
establishing an operation interface corresponding to the labeling item; the operation interface comprises the plurality of labeling controls and/or the plurality of label sets, and the operation interface is used for labeling the operation object according to the plurality of labeling controls and/or the plurality of label sets.
8. The system of claim 7, wherein the annotation item comprises a plurality of annotation activities; the server is specifically configured to:
binding a plurality of labeling controls for the plurality of labeling activities;
binding a label set for each labeling control; the labeling control is used for determining a marking position of the operation object, and the label set is used for describing the marking position.
9. The system according to any one of claims 7 to 8, wherein the operation interface is configured to receive a first selection operation of the annotation control by a user;
the server is further used for determining a target labeling control according to the first selection operation;
the operation interface is further used for receiving a marking operation input aiming at the operation object under the target marking control, and the marking operation corresponds to the target marking control;
and the server is also used for labeling the operation object according to the labeling operation.
10. The system of claim 9, wherein the operation interface is specifically configured to:
displaying a label set corresponding to the target labeling control according to the labeling operation;
receiving a second selection operation aiming at the label set corresponding to the target labeling control;
the server is specifically configured to determine a target tag according to the second selection operation, and determine the labeling information of the operation object according to the target tag.
11. The system of claim 10, wherein the tab set comprises custom tabs, and wherein the operator interface is further configured to:
receiving input operation of the custom tag;
the server is further used for determining the content of the custom tag according to the input operation; and determining the labeling information of the operation object according to the content of the custom tag.
12. The system of any of claims 9 to 11, wherein the server is further configured to:
binding a network model for the labeling control; the network model is used for determining the labeling information of the operation object;
inputting the operation object and the labeling operation to the network model;
obtaining a target label corresponding to the operation object through the network model;
and determining the labeling information of the operation object according to the target label.
13. A computer-readable storage medium storing one or more computer-executable instructions, wherein when the computer-executable instructions are executed by a processor, the processor performs the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011192361.2A CN114442876A (en) | 2020-10-30 | 2020-10-30 | Management method, device and system of marking tool |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011192361.2A CN114442876A (en) | 2020-10-30 | 2020-10-30 | Management method, device and system of marking tool |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114442876A true CN114442876A (en) | 2022-05-06 |
Family
ID=81357365
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011192361.2A Pending CN114442876A (en) | 2020-10-30 | 2020-10-30 | Management method, device and system of marking tool |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114442876A (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160147840A1 (en) * | 2014-11-21 | 2016-05-26 | International Business Machines Corporation | System for retrieving, visualizing and editing semantic annotations |
| CN108830466A (en) * | 2018-05-31 | 2018-11-16 | 长春博立电子科技有限公司 | A kind of image content semanteme marking system and method based on cloud platform |
| CN108897869A (en) * | 2018-06-29 | 2018-11-27 | 北京百度网讯科技有限公司 | Corpus labeling method, device, equipment and storage medium |
| CN109325213A (en) * | 2018-09-30 | 2019-02-12 | 北京字节跳动网络技术有限公司 | Method and apparatus for labeled data |
| CN110457494A (en) * | 2019-08-01 | 2019-11-15 | 新华智云科技有限公司 | Data mask method, device, electronic equipment and storage medium |
| KR102081037B1 (en) * | 2020-01-28 | 2020-02-24 | 주식회사 루닛 | Method for managing annotation job, apparatus and system supporting the same |
| WO2020044099A1 (en) * | 2018-08-29 | 2020-03-05 | 优视科技新加坡有限公司 | Service processing method and apparatus based on object recognition |
| CN110929729A (en) * | 2020-02-18 | 2020-03-27 | 北京海天瑞声科技股份有限公司 | Image annotation method, image annotation device and computer storage medium |
| CN110972499A (en) * | 2018-07-07 | 2020-04-07 | 初之光信息科技(新加坡)有限公司 | Labeling system of neural network |
| US20200167689A1 (en) * | 2018-11-28 | 2020-05-28 | Here Global B.V. | Method, apparatus, and system for providing data-driven selection of machine learning training observations |
| CN111367445A (en) * | 2020-03-31 | 2020-07-03 | 中国建设银行股份有限公司 | Image annotation method and device |
| CN111580947A (en) * | 2020-04-29 | 2020-08-25 | 中国科学院空天信息创新研究院 | Online collaborative remote sensing image annotation system based on artificial intelligence |
-
2020
- 2020-10-30 CN CN202011192361.2A patent/CN114442876A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160147840A1 (en) * | 2014-11-21 | 2016-05-26 | International Business Machines Corporation | System for retrieving, visualizing and editing semantic annotations |
| CN108830466A (en) * | 2018-05-31 | 2018-11-16 | 长春博立电子科技有限公司 | A kind of image content semanteme marking system and method based on cloud platform |
| CN108897869A (en) * | 2018-06-29 | 2018-11-27 | 北京百度网讯科技有限公司 | Corpus labeling method, device, equipment and storage medium |
| CN110972499A (en) * | 2018-07-07 | 2020-04-07 | 初之光信息科技(新加坡)有限公司 | Labeling system of neural network |
| WO2020044099A1 (en) * | 2018-08-29 | 2020-03-05 | 优视科技新加坡有限公司 | Service processing method and apparatus based on object recognition |
| CN109325213A (en) * | 2018-09-30 | 2019-02-12 | 北京字节跳动网络技术有限公司 | Method and apparatus for labeled data |
| US20200167689A1 (en) * | 2018-11-28 | 2020-05-28 | Here Global B.V. | Method, apparatus, and system for providing data-driven selection of machine learning training observations |
| CN110457494A (en) * | 2019-08-01 | 2019-11-15 | 新华智云科技有限公司 | Data mask method, device, electronic equipment and storage medium |
| KR102081037B1 (en) * | 2020-01-28 | 2020-02-24 | 주식회사 루닛 | Method for managing annotation job, apparatus and system supporting the same |
| CN110929729A (en) * | 2020-02-18 | 2020-03-27 | 北京海天瑞声科技股份有限公司 | Image annotation method, image annotation device and computer storage medium |
| CN111367445A (en) * | 2020-03-31 | 2020-07-03 | 中国建设银行股份有限公司 | Image annotation method and device |
| CN111580947A (en) * | 2020-04-29 | 2020-08-25 | 中国科学院空天信息创新研究院 | Online collaborative remote sensing image annotation system based on artificial intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Rencher | A review of “methods of multivariate analysis, ” | |
| WO2021109928A1 (en) | Creation method, usage method and apparatus for machine learning scheme template | |
| CN108733722A (en) | A kind of dialogue robot automatic generation method and device | |
| CN109408058B (en) | Front-end auxiliary development method and device based on machine learning | |
| US10540257B2 (en) | Information processing apparatus and computer-implemented method for evaluating source code | |
| CN112331348B (en) | Analysis method and system for set marking, data, project management and non-programming modeling | |
| KR20200138074A (en) | System and method for integrating machine learning and crowd-sourced data annotation | |
| CN110058756B (en) | Image sample labeling method and device | |
| CN113469358A (en) | Neural network training method and device, computer equipment and storage medium | |
| CN110796482A (en) | Financial data classification method and device for machine learning model and electronic equipment | |
| CN116648698A (en) | Dynamic facet ordering | |
| CN110717312B (en) | Text labeling method and device | |
| Fengler et al. | Beyond drift diffusion models: Fitting a broad class of decision and reinforcement learning models with HDDM | |
| CN114049631A (en) | Data labeling method and device, computer equipment and storage medium | |
| CN114270313A (en) | Computer-implemented method, computer program and physical computing environment | |
| CN111523315B (en) | Data processing method, text recognition device and computer equipment | |
| CN113296769A (en) | Data processing method, visual draft processing method, system and electronic equipment | |
| CN117390336A (en) | Webpage process automation method, device, equipment and storage medium | |
| CN112084376A (en) | Map knowledge based recommendation method and system and electronic device | |
| CN114442876A (en) | Management method, device and system of marking tool | |
| CN111859862A (en) | Text data labeling method and device, storage medium and electronic device | |
| CN115620095A (en) | Hand information labeling method, device, equipment and storage medium | |
| CN115760775A (en) | Tomato disease detection method, device, equipment and storage medium | |
| US20220147862A1 (en) | Explanatory confusion matrices for machine learning | |
| JP6894461B2 (en) | Information processing equipment, programs, and information processing methods |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220507 Address after: 523799 Room 101, building 4, No. 15, Huanhu Road, Songshanhu Park, Dongguan City, Guangdong Province Applicant after: Petal cloud Technology Co.,Ltd. Address before: 523808 Southern Factory Building (Phase I) Project B2 Production Plant-5, New Town Avenue, Songshan Lake High-tech Industrial Development Zone, Dongguan City, Guangdong Province Applicant before: HUAWEI DEVICE Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |