CN113268621B

CN113268621B - Picture sorting method and device, electronic equipment and storage medium

Info

Publication number: CN113268621B
Application number: CN202010096648.9A
Authority: CN
Inventors: 尚萌; 谢红伟
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2020-02-17
Filing date: 2020-02-17
Publication date: 2024-04-30
Anticipated expiration: 2040-02-17
Also published as: CN113268621A

Abstract

The embodiment of the application discloses a picture ordering method, a picture ordering device, electronic equipment and a storage medium, which relate to the field of artificial intelligence and specifically comprise the following steps: calculating at least one picture attribute corresponding to each target ordered picture, and generating comparison feature vectors corresponding to each target ordered picture according to the picture attribute; calculating the picture score of each target ordered picture according to the matching degree between each comparison feature vector and the standard feature vector of each picture in the standard high-quality picture set and/or the standard low-quality picture set; and sorting the target sorting pictures according to the picture scores of the target sorting pictures. According to the technical scheme provided by the embodiment of the application, the influence of human factors in the picture sorting process can be reduced to the greatest extent, the accuracy of sorting high-quality pictures is improved, the hit rate of the first screen picture on the user requirement is improved, and the click rate of the picture is further improved.

Description

Picture sorting method and device, electronic equipment and storage medium

Technical Field

The embodiment of the application relates to an image processing technology, in particular to the field of artificial intelligence, and specifically relates to a picture ordering method, a picture ordering device, electronic equipment and a storage medium.

Background

With the continuous development of computer technology, the demand of the picture of the user is increasing, and the demand of the picture of the user can be generally met by feeding back the picture search result or the picture recommendation result to the user. Before providing the picture searching or recommending result, the pictures are required to be ranked first, and the pictures are fed back according to the ranking result, the ranking purpose is to enable the high-quality pictures to be ranked forward, the probability that the first screen content directly hits the user requirement is improved, and therefore the click rate of the user on the pictures is improved.

In the prior art, when sorting pictures, the quality of the pictures is calculated mainly according to multi-dimensional objective factors (such as definition or aesthetic degree) of the pictures, scoring is performed on each picture, and the pictures are sorted according to scoring results. The scoring mode mainly comprises the following steps: and presetting scoring weights corresponding to the objective factors respectively, and obtaining a final scoring result by a weighted summation mode.

The inventors have found that the following drawbacks exist in the prior art in the process of implementing the present application: the scoring result of the picture strongly depends on the setting of the scoring weight, and the scoring weight is generally set by user definition, so that the scoring mode has a certain limitation on the ordering of the pictures.

Content of the application

The embodiment of the application discloses a picture ordering method, a picture ordering device, electronic equipment and a storage medium, which can reduce the influence of human factors in the picture ordering process to the greatest extent and improve the accuracy of high-quality picture ordering.

In a first aspect, an embodiment of the present application discloses a picture ordering method, including:

Calculating at least one picture attribute corresponding to each target ordered picture, and generating comparison feature vectors corresponding to each target ordered picture according to the picture attribute;

Calculating the picture score of each target ordered picture according to the matching degree between each comparison feature vector and the standard feature vector of each picture in the standard high-quality picture set and/or the standard low-quality picture set;

and sorting the target sorting pictures according to the picture scores of the target sorting pictures.

One embodiment of the above application has the following advantages or benefits: by calculating the matching degree between the comparison feature vector of at least one picture attribute corresponding to each target sorting picture and the standard feature vector of each picture in the standard quality picture set and/or the standard feature vector of each picture in the standard quality picture set, the influence of human factors in the picture sorting process can be reduced to the greatest extent, the accuracy of sorting the quality pictures is improved, the hit rate of the first screen picture on the user requirement is improved, and the click rate of the pictures is further improved.

In addition, the picture sorting method according to the above embodiment of the present application may further have the following additional technical features:

Optionally, calculating the picture score of each target ordered picture according to the matching degree between each comparison feature vector and the standard feature vector of each picture in the standard quality picture set and/or the standard quality picture set, including:

Respectively inputting the comparison feature vectors of the target ordered pictures into a pre-trained picture scoring model to obtain picture scores of the target ordered pictures;

the picture scoring model is obtained by training standard feature vectors of pictures in a standard high-quality picture set and a standard low-quality picture set.

One embodiment of the above application has the following advantages or benefits: the method has the advantages that the machine learning model is used for learning the picture characteristics of each picture in the standard high-quality picture set and the poor-quality picture set in advance, and the learned picture scoring model is used for scoring each target sequencing picture, so that a novel mode capable of automatically learning and generating the scoring weight of each picture attribute is provided, the implementation is simple, and the accuracy is high.

Optionally, before calculating at least one picture attribute corresponding to each target ordered picture, the method further includes:

Acquiring an original picture set, and acquiring a standard high-quality picture set according to picture marking results of a plurality of target marking platforms on the original picture set;

constructing a standard inferior picture set according to the standard value range of the at least one picture attribute;

and training a preset machine learning model according to standard feature vectors respectively corresponding to each picture in the standard high-quality picture set and the standard low-quality picture set to obtain the picture scoring model.

One embodiment of the above application has the following advantages or benefits: the specific implementation mode of generating the picture scoring model by training the standard inferior picture set as the training sample by using the standard quality picture set generated by the target labeling platform and the standard value range of at least one picture attribute is provided, so that the quality of the training sample is ensured to the greatest extent, and the scoring accuracy of the picture scoring model can be improved.

Optionally, obtaining an original picture set, and obtaining a standard high-quality picture set according to picture labeling results of a plurality of target labeling platforms on the original picture set, including:

Acquiring the original picture set as a target picture set;

Generating a plurality of positive and negative sample pairs according to the target picture set, and sending the positive and negative sample pairs to a plurality of target labeling platforms for labeling alternative high-quality pictures;

Determining a second picture set from the candidate high-quality pictures fed back by each target labeling platform as a new target picture set according to a preset cross judging mechanism;

Returning to execute the operation of generating a plurality of positive and negative sample pairs according to the target picture set and sending the positive and negative sample pairs to a plurality of target labeling platforms for labeling the alternative high-quality pictures until the end labeling condition is met;

and generating a standard high-quality picture set according to each picture in the current target picture set.

One embodiment of the above application has the following advantages or benefits: the method for marking the alternative high-quality pictures included in the positive and negative sample pairs in multiple rounds by the target marking platform under a preset cross judgment mechanism can accurately determine a standard high-quality picture set corresponding to an original picture set, and the quality of a positive sample used for training a picture scoring model is guaranteed to the greatest extent.

Optionally, before generating a plurality of positive and negative sample pairs according to the target picture set, and sending the positive and negative sample pairs to a plurality of target labeling platforms for labeling the candidate high-quality pictures, the method further comprises:

generating a plurality of standard high-quality pictures in advance, and adding dark pile data into each standard high-quality picture;

Constructing a plurality of standard positive and negative sample pairs according to the plurality of standard high-quality pictures, and sending the standard positive and negative sample pairs to a plurality of alternative labeling platforms;

and determining the target annotation platform in the multiple alternative annotation platforms according to the inclusion condition of the dark pile data in the alternative high-quality pictures fed back by the multiple alternative annotation platforms aiming at the standard positive and negative samples.

One embodiment of the above application has the following advantages or benefits: by adding the hidden pile data into the pre-generated high-quality pictures with a plurality of standards, the target marking platform with the high-quality picture marking standard consistent with the marking standard required by an actual system can be determined in the multiple alternative marking platforms, so that the quality of a positive sample used for training the picture marking model is further ensured.

optionally, generating a standard high-quality picture set according to each picture in the current target picture set includes:

Predicting and scoring each picture in the current target picture set, and generating the standard high-quality picture set after filtering out pictures of which the scoring result does not belong to high-quality pictures in the current target picture set; and/or

And calculating at least one picture attribute of each picture in the current target picture set, and filtering out pictures of which the picture attributes are not matched with the standard value range of the picture attributes in the current target picture set to generate the standard high-quality picture set.

One embodiment of the above application has the following advantages or benefits: the quality of positive samples used for training the picture scoring model is further ensured by filtering out pictures in the standard high-quality picture set, which may affect the accuracy of the model.

Optionally, after generating the standard quality picture set according to each picture in the current target picture set, the method further includes:

Constructing a plurality of picture feature vectors of new standard quality pictures according to the standard value range of the at least one picture attribute and the numerical distribution of the picture attribute of each picture in the standard quality picture set;

constructing a standard inferior picture set according to the standard value range of the at least one picture attribute, wherein the standard inferior picture set comprises:

Determining the quantity value of the standard quality picture according to the quantity value of the picture feature vector of the standard quality picture and the preset proportion value of the standard quality picture and the standard quality picture;

and constructing a standard inferior picture set matched with the number value of the standard inferior pictures.

One embodiment of the above application has the following advantages or benefits: according to the standard value range of the at least one picture attribute and the numerical distribution of the picture attribute of each picture in the standard high-quality picture set, a plurality of picture feature vectors of new standard high-quality pictures are constructed, so that the positive samples used for training the picture scoring model can be ensured to the greatest extent to ensure certain scoring accuracy under different values of the picture attribute, and the universality and the scoring accuracy of the picture scoring model are improved.

optionally, the machine learning model includes: gradient lifting decision tree model.

Optionally, the picture attribute includes at least one of: correlation, aesthetics, sharpness, brightness, whether face is included, whether watermark is included, and whether it is a sensitive picture.

In a second aspect, an embodiment of the present application discloses a picture ordering apparatus, including:

The comparison feature vector generation module is used for calculating at least one picture attribute corresponding to each target ordering picture respectively and generating comparison feature vectors corresponding to each target ordering picture respectively according to the picture attribute;

The picture score calculation module is used for calculating the picture score of each target ordered picture according to the matching degree between each comparison feature vector and the standard high-quality picture set and/or the standard feature vector of each picture in the standard low-quality picture set;

and the picture sorting module is used for sorting the target sorting pictures according to the picture scores of the target sorting pictures.

In a third aspect, an embodiment of the present application further provides an electronic device, including:

at least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of the embodiments of the present application.

In a fourth aspect, embodiments of the present application also provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of any one of the embodiments of the present application.

Other effects of the above alternative will be described below in connection with specific embodiments.

Drawings

The drawings are included to provide a better understanding of the present application and are not to be construed as limiting the application. Wherein:

fig. 1 is a flowchart of a picture sorting method according to a first embodiment of the present application;

Fig. 2 is a flowchart of a picture sorting method according to a second embodiment of the present application;

fig. 3 is a schematic structural diagram of a picture sorting apparatus according to a third embodiment of the present application;

fig. 4 is a block diagram of an electronic device for implementing a method of picture ordering in accordance with an embodiment of the present application.

Detailed Description

Exemplary embodiments of the present application will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present application are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

First embodiment

Fig. 1 is a flowchart of a picture sorting method according to a first embodiment of the present application. The method of the embodiment of the present application may be implemented by the picture sorting device provided by the embodiment of the present application, where the device may be implemented in a software and/or hardware manner, and may be generally integrated in a server, and used in conjunction with a client that provides a picture sorting result to a user, where the server may typically be a search engine server or an information recommendation server, etc.

As shown in fig. 1, the method may include:

S110, calculating at least one picture attribute corresponding to each target ordered picture, and generating comparison feature vectors corresponding to each target ordered picture according to the picture attribute.

In this embodiment, the target ordered pictures specifically refer to a plurality of pictures to be provided to a client for display.

In a specific example, a user inputs a certain picture search condition in the picture search engine client, the picture search engine server obtains a picture search result based on the picture search condition, the picture search result includes a plurality of pictures to be provided for the picture search engine client to display, and the pictures included in the picture search result need to be provided for the picture search engine client after being ordered, so that each picture included in the picture search result can be used as a target ordered picture.

In another specific example, a certain application client may recommend pictures to a user in a Feed stream (information stream) manner, and a certain refresh operation of the user triggers the information recommendation server to obtain a picture recommendation result, where the picture recommendation result includes a plurality of pictures to be provided to the application client for display, and the pictures included in the picture recommendation result need to be provided to the application client after being ordered, so that each picture included in the picture recommendation result may be used as a target ordered picture.

Considering that each target ordering picture can hit the actual demand of the user to a certain extent, if the number of the pictures is large, the pictures need to be paged and provided for the user to display, so how to provide the higher-quality pictures in the target ordering pictures to the user at a position relatively forward is very important.

In this embodiment, by extracting the picture attribute in each target ranking picture, how to score each target ranking picture is determined, and then the ranking of the target ranking pictures can be performed according to the scoring result. Correspondingly, the picture attribute specifically refers to attribute information which can be used for measuring the picture quality.

Wherein the picture quality may include at least one of: correlation, aesthetics, sharpness, brightness, whether a face is included, whether a watermark is included, and whether a sensitive picture (e.g., a yellow picture or a reaction picture, etc.).

Generally, when a picture is stored in a picture library, a POI (Point of Interest ) class, such as entertainment class, scenery class, or food class, to which the picture belongs is predetermined. The relevance refers to a relationship between the content of a picture and the POI class to which the picture belongs.

Alternatively, a classification model may be trained in advance, and a picture may be input into the classification model to obtain a picture classification result (for example, a certain scenic spot, a certain cartoon character, a certain food, etc.), so that the relevance of the picture may be obtained according to the picture classification result of the picture, the POI class to which the picture belongs, and a mapping relationship between the preset picture classification result and the POI class. Typically, the value of the correlation may be a discrete value such as 0,0.5,1.

For example, if the picture classification result of a picture is scenic spot a, the POI cluster to which the picture belongs is scenic spot, and the mapping result of "scenic spot a-scenic spot class-1" is found by querying the mapping relation, it is determined that the correlation degree corresponding to the picture is 1.

The beauty degree specifically means that a picture accords with the beauty degree perceived by a general person, specifically, a beauty degree evaluation model can be trained in advance, and the beauty degree score of the picture can be obtained by inputting a picture into the beauty degree evaluation model, and the numerical range of the scoring value of the beauty degree can be typically [0,1].

Definition refers to the definition degree of a picture detail shadow and its boundary, specifically, a definition recognition model may be trained in advance, and by inputting a picture into the definition recognition model, a definition score of the picture may be obtained, and typically, the numerical range of the score value of the definition may be [0,1].

The brightness refers to the brightness of a picture, and the brightness of a picture may be calculated by various calculation methods, alternatively, three color components of R (red), G (green) and B (blue) of each pixel in a picture may be calculated first, then the brightness value corresponding to each pixel is calculated by the formula (0.241×r ²)+0.691*(G²)+0.068*(B²)), and then the brightness values of the pixels are summed up and normalized after square root, as the brightness score of the picture, typically, the numerical range of the scoring value of the brightness may be [0,1].

Of course, it is understood that other types of picture attributes may be included, such as picture size (length, width, and height), and picture compression rate, etc., which are not limited by the present embodiment.

Alternatively, the method of determining whether a picture includes a face, whether a watermark is included, and whether a picture is a sensitive picture may be implemented for invoking a function interface implemented in advance, for example, by inputting a picture into a face recognition interface, if the face recognition interface returns to 1, determining that the picture includes a face, and if the face recognition interface returns to 0, determining that the picture does not include a face.

Generally, the higher the correlation, the aesthetic, the sharpness and the brightness of a picture, the higher the quality of the picture, and the higher the quality of the picture when a picture does not include a human face, does not include a watermark and is not a sensitive picture. In this embodiment, after each picture attribute of each target ordered picture is obtained by calculation, each picture attribute is combined to obtain a comparison feature vector corresponding to each target ordered picture.

In a specific example, if a picture has a correlation of 1, an aesthetic degree of 0.8, a definition of 0.7, a brightness of 0.65, whether a face is included as 0, whether a watermark is included as 0, and whether a sensitive picture is 0, the obtained picture attributes may be combined to obtain a comparison feature vector of a shape like (1,0.8,0.7,0.65,0,0,0).

S120, calculating the picture score of each target ordered picture according to the matching degree between each comparison feature vector and the standard feature vector of each picture in the standard quality picture set and/or the standard quality picture set.

In this embodiment, the inventors creatively propose: the attribute weight of each picture attribute is not manually set any more, the mode of the attribute weight which is more in line with the actual situation is automatically determined directly according to the characteristics of each picture in the standard high-quality picture set and the standard low-quality picture set, and the picture score of each target ordered picture is calculated.

Specifically, a standard high-quality picture set and a standard low-quality picture set may be acquired first. The standard quality picture set stores the actual determined quality picture, and the standard quality picture set stores the actual determined quality picture.

The standard quality picture set can be obtained from a labeling platform or other third party platforms, and the standard quality picture set can be obtained by obtaining a picture with a particularly low attribute of a certain picture, for example, obtaining a picture with a correlation degree, definition, or brightness equivalent of 0 or a minimum value to directly generate the standard quality picture set.

After the standard high-quality picture set is obtained, at least one picture attribute of each standard high-quality picture can be further obtained, and then the picture attributes can be combined according to a mode similar to S110 to obtain standard high-quality feature vectors of each picture in the standard high-quality picture set, wherein the standard high-quality feature vectors reflect the common picture features of the high-quality pictures to a certain extent;

Similarly, after the standard inferior picture set is obtained, at least one picture attribute of each standard inferior picture can be further obtained, and then the above-mentioned picture attributes can be combined according to a similar manner to S110 to obtain standard inferior feature vectors of each picture in the standard inferior picture set, where the above-mentioned standard inferior feature vectors reflect the common picture features that the inferior pictures should have to a certain extent.

In an optional implementation manner of this embodiment, according to the matching degree between each comparison feature vector and the standard feature vector of each picture in the standard quality picture set and/or the standard quality picture set, the manner of calculating the picture score of each target ordered picture may be:

Taking the calculation of the picture score of the picture a as an example, the vector similarity between the comparison feature vector of the picture a and the standard feature vector (i.e., the standard high-quality feature vector) of each picture in the standard high-quality picture set may be calculated first, and then the calculated vector similarity may be statistically averaged to obtain a first similarity S1, and then may be according to the formula: s=b×s1, and a picture score of the picture a is calculated. Wherein B is a preset total score;

Or, after calculating the vector similarity between the comparison feature vector of the picture a and the standard feature vector (i.e., the standard inferior feature vector) of each picture in the standard inferior picture set, the calculated vector similarity may be statistically averaged to obtain a second similarity S2, and then may be according to the formula: : s=b (1-S2), calculating a picture score for the picture a;

Or the vector similarity of the comparison feature vector of the picture a and each standard high-quality feature vector can be calculated first, then the calculated vector similarity is subjected to statistical average to obtain a first similarity S1, then the vector similarity of the comparison feature vector of the picture a and each standard low-quality feature vector is calculated, then the calculated vector similarity is subjected to statistical average to obtain a second similarity S2, and then the method can be used according to the formula: s=b×s1×1-S2, and a picture score of the picture a is calculated.

In another optional implementation manner of this embodiment, the comparison feature vectors of the target ordered pictures are respectively input into a pre-trained picture scoring model to obtain a picture score of each target ordered picture; the picture scoring model is obtained by training standard quality feature vectors of pictures in a standard quality picture set and standard quality feature vectors of pictures in a quality picture set.

In this embodiment, by constructing a picture scoring model and learning each standard high-quality feature vector and each standard low-quality feature vector by using model parameters in the picture scoring model, the picture scoring model can automatically learn attribute weights of each picture attribute without user-defined settings.

S130, sorting the target sorting pictures according to the picture scores of the target sorting pictures.

In this embodiment, since the picture score of one picture reflects the quality degree of the picture, the higher the picture score, the better the picture. Therefore, after obtaining the picture scores with the respective target ranking pictures, the respective target ranking pictures can be ranked in order of the picture scores from high to low.

Correspondingly, when each target ordered picture is provided for the client for display, each picture can be displayed in sequence according to the ordering result, so that the effect that the high-quality picture is preferentially displayed is achieved.

According to the technical scheme, the influence of human factors in the picture sorting process can be reduced to the greatest extent by calculating the comparison feature vector of at least one picture attribute corresponding to each target sorting picture, and matching degree between the comparison feature vector and the standard feature vector of each picture in the standard quality picture set and/or the standard feature vector of each picture in the standard quality picture set, so that the accuracy of sorting the high-quality pictures is improved, the hit rate of the first-screen picture on the user requirement is improved, and the click rate of the pictures is further improved.

Second embodiment

Fig. 2 is a flowchart of a picture sorting method according to a second embodiment of the present application, in this embodiment, a training picture scoring model is used to calculate a picture score of each target sorted picture, and a specific process of training to obtain the picture scoring model and a mode of obtaining a standard quality picture set are further defined, as shown in fig. 2, and the method may include:

S210, acquiring an original picture set as a target picture set.

In this embodiment, a specific training process of the picture scoring model is implemented through S210-S270. The key of the whole training process is to obtain a standard quality picture set which accurately represents the quality picture characteristics.

Before the standard high-quality picture set is acquired, a large number of original pictures can be acquired from a picture search engine to form an original picture set, and the standard high-quality picture set is determined in the original picture set.

S220, generating a plurality of positive and negative sample pairs according to the target picture set, and sending the positive and negative sample pairs to a plurality of target labeling platforms, wherein the target labeling platforms are used for labeling alternative high-quality pictures according to the positive and negative sample pairs.

In this embodiment, the pictures included in the target picture set may be pre-ordered according to a preset high-quality picture ordering standard, for example, according to one or more of a correlation degree, an aesthetic degree and a definition degree of the pictures, and a plurality of positive and negative sample pairs are generated according to a pre-ordering result, where a positive sample is a picture estimated to be a standard high-quality picture, and a negative sample is a picture estimated to be a standard low-quality picture.

The method for generating the plurality of positive and negative sample pairs according to the pre-sequencing result may be that one picture at a middle position in the pre-sequencing result is obtained, a positive sample set and a negative sample set are generated according to the picture at the middle position, and one picture in the positive sample set and one picture in the negative sample set are sequentially obtained according to the picture sequencing result in the positive sample set and the negative sample set to form the positive and negative sample pairs.

In one specific example: 1000 pictures are obtained after pre-ordering, then {1, 500} pictures can be formed into a positive sample set, and {501,1000} pictures can be formed into a negative sample set. And then the 1 st picture and the 501 st picture can be taken to form a positive and negative sample pair, the 2 nd picture and the 502 th picture can be taken to form a positive and negative sample pair, and the like, and 500 positive and negative sample pairs are formed in total.

In this embodiment, after obtaining multiple positive and negative sample pairs, the multiple positive and negative samples may be sent to multiple target labeling platforms, where the target labeling platform selects, in each received positive and negative sample pair, a picture with better quality as an alternative high-quality picture, and feeds back the labeled alternative high-quality picture accordingly. The target labeling platform is used for labeling alternative high-quality pictures according to the positive and negative sample pairs.

The target labeling platform can be a manual labeling platform or a third party labeling platform and the like, and is used for labeling a picture processing platform with better quality in a positive and negative sample pair after the positive and negative sample pair is received.

Specifically, each target labeling platform can label all or part of a plurality of positive and negative sample pairs, so as to ensure certain labeling redundancy, only a set number (for example, 3 or 5 or the like) of target labeling platforms can be ensured for each positive and negative sample pair, and the target labeling platforms can feed back labeled alternative high-quality pictures.

S230, determining a second picture set as a new target picture set from the candidate high-quality pictures fed back by each target labeling platform according to a preset cross judgment mechanism.

In this embodiment, in order to ensure availability and accuracy of the labeling result of the target labeling platform, a cross judging mechanism is adopted, and a second picture set is determined from the candidate high-quality pictures fed back by each target labeling platform to serve as a new target picture set.

The so-called cross judging mechanism specifically refers to that a first number of target labeling platforms feed back labeled alternative high-quality pictures to a certain positive and negative sample pair X, and if target labeling platforms (for example, 0.8, 0.9 or 0.95) with set proportions label the picture Y in the positive and negative sample pair X as the alternative high-quality picture, the picture Y is added into a second picture set to serve as a new target picture set.

S240, judging whether the end marking condition is met: if yes, executing S250; otherwise, execution returns to S220.

The ending labeling condition may be that the current iteration round has exceeded a set number of times, for example, 3 times or 5 times, or that the number of pictures included in the current target picture set is less than or equal to a set number threshold (for example, 1000 or 2000), which is not limited in this embodiment.

In this embodiment, when the ending labeling condition is not satisfied, the new target image set may be pre-ordered again according to the image quality, and according to the pre-ordering result, a new positive and negative sample pair is obtained again, and the new positive and negative sample pair is sent again to multiple target labeling platforms for labeling of alternative high-quality images, so as to screen out a more accurate standard high-quality image set.

S250, generating a standard high-quality picture set according to each picture in the current target picture set.

In this embodiment, the current target picture set may be directly used as a standard high-quality picture set, or the pictures included in the current target picture set may be further screened, and after the pictures with high-quality risks are filtered, the standard high-quality picture set is generated.

Optionally, generating the standard high-quality picture set according to each picture in the current target picture set may specifically include:

and predicting and scoring each picture in the current target picture set, and generating the standard high-quality picture set after filtering out pictures of which the scoring result does not belong to high-quality pictures in the current target picture set.

Specifically, each picture in the current target picture set can be predicted and scored through a basic scoring model, some experience parameters can be used as model parameters by the model, and the scoring result of each picture can be obtained roughly under a coarse precision. If it is determined that a picture in the target picture set does not belong to a good-quality picture according to the scoring result of the picture, for example, the score is lower than 50 points, the picture is indicated to have a good-quality risk (i.e., a certain probability does not belong to a good-quality picture), and then the picture can be filtered from the current target picture set, so that the good quality of the picture in the target picture set is ensured to the greatest extent.

Optionally, generating the standard high-quality picture set according to each picture in the current target picture set may further specifically include:

Specifically, at least one picture attribute of each picture in the current target picture set can be obtained respectively, if the picture attribute of a certain picture is far different from the standard value range of the picture attribute, the picture can also be described as having high-quality risk, and then the picture needs to be filtered out from the current target picture set, so that the high-quality of the picture in the target picture set is ensured to the greatest extent.

For example, the definition of a certain picture in the current target picture set is 0.3, and the standard value range of the preset definition is [0.6,1], and since the difference between the definition and the lowest tolerance of the preset definition is greater than the preset tolerance threshold (for example, 0.1 or 0.2, etc.), the picture can be directly deleted from the current target picture set.

S260, constructing a standard inferior picture set according to the standard value range of the at least one picture attribute.

In this embodiment, it is considered that when one or more picture attributes of a picture are far different from the standard value range of the corresponding picture attribute, the picture is generally a standard-quality picture, so that a relatively simple manner may be used to construct a standard-quality picture set.

For example, if the standard value range of the preset correlation is [0.8,1], a picture with the correlation less than or equal to 0.4 can be obtained and added into the standard inferior picture set; or if the preset standard value range of the aesthetic degree is [0.6,1] and the standard value range of the brightness is [0.5,0.8], obtaining the picture with the aesthetic degree raining equal to 0.2 and the brightness less than or equal to 0.1 and adding the picture into the standard inferior picture set.

It should be noted that, in general, when performing the picture scoring, the probability that the picture to be scored is a poor picture is far greater than that of a good picture, so in order to ensure the scoring accuracy of the picture scoring model, the number of pictures in the standard poor picture set as a negative sample is far greater than that of the standard good picture set as a positive sample.

Specifically, the ratio of the number of pictures in the standard quality picture set to the number of pictures in the standard quality picture set (for example, 1:3 or 1:4) may be predefined, so that the number of standard quality pictures included in the standard quality picture set may be determined according to the number of standard quality pictures included in the standard quality picture set, and then the number of standard quality pictures may be configured to be added to the standard quality picture set.

S270, training a preset machine learning model according to standard feature vectors corresponding to each picture in the standard high-quality picture set and the standard low-quality picture set, and obtaining the picture scoring model.

Specifically, the machine learning model may include: a gradient lifting decision tree model, a random forest classifier, a least squares regression tree model, or the like, which is not limited in this embodiment.

The inventor finds that when the image scoring model is obtained by training the gradient lifting decision tree model through multiple experiments, the scoring effect is best, and alternatively, the image scoring model can be obtained by training the gradient lifting decision tree model.

S280, calculating at least one picture attribute corresponding to each target ordered picture, and generating comparison feature vectors corresponding to each target ordered picture according to the picture attribute.

S290, respectively inputting the comparison feature vectors of the target ordered pictures into a pre-trained picture scoring model to obtain picture scores of the target ordered pictures.

The picture scoring model is obtained by training standard feature vectors of all pictures in a standard high-quality picture set and a standard low-quality picture set, namely: the picture scoring model is obtained by training standard quality feature vectors of pictures in a standard quality picture set and standard quality feature vectors of pictures in a quality picture set.

And S2100, sorting the target sorting pictures according to the picture scores of the target sorting pictures.

According to the technical scheme, the standard high-quality picture set with high quality and the poor-quality picture set are firstly obtained, and the picture scoring model is obtained based on the standard feature vector training of each picture in the standard high-quality picture set and the poor-quality picture set, so that the quality of training samples is ensured to the greatest extent, the scoring accuracy of the picture scoring model can be improved, and the sorting accuracy of pictures is finally improved.

On the basis of the above embodiments, before generating a plurality of positive and negative sample pairs according to the target picture set and sending the positive and negative sample pairs to a plurality of target labeling platforms, the method may further include:

Constructing a plurality of standard positive and negative sample pairs according to the plurality of standard high-quality pictures, and sending the plurality of standard positive and negative sample pairs to a plurality of alternative labeling platforms;

acquiring alternative high-quality pictures fed back by the plurality of alternative labeling platforms aiming at the standard positive and negative samples;

And determining the target annotation platform in the plurality of alternative annotation platforms according to the inclusion condition of each alternative high-quality picture on the dark pile data.

In this embodiment, the inventors further consider that: and if the judging standard of the high-quality picture of the target labeling platform is not in accordance with the preset judging standard of the high-quality picture, the operation of further screening the qualification of the target labeling platform is provided.

The method has the advantages that the method can determine the target marking platform in the plurality of alternative marking platforms by adding the hidden pile data into the plurality of standard high-quality pictures generated in advance, and the high-quality picture marking standard of the target marking platform is consistent with the marking standard required by an actual system so as to further ensure the quality of the positive sample used for training the picture marking model.

That is, first, a plurality of standard quality pictures are generated according to a predetermined quality picture criterion, which is a criterion for verifying quality pictures of the respective alternative labeling platforms. After generating a plurality of standard high-quality pictures, adding dark pile data into the standard high-quality pictures, constructing a plurality of standard positive and negative sample pairs, sending the standard positive and negative sample pairs to a plurality of alternative marking platforms, and marking the alternative high-quality pictures by the alternative marking platforms. The purpose of adding the hidden pile data is to identify whether the selected marked alternative high-quality picture of an alternative marking platform is consistent with the standard high-quality picture. And then after obtaining the candidate high-quality pictures fed back by the candidate labeling platforms aiming at the standard positive and negative samples, determining the target labeling platform in the candidate labeling platforms according to the inclusion condition of the hidden pile data in each candidate high-quality picture.

For example, in the process that a certain alternative labeling platform feeds back alternative high-quality pictures included in the positive and negative sample pairs 100 times, only 10 hits of the selected alternative high-quality pictures are added into standard high-quality pictures of the dark pile data, the hit rate of the alternative labeling platform to the standard high-quality pictures is lower than a preset hit rate threshold (for example, 60%), the alternative labeling platform is not selected as a target labeling platform in the labeling process of the subsequent positive and negative sample pairs, the alternative labeling platform is guided to continuously learn the preset high-quality picture judgment standard until the hit rate to the standard high-quality pictures is higher than the preset hit rate threshold, and the alternative labeling platform is selected as the target labeling platform.

On the basis of the above embodiments, after generating the standard quality picture set according to each picture in the current target picture set, the method may further include:

And constructing a plurality of picture feature vectors of the new standard quality pictures according to the standard value range of the at least one picture attribute and the numerical distribution of the picture attribute of each picture in the standard quality picture set.

Generally, if the number of the quality samples is small, or the numerical distribution of a certain picture attribute in the quality samples is uneven (for example, only 50 positive samples with picture definition being greater than 80% and 5000 positive samples with picture definition being between 50% and 80%), the scoring effect of the picture scoring model obtained by final training is weakened, so that a plurality of quality samples meeting the condition of uniform data distribution can be constructed according to the value range of the quality samples under each picture attribute, and specifically, standard quality feature vectors of standard quality pictures can be directly constructed without directly constructing standard quality pictures.

Correspondingly, constructing a standard inferior picture set according to the standard value range of the at least one picture attribute, including:

The advantages of this arrangement are that: according to the standard value range of the at least one picture attribute and the numerical distribution of the picture attribute of each picture in the standard high-quality picture set, a plurality of picture feature vectors of new standard high-quality pictures are constructed, so that the positive samples used for training the picture scoring model can be ensured to the greatest extent to ensure certain scoring accuracy under different values of the picture attribute, and the universality and the scoring accuracy of the picture scoring model are improved.

Third embodiment

Fig. 3 is a schematic structural diagram of a picture sorting apparatus 30 according to a third embodiment of the present application, which can execute a picture sorting method according to any of the embodiments of the present application, and has functional modules and beneficial effects corresponding to the execution method. As shown in fig. 3, the apparatus may include:

the comparison feature vector generation module 310 is configured to calculate at least one picture attribute corresponding to each target ordered picture, and generate a comparison feature vector corresponding to each target ordered picture according to the picture attribute;

The picture score calculating module 320 is configured to calculate a picture score of each target ordered picture according to a matching degree between each comparison feature vector and a standard high-quality picture set and/or a standard feature vector of each picture in a standard low-quality picture set;

the picture sorting module 330 is configured to sort each target sorted picture according to the picture score of each target sorted picture.

According to the technical scheme, the influence of human factors in the picture sorting process can be reduced to the greatest extent by calculating the comparison feature vector of at least one picture attribute corresponding to each target sorting picture, and the standard quality feature vector of each picture in the standard quality picture set and/or the matching degree between the comparison feature vector and the standard quality feature vector of each picture in the standard quality picture set, so that the influence of human factors in the picture sorting process can be reduced to the greatest extent, the accuracy of sorting of the quality pictures is improved, the hit rate of the first screen picture on the user requirement is improved, and the click rate of the pictures is further improved.

On the basis of the above embodiments, the picture score calculating module may include:

The model input unit is used for respectively inputting the comparison feature vectors of the target ordered pictures into a pre-trained picture scoring model to obtain picture scores of the target ordered pictures;

On the basis of the above embodiments, the method may further include:

The standard high-quality picture set acquisition unit is used for acquiring an original picture set before calculating at least one picture attribute corresponding to each target ordering picture respectively, and acquiring a standard high-quality picture set according to picture marking results of a plurality of target marking platforms on the original picture set;

The standard inferior picture set construction unit is used for constructing a standard inferior picture set according to the standard value range of the at least one picture attribute;

The picture scoring model generation unit is used for training a preset machine learning model according to standard feature vectors corresponding to each picture in the standard high-quality picture set and the standard low-quality picture set respectively to obtain the picture scoring model.

On the basis of the above embodiments, the standard quality picture set acquisition unit may include:

A target picture set acquisition subunit, configured to acquire the original picture set as a target picture set;

The positive and negative sample pair sending subunit is used for generating a plurality of positive and negative sample pairs according to the target picture set and sending the positive and negative sample pairs to a plurality of target labeling platforms, wherein the target labeling platforms are used for labeling alternative high-quality pictures according to the positive and negative sample pairs;

The second picture set determining subunit is used for determining a second picture set from the candidate high-quality pictures fed back by each target labeling platform as a new target picture set according to a preset cross judging mechanism;

The repeated execution subunit is used for triggering the repeated execution of the positive and negative samples on the sending subunit until the end marking condition is met;

and the standard high-quality picture set generation subunit is used for generating a standard high-quality picture set according to each picture in the current target picture set.

On the basis of the above embodiments, the method may further include a target labeling platform determining subunit, configured to:

generating a plurality of positive and negative sample pairs according to the target picture set, and before the positive and negative sample pairs are sent to a plurality of target labeling platforms, generating a plurality of standard high-quality pictures in advance, and adding dark pile data into each standard high-quality picture;

On the basis of the above embodiments, the standard quality picture set generating subunit may specifically be used for:

On the basis of the above embodiments, the method may further include a new image feature vector generation subunit, configured to:

after a standard high-quality picture set is generated according to each picture in a current target picture set, constructing a plurality of picture feature vectors of new standard high-quality pictures according to a standard value range of at least one picture attribute and numerical distribution of picture attributes of each picture in the standard high-quality picture set;

The standard inferior picture set construction unit is specifically used for:

On the basis of the above embodiments, the machine learning model may include: gradient lifting decision tree model.

On the basis of the above embodiments, the picture attribute may include at least one of: correlation, aesthetics, sharpness, brightness, whether face is included, whether watermark is included, and whether it is a sensitive picture.

The image sorting device 30 provided by the embodiment of the application can execute the image sorting method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method. Technical details not described in detail in this embodiment may be referred to the picture ordering method provided in any embodiment of the present application.

Fourth embodiment

According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

As shown in fig. 4, a block diagram of an electronic device according to a picture ordering method according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the applications described and/or claimed herein.

As shown in fig. 4, the electronic device includes: one or more processors 401, memory 402, and interfaces for connecting the components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 401 is illustrated in fig. 4.

Memory 402 is a non-transitory computer readable storage medium provided by the present application. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the picture ordering method provided by the present application. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to execute the picture ordering method provided by the present application.

The memory 402 is used as a non-transitory computer readable storage medium for storing non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules (e.g., the alignment feature vector generation module 310, the picture score calculation module 320, and the picture ordering module 330 shown in fig. 3) corresponding to the picture ordering method in the embodiment of the application. The processor 401 executes various functional applications of the server and data processing, i.e. implements the picture ordering method in the above-described method embodiments, by running non-transitory software programs, instructions and modules stored in the memory 402.

Memory 402 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created according to the use of the electronic device implementing the picture ordering method, and the like. In addition, memory 402 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 402 may optionally include memory remotely located with respect to processor 401, which may be connected via a network to an electronic device implementing the picture ordering method. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The electronic device for implementing the picture ordering method may further include: an input device 403 and an output device 404. The processor 401, memory 402, input device 403, and output device 404 may be connected by a bus or otherwise, for example in fig. 4.

Input device 403 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device of XXX, such as a touch screen, keypad, mouse, trackpad, touchpad, pointer stick, one or more mouse buttons, trackball, joystick, and like input devices. The output device 404 may include a display apparatus, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASIC (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed embodiments are achieved, and are not limited herein.

The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.

Claims

1. A picture ordering method, comprising:

Calculating the picture score of each target ordered picture according to the matching degree between each comparison feature vector and the standard feature vector of each picture in the standard quality picture set and the standard quality picture set; the standard high-quality picture set is obtained by marking a plurality of positive and negative sample pairs by a plurality of target marking platforms;

sorting the target sorting pictures according to the picture scores of the target sorting pictures;

The method for acquiring the target labeling platform comprises the following steps:

determining the target annotation platform in the plurality of alternative annotation platforms according to the inclusion condition of each alternative high-quality picture on the hidden pile data;

The purpose of adding the hidden pile data is to identify whether the selected marked alternative high-quality picture of an alternative marking platform is consistent with the standard high-quality picture.

2. The method of claim 1, wherein calculating the picture score for each target ranked picture based on the degree of matching between each aligned feature vector and the standard feature vector for each picture in the standard premium picture set and the standard premium picture set, comprises:

3. The method of claim 2, further comprising, prior to calculating the at least one picture attribute corresponding to each target ordered picture, respectively:

4. The method of claim 3, wherein obtaining an original picture set and obtaining a standard quality picture set according to picture labeling results of a plurality of target labeling platforms on the original picture set comprises:

Acquiring the original picture set as a target picture set;

Generating a plurality of positive and negative sample pairs according to the target picture set, and sending the positive and negative sample pairs to a plurality of target labeling platforms, wherein the target labeling platforms are used for labeling alternative high-quality pictures according to the positive and negative sample pairs;

Returning to execute the operation of generating a plurality of positive and negative sample pairs according to the target picture set and sending the positive and negative sample pairs to a plurality of target labeling platforms until the ending labeling condition is met;

5. The method of claim 4, wherein generating a standard quality picture set from each picture in the current target picture set comprises:

6. The method of claim 4, further comprising, after generating the standard quality picture set from each picture in the current target picture set:

7. The method of claim 3, wherein the machine learning model comprises: gradient lifting decision tree model.

8. The method of claim 1, wherein the picture attributes comprise at least one of: correlation, aesthetics, sharpness, brightness, whether face is included, whether watermark is included, and whether it is a sensitive picture.

9. A picture ordering apparatus, comprising:

the comparison feature vector generation module is used for calculating at least one picture attribute corresponding to each target ordering picture respectively and generating comparison feature vectors corresponding to each target ordering picture respectively according to the picture attribute; the standard high-quality picture set is obtained by marking a plurality of positive and negative sample pairs by a plurality of target marking platforms;

the picture score calculation module is used for calculating the picture score of each target ordered picture according to the matching degree between each comparison feature vector and the standard feature vector of each picture in the standard quality picture set;

the picture sorting module is used for sorting the target sorting pictures according to the picture scores of the target sorting pictures;

The target labeling platform comprises a target labeling platform determining subunit, a target labeling platform determining subunit and a target labeling platform determining subunit, wherein the target labeling platform determining subunit is used for:

10. The apparatus of claim 9, wherein the picture score calculation module comprises:

11. The apparatus as recited in claim 10, further comprising:

12. The apparatus of claim 11, wherein the standard premium picture set acquisition unit comprises:

13. The apparatus according to claim 12, wherein the standard quality picture set generation subunit is specifically configured to:

14. The apparatus of claim 12, further comprising a new picture feature vector generation subunit operable to:

The standard inferior picture set construction unit is specifically used for:

15. The apparatus of claim 11, wherein the machine learning model comprises: gradient lifting decision tree model.

16. The apparatus of claim 9, wherein the picture attribute comprises at least one of: correlation, aesthetics, sharpness, brightness, whether face is included, whether watermark is included, and whether it is a sensitive picture.

17. An electronic device, comprising:

at least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

18. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-8.