CN111104536A - Picture searching method, device, terminal and storage medium - Google Patents
Picture searching method, device, terminal and storage medium Download PDFInfo
- Publication number
- CN111104536A CN111104536A CN201911349275.5A CN201911349275A CN111104536A CN 111104536 A CN111104536 A CN 111104536A CN 201911349275 A CN201911349275 A CN 201911349275A CN 111104536 A CN111104536 A CN 111104536A
- Authority
- CN
- China
- Prior art keywords
- keyword
- search
- list
- picture
- keyword list
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/55—Clustering; Classification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application discloses a picture searching method, a picture searching device, a terminal and a storage medium, and belongs to the field of computers. The method comprises the following steps: when an image searching instruction is received, obtaining a searching keyword; determining at least one associated keyword corresponding to the search keyword from a first keyword list, wherein the first keyword contained in the first keyword list corresponds to a classification label of the image classification model; and determining a target picture in the album according to the search keyword and the associated keyword. By adopting the method provided by the embodiment of the application, when the picture searching instruction is received, at least one associated keyword which has a synonymous relationship or a parent-child relationship with the searching keyword is obtained from the first keyword list, and then the picture of the searching keyword and the classified label corresponding to the associated keyword is obtained, so that the searching deviation caused by the inconsistency between the searching keyword input by the user and the picture label is avoided, and the accuracy of the picture searching result is improved.
Description
Technical Field
The embodiment of the application relates to the field of computers, in particular to a picture searching method, a picture searching device, a picture searching terminal and a storage medium.
Background
With the continuous updating and development of the terminal album function, a user can store pictures more conveniently, the demand for the terminal album is gradually increased, the difficulty of searching the pictures by the user is increased when the quantity of the stored pictures of the terminal album is large, and the pictures can be quickly searched by the user due to the picture searching function of the terminal album.
In the related technology, after a terminal acquires a picture, the object type in the picture is deduced by using a neural network model, the picture is matched with a corresponding label according to the deduction result, and when the user needs to search the picture, the user inputs a keyword corresponding to the label to the terminal, so that the corresponding picture can be acquired.
However, with the method of searching for a picture using a single tag in the related art, a user needs to input the same content as the preset tag of the album to obtain the corresponding picture, but the user usually cannot predict the keyword corresponding to the album tag, and the types of the tags that can be matched by the neural network model are few, and the keyword input by the user has diversity, which easily causes that the matched picture is inaccurate or the picture corresponding to the input keyword cannot be obtained.
Disclosure of Invention
The embodiment of the application provides a picture searching method, a picture searching device, a terminal and a storage medium. The technical scheme is as follows:
in one aspect, an embodiment of the present application provides an image search method, where the method includes:
when an image searching instruction is received, obtaining a searching keyword;
determining at least one associated keyword corresponding to the search keyword from a first keyword list, wherein the first keyword contained in the first keyword list corresponds to a classification tag of an image classification model, the image classification model is used for setting the classification tag for an image in an album, and the association relationship between the associated keyword and the search keyword comprises a synonymy relationship or a parent-child relationship;
and determining a target picture in the album according to the search keyword and the associated keyword, wherein the classification label corresponding to the target picture is matched with the search keyword or the associated keyword.
In another aspect, an embodiment of the present application provides an image search apparatus, where the apparatus includes:
the first acquisition module is used for acquiring search keywords when receiving an image search instruction;
the first determining module is used for determining at least one associated keyword corresponding to the search keyword from a first keyword list, wherein the first keyword contained in the first keyword list corresponds to a classification tag of an image classification model, the image classification model is used for setting the classification tag for an image in an album, and the association relationship between the associated keyword and the search keyword comprises a synonymy relationship or a parent-child relationship;
and the second determining module is used for determining a target picture in the photo album according to the search keyword and the associated keyword, wherein the classification label corresponding to the target picture is matched with the search keyword or the associated keyword.
In another aspect, an embodiment of the present application provides a terminal, where the terminal includes a processor and a memory; the memory stores at least one instruction for execution by the processor to implement the picture search method of the above aspect.
In another aspect, an embodiment of the present application provides a computer-readable storage medium, where the storage medium stores at least one instruction, and the at least one instruction is used for being executed by a processor to implement the picture search method according to the above aspect.
The technical scheme provided by the embodiment of the application has the beneficial effects that at least:
in the embodiment of the application, the photo album picture is identified through the picture classification model, the classification tags obtained by the identification result are stored in association with the picture, when a picture search instruction is received, at least one associated keyword which has a synonymous relationship or a parent-child relationship with the search keyword is obtained from the first keyword list, and then the picture of the search keyword and the classification tag corresponding to the associated keyword is obtained.
Drawings
FIG. 1 is a schematic diagram of an implementation environment shown in accordance with an example embodiment;
FIG. 2 is a flow diagram illustrating a picture search method according to an example embodiment;
FIG. 3 is a diagram illustrating a terminal picture search interface in accordance with an illustrative embodiment;
FIG. 4 is a flowchart illustrating a picture search method according to another exemplary embodiment;
FIG. 5 is an illustration of a first keyword list shown in accordance with an exemplary embodiment;
FIG. 6 is a diagram illustrating a first keyword list, according to another exemplary embodiment;
FIG. 7 is an illustration of a first keyword list and a second keyword list shown in accordance with an exemplary embodiment;
FIG. 8 is a flowchart illustrating a picture search method according to another exemplary embodiment;
FIG. 9 is a flowchart illustrating a picture search method according to another exemplary embodiment;
fig. 10 is a block diagram illustrating a configuration of a picture search apparatus according to an exemplary embodiment;
fig. 11 is a block diagram illustrating a structure of a terminal according to an exemplary embodiment.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Reference herein to "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
In the related technology, when a terminal acquires a picture to be stored, an object in the picture is identified through a picture classification model, a corresponding classification label is matched with the picture, when a user wants to search one or a class of pictures, a keyword representing the class of pictures is input in an album interface, the terminal searches for the pictures with the classification labels consistent with the keyword, and a query result is returned. However, the user needs to obtain the desired picture only when the input keyword is completely consistent with the content of the classification label matched with the picture by the classification recognition model, but the user usually cannot predict the classification label matched with the picture, and the input keyword has diversity when the user searches the picture, for example, the user may input "animal" to search when the user wants to search the picture played in a zoo, but the classification label matched with the related picture in the actual album is a more specific subclass such as "cat" or "tiger", and the query requirement of the user cannot be met.
In order to solve the above technical problem, an embodiment of the present application provides an image searching method. Referring to fig. 1, a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application is shown. The implementation environment includes a terminal 101 and a server 102.
The terminal 101 is an electronic device installed with an album application, which may be a smart phone, a tablet computer, an electronic book reader, a personal portable computer, or the like. In this embodiment, the album application program has a function of classifying pictures, and optionally, a picture classification model is set in the album application program, and the album application program can determine the type of each picture through the picture classification model and set a corresponding classification tag for each picture.
In a possible implementation manner, the image searching method provided by the embodiment of the application may be implemented as an album application or a part of the album application. When a user wants to search for pictures, the album application program can be manually opened, and search keywords are input; the photo album application program determines at least one associated keyword (matched with the classification label of the picture classification model) having an associated relation with the search keyword according to the search keyword, so that a matched picture is searched and displayed according to the search keyword and the associated keyword.
The terminal 101 and the server 102 are connected by a wired or wireless network.
The server 102 is a server, a server cluster formed by a plurality of servers, or a cloud server.
In one possible embodiment, as shown in fig. 1, the server 102 is a background server for the registered applications in the terminal 101. Optionally, after the album application in the terminal 102 obtains the search keyword, the search keyword is sent to the server 102, the server 102 determines an associated keyword corresponding to the search keyword, and feeds the associated keyword back to the terminal 102, so that the terminal 102 performs image search in the album according to the received associated keyword and the search keyword (that is, the image search function is completed by the interaction between the terminal and the server).
For convenience of description, the following embodiments are described as examples in which the terminal 101 performs a picture search method.
Referring to fig. 2, a flowchart of a picture searching method according to an embodiment of the present application is shown. In this embodiment, an example in which an image search method is used for a terminal is described, where the method includes:
In a possible implementation mode, a terminal album interface comprises a search function, and when a user wants to search for pictures, the user inputs search keywords into the terminal and sends a picture search instruction. The search keywords are used for indicating objects contained in the pictures to be searched, and the terminal matches the corresponding pictures based on the search keywords.
Optionally, the user inputs a search keyword in a search bar of the album interface by characters, triggers a search control, and sends a picture search instruction to the terminal.
Optionally, the user inputs a search keyword and a picture search instruction to the terminal through voice, and the terminal can automatically open an album or an application program according to the voice instruction to search for a corresponding picture.
Illustratively, when a user wants to search for a picture about a cat stored in the terminal, the album function is started, a search keyword "cat" is input, a search control is triggered, and the terminal receives a picture search instruction and acquires the search keyword "cat".
In one possible embodiment, the picture classification model is used to set classification tags for pictures in the album. The terminal identifies the picture to be stored by adopting the picture classification model to obtain an object in the picture to be stored, and attaches a classification label corresponding to the object to the picture to be stored for storage. For example, the mobile terminal may adopt a lightweight convolutional neural network, for example, a mobile terminal neural network (MobileNet) as a picture classification model, and a terminal with a strong capability of calculating and processing data may adopt a more accurate deep convolutional neural network model, such as other classification models like AlexNet, which is not limited in this embodiment.
The recognition result of the image classification model can only be a single label matched with the image, and the user cannot know the content of the classification label matched with each image at the terminal, while the search keywords used by the user in searching the image have diversity, the classification label usually matched with the image classification model belongs to a fine category, such as cat, dog, apple and the like, and when the user wants to obtain a plurality of images of a certain category, or the specific content in the image cannot be determined due to memory deviation, the search keywords such as animal, fruit and the like may be input and cannot be matched with the target image.
In a possible implementation manner, the terminal database stores a first keyword list, and first keywords in the first keyword list are matched with the classification tags of the picture recognition model, that is, all the classification tags have one corresponding first keyword in the first keyword list, and the first keyword list does not have first keywords other than the first keyword corresponding to the classification tag. After the terminal acquires the search keyword, associated keywords of the search keyword are inquired from the first keyword list, and the associated relationship between the associated keywords and the search keyword comprises a synonymy relationship or a parent-child relationship.
Illustratively, the terminal obtains a search keyword "fruit" and obtains an associated keyword from the first keyword list to obtain "fruit" and subclass keywords "pineapple" and "banana".
In a possible implementation manner, the terminal determines the pictures with the classification labels consistent with the search keywords and the associated keywords according to the search keywords and the associated keywords, and returns the search results.
Illustratively, as shown in fig. 3, a user inputs a search keyword "fruit" in an album search bar, a terminal queries "fruit" and associated keywords thereof from a first keyword list to obtain four associated keywords (subclass keywords) of "apple", "banana", "pineapple" and "lemon", searches pictures with classification labels of "fruit", "apple", "banana", "pineapple" and "lemon" in an album picture, and finally obtains three pictures with classification labels of "apple", "banana" and "pineapple", where the classification labels of the three pictures are different from the search keyword.
Illustratively, a user inputs a search keyword 'pineapple' in an album search bar, a terminal queries the 'pineapple' and associated keywords thereof from a first keyword list to obtain an associated keyword (synonymous keyword) of 'pineapple', and searches pictures with classification labels of 'pineapple' and 'pineapple' in an album picture to finally obtain a picture with a classification label of 'pineapple'.
In summary, in the embodiment of the application, the album picture is identified through the picture classification model, the classification tags obtained from the identification result are stored in association with the picture, when a picture search instruction is received, at least one associated keyword having a synonymous relationship or a parent-child relationship with the search keyword is obtained from the first keyword list, and then the picture of the search keyword and the classification tag corresponding to the associated keyword is obtained.
After the image classification model identifies the image to be stored, the matched classification label is stored in association with the image, and one image is only matched with one classification label, so that the terminal needs to search the image according to the search keyword which is completely consistent with the classification label. However, the search keyword input by the user may not belong to the first keyword list, but a word having a similar meaning to the first keyword, and the query of the associated keyword based on the first keyword list cannot be performed. In a possible implementation manner, a first keyword list and a second keyword list are set in the terminal database in advance, a second keyword in the second keyword list is a synonym of the first keyword, and the second keyword list includes a synonymy relationship between the second keyword and the first keyword. When the terminal acquires the search keyword, the associated keyword of the search keyword is inquired through the first keyword list and the second keyword list, then the picture with the classification label consistent with the search keyword and the associated keyword is searched from the photo album, at least one target picture is obtained, and the possible result is acquired through the method, so that the search requirement of the user can be met.
Referring to fig. 4, a flowchart of a picture searching method according to another embodiment of the present application is shown. In this embodiment, an example in which an image search method is used for a terminal is described, where the method includes:
The implementation of step 401 may refer to step 201, and this embodiment is not described herein again.
In a possible implementation manner, the terminal database is provided with a first keyword list, wherein the stored first keywords correspond to the classification tags of the picture classification model, the first keyword list includes first keyword identifiers, and corresponding relations between the first keywords and parent keyword identifiers, the first keyword identifiers are identifiers corresponding to the first keywords, and the parent keyword identifiers are identifiers corresponding to the parent keywords. After the terminal obtains the search keyword, the terminal firstly searches the search keyword in the first keyword list.
Illustratively, as shown in fig. 5, the terminal obtains that the search keyword is "fruit", finds "fruit" in the first keyword list, and obtains its first keyword identifier "2".
Optionally, if a search keyword is searched in the first keyword list, step 403 is executed; if the search keyword is not found in the first keyword list, steps 404 to 406 are performed.
In step 403, if the search keyword is found in the first keyword list, determining a sub-category keyword corresponding to the search keyword in the first keyword list as an associated keyword.
The classification label of the image identification model is usually a specific refined classification label, and the search keyword input by the user corresponds to the classification label or a parent label thereof, so that after the terminal acquires the search keyword, the terminal can acquire the image which the user may want to query only by determining the subclass keyword of the search keyword as the associated keyword in the first keyword list, and does not need to search the parent keyword.
In one possible embodiment, a first keyword corresponds to only one parent keyword, or there is no parent keyword, and if there is a parent keyword, the parent keyword is the first keyword in the first keyword list with direct parent-child relationship, but not the first keyword with indirect parent-child relationship, for example, "animal" and "creature" are both parent keywords of "cat", and "cat" in the first keyword list is only associated with "animal".
Illustratively, as shown in fig. 5, a first column of the first keyword list is a first keyword identifier, a second column is the first keyword, the first keyword is a classification tag of the image recognition model, and a third column is a parent keyword identifier, where the parent keyword also belongs to the first keyword list. For example, three first keywords "pineapple", "fruit" and "food" are stored in the first keyword list, the first keyword identifiers are "1", "2" and "3", respectively, the parent keyword of the pineapple "is" fruit ", the corresponding parent keyword identifier is" 2 ", the parent keyword of the fruit" is "food", the corresponding parent keyword identifier is "3", and the parent keyword of the food "does not exist in the first keyword list. If the search keyword is "food", the "fruit" and "pineapple" are determined as the associated keywords.
In one possible embodiment, step 403 includes the following steps one to three:
firstly, determining a level 1 candidate keyword which takes a search keyword as a parent keyword from a first keyword list.
In one possible implementation mode, each first keyword in the first keyword list corresponds to at most one parent keyword, and the terminal firstly determines the 1 st-level candidate keyword of the search keyword, namely the first keyword with the identifier of the parent keyword being consistent with the identifier of the first keyword of the search keyword.
Illustratively, as shown in fig. 6, if the search keyword is "vehicle", the terminal obtains a first keyword identifier "13" of "vehicle" from the first keyword list, queries "13" in the column where the parent keyword identifier is located, and obtains a corresponding first keyword and a corresponding first keyword identifier, that is, "bicycle" and "5"; if the search keyword is "food", the terminal acquires a first keyword identifier "6" of "food" from the first keyword list, and acquires a first keyword and a first keyword identifier corresponding thereto, i.e., "fruit" and "3", "dessert" and "4".
And secondly, for the nth-level candidate keyword, determining an nth + 1-level candidate keyword which takes the nth-level candidate keyword as a parent keyword from the first keyword list, wherein n is greater than or equal to 1.
In a possible implementation manner, since the search keyword input by the user may include a plurality of levels of sub-category keywords, after determining the level 1 candidate keyword of the search keyword, the terminal determines the next level candidate keyword of the level 1 candidate keyword, and searches step by step until each candidate keyword does not have the next level candidate keyword.
Illustratively, the terminal acquires a search keyword "food", acquires a first keyword and a first keyword identifier corresponding to the search keyword, namely "fruit" and "3", "dessert" and "4", then respectively queries a next-level candidate keyword of "fruit" and "dessert", namely, searches for the first keyword using "3" and "4" as a parent keyword identifier, acquires a next-level candidate keyword "pineapple" and "banana" of "fruit", and respective first keyword identifiers "1" and "2", wherein the first keyword list does not include the first keyword using "4" as a parent keyword identifier, namely, the first keyword list does not include the next-level candidate keyword of "dessert", and similarly, the next-level candidate keyword does not exist for "pineapple" and "banana".
And thirdly, determining the candidate keywords at all levels as associated keywords.
After the terminal determines all levels of candidate keywords of the search keywords in the first candidate word list, all levels of candidate keywords are determined as associated keywords, and the associated keywords comprise all subclasses of keywords of the search keywords.
Illustratively, the terminal queries the candidate keywords "pineapple" and "banana" at the level 2 of "food" in the first candidate word list, and determines that the candidate keywords at the next level do not exist, and then determines the queried "dessert", "fruit", "pineapple" and "banana" as the associated keywords of "food".
In step 404, if the search keyword is not found in the first keyword list, the search keyword is found in a second keyword list, where the second keyword stored in the second keyword list is different from the first keyword stored in the first keyword list.
The classification label of the image recognition model can generally cover the category of the photo album image, but for the same category of image, the name of the image may have a plurality of synonyms, and the classification label of the image recognition model is relatively single, so that when the user inputs the synonym of the classification label, the search keyword cannot be found in the first keyword list.
In a possible implementation manner, the terminal database stores a second keyword list, and the second keyword is different from the first keyword stored in the first keyword list and comprises search keywords which are different from the classification labels of the image recognition model and can be input by the user. And if the result that the terminal searches for the search keyword in the first storage list does not exist, searching for the search keyword in the second keyword list.
In a possible implementation manner, a second keyword list is stored in the terminal database, and the second keyword in the second keyword list is a synonym of the first keyword.
Illustratively, as shown in fig. 7, the first column of the second keyword list is a second keyword identifier, the second column is a second keyword, the third column is a synonymous keyword identifier, and the synonymous keyword identifier is a first keyword identifier of the first keyword that is synonymous with the second keyword. For example, "food" and "food" in the second keyword list are synonyms of "food" in the first keyword list, and the synonym keyword thereof is identified as the first keyword identification "3" of "food"; the pineapple in the second keyword list is a synonym of the pineapple in the first keyword list, and the synonym of the pineapple is identified as the first keyword of the pineapple, namely 1.
Illustratively, the terminal obtains a search keyword 'food', firstly searches for 'food' in the first keyword list, if the result is that 'food' does not exist, searches for 'food' from the second keyword list, searches for 'food', and obtains a synonymous keyword identifier '3'.
In a possible implementation manner, the search keyword is a synonym of a certain classification tag, and the classification tag has a subclass tag, so after the terminal finds the synonym of the search keyword, it needs to find the subclass keyword of the synonym keyword from the first keyword list.
The step 403 may be referred to in the process of searching the sub-category keywords corresponding to the synonymous keyword, which is not described herein again in this embodiment.
Illustratively, as shown in fig. 7, the terminal obtains the search keyword "food", first searches for "food" in the first keyword list, if the result is that the search keyword does not exist, searches for "food" from the second keyword list, searches for "food", obtains the synonymous keyword identifier "3", determines the first keyword having the first keyword identifier "3" from the first keyword list, and if the result is "food", continues to search for all the ranked subclasses of the "food" from the first keyword list, and if the result is "fruit" and "pineapple", the associated keywords of the search keyword "food" are "food", "fruit" and "pineapple".
The implementation of step 407 can refer to step 203 described above, and this embodiment is not described herein again.
In the embodiment of the application, the first keyword list and the second keyword list are stored in the database of the terminal, the synonym keywords and the subclass keywords of the search keywords are obtained, and the pictures corresponding to the classification labels are searched according to the search keywords, the synonym keywords and the subclass keywords, so that the situation that the terminal cannot obtain correct picture search results when the search keywords input by a user are synonyms or father words of the classification labels of the target pictures is avoided, and the accuracy of picture search is improved.
The second keyword list preset in the database of the terminal is difficult to cover possible synonyms of all classification labels, and a user may actively add related labels, picture character records and the like when storing pictures, or independently establish an album and set titles for a class of pictures, so that the terminal can update the content of the second keyword list according to related information added by the user.
On the basis of fig. 4, as shown in fig. 8, step 401 may further include steps 408 to 412.
In a possible implementation manner, when the terminal detects that the to-be-stored picture carries the classification tag, or receives a tag modification operation on a stored picture, the terminal obtains the classification tag output by the picture identification model or the classification tag before modification.
Illustratively, for a picture with a classification label of "potato", the user modifies the classification label of the picture into "potato", and when the terminal detects the label modification operation, the terminal obtains the classification label of "potato" before modification and the classification label of "potato" after modification.
And 409, if the modified classification label does not belong to the first keyword list and the second keyword list, updating the second keyword list according to the classification label before modification and the classification label after modification.
And the terminal searches the modified classification labels from the first keyword list and the second keyword list, if the first keyword does not exist or the second keyword is consistent with the modified classification labels, the modified classification labels are determined to be synonyms of the classification labels before modification, and the second keyword list is updated.
Illustratively, the terminal searches for the 'potatoes' in the first keyword list and the second keyword list, if the first keyword and the second keyword which are consistent with each other are not searched, the second keyword 'potatoes' is added to the second keyword list and matched with the second keyword identification, and the first keyword identification of the 'potatoes' in the first keyword list is added to the synonymy keyword identification corresponding to the row of the 'potatoes'.
In a possible implementation manner, in order to improve the accuracy of data updating (avoid updating the modified classification tag to the second keyword list when the expression meanings of the modified classification tag are greatly different from those of the modified classification tag), the terminal obtains word vectors corresponding to the modified classification tag and the modified classification tag respectively, calculates the similarity between the word vectors (for example, calculates the cosine distance between the word vectors), and updates the second keyword list according to the modified classification tag and the modified classification tag when the similarity is higher than a similarity threshold (for example, 80%).
In a possible implementation manner, under the condition of permission of a user, after the terminal detects that the second keyword list is updated, the terminal sends a keyword data packet containing the second keyword list to the server, so that the server can update and expand the second keyword list in time. The keyword data packet comprises a second keyword list and a user identifier, the server acquires user information corresponding to the user identifier and performs clustering updating on the second keyword list according to the user information, wherein the user information comprises at least one of geographic position, age, preference and gender.
Optionally, the terminal sends the keyword data packet to the server at predetermined time intervals.
In a possible implementation manner, the server is associated with a plurality of terminals, and after the terminals send the locally updated keyword data packets to the server, the server performs clustering update on the second keyword list corresponding to the same geographic position according to the geographic position included in the user information. For example, clustering is performed on a certain synonym which is updated in a certain region, a second keyword list for the region is made, and the second keyword list after clustering update is issued to all the associated terminals of the corresponding region; or for the classification labels used in the specific age group user set, making a specific second keyword list, and issuing the second keyword list after clustering update to the associated terminal of the corresponding age group user. And the terminal receives the updating data packet sent by the server and updates the second keyword list stored locally. And updating the second keyword lists of the user terminals in all regions or different age groups in a clustering updating mode, so that the specific title of a certain region or a certain age group to a certain object can be increased. And the operation of autonomously changing the labels by the user is utilized, so that the difficulty of expansion of synonyms is reduced while the second keyword list is enriched.
And the terminal updates the second keyword list stored in the local database into a cluster-updated second keyword list sent by the server and stores the updated second keyword list.
In the embodiment of the application, the terminal updates the second keyword list stored locally based on the operation of actively adding or modifying the classification label by the user and sends the second keyword list to the server, and the server performs cluster updating based on the terminal information and the updated second keyword list and makes a specific second updating list aiming at a specific user group; and the operation that the user changes the label independently is utilized, the difficulty of expansion of the synonym is reduced while the second keyword list is enriched.
The first keyword in the first keyword list corresponds to a classification tag of the image recognition model, and the classification tag may change correspondingly after the image recognition model is updated, so in a possible implementation manner, the terminal updates according to the model data of the image classification model sent by the server and the first keyword list. On the basis of fig. 8, as shown in fig. 9, step 401 may further include steps 413 and 414 before.
And after the server updates the image classification model, the server sends the model updating data packet to all associated terminals, and the terminals automatically receive the model updating data packet or remind users of receiving the model updating data packet based on the updating notification of the server.
Illustratively, before the image recognition model is updated, the number of the classification tags is 88, the number of the keywords in the first keyword list is 88, and after the image recognition model is updated, the number of the classification tags is increased to 120, and then the number of the keywords in the first keyword list is also increased to 120.
And the terminal receives a model updating data packet sent by the server, and stores corresponding data in the data packet to the local, wherein the model updating data packet comprises the model data of the updated image classification model and the updated first keyword list.
In the embodiment of the application, the server updates the picture identification model and issues the data packet to the associated terminal, and the terminal updates the model data of the picture identification model issued by the server and the first keyword list, so that the content of the first keyword list is expanded, the range of the classification labels is expanded, and the accuracy of the picture search result is improved.
Referring to fig. 10, a block diagram of a picture search apparatus according to an exemplary embodiment of the present application is shown. The apparatus may be implemented as all or a portion of the terminal in software, hardware, or a combination of both. The device includes:
a first obtaining module 1001, configured to obtain a search keyword when an image search instruction is received;
a first determining module 1002, configured to determine at least one associated keyword corresponding to the search keyword from a first keyword list, where the first keyword included in the first keyword list corresponds to a classification tag of an image classification model, the image classification model is used to set the classification tag for an image in an album, and an association relationship between the associated keyword and the search keyword includes a synonymy relationship or a parent-child relationship;
a second determining module 1003, configured to determine, according to the search keyword and the associated keyword, a target picture in the album, where a classification tag corresponding to the target picture is matched with the search keyword or the associated keyword.
Optionally, the first determining module 1002 includes:
a first search unit configured to search the first keyword list for the search keyword;
a first determining unit, configured to determine, if the search keyword is found in the first keyword list, a sub-category keyword corresponding to the search keyword in the first keyword list as the associated keyword.
Optionally, the first keyword list includes a first keyword identifier and a corresponding relationship between the first keyword and a parent keyword identifier, where the first keyword identifier is an identifier corresponding to the first keyword, and the parent keyword identifier is an identifier corresponding to a parent keyword corresponding to the first keyword;
the first determining unit is further configured to:
determining a level 1 candidate keyword which takes the search keyword as a parent keyword from the first keyword list;
for the nth-level candidate keyword, determining an nth + 1-level candidate keyword which takes the nth-level candidate keyword as a parent keyword from the first keyword list, wherein n is more than or equal to 1;
and determining the candidate keywords at all levels as the associated keywords.
Optionally, the first determining module 1002 further includes:
a second searching unit, configured to search for the search keyword in a second keyword list if the search keyword is not found in the first keyword list, where a second keyword stored in the second keyword list is different from the first keyword stored in the first keyword list;
a third searching unit, configured to search a synonymous keyword corresponding to the search keyword from the second keyword list if the search keyword is found in the second keyword list, where the second keyword list includes a synonymous relationship between the second keyword and the first keyword, and the synonymous keyword belongs to the first keyword list;
a second determining unit, configured to determine the synonymous keyword and a sub-category keyword corresponding to the synonymous keyword in the first keyword list as the associated keyword.
Optionally, the apparatus further comprises:
the second obtaining module is used for obtaining a classification label before modification and a classification label after modification when receiving label modification operation on a stored picture, wherein the classification label before modification belongs to the first keyword list;
and the updating module is used for updating the second keyword list according to the classification label before modification and the classification label after modification if the classification label after modification does not belong to the first keyword list and the second keyword list.
Optionally, the apparatus further comprises:
the server is used for acquiring user information corresponding to the user identification and clustering and updating the second keyword list according to the user information, wherein the user information comprises at least one of geographic position, age, preference and gender;
the first receiving module is used for receiving an update data packet sent by the server, wherein the update data packet comprises the second keyword list after clustering update;
and the first storage module is used for storing the updated second keyword list.
Optionally, the apparatus further comprises:
the second receiving module is used for receiving a model updating data packet sent by the server, wherein the model updating data packet comprises updated model data of the image classification model and an updated first keyword list;
and the second storage module is used for storing the updated first keyword list.
Referring to fig. 11, a block diagram of a terminal 1100 according to an exemplary embodiment of the present application is shown. The terminal 1100 may be an electronic device in which an application is installed and run, such as a smart phone, a tablet computer, an electronic book, a portable personal computer, and the like. Terminal 1100 in the present application may include one or more of the following components: a processor 1110, a memory 1120, and a screen 1130.
The Memory 1120 may include a Random Access Memory (RAM) or a Read-Only Memory (ROM). Optionally, the memory 1120 includes a non-transitory computer-readable medium. The memory 1120 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 1120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, and the like), instructions for implementing the above-described method embodiments, and the like, and the operating system may be an Android (Android) system (including a system based on Android system depth development), an IOS system developed by apple inc (including a system based on IOS system depth development), or other systems. The stored data area may also store data created by terminal 1100 during use (e.g., phone book, audio-visual data, chat log data), etc.
The screen 1130 may be a capacitive touch display screen for receiving a touch operation of a user thereon or nearby using any suitable object such as a finger, a stylus, or the like, and displaying a user interface of each application. The touch display screen is generally provided on the front panel of the terminal 1100. The touch display screen may be designed as a full-face screen, a curved screen, or a profiled screen. The touch display screen can also be designed to be a combination of a full-face screen and a curved-face screen, and a combination of a special-shaped screen and a curved-face screen, which is not limited in the embodiment of the present application.
In addition, those skilled in the art will appreciate that the configuration of terminal 1100 illustrated in the above-described figures does not constitute a limitation of terminal 1100, and that terminals may include more or less components than those illustrated, or some components may be combined, or a different arrangement of components. For example, the terminal 1100 further includes a radio frequency circuit, a shooting component, a sensor, an audio circuit, a Wireless Fidelity (Wi-Fi) component, a power supply, a bluetooth component, and other components, which are not described herein again.
The embodiment of the present application further provides a computer-readable storage medium, where at least one instruction is stored, and the at least one instruction is loaded and executed by the processor to implement the picture search method according to the above embodiments.
The embodiment of the present application further provides a computer program product, where at least one instruction is stored, and the at least one instruction is loaded and executed by the processor to implement the picture search method according to the above embodiments.
Those skilled in the art will recognize that, in one or more of the examples described above, the functions described in the embodiments of the present application may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable storage medium. Computer-readable storage media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (10)
1. A picture searching method, characterized in that the method comprises:
when an image searching instruction is received, obtaining a searching keyword;
determining at least one associated keyword corresponding to the search keyword from a first keyword list, wherein the first keyword contained in the first keyword list corresponds to a classification tag of an image classification model, the image classification model is used for setting the classification tag for an image in an album, and the association relationship between the associated keyword and the search keyword comprises a synonymy relationship or a parent-child relationship;
and determining a target picture in the album according to the search keyword and the associated keyword, wherein the classification label corresponding to the target picture is matched with the search keyword or the associated keyword.
2. The method of claim 1, wherein determining at least one associated keyword corresponding to the search keyword from the first keyword list comprises:
searching the search keyword in the first keyword list;
if the search keyword is found in the first keyword list, determining a sub-class keyword corresponding to the search keyword in the first keyword list as the associated keyword.
3. The method according to claim 2, wherein the first keyword list includes a first keyword identifier, and a corresponding relationship between the first keyword and a parent keyword identifier, the first keyword identifier is an identifier corresponding to the first keyword, and the parent keyword identifier is an identifier corresponding to a parent keyword of the first keyword;
determining the subclass keywords corresponding to the search keywords in the first keyword list as the associated keywords, including:
determining a level 1 candidate keyword which takes the search keyword as a parent keyword from the first keyword list;
for the nth-level candidate keyword, determining an nth + 1-level candidate keyword which takes the nth-level candidate keyword as a parent keyword from the first keyword list, wherein n is more than or equal to 1;
and determining the candidate keywords at all levels as the associated keywords.
4. The method of claim 2, wherein after finding the search keyword in the first keyword list, the method further comprises:
if the search keyword is not found in the first keyword list, searching the search keyword in a second keyword list, wherein the second keyword stored in the second keyword list is different from the first keyword stored in the first keyword list;
if the search keyword is found in the second keyword list, finding a synonymous keyword corresponding to the search keyword from the second keyword list, wherein the second keyword list comprises a synonymous relationship between the second keyword and the first keyword, and the synonymous keyword belongs to the first keyword list;
and determining the synonymous keywords and the subclass keywords corresponding to the synonymous keywords in the first keyword list as the associated keywords.
5. The method of claim 4, further comprising:
when a label modification operation on a stored picture is received, obtaining a classification label before modification and a classification label after modification, wherein the classification label before modification belongs to the first keyword list;
and if the modified classification label does not belong to the first keyword list and the second keyword list, updating the second keyword list according to the classification label before modification and the classification label after modification.
6. The method of claim 4, further comprising:
sending a keyword data packet to a server, wherein the keyword data packet comprises the second keyword list and a user identifier, the server is used for acquiring user information corresponding to the user identifier and performing clustering update on the second keyword list according to the user information, and the user information comprises at least one of geographic position, age, preference and gender;
receiving an update data packet sent by the server, wherein the update data packet comprises the second keyword list after clustering update;
and storing the updated second keyword list.
7. The method of any of claims 1 to 6, further comprising:
receiving a model updating data packet sent by the server, wherein the model updating data packet comprises updated model data of the image classification model and an updated first keyword list;
and storing the updated first keyword list.
8. An apparatus for searching pictures, the apparatus comprising:
the first acquisition module is used for acquiring search keywords when receiving an image search instruction;
the first determining module is used for determining at least one associated keyword corresponding to the search keyword from a first keyword list, wherein the first keyword contained in the first keyword list corresponds to a classification tag of an image classification model, the image classification model is used for setting the classification tag for an image in an album, and the association relationship between the associated keyword and the search keyword comprises a synonymy relationship or a parent-child relationship;
and the second determining module is used for determining a target picture in the photo album according to the search keyword and the associated keyword, wherein the classification label corresponding to the target picture is matched with the search keyword or the associated keyword.
9. A terminal, characterized in that the terminal comprises a processor and a memory; the memory stores at least one instruction for execution by the processor to implement the picture search method of any of claims 1 to 7.
10. A computer-readable storage medium storing at least one instruction for execution by a processor to implement the picture search method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911349275.5A CN111104536A (en) | 2019-12-24 | 2019-12-24 | Picture searching method, device, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911349275.5A CN111104536A (en) | 2019-12-24 | 2019-12-24 | Picture searching method, device, terminal and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111104536A true CN111104536A (en) | 2020-05-05 |
Family
ID=70423757
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911349275.5A Pending CN111104536A (en) | 2019-12-24 | 2019-12-24 | Picture searching method, device, terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111104536A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111966850A (en) * | 2020-07-21 | 2020-11-20 | 珠海格力电器股份有限公司 | Picture screening method and device, electronic equipment and storage medium |
| CN115203457A (en) * | 2022-07-15 | 2022-10-18 | 小米汽车科技有限公司 | Image retrieval method, image retrieval device, vehicle, storage medium and chip |
| CN118132859A (en) * | 2024-05-10 | 2024-06-04 | 中国石油大学(华东) | Cloud disk picture asset accurate searching system and method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106708834A (en) * | 2015-08-07 | 2017-05-24 | 腾讯科技（深圳）有限公司 | Object searching method, device and server |
| CN108038192A (en) * | 2017-12-11 | 2018-05-15 | 广东欧珀移动通信有限公司 | Application searches method and apparatus, electronic equipment, computer-readable recording medium |
| CN109992689A (en) * | 2019-03-26 | 2019-07-09 | 华为技术有限公司 | Searching method, terminal and medium |
-
2019
- 2019-12-24 CN CN201911349275.5A patent/CN111104536A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106708834A (en) * | 2015-08-07 | 2017-05-24 | 腾讯科技（深圳）有限公司 | Object searching method, device and server |
| CN108038192A (en) * | 2017-12-11 | 2018-05-15 | 广东欧珀移动通信有限公司 | Application searches method and apparatus, electronic equipment, computer-readable recording medium |
| CN109992689A (en) * | 2019-03-26 | 2019-07-09 | 华为技术有限公司 | Searching method, terminal and medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111966850A (en) * | 2020-07-21 | 2020-11-20 | 珠海格力电器股份有限公司 | Picture screening method and device, electronic equipment and storage medium |
| CN115203457A (en) * | 2022-07-15 | 2022-10-18 | 小米汽车科技有限公司 | Image retrieval method, image retrieval device, vehicle, storage medium and chip |
| CN115203457B (en) * | 2022-07-15 | 2023-11-14 | 小米汽车科技有限公司 | Image retrieval method, device, vehicle, storage medium and chip |
| CN118132859A (en) * | 2024-05-10 | 2024-06-04 | 中国石油大学(华东) | Cloud disk picture asset accurate searching system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11003678B2 (en) | Method and system for presenting a search result in a search result card | |
| US20080208847A1 (en) | Relevance ranking for document retrieval | |
| US20130282702A1 (en) | Method and system for search assistance | |
| CN105912630B (en) | information expansion method and device | |
| US20130282682A1 (en) | Method and System for Search Suggestion | |
| US20240256621A1 (en) | Query completion based on location | |
| CN111104536A (en) | Picture searching method, device, terminal and storage medium | |
| US10776571B2 (en) | Dispatch of user input to multiple input fields in a user interface | |
| CN110309405B (en) | Project recommendation method and device and storage medium | |
| CN107911448B (en) | Content pushing method and device | |
| US10268655B2 (en) | Method, device, server and storage medium of searching a group based on social network | |
| US20140379719A1 (en) | System and method for tagging and searching documents | |
| JP6932360B2 (en) | Object search method, device and server | |
| CN111651570A (en) | Text sentence processing method and device, electronic equipment and storage medium | |
| CN105550217B (en) | Scene music searching method and scene music searching device | |
| WO2017136295A1 (en) | Adaptive seeded user labeling for identifying targeted content | |
| CN114564666A (en) | Encyclopedic information display method, encyclopedic information display device, encyclopedic information display equipment and encyclopedic information display medium | |
| CN110196833B (en) | Application searching method, device, terminal and storage medium | |
| CN112395396A (en) | Question-answer matching and searching method, device, system and storage medium | |
| CN105589852A (en) | Information recommendation method and device | |
| KR20190031536A (en) | Application Information Triggering | |
| CN110569419A (en) | question-answering system optimization method and device, computer equipment and storage medium | |
| US7801889B2 (en) | Search system for providing information of keyword input frequency by category and method thereof | |
| US20140280098A1 (en) | Performing application search based on application gaminess | |
| CN116016421A (en) | Method, computing device readable storage medium, and computing device for facilitating media-based content sharing performed in a computing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |