CN115269901A

CN115269901A - Method, device and equipment for generating extended image

Info

Publication number: CN115269901A
Application number: CN202110486627.2A
Authority: CN
Inventors: 赵奕涵
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-05-01
Filing date: 2021-05-01
Publication date: 2022-11-01

Abstract

The application provides an extended image generation method, device and equipment, and relates to the technical field of computers and communication. The method comprises the following steps: acquiring a reference image and acquiring reference elements in the reference image; performing multi-dimensional classification on the reference elements to obtain label classification results of the reference elements, wherein the label classification results are used for determining reference label combinations of the reference elements, and the reference label combinations comprise reference labels of the reference elements in multiple dimensions; matching a reference label combination corresponding to the label classification result with material label combinations of a plurality of material elements according to dimensionality, and determining a target material element, the matching degree of which with the reference label of the reference element meets a first preset matching condition, from the plurality of material elements; the material label combination of the material elements consists of material labels in at least one dimension; and generating an extended image corresponding to the reference image based on the target material element, and automatically generating the extended image.

Description

Extended image generation method, device and equipment

Technical Field

The present application relates to the field of computer and communication technologies, and in particular, to an extended image generation method, apparatus, and device.

Background

With the progressive development of society, people have higher and higher requirements on design drawings, and designers need to provide a plurality of design drawings with similar contents aiming at the same design requirement.

In the prior art, designers usually design a reference image or obtain the reference image from a client, and then develop an expansion image similar to the reference image in content according to the reference image, so that the expansion process does not need much creativity, the working machinery is repeated, but a great deal of time and energy are spent.

Disclosure of Invention

The application aims to provide an extended image generation method, device, equipment and storage medium, which can automatically generate an extended image.

According to an aspect of an embodiment of the present application, there is provided an extended image generation method, including: acquiring a reference image and acquiring reference elements in the reference image; performing multi-dimensional classification on the reference elements to obtain label classification results of the reference elements, wherein the label classification results are used for determining reference label combinations of the reference elements, and the reference label combinations comprise reference labels of the reference elements in multiple dimensions; matching the reference label combination corresponding to the label classification result with material label combinations of a plurality of material elements according to dimensions, and determining a target material element from the plurality of material elements, wherein the matching degree of the target material element with the reference label of the reference element meets a first preset matching condition; wherein a material label combination of the material elements consists of material labels in at least one dimension; and generating an expansion image corresponding to the reference image based on the target material element.

According to an aspect of an embodiment of the present application, there is provided an extended image generating apparatus including: the acquisition module is configured to acquire a reference image and acquire a reference element in the reference image; a classification module configured to perform multi-dimensional classification on the reference element to obtain a tag classification result of the reference element, where the tag classification result is used to determine a reference tag combination of the reference element, and the reference tag combination includes reference tags of the reference element in multiple dimensions; the matching module is configured to match a reference label combination corresponding to the label classification result with material label combinations of a plurality of material elements according to dimensions, and determine a target material element from the plurality of material elements, wherein the matching degree of the target material element with the reference label of the reference element meets a first preset matching condition; wherein a material label combination of the material elements consists of material labels in at least one dimension; and the generating module is configured to generate an expansion image corresponding to the reference image based on the target material element.

In an embodiment of the application, based on the foregoing scheme, the tag classification result includes a reference tag combination of the reference element, and the matching module is configured to: for the material label combination of each material element, comparing the labels of the reference label combination and the material label combination in the same dimension; determining a target dimension and a corresponding target dimension number of the material label combination and the reference label combination having the same label, wherein the reference label matching degree of the material element and the reference element is determined by the target dimension number; when the target dimensionality number corresponding to the material elements reaches a first preset number threshold value, judging that the matching degree of the reference labels of the material elements and the reference elements meets a first preset matching condition; and determining the material elements of which the target dimension number reaches a first preset number threshold value as the target material elements based on the target dimension number corresponding to the material label combination of each material element in the plurality of material elements.

In an embodiment of the present application, based on the foregoing solution, the matching module is further configured to: semantic expansion is carried out on each reference label in the reference label combination to obtain an expanded label combination of the reference element, wherein the expanded label combination is composed of expanded labels in the multiple dimensions; matching the expansion tag combination with the material tag combination according to dimensions, and determining expansion material elements of which the matching degree with the expansion tags of the reference elements meets a second preset matching condition from the plurality of material elements; the generation module is configured to: and generating an expansion image corresponding to the reference image based on the target material element and the expansion material element.

In an embodiment of the present application, based on the foregoing solution, the matching module is configured to: obtaining word vectors corresponding to texts of all reference labels; calculating the distance between the word vector corresponding to the text of each reference label and the word vector corresponding to the text of the candidate label; and selecting the extended labels with the distance within a set range from the candidate labels to combine to obtain the extended label combination.

In an embodiment of the application, based on the foregoing scheme, the tag classification result includes a reference tag probability distribution of the reference element in multiple dimensions, where the reference tag probability distribution includes reference tag probabilities of the reference element corresponding to multiple dimension tags in each dimension, and the reference tag is a dimension tag with a highest reference tag probability in each dimension; the matching module is configured to: acquiring reference label probability distribution of the reference label combination in each dimensionality; acquiring material label probability distribution of the material label combination corresponding to each dimension in the dimensions, wherein the material label probability distribution corresponding to each dimension comprises material label probability of the material element corresponding to the dimension labels, and the material label is a dimension label with the highest material label probability in each dimension; and matching the reference label probability distribution with the material label probability distribution according to dimensionality, and determining a target material element, of which the matching degree with the reference label of the reference element meets the first preset matching condition, from the plurality of material elements.

In an embodiment of the present application, based on the foregoing solution, the matching module is configured to: calculating a first difference value of the reference label probability and the material label probability corresponding to the same dimension label in the same dimension, and determining a first label difference degree of the reference label combination and the material label combination corresponding to each label in each dimension based on the first difference value; determining a first element difference degree of the reference element and the material element based on a first label difference degree corresponding to each dimension label in each dimension, wherein the reference label matching degree of the material element and the reference element is determined by the first element difference degree; when the first element difference degree corresponding to the material elements reaches a first difference degree threshold value, judging that the matching degree of the reference labels of the material elements and the reference elements meets a first preset matching condition; and determining the material elements with the first element difference degree lower than a first difference degree threshold value as the target material elements based on the first element difference degree corresponding to each material element in the plurality of material elements.

In an embodiment of the present application, based on the foregoing solution, the matching module is configured to: summing the first label difference degrees corresponding to the labels of the dimensions to obtain the first dimension difference degrees of the dimensions corresponding to the reference element and the material element; and summing the first dimension difference degrees of the material elements corresponding to all dimensions to serve as the first element difference degree of the reference element and the material elements.

In an embodiment of the present application, based on the foregoing solution, the generating module is configured to: and replacing the reference elements in the reference image with the target material elements to obtain the expanded image.

According to an aspect of embodiments of the present application, there is provided a computer program storage medium storing computer program instructions which, when executed by a computer, cause the computer to perform the method of any one of the above.

According to an aspect of an embodiment of the present application, there is provided an electronic device including: a processor; a memory having computer readable instructions stored thereon which, when executed by the processor, implement a method as in any above.

According to an aspect of embodiments herein, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the various alternative embodiments described above.

The technical scheme provided by the embodiment of the application can have the following beneficial effects:

in the technical solutions provided by some embodiments of the present application, a reference image is obtained, and a reference element in the reference image is obtained; performing multi-dimensional classification on the reference elements to obtain label classification results of the reference elements, wherein the label classification results are used for determining reference label combinations of the reference elements, and the reference label combinations comprise reference labels of the reference elements in multiple dimensions; matching the reference label combination corresponding to the label classification result with the material label combinations of the plurality of material elements according to dimensions, and determining a target material element from the plurality of material elements, wherein the matching degree of the reference label of the reference element meets a first preset matching condition; the material label combination of the material elements consists of material labels in at least one dimension; the extended image corresponding to the reference image is generated based on the target material elements, so that the extended image of the reference image can be automatically generated, and the extended image corresponding to the reference image is generated by matching the material elements with similar dimensions according to the dimensions, and the generated extended image is similar to the reference image in multiple dimensions, so that the image extension quality can be ensured, and the image output efficiency can be greatly improved. In a practical application, for example, in the field of design, one design drawing can be automatically converted into a plurality of design drawings through the method, so that the participation of designers is greatly reduced while the design drawings are enriched, and the labor cost is reduced; meanwhile, the designer design rules and specifications are precipitated through multi-dimensional classification, so that the design output efficiency is improved, and the high-quality image quality meeting the design rules and specifications is ensured.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

FIG. 1 shows a schematic diagram of an exemplary system architecture to which aspects of embodiments of the present application may be applied;

FIG. 2A is a diagram illustrating a data sharing system to which aspects of one embodiment of the present application may be applied;

FIG. 2B shows a block chain diagram to which one embodiment of the present application may be applied;

FIG. 2C is a diagram illustrating the generation of new tiles in a blockchain to which one embodiment of the present application may be applied;

FIG. 3 schematically shows a flow diagram of an extended image generation method according to an embodiment of the application;

FIG. 4A schematically illustrates a reference image schematic according to an embodiment of the present application;

FIG. 4B schematically illustrates an expanded image obtained from FIG. 4A as a reference image according to the present application;

FIG. 5 schematically shows a flow diagram of an extended image generation method according to an embodiment of the application;

fig. 6 schematically shows a CNN algorithm classification flow diagram according to an embodiment of the present application;

FIG. 7 schematically shows a flow diagram of an extended image generation method according to one embodiment of the present application;

FIG. 8 schematically shows a block diagram of an extended image generation apparatus according to an embodiment of the present application;

FIG. 9 is a hardware diagram illustrating an electronic device according to an example embodiment.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the subject matter of the present application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the application.

The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.

The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.

Fig. 1 shows a schematic diagram of an exemplary system architecture 100 to which the technical solutions of the embodiments of the present application can be applied.

As shown in fig. 1, system architecture 100 may include clients 101, network 102, and server 103. Network 102 serves as a medium for providing communication links between clients 101 and servers 103. Network 102 may include various connection types, such as wired communication links, wireless communication links, and so forth, although the application is not limited thereto.

It should be understood that the number of clients 101, networks 102, and servers 103 in fig. 1 is merely illustrative. There may be any number of clients 101, networks 102, and servers 103, as desired for implementation. For example, the server 103 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, a middleware service, a domain name service, a security service, a CDN, and a big data and artificial intelligence platform. The client 101 may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, and the like.

In one embodiment of the present application, the server 103 obtains the reference image and obtains the reference elements in the reference image; performing multi-dimensional classification on the reference elements to obtain label classification results of the reference elements, wherein the label classification results are used for determining reference label combinations of the reference elements, and the reference label combinations comprise reference labels of the reference elements in multiple dimensions; matching a reference label combination corresponding to the label classification result with material label combinations of a plurality of material elements according to dimensionality, and determining a target material element, the matching degree of which with the reference label of the reference element meets a first preset matching condition, from the plurality of material elements; the material label combination of the material elements consists of material labels in at least one dimension; and generating an extended image corresponding to the reference image based on the target material elements, and automatically generating the extended image.

It should be noted that the extended image generation method provided in the embodiment of the present application is generally executed by the server 103, and accordingly, the extended image generation apparatus is generally disposed in the server 103. However, in other embodiments of the present application, the client 101 may also have a similar function as the server 103, so as to execute the extended image generation method provided in the embodiments of the present application.

Fig. 2A shows a schematic diagram of an exemplary data sharing system 200 to which technical solutions of embodiments of the present invention can be applied.

Referring to the data sharing system 200 shown in fig. 2A, the data sharing system 200 refers to a system for performing data sharing between nodes. Each node 201 may receive input information and maintain shared data within the data sharing system 200 based on the received input information while operating normally. In order to ensure information intercommunication in the data sharing system 200, an information connection may exist between each node 201 in the data sharing system 200, and information transmission may be performed between the nodes 201 through the information connection. For example, when an arbitrary node 201 in the data sharing system 200 receives input information, other nodes 201 in the data sharing system 200 acquire the input information according to a consensus algorithm, and store the input information as data in shared data, so that the data stored on all nodes 201 in the data sharing system 200 are consistent.

Each node 201 in the data sharing system 200 has a node identifier corresponding thereto, and each node 201 in the data sharing system 200 may store the node identifiers of other nodes 201 in the data sharing system 200, so as to broadcast the generated block to other nodes 201 in the data sharing system 200 according to the node identifiers of other nodes 201. Each node 201 may maintain a node identifier list as shown in the following table, and store the node name and the node identifier in the node identifier list correspondingly. The node identifier may be an IP (Internet Protocol) address and any other information that can be used to identify the node, and table 1 only illustrates the IP address as an example.

Node name	Node identification
		Node 1	117.114.151.174
Node 2	117.116.189.145
		…	…
Node N	119.123.789.258

TABLE 1

Fig. 2B shows a schematic diagram of a blockchain to which an embodiment of the present application may be applied.

Each node in the data sharing system 200 stores one and the same blockchain. The block chain is composed of a plurality of blocks, as shown in fig. 2B, the block chain is composed of a plurality of blocks, the starting block includes a block header and a block main body, the block header stores an input information characteristic value, a version number, a timestamp and a difficulty value, and the block main body stores input information; the next block of the starting block takes the starting block as a parent block, the next block also comprises a block head and a block main body, the block head stores the input information characteristic value of the current block, the block head characteristic value of the parent block, the version number, the timestamp and the difficulty value, and so on, so that the block data stored in each block in the block chain is associated with the block data stored in the parent block, and the safety of the input information in the block is ensured.

Fig. 2C shows a schematic diagram of new block generation in a blockchain to which an embodiment of the present application may be applied.

When each block in the block chain is generated, referring to fig. 2C, when the node where the block chain is located receives the input information, the input information is verified, after the verification is completed, the input information is stored in the memory pool, and the hash tree used for recording the input information is updated; and then, updating the updating time stamp to the time when the input information is received, trying different random numbers, and calculating the characteristic value for multiple times, so that the calculated characteristic value can meet the following formula:

SHA256(SHA256(version+prev_hash+merkle_root+ntime+nbits+x))＜TARGET

SH256 is a feature value algorithm used to calculate feature values; version is version information of the relevant block protocol in the block chain; prev _ hsh is a block head characteristic value of a parent block of the current block; the quick _ root is a characteristic value of the input information; ntime is the update time of the update timestamp; nbits is the current difficulty, is a fixed value within a period of time, and is determined again after exceeding a fixed time period; x is a random number; TRGET is a feature value threshold, which can be determined from nbits.

Therefore, when the random number meeting the formula is obtained through calculation, the information can be correspondingly stored, and the block head and the block main body are generated to obtain the current block. Then, the node where the block chain is located sends the newly generated blocks to the other nodes 201 in the data sharing system 200 where the newly generated blocks are located respectively according to the node identifiers of the other nodes 201 in the data sharing system 200, the newly generated blocks are verified by the other nodes 201, and the newly generated blocks are added to the block chain stored by the newly generated blocks after the verification is completed.

In one embodiment of the present application, the input information stored in the data sharing system 200 may be material elements, and may also be candidate tags for selecting the expansion tags. By using the block chain basis processing method for generating the extended image, the accuracy of data acquisition can be improved, and the accuracy of extended image generation is improved.

The implementation details of the technical solution of the embodiment of the present application are set forth in detail below:

fig. 3 schematically shows a flowchart of an extended image generation method according to an embodiment of the present application, and an execution subject of the extended image generation method may be a server, such as the server 103 shown in fig. 1.

Referring to fig. 3, the extended image generating method at least includes steps S310 to S340, which are described in detail as follows:

in step S310, a reference image is acquired, and reference elements in the reference image are acquired.

In one embodiment of the present application, the reference image may be an image uploaded by the client 101. One or more reference elements may be included in the reference image, and the reference elements may be text, graphics, pictures, tables, or the like, for example, fig. 4A schematically illustrates a reference image diagram according to an embodiment of the present application, and fig. 4A may be the 401A hot dog, 402A pizza, 403A donut, 404A drink, 405A text, 406A background, or the like of fig. 4A.

In an embodiment of the present application, the reference image and the reference element in the reference image may be obtained from a design map source file, where the design map source file may be a source file designed by using design software such as Photoshop or Sketch, and includes information such as an element of each layer and a position and a size thereof.

In one embodiment of the present application, the reference elements may be all elements included in the reference image, and in other embodiments of the present application, the reference elements may be selected from the elements included in the reference image, and may be selected based on a user instruction or based on a preset rule.

In step S320, performing multidimensional classification on the reference element to obtain a tag classification result of the reference element, where the tag classification result is used to determine a reference tag combination of the reference element, and the reference tag combination includes reference tags of the reference element in multiple dimensions.

In one embodiment of the present application, the multiple dimensions may include a picture quality dimension, a color brilliance degree dimension, a style dimension, an emotion dimension, a manipulation dimension, an object dimension, and the like, wherein the tags in the picture quality dimension may include high definition, medium definition, low definition, and the like; labels in the color brilliance degree dimension may include highly brilliant, moderately brilliant, lowly brilliant, and the like; labels in the style dimension may include modern, classical, popular, extremely simple, pictorial, etc.; labels in the emotional dimension may include hot blood, dark black, loveliness, etc.; labels in the manual dimension can include watercolors, oil paintings, traditional Chinese paintings and the like; the object dimension label can be set according to the actual expressed content of the element, such as hamburger, potato chips, sweet rings, hotdogs and the like.

In the embodiment, the reference elements are classified not only by considering image dimensions such as picture quality dimension, color brightness degree dimension and the like, but also by considering content dimensions such as style dimension, emotion dimension, manipulation dimension, object dimension and the like, so that the reference elements can be interpreted in more dimensions, and target material elements can be found more accurately.

In an embodiment of the present application, the tag classification result may include a reference tag combination of a reference element, and the feature map of the reference element may be subjected to convolution calculation using a plurality of different convolution kernels to obtain a plurality of convolution images; combining the plurality of convolution images to obtain a combined feature map; and determining reference labels of the reference elements in all dimensions based on the combined feature map, and combining the reference labels of the reference elements in all dimensions to obtain a reference label combination of the reference elements.

In one embodiment of the present application, the reference labels of the reference elements in multiple dimensions may be determined using a Neural network model, wherein the Neural network model may be a Convolutional Neural Network (CNN), such as a mobile end Neural network (mobilet), a deep residual network (ResNet) Convolutional Neural network model, an open end Neural network (inclusion), or the like.

In an embodiment of the present application, after obtaining labels of reference elements output by the neural network model in each dimension, the labels output by the neural network model may be subjected to deduplication processing, and the labels output by the neural network model after deduplication processing are used as reference labels.

In one embodiment of the present application, prior to using the neural network model, the neural network model may be trained using a set of training images, the training images in the set of training images being known corresponding to the training labels in the respective dimensions. And inputting the training images into the neural network model to obtain the prediction labels output by the neural network model, and if the prediction labels corresponding to the same training image are inconsistent with the training labels, adjusting the neural network model until the prediction labels corresponding to the same training image are consistent with the training labels.

In an embodiment of the present application, the tag classification result includes reference tag probability distributions of the reference elements in multiple dimensions, and the reference tag probability distributions include reference tag probabilities of the reference elements corresponding to multiple dimension tags in each dimension, for example, the reference tag probability distribution of the reference elements in the manipulation dimension is: modern 80%, classical 10% and pictorial 10%, wherein the reference element has a reference label probability corresponding to the modern dimension label in the manipulation dimension of 80%, the reference element has a reference label probability corresponding to the classical dimension label in the manipulation dimension of 10%, and the reference element has a reference label probability corresponding to the pictorial dimension label in the manipulation dimension of 10%; the reference label probability distribution of the reference element in the emotion dimension is: 90% of hot blood, 8% of dark and 2% of loved ones, wherein the reference element has a reference label probability corresponding to a hot blood dimension label in an emotion dimension of 90%, the reference element has a reference label probability corresponding to a dark dimension label in the emotion dimension of 8%, and the reference element has a reference label probability corresponding to a loved dimension label in the emotion dimension of 2%; the reference label probability distribution of the reference element in the drawing type dimension is as follows: the watercolor is 75%, the oil painting is 20%, the traditional Chinese painting is 5% and the like, the reference element probability corresponding to the watercolor dimension label in the conversation type dimension is 75%, the reference element probability corresponding to the oil painting dimension label in the conversation type dimension is 20%, and the reference element probability corresponding to the traditional Chinese painting dimension label in the conversation type dimension is 5%.

In one embodiment of the present application, a reference label probability distribution of a reference element in multiple dimensions may be determined by a neural network model.

In one embodiment of the present application, a dimension label with the highest reference label probability in each dimension may be used as a reference label.

In one embodiment of the present application, the reference label probability distributions of the reference elements in the dimensions can be combined to serve as the reference label combination of the reference elements, for example, the reference label combination of the reference elements can be [ 80% modern, 10% classical, 10% art-insertion (hot blood 90%, dark 8%, lovely 2%) (watercolor 75%, oil painting 20%, chinese painting 5%) ].

In one embodiment of the present application, the reference element may be combined with the reference tag probability of the corresponding reference tag, for example, the reference element may be combined with the reference tag (modern 80%, hot blood 90%, watercolor 75%).

In an embodiment of the application, one or more reference tags of the reference element in each dimension may be selected from the dimension tags according to the reference tag probability that the reference element corresponds to each dimension tag.

Continuing to refer to fig. 3, in step S330, matching the reference label combination corresponding to the label classification result with the material label combinations of the plurality of material elements according to dimensions, and determining a target material element from the plurality of material elements, the matching degree of which with the reference label of the reference element meets a first predetermined matching condition; wherein a material label combination of material elements consists of material labels in at least one dimension.

In one embodiment of the present application, a material element corresponding to a material tag combination completely identical to a reference tag combination may be used as a target material element.

In an embodiment of the application, the reference tag combination and the material tag combination may be compared to obtain the number of the same tags in the reference tag combination and the material tag combination, and the material element with the largest number of the same tags is used as the target material element, or the material element with the number of the same tags reaching a set ratio is used as the target material element.

In one embodiment of the application, tags in the same dimension of a reference tag combination and a material tag combination can be compared for the material tag combination of each material element; determining target dimensions and corresponding target dimension numbers of the material label combination and the reference label combination with the same label; and determining the material elements of which the target dimensionality reaches a first preset quantity threshold value as target material elements based on the target dimensionality quantity corresponding to the material label combination of each material element.

In one embodiment of the present application, the material label combination can be obtained by processing the material elements through a neural network model.

In an embodiment of the application, the probability distribution of the reference label combination corresponding to each dimension in a plurality of dimensions can be obtained; acquiring material label probability distribution of a material label combination corresponding to each dimension in a plurality of dimensions, wherein the material label probability distribution corresponding to each dimension comprises material label probabilities of material elements corresponding to the plurality of dimension labels in each dimension, and the material label is a dimension label with the highest material label probability in each dimension; and matching the reference label probability distribution with the material label probability distribution according to the dimensionality, and determining a target material element which meets a first preset matching condition with the reference label matching degree of the reference element from the plurality of material elements.

In an embodiment of the application, a first difference value between a reference label probability and a material label probability corresponding to a label in the same dimension can be calculated, and a first label difference degree of the reference label combination and the material label combination corresponding to each label in each dimension is determined based on the first difference value; determining a first element difference degree between a reference element and a material element based on a first label difference degree corresponding to each dimension label in each dimension; and determining target material elements based on the material elements with the first element difference lower than a first difference threshold, wherein the reference label distribution probability and the material label distribution probability corresponding to the same dimension label refer to the reference label distribution probability of the reference label corresponding to the same label and the material label distribution probability of the material label corresponding to the same label when the reference label is the same as the material label.

In one embodiment of the present application, a material element, of which the first element difference degree is lower than the first difference degree threshold value and of which the first element difference degree is the lowest, may be taken as a target material element.

In one embodiment of the present application, a target material element may be selected from material elements having a first element degree of difference below a first degree of difference threshold.

In an embodiment of the application, absolute values of first differences of the reference label probabilities and the material label probabilities corresponding to the plurality of dimension labels in the same dimension can be made, so that first label difference degrees of the reference label combinations and the material label combinations corresponding to the labels in the dimensions are obtained.

In other embodiments of the application, a first difference between the reference label probability and the material label probability corresponding to a plurality of dimension labels in the same dimension may be squared to obtain a first label difference degree of the reference label combination and the material label combination corresponding to each label in each dimension.

In other embodiments of the application, the first difference between the reference label probability and the material label probability corresponding to the plurality of dimension labels in the same dimension may be averaged after being taken as an absolute value, so as to obtain the first label difference degree of the reference label combination and the material label combination corresponding to each label in each dimension.

In an embodiment of the application, the first tag difference degrees of the multiple dimension tags in each dimension can be summed to obtain the first dimension difference degrees of each dimension corresponding to the reference element and the material element; and summing the first dimension difference degrees of the plurality of dimensions to serve as the first element difference degree of the reference element and the material element.

In one embodiment of the present application, the material label distribution probability can be obtained by processing the material elements through a neural network model.

In an embodiment of the application, a weighted sum of the first dimension difference degrees of the material elements corresponding to the dimensions may be obtained to obtain the first element difference degree of the reference element and the material element, where the weight may be set corresponding to the dimensions.

In an embodiment of the application, the reference tags and the material tags are obtained by using means such as machine learning, and can be directly input by a designer during warehousing, and the probability that the reference elements correspond to the manually input reference tags in the manual input dimension can be 100%. The material elements may have a material tag probability corresponding to manual input in the manual input dimension of 100%.

With continued reference to fig. 3, in step S350, an extended image corresponding to the reference image is generated based on the target material elements.

In an embodiment of the application, the reference element may be replaced by a target material element, so as to obtain an extended image.

In an embodiment of the present application, if the material elements satisfying the first predetermined matching condition cannot be found, or the material elements which fluctuate from the material elements satisfying the first predetermined matching condition and are lower than the set error cannot be found, if the fluctuation exceeds 10% of the first predetermined number threshold, it indicates that the target material elements cannot be found, and the reference elements are not replaced.

In one embodiment of the application, the matching degree of the reference labels of the material elements and the reference elements is determined by the number of target dimensions; when the target dimension number corresponding to the material elements reaches a first preset number threshold, the matching degree of the reference labels of the material elements and the reference elements is judged to meet a first preset matching condition.

In one embodiment of the application, different extension images can be generated by taking different elements in the reference image as reference elements.

In one embodiment of the present application, FIG. 4B schematically illustrates an expanded image schematic according to the present application, as shown in FIG. 4B, where the 401A hot dog may be replaced with a 401B sandwich, the 402A pizza may be replaced with a 402B chicken leg, the 403A donut may be replaced with a 403B meat roll, and the 404A beverage may be replaced with 404B coffee, according to FIG. 4A as a reference image. In other embodiments of the present application, a light color such as yellow-green in the 406A background may be changed to a dark color such as orange, etc. in the 406B background.

In the embodiment of fig. 3, by acquiring a reference image, and acquiring reference elements in the reference image; performing multi-dimensional classification on the reference elements to obtain label classification results of the reference elements, wherein the label classification results are used for determining reference label combinations of the reference elements, and the reference label combinations comprise reference labels of the reference elements in multiple dimensions; matching a reference label combination corresponding to the label classification result with material label combinations of a plurality of material elements according to dimensionality, and determining a target material element, the matching degree of which with the reference label of the reference element meets a first preset matching condition, from the plurality of material elements; the material label combination of the material elements consists of material labels in at least one dimension; and generating an extended image corresponding to the reference image based on the target material element, and automatically generating the extended image.

In the embodiment of fig. 3, in step S350, before generating an extended image corresponding to the reference image based on the target material element, semantic extension may be performed on each reference label in the reference label combination to obtain an extended label combination used for representing semantic features of the reference element, where the extended label combination is composed of extended labels in multiple dimensions; matching the expansion tag combination with the material tag combination according to dimensionality, and determining a target material element which meets a second preset matching condition with the expansion tag matching degree of the reference element from the plurality of material elements.

In an embodiment of the application, word vectors corresponding to texts of all reference tags can be obtained; calculating the distance between the word vector corresponding to the text of each reference label and the word vector corresponding to the text of the candidate label; and selecting the extended labels with the distance within a set range from the candidate labels for combination to obtain an extended label combination.

In one embodiment of the present application, a neural network language model may be used to determine word vectors corresponding to the text of the reference tags and word vectors corresponding to the text of the candidate tags.

In an embodiment of the present application, a cosine distance or an euclidean distance between a word vector corresponding to the text of the reference tag and a word vector corresponding to the text of the reference tag may be calculated as a distance between the word vector corresponding to the text of the reference tag and a word vector corresponding to the text of the candidate tag.

In an embodiment of the application, after semantic expansion is performed on each reference label in the reference label combination to obtain an expansion label, one or more expansion labels corresponding to the reference label may be selected in each dimension to form an expansion label combination.

In one embodiment of the present application, the extension tag may be the same as the reference tag.

In an embodiment of the application, the expansion tag can be obtained by expanding according to the font of each reference tag.

In an embodiment of the application, when the reference tags are expanded, the reference tags can be screened according to the distribution probability of the reference tags, and only the reference tags with the distribution probability of the reference tags reaching a set probability threshold are expanded.

In one embodiment of the present application, the expansion can be performed by manually entered reference tags, such as fruit. The vocabulary can be expanded through the technologies such as word vector and the like, and a series of related words such as watermelon, apple and the like can be expanded through fruits. The technique first requires a word vector library, such as Tencent _ AILab _ ChineseEmbelling. The word vector library may obtain related words by calculating similarity between words, such as a cosine similarity algorithm, or may be obtained by loading a Natural Language Processing (NLP) frame, such as a semantic topic library (Gensim) and an Annoy, into the word vector library. Similar words have quantifiable distance between them. The distance may be normalized, such as 0.8 for hamburger to cola and 0.75 for hamburger to french fries.

In one embodiment of the application, tags of the expansion tag combination and the material tag combination in the same dimension can be compared for the material tag combination of each material element; determining expansion dimensions and corresponding expansion dimension numbers of the same label of the material label combination and the reference label combination; and determining the material elements of which the target dimensionality reaches a second preset quantity threshold value as target material elements based on the expansion dimensionality quantity corresponding to the material label combination of each material element.

In an embodiment of the application, probability distribution of expansion labels of expansion label combinations corresponding to each dimension in multiple dimensions can be obtained; acquiring material label probability distribution of a material label combination corresponding to each dimension in a plurality of dimensions, wherein the material label probability distribution corresponding to each dimension comprises material label probabilities of material elements corresponding to the plurality of dimension labels in each dimension, and the material label is a dimension label with the highest material label probability in each dimension; and matching the probability distribution of the expansion tags with the probability distribution of the material tags according to the dimensionality, determining expansion material elements, the matching degree of which with the expansion tags of the expansion elements meets a second preset matching condition, from the plurality of material elements, and generating an expansion image corresponding to the reference image based on the target material elements and the expansion material elements.

In an embodiment of the application, a second difference value between the expanded label probability and the material label probability corresponding to the same dimension label in the same dimension can be calculated, and a second label difference degree of each label corresponding to each dimension between the expanded label combination and the material label combination is determined based on the second difference value; determining a second element difference degree of the expansion elements and the material elements based on a second label difference degree corresponding to each dimension label in each dimension; and determining target material elements based on the material elements with the second element difference lower than the second difference threshold, wherein the expansion label distribution probability and the material label distribution probability corresponding to the same dimension label refer to the expansion label distribution probability of the expansion label corresponding to the same label and the material label distribution probability of the material label corresponding to the same label when the expansion label is the same as the material label.

In an embodiment of the present application, a material element whose second element difference is lower than a second difference threshold and whose second element difference is the lowest may be used as the target material element.

In one embodiment of the present application, the target material element may be selected from material elements whose second element difference is lower than the second difference threshold.

In an embodiment of the application, absolute values of second differences of the extended label probabilities and the material label probabilities corresponding to the multiple dimension labels in the same dimension can be made, so that second label difference degrees of the extended label combinations and the material label combinations corresponding to the labels in the dimensions are obtained.

In other embodiments of the application, a second difference between the expanded tag probability and the material tag probability corresponding to the multiple dimension tags in the same dimension may be squared to obtain a second tag difference degree of each tag corresponding to each dimension between the expanded tag combination and the material tag combination.

In other embodiments of the application, the second difference values of the expansion tag probabilities and the material tag probabilities corresponding to the multiple dimension tags in the same dimension may be averaged after being taken as absolute values, so as to obtain the second tag difference degrees of the expansion tag combination and the material tag combination corresponding to each tag in each dimension.

In an embodiment of the application, the second label difference degrees of the multiple dimension labels in each dimension can be summed to obtain the second dimension difference degrees of each dimension corresponding to the extension elements and the material elements; and summing the second dimension difference degrees of the multiple dimensions to serve as a second element difference degree of the expansion element and the material element.

In an embodiment of the application, a weighted sum of the second dimension difference degrees of the material elements corresponding to the dimensions can be obtained to obtain the second element difference degree of the extension elements and the material elements, wherein the weight can be set corresponding to the dimensions.

In an embodiment of the application, the reference label probability distribution of the reference label corresponding to the expansion label may be used as the expansion label probability distribution of the expansion label.

In an embodiment of the application, the expanded label probability distribution of the expanded label combination in each dimension can be determined based on the reference label probability distribution of the reference label corresponding to the expanded label and the distance between the word vector of the reference label corresponding to the expanded label and the word vector of the expanded label. Specifically, based on the vector difference between the word vector of the reference tag corresponding to the expansion tag and the word vector of the expansion tag, the expansion tag restoration probability for representing the similarity between the reference tag corresponding to the expansion tag and the expansion tag is determined, and the product of the reference tag probability corresponding to the expansion tag and the expansion tag restoration probability is calculated to obtain the expansion tag probability distribution.

In an embodiment of the application, a corresponding relationship between the vector difference degree and the extended label reduction probability may be preset, so that the extended label reduction probability corresponding to the vector difference may be determined based on the corresponding relationship.

In one embodiment of the present application, a vector distance between the word vector of the reference tag and the word vector of the expansion tag may be calculated as a vector disparity.

FIG. 5 schematically shows a flowchart of an extended image generation method according to an embodiment of the present application, and the main process is divided into 1) reading information of each element of a design drawing source file; 2) Carrying out multi-dimensional classification on the element images through an algorithm; 3) Matching materials consistent with each dimension of the element image from a preprocessed material library for replacement; 4) And outputting the replaced expansion diagram. The auxiliary process comprises the steps of 1) carrying out multi-dimensional classification on the material pictures by using a machine learning algorithm; 2) Marking the classified pictures according to the types of all dimensions, and storing the marked pictures into a material library. The following steps are described. The execution subject of the extended image generation method may be a server, such as the server 103 shown in fig. 1.

Referring to fig. 5, the extended image generating method at least includes steps S510 to S580, and the following steps are described in detail:

in step S510, a plan is acquired;

in step S520, design primitive information is obtained;

in step S530, performing a dimension-defined classification on the design primitive by using a CNN algorithm;

in step S540, a material picture is acquired;

in step S550, classifying the materials in the material library by the designer' S established dimensionality using the CNN algorithm, and storing the dimensionality labels;

in step S560, a material library is acquired

In step S570, material elements matching the same classification dimension label in the material library are replaced;

in step S580, a development map is output.

In the embodiment of fig. 5, the design drawing is used as a reference image, the design drawing elements are used as reference elements, the material in the material library is used as material elements, the target material elements corresponding to the design drawing elements are searched from the material library, and the output expansion drawing is the expansion image. Step S540 may be executed before step S510, or may be executed after step S530, which is not limited herein.

In an embodiment of the present application, in step S530, when the CNN algorithm is used to perform the classification of the design primitive with the predetermined dimension, the input of the CNN algorithm classifier is the picture itself. It is also possible to add other information of the picture such as the size on the design drawing to construct the picture more accurately.

In an embodiment of the present application, layers of a design map source file, design map elements on the layers, and other related information such as size and position may be obtained by a corresponding plug-in or by reading and parsing a file using a code.

In the embodiment of fig. 5, the matching of the material pictures can be performed by matching the tags of the respective dimensions. For example, if the reference elements in the design drawing are classified into (modern, hot blood and watercolor) in each dimension, the probability difference degrees of the labels corresponding to each dimension are searched from the material library, and if the material library has (modern, hot blood and watercolor) material pictures, the material pictures can be selected; if not, the (modern, hot blood and oil painting) can be used for replacement, and the selection is carried out according to the consistent dimensionality, and the more dimensionality is the better. In addition, matching can also be based on the reference label probability distribution of each label in each dimension of the element, so that the matching is more accurate. For example, the material library has a plurality of material pictures (modern, hot blood and watercolor). And then comparing the reference label probability distribution of each label. The probability distribution on the original design drawing is (modern 80%, classical 10%, and pictorial 10%), (hot blood 90%, dark 8%, and lovely 2%), (watercolor 75%, oil painting 20%, and Chinese painting 5%), and then the material picture with the most consistent probability distribution under each dimension is searched from the material library, the consistency is obtained by calculating the absolute value of the difference of each probability, summing up the absolute values to obtain the element difference, and then the material picture element with the minimum error value is selected as the target material picture element. For example, the material pictures in the material library are (modern 70%, classical 20%, and pictorial 10%), (hot blood 80%, dark 18%, lovely 2%), (watercolor 80%, oil painting 15%, and traditional painting 5%). The element difference is (| 80% -70% | + |10% -20% | + |10% -10% > |) + (| 90% -80% | + |8% -18% | + |2% -2% > |) + (| 75% -80% | + |20% -15% | + |5% -10% > |) =0.55, wherein the algorithm of the element difference is very diverse, for example, the absolute value distance can be replaced by the sum of squares.

In one embodiment of the present application, in step S550, the material tags are stored in the system together with the material pictures to form a material library. The material label can be specific (classicality, watercolor and hot blood), or the material label and the material label probability distribution are stored in a combined mode, and the accuracy of the material label is higher. After the reasonable pretreatment is carried out on the material library, the material library can be effectively and fully utilized in future design drawings, and the utilization rate of design materials is improved.

Fig. 6 schematically shows a CNN algorithm classification flow diagram according to an embodiment of the present application, where an execution subject of the CNN algorithm classification method may be a server, such as the server 103 shown in fig. 1.

In fig. 6, the designer classifies according to the given dimension, and each type prepares an example picture to pre-train the network for further training the network to achieve the purpose of classifying according to the style dimension determined by the designer. The pre-trained network comprises a MobileNet with a pre-training weight, a pooling layer (Polling) and a classification layer (Softmax), mass materials are obtained and input into the MobileNet with the pre-training weight, the output result of the MobileNet is input into the pooling layer, the structure of the pooling layer is input into the classification layer, classification probability distribution is obtained and serves as material label distribution probability, and the mass material graph is automatically classified by a machine through an realized algorithm classifier. For example, a material picture can be classified into classical style in style dimension, watercolor in manipulation dimension, and hot blood in emotion dimension. In addition to the deterministic classification, a probability distribution can also be obtained for each dimension, such as 80% machine inferred as classical, 10% as fair, 10% as extremely simple, in the style dimension, and the like.

In the embodiment of fig. 6, the present technical solution implements: according to a machine learning classification algorithm, according to a plurality of design dimensions preset by a designer and classification of each dimension, analyzing reference elements on a design drawing, and replacing pictures matched with the same design dimensions from a material library, thereby automatically generating more expansion drawings.

In one embodiment of the present application, CNN may be used to classify reference elements. Analysis of the design drawing elements using an algorithm: the adopted algorithm is consistent with the algorithm for constructing the design drawing material library in the auxiliary process. Analyzing the background of the design drawing and the reference labels of the reference elements in various dimensions. The result may be a reference label or a reference label probability distribution over individual reference labels. The setting of dimension and label can be as meticulous as possible, according to different demands, can be different. For example, if all the expansion maps only need to be animated, the dimension and the classification only need to be set on the basis of animation.

Fig. 7 schematically shows a flowchart of an extended image generation method according to an embodiment of the present application, and an execution subject of the extended image generation method may be a server, such as the server 103 shown in fig. 1.

Referring to fig. 7, the extended image generation method adds steps S710 to S740 compared to the extended image generation method in the embodiment of fig. 6, and the following is described in detail:

after reading the design element information, in step S710, identifying the design element content;

in step S720, expanding the recognized content vocabulary using word vector technology;

in step S730, identifying the design element content to form a content tag for storage;

in step S740, matching the same classification dimension from the material library, and replacing the content with the expanded vocabulary to output an expanded graph.

In the embodiment of fig. 7, more expansion diagrams with similar contents can be obtained as expansion images by using the design drawing as a reference image, using the design drawing elements as reference elements, using the content tags formed by the recognized content vocabularies as reference tags, and using the expansion vocabularies as expansion tags to expand the content vocabularies of the content dimensions.

Embodiments of the apparatus of the present application are described below, which may be used to perform the extended image generation method in the above embodiments of the present application. For details that are not disclosed in the embodiments of the apparatus of the present application, please refer to the embodiments of the extended image generation method described above in the present application.

Fig. 8 schematically shows a block diagram of an extended image generation apparatus according to an embodiment of the present application.

Referring to fig. 8, an extended image generation apparatus 800 according to an embodiment of the present application includes an obtaining module 801, a classifying module 802, a matching module 803, and a generating module 804.

According to an aspect of the embodiment of the present application, based on the foregoing solution, the obtaining module 801 is configured to obtain a reference image and obtain a reference element in the reference image; the classification module 802 is configured to perform multi-dimensional classification on the reference elements to obtain tag classification results of the reference elements, where the tag classification results are used to determine reference tag combinations of the reference elements, and the reference tag combinations include reference tags of the reference elements in multiple dimensions; the matching module 803 is configured to match a reference label combination corresponding to the label classification result with material label combinations of the plurality of material elements according to dimensions, and determine a target material element from the plurality of material elements, the matching degree of which with the reference label of the reference element meets a first predetermined matching condition; the material label combination of the material elements consists of material labels in at least one dimension; the generating module 804 is configured to generate an extended image corresponding to the reference image based on the target material element.

In an embodiment of the present application, based on the foregoing scheme, the tag classification result includes a reference tag combination of the reference element, and the matching module 803 is configured to: aiming at the material label combination of each material element, comparing the labels of the reference label combination and the material label combination in the same dimension; determining target dimensions and corresponding target dimension numbers of the material label combination and the reference label combination with the same label; the matching degree of the reference labels of the material elements and the reference elements is determined by the target dimension number; when the target dimensionality number corresponding to the material elements reaches a first preset number threshold value, judging that the matching degree of the reference labels of the material elements and the reference elements meets a first preset matching condition; and determining the material elements of which the target dimensionality quantity reaches a first preset quantity threshold value as target material elements based on the target dimensionality quantity corresponding to the material label combination of each material element in the plurality of material elements.

In an embodiment of the present application, based on the foregoing solution, the matching module 803 is further configured to: semantic expansion is carried out on each reference label in the reference label combination respectively to obtain an expanded label combination of the reference element, wherein the expanded label combination is composed of expanded labels in multiple dimensions; matching the expansion tag combination with the material tag combination according to dimensions, and determining expansion material elements of which the matching degree with the expansion tags of the reference elements meets a second preset matching condition from the plurality of material elements; the generation module 804 is configured to: and generating an expansion image corresponding to the reference image based on the target material element and the expansion material element.

In an embodiment of the present application, based on the foregoing solution, the matching module 803 is configured to: acquiring word vectors corresponding to texts of all reference labels; calculating the distance between the word vector corresponding to the text of each reference label and the word vector corresponding to the text of the candidate label; and selecting the extended labels with the distance within a set range from the candidate labels for combination to obtain an extended label combination.

In an embodiment of the application, based on the foregoing scheme, the tag classification result includes reference tag probability distributions of the reference elements in multiple dimensions, where the reference tag probability distributions include reference tag probabilities of the reference elements corresponding to multiple dimension tags in each dimension, and a reference tag is a dimension tag with a highest reference tag probability in each dimension; the matching module 803 is configured to: acquiring reference label probability distribution of a reference label combination corresponding to each dimension in a plurality of dimensions; acquiring material label probability distribution of a material label combination corresponding to each dimension in a plurality of dimensions, wherein the material label probability distribution corresponding to each dimension comprises material label probabilities of material elements corresponding to the plurality of dimension labels in each dimension, and the material label is a dimension label with the highest material label probability in each dimension; and matching the reference label probability distribution with the material label probability distribution according to the dimensionality, and determining a target material element which meets a first preset matching condition with the reference label matching degree of the reference element from the plurality of material elements.

In an embodiment of the present application, based on the foregoing scheme, the matching module 803 is configured to: calculating a first difference value of the reference label probability and the material label probability corresponding to the same dimension label in the same dimension, and determining a first label difference degree of the reference label combination and the material label combination corresponding to each label in each dimension based on the first difference value; determining a first element difference degree of a reference element and a material element based on a first label difference degree corresponding to each dimension label in each dimension, wherein the reference label matching degree of the material element and the reference element is determined by the first element difference degree; when the difference degree of a first element corresponding to the material element reaches a first difference degree threshold value, judging that the matching degree of the reference label of the material element and the reference element meets a first preset matching condition; and determining the material elements with the first element difference degree lower than a first difference degree threshold value as target material elements based on the first element difference degree corresponding to each material element in the plurality of material elements.

In an embodiment of the present application, based on the foregoing solution, the matching module 803 is configured to: summing the first label difference degrees of the plurality of dimension labels in each dimension to obtain the first dimension difference degrees of each dimension corresponding to the reference element and the material element; and summing the first dimension difference degrees of the material elements corresponding to all dimensions to serve as the first element difference degree of the reference element and the material elements.

In an embodiment of the application, based on the foregoing solution, the generating module 804 is configured to: and replacing the reference elements in the reference image with target material elements to obtain an expanded image.

As will be appreciated by one skilled in the art, aspects of the present application may be embodied as a system, method or program product. Accordingly, various aspects of the present application may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.

An electronic apparatus 90 according to this embodiment of the present application is described below with reference to fig. 9. The electronic device 90 shown in fig. 9 is only an example, and should not bring any limitation to the functions and the use range of the embodiment of the present application.

As shown in fig. 9, the electronic device 90 is in the form of a general purpose computing device. The components of the electronic device 90 may include, but are not limited to: the at least one processing unit 91, the at least one memory unit 92, a bus 93 connecting different system components (including the memory unit 92 and the processing unit 91), and a display unit 94.

Wherein the storage unit stores program code that can be executed by the processing unit 91 to cause the processing unit 91 to perform the steps according to various exemplary embodiments of the present application described in the section "methods of embodiments" mentioned above in this specification.

The storage unit 92 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM) 921 and/or a cache memory unit 922, and may further include a read only memory unit (ROM) 923.

Storage unit 92 may also include a program/utility 924 having a set (at least one) of program modules 925, such program modules 925 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Bus 93 may be one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The electronic device 90 may also communicate with one or more external devices (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 90, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 90 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 95. Also, the electronic device 90 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via a network adapter 96. As shown, the network adapter 96 communicates with the other modules of the electronic device 90 over a bus 93. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 90, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, to name a few.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, and may also be implemented by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the embodiments of the present application.

According to an embodiment of the present application, there is also provided a computer-readable storage medium having a program product stored thereon, wherein the program product is capable of implementing the above-mentioned method of the present specification. In some possible embodiments, various aspects of the present application may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the present application described in the "exemplary methods" section above of this specification, when the program product is run on the terminal device.

According to one embodiment of the present application, a program product for implementing the above method may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

Furthermore, the above-described figures are only schematic illustrations of the processes involved in the methods according to exemplary embodiments of the present application and are not intended to be limiting. It will be readily appreciated that the processes illustrated in the above figures are not intended to indicate or limit the temporal order of the processes. In addition, it is also readily understood that these processes may be performed, for example, synchronously or asynchronously in multiple modules.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. An extended image generation method, comprising:

acquiring a reference image and acquiring reference elements in the reference image;

performing multi-dimensional classification on the reference elements to obtain label classification results of the reference elements, wherein the label classification results are used for determining reference label combinations of the reference elements, and the reference label combinations comprise reference labels of the reference elements in multiple dimensions;

matching a reference label combination corresponding to the label classification result with material label combinations of a plurality of material elements according to dimensionality, and determining a target material element, of the plurality of material elements, of which the matching degree with the reference label of the reference element meets a first preset matching condition; wherein a material label combination of the material elements consists of material labels in at least one dimension;

and generating an expansion image corresponding to the reference image based on the target material element.

2. An extended image generation method as claimed in claim 1, wherein the tag classification result includes a reference tag combination of the reference element, the reference tag combination corresponding to the tag classification result is matched with material tag combinations of multiple material elements according to dimensions, and a target material element whose matching degree with the reference tag of the reference element meets a first predetermined matching condition is determined from the multiple material elements, including:

for the material label combination of each material element, comparing the labels of the reference label combination and the material label combination in the same dimension; determining the target dimensions of the material label combination and the reference label combination with the same label and the corresponding target dimension number; the matching degree of the reference labels of the material elements and the reference elements is determined by the target dimension number; when the target dimension number corresponding to the material elements reaches a first preset number threshold, judging that the matching degree of the reference labels of the material elements and the reference elements meets a first preset matching condition;

and determining the material elements of which the target dimensionality reaches a first preset quantity threshold value as the target material elements based on the target dimensionality quantity corresponding to the material label combination of each material element in the plurality of material elements.

3. An extended image generation method as claimed in claim 2, wherein before generating an extended image corresponding to the reference image based on the target material elements, the method further comprises: semantic expansion is carried out on each reference label in the reference label combination respectively to obtain an expansion label combination of the reference element, wherein the expansion label combination is composed of expansion labels in the multiple dimensions; matching the expansion tag combination with the material tag combination according to dimensions, and determining expansion material elements of which the matching degree with the expansion tags of the reference elements meets a second preset matching condition from the plurality of material elements;

the generating of the extended image corresponding to the reference image based on the target material element includes:

and generating an extended image corresponding to the reference image based on the target material element and the extended material element.

4. An extended image generation method according to claim 3, wherein the semantic extension is performed on each reference label in the reference label combination to obtain an extended label combination for representing semantic features of the reference elements, and the method includes:

obtaining word vectors corresponding to texts of all reference labels;

calculating the distance between the word vector corresponding to the text of each reference label and the word vector corresponding to the text of the candidate label;

and selecting the extended labels with the distance within a set range from the candidate labels for combination to obtain the extended label combination.

5. An expanded image generation method according to claim 1, wherein the label classification result includes a reference label probability distribution of the reference element in multiple dimensions, the reference label probability distribution includes reference label probabilities of the reference element corresponding to multiple dimension labels in each dimension, and the reference label is a dimension label with a highest reference label probability in each dimension;

the step of matching the reference label combination corresponding to the label classification result with the material label combinations of a plurality of material elements according to dimensions, and determining a target material element from the plurality of material elements, wherein the matching degree of the target material element with the reference label of the reference element meets a first preset matching condition, includes:

acquiring reference label probability distribution of the reference label combination in each dimensionality;

acquiring material label probability distribution of the material label combination corresponding to each dimension in the dimensions, wherein the material label probability distribution corresponding to each dimension comprises material label probability of the material element corresponding to the dimension labels, and the material label is a dimension label with the highest material label probability in each dimension;

and matching the reference label probability distribution with the material label probability distribution according to dimensionality, and determining a target material element, of which the matching degree with the reference label of the reference element meets the first preset matching condition, from the plurality of material elements.

6. An extended image generation method as claimed in claim 5, wherein the matching the reference label distribution probability and the material label distribution probability according to the dimension, and determining a target material element from the plurality of material elements, the matching degree of which with the reference label of the reference element meets the first predetermined matching condition, comprises:

calculating a first difference value of the reference label probability and the material label probability corresponding to the same dimension label in the same dimension, and determining a first label difference degree of the reference label combination and the material label combination corresponding to each label in each dimension based on the first difference value;

determining a first element difference degree of the reference element and the material element based on a first label difference degree corresponding to each dimension label in each dimension, wherein the reference label matching degree of the material element and the reference element is determined by the first element difference degree; when the first element difference degree corresponding to the material elements reaches a first difference degree threshold value, judging that the matching degree of the reference labels of the material elements and the reference elements meets a first preset matching condition;

and determining the material elements with the first element difference degree lower than a first difference degree threshold value as the target material elements based on the first element difference degree corresponding to each material element in the plurality of material elements.

7. An extended image generation method as claimed in claim 6, wherein determining the first element difference degree between the reference element and the material element based on the first tag difference degree corresponding to each dimension tag in each dimension comprises:

summing the first label difference degrees corresponding to the dimension labels in each dimension to obtain the first dimension difference degrees of the reference element and the material element corresponding to each dimension;

and summing the first dimension difference degrees of the material elements corresponding to all dimensions to obtain the first element difference degree of the reference element and the material elements.

8. An extended image generation method according to claim 1, wherein the generating an extended image corresponding to the reference image based on the target material element includes:

and replacing the reference elements in the reference image with the target material elements to obtain the expanded image.

9. An extended image generation apparatus, comprising:

the acquisition module is configured to acquire a reference image and acquire a reference element in the reference image;

a classification module configured to perform multi-dimensional classification on the reference element to obtain a tag classification result of the reference element, where the tag classification result is used to determine a reference tag combination of the reference element, and the reference tag combination includes reference tags of the reference element in multiple dimensions;

the matching module is configured to match a reference label combination corresponding to the label classification result with material label combinations of a plurality of material elements according to dimensions, and determine a target material element from the plurality of material elements, wherein the matching degree of the target material element with the reference label of the reference element meets a first preset matching condition; wherein a material label combination of the material elements consists of material labels in at least one dimension;

and the generating module is configured to generate an expansion image corresponding to the reference image based on the target material element.

10. An electronic device, comprising:

a memory storing computer readable instructions;

a processor to read computer readable instructions stored by the memory to perform the method of any of claims 1-8.