CN117152476A - Automatic extraction method for multi-level transformation information of design image - Google Patents

Automatic extraction method for multi-level transformation information of design image Download PDF

Info

Publication number
CN117152476A
CN117152476A CN202311186279.2A CN202311186279A CN117152476A CN 117152476 A CN117152476 A CN 117152476A CN 202311186279 A CN202311186279 A CN 202311186279A CN 117152476 A CN117152476 A CN 117152476A
Authority
CN
China
Prior art keywords
design element
basic design
elements
image
basic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311186279.2A
Other languages
Chinese (zh)
Inventor
曹力
徐宜科
张腾腾
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hefei University of Technology
Original Assignee
Hefei University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hefei University of Technology filed Critical Hefei University of Technology
Priority to CN202311186279.2A priority Critical patent/CN117152476A/en
Publication of CN117152476A publication Critical patent/CN117152476A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)

Abstract

The application discloses an automatic extraction method of multi-level transformation information of a design image, which comprises the following steps: 1, detecting basic design elements of a design image; 2, matching all the extracted basic design elements and inputting the basic design elements into a target transformation parameter regression neural network to obtain a first layer transformation parameter; 3, matching the similar composite design elements and inputting the matching elements into the same transformation parameter regression network to obtain a second layer transformation parameter; 4, carrying out vectorization treatment on the basic design element to obtain vectorization expression of the basic design element; and 5, saving the vector expression of the basic design elements and the multi-layer transformation information of the basic design elements as a parameterized description file. The application can automatically extract multi-level transformation information from the complex pattern design image so as to modify or replace design elements more efficiently, thereby generating a new layout scheme.

Description

Automatic extraction method for multi-level transformation information of design image
Technical Field
The application relates to the technology in the fields of image processing and information extraction, in particular to an automatic extraction method of multi-level transformation information and vectorization information of design elements of an image.
Background
In planar art design, transformation information in design images is rich, and for a type of design image with a large number of graphic elements multiplexing and affine transformation, transformation information based on basic graphic elements contained in the design image is an important factor for forming a design image design element. In a two-dimensional plane XOY coordinate system, normalized homogeneous coordinate expression is introduced, affine transformation information of the graphic element can be described by different 3X 3 transformation matrixes, and the technical problems existing in the field at present are as follows: 1. the processing process of the design image information in the traditional design software is less to emphasize the extraction of the image transformation information, and the structural information among the image design elements is ignored, so that the design has strong reusability, rich transformation relations and particularly strong work repeatability when the design image containing multi-level transformation relations is designed, and the design efficiency is low; 2. in the existing method for generating the new image with the similar structure by learning the implicit structure information of the image, the feature similarity of the gram matrix between the images is mostly used as an optimization target, the integrity of the appearance of the design element in the generated result is difficult to ensure, and the editing of the generated result is not facilitated; 3. based on complete basic design elements, the method for carrying out optimization solution on the layout information of the target structural image can ensure the editability of the result and the integrity of the elements, but initialization parameters such as a basic design element set, the number of element examples, the optimization times and the like are often required to be given, and the structural information of higher hierarchy possibly existing in the design image cannot be extracted.
Disclosure of Invention
Aiming at the current situation and the problems, the application provides an automatic extraction method of multi-level transformation information of a design image, so that the multi-level transformation information can be automatically extracted from a complex pattern design image, and design elements can be modified or replaced more efficiently, thereby generating a new layout scheme.
In order to achieve the aim of the application, the application adopts the following technical scheme:
the application relates to an automatic extraction method of multi-level conversion information of design elements in a design image, which is characterized by comprising the following steps:
step 1, inputting a design image I containing a plurality of basic design elements and composite design elements formed by the basic design elements into a neural network model M1 for processing, and outputting a set E= { E of various basic design elements 1 ,E 2 ,…,E i ,…,E n E, where E i Represents a set of i-th basic design elements, n represents a total number of categories of basic design elements, andwherein (1)>Representing an S-th basic design element instance in the i-th basic design element set, wherein S represents the number of basic design element instances in the i-th basic design element set; and is also provided withWherein i represents an instance->Category index number,/->Is a basic design element example->In the design image IIs a position rectangular frame; />Is a basic design element example->A segmentation mask image in the design image I, wherein a black background in the segmentation mask image is characterized by a binary code of "0", and a white foreground in the segmentation mask image is characterized by a binary code of "1";
step 2, according to the position rectangle frame pair E of each basic design element example in the i-th class i Clustering to form a cluster of basic design element examples with similar spatial distribution, and combining basic design element examples under the same cluster into a composite design element to obtain an i-th composite design element set
Wherein,representing the kth composite design element in the ith class; k represents the number of clusters; and-> Wherein (1)>Represents the kth composite design element->V of the V-th basic design element instance, V k Representing the number of instances in the kth cluster; thereby clustering the basic design element examples of all classes and obtaining +>
Step 3, according toSimilarity between the composite design elements of +.>Performing secondary classification to obtain class i secondary classification set +.>Wherein F is i,u Representing the result of the ith secondary classification in the ith composite design element set, U i Representation->The number of categories of the secondary classification of (a); and->Wherein (1)>Is F i,u W is the W-th secondary composite design element, W u Is F i,u The number of secondary composite design elements;
step 4, collecting the set F i,u The first two-time composite design elementAnd according to the position rectangle box pair of each basic design element instance in the i-th class +.>The contained basic design element instance set +.>Matching and combining to obtain a combined image pair set +.>Wherein,the representation comprises->1 st basic design element example +.>And->X-th basic design element instance +.>Is a pair of images of (a);
will beInputting into a regression neural network model M2 for processing to obtain a first composite design element +.>Corresponding transformation parameter set-> Wherein T1 i,u [x]Representation->Corresponding transformation parameters;
step 5, according to the process of step 4, G i Processing the middle U secondary classification results to obtain G i Is a first layer transform parameter set of (1)Thereby obtaining a first layer transformation parameter total set T1= { T1 1 ,T1 2 ,…,T1 i ,…,T1 n }；
Step 6, according to the position rectangle frame of each basic design element example in the i-th class, for F i,u Each secondary composite design element in the image is matched and combined to obtain an image pair set after the composite design elements are combinedWherein (1)>Representing and including F i,u The 1 st secondary composite design element +.>And w-th secondary composite design element +.>Is a pair of images of (a);
will beInputting into the regression neural network model M2 for processing, outputting F i,u Corresponding transformation parameter set T2 i,u ＝{T2 i,u [1],T2 i,u [2],…,T2 i,u [w],…,T2 i,u [W u ]}, wherein T2 i,u [w]Representation->Corresponding transformation parameters;
step 7, according to the process of step 6, G i Processing the middle U secondary classification results to obtain G i Is a second layer transform parameter set of (2)Thereby obtaining a second layer transformation parameter total set T2= { T2 1 ,T2 2 ,…,T2 i ,…,T2 n }；
Step 8, slave e= { E 1 ,E 2 ,…,E i ,…,E n Sequentially extracting 1 st basic design element from various basic design element sets and forming an image setWherein (1)>Representing the 1 st basic design element in the i-th basic design element set;
for a pair ofVectorizing each basic design element in the set to obtain a vector information set S= { S 1 ,S 2 ,…,S i ,…,S n S, where S i Representation->Vector information of (2);
and 9, saving the vector information set S and the vector information sets T1 and T2 as parameterized description files for subsequent modification editing or rendering operation.
The electronic device of the application comprises a memory and a processor, characterized in that the memory is used for storing a program for supporting the processor to execute the automatic extraction method, and the processor is configured for executing the program stored in the memory.
The application relates to a computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, performs the steps of the automatic extraction method.
Compared with the prior art, the application has the beneficial effects that:
1. the application utilizes an example detection and segmentation method to extract basic design element information in a design image, wherein the basic design element information comprises categories, a space bounding box and a binarization mask of the basic design element, then the basic design element information is used for extracting inter-element transformation information, and the basic design element is vectorized to obtain a reconstruction result for structuring the original design image information. And the extracted structural reconstruction information is utilized, so that the repetitive labor in the process of designing the image by the multiple multiplexing elements is reduced.
2. According to the application, through detection of basic design element information, clustering division is directly carried out on individual basic design element examples, and explicit transformation information extraction is carried out through a transformation information prediction network, so that compared with a generation model based on an optimized gram feature matrix, the method can ensure the integrity of basic design elements in a result, and is more beneficial to editing operations such as modification of explicit transformation information and the like.
3. According to the application, basic design element information is automatically extracted through the detection model, clustering division operation is carried out on the basic design element information for a plurality of times, and final structural expression of the design image is obtained through carrying out transformation information extraction on the result of each clustering division.
Drawings
FIG. 1 is a process flow diagram corresponding to the method of the present application;
FIG. 2 is a flow chart corresponding to the method of the present application;
FIG. 3 is a training flow chart of the element transformation information regression model designed by the method of the application;
fig. 4 is a schematic diagram of a multi-level transformation information extraction process according to an embodiment of the present application.
Detailed Description
In this embodiment, an automatic extraction method of multi-level transformation information of design elements mainly includes the following steps:
s1) detecting basic design elements of a design image, processing to obtain category, spatial position and mask information of all elements, carrying out cluster analysis on the basic design elements of the same category according to spatial density, and forming a composite design element by the basic design elements of the same category under each density cluster;
in the step S1), the inter-space clustering analysis method is a density clustering algorithm which takes the central coordinate value of the bounding box of the same basic design element as a clustering object, and the spatial clustering division of the basic design element is obtained by setting a clustering threshold.
S2) carrying out similarity judgment processing on the composite design elements obtained in the step S1), carrying out secondary clustering according to similarity indexes calculated between the composite design elements on the basis of traditional feature descriptors or high-level semantic features, taking a first composite design element instance under each cluster as a transformed reference object, then carrying out matching combination on basic design element instances contained in the reference object, inputting the combined image into a transformation parameter regression neural network model, and outputting corresponding first-layer transformation parameters between each group of matched images by the transformation parameter regression neural network model;
in step S2), the transformation parameter regression neural network predicts parameters of a transformation matrix corresponding between a pair of design elements contained in the image based on a residual convolution network structure of a channel attention mechanism.
S3) matching and combining the similar composite design elements obtained by clustering in the step S2), inputting the composite design element combined image into the same transformation parameter regression neural network model in the step S2), and outputting a second layer of transformation parameters corresponding to each group of matched images by the transformation parameter regression neural network model;
s4) carrying out automatic vectorization on each type of basic design element image output by the detection model in the step S1) to obtain vectorization expression of each type of basic design element, and storing vector parameters in a certain structural format to be used as reusable vectorization basic design element information.
And S5) finally, storing the vectorization expression of the basic design elements obtained in the step S4) and the multilevel transformation information obtained in the output of the steps S2) and S3) as parameterization description information for generating and editing operations in the design of the vector pattern.
The final parameterized description information takes json text language as a carrier, and the json file contains vector information of basic design elements and corresponding multi-level transformation information, so that efficient modification and replacement of the design elements in the parameterized description file are supported, and a new design image is generated.
The key flow in the present application will be described in more detail with reference to the accompanying drawings and specific embodiments.
Step 1: detection of basic design elements. In the basic design element detection flow of fig. 1, the detection process of extracting basic design elements is illustrated by a detection model based on a neural network, and the detection step can select different detection algorithms according to different application scenarios. In this embodiment, the basic design element detection neural network model adopts a target detection and segmentation model with a Mask R-CNN structure, which adds a deconvolution integral branch network to the fast R-CNN structure to predict the target object Mask in each candidate region, and introduces the RoI alignment constraint to perform bilinear interpolation on the feature grid to reduce the prediction error of the Mask.
Step 1.1: the application acquires the data set of the basic design element image through network retrieval and manual production, and marks the basic design element in the data set by using a bounding box and a mask. The neural network model is subjected to fine adjustment on the pre-training model through basic design element image data so as to improve the detection effect and accuracy of the neural network model.
Step 1.2: in the training process of the detection neural network model, the data set is divided into a training set and a testing set, and the training set and the testing set are respectively used for updating model parameters and evaluating model performance. Through repeated iterative training, the errors on the training set and the testing set reach the expected level, and therefore a usable basic design element detection model M1 is obtained.
Step 1.3: in the basic design element detection implementation stage, first, a design image I is input into a basic design element detection model M1 to perform basic design element detection. Model M1 consists of three parts: a Backbone Network (Backbone), a regional proposal Network (Region Proposal Network), and a Head Network (Head Network). Design image I firstThe characteristics of the I under multiple resolutions are obtained through the processing of a characteristic pyramid backbone network ResNet-FPN based on a residual error structure of M1; then the regional suggestion network in M1 processes the characteristic information to obtain candidate regions of the characteristics; finally, the candidate regions of the characteristics are processed through the head network in M1, and basic design elements corresponding to the candidate regions are obtained through outputPosition rectangle box in input design image I +.>And instance partition mask->I.e. < ->Note the basic design element set e= { E for all classes 1 ,E 2 ,…,E i ,…,E n E, where E i Represents a set of i-th basic design elements, n represents the total number of categories of basic design elements, and +.>Wherein (1)>Representing the S-th basic design element instance in the i-th basic design element set, and S represents the number of basic design element instances in the i-th basic design element set.
Step 2: detecting all basic design element information E= { E output by the segmentation model for basic design elements 1 ,E 2 ,…,E i ,…,E n Basic design elements of the same classAnd performing density clustering according to the space distance between the elements. Spatial distance density clustering methodFor basic design element->Central coordinate value->And (3) carrying out clustering operation, wherein the clustering process examines the connectivity of element distribution from the spatial distribution density of the basic design elements, and continuously expands the clustering clusters according to the connectivity to obtain final basic design element set division.
Step 2.1: defining the distance between basic design elements with index i as the distance of the element edges
If the neighborhood threshold of the element is E, any elementIs defined as a set of:
step 2.2: the maximum set formed by the neighborhood relation among the elements forms a density cluster, and the cluster is recorded asWhere i is the basic design element class index and k represents the kth set of partitions. The dashed boxes in fig. 4 are the results of density clustering of basic design elements, each density clustering result corresponds to one composite design element, and each dashed box in the figure contains 3 same-class basic design elements.
Step 3: first layer transform information solution between basic design elements
Step 3.1: for a composite design elementBasic design element set in corresponding partition setFirst selecting the first basic design element instance +.>As a reference object of transformation, matching the basic design element pair with the rest design elements in the divided set to obtain a basic design element pair combined image under the divided set, and marking the basic design element pair as a tuple->
Step 3.2: the regression neural network needed for solving the transformation parameters is M2, and the backbone part of the regression neural network is based on a convolution residual network taking ResNet18 as a backbone.
Step 3.2.1: the input of the network is a combined image of the transformation information to be extractedThe dimension is w×h×c, as shown in fig. 2, the M2 network includes a convolutional pooling layer, 3 residual network blocks, and a full-connection layer, where the convolutional kernel size is kernel_size=5×5, and the step size stride=2.
Step 3.2.2: in order to more effectively utilize the feature graphs of different channels and improve the prediction accuracy, a feature channel attention mechanism SE is introduced into the network, wherein the feature channel attention mechanism SE is a module for enhancing the dependence relationship among channels in the convolutional neural network, and the importance of each channel can be adaptively adjusted, so that the image recognition performance is improved. The SE module achieves channel attention in two steps:
squeeze: using global average pooling layer F Average pool (. About.) spatial information to be entered into a feature mapCompression to a channel descriptionSymbol, i.e. a one-dimensional vector +.>The vector contains global information for each channel.
The specification: generating a channel weight vector using two full connection layers and a Sigmoid activation function with channel descriptors as inputsThe vector represents the importance of each channel and is then multiplied by the input feature map to obtain a weighted feature map +.>
Step 3.2.3: fig. 3 is a flowchart of a training step of the transformation information regression network M2, where training data of the transformation information regression network may be obtained by randomly generating affine transformation images between two similar basic design elements, recording transformation parameters applied randomly as labels of the training images, scaling the dimensions of the training images and the transformation parameter information to w×h×c, and training the target parameter regression network to obtain a usable transformation parameter regression model.
Step 3.3: tuple(s)The transformation between instances of design elements in (a) can be thought of as the superposition of a series of affine transformations, in a Cartesian two-dimensional coordinate system, a basic affine transformation being one transformation from one two-dimensional coordinate system to another, including scaling, translation, rotation, reflection, miscut, etc. The affine transformation is characterized in that the straightness and the parallelism of the straight lines are maintained, namely the straight lines before and after transformation are still straight lines, and the original parallel straight lines are still parallel.
Affine transformation of two-dimensional coordinates can be represented by a 3 x 3 matrix, i.e
Where x1, x2, tx, y1, y2, and ty are six free parameters that determine the type and degree of transformation. The multiple superposition of affine transformation covers the majority transformation between the target design elements, the superposition transformation matrix of affine transformation can be obtained by the left multiplication operation of a plurality of 3×3 matrixes, the result is still a 3×3 matrix, the full connection layer of the regression network outputs the predicted parameters (x 1, x2, tx, y1, y2, ty) of the 6 transformation matrixes, and the parameters are matched with the fixed matrix parameters [0,1]Constituent design elementsAndthe transformation matrix between them is denoted as T1 i,k [v]。
Step 3.4: as shown in fig. 1 and fig. 2, the matched combined image is input into a target transformation parameter regression model M2, a transformation matrix set among basic design elements inside all composite design elements is solved, and the transformation information of the first layer is recorded as t1= { T1 1 ,T1 2 ,…,T1 i ,…,T1 n }, whereinT1 i,u ＝{T1 i,u [1],T1 i,u [2],…,T1 i,u [x],…,T1 i,u [X u ]}. The middle part of fig. 4 illustrates a process of obtaining two combined images by matching one basic design element in one composite design element as a reference object with other basic design elements in the composite design element, and solving corresponding first-layer transformation information through a transformation parameter regression network M2.
Step 4: composite design element similarity partitioning and second layer transformation information solving
Step 4.1: composite design element set formed by i-th basic design elements The composite design elements in the set still have certain possibility to be similar in vision, and the similarity index threshold value tau is set for +.>The composite design elements in the model (3) are subjected to secondary classification, and similarity discrimination indexes similarity of the composite design elements can be based on image feature descriptions such as scale invariant feature SIFT and acceleration robust feature SURF as comparison standards. Wherein SIFT features respond to and calculate local maxima in image space and scale space by a laplace filter; the SURF feature is a simplified version of SIFT that calculates the hessian matrix ++for each pixel at different scales by a simplified laplace filter>Judging whether the characteristic points are characteristic points or not according to determinant values of the hessian matrix; the similarity index between the composite design elements can also be used for extracting the visual information feature vectors c of the two composite design elements through the convolutional neural network 1 ,c 2 By outputting cosine values between eigenvectors +.>And judging the similarity between the composite design elements as a reference standard for similarity comparison of the targets. Similarity index similarity value is [0,1]Satisfying the similarity>The composite design element of τ is determined to be a similar composite design element, +.>The similar composite design elements in the model are formed into a new partition which is denoted as G i ＝{F i,1 ,F i,2 ,…,F i,u ,…,F i,U }, whereinIs->U is the set +.>Dividing the subset F i,u Composite design element->The relationship may also be described by a determined transformation relationship.
Step 4.3: with detected sub-sets of divisionsFirst song composite design element instance +.>For the transformed reference elements, in turn with the set F i,u The composite design elements in (a) are matched into a combined image, and marked as a tuple +.>Combining the composite design elements into an image->As an input to the transformation parameter recurrent neural network M2, as shown in fig. 1 and 2, the output is processed via model M2 to obtain +.>Corresponding transformation parameter matrix T2 i,u [w]Sequentially processing combined images in the secondary classification sets in all the classes, solving a transformation matrix among composite design elements under n basic element classes, and taking the transformation matrix as a second-layer transformation matrix and marking as T2= { T2 1 ,T2 2 ,…,T2 i ,…,T2 n }, wherein-> T2 i,u ＝{T2 i,u [1],T2 i,u [2],…,T2 i,u [w],…,T2 i,u [W u ]}. The final part of fig. 4 illustrates the process of matching one of the composite design elements in the same division with the other composite design elements in the division to obtain a combined image, and processing the combined image by M2 to obtain corresponding second layer transformation information.
Step 5: vectorizing basic design elements
As shown in the flow chart of fig. 2, the output of the basic design element detection stepAs an input of an image vectorization algorithm, a vectorization result S= { S of the basic design element bitmap is obtained through processing 1 ,S 2 ,…,S i ,…,S n }. The basic processing steps of the vectorization algorithm are as follows:
step 5.1: bitmaps of basic design elementsThe paths are decomposed by edge detection operator Canny or k-means clustering, which form boundaries between black and white areas. By moving along the pixel edge and according to a steering policy, it is decided whether to turn left or right at the corner. Each time a closed path is found, it is removed from the bitmap and the next path is continued to be found until no black pixels remain.
Step 5.2: each path is approximated as an optimal polygon. This step is implemented by a dynamic programming algorithm that finds a polygon approximation that minimizes the error in polynomial time. The specific process of the algorithm is as follows:
step 5.2.1: defining an error functionTo approximate the error in a sub-sequence from point i to point j in the path with a straight line segment, the error is measured as the sum of the distances from all points in the sub-sequence to the straight line segment, where j→i +1 represents the number of points through which point i to point j in the path passes,representing point v k To straight line->Euclidean distance of (c).
Step 5.2.2: the optimum function OPT (i, j) is defined as the minimum error resulting from approximating the subsequence from the i-th point to the j-th point on the path with a polygon. This minimum error can be solved with dynamic programming, namely:
OPT(i,j)＝min{Error(i,k)+OPT(k,j)}
where i < k < j. A Global optimum function Global (n, m) is defined as the minimum error resulting from approximating n points on a path with m straight line segments. This minimum error can also be solved with dynamic programming, namely:
Global(n,m)＝min{Global(k,m-1)+Error(k,n)}
wherein 1 is<k<n. By the three functions, we can obtain the basic design elementOptimal polygonal approximation of all points on the contour path +.>
Step 5.3: taking a polygonThe vertex is the end point of the third-order Bezier curve, the control point calculation and optimization are carried out on the polygon, and a section of sequence segment with the outline as the point is recorded ij ＝[p i ,p i+1 ,…,p j ]From the Bernstant function of the third-order Bezier curve>Wherein t is E [0,1 ]]，p t0 、p t3 Representing a segment of a contour ij Endpoint, pt of (1) 1 ，pt 2 For the three-order Bezier curve control point coordinates to be solved, t=0, 1/3,2/3,1 are taken respectively, and let B (t) =segment ij [i+t×(j-i)]Written in matrix form +.> To obtain->Thereby making a polygonConversion to a smooth contour s= { S 1 ,S 2 ,…,S i ,…,S n }。
Step 5.5: finally, vector curve information { S 1 ,S 2 ,…,S i ,…,S n To SVG files.
Step 6: outputting vector information and multilevel transformation information of basic design elements;
the first layer transformation parameters T1 among the basic design elements, the second layer transformation parameters T2 among the composite design elements and the vector information S of the basic design elements are respectively obtained through the steps 1-5, the information extracted by the data record form organization is defined, and the original design image is described again.
The following is a json text description example of the design image containing multi-level transformation information by the extraction method:
the structured description of the design image is in the form of: designing basic information project-properties of an image, wherein the basic information project-properties comprise basic attributes such as width, height, background color and the like; the elements node contains all the extracted basic design elements and composite design element template information, wherein the data attribute is a reference position index of the design elements; all elements in the image record the reference information of template elements in the set elements based on the transformation information, and each referenced transformation attribute record corresponds to transformation matrix information of different layers; the root-layer contains all the composite design element nodes, each of which contains a reference to the basic design element in the elements node in its own property child.
In this embodiment, an electronic device includes a memory for storing a program supporting the processor to execute the above method, and a processor configured to execute the program stored in the memory.
In this embodiment, a computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the method described above.

Claims (3)

1. An automatic extraction method for multi-level transformation information of a design image is characterized by comprising the following steps:
step 1, inputting a design image I containing a plurality of basic design elements and composite design elements formed by the basic design elements into a neural network model M1 for processing, and outputting a set E= { E of various basic design elements 1 ,E 2 ,…,E i ,…,E n E, where E i Represents a set of i-th basic design elements, n represents a total number of categories of basic design elements, andwherein (1)>Representing an S-th basic design element instance in the i-th basic design element set, wherein S represents the number of basic design element instances in the i-th basic design element set; and->Wherein i represents an instance->Category index number,/->Is a basic design element example->A positional rectangular frame in the design image I; />Is a basic design element example->A segmentation mask image in the design image I, wherein a black background in the segmentation mask image is characterized by a binary code of "0", and a white foreground in the segmentation mask image is characterized by a binary code of "1";
step 2, according to the position rectangle frame pair F of each basic design element example in the i-th class i Clustering to form a cluster of basic design element examples with similar spatial distribution, and combining basic design element examples under the same cluster into a composite design element to obtain an i-th composite design element setWherein (1)>Representing the kth composite design element in the ith class; k represents the number of clusters; and-> Wherein (1)>Represents the kth composite design element->V of the V-th basic design element instance, V k Representing the number of instances in the kth cluster; thereby clustering the basic design element examples of all classes and obtaining +>
Step 3, according toSimilarity between the composite design elements of +.>Performing secondary classification to obtain class i secondary classification set +.>Wherein F is i,u Representing the result of the ith secondary classification in the ith composite design element set, U i Representation->The number of categories of the secondary classification of (a);and->Wherein (1)>Is F i,u W is the W-th secondary composite design element, W u Is F i,u The number of secondary composite design elements;
step 4, collecting the set F i,u The first two-time composite design elementAnd according to the position rectangle box pair of each basic design element instance in the i-th class +.>The contained basic design element instance set +.>Matching and combining to obtain a combined image pair set +.>Wherein (1)>The representation comprises->1 st basic design element example +.>And->X-th basic design element instance +.>Is a pair of images of (a);
will beInputting into a regression neural network model M2 for processing to obtain a first composite design element +.>Corresponding transformation parameter set T1 i,u ＝{T1 i,u [1],T1 i,u [2],…,T1 i,u [x],…,T1 i,u [X u ]-a }; wherein T1 i,u [x]Representation->Corresponding transformation parameters;
step 5, according to the process of step 4, G i Processing the middle U secondary classification results to obtain G i Is a first layer transform parameter set of (1)Thereby obtaining a first layer transformation parameter total set T1= { T1 1 ,T1 2 ,…,T1 i ,…,T1 n }；
Step 6, according to the position rectangle frame of each basic design element example in the i-th class, for F i,u Each secondary composite design element in the image is matched and combined to obtain an image pair set after the composite design elements are combinedWherein (1)>Representing and including F i,u The 1 st secondary composite design element +.>And w-th secondary composite design element +.>Is a pair of images of (a);
will beInputting into the regression neural network model M2 for processing, outputting F i,u Corresponding transformation parameter set T2 i,u ＝{T2 i,u [1],T2 i,u [2],…,T2 i,u [w],…,T2 i,u [w u ]}, wherein T2 i,u [w]Representation->Corresponding transformation parameters;
step 7, according to the process of step 6, G i Processing the middle U secondary classification results to obtain G i Is a second layer transform parameter set of (2)Thereby obtaining a second layer transformation parameter total set T2= { T2 1 ,T2 2 ,…,T2 i ,…,T2 n }；
Step 8, slave e= { E 1 ,E 2 ,…,E i ,…,E n Sequentially extracting 1 st basic design element from various basic design element sets and forming an image setWherein (1)>Representing the 1 st basic design element in the i-th basic design element set;
for a pair ofVectorizing each basic design element in the set to obtain a vector information set S= { S 1 ,S 2 ,…,S i ,…,S n S, where S i Representation->Vector information of (2);
and 9, saving the vector information set S and the vector information sets T1 and T2 as parameterized description files for subsequent modification editing or rendering operation.
2. An electronic device comprising a memory and a processor, wherein the memory is configured to store a program that supports the processor to perform the automatic extraction method of claim 1, the processor being configured to execute the program stored in the memory.
3. A computer readable storage medium having a computer program stored thereon, characterized in that the computer program when run by a processor performs the steps of the automatic extraction method of claim 1.
CN202311186279.2A 2023-09-14 2023-09-14 Automatic extraction method for multi-level transformation information of design image Pending CN117152476A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311186279.2A CN117152476A (en) 2023-09-14 2023-09-14 Automatic extraction method for multi-level transformation information of design image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311186279.2A CN117152476A (en) 2023-09-14 2023-09-14 Automatic extraction method for multi-level transformation information of design image

Publications (1)

Publication Number Publication Date
CN117152476A true CN117152476A (en) 2023-12-01

Family

ID=88911884

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311186279.2A Pending CN117152476A (en) 2023-09-14 2023-09-14 Automatic extraction method for multi-level transformation information of design image

Country Status (1)

Country Link
CN (1) CN117152476A (en)

Similar Documents

Publication Publication Date Title
CN110532920B (en) Face recognition method for small-quantity data set based on FaceNet method
Ioannidou et al. Deep learning advances in computer vision with 3d data: A survey
Bouti et al. A robust system for road sign detection and classification using LeNet architecture based on convolutional neural network
CN111191583B (en) Space target recognition system and method based on convolutional neural network
Li et al. Multi-scale neighborhood feature extraction and aggregation for point cloud segmentation
CN111178251B (en) Pedestrian attribute identification method and system, storage medium and terminal
CN111127631B (en) Three-dimensional shape and texture reconstruction method, system and storage medium based on single image
WO2014205231A1 (en) Deep learning framework for generic object detection
Lu et al. Traffic sign recognition via multi-modal tree-structure embedded multi-task learning
Ahmad et al. 3D capsule networks for object classification from 3D model data
CN113449784A (en) Image multi-classification method, device, equipment and medium based on prior attribute map
Khan et al. A comparative study of graphic symbol recognition methods
Bello et al. FFPointNet: Local and global fused feature for 3D point clouds analysis
Singh et al. CNN based approach for traffic sign recognition system
Hoang et al. Pixel-level clustering network for unsupervised image segmentation
Luo et al. Piecewise linear regression-based single image super-resolution via Hadamard transform
CN117058554A (en) Power equipment target detection method, model training method and device
CN116778339A (en) Method and system for selecting hyperspectral wave bands by aid of local view auxiliary discrimination
CN116597275A (en) High-speed moving target recognition method based on data enhancement
US20230073223A1 (en) Method for detecting anomalies in images using a plurality of machine learning programs
CN117152476A (en) Automatic extraction method for multi-level transformation information of design image
Strassburg et al. On the influence of superpixel methods for image parsing
Mujtaba et al. Automatic solar panel detection from high-resolution orthoimagery using deep learning segmentation networks
CN111627033A (en) Hard sample instance segmentation method and device and computer readable storage medium
Roy et al. Recognizing Bangla Handwritten Numerals: A Hybrid Model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination