CN113505866B - Image analysis method and device based on edge material data enhancement - Google Patents

Image analysis method and device based on edge material data enhancement Download PDF

Info

Publication number
CN113505866B
CN113505866B CN202111065726.XA CN202111065726A CN113505866B CN 113505866 B CN113505866 B CN 113505866B CN 202111065726 A CN202111065726 A CN 202111065726A CN 113505866 B CN113505866 B CN 113505866B
Authority
CN
China
Prior art keywords
edge
image
edge material
original image
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111065726.XA
Other languages
Chinese (zh)
Other versions
CN113505866A (en
Inventor
蔡聪怀
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202111065726.XA priority Critical patent/CN113505866B/en
Publication of CN113505866A publication Critical patent/CN113505866A/en
Application granted granted Critical
Publication of CN113505866B publication Critical patent/CN113505866B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The application discloses an image analysis method and device based on edge material data enhancement, wherein the method comprises the following steps: the method comprises the steps of inputting original images into a target model subjected to edge material data enhancement training to obtain analysis results corresponding to the original images, wherein the target images are obtained by enhancing edge material data of first original images in a training set, the target model is trained to obtain a target model subjected to edge material data enhancement training, the target images are obtained by splicing edge materials in an edge material library and the first original images in the training set, the edge materials are obtained by performing edge detection and identification on second original images, the second original images are any images, the target model can be trained by using the image data enhanced by the edge materials, the edge editing resistance of the target model is effectively improved, and the analysis capability of the target model on the image data and the video data is further improved.

Description

Image analysis method and device based on edge material data enhancement
Technical Field
The present disclosure relates generally to the field of image processing technologies, and in particular, to the field of artificial intelligence technologies, and in particular, to an image analysis method and apparatus based on edge material data enhancement.
Background
LeNet-5 model training used data enhancement methods for the first time, and in the following years, data enhancement was widely applied to model training to improve the effect of models. In the related art, the data enhancement method generally adopted mainly includes: geometric transformations, color space conversion, and random erasure. However, with the development of internet technology, the creator of the network video often adds various kinds of edge material information to the content picture, and the traditional data enhancement method cannot produce a good effect on the content picture.
Disclosure of Invention
In view of the foregoing defects or shortcomings in the prior art, it is desirable to provide an image analysis method and apparatus based on edge material data enhancement, which train a target model using image data enhanced by edge material data and effectively improve the edge editing resistance of the target model.
In a first aspect, an embodiment of the present application provides an image analysis method based on edge material data enhancement, including:
acquiring an original image to be analyzed;
inputting the original image into a target model subjected to edge material data enhancement training to obtain an analysis result corresponding to the original image, wherein the target model is trained by utilizing a target image obtained by enhancing edge material data of a first original image in a training set to obtain the target model subjected to the edge material data enhancement training, the target image is obtained by splicing an edge material in an edge material library with the first original image in the training set, the edge material is obtained by performing edge detection and identification on a second original image, and the second original image is an arbitrary image.
In a second aspect, an embodiment of the present application provides an image analysis apparatus based on edge material data enhancement, including:
the acquisition module is used for acquiring an original image to be analyzed;
the analysis module is used for inputting the original images into a target model subjected to edge material data enhancement training to obtain analysis results corresponding to the original images, wherein the target model is trained by utilizing a target image obtained by enhancing edge material data of a first original image in a training set to obtain the target model subjected to edge material data enhancement training, the target image is obtained by splicing edge materials in an edge material library with the first original image in the training set, the edge materials are obtained by performing edge detection and identification on a second original image, and the second original image is any image.
In some embodiments, the training of the target model by using the target image obtained by enhancing the edge material data of the first original image in the training set based on the image analysis device enhanced by the edge material data includes:
the first acquisition module is used for acquiring a first original image in a training set;
the random module is used for selecting a plurality of edge materials from the edge material library according to a preset rule;
the splicing module is used for splicing the edge materials to the first original image respectively to obtain a target image for model training;
and the training module is used for training a target model according to the target image.
In some embodiments, the image analysis apparatus enhanced based on the edge material data further includes:
the second acquisition module is used for acquiring the second original image;
the extraction module is used for extracting first image characteristics of the second original image;
the detection module is used for inputting the first image characteristic into an edge detection model to obtain texture information and corresponding parameter values contained in an edge area in the first image characteristic;
and the storage module is used for storing the edge area with the parameter value meeting the preset condition into the edge material library as the edge material.
In some embodiments, the edge detection model includes a target detection task branch and an edge detection task branch, the parameter values include a confidence value and an edge coordinate value, and the edge coordinate value is a vertex coordinate value corresponding to the edge region, and the detection module is further configured to:
sending the first image feature to the target detection task branch to obtain the confidence value and the edge coordinate value corresponding to the edge region in the first image feature; and
and sending the first image feature to the edge detection task branch to obtain texture information of the edge area in the first image feature.
In some embodiments, the target detection task branch includes a candidate area selection network and two full connectivity layer branches, and the detection module is further configured to:
inputting the first image characteristic into the candidate area selection network to obtain a candidate edge area;
inputting the candidate edge area to a first full-connection layer branch to obtain a confidence value corresponding to the candidate edge area;
and inputting the candidate edge region into a second full-connection layer branch to obtain an edge coordinate value corresponding to the candidate edge region.
In some embodiments, the parameter values include a confidence value and an edge coordinate value, the preset condition is a confidence condition, and the storage module is further configured to:
and storing the edge area with the confidence value larger than a preset threshold value into the edge material library as the edge material.
In some embodiments, the preset rule includes randomly selecting the edge material, and the random module is further configured to:
and randomly selecting a plurality of edge materials from the edge material library.
In some embodiments, the splicing module is further configured to:
adjusting the size of the first original image according to the edge coordinate value corresponding to each edge material to obtain an intermediate image corresponding to each edge material;
splicing the edge material to the intermediate image corresponding to the edge material to obtain the target image; or
Aiming at each edge material, adjusting the size of the edge material according to the edge coordinate value of the first original image to obtain a target edge material;
and splicing the target edge material to the first original image to obtain the target image.
In some embodiments, the preset rule includes selecting the edge material according to the size of the first original image, and the random module is further configured to:
and selecting a plurality of edge materials, the sizes of main content areas corresponding to the edge materials of which are consistent with the size of the first original image, from the edge material library, wherein the main content areas are areas surrounded by edge coordinate values of the edge materials.
In some embodiments, the edge material comprises the edge material without edges.
In a third aspect, embodiments of the present application provide an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the computer program to implement the method described in the embodiments of the present application.
In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the method as described in the embodiments of the present application.
According to the image analysis method based on the edge material data enhancement, the target model obtained by training the target image containing the edge material information is used for analyzing the original image to be analyzed, the editing resistance of the target model to the edge material generated in the training process can be effectively utilized, and the analysis accuracy of the target model to the original image to be analyzed is improved.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
fig. 1 is a schematic diagram of an exemplary system architecture of an image analysis method based on edge material data enhancement according to an embodiment of the present application;
fig. 2 is a flowchart of an image analysis method based on edge material data enhancement according to an embodiment of the present disclosure;
fig. 3 is a flowchart of another image analysis method based on edge material data enhancement according to an embodiment of the present application;
fig. 4 is a schematic diagram of an image analysis method based on edge material data enhancement according to an embodiment of the present application;
FIG. 5 is an example of an object model detecting an object according to an embodiment of the present disclosure;
fig. 6 is a flowchart of another image analysis method based on edge material data enhancement according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an edge detection model according to an embodiment of the present disclosure;
FIG. 8 is an example of two types of edge material proposed in an embodiment of the present application;
FIG. 9 is a schematic diagram of an image analysis method based on edge material data enhancement according to an embodiment of the present application;
fig. 10 is a block diagram of an image analysis apparatus based on edge material data enhancement according to an embodiment of the present application;
fig. 11 is a schematic block diagram of another image analysis apparatus based on edge material data enhancement according to an embodiment of the present application;
fig. 12 shows a schematic structural diagram of a computer system suitable for implementing the electronic device or the server according to the embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
For a clearer description of the present application, the following are explanations of terms of related art:
edge material: broadly refers to black and white borders, frosted glass, solid colored borders, etc., that are added around a region of subject content in an image or video.
Feature vector: vectors representing images or videos, images or videos with close vision, and distance of feature vectors are also close.
Data enhancement: the method is a training means for improving the generalization ability of the model.
In the related art, a commonly used data enhancement method mainly includes: three kinds of geometric transformation, color space conversion and random erasure. The geometric transformation is to perform operations such as horizontal turning, vertical turning, random region clipping, image rotation, image translation and the like on the image, and the operations effectively ensure the invariance of the model to the geometric spatial change of the image. The color space enhancement mainly adopts the mode of randomly transforming the tone, the brightness, the transparency, the color channel and the like of the image, so that the color deviation in the data can be eliminated during model training. Random erasure is the random selection of a sub-region of an image, filled with 0 or 255, averaged pixel values, or random values, which ensures that the model focuses on the entire image region and not just a portion thereof.
However, with the support of the internet, the image changes are more diverse, many content creators, such as video numbers, often add edge material information to a content picture, however, the traditional data enhancement method does not consider such a possible data situation, and thus the model cannot obtain the edge resistance of the image through conventional data enhancement.
Based on the above, the application provides an image analysis method and device based on edge material data enhancement.
The image analysis method based on edge material data enhancement provided by the embodiment of the application can be used for equipment with image recognition capability, for example, the image analysis method can be independently executed through the terminal equipment or the server, and can also be applied to a network scene of communication between the terminal equipment and the server and executed through cooperation between the terminal equipment and the server. The terminal device can be a smart phone, a notebook computer, a desktop computer, a personal digital assistant, a tablet computer and the like. The server may be understood as an application server or a Web server, and when actually deployed, the server may be an independent server or a cluster server.
Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.
The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.
Computer Vision technology (CV), which is a science for researching how to make a machine "see", and further refers to using a camera and a Computer to replace human eyes to perform machine Vision such as identification, tracking and measurement on a target, and further performing image processing, so that the Computer processing becomes an image more suitable for human eyes to observe or transmitted to an instrument to detect. As a scientific discipline, computer vision research-related theories and techniques attempt to build artificial intelligence systems that can capture information from images or multidimensional data. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D technologies, virtual reality, augmented reality, synchronous positioning, map construction, and other technologies, and also include common biometric technologies such as face recognition and fingerprint recognition.
Key technologies for Speech Technology (Speech Technology) are automatic Speech recognition Technology (ASR) and Speech synthesis Technology (TTS), as well as voiceprint recognition Technology. The computer can listen, see, speak and feel, and the development direction of the future human-computer interaction is provided, wherein the voice becomes one of the best viewed human-computer interaction modes in the future.
Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that enable efficient communication between humans and computers using natural language. Natural language processing is a science integrating linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language that people use everyday, so it is closely related to the research of linguistics. Natural language processing techniques typically include text processing, semantic understanding, machine translation, robotic question and answer, knowledge mapping, and the like.
Machine Learning (ML) is a multi-domain cross discipline, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formal education learning.
The automatic driving technology generally comprises technologies such as high-precision maps, environment perception, behavior decision, path planning, motion control and the like, and has wide application prospects.
Along with the research and progress of artificial intelligence technology, the artificial intelligence technology develops research and application in a plurality of fields, for example, common intelligent house, intelligent wearable equipment, virtual assistant, intelligent sound box, intelligent marketing, unmanned driving, automatic driving, unmanned aerial vehicle, robot, intelligent medical treatment, intelligent customer service and the like. It is believed that with the development of technology, artificial intelligence technology will find application in more fields and will play an increasingly important role.
The scheme provided by the embodiment of the application relates to the computer vision technology of artificial intelligence and the like, and is specifically explained by the following embodiment.
Referring to fig. 1, fig. 1 is a schematic diagram of an exemplary system architecture of an image analysis method based on edge material data enhancement according to an embodiment of the present application. As shown in fig. 1, the system architecture includes a terminal device 1, a network 2 and a server 3, and the network 2 is used to provide a medium of a communication link between the terminal device 1 and the server 3. The network 2 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
It should be understood that the number of terminal devices 1, networks 2 and servers 3 in fig. 1 is merely illustrative, and there may be any number of terminal devices 1 or servers 3, as desired for an implementation. The user can use the terminal device 1 to interact with the server 3 via the network 2 to receive or transmit information or the like. The terminal device 1 may be various electronic devices with a display screen, including but not limited to smart phones, tablet computers, portable computers and desktop computers, smart voice interaction devices, smart home appliances, vehicle-mounted terminals, and the like.
The server 3 may be a proxy server providing various servers, for example, the server 3 may receive an image or a video input by a user to the terminal device 1, where the video may be regarded as a series of image frames, and the server 3 performs analysis processing on the received image or video by using a preset image analysis model. The preset image analysis model can be a trained deep learning model such as an image recognition model and an image segmentation model. Before analyzing and processing the received image or video by using the image analysis model, the server needs to perform model training on the analysis model by using the training sample enhanced by the edge material data.
Fig. 2 is a flowchart of an image analysis method based on edge material data enhancement according to an embodiment of the present application. It should be noted that, an execution subject of the image analysis method based on edge material data enhancement according to this embodiment is an image analysis apparatus based on edge material data enhancement, and the image analysis apparatus based on edge material data enhancement may be implemented in a software and/or hardware manner.
As shown in fig. 2, the image analysis method based on edge material data enhancement includes the following steps:
step 101, obtaining an original image to be analyzed.
The original image may be a single frame image, or may be a continuous multi-frame image in the video data.
102, inputting an original image into a target model subjected to edge material data enhancement training to obtain an analysis result corresponding to the original image, wherein the target model is trained by utilizing a target image obtained by enhancing edge material data of a first original image in a training set to obtain the target model subjected to edge material data enhancement training, the target image is obtained by splicing an edge material in an edge material library and the first original image in the training set, the edge material is obtained by performing edge detection and identification on a second original image, and the second original image is an arbitrary image.
The target model may be an image analysis model that performs recognition analysis, classification analysis, and the like on an image, and the original training set is a training set when edge material data enhancement is not performed.
Specifically, before the target model is used for analyzing the original image, the target model needs to be trained, before the target model is trained, the edge material in the edge material library needs to be spliced with the first original image in the training set to obtain the target image enhanced by the edge material data, and then the target model is subjected to model training by using the target image enhanced by the edge material data, so that the resistance of the target model to the edge material is effectively improved.
For example, the target model may be an image classification model, and at this time, the original image to be analyzed is input into the target model trained by using the edge material library, so that the target model can classify the original image according to the intermediate image content of each image without the influence of the edge material, and classify the intermediate image spliced with a plurality of different edge materials into a plurality of different classes.
Therefore, the target model obtained by training the target image containing the edge material information is used for analyzing the original image to be analyzed, the editing resistance of the target model to the edge material generated in the training process can be effectively utilized, and the analysis accuracy of the target model to the original image to be analyzed is improved.
In one or more embodiments, as shown in fig. 3 and 4, the image analysis method based on edge material data enhancement further includes:
step 201, a first original image in a training set is obtained.
Wherein, the training set is an original training set used for training the target model. The training set may include first original images for training a target and label information corresponding to each of the first original images. For example, when the target model is an image recognition model, the training set includes at least one first original image and a content label corresponding to each first original image, and each first original image and label information corresponding to each first original image are input to the target model, so that the target model is trained according to the first original image and the label information until the recognition result of the target model on the first original image is consistent with the corresponding content label, and then the training of the target model is completed.
Step 202, selecting a plurality of edge materials from the edge material library according to a preset rule.
The edge material library is an edge material set established according to original images with edges of the sea. Specifically, the method can perform target detection and edge detection on an original image containing an edge to intercept an edge material in the image, and store the intercepted edge material in an edge material library.
The preset rule may be set according to an actual situation, and optionally, may be set according to a type of the target model, that is, may be set according to a requirement on image processing accuracy. For example, when the image accuracy requirement is low, the edge material may be selected randomly, and when the image accuracy requirement is high, the edge material may be selected according to the size of the image. The requirement of the image precision is whether modification adjustment, such as size adjustment, can be performed on the original image, if the modification is possible, the requirement of the image precision is determined to be low, and if the modification is not possible, the requirement of the image precision is determined to be high.
Wherein, the number of the selected edge materials is a natural number which is more than or equal to 1.
And step 203, respectively splicing the plurality of edge materials to the first original image to obtain a plurality of target images.
Wherein, the splicing mode can be a processing mode of tightly connecting and overlapping the edge of the edge material and the edge of the first original image,
And step 204, training a target model according to the target image.
It should be understood that the purpose of training the target model by using the target image is to enable edge materials in the target image not to influence the recognition result of the target model on the target image.
For example, in an image deduplication scene, when the target model identifies two different target images (as shown in fig. 5) generated by stitching different edge materials with the same main content image, the identification results are the same, that is, the target model outputs the image identification result only according to the content of the main content image, and it is able to effectively ensure that the two images are recalled and filtered. Or, in the scene of the drama name identification, when the same frame of drama is spliced with different edge materials, the target model can accurately identify the film to which the frame of drama belongs.
Specifically, when the target model is trained, a first original image in a training set is obtained, then an edge material library for edge data enhancement is obtained, a plurality of edge materials are selected from the edge material library, the selected edge materials are spliced to the first original image respectively to obtain a plurality of target images, and then the target model is trained by using the target images.
Therefore, according to the image analysis method based on edge material data enhancement, the first original image is respectively spliced with the plurality of edge materials, the data enhancement effect of the first original image by using the edge materials is effectively achieved, and then the target model trained by using the target image obtained through edge data enhancement has good robustness for edge transformation, the edge editing resistance of the target model to image information is effectively improved, and the generalization capability of the target model is improved.
It should be understood that, because the data enhancement based on the edge material in the present application is to improve the edge editing resistance of the target model, no additional training parameters are introduced in the process of training the target model, that is, no influence is caused on the labels and attributes of the main content region, and the resource consumption of training is not increased.
Optionally, when the plurality of edge materials are respectively spliced to the first original image, the method may further include: for each edge material, data enhancement operations such as geometric transformation, color space conversion, random erasure and the like can be performed on the edge material to obtain a plurality of target edge materials, and then the plurality of target edge materials are respectively spliced with the first original image to obtain a plurality of target images.
Therefore, by enhancing the data of the edge material, the robustness of the target model to the edge transformation can be further enhanced, the edge editing resistance of the target model to the image information can be improved, and the generalization capability of the target model can be improved.
In one or more embodiments, as shown in fig. 6, a method of obtaining an edge corpus comprises:
step 301, a second original image is acquired.
The second original image may be an online image or an offline image, that is, the second image may be an information stream image from a network or an image stored locally.
It should be understood that the second original image may or may not include the edge material, and when the second original image includes the edge material, the time for constructing the edge material library can be further saved, and the efficiency for establishing the edge material library is effectively improved.
Step 302, extracting a first image feature of a second original image.
The first image feature is a feature vector corresponding to the second original image, and the first image feature includes all data information in the second original image, for example, texture information of the second original image. Optionally, the convolution layer may be used to convolve the second original image to obtain the first image feature, and preferably, a Deep residual network (ResNet) is used to obtain the first image feature of the second original image.
Step 303, inputting the first image feature into the edge detection model to obtain texture information and corresponding parameter values included in the edge region in the first image feature.
The edge detection model is a trained deep learning model, namely the edge detection model is trained by using a corresponding training set, so that edge materials in the input image can be effectively detected.
The parameter values comprise confidence values and edge coordinate values, the confidence values represent probability values that the edge regions of the second original image are edge materials, and the edge coordinate values are vertex coordinate values corresponding to the edge regions.
And 304, storing the edge area meeting the preset condition into an edge material library as an edge material.
Optionally, the preset condition is a confidence condition, and specifically, the edge region with the confidence value greater than a preset threshold is stored in the edge material library. The preset threshold is a preset threshold, and may be a threshold obtained through limited experiments, or a threshold obtained through limited computer simulation. Preferably, in one or more embodiments, the preset threshold may be 98%. For example, when the confidence value is greater than 98%, the texture information and the edge coordinate value of the edge region corresponding to the confidence value are stored in the edge material library.
Specifically, after the first image feature corresponding to the second original image is input to the edge detection model, the edge detection model determines, according to the first image feature, that the edge region of the second original image is a confidence value of the edge material, an edge coordinate value of the edge region, and texture information of the edge region, when the confidence value of the edge region is greater than a preset threshold, the edge region in the second original image is determined to be the edge material, and the texture information and the edge coordinate value of the edge material are stored in the edge material library, so that later splicing with the first original image is performed according to the texture information and the edge coordinate value of the edge material, when the confidence value of the edge region is less than or equal to the preset threshold, it is determined that the edge region of the second original image is not the edge material, that is, the edge region of the second original image is also a part of the main content of the second original image, at this time, the texture information and the edge coordinate values of the edge region obtained by the edge detection model are discarded.
Therefore, the edge materials are extracted from the second original image by using the edge detection model, the efficiency and the quantity of the extraction of the edge materials are effectively improved, and the extracted edge materials can be effectively ensured to be consistent with the edge materials applied actually (in the second original image), so that the reliability of the edge materials in the edge material library is effectively improved. Particularly, when the second original image and the first original image are both from a network, the target model trained by the edge material used in the second original image can be better adapted to the variation trend of the edge material in the first original image, and better robustness and generalization capability are maintained.
In one or more embodiments, as shown in fig. 7, the edge detection model includes a target detection task branch and an edge detection task branch, the parameter values include a confidence value and edge coordinate values, and the edge coordinate values are vertex coordinate values corresponding to the edge region.
Step 303, inputting the first image feature into the edge detection model to obtain texture information and corresponding parameter values included in the edge region in the first image feature, including: sending the first image feature to a target detection task branch to obtain a confidence value and an edge coordinate value corresponding to an edge region in the first image feature; and sending the first image feature to an edge detection task branch to obtain texture information of an edge area in the first image feature.
Further, the target detection task branch includes an area selection network and two full-connection layer branches, and sends the first image feature to the target detection task branch to obtain a confidence value and an edge coordinate value corresponding to an edge area in the first image feature, including: inputting the first image characteristic into a candidate area selection network to obtain a candidate edge area, inputting the candidate edge area into a first full-connection layer branch to obtain a confidence value corresponding to the candidate edge area, and inputting the candidate edge area into a second full-connection layer branch to obtain an edge coordinate value corresponding to the candidate edge area.
It should be understood that the deep learning model needs to be trained according to a training set, and when the trained target content is less, the accuracy of the model is beneficial to be improved, so that the branch for acquiring the parameter value of the edge region, the branch for acquiring the texture of the edge region, and the branch for acquiring the confidence value and the edge coordinate value corresponding to the candidate edge region are respectively set in the present application, so that each branch can concentrate more on the data output by itself, and the accuracy of each data output is effectively improved.
Specifically, as shown in fig. 6, the edge detection model includes a target detection task branch and an edge detection task branch, the target detection task branch includes a candidate area selection network and two full-connection layer branches, the candidate area selection network and the two full-connection layer branches are connected through a full-connection layer, and the edge detection branch includes a candidate area selection network and two full-connection layers connected in series. The candidate area selection network in the target detection task branch is the same as the candidate area selection network in the edge detection branch, so that the texture information, the confidence value and the edge coordinate value of each obtained candidate edge area can be in one-to-one correspondence. Alternatively, the candidate Region selection Network may generate a candidate Region model (RPN).
In one or more embodiments, the preset rule includes randomly selecting edge materials, and selecting a plurality of edge materials from an edge material library according to the preset rule includes: and randomly selecting a plurality of edge materials from the edge material library.
Further, the step of respectively splicing the plurality of edge materials to the first original image to obtain a plurality of target images includes: adjusting the size of the first original image according to the edge coordinate value corresponding to each edge material to obtain an intermediate image corresponding to each edge material; splicing the edge material to the corresponding intermediate image to obtain a target image; or, for each edge material, adjusting the size of the edge material according to the edge coordinate value of the first original image to obtain a target edge material, and splicing the target edge material to the first original image to obtain a target image.
That is, when selecting edge materials from the edge material library, the edge materials may be randomly selected from the edge material library, for example, sequence coding is performed on the edge materials in the edge material library, then a random number is generated in a coding range, and the edge materials with the sequence numbers corresponding to the random numbers are used as the selected edge materials.
It should be noted that, because the edge material is usually added by the image creator according to the requirement, the size, shape, direction and possibility of the edge material are different, as shown in fig. 5, the first edge material and the third edge material are both of the upper and lower structure, and the second edge material is only of the lower half portion, obviously, the space size of the main content area reserved by the three edge materials is different, at this time, when the edge material is spliced to the first original image, the size of the edge material or the first original image needs to be adjusted, so that the edge material and the first original image are tightly connected, and the splicing purpose is achieved.
Specifically, taking an edge material as an example, adjusting the size of a first original image according to the edge coordinate position corresponding to the edge material, that is, the coordinates of four vertexes of the first original image are the same as the four vertex coordinate positions of the edge material to obtain an intermediate image corresponding to the edge material, and then splicing the edge material onto the intermediate image whose four vertex positions are the same as the four vertex positions of the edge material to obtain a target image, thereby realizing the splicing of the edge material and the first original image by adjusting the first original image.
Or, according to the edge coordinate values (coordinate values of four vertexes) of the first original image to be spliced, adjusting the size of each edge material, namely, adjusting the four vertex coordinates of the edge material to coincide with the four vertex coordinates of the first original image to obtain a target edge material, and then splicing the target edge material to the first original image to obtain the target image.
In one or more embodiments, the preset rule includes selecting edge materials according to the size of the first original image, and selecting a plurality of edge materials from an edge material library according to the preset rule, including: and selecting a plurality of edge materials, the sizes of main content areas corresponding to the edge materials of which are consistent with the size of the first original image, from an edge material library, wherein the main content areas are areas surrounded by edge coordinate values of the edge materials.
That is to say, before selecting the edge material, the four vertex coordinates of the edge material may be determined, then the first width value and the first height value of the space surrounded by the four vertex coordinates are calculated, then the four vertex coordinates of the first original image to be spliced are obtained, the second width value and the second height value of the first original image are determined according to the four vertex coordinates of the first original image, at this time, the edge material corresponding to the first width value and the first height value which are both consistent with the second width value and the second height value is selected from the material library, and then the selected edge material is spliced to the first original image.
It should be understood that, because the size of the main content region surrounded by the selected edge material is consistent with the size of the first original image, the size of the first original image or the edge material does not need to be adjusted, the splicing speed is effectively saved, the loss of graphic information caused by size adjustment is avoided, and the image accuracy is improved.
It should be noted that, when the edge material has an up-down structure, as shown in (a) of fig. 8,the coordinates of the lower two vertexes of the upper part of the edge material are the upper left vertex coordinate a and the upper right vertex coordinate B in the four vertex coordinates of the edge material, the coordinates of the upper two vertexes of the lower part of the edge material are the lower left vertex coordinate C and the lower right vertex coordinate D in the four vertex coordinates of the edge material, and the four vertex coordinates of the first original image coincide with the coordinates A, B, C and D no matter the first original image is adjusted or the edge coordinates are adjusted. When the edge material has a single-sided edge structure, as shown in fig. 8 (b), and has only a lower half structure, the vertex a of the image area is defined’、B’The coordinates of the upper two vertexes of the lower part of the edge material are the lower left vertex coordinate C in the four vertex coordinates of the edge material’And the lower right vertex coordinate D’When adjusting the first original image or the edge coordinates, the four vertex coordinates and the coordinate a of the first original image are used’、B’、C’And D’And (4) overlapping. Accordingly, the lower two vertices of the image area are taken as the vertices of the edge material, as is the same when the edge material is only in the upper half. It should be understood that, when the edge material is a transverse edge material, the principle of adjusting the edge material and the first original image is the same as that of the edge material in the longitudinal direction in fig. 8, and the details of this application are not repeated herein.
In one or more embodiments, the plurality of edge material further includes edge material that is edge-free, i.e., four vertices of the edge material coincide with four vertices of the image region.
Therefore, the sample types of the marginal materials can be enriched through the marginal materials without the edges, and the risk of overfitting during target model training is effectively reduced.
As a specific embodiment, as shown in fig. 8, a large amount of second original images may be obtained, the obtained large amount of second original images are input into an edge detection model, and an edge material is obtained through analysis of the edge detection model.
When a target model needs to be trained, a first original image in a training set for training the target model is obtained, a plurality of edge materials are randomly selected from an edge material library, then the position coordinates of four vertexes of the first original image are adjusted to be coincident with the coordinates of the four vertexes of each edge material, the adjusted first original image and the edge materials are spliced to obtain a spliced target image, the target image is input into the target model, and the target model is trained.
As shown in fig. 9, the target model may be a classification recognition model, which may include a pooling layer, four convolution layers, and two fully-connected layers, and is trained to effectively recognize the type of the main content region of the image, wherein the type recognition of the main content may include, but is not limited to, drama recognition, portrait recognition, and the like.
In one or more embodiments, the edge detection model may be located in an offline server and the target model may be located in an online server. That is to say, the edge detection model and the target model can be set in different servers, so that the data processing amount of the server where the target model is located is effectively reduced, and the training speed and the analysis efficiency of the target model are improved.
Specifically, the offline server is configured with an edge detection model to extract different edge materials from a large number of acquired information stream pictures and/or videos and store the edge materials into an edge material library, so that a target model arranged in the online server can be trained conveniently, and an original image can be analyzed by using the trained target model. The edge detection model comprises a target detection task branch and an edge detection task branch. Specifically, an information flow picture and/or video used as a second original image is obtained and input into the edge detection model, and first image features corresponding to each second original image are obtained through a residual error network model ResNet101 set in the edge detection model. And then the first image characteristics are respectively input into the target detection task branch and the edge detection task branch. On one hand, in the target detection task branch, the target detection task branch utilizes a candidate area selection network to generate possible main content areas, then the obtained main content areas are respectively input into two paths of full connection layers to obtain confidence degrees and coordinate values corresponding to the main content areas of the content, then the confidence degrees of the main content areas are replaced by the confidence degrees of edge areas corresponding to a second original image, and the coordinate values of the main content areas are replaced by the coordinate values of the edge areas corresponding to the second original image, wherein the confidence values of the main content areas are the confidence values of the edge areas corresponding to the second original image, and the coordinate values of the subject content areas are the coordinate values of the edge areas corresponding to the second original image. On the other hand, in the edge detection task branch, a candidate area selection network which is the same as the target detection task branch is also arranged, so that an edge area corresponding to the second original image is determined according to a main content area determined by the candidate area selection network, and then texture information of a boundary extracted through a multilayer convolution network is obtained, and texture information of an edge material corresponding to the second original image is obtained. Finally, the edge detection model determines whether to store the edge material corresponding to the second original image into the edge material library according to the confidence value corresponding to the second original image, and when the edge material corresponding to the second original image is to be stored into the edge material library, the coordinate value and the texture information obtained by the second original image through the edge detection model are stored into the edge material library together.
And then, selecting a plurality of edge materials from the edge material library according to the online training requirement and preset rules by the target model arranged in the online server, and splicing the edge materials to the first original image in different sizes to obtain a target image, thereby realizing the purpose of enhancing the edge material data of the first original image. And the selected edge materials have the probability of having edge materials without edge coordinate values and texture information. And then, inputting the target image subjected to the edge material data enhancement into a target model to train the target model. After the target model is trained, the original image to be analyzed is input into the trained target model, so that an image analysis result with resistance to edge materials can be obtained.
The method comprises the steps of training a target model by using the method of not enhancing data, enhancing random horizontal turnover data, enhancing random color data, enhancing random erasure data and enhancing random edge material data provided by the method, wherein the steps are the same as the steps, such as 110000, of training the target model respectively, and the F1 score is obtained, wherein the F1 score is used for evaluating the accuracy of the analysis result of the target model.
Training mode Number of training steps F1 score
No data enhancement training 110000 67.33%
Random horizontal turnover training 110000 67.32%
Stochastic color enhancement training 110000 67.55%
Random erasure training 110000 68.01%
Random lace transformation training 110000 68.15%
Therefore, the target model enhanced by the edge material data has higher accuracy in the analysis of the original image compared with the target model not subjected to data enhancement training or the target model subjected to other types of data enhancement training.
To sum up, the original image that the analysis was treated through the target model that trains is analyzed to this application embodiment, wherein, before treating the original image that analyzes and carrying out image analysis, need utilize and carry out the image of data enhancement through edge material storehouse and train to make the target model that trains have better interference killing feature to the edge material, and, through carrying out the edge detection discernment to arbitrary second original image and obtaining the edge material, can effectively enrich the sample size of edge material, improve target model's robustness and generalization ability.
It should be noted that while the operations of the method of the present invention are depicted in the drawings in a particular order, this does not require or imply that the operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results.
With further reference to fig. 10, fig. 10 is a block diagram of an image analysis apparatus based on edge material data enhancement according to an embodiment of the present application.
As shown in fig. 10, the image analysis device 10 enhanced based on edge material data includes:
an obtaining module 11, configured to obtain an original image to be analyzed;
the analysis module 12 is configured to input the original image into a target model subjected to edge material data enhancement training, so as to obtain an analysis result corresponding to the original image, where the target model is trained by using a target image obtained by performing edge material data enhancement on a first original image in a training set, so as to obtain a target model subjected to edge material data enhancement training, the target image is obtained by splicing an edge material in an edge material library with the first original image in the training set, the edge material is obtained by performing edge detection and identification on a second original image, and the second original image is an arbitrary image.
In some embodiments, the analysis module 12, further comprises:
a first obtaining module 121, configured to obtain a first original image in a training set;
a random module 122, configured to select a plurality of edge materials from the edge material library according to a preset rule;
the splicing module 123 is configured to splice the plurality of edge materials to the first original image respectively to obtain a target image for model training;
and a training module 124 for training the target model according to the target image.
In some embodiments, as shown in fig. 11, the analysis module 12 further includes:
a second obtaining module 125, configured to obtain a second original image;
an extraction module 126, configured to extract a first image feature of the second original image;
the detection module 127 is configured to input the first image feature into the edge detection model, so as to obtain texture information and corresponding parameter values included in an edge region in the first image feature;
and the storage module 128 is configured to store the edge area with the parameter value meeting the preset condition into the edge material library as an edge material.
In some embodiments, the edge detection model includes a target detection task branch and an edge detection task branch, the parameter values include a confidence value and edge coordinate values, the edge coordinate values are vertex coordinate values corresponding to the edge region,
the detection module 127 is further configured to: sending the first image feature to a target detection task branch to obtain a confidence value and an edge coordinate value corresponding to an edge region in the first image feature; and
and sending the first image feature to an edge detection task branch to obtain texture information of an edge area in the first image feature.
In some embodiments, the target detection task branch includes a candidate area selection network and two full connectivity layer branches, and the detection module 127 is further configured to: inputting the first image characteristic into a candidate area selection network to obtain a candidate edge area;
inputting the candidate edge area to the first full-connection layer branch to obtain a confidence value corresponding to the candidate edge area;
and inputting the candidate edge region into the second full-connection layer branch to obtain an edge coordinate value corresponding to the candidate edge region.
In some embodiments, the parameter values include confidence values and edge coordinate values, the preset condition is a confidence condition, and the storage module 128 is further configured to: and storing the edge area with the confidence value larger than the preset threshold value into an edge material library as an edge material.
In some embodiments, the preset rule includes randomly selecting edge materials, and the random module 12 is further configured to: and randomly selecting a plurality of edge materials from the edge material library.
In some embodiments, the splicing module 123 is further configured to:
adjusting the size of the first original image according to the edge coordinate value corresponding to each edge material to obtain an intermediate image corresponding to each edge material;
splicing the edge material to the corresponding intermediate image to obtain a target image; or
Aiming at each edge material, adjusting the size of the edge material according to the edge coordinate value of the first original image to obtain a target edge material;
and splicing the target edge material to the first original image to obtain a target image.
In some embodiments, the preset rule includes selecting edge material according to the size of the first original image, and the random module 122 is further configured to: and selecting a plurality of edge materials, the sizes of main content areas corresponding to the edge materials of which are consistent with the size of the first original image, from an edge material library, wherein the main content areas are areas surrounded by edge coordinate values of the edge materials.
In some embodiments, the plurality of edge material includes edge material without edges.
It should be understood that the units or modules recited in the image analysis apparatus 10 enhanced based on the edge material data correspond to the respective steps in the method described with reference to fig. 2. Thus, the operations and features described above for the method are also applicable to the image analysis apparatus 10 enhanced based on edge material data and the units included therein, and will not be described in detail here. The image analysis apparatus 10 enhanced based on the edge material data may be implemented in a browser or other security applications of the electronic device in advance, or may be loaded into the browser or other security applications of the electronic device by downloading or the like. The corresponding units in the image analysis apparatus 10 enhanced based on the edge material data may be cooperated with the units in the electronic device to implement the solution of the embodiment of the present application.
The division into several modules or units mentioned in the above detailed description is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
In summary, according to the embodiment of the application, the target model obtained by training the target image containing the edge material information is used for analyzing the original image to be analyzed, so that the editing resistance of the target model to the edge material generated in the training process can be effectively utilized, and the analysis accuracy of the target model to the original image to be analyzed is improved.
It should be noted that, for details that are not disclosed in the image analysis apparatus based on edge material data enhancement according to the embodiment of the present application, please refer to details disclosed in the above embodiments of the present application, which are not described herein again.
Referring now to fig. 12, fig. 12 illustrates a schematic diagram of a computer system suitable for use in implementing an electronic device or server according to embodiments of the present application,
as shown in fig. 12, the computer system 1100 includes a Central Processing Unit (CPU) 1101, which can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM) 1102 or a program loaded from a storage section 1108 into a Random Access Memory (RAM) 1103. In the RAM1103, various programs and data necessary for operation instructions of the system are also stored. The CPU1101, ROM1102, and RAM1103 are connected to each other by a bus 1104. An input/output (I/O) interface 1105 is also connected to bus 1104.
The following components are connected to the I/O interface 1105; an input portion 1106 including a keyboard, mouse, and the like; an output portion 1107 including a signal output unit such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 1108 including a hard disk and the like; and a communication section 1109 including a network interface card such as a LAN card, a modem, or the like. The communication section 1109 performs communication processing via a network such as the internet. A driver 1110 is also connected to the I/O interface 1105 as necessary. A removable medium 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1110 as necessary, so that a computer program read out therefrom is mounted into the storage section 1108 as necessary.
In particular, according to embodiments of the present application, the process described above with reference to the flowchart fig. 2 may be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program comprises program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 1109 and/or installed from the removable medium 1111. The above-described functions defined in the system of the present application are executed when the computer program is executed by a Central Processing Unit (CPU) 1101.
It should be noted that the computer readable medium shown in the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operational instructions of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present application may be implemented by software or hardware. The described units or modules may also be provided in a processor, and may be described as: a processor comprises a first obtaining module, a random module, a splicing module and a training module. Where the names of these units or modules do not in some cases constitute a limitation on the unit or module itself, for example, the first acquisition module, may also be described as "acquiring the first raw image in the training set".
As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments, or may exist separately without being assembled into the electronic device. The computer readable storage medium stores one or more programs which, when executed by one or more processors, perform the image analysis method based on edge material data enhancement described herein.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the disclosure. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.

Claims (13)

1. An image analysis method based on edge material data enhancement is characterized by comprising the following steps:
acquiring an original image to be analyzed;
inputting the original image into a target model subjected to edge material data enhancement training to obtain an analysis result corresponding to the original image, wherein the target model is trained by utilizing a target image obtained by enhancing edge material data of a first original image in a training set to obtain the target model subjected to the edge material data enhancement training, the target image is obtained by splicing an edge material in an edge material library with the first original image in the training set, the edge material is obtained by performing edge detection and identification on a second original image, and the second original image is an arbitrary image;
the edge detection and identification of the second original image to obtain the edge material includes:
acquiring the second original image;
extracting first image features of the second original image;
inputting the first image feature into an edge detection model to obtain texture information and corresponding parameter values contained in an edge region of the first image feature, wherein the parameter values comprise confidence values and edge coordinate values, the confidence values represent probability values that the edge region of the second original image is an edge material, and the edge coordinate values are vertex coordinate values corresponding to the edge region;
and storing the edge area with the parameter value meeting the preset condition into the edge material library as the edge material.
2. The method of claim 1, wherein training the target model using a target image obtained by enhancing edge material data of a first original image in a training set comprises:
acquiring the first original image in the training set;
selecting a plurality of edge materials from an edge material library according to a preset rule;
respectively splicing the edge materials to the first original image to obtain a plurality of target images;
and training the target model according to the target image.
3. The method of claim 1, wherein the edge detection model comprises a target detection task branch and an edge detection task branch, wherein the parameter values comprise a confidence value and an edge coordinate value, and wherein the edge coordinate value is a vertex coordinate value corresponding to the edge region,
the inputting the first image feature into an edge detection model to obtain texture information and corresponding parameter values included in an edge region of the first image feature includes:
sending the first image feature to the target detection task branch to obtain the confidence value and the edge coordinate value corresponding to the edge region in the first image feature; and
and sending the first image feature to the edge detection task branch to obtain texture information of the edge area in the first image feature.
4. The method of claim 3, wherein the target detection task branch comprises a candidate area selection network and two full connectivity layer branches,
the sending the first image feature to the target detection task branch to obtain the confidence value and the edge coordinate value corresponding to the edge region in the first image feature includes:
inputting the first image characteristic into the candidate area selection network to obtain a candidate edge area;
inputting the candidate edge area to a first full-connection layer branch to obtain a confidence value corresponding to the candidate edge area;
and inputting the candidate edge region into a second full-connection layer branch to obtain an edge coordinate value corresponding to the candidate edge region.
5. The method according to claim 1, wherein the parameter values include confidence values and edge coordinate values, the preset condition is a confidence condition, and the storing the edge regions of which the parameter values satisfy the preset condition into the edge material library as the edge material includes:
and storing the edge area with the confidence value larger than a preset threshold value into the edge material library as the edge material.
6. The method of claim 2, wherein the predetermined rule comprises randomly selecting the edge material, and wherein selecting a plurality of edge materials from an edge material library according to the predetermined rule comprises:
and randomly selecting a plurality of edge materials from the edge material library.
7. The method according to claim 6, wherein the stitching the plurality of edge materials to the first original image respectively to obtain a plurality of target images comprises:
adjusting the size of the first original image according to the edge coordinate value corresponding to each edge material to obtain an intermediate image corresponding to each edge material;
splicing the edge material to the intermediate image corresponding to the edge material to obtain the target image; or
Aiming at each edge material, adjusting the size of the edge material according to the edge coordinate value of the first original image to obtain a target edge material;
and splicing the target edge material to the first original image to obtain the target image.
8. The method of claim 2, wherein the preset rule comprises selecting the edge material according to the size of the first original image, and wherein the selecting a plurality of edge materials from an edge material library according to the preset rule comprises:
and selecting a plurality of edge materials, the sizes of main content areas corresponding to the edge materials of which are consistent with the size of the first original image, from the edge material library, wherein the main content areas are areas surrounded by edge coordinate values of the edge materials.
9. The method of any of claims 1-8, wherein the edge material comprises the edge material without edges.
10. An image analysis apparatus based on edge material data enhancement, comprising:
the acquisition module is used for acquiring an original image to be analyzed;
the analysis module is used for inputting the original images into a target model subjected to edge material data enhancement training to obtain analysis results corresponding to the original images, wherein the target model is trained by utilizing a target image obtained by enhancing edge material data of a first original image in a training set to obtain the target model subjected to edge material data enhancement training, the target image is obtained by splicing edge materials in an edge material library with the first original image in the training set, the edge materials are obtained by performing edge detection and identification on a second original image, and the second original image is any image;
wherein, the analysis module further comprises:
the extraction module is used for extracting first image characteristics of the second original image;
the detection module is used for inputting the first image feature into an edge detection model to obtain texture information and corresponding parameter values contained in an edge region of the first image feature, wherein the parameter values comprise confidence values and edge coordinate values, the confidence values represent probability values that the edge region of the second original image is an edge material, and the edge coordinate values are vertex coordinate values corresponding to the edge region;
and the storage module is used for storing the edge area with the parameter value meeting the preset condition into the edge material library as the edge material.
11. The apparatus of claim 10, wherein the analysis module further comprises:
a first obtaining module, configured to obtain the first original image in the training set;
the random module is used for selecting a plurality of edge materials from the edge material library according to a preset rule;
the splicing module is used for splicing the edge materials to the first original image respectively to obtain a target image for model training;
and the training module is used for training a target model according to the target image.
12. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor, when executing the program, implements the image analysis method based on edge material data enhancement according to any one of claims 1 to 9.
13. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method for image analysis based on enhancement of edge material data according to any one of claims 1 to 9.
CN202111065726.XA 2021-09-13 2021-09-13 Image analysis method and device based on edge material data enhancement Active CN113505866B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111065726.XA CN113505866B (en) 2021-09-13 2021-09-13 Image analysis method and device based on edge material data enhancement

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111065726.XA CN113505866B (en) 2021-09-13 2021-09-13 Image analysis method and device based on edge material data enhancement

Publications (2)

Publication Number Publication Date
CN113505866A CN113505866A (en) 2021-10-15
CN113505866B true CN113505866B (en) 2021-12-03

Family

ID=78016794

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111065726.XA Active CN113505866B (en) 2021-09-13 2021-09-13 Image analysis method and device based on edge material data enhancement

Country Status (1)

Country Link
CN (1) CN113505866B (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113344957A (en) * 2021-07-19 2021-09-03 北京城市网邻信息技术有限公司 Image processing method, image processing apparatus, and non-transitory storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110866931B (en) * 2019-11-18 2022-11-01 东声(苏州)智能科技有限公司 Image segmentation model training method and classification-based enhanced image segmentation method
CN113112511B (en) * 2021-04-19 2024-01-05 新东方教育科技集团有限公司 Method and device for correcting test paper, storage medium and electronic equipment

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113344957A (en) * 2021-07-19 2021-09-03 北京城市网邻信息技术有限公司 Image processing method, image processing apparatus, and non-transitory storage medium

Also Published As

Publication number Publication date
CN113505866A (en) 2021-10-15

Similar Documents

Publication Publication Date Title
CN111563502B (en) Image text recognition method and device, electronic equipment and computer storage medium
CN111709409B (en) Face living body detection method, device, equipment and medium
CN111754596B (en) Editing model generation method, device, equipment and medium for editing face image
CN111553267B (en) Image processing method, image processing model training method and device
CN111275784B (en) Method and device for generating image
CN111680678B (en) Target area identification method, device, equipment and readable storage medium
CN111739027B (en) Image processing method, device, equipment and readable storage medium
CN112272295B (en) Method for generating video with three-dimensional effect, method for playing video, device and equipment
CN113411550B (en) Video coloring method, device, equipment and storage medium
CN113449700B (en) Training of video classification model, video classification method, device, equipment and medium
CN113761153A (en) Question and answer processing method and device based on picture, readable medium and electronic equipment
CN113706562B (en) Image segmentation method, device and system and cell segmentation method
CN112132770A (en) Image restoration method and device, computer readable medium and electronic equipment
CN110807379A (en) Semantic recognition method and device and computer storage medium
CN113570685A (en) Image processing method and device, electronic device and storage medium
CN116701706B (en) Data processing method, device, equipment and medium based on artificial intelligence
CN113505866B (en) Image analysis method and device based on edge material data enhancement
CN111008622A (en) Image object detection method and device and computer readable storage medium
CN114419514B (en) Data processing method, device, computer equipment and storage medium
CN112862840B (en) Image segmentation method, device, equipment and medium
CN113673567B (en) Panorama emotion recognition method and system based on multi-angle sub-region self-adaption
CN115115699A (en) Attitude estimation method and device, related equipment and computer product
CN112749691A (en) Image processing method and related equipment
CN110472728B (en) Target information determining method, target information determining device, medium and electronic equipment
CN117011324A (en) Image processing method, device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40053597

Country of ref document: HK