CN111161289B - Method and device for improving contour precision of object in image - Google Patents
Method and device for improving contour precision of object in image Download PDFInfo
- Publication number
- CN111161289B CN111161289B CN201911368035.XA CN201911368035A CN111161289B CN 111161289 B CN111161289 B CN 111161289B CN 201911368035 A CN201911368035 A CN 201911368035A CN 111161289 B CN111161289 B CN 111161289B
- Authority
- CN
- China
- Prior art keywords
- image
- current
- contour
- layer
- contour line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method, a device and a computer program product for improving the contour precision of an object in an image. The method comprises the following steps: constructing an image pyramid layer by layer according to the output image and the original image of the segmentation model, and determining the current contour line of an object in the current layer image based on the contour of the object in the previous layer image while constructing each layer of image; expanding the current contour line; optimizing object contour precision in the current layer image by utilizing the current contour line and an expansion result thereof; and processing the images layer by layer in the mode until the target contour conforming to the precision of the original image is obtained. The invention can match the object contour of the image processed by the segmentation model with the object contour of the original image, and does not relate to the processing of the input image of the segmentation model, so that the operation pressure of an algorithm network is not increased.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, and a computer program product for improving the accuracy of an object contour in an image.
Background
The problem of image segmentation in the field of image processing is related to the extraction of the contour of an object in an image, and is widely used in various other related fields such as military, medical and video post-production. In recent years, with the development of algorithms such as deep learning, the effects of general image segmentation (contour extraction) and the like have been greatly improved, and the method has been increasingly put into practical use. However, algorithms such as deep learning are limited in dealing with contour extraction problems by the following factors:
1) An image segmentation network model (the present invention is simply referred to as a segmentation model, such as a neural network, etc.) requires that the size of the input image be fixed. On the premise that the image fraction gradually increases at the present stage, an original image (i.e. an uncompressed original image to be processed) is usually required to be subjected to compression processing to obtain a low-resolution input image for a segmentation model, and the size of an output image (for example, a mask image) obtained by extracting the object contour through the segmentation model is consistent with that of the input image, namely, the object contour in the output image is based on a contour curve under the low-resolution condition.
2) The amount of computation is proportional to the image size. The way of improving the resolution of the input image and even not performing compression processing cannot be simply considered from the algorithm input end, which is extremely unrealistic for the calculation power of the current application scene.
Under the premise of the technology, when the image is segmented by adopting a segmentation model in the prior art and is often restored to the high resolution of the original image through subsequent amplification processing, the contour precision of the object is greatly influenced, and particularly, the low-precision object contour in the image is difficult to meet the user expectations in consideration of the visual sensitivity of human eyes.
Disclosure of Invention
The invention provides a method, a device and a computer program product for improving the precision of an object contour in an image, which can efficiently and low-consumption improve the problem of low precision of the object contour in a low-resolution image after image segmentation by optimizing the object contour of the high-resolution image output by a segmentation model in the above mode.
The technical scheme adopted by the invention is as follows:
a method for improving the outline precision of an object in an image comprises the following steps:
constructing an image pyramid layer by layer according to the output image and the original image of the segmentation model, and determining the current contour line of an object in the current layer image based on the contour of the object in the previous layer image while constructing each layer of image;
expanding the current contour line;
optimizing object contour precision in the current layer image by utilizing the current contour line and an expansion result thereof;
and processing the images layer by layer in the mode until the target contour conforming to the precision of the original image is obtained.
Optionally, constructing the image pyramid layer by layer according to the output image of the segmentation model and the original image, and determining the current contour line of the object in the current layer image based on the object contour in the previous layer image while constructing each layer of image includes:
acquiring a first resolution of an output image and a second resolution of an original image;
setting a resolution ratio for constructing each layer of image in the image pyramid according to the first resolution and the second resolution;
using the resolution ratio and the first resolution to obtain a current resolution;
based on the current resolution, converting the object contour in the output image to a current contour line and converting the output image to a current layer image in an image pyramid.
Optionally, the converting the object contour in the output image into the current contour line based on the current resolution includes:
based on the current resolution, performing coordinate conversion on each pixel point of the object contour in the output image to obtain a plurality of discrete initial contour points;
and processing each initial contour point into a continuous curve to obtain the current contour line.
Optionally, the expanding the current contour line includes:
and expanding each pixel point in the current contour line one by one according to the resolutions of the previous layer image and the current layer image, and determining the search range of the substitute point of each pixel point.
Optionally, the optimizing the object contour precision in the current layer image by using the current contour line and the expansion result thereof includes:
determining a target pixel point from each searching range by utilizing the current layer image and the current contour line;
and determining a contour curve of an object in the current layer image by all the target pixel points, and performing cyclic search in the search range based on the contour curve to obtain an iteratively updated target contour for the current layer image.
Optionally, the determining, by using the current layer image and the current contour line, the target pixel point from each of the search ranges includes:
and searching an optimal substitution point for the current pixel point in the search range according to the gradient change relation of one pixel point in the current contour line relative to the current layer image and the adjacent pixel points, and taking the optimal substitution point as the target pixel point.
An apparatus for improving the contour accuracy of an object in an image, comprising:
the image pyramid processing module is used for constructing an image pyramid layer by layer according to the output image and the original image of the segmentation model, and determining the current contour line of an object in the current layer image based on the object contour in the previous layer image while constructing each layer of image;
the current contour expansion module is used for expanding the current contour line;
the contour precision optimization module is used for optimizing the contour precision of the object in the current layer image by utilizing the current contour line and the expansion result thereof;
and the high-precision target contour determination module is used for processing the target contours layer by layer in the mode until the target contours conforming to the precision of the original images are obtained.
Optionally, the current profile extension module is specifically configured to:
and expanding each pixel point in the current contour line one by one according to the resolutions of the previous layer image and the current layer image, and determining the search range of the substitute point of each pixel point.
Optionally, the profile accuracy optimization module is specifically configured to:
and searching an optimal substitution point for the current pixel point in the search range according to the gradient change relation of one pixel point in the current contour line relative to the current layer image and the adjacent pixel points, and taking the optimal substitution point as a target pixel point.
A computer program product which, when run on a computer device, causes the computer device to perform a method of improving the accuracy of an object profile in an image as described above.
The invention is characterized in that a current layer image with higher resolution than a previous layer image (low resolution) is built layer by layer between the top layer and the bottom layer of the image pyramid through the idea of the image pyramid, and simultaneously, the object contour of the current layer image can be initialized in parallel, namely, two post-processing conditions are respectively obtained: a current contour line and a current layer image. And then searching and expanding the current contour line, and optimizing the object contour by utilizing the current contour line, the expansion result and the current layer image, and so on, processing the object contour layer by layer downwards until the final optimization result of the pyramid bottom layer under the high resolution of the original image is obtained, namely, the object contour of the image processed by the segmentation model is matched with the object contour of the original image. The process does not involve processing the input image of the segmentation model, so that the operation pressure of the algorithm network is not increased, and the implementation mode of the invention is simple, convenient and efficient, thereby saving the whole operation resource.
Drawings
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described with reference to the accompanying drawings, in which:
FIG. 1 is a flowchart of an embodiment of a method for improving the accuracy of an object contour in an image according to the present invention;
FIG. 2 is a schematic diagram of an embodiment of an image pyramid provided by the present invention;
FIG. 3 is a flowchart of a step S1 according to an embodiment of the present invention;
FIG. 4 is a flow chart of an embodiment of obtaining a current contour line provided by the present invention;
FIG. 5 is a flowchart of a step S3 according to an embodiment of the present invention;
FIG. 6 is a block diagram of an embodiment of an apparatus for improving the accuracy of an object profile in an image according to the present invention;
fig. 7 is a schematic diagram of an embodiment of an apparatus for improving the accuracy of an object contour in an image according to the present invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention.
First, as a design premise of a technical implementation means and a specific implementation example thereof, which are involved in the creation of the present invention, the technical problem involved in the present invention needs to be described again.
As mentioned above, when the deep learning algorithm is used, it is necessary to reduce the input image to a predetermined size, typically a relatively small size, for example, about 200×200, and then enlarge the segmented mask image obtained by the segmentation model to the original image. This process creates problems with profile accuracy. Because digital images tend to be rasterized, a pixel can be considered a grid. If an image is reduced by a factor of two in both the length and width dimensions, one pixel after reduction corresponds to a 2×2 area in the original image. If this pixel on the small image is exactly one edge pixel of the object in the image, then four pixels in the corresponding 2 x 2 area after enlargement, which are all belonging to the object, and possibly only part of the pixels belonging to the object, are at least two possibilities, which is the problem of the accuracy of the object contour in the low resolution image in the high resolution image.
Based on this, the present invention provides an embodiment of a method for improving the accuracy of an object contour in an image, as shown in fig. 1, including:
step S1, constructing an image pyramid layer by layer according to an output image and an original image of a segmentation model, and determining a current contour line of an object in a current layer image based on an object contour in a previous layer image while constructing each layer of image;
s2, expanding the current contour line;
s3, optimizing object contour precision in the current layer image by utilizing the current contour line and an expansion result thereof;
and step S4, processing the images layer by layer in the mode until the target contour conforming to the precision of the original image is obtained.
The image pyramid mentioned therein may refer to the multi-layer structure shown in fig. 2, i.e. the multi-image layer obtainable by resolution scaling, and it is obvious that in the inventive concept the image output by the segmentation model is the object to be processed, which is then located at the top layer in the image pyramid, whereas the original image may be the bottom layer of the image pyramid as the size of the original image is the final target after enlargement. The main idea of the invention is to perform step-by-step refinement from the contour of the image with the lowest resolution along the image pyramid until the high-precision contour on the image with the highest resolution is obtained. For this, three points are pointed out here:
one, in some embodiments, any layer may be used as a bottom layer based on different requirements, which will not be described herein.
Secondly, the embodiment does not construct a complete multi-layer image pyramid first and then performs optimization processing. On the premise of defining top and bottom images, performing layer-by-layer optimization on the currently constructed image layer and the corresponding current contour line in the layer-by-layer construction process, and then constructing and optimizing layer by layer until the bottom image is processed.
Thirdly, in the process of acquiring the current contour line in the embodiment, although contour extraction can be performed on the current layer image after the current layer image is constructed, considering the problem of consumption of operation resources, it is preferable that the current layer image and the current contour line are acquired as different processing paths, and the two processing paths are not in sequence or cause and effect.
In particular, reference may be made to the specific implementation shown in fig. 3:
step S11, acquiring a first resolution of an output image and a second resolution of an original image;
step S12, setting a resolution ratio for constructing each layer of image in the image pyramid according to the first resolution and the second resolution;
step S13, the current resolution is obtained by utilizing the resolution ratio and the first resolution;
step S14, converting the object contour in the output image into a current contour line and converting the output image into a current layer image in an image pyramid based on the current resolution.
It can be seen that, in this embodiment, instead of directly constructing a multi-layer pyramid, the resolution ratio for constructing the multi-layer pyramid is determined by the resolutions of the top layer and the bottom layer images, that is, the ratio relationship between the previous layer image and the next layer image is amplified, and the resolution of the next layer image, that is, the current resolution, can be obtained when the previous layer image and the next layer image are processed step by step. The rule adopted later can be a parallel step, on the basis of the current resolution, the current layer image can be obtained through amplifying operation, on the one hand, the contour in the previous layer image can be directly processed, and the initial contour corresponding to the higher resolution of the current layer image, namely the current contour line, can be obtained.
Here, the present invention further provides at least one simple implementation concept for obtaining the current contour line, and referring to fig. 4, the method may include the following steps:
step S141, based on the current resolution, carrying out coordinate conversion on each pixel point of the object contour in the output image to obtain a plurality of discrete initial contour points;
and step S142, processing each initial contour point into a continuous curve to obtain the current contour line.
The method considers that the object contour converted by the resolution ratio relation is in a sparse and discontinuous state, so that the initial contour points are reprocessed after the coordinate conversion to be in a continuous state, and the related processing means can be a connecting line directly made between the discrete and adjacent initial contour points two by two to form a current contour line, and the connecting line can be made according to the position change trend of the discrete and adjacent initial contour points. The former has small calculation force loss, the current contour line formed by the latter is closer to the actual contour, and the former and the latter can be selected according to the requirement in the actual operation.
From the above embodiments, two input conditions for the subsequent processing can be obtained: a current contour line and a current layer image. The present invention proposes to expand the current contour line, which is an intermediate process, and the purpose of this step is to consider that the process of optimizing the contour line is the process of searching for the optimal contour point/line, but since the current image resolution is often relatively large, if the current image resolution is randomly searched, the cache hit rate is often reduced, and thus the performance of the relevant computer equipment (embedded) is degraded, and in fact, the image area related to the contour in the image is usually only a small part area of the original image. The purpose of this step is therefore to expand a legal search area based on the current contour for the subsequent optimization operation, in other words, the expansion is referred to as a limited expansion.
Specifically, the expanding the current contour line may be to expand each pixel point in the current contour line one by one according to the resolutions of the previous layer image and the current layer image, so as to determine a search range of the substitute point of each pixel point. In this example, by taking a point (more specifically, may refer to coordinates of the point), according to the scaling relationship of the image, a trusted region is explored around each point, and the searching region may be a rectangular frame or other set shape, which in this example, stands on a global and local view angle to search for related points outwards with each pixel point in the current contour line as a center, so that each point in the searching range can be used as an alternative to the point in the current contour line. Of course, it will be understood by those skilled in the art that, in an actual implementation process, a corresponding profile list may be formed for each pixel point in the current profile, which will be described later.
Regarding the following optimization step, the core idea is to determine a relatively optimal target point from the extended search range under certain conditions to replace the point in the original current contour line, that is, update the current contour line, so from the preferred point of view, the embodiment shown in fig. 5 may be referred to as follows:
step S31, determining a target pixel point from each searching range by utilizing the current layer image and the current contour line;
and step S32, determining a contour curve of an object in the current layer image by all the target pixel points, and carrying out cyclic search in the search range based on the contour curve to obtain an iteratively updated target contour for the current layer image.
For example, a candidate point may be randomly selected (or may be directly started by a first pixel point) within a search range of a first pixel point of the current contour line, a score of the random candidate point is determined by using an image parameter relationship between the random candidate point and the current layer image and between the random candidate point and the current contour line itself, then a second pixel point adjacent to the first pixel point is obtained on the current contour line along a preset direction, and a current best target pixel point for the second pixel point is determined within a search range of the second pixel point in combination with the determined random backup point and an image parameter relationship between the current layer image and the current contour line itself, and a corresponding score may be obtained. And taking the contour curve obtained by the round of optimization into consideration as a new current contour line, and recycling the operation of searching the target pixel point in the determined searching range, wherein the contour curve with the highest score is taken as the target contour of the object in the current layer image after limited iterative updating.
Further, for the above process of determining the target pixel point from each of the search ranges by using the current layer image and the current contour line, in some embodiments, the process may refer to searching for an optimal substitute point for the current pixel point in the search range according to a gradient change relationship of one pixel point in the current contour line relative to the current layer image and an adjacent pixel point, and using the optimal substitute point as the target pixel point. The process includes at least two layers of meaning, namely, the step change of the position and the color of the target pixel point relative to the current layer image is considered, and meanwhile, the color deviation of the target pixel point relative to other points on the contour line is considered, namely, in the embodiments, the optimization target is to find the target pixel point which has smoother transition relative to the image content and the contour line.
After the processing of the above processes, the optimal precision contour line of the current layer image in the image pyramid is obtained, then, as long as the complete process is adopted, similar processing is carried out in the image pyramid from low resolution to high resolution in sequence until the contour line of the image with the highest resolution (the second resolution) is obtained, and the precision optimization of the object contour in the output image of the segmentation model is completed based on the target contour with the highest resolution.
In summary, the concept of the present invention is to construct a current layer image with higher resolution than a previous layer image (low resolution) layer by layer between a top layer and a bottom layer of an image pyramid through the idea of the image pyramid, and simultaneously initialize the object contour of the current layer image in parallel, that is, obtain two post-processing conditions respectively: a current contour line and a current layer image. And then searching and expanding the current contour line, and optimizing the object contour by utilizing the current contour line, the expansion result and the current layer image, and so on, processing the object contour layer by layer downwards until the final optimization result of the pyramid bottom layer under the high resolution of the original image is obtained, namely, the object contour of the image processed by the segmentation model is matched with the object contour of the original image. The process does not involve processing the input image of the segmentation model, so that the operation pressure of an algorithm network is not increased, the implementation mode of the invention is simple and efficient, and thus, the whole operation resources can be saved, and regarding the advantage, the invention can be further supplemented by taking an application scene of carrying out the real-time segmentation of the portrait contours on the mobile phone as an example, the calculation force of the mobile phone is limited, and the algorithm processing such as deep learning is added during contour extraction, so that the resources reserved for the later precision improvement are very tense, and the invention also needs to make the actual calculation amount of the scheme as small as possible on the premise of ensuring the feasibility of the scheme, thereby fully utilizing the operation resources in each application scene.
On the basis of the above embodiments and preferred embodiments, for the convenience of understanding and implementation, the following specific examples of operation are provided on the basis of the foregoing.
The main premise of the image pyramid is to determine the resolution ratio between the layers of the pyramid, and the image pyramid can be constructed by using the ratio (as shown in fig. 2), wherein the highest layer corresponds to the image with the lowest resolution (such as a mask image after the segmentation model is processed and output), and the lowest layer corresponds to the image with the highest resolution (the original image which is not zoomed before the segmentation model is processed), so that how to determine the image pyramid can be completely adjusted according to specific conditions.
After the continuous curve with higher resolution is formed through coordinate transformation, the operation of expanding the contour line can be performed by generating a list, namely, forming a contour map list containing triples, wherein the triples define a contour map B: the local coordinates Bc of the contour point and the sub-image Bim near the contour point are respectively, and the rectangular frame coordinates Box of the region in the original image. With the information, not only the contour point and the pixel information in the field can be conveniently obtained, but also the coordinates can be more conveniently converted into the original image or from the original imageTo the contour map, global coordinates of contour points calculated from the contour map are denoted by (x (B), y (B), for example. For example by B -1 Representing a contour diagram in front of B 1 A contour map is shown after B, with B representing the corresponding contour point, and sometimes for convenience B also referring to the coordinates of the contour point in the original map, which can be easily distinguished from the context. Then, based on the points adjacent to each other before and after a certain point, the normal vector Nb can be calculated, and the x component can be represented by x (Nb) and the y component can be represented by y (Nb). In some specific algorithm embodiments, bi-1 is used to represent the previous contour point of the contour point bi, and bi+1 is used to represent the next contour point of bi, however, it should be noted that this is a simple example of a representation, since in practice i+1 should return to 1 if exceeding the length N of the contour, the description of the present invention is omitted.
The contour line with higher resolution is obtained by the resolution of the current layer, and the contour point in the image of the previous layer can be considered to be simply expanded, namely, the coordinate of the contour point is converted at higher resolution, but the precision of the contour point is also converted at low resolution, so that the contour line needs to be further expanded and promoted. By combining the profile map list, a strategy such as a greedy algorithm can be further used for iteratively updating the profile points/lines of the object in the current layer image, so that the accuracy of the profile points/lines is improved. When each contour point is circularly updated, repeated searching is not needed because the contour line determined by the front wheel is already acquired, and the searching range of each updating process is also determined by the contour map list, so that the searching operation can be stopped in time once the contour map list is exceeded. The whole iterative updating operation can be regarded as an optimization process, and the optimization target can be the following objective function S maximum:
wherein, the liquid crystal display device comprises a liquid crystal display device,representing contour point b i Score of->Representing contour point b i Gradient in the image, whereas lambda represents a compromise between gradient and colour difference, which can be set to 0.3 in general,/for example>Representing the color difference of the contour point from the neighboring contour point:
the above examples are merely specific operation schemes based on the foregoing schemes, and the operation processes, concept definitions, symbol marks and the like are not limited. In addition, corresponding to the above scheme, the invention also provides an embodiment of an apparatus for improving the contour precision of an object in an image, as shown in fig. 6, which specifically may include the following components:
the image pyramid processing module 1 is used for constructing an image pyramid layer by layer according to an output image and an original image of the segmentation model, and determining a current contour line of an object in a current layer image based on an object contour in a previous layer image while constructing each layer of image;
a current contour expansion module 2, configured to expand the current contour line;
a contour precision optimizing module 3, configured to optimize object contour precision in the current layer image by using the current contour line and an extension result thereof;
the high-precision target contour determination module 4 is used for processing layer by layer in the above manner until the target contour conforming to the precision of the original image is obtained.
Further, the current profile extension module is specifically configured to:
and expanding each pixel point in the current contour line one by one according to the resolutions of the previous layer image and the current layer image, and determining the search range of the substitute point of each pixel point.
Further, the profile accuracy optimization module is specifically configured to:
and searching an optimal substitution point for the current pixel point in the search range according to the gradient change relation of one pixel point in the current contour line relative to the current layer image and the adjacent pixel points, and taking the optimal substitution point as a target pixel point.
It should be understood that the above division of the components of the object contour precision improving apparatus in the image shown in fig. 6 is only a division of a logic function, and may be fully or partially integrated into a physical entity or may be physically separated. And these components may all be implemented in software in the form of a call through a processing element; or can be realized in hardware; it is also possible that part of the components are implemented in the form of software called by the processing element and part of the components are implemented in the form of hardware. For example, some of the above modules may be individually set up processing elements, or may be integrated in a chip of the electronic device. The implementation of the other components is similar. In addition, all or part of the components can be integrated together or can be independently realized. In implementation, each step of the above method or each component above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.
For example, the above components may be one or more integrated circuits configured to implement the above methods, such as: one or more specific integrated circuits (Application Specific Integrated Circuit; hereinafter ASIC), or one or more microprocessors (Digital Singnal Processor; hereinafter DSP), or one or more field programmable gate arrays (Field Programmable Gate Array; hereinafter FPGA), etc. For another example, these components may be integrated together and implemented in the form of a System-On-a-Chip (SOC).
In view of the foregoing examples and their preferred embodiments, those skilled in the art will appreciate that in practice the present invention is applicable to a variety of embodiments, and the present invention is schematically illustrated by the following carriers:
(1) An apparatus for improving the contour accuracy of an object in an image may include:
one or more processors, memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs comprising instructions, which when executed by the apparatus, cause the apparatus to perform the steps/functions of the aforementioned embodiment of the method of improving object profile accuracy in images or equivalent implementations.
Fig. 7 is a schematic structural diagram of at least one embodiment of an apparatus for improving the accuracy of an outline of an object in an image, where the apparatus may be an electronic apparatus or a circuit apparatus built in the electronic apparatus. The electronic equipment can be a cloud server, a mobile terminal (a mobile phone, a wearable device and a tablet personal computer), an intelligent screen, intelligent teaching equipment and the like. The specific form of the device for improving the contour accuracy of the object in the image is not limited in this embodiment.
As shown in fig. 7, the apparatus 900 for improving the accuracy of the contour of an object in an image includes a processor 910 and a memory 930. Wherein the processor 910 and the memory 930 may communicate with each other via an internal connection, and transfer control and/or data signals, the memory 930 is configured to store a computer program, and the processor 910 is configured to call and execute the computer program from the memory 930. The processor 910 and the memory 930 may be combined into a single processing device, more commonly referred to as separate components, and the processor 910 is configured to execute program code stored in the memory 930 to perform the functions described above. In particular, the memory 930 may also be integrated within the processor 910 or may be separate from the processor 910.
In addition, in order to further improve the functionality of the device 900 for improving the accuracy of the contour of an object in an image, the device 900 may further comprise one or more of an input unit 960, a display unit 970, an audio circuit 980, a camera 990, a sensor 901, etc., which may further comprise a speaker 982, a microphone 984, etc. Wherein the display unit 970 may include a display screen.
Further, the apparatus 900 for improving the contour precision of an object in an image may further include a power supply 950 for supplying power to various devices or circuits in the apparatus 900.
It should be appreciated that the object profile accuracy improving apparatus 900 in the image shown in fig. 7 can implement the respective processes of the method provided in the foregoing embodiment. The operations and/or functions of the various components in the device 900 may be respectively for implementing the corresponding flows in the method embodiments described above. Reference is specifically made to the foregoing descriptions of embodiments of methods, apparatuses and so forth, and detailed descriptions thereof are appropriately omitted for the purpose of avoiding redundancy.
It should be understood that, the processor 910 in the apparatus 900 for improving the accuracy of the contour of an object in an image shown in fig. 7 may be a system on a chip SOC, and the processor 910 may include a central processing unit (Central Processing Unit; hereinafter referred to as "CPU") and may further include other types of processors, for example: an image processor (Graphics Processing Unit; hereinafter referred to as GPU) or the like, as will be described in detail below.
In general, portions of the processors or processing units within the processor 910 may cooperate to implement the preceding method flows, and corresponding software programs for the portions of the processors or processing units may be stored in the memory 930.
(2) A readable storage medium having stored thereon a computer program or the above-mentioned apparatus, which when executed, causes a computer to perform the steps/functions of the foregoing example or equivalent implementation of a method for improving the accuracy of an object profile in an image.
In several embodiments provided by the present invention, any of the functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, certain aspects of the present invention may be embodied in the form of a software product as described below, in essence, or as a part of, contributing to the prior art.
(3) A computer program product (which may comprise the apparatus described above) which, when run on a terminal device, causes the terminal device to perform the method of improving the accuracy of the profile of an object in an image of the previous embodiment or equivalent.
From the above description of embodiments, it will be apparent to those skilled in the art that all or part of the steps of the above described methods may be implemented in software plus necessary general purpose hardware platforms. Based on such understanding, the above-described computer program product may include, but is not limited to, an APP; the foregoing description is further to be supplemented by the fact that the device/terminal may be a computer device (e.g., a mobile phone, a PC terminal, a cloud platform, a server cluster, or a network communication device such as a media gateway, etc.). Moreover, the hardware structure of the computer device may further specifically include: at least one processor, at least one communication interface, at least one memory and at least one communication bus; the processor, the communication interface and the memory can all communicate with each other through a communication bus. The processor may be a central processing unit CPU, DSP, microcontroller or digital signal processor, and may further include a GPU, an embedded Neural network processor (Neural-network Process Units; hereinafter referred to as NPU) and an image signal processor (Image Signal Processing; hereinafter referred to as ISP), and the processor may further include an ASIC (application specific integrated circuit) or one or more integrated circuits configured to implement embodiments of the present invention, and in addition, the processor may have a function of operating one or more software programs, and the software programs may be stored in a storage medium such as a memory; and the aforementioned memory/storage medium may include: nonvolatile Memory (non-volatile Memory), such as a non-removable magnetic disk, a USB flash disk, a removable hard disk, an optical disk, and the like, and Read-Only Memory (ROM), random access Memory (Random Access Memory; RAM), and the like.
In the embodiments of the present invention, "at least one" means one or more, and "a plurality" means two or more. "and/or", describes an association relation of association objects, and indicates that there may be three kinds of relations, for example, a and/or B, and may indicate that a alone exists, a and B together, and B alone exists. Wherein A, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of the following" and the like means any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c may represent: a, b, c, a and b, a and c, b and c or a and b and c, wherein a, b and c can be single or multiple.
Those of skill in the art will appreciate that the various modules, units, and method steps described in the embodiments disclosed herein can be implemented in electronic hardware, computer software, and combinations of electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
And, each embodiment in the specification is described in a progressive manner, and the same and similar parts of each embodiment are mutually referred to. In particular, for embodiments of the apparatus, device, etc., as they are substantially similar to method embodiments, the relevance may be found in part in the description of method embodiments. The above-described embodiments of apparatus, devices, etc. are merely illustrative, in which modules, units, etc. illustrated as separate components may or may not be physically separate, i.e., may be located in one place, or may be distributed across multiple places, e.g., nodes of a system network. In particular, some or all modules and units in the system can be selected according to actual needs to achieve the purpose of the embodiment scheme. Those skilled in the art will understand and practice the invention without undue burden.
The construction, features and effects of the present invention are described in detail according to the embodiments shown in the drawings, but the above is only a preferred embodiment of the present invention, and it should be understood that the technical features of the above embodiment and the preferred mode thereof can be reasonably combined and matched into various equivalent schemes by those skilled in the art without departing from or changing the design concept and technical effects of the present invention; therefore, the invention is not limited to the embodiments shown in the drawings, but is intended to be within the scope of the invention as long as changes made in the concept of the invention or modifications to the equivalent embodiments do not depart from the spirit of the invention as covered by the specification and drawings.
Claims (7)
1. The method for improving the contour precision of the object in the image is characterized by comprising the following steps of:
constructing an image pyramid layer by layer according to the output image and the original image of the segmentation model, and determining the current contour line of an object in the current layer image based on the contour of the object in the previous layer image while constructing each layer of image;
expanding the current contour line comprises the following steps: according to the resolution ratio of the previous layer image and the current layer image, expanding each pixel point in the current contour line one by one, and determining the search range of the substitute point of each pixel point;
optimizing object contour accuracy in the current layer image by using the current contour line and an expansion result thereof, wherein the method comprises the steps of determining a relatively optimal target point from the expanded search range to replace a point in the original current contour line and updating the current contour line;
and processing the images layer by layer in the mode until the target contour conforming to the precision of the original image is obtained.
2. The method for improving the contour precision of an object in an image according to claim 1, wherein constructing an image pyramid layer by layer according to an output image of a segmentation model and an original image, and determining a current contour line of the object in a current layer image based on the contour of the object in a previous layer image while constructing each layer image comprises:
acquiring a first resolution of an output image and a second resolution of an original image;
setting a resolution ratio for constructing each layer of image in the image pyramid according to the first resolution and the second resolution;
using the resolution ratio and the first resolution to obtain a current resolution;
based on the current resolution, converting the object contour in the output image to a current contour line and converting the output image to a current layer image in an image pyramid.
3. The method according to claim 2, wherein converting the object contour in the output image into a current contour line based on the current resolution comprises:
based on the current resolution, performing coordinate conversion on each pixel point of the object contour in the output image to obtain a plurality of discrete initial contour points;
and processing each initial contour point into a continuous curve to obtain the current contour line.
4. The method for improving the contour precision of an object in an image according to claim 1, wherein optimizing the contour precision of the object in the current layer image by using the current contour line and the expansion result thereof comprises:
determining a target pixel point from each searching range by utilizing the current layer image and the current contour line;
and determining a contour curve of an object in the current layer image by all the target pixel points, and performing cyclic search in the search range based on the contour curve to obtain an iteratively updated target contour for the current layer image.
5. The method for improving the accuracy of the contour of an object in an image according to claim 4, wherein determining a target pixel from each of the search ranges by using the current layer image and the current contour line comprises:
and searching an optimal substitution point for the current pixel point in the search range according to the gradient change relation of one pixel point in the current contour line relative to the current layer image and the adjacent pixel points, and taking the optimal substitution point as the target pixel point.
6. An apparatus for improving the accuracy of an outline of an object in an image, comprising:
the image pyramid processing module is used for constructing an image pyramid layer by layer according to the output image and the original image of the segmentation model, and determining the current contour line of an object in the current layer image based on the object contour in the previous layer image while constructing each layer of image;
the current contour expansion module is used for expanding the current contour line and comprises expanding each pixel point in the current contour line one by one according to the resolution of the previous layer image and the current layer image, and determining the search range of the substitution point of each pixel point;
the contour precision optimizing module is used for optimizing the contour precision of the object in the current layer image by utilizing the current contour line and the expansion result thereof, and comprises the steps of determining a relatively optimal target point from the expanded searching range to replace a point in the original current contour line and updating the current contour line;
and the high-precision target contour determination module is used for processing the target contours layer by layer in the mode until the target contours conforming to the precision of the original images are obtained.
7. The apparatus for improving the contour precision of an object in an image according to claim 6, wherein the contour precision optimizing module is specifically configured to:
and searching an optimal substitution point for the current pixel point in the search range according to the gradient change relation of one pixel point in the current contour line relative to the current layer image and the adjacent pixel points, and taking the optimal substitution point as a target pixel point.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911368035.XA CN111161289B (en) | 2019-12-26 | 2019-12-26 | Method and device for improving contour precision of object in image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911368035.XA CN111161289B (en) | 2019-12-26 | 2019-12-26 | Method and device for improving contour precision of object in image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111161289A CN111161289A (en) | 2020-05-15 |
| CN111161289B true CN111161289B (en) | 2023-08-08 |
Family
ID=70558487
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911368035.XA Active CN111161289B (en) | 2019-12-26 | 2019-12-26 | Method and device for improving contour precision of object in image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111161289B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113689355B (en) * | 2021-09-10 | 2022-07-08 | 数坤(北京)网络科技股份有限公司 | Image processing method, image processing device, storage medium and computer equipment |
| CN114278281B (en) * | 2021-12-24 | 2023-11-21 | 北京西华益昌技术开发有限责任公司 | Measurement resolution optimization method, device and equipment of measurement device and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101493936A (en) * | 2008-05-30 | 2009-07-29 | 内蒙古科技大学 | Multi- resolution non-rigid head medicine image registration method based on image edge |
| CN104156959A (en) * | 2014-08-08 | 2014-11-19 | 中科创达软件股份有限公司 | Video matting method and device |
| CN107123146A (en) * | 2017-03-20 | 2017-09-01 | 深圳市华汉伟业科技有限公司 | The mark localization method and system of a kind of scaling board image |
| CN107481252A (en) * | 2017-08-24 | 2017-12-15 | 上海术理智能科技有限公司 | Dividing method, device, medium and the electronic equipment of medical image |
| CN107833226A (en) * | 2017-10-26 | 2018-03-23 | 中国测绘科学研究院 | A kind of C V models based on exponential type image fusion sequence are to the fast automatic dividing method in SAR images coastline |
| CN108010082A (en) * | 2017-12-28 | 2018-05-08 | 上海觉感视觉科技有限公司 | A kind of method of geometric match |
| CN108960246A (en) * | 2018-06-14 | 2018-12-07 | 国网江苏省电力有限公司电力科学研究院 | A kind of binary conversion treatment device and method for image recognition |
| CN109934843A (en) * | 2019-01-28 | 2019-06-25 | 北京华捷艾米科技有限公司 | A kind of real-time profile, which refines, scratches image space method and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6422250B2 (en) * | 2014-07-08 | 2018-11-14 | キヤノン株式会社 | Image processing method, image processing apparatus, program, and recording medium |
| JP6570370B2 (en) * | 2015-08-21 | 2019-09-04 | キヤノン株式会社 | Image processing method, image processing apparatus, program, and recording medium |
-
2019
- 2019-12-26 CN CN201911368035.XA patent/CN111161289B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101493936A (en) * | 2008-05-30 | 2009-07-29 | 内蒙古科技大学 | Multi- resolution non-rigid head medicine image registration method based on image edge |
| CN104156959A (en) * | 2014-08-08 | 2014-11-19 | 中科创达软件股份有限公司 | Video matting method and device |
| CN107123146A (en) * | 2017-03-20 | 2017-09-01 | 深圳市华汉伟业科技有限公司 | The mark localization method and system of a kind of scaling board image |
| CN107481252A (en) * | 2017-08-24 | 2017-12-15 | 上海术理智能科技有限公司 | Dividing method, device, medium and the electronic equipment of medical image |
| CN107833226A (en) * | 2017-10-26 | 2018-03-23 | 中国测绘科学研究院 | A kind of C V models based on exponential type image fusion sequence are to the fast automatic dividing method in SAR images coastline |
| CN108010082A (en) * | 2017-12-28 | 2018-05-08 | 上海觉感视觉科技有限公司 | A kind of method of geometric match |
| CN108960246A (en) * | 2018-06-14 | 2018-12-07 | 国网江苏省电力有限公司电力科学研究院 | A kind of binary conversion treatment device and method for image recognition |
| CN109934843A (en) * | 2019-01-28 | 2019-06-25 | 北京华捷艾米科技有限公司 | A kind of real-time profile, which refines, scratches image space method and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 虞红伟.基于活动轮廓模型的乳腺X线图像肿块分割方法的研究.《万方数据库》.2011,第14-18,36-40页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111161289A (en) | 2020-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10803554B2 (en) | Image processing method and device | |
| KR102281017B1 (en) | Neural network model training method, apparatus and storage medium for image processing | |
| US10650495B2 (en) | High resolution style transfer | |
| CN108961303B (en) | Image processing method and device, electronic equipment and computer readable medium | |
| CN107506792B (en) | Semi-supervised salient object detection method | |
| CN111161289B (en) | Method and device for improving contour precision of object in image | |
| CN110428386B (en) | Map grid merging method and device, storage medium and electronic device | |
| US11443481B1 (en) | Reconstructing three-dimensional scenes portrayed in digital images utilizing point cloud machine-learning models | |
| CN114004754B (en) | Scene depth completion system and method based on deep learning | |
| CN111383232A (en) | Matting method, matting device, terminal equipment and computer-readable storage medium | |
| US20220351413A1 (en) | Target detection method, computer device and non-transitory readable storage medium | |
| CN111681165A (en) | Image processing method, image processing device, computer equipment and computer readable storage medium | |
| CN114926342A (en) | Image super-resolution reconstruction model construction method, device, equipment and storage medium | |
| Zhang et al. | Lightweight image super-resolution with superpixel token interaction | |
| JP6937782B2 (en) | Image processing method and device | |
| WO2024041108A1 (en) | Image correction model training method and apparatus, image correction method and apparatus, and computer device | |
| JP2021144428A (en) | Data processing device and data processing method | |
| CN116310899A (en) | YOLOv 5-based improved target detection method and device and training method | |
| CN113901247A (en) | Optical image target detection method and computing device | |
| Deka et al. | Joint sparse representation-based single image super-resolution for remote sensing applications | |
| US20230351719A1 (en) | Method and electronic device for determining optimal global attention in deep learning model | |
| CN111882588B (en) | Image block registration method and related product | |
| US20240031512A1 (en) | Method and system for generation of a plurality of portrait effects in an electronic device | |
| WO2023056833A1 (en) | Background picture generation method and apparatus, image fusion method and apparatus, and electronic device and readable medium | |
| CN115761402A (en) | Image processing method, electronic device and chip |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |