CN113689337B - Ultrasonic image super-resolution reconstruction method and system based on generation countermeasure network - Google Patents
Ultrasonic image super-resolution reconstruction method and system based on generation countermeasure network Download PDFInfo
- Publication number
- CN113689337B CN113689337B CN202110994950.0A CN202110994950A CN113689337B CN 113689337 B CN113689337 B CN 113689337B CN 202110994950 A CN202110994950 A CN 202110994950A CN 113689337 B CN113689337 B CN 113689337B
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- network
- super
- ultrasonic image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000003384 imaging method Methods 0.000 claims abstract description 21
- 238000013178 mathematical model Methods 0.000 claims abstract description 7
- 238000002604 ultrasonography Methods 0.000 claims description 30
- 238000000605 extraction Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 5
- 238000011156 evaluation Methods 0.000 claims description 4
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 3
- 230000003321 amplification Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 3
- 238000012285 ultrasound imaging Methods 0.000 claims description 3
- 230000001960 triggered effect Effects 0.000 claims description 2
- 238000013139 quantization Methods 0.000 claims 2
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000003745 diagnosis Methods 0.000 description 3
- 238000001303 quality assessment method Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000015556 catabolic process Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 238000002059 diagnostic imaging Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10132—Ultrasound image
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an ultrasonic image super-resolution reconstruction method based on a generated countermeasure network. The method comprises the following steps: s1: obtaining an ultrasonic image by an ultrasonic imaging device; s2: performing image downsampling on the existing high-resolution ultrasonic image to obtain a matched low-resolution ultrasonic image; s3: reconstructing a super-resolution ultrasonic image from the low-resolution ultrasonic image through a generation network; s4: the judging network judges the super-resolution ultrasonic image and the high-resolution ultrasonic image respectively, and outputs probability for true and false judgment; s5: generating a network and judging the network to perform a game of maximum and minimum value between the network and the judging network through mutual learning and countermeasure until the network reaches a stable state; s6: and (5) giving quantitative indexes by using a mathematical model to evaluate the similarity of the super-resolution ultrasonic image and the high-resolution ultrasonic image. The invention can effectively improve the ultrasonic image imaging quality of the handheld ultrasonic imaging equipment.
Description
Technical Field
The invention belongs to the field of imaging enhancement, and relates to an ultrasonic image super-resolution reconstruction method and system based on a generated countermeasure network.
Background
Ultrasound imaging is one of the most commonly used medical imaging methods at present, has real-time property and portability, and occupies a main position in medical imaging technology. The B-type ultrasonic imaging technology plays an important role in being used for assisting medical diagnosis, and the traditional B-type ultrasonic diagnosis equipment is huge in size, high in price, limited to clinical medical treatment and unfavorable for accident site diagnosis and family medical treatment scenes. With the development of embedded hardware devices, portable and handheld type B ultrasonic diagnostic devices are made possible. However, the volume of the device is greatly reduced, so that the quality of an ultrasonic image displayed by the device is poorer than that of the traditional device, and more background noise exists, so that the quality of the ultrasonic image is an important task.
Currently, super-resolution reconstruction is an important means to improve image quality. The effects of deblurring, speckle removal, edge enhancement and the like can be achieved by performing super-resolution reconstruction on the low-resolution image. The super-resolution of the image refers to recovering natural and clear textures from the low-resolution image, and finally obtaining a high-resolution image is a very important problem in the field of image enhancement. Low resolution images are typically obtained through a series of degradation operations that introduce a series of noise while losing a significant amount of image detail. The super-resolution reconstruction process based on deep learning is essentially that training based on learning is carried out by adopting paired training data through a network model, and then the inverse operation of the degradation operation is fitted, so that a reconstructed high-resolution image is obtained.
Most of the current image reconstruction methods are applied to natural images, and medical images are less researched. Medical images are greatly different from natural images due to their unique imaging modalities. The natural image has bright color, rich content and various details such as image edges. Because the level of the pixel is very varied, human eyes and algorithms can easily locate and detect the corresponding structure, the processing of segmentation, identification and the like is relatively easy, and the obtained result is often high in reliability. The medical image is a gray image, the image has single texture, simple information and few types, and most of human organs have similar structures. The medical image scanned by the instrument has similar conditions in gray level change, pixel value change, edge information and the like. The pixel values of each layer of matrix of the RGB channel of the ultrasonic image are the same, and compared with a natural image, the method is very simple, so that not only is manual labeling difficult, but also a computer algorithm cannot judge correctly. So that algorithms that can be successfully applied to natural images are not necessarily applicable to medical images. Based on this background, the present invention expands super-resolution reconstruction studies based on generating an countermeasure network for low quality ultrasound images.
Disclosure of Invention
The invention aims to solve the technical problems that the portable ultrasonic imaging equipment can acquire ultrasonic images, but the volume of the portable equipment is greatly reduced, so that the quality of the images displayed by the equipment is poor, the problems of more background noise, electromagnetic interference noise and the like exist, and the imaging quality of the ultrasonic images is affected.
The technical scheme for solving the problems is that the invention provides a method for generating ultrasonic image super-resolution reconstruction of an countermeasure network, which is characterized by comprising the following steps:
step S1: obtaining an ultrasonic image by an ultrasonic imaging device;
step S2: performing image downsampling on the existing high-resolution ultrasonic image to obtain a matched low-resolution ultrasonic image;
step S3: reconstructing a super-resolution ultrasonic image from the low-resolution ultrasonic image through a generation network;
step S4: the judging network judges the super-resolution ultrasonic image and the high-resolution ultrasonic image respectively, and outputs probability for true and false judgment;
step S5: generating a network and judging the network to perform a game of maximum and minimum value between the network and the judging network through mutual learning and countermeasure until the network reaches a stable state;
step S6: and (5) giving quantitative indexes by using a mathematical model to evaluate the similarity of the super-resolution ultrasonic image and the high-resolution ultrasonic image.
Step S1 comprises the steps of:
in this step, the medical staff or the operator of the ultrasonic imaging apparatus scans and acquires a proper ultrasonic image through the ultrasonic imaging apparatus according to his own experience, and the ultrasonic imaging apparatus is already disclosed and will not be described herein.
Step S2 comprises the steps of:
the invention needs to downsample the high-resolution ultrasonic image, reserves as much source image information as possible, and minimizes the loss of the image information. Therefore, considering that the super-resolution processing methods based on learning all use bicubic interpolation to downsample the high-resolution image to obtain the corresponding low-resolution image, the invention selects bicubic interpolation to downsample.
Bicubic interpolation calculates pixels of the target point from the correlation formula based on gray values of 16 points around the source image. The method takes all the surrounding 16 points into consideration, and the algorithm enlarges the receptive field and comprises the direct adjacent points and the adjacent points in a circle around, so that the utilization of surrounding information is more perfect. Through operation, each pixel of the reconstruction result contains more neighborhood information, but the calculation amount is larger. The bicubic interpolation algorithm needs to select a proper interpolation basis function for fitting, wherein the most common interpolation basis function is:
the bicubic interpolation function is:
f(i+u,j+v)＝A·B·C
wherein w represents the distance from the pixel point of the image to the P point, and the P point is the position of the image which is subjected to bicubic interpolation downsampling processing and corresponds to the source image; i and j respectively represent integer parts of coordinates of the pixel points of the image; u, v represent the fractional part of the image pixel coordinates, respectively.
Wherein A, B, C represents a matrix, the form of which is shown in the following formula:
A＝[S(1+u)S(u)S(1-u)S(2-u)]
C＝[S(1+v)S(v)S(1-v)S(2-v)] T
where f (i, j) represents the pixel value of the coordinate point of the source image (i, j).
Step S3 comprises the steps of:
the low-resolution ultrasonic image is reconstructed into a super-resolution ultrasonic image through a generating network, wherein the generating network is provided with 10 layers, the low-resolution ultrasonic image is firstly subjected to characteristic extraction of a convolution layer and comprises a layer of convolution treatment, then is subjected to a residual block overlapped with 5 layers, the result of each layer is overlapped with the output of the previous layer, and is subjected to a layer of convolution, the result of the convolution is overlapped with the result of the first layer, the characteristic loss is prevented, and the output is subjected to double amplification treatment of two layers to reconstruct the super-resolution ultrasonic image four times as large.
Step S4 comprises the steps of:
the discrimination network firstly carries out feature extraction on the image generated by the generation network, the feature extraction comprises 8 convolution layers, then two full-connection layers are adopted to finally obtain discrimination results, the discrimination network respectively judges the super-resolution ultrasonic image and the high-resolution ultrasonic image, the output probability is used for true and false judgment until the obtained numerical value tends to 0.5, and the training is gradually stable.
The discrimination network firstly performs feature extraction on the high-resolution ultrasonic image or the super-resolution ultrasonic image generated by the generation network, wherein the feature extraction comprises convolution, an LReLU activation layer and batch normalization, and then the discrimination result is finally obtained through two-layer full-connection layer processing and sigmoid function activation.
Step S5 comprises the steps of:
the generating network and the judging network learn and fight each other, and game with a maximum and minimum value is carried out between the generating network and the judging network until the value obtained by the judging network tends to 0.5, and the network training is gradually stable.
Step S6 includes the steps of:
the purpose of super-resolution reconstruction of low-resolution ultrasound images is to generate as much as possible the same image as high-resolution ultrasound images, so that image quality assessment is a critical issue. Objective evaluation uses a mathematical model to give a quantitative index, and peak signal to noise ratio (PSNR) and Structural Similarity (SSIM) are common quantitative indexes.
First, peak signal-to-noise ratio, i.e., the ratio of the energy of the peak signal to the average energy of the noise, is one of the most widely used objective image quality assessment indicators. Since the Mean Square Error (MSE) is the energy mean value of the difference between the real image and the noisy image, the difference between the two is noise, and thus the PSNR index is the ratio of the peak signal energy to the MSE, and the formula is as follows:
where X, Y denotes the current image and the reference image, H, W is the height and width of the image, respectively.
The unit of peak signal-to-noise ratio is dB, with a larger value indicating less distortion.
Second, structural similarity. In the case of spatial similarity, there is a very high structural correlation between the pixels of the image. The structural correlation index is triggered by the similarity of the image structure and consists of three comparison modules, namely brightness, contrast and structure. The value of SSIM ranges from 0 to 1, the closer to 1, the more similar the two images. Compared with PSNR, the evaluation method can reflect the visual quality of the image, and is widely used in the field of image super-resolution.
Based on the method, the invention also provides an ultrasonic image super-resolution reconstruction system based on the generation of the countermeasure network, which comprises the following steps: a memory and a processor; the memory has stored thereon a computer program which, when executed by the processor, implements the method described above.
The invention also proposes a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the above-mentioned method.
The invention has the beneficial effects that: by implementing the ultrasonic image super-resolution reconstruction method based on the generation countermeasure network, the ultrasonic image imaging quality of the handheld ultrasonic imaging equipment can be effectively improved, and medical staff or operators of the ultrasonic imaging equipment can be helped to acquire clearer ultrasonic images through the handheld ultrasonic imaging equipment.
Drawings
Fig. 1 is a flow chart of the ultrasound image super-resolution reconstruction method based on generating an countermeasure network of the present invention.
Fig. 2 is an exemplary image downsampled low resolution ultrasound image of an embodiment.
Fig. 3 is an example of a super-resolution ultrasound image reconstructed by super-resolution based on ultrasound images generated against a network according to an embodiment.
Fig. 4 is a second example of a downsampled low resolution ultrasound image of an embodiment.
Fig. 5 is a second example of a super-resolution ultrasound image reconstructed by super-resolution of an ultrasound image based on generation of a countermeasure network according to an embodiment.
Fig. 6 is a third example of a downsampled low resolution ultrasound image of an embodiment.
Fig. 7 is a third example of a super-resolution ultrasound image reconstructed by super-resolution of an ultrasound image based on generation of a countermeasure network according to an embodiment.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The following is a specific embodiment of the ultrasonic image super-resolution reconstruction method based on the generation countermeasure network:
as shown in fig. 1, a flowchart of an ultrasound image super-resolution reconstruction method based on generating an countermeasure network according to an embodiment of the present invention includes the following steps:
step S1: obtaining an ultrasonic image by an ultrasonic imaging device;
in this step, the medical staff or the operator of the ultrasonic imaging apparatus scans and acquires a proper ultrasonic image through the ultrasonic imaging apparatus according to his own experience, and the ultrasonic imaging apparatus is already disclosed and will not be described herein.
Step S2: performing image downsampling on the existing high-resolution ultrasonic image to obtain a matched low-resolution ultrasonic image;
in this step, the results obtained tend to be different due to the different a priori knowledge of the low resolution ultrasound images. The task is to learn the priori knowledge to make the model obtain a complete mapping system, but because no image pair with matched high resolution and low resolution exists, the high resolution image needs to be downsampled, and as much source image information as possible is reserved, so that the image information loss is minimized. Therefore, considering that the super-resolution processing methods based on learning all use bicubic interpolation to downsample the high-resolution image to obtain the corresponding low-resolution image, the invention selects bicubic interpolation to downsample.
At present, the super-resolution network with better effect uses a bicubic interpolation downsampling method, but different ultrasonic image downsampling often can obtain different results, so that the ultrasonic image results generated by the network are different. An ultrasound image of size mxn is downsampled s times to produce a low resolution ultrasound image of size (M/s) x (N/s), and a pixel is calculated from the pixel values in the window of size sxs in the source image by different calculation methods, e.g., if a median downsampling method is used, the pixel value at this point is the median of all pixels in the window.
Step S3: the low-resolution ultrasonic image is reconstructed into a super-resolution ultrasonic image through a generating network.
The generating countermeasure network consists of a generating network and a judging network, and the super-resolution ultrasonic image is reconstructed by the low-resolution ultrasonic image through the generating network. The generating network has 10 layers, the low-resolution ultrasonic image is firstly subjected to the characteristic extraction of a convolution layer, wherein the characteristic extraction comprises a layer of convolution processing, then subjected to residual block processing of 5 layers of overlapping, the result of each layer is overlapped with the output result of the previous layer, and then subjected to a layer of convolution, and the result is overlapped with the result of the first layer of convolution to prevent the characteristic loss. And finally, outputting the result through two layers of double amplification processing to reconstruct a four-times large super-resolution ultrasonic image.
Step S4: the judging network judges the super-resolution ultrasonic image and the high-resolution ultrasonic image respectively, and outputs the probability for true and false judgment.
The generating countermeasure network consists of a generating network and a judging network, the judging network firstly carries out feature extraction on the image generated by the generating network, wherein the feature extraction comprises 8 convolution layers, then the judging network carries out two-layer full-connection layers to finally obtain a judging result, the judging network respectively judges the super-resolution ultrasonic image and the high-resolution ultrasonic image, the output probability is used for true and false judgment until the obtained numerical value tends to 0.5, and training is gradually stable.
The discrimination network firstly performs feature extraction on the high-resolution ultrasonic image or the super-resolution ultrasonic image generated by the generation network, wherein the feature extraction comprises convolution, an LReLU activation layer and batch normalization, and then the discrimination result is finally obtained through two-layer full-connection layer processing and sigmoid function activation.
The function of the discrimination network is to learn the characteristics between the low-resolution ultrasonic image and the super-resolution ultrasonic image, and feed the characteristics back to the generation network so as to promote the generation network to further generate clearer details.
Step S5: generating a network and judging the network to perform a game of maximum and minimum value between the network and the judging network through mutual learning and countermeasure until the network reaches a stable state;
the generation network and the discrimination network need to be constantly learned and antagonized. The aim of generating the network is to deceive the discrimination network as far as possible, the aim of the discrimination network is to distinguish the output of the generation network from the real sample as far as possible, the two networks are mutually opposed, the parameters are continuously adjusted, the final aim is that the discrimination network can not judge whether the output result of the generation network is real or not until the value of the output of the discrimination network approaches 0.5, and the training of the whole generation opposed network is gradually stable.
Step S6: and (5) giving quantitative indexes by using a mathematical model to evaluate the similarity of the super-resolution ultrasonic image and the high-resolution ultrasonic image.
The purpose of super-resolution reconstruction of low-resolution ultrasound images is to generate as much as possible the same image as high-resolution ultrasound images, so that image quality assessment is a critical issue. Objective evaluation uses a mathematical model to give a quantitative index, and peak signal to noise ratio (PSNR) and Structural Similarity (SSIM) are common quantitative indexes.
Fig. 2, fig. 4, fig. 6 show a low-resolution ultrasonic image obtained by downsampling a high-resolution ultrasonic image, fig. 3, fig. 5, fig. 7 show a super-resolution ultrasonic image reconstructed by an ultrasonic image super-resolution reconstruction method based on a generated countermeasure network, and as a result, the super-resolution ultrasonic image reconstructed by the super-resolution has clear bright and dark boundaries, uniform brightness, clear structure, obvious boundaries and real details, and the overall visual effect is obviously better than that of the low-resolution ultrasonic image.
The summary of the invention is not limited to the above embodiments. Variations and advantages that would occur to one skilled in the art are included in the invention without departing from the spirit and scope of the inventive concept, and the scope of the invention is defined by the appended claims. The super-resolution reconstruction method of the ultrasonic image based on the generation countermeasure network enables the generation of high-quality ultrasonic images through low-quality ultrasonic images.
The foregoing is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions easily contemplated by those skilled in the art within the technical scope of the present invention should be included in the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.
Claims (8)
1. A method for super-resolution reconstruction of ultrasound images based on generation of a countermeasure network, comprising the steps of:
step S1: obtaining an ultrasonic image by an ultrasonic imaging device;
step S2: performing image downsampling on the existing high-resolution ultrasonic image to obtain a matched low-resolution ultrasonic image;
step S3: reconstructing a super-resolution ultrasonic image from the low-resolution ultrasonic image through a generation network;
step S4: the judging network judges the super-resolution ultrasonic image and the high-resolution ultrasonic image respectively, and outputs probability for true and false judgment;
step S5: generating a network and judging the network to perform a game of maximum and minimum value between the network and the judging network through mutual learning and countermeasure until the network reaches a stable state;
step S6: and (5) giving quantitative indexes by using a mathematical model to evaluate the similarity of the super-resolution ultrasonic image and the high-resolution ultrasonic image.
2. The method for generating super-resolution reconstruction of ultrasound images over an countermeasure network according to claim 1, wherein in step S1, a medical staff member or an operator of the ultrasound imaging apparatus scans and acquires the ultrasound images by the ultrasound imaging apparatus.
3. The method for generating an ultrasound image super-resolution reconstruction of an countermeasure network according to claim 1, wherein in the step S2, a bicubic interpolation method is selected for downsampling, including: according to gray values of 16 points around the source image, calculating pixels of the target point, wherein an interpolation basis function is as follows:
the bicubic interpolation function is:
f(i+u,j+v)＝A·B·C
wherein w represents the distance from the pixel point of the image to the P point, and the P point is the position of the image which is subjected to bicubic interpolation downsampling processing and corresponds to the source image; i and j respectively represent integer parts of coordinates of the pixel points of the image; u, v respectively represent the decimal part of the coordinates of the image pixel points;
wherein A, B, C represents a matrix, the form of which is shown in the following formula:
A＝[S(1+u)S(u)S(1-u)S(2-u)]
C＝[S(1+v)S(v)S(1-v)S(2-v)]T
where f (i, j) represents the pixel value of the coordinate point of the source image (i, j).
4. The method for reconstructing super-resolution of ultrasound image based on generation countermeasure network as claimed in claim 1, wherein in step S3, the low-resolution ultrasound image is reconstructed into super-resolution ultrasound image through generation network, wherein the generation network has 10 layers, the low-resolution ultrasound image is firstly subjected to feature extraction of convolution layer, including one layer of convolution processing, then 5 layers of overlapping residual blocks, the result of each layer is overlapped with the output of the previous layer, then one layer of convolution is performed, the result is overlapped with the result of the first layer of convolution, feature loss is prevented, and the output is subjected to two layers of double amplification processing, thus reconstructing four times of large super-resolution ultrasound image.
5. The method for reconstructing super-resolution of ultrasonic image based on generation countermeasure network as claimed in claim 1, wherein in the step S4, the discrimination network firstly performs feature extraction on the image generated by the generation network, including 8 convolution layers, and then passes through two full connection layers to finally obtain discrimination results, the discrimination network respectively performs discrimination on the super-resolution ultrasonic image and the high-resolution ultrasonic image, the output probability is used for true and false discrimination until the obtained value tends to 0.5, and training is gradually stable;
the discrimination network firstly performs feature extraction on the high-resolution ultrasonic image or the super-resolution ultrasonic image generated by the generation network, wherein the feature extraction comprises convolution, an LReLU activation layer and batch normalization, and then the discrimination result is finally obtained through two-layer full-connection layer processing and sigmoid function activation.
6. The method for generating an ultrasound image super-resolution reconstruction of an countermeasure network according to claim 1, wherein in the step S6, the image quality is evaluated, and a mathematical model is used for objective evaluation to give a quantization index, wherein the quantization index is peak signal-to-noise ratio and structural similarity;
the peak signal-to-noise ratio, that is, the ratio of the energy of the peak signal to the average energy of the noise, is the mean value of the energy of the difference between the real image and the noisy image, and the difference between the real image and the noisy image is the noise, so that the PSNR index is the ratio of the peak signal energy to the MSE, and the formula is as follows:
where X, Y denotes the current image and the reference image, H, W is the height and width of the image, respectively;
the unit of peak signal-to-noise ratio is dB, and the larger the value is, the smaller the distortion is;
under the condition of spatial similarity, extremely high structural correlation exists between pixels of images, a structural correlation index is triggered from the similarity of the image structures and consists of three contrast modules of brightness, contrast and structures, the value range of the structural similarity is between 0 and 1, and the closer to 1, the more similar the two images are represented.
7. An ultrasound image super-resolution reconstruction system based on generation of an countermeasure network, comprising: a memory and a processor; the memory having stored thereon a computer program which, when executed by the processor, implements the method according to any of claims 1-6.
8. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110994950.0A CN113689337B (en) | 2021-08-27 | 2021-08-27 | Ultrasonic image super-resolution reconstruction method and system based on generation countermeasure network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110994950.0A CN113689337B (en) | 2021-08-27 | 2021-08-27 | Ultrasonic image super-resolution reconstruction method and system based on generation countermeasure network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113689337A CN113689337A (en) | 2021-11-23 |
| CN113689337B true CN113689337B (en) | 2023-09-19 |
Family
ID=78583363
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110994950.0A Active CN113689337B (en) | 2021-08-27 | 2021-08-27 | Ultrasonic image super-resolution reconstruction method and system based on generation countermeasure network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113689337B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114926468B (en) * | 2022-07-22 | 2022-12-06 | 深圳华声医疗技术股份有限公司 | Ultrasonic image quality control method, ultrasonic device, and storage medium |
| CN115190263B (en) * | 2022-09-13 | 2022-12-20 | 广州市保伦电子有限公司 | Video scaling method, device, equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107154023A (en) * | 2017-05-17 | 2017-09-12 | 电子科技大学 | Face super-resolution reconstruction method based on generation confrontation network and sub-pix convolution |
| CN111583109A (en) * | 2020-04-23 | 2020-08-25 | 华南理工大学 | Image super-resolution method based on generation countermeasure network |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018053340A1 (en) * | 2016-09-15 | 2018-03-22 | Twitter, Inc. | Super resolution using a generative adversarial network |
-
2021
- 2021-08-27 CN CN202110994950.0A patent/CN113689337B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107154023A (en) * | 2017-05-17 | 2017-09-12 | 电子科技大学 | Face super-resolution reconstruction method based on generation confrontation network and sub-pix convolution |
| CN111583109A (en) * | 2020-04-23 | 2020-08-25 | 华南理工大学 | Image super-resolution method based on generation countermeasure network |
Non-Patent Citations (1)
| Title |
|---|
| 基于WGAN单帧人脸图像超分辨率算法;周传华;吴幸运;李鸣;;计算机技术与发展(09);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113689337A (en) | 2021-11-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109791692B (en) | System and method for computer-aided detection using multiple images from different perspectives of a region of interest to improve detection accuracy | |
| Loizou et al. | Quality evaluation of ultrasound imaging in the carotid artery based on normalization and speckle reduction filtering | |
| CN107886508B (en) | Differential subtraction method and medical image processing method and system | |
| CN113689337B (en) | Ultrasonic image super-resolution reconstruction method and system based on generation countermeasure network | |
| CN102800071B (en) | Method for reconstructing super resolution of sequence image POCS | |
| CN110599421B (en) | Model training method, video fuzzy frame conversion method, device and storage medium | |
| CN111862044B (en) | Ultrasonic image processing method, ultrasonic image processing device, computer equipment and storage medium | |
| CN100561518C (en) | Self-adaptation medical image sequence interpolation method based on area-of-interest | |
| CN114241077B (en) | CT image resolution optimization method and device | |
| Zhu et al. | Residual dense network for medical magnetic resonance images super-resolution | |
| CN113989407B (en) | Training method and system for limb part recognition model in CT image | |
| CN107292819A (en) | A kind of infrared image super resolution ratio reconstruction method protected based on edge details | |
| US20210366120A1 (en) | Medical image processing apparatus, and medical imaging apparatus | |
| CN112598619A (en) | Method for establishing intracranial vascular simulation three-dimensional narrowing model based on transfer learning | |
| CN112562058A (en) | Rapid establishing method of intracranial vascular simulation three-dimensional model based on transfer learning | |
| CN115136192A (en) | Out-of-distribution detection of input instances to a model | |
| CN114792287A (en) | Medical ultrasonic image super-resolution reconstruction method based on multi-image fusion | |
| CN112150571A (en) | Image motion artifact eliminating method, device, equipment and storage medium | |
| CN109949274B (en) | Image processing method, device and system | |
| Chilukuri et al. | Analysing Of Image Quality Computation Models Through Convolutional Neural Network | |
| Zhao et al. | Perception-oriented generative adversarial network for retinal fundus image super-resolution | |
| CN113299369B (en) | Medical image window adjusting optimization method | |
| Karthik et al. | Automatic quality enhancement of medical diagnostic scans with deep neural image super-resolution models | |
| CN110930394B (en) | Method and terminal equipment for measuring slope and pinnate angle of muscle fiber bundle line | |
| Ko et al. | Accurate depth estimation of skin surface using a light‐field camera toward dynamic haptic palpation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |