CN114998150A - Three-dimensional reconstruction method and device of ultrasonic image - Google Patents

Three-dimensional reconstruction method and device of ultrasonic image Download PDF

Info

Publication number
CN114998150A
CN114998150A CN202210681854.5A CN202210681854A CN114998150A CN 114998150 A CN114998150 A CN 114998150A CN 202210681854 A CN202210681854 A CN 202210681854A CN 114998150 A CN114998150 A CN 114998150A
Authority
CN
China
Prior art keywords
interpolation
tetrahedron
spatial
dimensional
lattice points
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210681854.5A
Other languages
Chinese (zh)
Other versions
CN114998150B (en
Inventor
付天宇
王一凡
杨健
林毓聪
范敬凡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Technology BIT
Original Assignee
Beijing Institute of Technology BIT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Institute of Technology BIT filed Critical Beijing Institute of Technology BIT
Priority to CN202210681854.5A priority Critical patent/CN114998150B/en
Publication of CN114998150A publication Critical patent/CN114998150A/en
Application granted granted Critical
Publication of CN114998150B publication Critical patent/CN114998150B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10028Range image; Depth image; 3D point clouds

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)
  • Image Processing (AREA)

Abstract

The three-dimensional reconstruction method and the three-dimensional reconstruction device for the ultrasonic image can reduce the influence of parameter imbalance caused by non-uniform acquisition density, have lower reconstruction errors, increase the application range, have independence on the interpolation process of different voxels, can perform parallel calculation, and greatly improve the reconstruction efficiency. The method comprises the following steps: (1) spatial mapping: regarding pixels in a two-dimensional image obtained from an ultrasonic acquisition system as plane point clouds only with values in the center, and obtaining the point clouds in a corresponding space coordinate system through space mapping; (2) dividing a tetrahedron: dividing the space under the new coordinate system into a plurality of tetrahedrons which are formed by taking the point cloud as a vertex; (3) tetrahedral interpolation: interpolating the lattice points in the tetrahedron, obtaining the vertex of the tetrahedron by using pixel mapping, and performing interpolation calculation on the lattice points at the corresponding positions of the voxels, wherein the interpolation method adopts the fusion of volume weight interpolation and spatial second-order polynomial interpolation, and the interpolation of each tetrahedron does not influence each other and performs parallel calculation to obtain the reconstructed three-dimensional data.

Description

Three-dimensional reconstruction method and device of ultrasonic image
Technical Field
The present invention relates to the technical field of medical image processing, and in particular, to a three-dimensional reconstruction method and a three-dimensional reconstruction device for an ultrasound image.
Background
The ultrasonic image is widely applied to clinical diagnosis due to the characteristics of low cost, no radiation and real-time performance, and the handheld ultrasonic image is more in line with the habits of doctors and the environment of an operating room due to the advantages of free scanning mode, capability of providing a larger imaging visual angle, higher image resolution and the like, is a main research direction of ultrasonic image guided interventional operations, and has great attention in recent years. The principle is that three-dimensional information is reconstructed from two-dimensional images acquired by an ultrasonic instrument by utilizing the space conversion relation between the images, the misdiagnosis risk can be reduced, and the method is widely applied to the aspects of auxiliary diagnosis and treatment.
Three-dimensional ultrasound reconstruction algorithms can be divided into three categories depending on the implementation: voxel-based methods, pixel-based methods, and function-based methods. Voxel-based methods include voxel nearest neighbor and distance weighted interpolation. Pixel-based methods include pixel nearest neighbor and kernel regression interpolation. The method based on the function comprises interpolation calculation functions such as a radial basis function and a Bessel function, wherein the radial basis function is subjected to block calculation and then is smoothly connected through global interpolation, the efficiency is low, and the Bessel function improves the reconstruction efficiency, but requires all collected images to be arranged according to the position sequence, namely single one-way scanning, so that the application scene is limited.
In practical applications, the following situations exist: the image is unevenly distributed in space, so that global parameters are difficult to select, the local self-adaptive method is low in calculation efficiency, extra filling operation is required aiming at possible holes, and the function-based interpolation method is limited by a function model. Recently, due to the development of a deep learning technology, Raphael et al adopt CNN to estimate motion information from an ultrasonic sequence, and combine an inertia measurement unit to realize a reconstruction task under a complex condition, Guo et al adopt 3D convolution to extract features in the ultrasonic sequence, combine an attention module to pay attention to a speckle region where motion information is easy to extract, and carry out 3D ultrasonic reconstruction under the condition without any tracking device. However, compared with the error of the conventional tracking device, the spatial position of the pixel restored by the deep learning technology at the present stage is not accurate enough, and is difficult to be used clinically. Therefore, a new reconstruction method is needed to meet the requirements of practical applications.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a three-dimensional reconstruction method of an ultrasonic image, which can reduce the influence of parameter imbalance caused by non-uniform acquisition density, has lower reconstruction error and increased application range, has independence in the interpolation process of different voxels, can perform parallel calculation, and greatly improves reconstruction efficiency.
The technical scheme of the invention is as follows: the three-dimensional reconstruction method of the ultrasonic image comprises the following steps:
(1) and (3) space mapping: regarding pixels in a two-dimensional image obtained from an ultrasonic acquisition system as plane point clouds only with values in the center, and obtaining the point clouds in a corresponding space coordinate system through space mapping;
(2) dividing a tetrahedron: dividing the space under the new coordinate system into a plurality of tetrahedrons which are formed by taking the point cloud as a vertex;
(3) tetrahedral interpolation: interpolating lattice points in a tetrahedron, obtaining tetrahedral vertexes by utilizing pixel mapping, and performing interpolation calculation on the lattice points at the positions corresponding to the voxels, wherein the interpolation method adopts the fusion of volume weight interpolation and spatial second-order polynomial interpolation, and the interpolation of each tetrahedron does not influence each other to perform parallel calculation, so as to obtain the reconstructed three-dimensional data.
According to the method, the purpose of self-adaptive neighborhood segmentation is achieved by dividing the tetrahedron in the space, a proper neighborhood is provided for interpolation points under different acquisition densities, and the influence of parameter imbalance caused by nonuniform acquisition densities is reduced; by adopting an interpolation strategy of fusing volume weight interpolation and space second-order polynomial interpolation for voxels to be interpolated in the space, the method has lower reconstruction error and increases the application range of the method; the interpolation process of different voxels has independence, parallel calculation can be performed, and the reconstruction efficiency is greatly improved.
There is also provided an apparatus for three-dimensional reconstruction of an ultrasound image, comprising:
the spatial mapping module is configured to regard pixels in a two-dimensional image obtained from the ultrasonic acquisition system as plane point clouds only with values in the center, and point clouds in a corresponding spatial coordinate system are obtained through spatial mapping;
a tetrahedron subdivision module configured to divide the space under the new coordinate system into a plurality of tetrahedrons composed of point clouds serving as vertexes;
and the tetrahedral interpolation module is configured to interpolate lattice points in a tetrahedron, obtain tetrahedral vertexes by utilizing pixel mapping, and perform interpolation calculation on the lattice points at the corresponding positions of the voxels, wherein the interpolation method adopts the fusion of volume weight interpolation and spatial second-order polynomial interpolation, and the interpolation of each tetrahedron does not influence each other to perform parallel calculation, so that the reconstructed three-dimensional volume data is obtained.
Drawings
Fig. 1 is a flowchart of a method of three-dimensional reconstruction of an ultrasound image according to the present invention.
Fig. 2 is a block diagram of a method of three-dimensional reconstruction of an ultrasound image according to the present invention.
Fig. 3 shows a subdivision interpolation diagram of the three-dimensional reconstruction method of an ultrasound image according to the present invention, (a) a spatial point cloud distribution diagram, (b) a tetrahedral subdivision result, and (c) a tetrahedral interpolation diagram.
Detailed Description
As shown in fig. 1, 2 and 3, the method for three-dimensional reconstruction of an ultrasound image includes the following steps:
(1) spatial mapping: regarding pixels in a two-dimensional image obtained from an ultrasonic acquisition system as plane point clouds only with values in the center, and obtaining the point clouds in a corresponding space coordinate system through space mapping;
(2) dividing a tetrahedron: dividing the space under the new coordinate system into a plurality of tetrahedrons which are formed by taking the point cloud as a vertex; FIG. 3(a) shows: the grid points represent the positions of the point cloud obtained after the pixels are mapped to the spatial coordinate system, and fig. 3(b) shows: the vertices are tetrahedrons made up of lattice points.
(3) Tetrahedral interpolation: interpolating lattice points within the tetrahedron, as shown in fig. 3 (c): the method comprises the steps of utilizing pixel mapping to obtain tetrahedral vertexes, carrying out interpolation calculation on grid points (hollow points indicated by arrows) at positions corresponding to voxels, fusing volume weight interpolation and spatial second-order polynomial interpolation, and enabling interpolation of each tetrahedron not to influence each other to carry out parallel calculation to obtain reconstructed three-dimensional volume data.
The method achieves the purpose of self-adaptive neighborhood segmentation by tetrahedrally subdividing the space, provides a proper neighborhood for interpolation points under different acquisition densities, and reduces the influence of unbalanced parameters caused by uneven acquisition densities; by adopting an interpolation strategy of fusing volume weight interpolation and space second-order polynomial interpolation for voxels to be interpolated in the space, the method has lower reconstruction error and increases the application range of the method; the interpolation process of different voxels has independence, parallel calculation can be performed, and the reconstruction efficiency is greatly improved.
Preferably, in the step (1), the spatial position transform matrix T ═ T obtained by the tracking device is used 1 ,…,T M And mapping the plane point cloud to the same space coordinate system to obtain a space point cloud.
Preferably, in the step (2), the split tetrahedron has uniqueness and most regularization, i.e. the minimum internal angle is the largest.
Preferably, in the step (3), grid points in the tetrahedron are interpolated by adopting a method of fusing volume weight interpolation and spatial second-order polynomial interpolation, and continuous volume weight interpolation is performed when the grid points are close to the surface of the tetrahedron; when the lattice point is close to the center of the tetrahedron, spatial second-order polynomial interpolation is executed, and the fitting capability is stronger.
Preferably, in the step (3), interpolation operations of each tetrahedron are not affected by each other, and have independence, and a GPU (Graphics Processing Unit) may be used to accelerate operations in parallel, so as to obtain the reconstructed three-dimensional volume data.
It will be understood by those skilled in the art that all or part of the steps in the method of the above embodiments may be implemented by hardware instructions related to a program, the program may be stored in a computer-readable storage medium, and when executed, the program includes the steps of the method of the above embodiments, and the storage medium may be: ROM/RAM, magnetic disks, optical disks, memory cards, and the like. Therefore, the invention also includes a three-dimensional reconstruction device of the ultrasonic image corresponding to the method of the invention. As shown in fig. 1, the apparatus includes:
the spatial mapping module is configured to regard pixels in a two-dimensional image obtained from the ultrasonic acquisition system as plane point clouds only with values in the center, and point clouds in a corresponding spatial coordinate system are obtained through spatial mapping;
a tetrahedron subdivision module configured to divide the space under the new coordinate system into a plurality of tetrahedrons composed of point clouds serving as vertexes;
and the tetrahedral interpolation module is configured to interpolate lattice points in a tetrahedron, obtain tetrahedral vertexes by utilizing pixel mapping, and perform interpolation calculation on the lattice points at the corresponding positions of the voxels, wherein the interpolation method adopts the fusion of volume weight interpolation and spatial second-order polynomial interpolation, and the interpolation of each tetrahedron does not influence each other to perform parallel calculation, so that the reconstructed three-dimensional volume data is obtained.
Preferably, in the spatial mapping module, the spatial position transformation matrix T ═ T obtained by the tracking device is used 1 ,…,T M And mapping the plane point cloud to the same space coordinate system to obtain a space point cloud.
Preferably, in the tetrahedron subdivision module, the minimum internal angle of the subdivided tetrahedron is the largest.
Preferably, in the tetrahedral interpolation module, a method of fusing volume weight interpolation and spatial second-order polynomial interpolation is adopted to interpolate lattice points in a tetrahedron, continuous volume weight interpolation is performed when the lattice points are close to the surface of the tetrahedron, and spatial second-order polynomial interpolation is performed when the lattice points are close to the center of the tetrahedron.
Preferably, in the tetrahedral interpolation module, a GPU is used to perform parallel acceleration operation to obtain the reconstructed three-dimensional volume data.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and all simple modifications, equivalent variations and modifications made to the above embodiment according to the technical spirit of the present invention still belong to the protection scope of the technical solution of the present invention.

Claims (10)

1. The three-dimensional reconstruction method of the ultrasonic image is characterized by comprising the following steps: which comprises the following steps:
(1) spatial mapping: regarding pixels in a two-dimensional image obtained from an ultrasonic acquisition system as plane point clouds only with values in the center, and obtaining the point clouds in a corresponding space coordinate system through space mapping;
(2) dividing a tetrahedron: dividing the space under the new coordinate system into a plurality of tetrahedrons which are formed by taking the point cloud as a vertex;
(3) tetrahedral interpolation: interpolating lattice points in a tetrahedron, obtaining tetrahedral vertexes by utilizing pixel mapping, and performing interpolation calculation on the lattice points at the positions corresponding to the voxels, wherein the interpolation method adopts the fusion of volume weight interpolation and spatial second-order polynomial interpolation, and the interpolation of each tetrahedron does not influence each other to perform parallel calculation, so as to obtain the reconstructed three-dimensional data.
2. The method for three-dimensional reconstruction of an ultrasound image according to claim 1, characterized in that: in the step (1), the spatial position transform matrix T obtained by the tracking device is { T ═ T } 1 ,…,T M And mapping the plane point cloud to the same space coordinate system to obtain a space point cloud.
3. The method for three-dimensional reconstruction of an ultrasound image according to claim 2, characterized in that: in the step (2), the minimum internal angle of the split tetrahedron is the largest.
4. The method for three-dimensional reconstruction of an ultrasound image according to claim 3, wherein: in the step (3), a method of fusing volume weight interpolation and spatial second-order polynomial interpolation is adopted to interpolate lattice points in the tetrahedron, continuous volume weight interpolation is executed when the lattice points are close to the surface of the tetrahedron, and spatial second-order polynomial interpolation is executed when the lattice points are close to the center of the tetrahedron.
5. The method for three-dimensional reconstruction of an ultrasound image according to claim 4, wherein: in the step (3), the GPU is utilized to perform parallel acceleration operation to obtain the reconstructed three-dimensional data.
6. The three-dimensional reconstruction device of the ultrasonic image is characterized in that: it includes:
the spatial mapping module is configured to regard pixels in a two-dimensional image obtained from the ultrasonic acquisition system as plane point clouds only with values in the center, and point clouds in a corresponding spatial coordinate system are obtained through spatial mapping;
a tetrahedron subdivision module configured to divide the space under the new coordinate system into a plurality of tetrahedrons composed of point clouds serving as vertexes;
and the tetrahedral interpolation module is configured to interpolate lattice points in a tetrahedron, obtain tetrahedral vertexes by utilizing pixel mapping, and perform interpolation calculation on the lattice points at the corresponding positions of the voxels, wherein the interpolation method adopts the fusion of volume weight interpolation and spatial second-order polynomial interpolation, and the interpolation of each tetrahedron does not influence each other to perform parallel calculation, so that the reconstructed three-dimensional volume data is obtained.
7. The apparatus for three-dimensional reconstruction of an ultrasound image according to claim 6, wherein: in the spatial mapping module, a spatial position transformation matrix T obtained by the tracking device is set to { T ═ T 1 ,…,T M And mapping the plane point cloud to the same space coordinate system to obtain a space point cloud.
8. The apparatus for three-dimensional reconstruction of an ultrasound image according to claim 7, wherein: in the tetrahedron subdivision module, the minimum internal angle of a subdivided tetrahedron is the largest.
9. The apparatus for three-dimensional reconstruction of an ultrasound image according to claim 8, wherein: in the tetrahedral interpolation module, a method of fusing volume weight interpolation and spatial second-order polynomial interpolation is adopted to interpolate lattice points in a tetrahedron, continuous volume weight interpolation is executed when the lattice points are close to the surface of the tetrahedron, and spatial second-order polynomial interpolation is executed when the lattice points are close to the center of the tetrahedron.
10. The apparatus for three-dimensional reconstruction of an ultrasound image according to claim 9, wherein: in the tetrahedral interpolation module, GPU (graphics processing unit) is utilized to perform parallel acceleration operation, and the reconstructed three-dimensional volume data is obtained.
CN202210681854.5A 2022-06-15 2022-06-15 Three-dimensional reconstruction method and device for ultrasonic image Active CN114998150B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210681854.5A CN114998150B (en) 2022-06-15 2022-06-15 Three-dimensional reconstruction method and device for ultrasonic image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210681854.5A CN114998150B (en) 2022-06-15 2022-06-15 Three-dimensional reconstruction method and device for ultrasonic image

Publications (2)

Publication Number Publication Date
CN114998150A true CN114998150A (en) 2022-09-02
CN114998150B CN114998150B (en) 2024-04-30

Family

ID=83036052

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210681854.5A Active CN114998150B (en) 2022-06-15 2022-06-15 Three-dimensional reconstruction method and device for ultrasonic image

Country Status (1)

Country Link
CN (1) CN114998150B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117414154A (en) * 2023-09-05 2024-01-19 骨圣元化机器人(深圳)有限公司 Three-dimensional ultrasonic reconstruction method, device and ultrasonic system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102799657A (en) * 2012-07-05 2012-11-28 上海富瀚微电子有限公司 System and method for realizing real-time data point mapping processing based on three-dimensional checking
CN109636912A (en) * 2018-11-27 2019-04-16 中国地质大学(武汉) Tetrahedron subdivision finite element interpolation method applied to three-dimensional sonar image reconstruction
EP3961551A1 (en) * 2019-04-25 2022-03-02 Spreadtrum Communications (Shanghai) Co., Ltd. Tetrahedral interpolation calculation method and apparatus, gamut conversion method and apparatus, and medium
CN114463480A (en) * 2020-11-09 2022-05-10 北京理工大学 Ultrasonic volume reconstruction method and device based on pose parameter regularization

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102799657A (en) * 2012-07-05 2012-11-28 上海富瀚微电子有限公司 System and method for realizing real-time data point mapping processing based on three-dimensional checking
CN109636912A (en) * 2018-11-27 2019-04-16 中国地质大学(武汉) Tetrahedron subdivision finite element interpolation method applied to three-dimensional sonar image reconstruction
EP3961551A1 (en) * 2019-04-25 2022-03-02 Spreadtrum Communications (Shanghai) Co., Ltd. Tetrahedral interpolation calculation method and apparatus, gamut conversion method and apparatus, and medium
CN114463480A (en) * 2020-11-09 2022-05-10 北京理工大学 Ultrasonic volume reconstruction method and device based on pose parameter regularization

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117414154A (en) * 2023-09-05 2024-01-19 骨圣元化机器人(深圳)有限公司 Three-dimensional ultrasonic reconstruction method, device and ultrasonic system

Also Published As

Publication number Publication date
CN114998150B (en) 2024-04-30

Similar Documents

Publication Publication Date Title
Beers et al. High-resolution medical image synthesis using progressively grown generative adversarial networks
US11653900B2 (en) Data augmentation for training deep learning models with ultrasound images
CN109754394B (en) Three-dimensional medical image processing device and method
CN110675489B (en) Image processing method, device, electronic equipment and storage medium
KR101805619B1 (en) Apparatus and method for creating optimal 2-dimensional medical image automatically from 3-dimensional medical image
Huang et al. Bezier interpolation for 3-D freehand ultrasound
EP2203894B1 (en) Visualization of voxel data
CN102436672B (en) Ultrasonic image processing apparatus
JP4424799B2 (en) Three-dimensional imaging method and system
JP2016135252A (en) Medical image processing apparatus and medical image diagnostic apparatus
CN114037714A (en) 3D MR and TRUS image segmentation method for prostate system puncture
US8059914B2 (en) Method and apparatus for preprocessing ultrasound imaging
JP2012105750A (en) Ultrasonic image processing apparatus
CN103860197B (en) Free style 3-D supersonic imaging methods, devices and systems
CN117413300A (en) Method and system for training quantized nerve radiation field
CN103761767A (en) Quick three-dimensional ultrasound image reconstruction method based on sparse data
CN110599535A (en) High-resolution human body real-time dynamic reconstruction method and device based on hash table
CN114998150B (en) Three-dimensional reconstruction method and device for ultrasonic image
US20070201737A1 (en) System And Method For Vascular Visualization Using Planar Reformation Of Vascular Central Axis Surface With Biconvex Slab
JP7019594B2 (en) Imaging system and method
CN104224230B (en) Three-dimensional and four-dimensional ultrasonic imaging method and device based on GPU (Graphics Processing Unit) platform and system
Frisken et al. Efficient estimation of 3D euclidean distance fields from 2D range images
US6306092B1 (en) Method and apparatus for calibrating rotational offsets in ultrasound transducer scans
CN115239559A (en) Depth map super-resolution method and system for fusion view synthesis
Wen et al. GPU-based volume reconstruction for freehand 3D ultrasound imaging

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant