CN116051754A - Three-dimensional reconstruction device, method and system based on FPGA and storage medium - Google Patents
Three-dimensional reconstruction device, method and system based on FPGA and storage medium Download PDFInfo
- Publication number
- CN116051754A CN116051754A CN202310204311.9A CN202310204311A CN116051754A CN 116051754 A CN116051754 A CN 116051754A CN 202310204311 A CN202310204311 A CN 202310204311A CN 116051754 A CN116051754 A CN 116051754A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- image
- images
- reconstructed
- reconstruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the application provides a three-dimensional reconstruction device, method, system and storage medium based on an FPGA, and relates to the technical field of three-dimensional reconstruction. Wherein the device includes: the image acquisition module is used for acquiring at least one two-dimensional image of the target to be reconstructed; wherein each two-dimensional image corresponds to a projection angle of the target to be reconstructed; the image conversion module is used for converting the two-dimensional images of the projection angles in parallel to obtain intensity images corresponding to the projection angles; and the three-dimensional reconstruction module is used for carrying out three-dimensional reconstruction in parallel based on the intensity images of the projection angles to obtain the three-dimensional image of the target to be reconstructed. The embodiment of the application solves the problems of high cost and large volume of the three-dimensional reconstruction system in the related technology.
Description
Technical Field
The application relates to the technical field of three-dimensional reconstruction, in particular to a three-dimensional reconstruction device, method and system based on an FPGA and a storage medium.
Background
Three-dimensional reconstruction is a hotspot in medical imaging technology, and at present, three-dimensional reconstruction of an object is mainly achieved in two ways, namely, a large number of two-dimensional images based on the object are spliced to obtain a three-dimensional image, and three-dimensional frequency domain to airspace, airspace to frequency domain and the like are transformed based on three-dimensional information of the object to obtain a three-dimensional image.
However, since the three-dimensional image contains a large amount of information, no matter what way is used to implement three-dimensional reconstruction of the object, huge calculation overhead is generated, so the current three-dimensional reconstruction system mainly uses a high-performance computer or GPU to implement three-dimensional reconstruction of the object, but the high-performance computer or GPU is used to implement three-dimensional reconstruction, which has many disadvantages, on one hand, the high-performance computer or GPU has high cost, and on the other hand, the high-performance computer or GPU has large volume and is inconvenient to be embedded into a device convenient for transplantation.
Disclosure of Invention
The embodiments of the application provide a three-dimensional reconstruction device, method, system and storage medium based on an FPGA, which can solve the problems of high cost and large volume of a three-dimensional reconstruction system in the related technology. The technical scheme is as follows:
according to one aspect of embodiments of the present application, an FPGA-based three-dimensional reconstruction apparatus, the apparatus comprising: the image acquisition module is used for acquiring at least one two-dimensional image of the target to be reconstructed; wherein each two-dimensional image corresponds to a projection angle of the target to be reconstructed; the image conversion module is used for converting the two-dimensional images of the projection angles in parallel to obtain intensity images corresponding to the projection angles; and the three-dimensional reconstruction module is used for carrying out three-dimensional reconstruction in parallel based on the intensity images of the projection angles to obtain the three-dimensional image of the target to be reconstructed.
According to one aspect of an embodiment of the present application, a three-dimensional reconstruction method based on an FPGA, the method includes: acquiring at least one two-dimensional image of a target to be reconstructed; wherein each two-dimensional image corresponds to a projection angle of the target to be reconstructed; transforming the two-dimensional images of the projection angles in parallel to obtain intensity images corresponding to the projection angles; and carrying out three-dimensional reconstruction in parallel on the basis of the intensity images of the projection angles to obtain the three-dimensional image of the target to be reconstructed.
According to one aspect of embodiments of the present application, an FPGA-based three-dimensional reconstruction system includes an FPGA-based three-dimensional reconstruction device as described above.
According to one aspect of embodiments of the present application, a storage medium has stored thereon a computer program which, when executed by a processor, implements the FPGA-based three-dimensional reconstruction method as described above.
According to one aspect of embodiments of the present application, a computer program product comprises a computer program stored in a storage medium, a processor of a computer device reading the computer program from the storage medium, the processor executing the computer program such that the computer device, when executing, implements the FPGA-based three-dimensional reconstruction method as described above.
The beneficial effects that this application provided technical scheme brought are:
in the technical scheme, based on each two-dimensional image of the target to be reconstructed, the FPGA unit in the three-dimensional reconstruction system is utilized for parallel reconstruction to obtain a three-dimensional image of the target to be reconstructed; on one hand, the parallel acceleration processing mechanism based on the FPGA unit reduces the power consumption of three-dimensional reconstruction, ensures the speed of three-dimensional reconstruction, and on the other hand, the FPGA unit has low cost and small volume, is convenient to be embedded into equipment convenient for transplanting, and effectively solves the problems of high cost and large volume of the three-dimensional reconstruction system in the related technology.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that are required to be used in the description of the embodiments of the present application will be briefly described below.
FIG. 1 is a block diagram of a three-dimensional FPGA-based reconstruction system according to the present application;
FIG. 2 is a flow chart illustrating a method of three-dimensional reconstruction based on an FPGA, according to an exemplary embodiment;
FIG. 3 is a schematic diagram of a specific implementation of three-dimensional reconstruction according to the corresponding embodiment of FIG. 2;
FIG. 4 is a schematic diagram of an intensity image obtained using two-dimensional images according to the embodiment of FIG. 2;
FIG. 5 is a flow chart of step 330 in one embodiment of the corresponding embodiment of FIG. 2;
FIG. 6 is a flow chart of step 350 in one embodiment of the corresponding embodiment of FIG. 2;
FIG. 7 is a schematic illustration of a specific implementation of cross-sectional intensity data according to the corresponding embodiment of FIG. 6;
FIGS. 8 to 9 are schematic diagrams showing specific implementation of a three-dimensional reconstruction method based on an FPGA;
FIG. 10 is a block diagram illustrating a structure of an FPGA-based three-dimensional reconstruction device according to an exemplary embodiment;
fig. 11 is a block diagram illustrating a structure of an electronic device according to an exemplary embodiment.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for the purpose of illustrating the present application and are not to be construed as limiting the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.
Cell and tissue microscopic imaging is an important imaging technology in the field of modern biomedicine, and provides great assistance and strong evidence for doctors and researchers to study medical science and life science. However, the conventional two-dimensional imaging method is insufficient to meet the increasingly developed scientific research and medical diagnosis requirements, and the three-dimensional reconstruction technology has been developed. The three-dimensional reconstruction technology not only can display the microstructure inside the measured object in an omnibearing manner, but also can reflect the relation between the details inside the measured object and can provide information far greater than two-dimensional images, so the three-dimensional reconstruction technology is a current research hot spot. One classical method of three-dimensional reconstruction is tomographic reconstruction. The chromatography refers to the steps of obtaining the information of the measured object layer by layer and then reconstructing three-dimensionally through an algorithm. In microscopic imaging, the classical method of acquiring object information in layers is holographic imaging.
Holographic imaging is the advance of microscopic three-dimensional tomographic reconstruction, which utilizes multi-angle holograms to reconstruct images of three-dimensional objects. The holographic tomographic three-dimensional reconstruction algorithm may be classified into a tomographic reconstruction taking into consideration diffraction effects and a tomographic reconstruction taking into consideration no diffraction effects, depending on whether diffraction effects are considered. Considering that the chromatography of sample diffraction considers that light propagates along a straight line in a sample and the diffraction phenomenon occurs when light waves pass through an object, the algorithm is based on a Fourier diffraction projection theory and can reconstruct a three-dimensional image with higher resolution, but the algorithm theory is complex, and a Fourier mapping algorithm, a filtering counter-propagation algorithm and the like are based on the fact that three-dimensional Fourier transformation and the like can be utilized; the method considers that light passes through the sample linearly without diffraction effect regardless of the chromatography of the sample, and is based on the Fourier center slice theorem, and the main algorithms comprise filtered back projection, direct back projection, iterative reconstruction algorithm and the like.
The existing holographic chromatography three-dimensional reconstruction technology is mostly realized based on a high-performance computer or a GPU, and a method for realizing three-dimensional reconstruction by utilizing a CPU on the high-performance computer has high requirements on computer performance and low speed, and can accelerate calculation on the GPU, but the GPU has high cost and high power consumption, is not convenient to be embedded into equipment convenient to transplant, for example, is not suitable for an embedded portable microscope.
From the above, the related art still has the defects of high cost and large volume of the three-dimensional reconstruction system.
Therefore, the three-dimensional reconstruction device based on the FPGA can effectively reduce the volume of a three-dimensional reconstruction system while guaranteeing the three-dimensional reconstruction speed, reduce the cost of the three-dimensional reconstruction system and solve the problems of high cost and large volume of the three-dimensional reconstruction device in the related technology. The three-dimensional reconstruction device based on the FPGA can be deployed in a three-dimensional reconstruction system and can also be deployed in electronic equipment, for example, the electronic equipment can be a desktop computer, a notebook computer and a server which are configured with a von neumann system structure, and can also be mobile equipment carrying a singlechip and the like. Further, a three-dimensional reconstruction method based on the FPGA is provided, and accordingly, the three-dimensional reconstruction method based on the FPGA is suitable for a three-dimensional reconstruction device based on the FPGA.
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 shows a schematic block diagram of an FPGA-based three-dimensional reconstruction system, and as shown in fig. 1, the three-dimensional reconstruction system 100 includes a field programmable gate array FPGA module 110; the FPGA module 110 may include a plurality of transformation units 111, a plurality of reconstruction units 113, and memory banks 115, an AXI bus 117, and a high-speed interface 119.
The FPGA module 110 is configured to obtain at least one two-dimensional image of a target to be reconstructed; wherein each two-dimensional image corresponds to a projection angle of the object to be reconstructed; transforming the two-dimensional images of the projection angles in parallel to obtain intensity images corresponding to the projection angles; and carrying out three-dimensional reconstruction in parallel on the basis of the intensity images of the projection angles to obtain a three-dimensional image of the target to be reconstructed.
In one possible implementation, the FPGA-based three-dimensional reconstruction device may be deployed in the FPGA module 110.
Each transformation unit 111 is configured to perform fourier forward transformation on the two-dimensional image of each projection angle, so as to obtain each two-dimensional image subjected to fourier forward transformation; and performing inverse Fourier transform based on each two-dimensional image subjected to the Fourier forward transform to obtain an intensity image of each projection angle.
Each reconstruction unit 113 is configured to divide a target to be reconstructed into a plurality of sections; acquiring section intensity data corresponding to each section based on the intensity images of each section and each projection angle; the section intensity data comprise intensity images of projection angles corresponding to a section; respectively carrying out filtering back projection on the intensity data of each section in parallel to obtain slice images corresponding to each section; and carrying out three-dimensional reconstruction based on each slice image to obtain a three-dimensional image of the target to be reconstructed.
In one embodiment, the reconstruction unit 113 is further configured to perform filtering back projection on the intensity images of the projection angles corresponding to one section in parallel based on one section intensity data in the section intensity data, so as to obtain a slice image corresponding to the section intensity data.
Further, in an embodiment, the three-dimensional reconstruction system may further include an image scanning module, configured to scan the object to be reconstructed based on different projection angles, so as to obtain two-dimensional images corresponding to the projection angles.
Referring to the three-dimensional reconstruction system of fig. 1, a description will be given of a three-dimensional reconstruction process, specifically, as shown in fig. 1, a two-dimensional image of a target to be reconstructed is transferred into an FPGA module 110 through a high-speed interface 119, and the two-dimensional image of each projection angle enters a transformation unit 111, and is transformed in parallel in each transformation unit 111, where the number of transformation units 111 of the FPGA module 110 depends on the speed of generating the two-dimensional image and the processing speed of the transformation units 111, and the number of transformation units can be adaptively adjusted to ensure maximum transformation efficiency.
In one embodiment, the transformation unit may further include a first transformation unit and a second transformation unit, where the first transformation unit is configured to perform fourier positive transformation on the two-dimensional image, to obtain a two-dimensional image with fourier positive transformation completed; the second transformation unit is used for carrying out inverse Fourier transformation on the two-dimensional image subjected to the Fourier forward transformation to obtain an intensity image.
Subsequently, each intensity image generated by the transformation unit 111 is stored in the memory bank 115 through the AXI bus 117 in the FPGA module 110, and when the intensity images of all projection angles are stored, each reconstruction unit 113 performs parallel reconstruction based on each intensity image, so as to obtain a three-dimensional image of the target to be reconstructed.
In one embodiment, the reconstruction unit 113 may include a data indexing unit, a filtering unit and an interpolation back projection unit, where the indexing module is configured to divide the object to be reconstructed into a plurality of sections, so as to obtain section intensity data corresponding to each section; the filtering unit is used for filtering the intensity data of each section so as to remove noise; the interpolation back projection unit is used for carrying out filtering back projection on the intensity data of each section so as to obtain slice images corresponding to each section, and obtaining a three-dimensional image of the target to be reconstructed according to the slice images corresponding to each section.
Referring to fig. 2, an embodiment of the present application provides a three-dimensional reconstruction method based on an FPGA, which is suitable for the three-dimensional reconstruction system 100 in the schematic block diagram shown in fig. 1.
In the following method embodiments, for convenience of description, the execution subject of each step of the method is described as an example of the three-dimensional reconstruction system, but this configuration is not particularly limited.
As shown in fig. 2, the method may include the steps of:
at step 310, at least one two-dimensional image of the object to be reconstructed is acquired.
The two-dimensional image may be derived from a two-dimensional image obtained by scanning the object to be reconstructed in real time, or may be a two-dimensional image stored in advance in an electronic device for a historical period of time. Then, for the electronic apparatus, after the two-dimensional image is scanned, the two-dimensional image may be processed in real time, and further, reprocessing may be stored in advance, for example, the two-dimensional image may be processed when the occupation of the electronic apparatus is low, or the two-dimensional image may be processed according to an instruction of a worker. Thus, the three-dimensional reconstruction in the present embodiment may be performed on a two-dimensional image acquired in real time, or may be performed on a two-dimensional image acquired in a history period, which is not particularly limited herein.
Each two-dimensional image corresponds to a projection angle of an object to be reconstructed, and the object to be reconstructed may refer to an object to be reconstructed in three dimensions. The image scanning module can be utilized to scan the target to be reconstructed based on different projection angles, so as to obtain two-dimensional images corresponding to the projection angles.
It should be noted that, the projection angle refers to an angle when an imaging technique such as an X-ray, an electron beam, or a laser beam irradiates an object to be reconstructed from different directions in the three-dimensional reconstruction process. In the three-dimensional reconstruction process, in order to obtain a sufficient two-dimensional image, it is necessary to irradiate the object to be reconstructed from different directions, and collect two-dimensional images of a plurality of projection angles. The two-dimensional image of each projection angle can be regarded as a two-dimensional projection of the object to be reconstructed in the direction, and is important information for reconstructing a three-dimensional image.
In determining the projection angle, a number of factors need to be considered, such as the shape, size, imaging requirements, etc. of the object to be reconstructed. In general, selecting more projection angles will increase the accuracy and quality of the three-dimensional image, but will also increase the time and cost of three-dimensional reconstruction. Therefore, a trade-off between time and quality is required, choosing the most appropriate projection angle. For example, a better reconstruction can be obtained by selecting 30 to 200 projection angles.
The projection angle may be changed by rotating the object to be reconstructed, or the projection angle may be changed by rotating the light ray, which is not limited herein.
Fig. 3 shows a schematic diagram of a specific implementation of three-dimensional reconstruction, as shown in fig. 3, light and coherent light thereof are irradiated on an object to be reconstructed, a two-dimensional image is recorded on a CCD or CMOS sensor, at this time, the projection angle is 0 °, the light is rotated around the Z axis or the object to be reconstructed, so that the light generates a projection angle theta with respect to the object to be reconstructed, the two-dimensional image under the projection angle theta is continuously obtained, and the two-dimensional images under the projection angle theta are sequentially rotated, so that the angles theta are 1 °,2 °,3 °, … …,179 °, and thus, the two-dimensional image of the object to be reconstructed at 180 degrees can be obtained.
The transformation of the two-dimensional images can be achieved by angular spectrometry, and a schematic diagram of obtaining an intensity image from each two-dimensional image is shown in fig. 4, and in a rectangular coordinate system, the object plane coordinates are set to be
The light wave field is
Through a certain spatial distance->
The diffraction screen coordinates after diffraction are +.>
Diffraction field +.>
. Reference fourier transform and inverse transform symbols>
The angular spectrum diffraction integral can be written as formula (1) and formula (2):
Where fx, fy is the frequency domain coordinates, H (fx, fy) is the optical transfer function, j is the imaginary unit, λ is the wavelength of light, 2pi/λ=k. And reversely solving U0 (x 0, y 0) according to the above formula to obtain an intensity image, wherein the intensity image is represented by the following formula (3):
Where j is an imaginary unit, λ is the wavelength of light, 1/λ=k.
From the above, the angular spectrum method uses a two-dimensional fourier transform and an inverse two-dimensional fourier transform.
In one embodiment, the fourier transform may be implemented by a transform unit in the FPGA unit, in particular comprising the steps of: performing Fourier forward transformation on the two-dimensional images of each projection angle to obtain each two-dimensional image subjected to Fourier forward transformation; and performing inverse Fourier transform based on each two-dimensional image subjected to the Fourier forward transform to obtain an intensity image of each projection angle.
It is further explained that, since there are a plurality of transformation units in the FPGA unit, a plurality of two-dimensional images can be transformed in parallel by the plurality of transformation units, specifically, in one embodiment, as shown in fig. 5, the method comprises the steps of:
First, a plurality of conversion units may be used in parallel according to the speed of two-dimensional image input and an appropriate amount of calculation resources of the FPGA unit. For example, if the first two-dimensional image is not processed in the first transformation unit and the second two-dimensional image is input, the second two-dimensional image may be processed using a second transformation unit in an idle state.
Based on this, the set number may be determined according to the number of transformation units in the three-dimensional system to achieve the maximum parallel transformation efficiency, for example, the set number is equal to the number of transformation units, which is not limited herein.
Through the embodiment, the delay in the two-dimensional image processing process can be reduced by utilizing the parallel computation of the plurality of transformation units in the FPGA unit, and the speed of the two-dimensional image processing is increased, so that the efficiency of three-dimensional reconstruction is improved.
And 350, carrying out three-dimensional reconstruction in parallel based on the intensity images of the projection angles to obtain a three-dimensional image of the target to be reconstructed.
Wherein, each reconstruction unit in the three-dimensional system can be utilized to perform three-dimensional reconstruction, as shown in fig. 6, comprising the following steps:
And step 355, filtering back projection is performed on the section intensity data in parallel to obtain slice images corresponding to the sections.
In step 357, three-dimensional reconstruction is performed based on each slice image, resulting in a three-dimensional image of the object to be reconstructed.
The section intensity data comprises intensity images of projection angles corresponding to one section.
Referring back to fig. 3 for the cross-sectional intensity data, as shown in fig. 3, when z=0, the cross-section of the object to be reconstructed may correspond to the intensity images of each projection angle, so as to form an intensity image of 180 projection angles, and then the intensity images of 180 projection angles are the cross-sectional intensity data.
For example, as shown in fig. 7, a circle represents a cross section, light is scanned around the cross section, a one-dimensional intensity image can be obtained from the intensity image of each projection angle, and the intensity images of multiple angles are arranged according to angles, so as to form a cross section intensity data.
Further, a slice image of z=0 of the object to be reconstructed can be reconstructed from the cross-sectional intensity data according to the filtered back projection method. Therefore, the slice images corresponding to the cross sections can be reconstructed by traversing all z, and the three-dimensional images of the target to be reconstructed can be obtained by combining all slice images together for three-dimensional reconstruction.
With respect to combining all slice images together for three-dimensional reconstruction, it may be implemented using a computer algorithm, such as a model fitting algorithm, a voxelization algorithm, a view-based voxelization algorithm, or a point cloud-based algorithm, without limitation.
Further, since the FPGA unit includes a plurality of reconstruction units, a plurality of section intensity data can be transformed in parallel by the plurality of reconstruction units. The plurality of reconstruction units may be used in parallel according to the speed of the cross-sectional intensity data input, and the appropriate amount of computational resources of the FPGA unit. For example, if the first slice strength data has not been processed in the first reconstruction unit and the second slice strength data has been input, a second reconstruction unit in an idle state may be used to process the second slice strength data.
Of course, since the section intensity data includes intensity images of projection angles corresponding to one section, in one possible implementation manner, based on one section intensity data in each section intensity data, filtering back projection is performed on the intensity images of projection angles corresponding to one section in parallel by using the parallel computing feature of the reconstruction unit in the FPGA unit, so as to obtain a slice image corresponding to the section intensity data.
It will be appreciated that, since one section intensity data includes intensity images corresponding to each projection angle in one section, if the reconstruction unit simultaneously reconstructs the intensity images corresponding to each projection angle in parallel, the speed of obtaining the corresponding slice image from the section intensity data can be greatly increased.
Through the process, on one hand, the plurality of section intensity data are processed in parallel through the plurality of reconstruction units, so that the frame parallel acceleration processing of the plurality of section intensity data is realized, and the efficiency of three-dimensional reconstruction is accelerated; on the other hand, the filtered back projection is performed on the cross section intensity data in a plurality of rows in parallel by one reconstruction unit, so that the parallel acceleration processing of the cross section intensity data is realized, and the processing speed and the processing capacity of each reconstruction unit are improved.
In summary, in combination with the above embodiments, based on each two-dimensional image of the target to be reconstructed, parallel reconstruction is performed by using the FPGA unit in the three-dimensional reconstruction system, so as to obtain a three-dimensional image of the target to be reconstructed; on one hand, the parallel acceleration processing mechanism based on the FPGA unit reduces the power consumption of three-dimensional reconstruction, ensures the speed of three-dimensional reconstruction, and on the other hand, the FPGA unit has low cost and small volume, is convenient to be embedded into equipment convenient for transplanting, and effectively solves the problems of high cost and large volume of the three-dimensional reconstruction system in the related technology.
Fig. 8 is a schematic diagram of a specific implementation of a three-dimensional reconstruction method based on an FPGA, and fig. 9 is a flowchart of the three-dimensional reconstruction method, and fig. 8 will be described with reference to fig. 9:
through step 701, a two-dimensional image of each projection angle of the object to be detected is acquired by using an image scanning unit.
In step 703, fourier transform is performed on each two-dimensional image in parallel by each transform unit in the FPGA unit, and each intensity image is obtained.
Through step 705, each reconstruction unit in the FPGA unit is used to perform a line and frame double parallel three-dimensional reconstruction based on each intensity image, so as to obtain a three-dimensional image of the object to be reconstructed.
From the above, the three-dimensional reconstruction system of the scheme realizes three-dimensional reconstruction of the target to be reconstructed based on the thought of parallel acceleration of the rows and the frames of the FPGA unit; on one hand, the three-dimensional reconstruction speed of the analog GPU can be achieved, and on the other hand, the three-dimensional reconstruction system is small in size, low in cost and low in power consumption, so that the defects of high cost and large size of the three-dimensional reconstruction system in the related technology are overcome.
The following is an embodiment of the apparatus of the present application, which may be used to execute the FPGA-based three-dimensional reconstruction method related to the present application. For details not disclosed in the device embodiments of the present application, please refer to a method embodiment of the FPGA-based three-dimensional reconstruction method related to the present application.
Referring to fig. 10, in an embodiment of the present application, an FPGA-based three-dimensional reconstruction device 900 is provided, including but not limited to: an image acquisition module 910, an image transformation module 930, and a three-dimensional reconstruction module 950.
Wherein, the image acquisition module 910 is configured to acquire at least one two-dimensional image of an object to be reconstructed; wherein each two-dimensional image corresponds to a projection angle of the object to be reconstructed.
The image transformation module 930 is configured to transform the two-dimensional images of each projection angle in parallel, so as to obtain an intensity image corresponding to each projection angle.
The three-dimensional reconstruction module 950 is configured to perform three-dimensional reconstruction in parallel based on the intensity images of the projection angles, so as to obtain a three-dimensional image of the target to be reconstructed.
In one embodiment, the image transforming module 930 further includes a plurality of transforming units, each transforming unit configured to perform fourier positive transformation on the two-dimensional image of each projection angle, so as to obtain each two-dimensional image subjected to fourier positive transformation; and performing inverse Fourier transform based on each two-dimensional image subjected to the Fourier forward transform to obtain an intensity image of each projection angle.
In one embodiment, the three-dimensional reconstruction module 950 further includes a plurality of reconstruction units, each reconstruction unit for dividing the object to be reconstructed into a plurality of sections; acquiring section intensity data corresponding to each section based on the intensity images of each section and each projection angle; the section intensity data comprise intensity images of projection angles corresponding to a section; respectively carrying out filtering back projection on the intensity data of each section in parallel to obtain slice images corresponding to each section; and carrying out three-dimensional reconstruction based on each slice image to obtain a three-dimensional image of the target to be reconstructed.
In an embodiment, the reconstruction unit is further configured to perform filtering back projection on the intensity images of the projection angles corresponding to one section in parallel based on one section intensity data in the section intensity data, so as to obtain a slice image corresponding to the section intensity data.
It should be noted that, when the three-dimensional reconstruction device based on the FPGA and the three-dimensional reconstruction system based on the FPGA provided in the foregoing embodiments perform three-dimensional reconstruction based on the FPGA, only the division of the functional modules is used to illustrate the three-dimensional reconstruction, in practical application, the functional allocation may be completed by different functional modules according to needs, that is, the internal structures of the three-dimensional reconstruction device based on the FPGA and the three-dimensional reconstruction system based on the FPGA are divided into different functional modules, so as to complete all or part of the functions described above.
In addition, the embodiments of the FPGA-based three-dimensional reconstruction device, the FPGA-based three-dimensional reconstruction system, and the FPGA-based three-dimensional reconstruction method provided in the foregoing embodiments belong to the same concept, and the specific manner in which each module performs the operation has been described in detail in the method embodiment, which is not described herein again.
Referring to fig. 11, in an embodiment of the present application, an electronic device 4000 is provided, where the electronic device 4000 may include: desktop computers, notebook computers, servers, mobile devices carrying a single-chip microcomputer, etc., are suitable for use in the three-dimensional reconstruction system 100 of the fig. 1 implementation environment.
In fig. 11, the electronic device 4000 includes at least one processor 4001, at least one communication bus 4002, and at least one memory 4003.
Wherein the processor 4001 is coupled to the memory 4003, such as via a communication bus 4002. Optionally, the electronic device 4000 may further comprise a transceiver 4004, the transceiver 4004 may be used for data interaction between the electronic device and other electronic devices, such as transmission of data and/or reception of data, etc. It should be noted that, in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 is not limited to the embodiment of the present application.
The processor 4001 may be a CPU (Central Processing Unit ), general purpose processor, DSP (Digital Signal Processor, data signal processor), ASIC (Application Specific Integrated Circuit ), FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic device, transistor logic device, hardware components, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. The processor 4001 may also be a combination that implements computing functionality, e.g., comprising one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.
The communication bus 4002 may include a pathway to transfer information between the aforementioned components. The communication bus 4002 may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture ) bus, or the like. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 11, but not only one bus or one type of bus.
The memory 4003 has stored thereon a computer program, and the processor 4001 reads the computer program stored in the memory 4003 through the communication bus 4002.
The computer program, when executed by the processor 4001, implements the FPGA-based three-dimensional reconstruction method in the above embodiments.
Further, in the embodiments of the present application, a storage medium is provided, on which a computer program is stored, which when executed by a processor, implements the FPGA-based three-dimensional reconstruction method in the above embodiments.
In an embodiment of the present application, a computer program product is provided, which includes a computer program stored in a storage medium. The processor of the computer device reads the computer program from the storage medium, and the processor executes the computer program, so that the computer device executes the FPGA-based three-dimensional reconstruction method in the above embodiments.
Compared with the related art, the method and the device have the advantages that the FPGA units in the three-dimensional reconstruction system are utilized for parallel reconstruction, and the three-dimensional image of the target to be reconstructed is obtained; on one hand, the parallel acceleration processing mechanism based on the FPGA unit reduces the power consumption of three-dimensional reconstruction, can achieve the three-dimensional reconstruction speed similar to that of the GPU, and on the other hand, the FPGA unit has low cost, small volume and small energy consumption, is convenient to be embedded into equipment which is convenient to transplant, such as a microscope, and effectively solves the problems of high cost and large volume of the three-dimensional reconstruction system in the related technology.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for a person skilled in the art, several improvements and modifications can be made without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.
Claims (10)
1. An FPGA-based three-dimensional reconstruction device, the device comprising:
the image acquisition module is used for acquiring at least one two-dimensional image of the target to be reconstructed; wherein each two-dimensional image corresponds to a projection angle of the target to be reconstructed;
the image conversion module is used for converting the two-dimensional images of the projection angles in parallel to obtain intensity images corresponding to the projection angles;
and the three-dimensional reconstruction module is used for carrying out three-dimensional reconstruction in parallel based on the intensity images of the projection angles to obtain the three-dimensional image of the target to be reconstructed.
2. The apparatus of claim 1, wherein the image transformation module comprises a plurality of transformation units;
each transformation unit is used for carrying out Fourier forward transformation on the two-dimensional images of each projection angle to obtain each two-dimensional image after the Fourier forward transformation is completed; and carrying out inverse Fourier transform on the two-dimensional images after the positive Fourier transform is completed, and obtaining intensity images of the projection angles.
3. The apparatus of claim 1, wherein the three-dimensional reconstruction module further comprises a plurality of reconstruction units;
each reconstruction unit is used for dividing the target to be reconstructed into a plurality of sections; acquiring section intensity data corresponding to each section based on the intensity images of each section and each projection angle; wherein the section intensity data comprises intensity images of the projection angles corresponding to the section; respectively carrying out filtering back projection on the section intensity data in parallel to obtain slice images corresponding to the sections; and carrying out three-dimensional reconstruction based on each slice image to obtain a three-dimensional image of the target to be reconstructed.
4. The apparatus of claim 3, wherein the reconstruction unit is further configured to perform filtered back projection on the intensity image of each projection angle corresponding to one of the cross-sections in parallel based on one of the cross-section intensity data, to obtain a slice image corresponding to the cross-section intensity data.
5. A three-dimensional reconstruction method based on an FPGA, the method comprising:
acquiring at least one two-dimensional image of a target to be reconstructed; wherein each two-dimensional image corresponds to a projection angle of the target to be reconstructed;
transforming the two-dimensional images of the projection angles in parallel to obtain intensity images corresponding to the projection angles;
and carrying out three-dimensional reconstruction in parallel on the basis of the intensity images of the projection angles to obtain the three-dimensional image of the target to be reconstructed.
6. The method according to claim 5, wherein the performing three-dimensional reconstruction on the intensity image based on each projection angle in parallel to obtain the three-dimensional image of the object to be reconstructed includes:
dividing the target to be reconstructed into a plurality of sections;
obtaining section intensity data corresponding to each section based on the intensity images of each section and each projection angle, wherein the section intensity data comprises an intensity image of each projection angle corresponding to one section;
respectively carrying out filtering back projection on the section intensity data in parallel to obtain slice images corresponding to the sections;
and carrying out three-dimensional reconstruction based on each slice image to obtain a three-dimensional image of the target to be reconstructed.
7. The method of claim 5, wherein transforming the two-dimensional images for each projection angle in parallel results in an intensity image for each projection angle, comprising:
selecting a set number of two-dimensional images from the two-dimensional images of each projection angle for parallel transformation;
if the parallel conversion is completed for each two-dimensional image, selecting a set number of two-dimensional images from the rest two-dimensional images to carry out the parallel conversion until the parallel conversion for the two-dimensional images of each projection angle is completed.
8. An FPGA-based three-dimensional reconstruction system, characterized in that the system comprises a three-dimensional reconstruction device according to any one of claims 1 to 4.
9. The system of claim 8, wherein the system further comprises an image scanning module;
the image scanning module is used for scanning the target to be reconstructed based on different projection angles to obtain two-dimensional images corresponding to the projection angles.
10. A storage medium having stored thereon a computer program which, when executed by a processor, implements the FPGA-based three-dimensional reconstruction method of any one of claims 5 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310204311.9A CN116051754B (en) | 2023-03-06 | 2023-03-06 | Three-dimensional reconstruction device, method and system based on FPGA and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310204311.9A CN116051754B (en) | 2023-03-06 | 2023-03-06 | Three-dimensional reconstruction device, method and system based on FPGA and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116051754A true CN116051754A (en) | 2023-05-02 |
| CN116051754B CN116051754B (en) | 2023-09-22 |
Family
ID=86133278
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310204311.9A Active CN116051754B (en) | 2023-03-06 | 2023-03-06 | Three-dimensional reconstruction device, method and system based on FPGA and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116051754B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2324832A1 (en) * | 2000-10-31 | 2002-04-30 | Roger Lecomte | Real-time image reconstruction for computed tomography systems |
| US20040264634A1 (en) * | 2003-06-25 | 2004-12-30 | General Electric Company | Fourier based method, apparatus, and medium for optimal reconstruction in digital tomosynthesis |
| US20180018757A1 (en) * | 2016-07-13 | 2018-01-18 | Kenji Suzuki | Transforming projection data in tomography by means of machine learning |
| US20200340798A1 (en) * | 2018-01-29 | 2020-10-29 | Duke University | Systems and methods for enhanced resolution imaging based on multiple cross-sectional images acquired at different angles |
-
2023
- 2023-03-06 CN CN202310204311.9A patent/CN116051754B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2324832A1 (en) * | 2000-10-31 | 2002-04-30 | Roger Lecomte | Real-time image reconstruction for computed tomography systems |
| US20040264634A1 (en) * | 2003-06-25 | 2004-12-30 | General Electric Company | Fourier based method, apparatus, and medium for optimal reconstruction in digital tomosynthesis |
| US20180018757A1 (en) * | 2016-07-13 | 2018-01-18 | Kenji Suzuki | Transforming projection data in tomography by means of machine learning |
| US20200340798A1 (en) * | 2018-01-29 | 2020-10-29 | Duke University | Systems and methods for enhanced resolution imaging based on multiple cross-sectional images acquired at different angles |
Non-Patent Citations (1)
| Title |
|---|
| 张晓梦等: "基于ＦＰＧＡ实现ＣＴ图像重建加速的设计", 液晶与显示, pages 455 - 459 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116051754B (en) | 2023-09-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yaroslavsky | Digital holography and digital image processing: principles, methods, algorithms | |
| Müller et al. | The theory of diffraction tomography | |
| Myagotin et al. | Efficient volume reconstruction for parallel-beam computed laminography by filtered backprojection on multi-core clusters | |
| Birk et al. | A comprehensive comparison of GPU-and FPGA-based acceleration of reflection image reconstruction for 3D ultrasound computer tomography | |
| Ahrenberg et al. | Using commodity graphics hardware for real-time digital hologram view-reconstruction | |
| KR101978728B1 (en) | Method and device for classifying medical ultrasound image based on deep learning using smart device | |
| Quinto et al. | Local tomography in electron microscopy | |
| Liu et al. | Sparse representation based image super-resolution on the KNN based dictionaries | |
| CN116051754B (en) | Three-dimensional reconstruction device, method and system based on FPGA and storage medium | |
| Omidi et al. | PATLAB: A graphical computational software package for photoacoustic computed tomography research | |
| Franzen et al. | General nonlinearities in so (2)-equivariant cnns | |
| Rogan et al. | Improving the fast back projection algorithm through massive parallelizations | |
| Mujahid et al. | GPU-accelerated multivariate empirical mode decomposition for massive neural data processing | |
| CA2527078A1 (en) | Method for fast image reconstruction with compact radiation source and detector arrangement using computerized tomography | |
| Jiang et al. | Ring and radius sampling based phasor field diffraction algorithm for non-line-of-sight reconstruction | |
| CN111640054A (en) | Three-dimensional reconstruction method based on GPU acceleration | |
| CN112288847A (en) | Light field three-dimensional reconstruction method based on fast Fourier transform | |
| Romero et al. | Using GPUs for beamforming acceleration on SAFT imaging | |
| Chen et al. | An FPGA-based autofocusing hardware architecture for digital holography | |
| Macías-Garza et al. | The missing cone problem and low-pass distortion in optical serial sectioning microscopy | |
| Rivera-Lopez et al. | Fast computation of 3D Tchebichef moments for higher orders | |
| Reid et al. | Challenges in using GPUs for the reconstruction of digital hologram images | |
| Thomas et al. | Computationally efficient filtered-backprojection algorithm for tomographic image reconstruction using Walsh transform | |
| WO2023108787A1 (en) | Gradient-domain matrix elastic network infimal convolution regularization accelerated magnetic resonance parameter imaging method, and terminal device and computer storage medium | |
| Li et al. | Deep Non-line-of-sight Imaging from Under-scanning Measurements |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |