CN116518878A - Efficient 4D reconstruction method and efficient 4D reconstruction system based on dual-mode imaging - Google Patents
Efficient 4D reconstruction method and efficient 4D reconstruction system based on dual-mode imaging Download PDFInfo
- Publication number
- CN116518878A CN116518878A CN202310784824.1A CN202310784824A CN116518878A CN 116518878 A CN116518878 A CN 116518878A CN 202310784824 A CN202310784824 A CN 202310784824A CN 116518878 A CN116518878 A CN 116518878A
- Authority
- CN
- China
- Prior art keywords
- sample
- hyperspectral
- annular
- camera
- calibration object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 49
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000013519 translation Methods 0.000 claims abstract description 49
- 238000010586 diagram Methods 0.000 claims description 16
- 230000009977 dual effect Effects 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 11
- 239000003086 colorant Substances 0.000 claims description 10
- 230000033001 locomotion Effects 0.000 claims description 10
- 238000012360 testing method Methods 0.000 claims description 6
- 230000002238 attenuated effect Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 3
- 230000000149 penetrating effect Effects 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 230000016507 interphase Effects 0.000 claims 1
- 238000001514 detection method Methods 0.000 abstract description 4
- 230000005855 radiation Effects 0.000 description 9
- 238000013170 computed tomography imaging Methods 0.000 description 4
- 238000001816 cooling Methods 0.000 description 4
- 229910052736 halogen Inorganic materials 0.000 description 4
- 150000002367 halogens Chemical class 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000000701 chemical imaging Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000003028 elevating effect Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000013021 overheating Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 239000013558 reference substance Substances 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N23/00—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00
- G01N23/02—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00 by transmitting the radiation through the material
- G01N23/04—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00 by transmitting the radiation through the material and forming images of the material
- G01N23/046—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00 by transmitting the radiation through the material and forming images of the material using tomography, e.g. computed tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10116—X-ray image
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Radiology & Medical Imaging (AREA)
- Pulmonology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Architecture (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Analysing Materials By The Use Of Radiation (AREA)
Abstract
The invention relates to the technical field of machine vision phenotype detection, in particular to a high-efficiency 4D reconstruction method and a high-efficiency 4D reconstruction system based on dual-mode imaging, wherein the method comprises the following steps: sequentially moving a reference object, a calibration object and a sample to be measured to a designated position through a conveying line, sequentially transferring the reference object, the calibration object and the sample to be measured to a double-shaft translation table through a mechanical arm, aligning a rotating shaft of the sample to be measured with a slit of a hyperspectral camera through the reference object, calibrating internal and external parameters of the hyperspectral camera through the calibration object, collecting X-ray projection images of the sample to be measured at different rotating angles through a flat panel detector assembly, and shooting annular hyperspectral projection images of the sample to be measured through the hyperspectral camera; and the workstation generates a 4D model of the sample to be tested according to the received X-ray projection image and the annular hyperspectral projection image of the sample to be tested. According to the invention, accurate spatial positioning of hyperspectral information is realized while the internal fault structure of the sample is reconstructed, and a 4D sample model with both internal structure and external texture information is obtained.
Description
Technical Field
The invention relates to the technical field of machine vision phenotype detection, in particular to a high-efficiency 4D reconstruction method and a high-efficiency 4D reconstruction system based on micro CT-hyperspectral dual-mode imaging.
Background
In recent years, remote sensing technology and computer vision technology greatly promote the development of the field of phenotype research, and the acquisition of phenotype data is gradually changed from an inefficient and lossy traditional manual mode to high-throughput and lossless automatic phenotype measurement. Modern phenotype detection methods mainly acquire sample images by means of visible light, infrared, fluorescence, hyperspectral, X-ray/CT and other imaging modes, and then extract sample phenotype shape parameters from the sample images by adopting a traditional or deep learning-based image processing method. The imaging methods such as visible light, infrared, fluorescence and the like can only acquire sample surface information but cannot detect internal structures, CT imaging can reconstruct the internal structures of the samples but cannot acquire surface textures, the phenotype information acquired by a single imaging mode is limited, and the existing phenotype detection method has a large lifting space in measurement breadth and dimension. The reflected light imaging and the transmission imaging are integrated, so that the external texture information and the internal structure of the sample can be obtained at the same time, the dimension of phenotype data acquisition can be effectively expanded, and the data mining potential is improved. CT and hyperspectral imaging are typical of two imaging techniques, namely transmitted light and reflected light, and are widely used in phenotypic studies, but these studies are usually performed by a single CT device or hyperspectral device, and the acquired data has a low dimension. In addition, the existing CT-hyperspectral dual-mode imaging system is complex in operation process, a ray projection image and a hyperspectral image cannot be shot at the same time, and the acquisition efficiency of phenotype data is low.
Disclosure of Invention
In view of the fact that phenotype data which can be obtained by a single imaging mode are limited, the invention provides a high-efficiency 4D reconstruction method and a high-efficiency 4D reconstruction system based on micro CT-hyperspectral dual-mode imaging, and accurate spatial positioning of hyperspectral information is achieved while the internal fault structure of a sample is reconstructed, so that a 4D sample model with internal structure and external texture information is obtained.
The invention provides a high-efficiency 4D reconstruction method based on dual-mode imaging, which comprises the following steps:
the method comprises the steps of conveying a reference object to a designated position through a conveying line, transferring the reference object to a double-shaft translation table provided with a carrying rotary table below through a mechanical arm, and aligning a rotary shaft of the carrying rotary table with a slit of a hyperspectral camera positioned on one side of the carrying rotary table by utilizing the reference object;
conveying the calibration object to a designated position through a conveying line, transferring the calibration object to a double-shaft translation table through a mechanical arm, and calibrating internal and external parameters of the hyperspectral camera by utilizing the calibration object;
conveying the sample to be tested to a designated position through a conveying line, and transferring the sample to be tested to a double-shaft translation table through a mechanical arm;
the object carrying rotary table is controlled to rotate at equal intervals, and meanwhile, the micro focal spot ray source emits X-rays so that the X-rays penetrate through a sample to be detected and reach the flat panel detector assembly;
Receiving X-rays through a flat panel detector assembly to acquire a plurality of X-ray projection images of a sample to be tested at different rotation angles within a rotation range, transmitting the X-ray projection images to a workstation, and shooting an annular hyperspectral projection image of the sample to be tested within the rotation range through a hyperspectral camera, and transmitting the annular hyperspectral projection image to the workstation;
and the workstation generates a 4D model of the sample to be tested according to the received X-ray projection images and the annular hyperspectral projection images of the sample to be tested.
Preferably, the step of aligning the rotation axis of the sample to be measured with the slit by using the reference substance includes:
positioning the rotation axis of the sample to be detected through the plumb line on the reference object, so that the plumb line on the reference object coincides with the rotation axis of the sample to be detected;
the orientation of the hyperspectral camera is adjusted by adjusting the mount so that the slit is aligned with the axis of rotation.
Preferably, the step of positioning the axis of rotation of the sample to be measured by a plumb line on the reference object such that the plumb line coincides with the axis of rotation of the sample to be measured comprises:
fixing a visible light camera right above the characteristic pattern of the reference object, shooting the characteristic pattern of the reference object through the visible light camera, and calculating the center point position of the characteristic pattern of the reference object;
And translating the reference object along the horizontal direction, so that the drift amount of the center point of the characteristic pattern of the reference object is minimized, and the plumb line on the reference object is overlapped with the rotation axis of the sample to be detected.
Preferably, after adjusting the orientation of the hyperspectral camera, further comprising:
the rotation center of a bottom plate of the calibration object is moved to a rotation shaft of a sample to be tested, and a calibration annular hyperspectral projection graph of the calibration object is shot in the process of rotating the calibration object for one circle;
based on the calibrated annular hyperspectral projection graph, the internal and external parameters of the hyperspectral camera are calculated according to the corresponding relation between the actual coordinates of the feature points on the calibrated object and the projected pixel coordinates.
Preferably, the step of moving the rotation center of the bottom plate of the calibration object onto the rotation axis of the sample to be measured, and shooting the calibration annular hyperspectral projection chart of the calibration object in the process of rotating the calibration object once comprises the following steps:
placing a calibration object on an object carrying rotary table to enable the calibration object to rotate at a constant speed, wherein the calibration object comprises a bottom plate and thin rods vertically fixed on the bottom plate, rings with different colors at intervals are formed on the bottom plate, textures with different colors at intervals are formed on the thin rods, connecting lines of the bottom ends of the thin rods and the central point of the bottom plate are distributed at equal angles, and the distances between the bottom ends of the thin rods and the central point of the bottom plate are sequentially increased;
Moving the center of the bottom plate of the calibration object to the rotation axis of the sample to be tested to coincide;
and shooting a calibration annular hyperspectral projection graph of the calibration object in the process of rotating the calibration object for one circle.
Preferably, the step of calculating the internal and external parameters of the hyperspectral camera based on the calibrated annular hyperspectral projection graph according to the correspondence between the actual coordinates of the feature points on the calibration object and the projected pixel coordinates includes:
establishing characteristic points of the calibration object based on the boundary of the alternate rings on the bottom plate and the texture edge of the thin rod;
based on the calibrated annular hyperspectral projection graph, calculating the internal and external parameters of the hyperspectral camera according to the corresponding relation between the actual coordinates of the characteristic points of the calibrated object and the projected pixel coordinates; the internal and external parameters of the hyperspectral camera comprise equivalent focal length, principal point coordinates, rotation angles from a world coordinate system to a camera coordinate system and translation components from the world coordinate system to the camera coordinate system.
Preferably, the step of establishing the feature points of the calibration object based on the boundary between the alternate rings on the base plate and the grain edges of the thin rods comprises the following steps:
measuring the radius of the outer edge of each ring on the bottom plate to obtain a first group of characteristic point plane coordinates;
and measuring the height of the edge of each fine rod texture relative to the bottom plate to obtain a second group of characteristic point plane coordinates.
Preferably, the step of generating a 4D model of the sample to be measured by the workstation from the received plurality of X-ray projections and the annular hyperspectral projection of the sample to be measured comprises:
the workstation adopts an FDK algorithm, and a fault diagram of each height of the sample to be detected is obtained according to the sequence reconstruction of a plurality of X-ray projection diagrams of the sample to be detected;
the workstation performs foreground segmentation on the fault diagrams of the heights of the sample to be detected, and a CT three-dimensional model of the sample to be detected is generated;
and the workstation performs registration according to the CT three-dimensional model of the sample to be detected and the annular hyperspectral projection graph to generate a 4D model of the sample to be detected.
The invention also provides a high-efficiency 4D reconstruction system based on dual-mode imaging, which comprises: the conveying line is used for conveying the reference object, the calibration object and the sample to be tested to a designated position in sequence; the XY biaxial translation stage is used for realizing two-dimensional movement in the horizontal direction; the object carrying rotary table is arranged at the bottom of the XY double-shaft translation table and is used for realizing rotation around the vertical direction; the mechanical arm is used for sequentially transferring the reference object, the calibration object and the sample to be tested to the XY biaxial translation stage; a micro focal spot ray source for emitting X-rays to a sample to be measured; the flat panel detector component is used for receiving the attenuated X-rays after penetrating through the sample to be detected, and obtaining a plurality of X-ray projection images of different rotation angles of the sample to be detected after the sample to be detected rotates one circle; the hyperspectral camera is positioned on one side of the sample to be detected, a slit of the hyperspectral camera is aligned with a rotating shaft of the object carrying rotating table, the hyperspectral camera is used for shooting the sample to be detected, and after the sample to be detected rotates for one circle, an annular hyperspectral projection diagram of the sample to be detected is obtained; the working station is used for controlling the start and stop of the object carrying rotary table, the acquisition of a ray projection image, the acquisition of an annular hyperspectral projection image, the reconstruction of a CT three-dimensional model and the completion of the reconstruction of a 4D model of a sample to be detected based on the CT three-dimensional model and the annular hyperspectral projection image; the PLC is used for realizing the communication among the workstation, the servo motor and the driver, and further controlling the start and stop of the object carrying rotary table; the reference object is used for aligning the rotating shaft of the object carrying rotating table with the slit of the hyperspectral camera; the calibration object is used for realizing the calibration of the internal and external parameters of the hyperspectral camera.
Preferably, the workstation comprises at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the dual mode imaging based efficient 4D reconstruction method of the present invention.
Compared with the prior art, the invention has the following beneficial effects:
according to the invention, through synchronous acquisition of the ray projection diagram and the annular hyperspectral projection diagram, the acquisition efficiency of phenotype data based on CT-hyperspectral dual-mode imaging can be improved, and through registration of CT imaging data and hyperspectral data, a high-precision 4D sample model with internal structure and external texture information can be obtained, so that the dimension and breadth of phenotype data acquisition are improved.
Drawings
FIG. 1 is a schematic diagram of a structure of a dual mode imaging-based efficient 4D reconstruction system provided in accordance with an embodiment of the present invention;
FIG. 2 is a schematic diagram of a structure of a reference object provided according to an embodiment of the present invention;
FIG. 3 is a schematic illustration of a feature pattern of a reference object provided in accordance with an embodiment of the present invention;
FIG. 4 is a schematic view of the structure of an adjusting bracket provided according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a calibration object according to an embodiment of the present invention;
FIG. 6 is a schematic view of the structure from the top view of FIG. 5;
FIG. 7 is a flow diagram of a dual mode imaging-based efficient 4D reconstruction method provided in accordance with an embodiment of the present invention;
FIG. 7a is a flow chart of a method for aligning a slit with a rotation axis of a sample to be measured according to an embodiment of the present invention;
FIG. 7b is a flowchart of a method for calculating internal and external parameters of a hyperspectral camera according to an embodiment of the present invention;
fig. 8 is a schematic diagram of a single-frame imaging model of a hyperspectral camera provided according to an embodiment of the present invention.
Reference numerals: workstation 1, PLC controller 2, transfer chain 3, arm 4, little focal spot radiation source 5, ray source cooling device 6, elevating platform 7, flat panel detector subassembly 8, translation platform 9, year revolving stage 10, XY biax translation platform 11, halogen light source 12, hyperspectral camera 13, adjusting support 14, keyset 14-1, lift pillar 14-2, revolving stage 14-3, unipolar translation platform 14-4, reference 15, base 15-1, flat 15-2, flat support 15-3, characteristic pattern 15-4, plumb line 15-5 and plumb 15-6, marker 16, bottom plate 16-1, fine rod 16-2, sample 17 under test.
Detailed Description
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, like modules are denoted by like reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, a detailed description thereof will not be repeated.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not to be construed as limiting the invention.
Fig. 1 illustrates a structure of a dual mode imaging-based efficient 4D reconstruction system provided in accordance with an embodiment of the present invention.
As shown in fig. 1, the efficient 4D reconstruction system based on dual-mode imaging provided by the embodiment of the invention includes a workstation 1, a PLC controller 2, a conveyor line 3, a mechanical arm 4, a micro focal spot radiation source 5, a radiation source cooling device 6, a lifting table 7, a flat panel detector 8, a translation table 9, a carrying rotary table 10, an XY biaxial translation table 11, a halogen light source 12, a hyperspectral camera 13, an adjusting bracket 14, a reference object 15 and a calibration object 16.
The conveying line 3 is used for conveying the reference object 15, the calibration object 16 and the sample 17 to be tested to a specified position. The conveying line 3 is a conventional product in the market, so the specific structure thereof will not be described herein.
The mechanical arm 4 is used for transferring the reference object 15, the calibration object 16 and the sample 17 to be measured to the XY biaxial translation stage 11 in sequence. The mechanical arm 4 adopts a common multi-degree-of-freedom robot in the market, so the specific structure thereof is not described herein.
The XY biaxial translation stage 11 is located above the carrier rotary stage 10 for moving the reference object 15, the calibration object 16, or the sample 17 to be measured placed thereon in the X or Y direction. The XY biaxial translation stage 11 is a two-dimensional translation adjustment mechanism commonly used in the market, and therefore the specific structure thereof is not described here.
The object carrying rotary table 10 is used for driving the reference object 15, the calibration object 16 or the sample 17 to be tested to rotate. The object carrying rotary table 10 is a common rotating mechanism with an object carrying function in the market, and therefore the specific structure thereof is not described herein.
The micro focal spot radiation source 5 is located at one side of the object carrying rotary table 10 for emitting X-rays towards the sample 17 to be measured. The micro focal spot ray source 5 mainly comprises a cathode filament, a vacuum pressurizing cavity anode target and the like, and is used for emitting stable X-ray beams to a sample to be tested, and the ray energy distribution and total energy of the X-ray beams are determined by voltages at two ends of the pressurizing cavity and currents passing through two ends of the filament. The micro focal spot radiation source 5 is independently placed beside the radiation source cooling device 6, and the micro focal spot radiation source 5 is cooled by the radiation source cooling device 6, so that the micro focal spot radiation source 5 is prevented from being damaged due to overheating.
The flat panel detector assembly 8 is located on the other side of the object carrying rotary table 10, X-ray projection images of the sample 17 to be detected at different rotation angles are collected through the flat panel detector assembly 8, and a CT three-dimensional model is built based on the X-ray projection images. The flat panel detector assembly 8 comprises a flat panel detector, a scintillator and a CCD, wherein the flat panel detector is used for receiving the X-rays attenuated after penetrating through a sample to be detected, converting the X-ray energy into optical signals through the scintillator, and converting the optical signals into image data through the CCD.
The translation stage 9 is used to adjust the distance of the load turntable 10 from the flat panel detector assembly 8.
The carrying rotary table 10 is placed on the translation table 9, and the movement of the carrying rotary table 10 is realized through the translation table 9 so as to adjust the distance between the carrying rotary table 10 and the flat panel detector assembly 8.
The flat panel detector assembly 8 is placed on the lifting table 7, the height of the flat panel detector assembly 8 is adjusted through the lifting table 7, the movement of the carrying rotary table 10 is realized through the adjustment of the translation table 9, and the distance between the carrying rotary table 10 and the flat panel detector assembly 8 is adjusted, so that a sample 17 to be detected can be in an X-ray shooting view under each rotation angle.
The hyperspectral camera 13 is located on one side of the carrying turntable 10, the slit of the hyperspectral camera 13 is aligned with the rotation axis of the carrying turntable 10, and the rotation axis of the carrying turntable 10 is the rotation axis of the sample 17 to be measured. After the sample 17 to be measured rotates one turn, the hyperspectral camera 13 can shoot to obtain an annular hyperspectral projection image of the sample 17 to be measured.
The hyperspectral camera 13 is placed on the adjusting bracket 14, the hyperspectral camera 13 can rotate around three non-coplanar shafts and translate along a certain fixed direction by the adjusting bracket 14, and the direction of the hyperspectral camera 13 is adjusted by the adjusting bracket 14, so that the slit of the hyperspectral camera 13 is aligned with the rotation shaft of the object carrying rotary table 10.
The halogen light source 12 is located at one side of the hyperspectral camera 13, and the halogen light source 12 provides a stable light source for hyperspectral imaging required for the hyperspectral camera 13.
The workstation 1 is used for controlling the start and stop of the object carrying rotary table 10, the acquisition of a ray projection image, the acquisition of an annular hyperspectral projection image, the reconstruction of a CT three-dimensional model and the completion of the reconstruction of a 4D model of a sample to be detected based on the registration of the CT three-dimensional model and the annular hyperspectral projection image.
The workstation 1 comprises at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a 4D model reconstruction method of the sample 17 under test.
Workstation 1 is in the form of a general purpose computing device. The workstation 1 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
The PLC controller 2 is used for realizing communication between the workstation 1 and the servo motor and driver, and further controlling the start and stop of the conveying line 3, the mechanical arm 4, the object carrying rotary table 10 and the XY double-shaft translation table 11.
The reference object 15 is used for matching with the object carrying rotary table 10 and the XY double-shaft translation table 11 to realize the alignment of the slit of the hyperspectral camera 13 and the rotation axis of the object carrying rotary table 10.
The calibration object 16 is used for matching with the object carrying rotary table 10 and the XY double-shaft translation table 11 to realize the calibration of the internal and external parameters of the hyperspectral camera 13.
Fig. 2 shows a structure of a reference object provided according to an embodiment of the present invention.
As shown in fig. 2, the reference object includes a base 15-1, a plate 15-2, a plate support 15-3, a characteristic pattern 15-4, a plumb line 15-5, and a plumb 15-6, the plate 15-2 is fixed to the base 15-1 by the plate support 15-3, the characteristic pattern 15-4 is formed on the upper surface of the plate 15-2, the plumb 15-6 is released by the plumb line 15-5 at a position corresponding to a center point of the characteristic pattern 15-4 on the lower surface of the plate 15-2, and textures of different colors are formed on the plumb line 15-5.
Fig. 3 shows a feature pattern of a reference object provided according to an embodiment of the present invention.
As shown in fig. 3, the feature pattern includes two sets of circles of different colors, each set including two opposing circles of the same color, for example, two opposing blue circles and two opposing green circles, and the plumb 15-6 is released downward at the intersection point of the centroid line of the two blue circles and the centroid line of the two green circles, and in addition, the outer circle is red, and the remaining lines are black. Of course, the present invention can also search for the center point of the feature pattern through different shapes. For example, two green circles are replaced with two blue triangles.
It should be noted that, the present invention can detect the center point of the feature pattern by color or shape, and if the feature pattern is not a circle, but another shape, the position of the released plumb line must be the intersection point of the centroid lines of two sets of opposite shapes.
It should also be noted that, if other asymmetric pattern configurations are used, the term "center point" may not be literally used, because the intersection point of two sets of centroid lines is most likely not at the exact center of the entire feature pattern.
Fig. 4 shows a structure of an adjusting bracket provided according to an embodiment of the present invention.
As shown in fig. 4, the adjusting bracket comprises an adapter plate 14-1, a lifting support 14-2, a rotary table 14-3 and a single-axis translation table 14-4, the rotary table 14-3 is fixedly connected with the moving part of the single-axis translation table 14-4, the bottom end of the lifting support 14-2 is fixed on the rotary table 14-3, the top end of the lifting support 14-2 supports the adapter plate 14-1, and the hyperspectral camera 13 is placed on the adapter plate 14-1. The lifting support 14-2, the rotary table 14-3 and the single-axis translation table 14-4 are all of the prior art, and the specific structure thereof is not described herein.
Before the annular hyperspectral projection image is acquired by the hyperspectral camera 13, the internal and external parameters of the hyperspectral camera 13 are required to be calibrated, and the calibration of the internal and external parameters of the hyperspectral camera 13 is realized by a calibration object.
Fig. 5 and 6 respectively show structures of calibration objects with different viewing angles according to an embodiment of the present invention.
As shown in fig. 5 and 6, the calibration object is composed of a bottom plate 16-1 and a plurality of thin rods 16-2 vertically fixed on the bottom plate, rings with different colors being alternately formed on the bottom plate 16-1, textures with different colors being alternately formed on each thin rod 16-2, the connecting lines of the bottom end of each thin rod 16-2 and the central point of the bottom plate 16-1 are distributed at equal angles, and the distance between the bottom end of the thin rod 16-2 and the central point of the bottom plate 16-1 is sequentially increased.
The foregoing details the efficient 4D reconstruction system based on dual-mode imaging provided by the embodiment of the present invention, corresponding to the foregoing sample 4D reconstruction system, and the second embodiment of the present invention further provides a sample 4D reconstruction method implemented by using the efficient 4D reconstruction system based on dual-mode imaging.
Fig. 7 shows a flow of an efficient 4D reconstruction method based on dual mode imaging according to an embodiment of the present invention.
As shown in fig. 7, the efficient 4D reconstruction method based on dual-mode imaging mainly includes the following steps:
and 701, conveying the reference object to a designated position through a conveying line, transferring the reference object to a double-shaft translation table with a carrying rotary table arranged below through a mechanical arm, and aligning a slit of a hyperspectral camera positioned on one side of the carrying rotary table with a rotary shaft of the carrying rotary table by utilizing the reference object.
In this step, the alignment of the slit of the hyperspectral camera and the rotation axis of the object carrying rotary table is realized by the reference object, so that the synchronous acquisition of the radial projection image and the annular hyperspectral projection image is realized by the flat panel detector assembly 8 and the hyperspectral camera 13 in the period of one rotation of the sample 17 to be detected in the subsequent steps, so that the acquisition efficiency of the phenotype data based on CT-hyperspectral dual-mode imaging is greatly improved, and the hyperspectral camera can also obtain the surface information of the sample to be detected to the maximum extent.
It should be noted that, only before the first use of the efficient 4D reconstruction system based on dual-mode imaging to perform dual-mode image capturing, the slit of the hyperspectral camera 13 needs to be aligned with the rotation axis of the sample 17 to be tested, and then the internal and external parameters of the hyperspectral camera 13 are calibrated, where in the present invention, the calibration of the internal and external parameters of the hyperspectral camera and the calculation of the internal and external parameters of the hyperspectral camera can be understood as meaning.
It will be appreciated that after alignment and calibration, the same set of efficient 4D reconstruction systems based on dual mode imaging are reused without any change in the system.
Specifically, the slit of the hyperspectral camera 13 is aligned with the rotation axis of the sample 17 to be measured in order to maximize the acquisition of the surface information of the sample 17 to be measured by the hyperspectral camera 13. The slit of the hyperspectral camera 13 is aligned with the rotation axis of the sample 17 to be measured, i.e. the rotation axis of the sample 17 to be measured is brought within the imaging plane of the hyperspectral camera 13. Since adjusting the orientation of the hyperspectral camera by eye observation alone does not allow the slit alignment to the accuracy required for data registration, the present invention uses the reference object shown in fig. 2 and the adjustment bracket shown in fig. 4 to align the slit of the hyperspectral camera 13 with the rotation axis of the sample 17 to be measured.
Fig. 7a shows the flow of the alignment method in the present invention.
As shown in fig. 7a, the alignment method includes the steps of:
and step 11, positioning the rotating shaft by adopting the plumb line on the reference object, namely enabling the plumb line of the reference object to coincide with the rotating shaft.
In the implementation process, first, a visible light camera may be fixed directly above a feature pattern of a reference object, the feature pattern of the reference object is photographed by the visible light camera, and a center point position of the feature pattern of the reference object is calculated. The specific operation steps are as follows:
(1) The top surface of the XY biaxial translation stage 11 is adjusted to be completely horizontal by a level gauge, and at this time, the rotation axis of the sample 17 to be measured is in the vertical direction.
(2) The reference object is placed on the XY biaxial translation stage 11, and the object carrying rotary table 10 is rotated at a constant speed by sending an instruction to the workstation 1, and the XY biaxial translation stage 11 rotates at a constant speed together with the reference object placed thereon along with the object carrying rotary table 10.
(3) A visible light camera is fixed right above the characteristic pattern 15-4 of the reference object, and the characteristic pattern 15-4 is photographed by the visible light camera, at this time, the console terminal of the workstation 1 displays the characteristic pattern 15-4 photographed by the visible light camera in real time.
And secondly, the reference object can be horizontally translated, so that the drift amount of the center point of the characteristic pattern of the reference object is minimized, and the plumb line on the reference object is overlapped with the rotation axis of the sample to be measured. The specific operation steps are as follows:
(1) And calculating the intersection point of the connecting line of the centers of mass of the two opposite patterns on the characteristic pattern 15-4 and the connecting line of the centers of mass of the other two opposite patterns to obtain the pixel coordinate of the right center of the characteristic pattern 15-4 (namely the position of the central point of the characteristic pattern 15-4).
(2) Tracking the movement track of the center point of the characteristic pattern 15-4, and translating the reference object along the X or Y direction, so that the drift amount of the center point of the reference object is minimized.
When the reference object is placed on the XY biaxial translation stage 11 and rotated along with the object-carrying rotary stage 10 during alignment, the center point of the feature pattern is not moved to the rotation axis, and the movement locus of the center point is a circle, wherein the center point of the circle is located on the rotation axis, the closer the center point is to the rotation axis, the smaller the radius of the circle of the movement locus is, and after moving to the rotation axis, the center point will remain at a point, that is, the so-called "drift amount" is minimized.
(3) Since the vertex of the plumb line 15-5 coincides with the location of the center point of the feature pattern 15-4, when the center point of the feature pattern 15-4 has minimal drift during the rotational movement, the plumb line 15-5 coincides with the rotational axis of the sample 17 to be measured.
(4) And evacuating the visible light camera.
It will be appreciated that the present invention achieves positioning of the axis of rotation using the plumb line on the reference object by tracking the center point motion trajectory of the feature pattern 15-4, translating the reference object in the X or Y direction, such that the plumb line of the reference object coincides with the axis of rotation.
And step 12, when the plumb line of the reference object is coincident with the rotation axis, adjusting the azimuth of the hyperspectral camera so that the slit of the hyperspectral camera is aligned with the rotation axis.
In the step, firstly, slit single-frame imaging results can be displayed in real time in hyperspectral camera acquisition control software, and the slit of the hyperspectral camera 13 can observe plumb lines 15-5 by adjusting the bracket 14 to translate the hyperspectral camera 13; the orientation of the hyperspectral camera 13 is then further fine-tuned, when the range of plumb lines 15-5 that the slit can observe is maximized, indicating that the slit of the hyperspectral camera 13 is aligned with the axis of rotation of the sample 17 to be measured. It can be seen that in this step, the slit of the hyperspectral camera can be aligned with the plumb line by adjusting the orientation of the hyperspectral camera, i.e. indirectly the slit of the hyperspectral camera 13 is aligned with the rotation axis.
Step 702, conveying the calibration object to a designated position through a conveying line, transferring the calibration object to a double-shaft translation table through a mechanical arm, and calibrating internal and external parameters of the hyperspectral camera by using the calibration object.
The calculation of the internal and external parameters of the hyperspectral camera 13 is for the registration of the subsequent CT three-dimensional model with the annular hyperspectral projection map.
The invention simplifies a single-frame imaging model of the hyperspectral camera 13 into the two-dimensional condition aperture imaging shown in fig. 8, and calculates the internal and external parameters of the hyperspectral camera 13 by adopting a special calibration object based on the simplified imaging model.
As shown in FIG. 8, the world coordinate system is recorded asThe camera coordinate system is->The image coordinate system is OX, the pixel coordinate system is +.>，/>For the optical axis of the hyperspectral camera 13, the world coordinate system +.>The axis coincides with the rotation axis of the sample 17 to be measured, since the slit of the hyperspectral camera 13 is aligned with the rotation axis of the sample 17 to be measured, +.>And->Two coplanar straight lines are to be +.>And->The plane is defined as the hyperspectral camera imaging plane, and the world coordinate system is specified>The axis is within the hyperspectral camera imaging plane and perpendicular to +.>。
One point in the world coordinate systemThe projection formula of the image point m (u) to this point is:
；
in the case of the formula (1),is of the phaseBuilt-in reference matrix->Is an external reference matrix->Is equivalent focal length +.>F is the focal length of the camera, dx is the size of a single pixel of the CCD of the camera, < >>For principal point coordinates, +.>For the rotation angle of the world coordinate system to the camera coordinate system, +.>And->Is a translational component of the world coordinate system to the camera coordinate system.
Equation (1) can be written as:
；
recording deviceWherein->，/>，/>，，/>，/>；
If the correspondence between two-dimensional points in N imaging planes and their image points is known, then there are:
；
let the above formula be abbreviated ap=0, wherein:
equation ap=0 includesA total of 6 unknowns, since only the ratio between the unknowns is valid, do not require +. >The solution of this equation is matrix +.>A feature vector corresponding to the minimum feature value of (a).
Solving for vector P such that ap=0, let:
；
according toThe internal and external parameters of the hyperspectral camera 13 are calculated:
a minimum of 5 known plane point-to-pixel correspondence is required from equation (3) to solve for the internal and external parameters of the hyperspectral camera 13. The larger the number of known correspondences, the smaller the solution error.
For each feature point, the actual coordinates thereof can be measuredAt the same time the coordinates of its projection point can be found from the annular hyperspectral projection map +.>From the actual coordinates>And the coordinates of its projection points +.>An equation can be listed +.>
The N feature points may list N or more forms of equations, and writing the N equations as a matrix form is equation (3).
In the formula (3) of the present invention,is a vector related to the internal and external parameters of the camera, and the calibration of the camera is equivalent to finding the vector for satisfying equation (3)>The solving method is as specified in the description between the formula (3) to the formula (4).
The internal and external parameters of the hyperspectral camera 13 are calibrated by using the calibration object shown in fig. 5. Note that the connection line of the bottom end of the ith (i=0, 1, …, n) thin rod and the center of the bottom plate Is->，/>And->The included angle is->The method comprises the steps of carrying out a first treatment on the surface of the The bottom end of the ith thin rod is positioned on the outer edge of the (i+1) th circular ring, and the top view of the calibration object is shown in fig. 6.
Fig. 7b shows a flow of a method for calculating internal and external parameters of a hyperspectral camera according to an embodiment of the present invention.
As shown in fig. 7b, the method for calculating the internal and external parameters of the hyperspectral camera 13 includes the following steps:
and 21, moving the rotation center of the base plate of the calibration object to a rotation shaft of the sample to be tested, and shooting a calibration annular hyperspectral projection graph of the calibration object in the process of rotating the calibration object for one circle.
In the specific implementation process, firstly, a calibration object is placed on an XY double-shaft translation table, so that the calibration object and the XY double-shaft translation table rotate together with an object carrying rotary table at a constant speed; the calibration object comprises a bottom plate and thin rods vertically fixed on the bottom plate, rings with different alternate colors are formed on the bottom plate, textures with different alternate colors are formed on the thin rods, connecting lines of the bottom ends of the thin rods and the central point of the bottom plate are distributed at equal angles, the distances between the bottom ends of the thin rods and the central point of the bottom plate are sequentially increased, and detailed description can refer to related description corresponding to fig. 5 and 6 in the first embodiment and is not repeated here. The specific operation steps are as follows:
(1) And the top surface of the XY biaxial translation stage is adjusted to be completely horizontal by adopting a level gauge, at the moment, the rotating shaft of the sample to be measured is in the vertical direction, and a calibration object is placed on the XY biaxial translation stage.
(2) The work station 1 sends an instruction to enable the object carrying rotary table 10 to rotate at a constant speed, and the XY biaxial translation table and the calibration objects placed on the XY biaxial translation table rotate at a constant speed along with the object carrying rotary table.
And secondly, moving the center of the bottom plate of the calibration object to the rotation axis of the sample to be tested to coincide. The specific operation steps are as follows:
(1) A visible light camera is fixed directly above the calibration object 16.
(2) The base plate 16-1 of the calibration object 16 is photographed using a visible light camera.
(3) The console terminal of the workstation 1 displays in real time the circle at the center of the base plate 16-1 photographed by the visible light camera.
(4) The centroid of the circle is calculated to obtain the center point position of the bottom plate 16-1.
(5) Tracking the movement track of the central point of the bottom plate 16-1, and translating the calibration object along the X or Y direction to minimize the drift amount of the central point of the bottom plate 16-1, wherein the central point of the bottom plate 16-1 is positioned on the rotation axis of the sample 17 to be tested, and all the thin rods 16-2 fixed on the bottom plate 16-1 are parallel to the rotation axis of the sample 17 to be tested.
(6) After the center point of the base plate 16-1 is moved onto the rotation axis of the sample 17 to be measured, the visible light camera is evacuated.
Thirdly, shooting a calibration annular hyperspectral projection graph of the calibration object in the process of rotating the calibration object for one circle.
And step 22, calculating the internal and external parameters of the hyperspectral camera according to the corresponding relation between the actual coordinates of the characteristic points on the calibration object and the projected pixel coordinates based on the calibrated annular hyperspectral projection graph.
In the implementation process, characteristic points of the calibration object are established based on the boundary of the alternate rings on the base plate and the texture edge of the thin rod. The specific operation steps are as follows:
(1) And measuring the radius of the outer edge of each ring on the bottom plate to obtain a first group of characteristic point plane coordinates.
Specifically, the outer edge radius of each ring on the base plate 16-1 is measuredObtaining a first group of characteristic point plane coordinates +.>。
(2) And measuring the height of the edge of each fine rod texture relative to the bottom plate to obtain a second group of characteristic point plane coordinates.
Specifically, the method comprises the following operation steps:
A. the height of the edge of the texture of each wand 16-2 relative to the base plate 16-1 is measured and the height of the jth textured edge of the ith wand 16-2 relative to the base plate 16-1 is recorded asObtaining a second group of characteristic point plane coordinates 。/>
B. The coordinates imaged by the edges of the rings of the base plate 16-1 are obtained from the annular hyperspectral projection map
And the first group of characteristic point projection pixel coordinates are marked as first group of characteristic point projection pixel coordinates, and the first group of characteristic point projection pixel coordinates correspond to the first group of characteristic point plane coordinates.
C. Obtaining the coordinates of the edges of the textures of the thin rods 16-2 from the annular hyperspectral projection map
And the second group of characteristic point projection pixel coordinates are marked as second group of characteristic point projection pixel coordinates, and the second group of characteristic point projection pixel coordinates correspond to the second group of characteristic point plane coordinates.
Secondly, calculating internal and external parameters of the hyperspectral camera based on a calibrated annular hyperspectral projection graph according to the corresponding relation between the plane coordinates of the characteristic points and the coordinates of the projection pixels on the calibrated object; the internal and external parameters of the hyperspectral camera comprise equivalent focal length, principal point coordinates, rotation angles from a world coordinate system to a camera coordinate system and translation components from the world coordinate system to the camera coordinate system.
Specifically, the method can be based on the corresponding relation between the first group of the projection pixel coordinates of the feature points and the plane coordinates of the feature points, the corresponding relation between the second group of the projection pixel coordinates of the feature points and the plane coordinates of the feature points, and the formula3) And equation (4) for solving the internal and external parameters of the hyperspectral camera 13 。
And step 703, conveying the sample to be tested to a designated position through a conveying line, and transferring the sample to be tested to an XY biaxial translation stage through a mechanical arm.
For steps 701-703, referring to fig. 1, the reference object 15, the calibration object 16 and the sample 17 to be measured may be sequentially placed on the conveying line 3, then the conveying line 3 is started, the reference object 15 is moved to the dotted line area, then the conveying line 3 is suspended, the reference object 15 is transferred to the XY biaxial translation stage 11 by the mechanical arm 4, the alignment of the slit of the hyperspectral camera 13 and the rotation axis of the object carrying rotary stage 10 is performed, after the alignment, the reference object 15 is moved back to the conveying line 3 by the mechanical arm 4, and the operation modes of the calibration object 16 and the sample 17 to be measured are the same.
Step 704, controlling the object carrying rotary table to rotate at equal intervals, and simultaneously enabling the micro focal spot ray source to emit X-rays so as to enable the X-rays to penetrate through the sample to be detected to reach the flat panel detector assembly.
Step 705, receiving X-rays through the flat panel detector assembly to collect a plurality of X-ray projection images of the sample to be measured at different rotation angles within a rotation range, transmitting the X-ray projection images to the workstation, and shooting an annular hyperspectral projection image of the sample to be measured within the rotation range through the hyperspectral camera, and transmitting the annular hyperspectral projection image to the workstation.
Step 706, the workstation generates a 4D model of the sample to be tested according to the received multiple X-ray projections and the annular hyperspectral projections of the sample to be tested.
In step 706, first, the workstation 1 may generate a CT three-dimensional model through a plurality of X-ray projection images of the sample 17 to be measured, and then, according to the correspondence between the spatial points in the CT three-dimensional model and the pixel points in the annular hyperspectral projection image, output the coordinates of the three-dimensional points and the spectrum information together to obtain a 4D (spatial three-dimensional-spectrum) model of the sample 17 to be measured. The specific operation steps are as follows:
(1) Using FDK algorithm to reconstruct from the sequence of X-ray projection images of the sample 17 to be measuredTo the tomogram of each height of the sample 17 to be measured, the foreground segmentation is carried out on all tomograms, and the first is thatThe foreground point (i, j) in the Zhang Duanceng graph can obtain a three-dimensional point coordinate (i, j, k), and the set of the three-dimensional point coordinates corresponding to the foreground points of all the tomograms is the CT three-dimensional model of the sample 17 to be detected.
(2) The workstation 1 registers according to the CT three-dimensional model of the sample to be detected generated in the step (1) and the photographed annular hyperspectral projection graph of the sample to be detected, and generates a 4D model of the sample to be detected.
In step (2), the specific registration method of the CT three-dimensional model and the annular hyperspectral projection map is as follows:
traversing the annular hyperspectral projection view of the sample 17 to be tested, and for any column on the annular hyperspectral projection viewAccording to the coordinates->Judging the intersection part I of the imaging plane and the sample 17 to be tested when the hyperspectral camera 13 shoots the v-th frame image, for the +.>When the point is a foreground point, carrying out back projection calculation on the point according to a formula (1) to judge the back projection light ray and the distance I of the point (u, v)>The nearest intersection point is the spatial point corresponding to the pixel point (u, v).
The method for judging the intersection part I of the imaging plane and the sample 17 to be tested when the hyperspectral camera 13 shoots the v-th frame image is as follows:
A. when the sample 17 to be measured is photographed, a vertical thin rod is placed on the carrying rotary table 6 together with the sample 17 to be measured.
B. Locating thin rods in tomograms, recordingThe connecting line of the centroid of the cross section of the slender rod and the central point of the tomogram is。
C. Positioning thin rods in a ring hyperspectral projection view, and marking the thin rods as。
D. Assume that the sample 17 to be measured is observed from the top view as clockwise rotation, and assume that the hyperspectral camera 13 takes the image of the v-th frame and takes the image of the v-th frame The rotation angle of the sample 17 to be measured in the frame image is +.>Record->Rotate counterclockwise along the center of the tomogram +.>And the obtained straight line is l, and for the v th column of the annular hyperspectral projection graph, the corresponding I is the set of the intersection part of the straight line l and the foreground point in all the fault graphs.
Therefore, the sample 4D reconstruction method realized by the high-efficiency 4D reconstruction system based on the dual-mode imaging can finish synchronous acquisition of a plurality of X-ray projection images of different rotation angles of the sample to be detected and the annular hyperspectral projection image of the sample to be detected through the flat panel detector assembly 8 and the hyperspectral camera 13 within one circle of rotation time of the sample to be detected.
Compared with the prior art, the X-ray projection image and the hyperspectral projection image can be obtained only by independently shooting the sample to be detected at least twice by using the CT equipment and the hyperspectral equipment, and the invention can simultaneously obtain a plurality of X-ray projection images and annular hyperspectral projection images of different rotation angles of the sample to be detected 17 through one operation, thereby not only greatly improving the operation efficiency, but also effectively improving the phenotype data dimension which can be obtained based on CT imaging and hyperspectral imaging by high-efficiency registration of the annular hyperspectral projection images and CT three-dimensional models generated based on the plurality of X-ray projection images.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A high-efficiency 4D reconstruction method based on dual-mode imaging, comprising:
transporting a reference object to a designated position through a conveying line, transferring the reference object to a double-shaft translation table provided with a carrying rotary table below through a mechanical arm, and aligning a rotating shaft of the carrying rotary table with a slit of a hyperspectral camera positioned on one side of the carrying rotary table by utilizing the reference object;
Conveying a calibration object to a designated position through the conveying line, transferring the calibration object to the double-shaft translation table through a mechanical arm, and calibrating internal and external parameters of the hyperspectral camera by utilizing the calibration object;
conveying the sample to be tested to a designated position through the conveying line, and transferring the sample to be tested to the double-shaft translation table through a mechanical arm;
controlling the object carrying rotary table to rotate at equal intervals, and simultaneously enabling the micro focal spot ray source to emit X-rays so as to enable the X-rays to penetrate through the sample to be detected to reach the flat panel detector assembly;
receiving the X-rays through the flat panel detector assembly to acquire a plurality of X-ray projection images of the sample to be tested at different rotation angles within a rotation range, transmitting the X-ray projection images to a workstation, and shooting an annular hyperspectral projection image of the sample to be tested within the rotation range through the hyperspectral camera, and transmitting the annular hyperspectral projection image to the workstation;
and the workstation generates a 4D model of the sample to be tested according to the received X-ray projection images and the annular hyperspectral projection images of the sample to be tested.
2. The efficient 4D reconstruction method based on dual mode imaging as claimed in claim 1, wherein the step of aligning the rotation axis of the sample to be measured with the slit using the reference object comprises:
Positioning the rotation axis of the sample to be detected through the plumb line on the reference object so that the plumb line on the reference object coincides with the rotation axis of the sample to be detected;
the orientation of the hyperspectral camera is adjusted by adjusting the stand so that the slit is aligned with the rotation axis.
3. The efficient 4D reconstruction method based on dual mode imaging of claim 2, wherein the step of positioning the rotational axis of the sample under test with a plumb line on the reference such that the plumb line coincides with the rotational axis of the sample under test comprises:
fixing a visible light camera right above the characteristic pattern of the reference object, shooting the characteristic pattern of the reference object through the visible light camera, and calculating the center point position of the characteristic pattern of the reference object;
and translating the reference object along the horizontal direction, so that the drift amount of the center point of the characteristic pattern of the reference object is minimized, and the plumb line on the reference object is overlapped with the rotation axis of the sample to be detected.
4. A dual mode imaging based efficient 4D reconstruction method according to claim 2 or 3 further comprising, after adjusting the orientation of the hyperspectral camera:
The rotation center of a bottom plate of a calibration object is moved to a rotation shaft of the sample to be tested, and a calibration annular hyperspectral projection image of the calibration object is shot in the process of rotating the calibration object for one circle;
and calculating the internal and external parameters of the hyperspectral camera according to the corresponding relation between the actual coordinates of the characteristic points on the calibration object and the projected pixel coordinates based on the calibrated annular hyperspectral projection graph.
5. The efficient 4D reconstruction method based on dual mode imaging as set forth in claim 4, wherein the step of moving the center of rotation of the base plate of the calibration object onto the rotation axis of the sample to be measured, and capturing a calibration annular hyperspectral projection map of the calibration object during one rotation of the calibration object, comprises:
placing a calibration object on the object carrying rotary table to enable the calibration object to rotate at a constant speed, wherein the calibration object comprises a bottom plate and thin rods vertically fixed on the bottom plate, rings with different colors at intervals are formed on the bottom plate, textures with different colors at intervals are formed on the thin rods, connecting lines of the bottom ends of the thin rods and the central point of the bottom plate are distributed at equal angles, and the distances between the bottom ends of the thin rods and the central point of the bottom plate are sequentially increased;
Moving the center of the bottom plate of the calibration object to the rotation axis of the sample to be tested to coincide;
and shooting a calibration annular hyperspectral projection graph of the calibration object in the process of rotating the calibration object for one circle.
6. The efficient 4D reconstruction method based on dual mode imaging as set forth in claim 4, wherein the step of calculating the internal and external parameters of the hyperspectral camera based on the calibrated annular hyperspectral projection map according to the correspondence between the actual coordinates of the feature points on the calibration object and the projected pixel coordinates includes:
establishing characteristic points of the calibration object based on the juncture of the alternate rings and the grain edges of the thin rods on the bottom plate;
based on the calibrated annular hyperspectral projection graph, calculating internal and external parameters of the hyperspectral camera according to the corresponding relation between the actual coordinates of the characteristic points of the calibrated object and the projected pixel coordinates; the internal and external parameters of the hyperspectral camera comprise equivalent focal length, principal point coordinates, rotation angles from a world coordinate system to a camera coordinate system and translation components from the world coordinate system to the camera coordinate system.
7. The efficient 4D reconstruction method based on dual mode imaging as set forth in claim 6, wherein said step of establishing feature points of said calibration object based on interfaces of inter-phase circles on said base plate and textured edges of thin rods comprises:
Measuring the radius of the outer edge of each ring on the bottom plate to obtain a first group of characteristic point plane coordinates;
and measuring the height of the edge of each fine rod texture relative to the bottom plate to obtain a second group of characteristic point plane coordinates.
8. The efficient 4D reconstruction method based on dual mode imaging of claim 1 wherein the step of generating a 4D model of the sample under test from the received plurality of X-ray projections and the annular hyperspectral projection of the sample under test by the workstation comprises:
the workstation adopts an FDK algorithm, and a fault chart of each height of the sample to be detected is obtained according to the sequence reconstruction of a plurality of X-ray projection charts of the sample to be detected;
the workstation performs foreground segmentation on the fault diagrams of the sample to be detected at all heights to generate a CT three-dimensional model of the sample to be detected;
and the workstation generates a 4D model of the sample to be tested according to the CT three-dimensional model of the sample to be tested and the annular hyperspectral projection graph.
9. A system for implementing the dual mode imaging-based efficient 4D reconstruction method of any one of claims 1 to 8, the system comprising:
the conveying line is used for conveying the reference object, the calibration object and the sample to be tested to a designated position in sequence;
The XY biaxial translation stage is used for realizing two-dimensional movement in the horizontal direction;
the object carrying rotary table is arranged at the bottom of the XY double-shaft translation table and is used for realizing rotation around the vertical direction;
the mechanical arm is used for sequentially transferring the reference object, the calibration object and the sample to be tested to the XY biaxial translation stage;
a micro focal spot ray source for emitting X-rays to the sample to be measured;
the flat panel detector component is used for receiving the attenuated X-rays penetrating through the sample to be detected, and obtaining a plurality of X-ray projection images of different rotation angles of the sample to be detected after the sample to be detected rotates for one circle;
the hyperspectral camera is positioned on one side of the sample to be detected, a slit of the hyperspectral camera is aligned with a rotation axis of the object carrying rotary table, the hyperspectral camera is used for shooting the sample to be detected, and an annular hyperspectral projection image of the sample to be detected is obtained after the sample to be detected rotates for one circle;
the work station is used for controlling the start and stop of the object carrying rotary table, the collection of the ray projection diagram, the collection of the annular hyperspectral projection diagram and completing the 4D model reconstruction of the sample to be detected based on the CT three-dimensional model reconstructed by the ray projection diagram and the annular hyperspectral projection diagram;
The PLC is used for realizing the communication among the workstation, the servo motor and the driver, and further controlling the start and stop of the object carrying rotary table;
the reference object is used for realizing the alignment of the rotating shaft of the object carrying rotating table and the slit of the hyperspectral camera;
the calibration object is used for realizing the calibration of the internal and external parameters of the hyperspectral camera.
10. The system of claim 9, wherein the workstation comprises at least one processor; and a memory communicatively coupled to the at least one processor;
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of sample 4D reconstruction of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310784824.1A CN116518878B (en) | 2023-06-29 | 2023-06-29 | Efficient 4D reconstruction method and efficient 4D reconstruction system based on dual-mode imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310784824.1A CN116518878B (en) | 2023-06-29 | 2023-06-29 | Efficient 4D reconstruction method and efficient 4D reconstruction system based on dual-mode imaging |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116518878A true CN116518878A (en) | 2023-08-01 |
| CN116518878B CN116518878B (en) | 2023-10-10 |
Family
ID=87394368
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310784824.1A Active CN116518878B (en) | 2023-06-29 | 2023-06-29 | Efficient 4D reconstruction method and efficient 4D reconstruction system based on dual-mode imaging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116518878B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117378318A (en) * | 2023-12-08 | 2024-01-12 | 海南大学三亚研究院 | Coconut seed germination detection and sorting method and system based on X-rays |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140270440A1 (en) * | 2013-03-15 | 2014-09-18 | Jean-Marc Inglese | Volume image reconstruction using data from multiple energy spectra |
| CN106821410A (en) * | 2017-03-20 | 2017-06-13 | 厦门大学 | Using the substance identification and system of EO-1 hyperion CT functional imagings |
| CN108072672A (en) * | 2017-12-14 | 2018-05-25 | 清华大学 | The on-Line Monitor Device and its monitoring method of a kind of ablation structure and morphology and product |
| CN108254396A (en) * | 2017-12-05 | 2018-07-06 | 江苏大学 | A kind of tomato seedling water stress detection method based on micro-CT and polarization-high light spectrum image-forming multiple features fusion |
-
2023
- 2023-06-29 CN CN202310784824.1A patent/CN116518878B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140270440A1 (en) * | 2013-03-15 | 2014-09-18 | Jean-Marc Inglese | Volume image reconstruction using data from multiple energy spectra |
| CN106821410A (en) * | 2017-03-20 | 2017-06-13 | 厦门大学 | Using the substance identification and system of EO-1 hyperion CT functional imagings |
| CN108254396A (en) * | 2017-12-05 | 2018-07-06 | 江苏大学 | A kind of tomato seedling water stress detection method based on micro-CT and polarization-high light spectrum image-forming multiple features fusion |
| CN108072672A (en) * | 2017-12-14 | 2018-05-25 | 清华大学 | The on-Line Monitor Device and its monitoring method of a kind of ablation structure and morphology and product |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117378318A (en) * | 2023-12-08 | 2024-01-12 | 海南大学三亚研究院 | Coconut seed germination detection and sorting method and system based on X-rays |
| CN117378318B (en) * | 2023-12-08 | 2024-03-12 | 海南大学三亚研究院 | Coconut seed germination detection and sorting method and system based on X-rays |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116518878B (en) | 2023-10-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109215108B (en) | Panoramic three-dimensional reconstruction system and method based on laser scanning | |
| CN108921901B (en) | Large-view-field camera calibration method based on precise two-axis turntable and laser tracker | |
| CN108844459B (en) | Calibration method and device of blade digital sample plate detection system | |
| CN104897060B (en) | Using the big visual field global measuring method of coordinate tracking control panel | |
| US8744133B1 (en) | Methods and systems for locating visible differences on an object | |
| CN116518878B (en) | Efficient 4D reconstruction method and efficient 4D reconstruction system based on dual-mode imaging | |
| EP2587450B1 (en) | Method and apparatus for generating a three-dimensional model of a region of interest using an imaging system | |
| CN113048918B (en) | Device and method for detecting consistency of emission axis and aiming axis | |
| CN105758623B (en) | A kind of large-aperture long-focus remote sensing camera distortion measurement device and method based on TDI CCD | |
| CN116518879B (en) | Sample 4D reconstruction method and system based on micro CT-hyperspectral dual-mode imaging | |
| WO2019140778A1 (en) | Three-dimensional reconstruction system and three-dimensional reconstruction method | |
| CN114264248B (en) | Monocular rotation structured light three-dimensional measurement method | |
| CN108614277A (en) | Double excitation single camera three-dimensional imaging scan table and scanning, imaging method | |
| CN113532318A (en) | Three-dimensional scanning system and method for positioning by using multiple groups of laser trackers | |
| CN108670284A (en) | A kind of determination method and system of CT scanner source focal point | |
| CN111561868A (en) | Method for realizing non-contact measurement of antenna profile by utilizing optical tracking structure optical scanner | |
| CN112254680B (en) | Multi freedom's intelligent vision 3D information acquisition equipment | |
| CN111665023A (en) | Telescope distortion measuring device and method | |
| CN107976146B (en) | Self-calibration method and measurement method of linear array CCD camera | |
| CN211292584U (en) | Imaging device | |
| US20110018965A1 (en) | Apparatus for optimizing geometric calibration | |
| CN108200424A (en) | A kind of adjustment method and system for multiple TDICCD detectors optical axis centerings | |
| CN104502378B (en) | X-ray CT (Computed Tomography) device | |
| CN102692422A (en) | Metering type high-precision x-ray microscope sample scanning table | |
| CN107202802B (en) | Computer tomography apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 570228 No. 58 Renmin Avenue, Hainan, Haikou Applicant after: HAINAN University Applicant after: Sanya Research Institute of Hainan University Address before: 572024 Room 501, 5 / F, building 3, Baitai Industrial Park, yazhouwan science and Technology City, Yazhou District, Sanya City, Hainan Province Applicant before: Sanya Research Institute of Hainan University Applicant before: HAINAN University |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |