WO2023197785A1 - 局域轨道函数的三维重构方法及装置 - Google Patents

局域轨道函数的三维重构方法及装置 Download PDF

Info

Publication number
WO2023197785A1
WO2023197785A1 PCT/CN2023/080253 CN2023080253W WO2023197785A1 WO 2023197785 A1 WO2023197785 A1 WO 2023197785A1 CN 2023080253 W CN2023080253 W CN 2023080253W WO 2023197785 A1 WO2023197785 A1 WO 2023197785A1
Authority
WO
WIPO (PCT)
Prior art keywords
function
local
parameters
image
dimensional
Prior art date
Application number
PCT/CN2023/080253
Other languages
English (en)
French (fr)
Inventor
于荣
毛梁泽
程志英
Original Assignee
清华大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 清华大学 filed Critical 清华大学
Publication of WO2023197785A1 publication Critical patent/WO2023197785A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present application relates to the field of three-dimensional imaging technology, and in particular to a three-dimensional reconstruction method and device of a local orbit function.
  • the electron microscope photos of the sample contain rich information that is difficult to obtain intuitively.
  • Applying a three-dimensional reconstruction algorithm to obtain the three-dimensional structural information of the sample from a series of electron microscope projection photos of the sample is essential for understanding the material composition at a fundamental level. It helps a lot with performance.
  • AET Anamic Electrical Tomography, atomic scale electron tomography
  • the AET algorithm obtains the three-dimensional coordinates of atoms through peak searching from the reconstructed three-dimensional density matrix, which has high hardware requirements, and during the peak searching process, cumbersome human intervention is required, which is difficult to avoid Errors caused by human intervention.
  • the three-dimensional coordinates of atoms it is impossible to correct the sample drift and mechanical tilt error of the sample stage when collecting images. The accuracy is poor and needs to be improved.
  • This application provides a method and device for three-dimensional reconstruction of local orbital functions to solve the problem in related technologies that only the three-dimensional coordinates of atoms can be obtained from the reconstructed three-dimensional density matrix, and the errors cannot be corrected, resulting in reconstruction problems.
  • the process has high hardware requirements and the reconstructed three-dimensional coordinates have poor accuracy.
  • the first embodiment of the present application provides a three-dimensional reconstruction method of a local orbit function, which includes the following steps: collecting image data of samples at multiple tilt angles; based on the image data, scattering is performed at equal intervals in real space. points and use linear accumulation to obtain the calculated image at each tilt angle of the multiple tilt angles; and calculate the loss function based on the calculated image at each tilt angle, and obtain the parameters of the loss function to be optimized gradient, and optimize the parameters to be optimized according to the gradient, screen out the atoms that meet the preset conditions, and recalculate the new loss function until the
  • the convergence condition is to reconstruct the three-dimensional space coordinates of the center of the local orbit function and the shape of the local orbit function in the real space to obtain a three-dimensional reconstruction result.
  • collecting image data of the sample at the multiple tilt angles includes: acquiring initial image data of the sample at the multiple tilt angles; The image data is subjected to alignment and noise reduction processing, and the processed image is normalized to obtain the image data.
  • the calculation formula of the loss function is:
  • W represents the loss function
  • M represents the total number of tilt angles
  • j represents the serial number of the tilt angle
  • i represents the local orbit function serial number
  • P represents the side length of the image
  • u represents the abscissa of each pixel in the image
  • v represents the ordinate of each pixel in the image
  • f j (u, v) represents the image calculated at the j-th angle
  • b j (u, v) represents the image obtained experimentally at the j-th angle.
  • filtering out atoms that meet preset conditions includes: after iteratively updating the parameters at each step, detecting the parameters of all local orbital functions before detecting any local When the parameters of the orbital function are less than the threshold obtained from all the parameters of the local orbital function, while deleting any of the local orbital functions, a local orbital function whose center distance is smaller than the preset pixel is obtained by establishing a binary tree. For the index, delete any local orbit function in the pair of local orbit functions; after each step iteratively updates the parameters and deletes the local orbit function, reduce the parameters of each local orbit function by a preset probability. is a preset multiple, and the protection time is set so that filtering operations and deletion operations are not allowed to be performed within the protection time.
  • the parameters to be optimized include the three-dimensional spatial coordinates of the center of each local orbit function, parameters describing its shape, three Euler angles corresponding to each corner, and each At least one of the drift of the sample under the angle and the mechanical tilt deviation of the sample stage.
  • the second embodiment of the present application provides a three-dimensional reconstruction device of a local orbit function, including: a collection module for collecting image data of samples at multiple tilt angles; an accumulation module for based on the image data , scatter points at equal intervals in the real space and use linear accumulation to obtain the calculated image at each tilt angle of the multiple tilt angles; and a reconstruction module used to calculate the image at each tilt angle according to Calculate the loss function, obtain the gradient of the loss function with respect to the parameters to be optimized, and optimize the parameters to be optimized according to the gradient, screen out atoms that meet the preset conditions, and recalculate the new loss function until the convergence conditions are met, The three-dimensional space coordinates of the center of the local orbit function and the shape of the local orbit function are reconstructed in the real space to obtain a three-dimensional reconstruction result.
  • the acquisition module includes: an acquisition unit, used to acquire initial image data of the sample at the multiple tilt angles; and a noise reduction unit, used to perform the initial image data on the sample.
  • the image data is subjected to alignment and noise reduction processing, and the processed image is normalized to obtain the image data.
  • the calculation formula of the loss function is:
  • W represents the loss function
  • M represents the total number of tilt angles
  • j represents the serial number of the tilt angle
  • i represents the local orbit function serial number
  • P represents the side length of the image
  • u represents the abscissa of each pixel in the image
  • v represents the ordinate of each pixel in the image
  • f j (u, v) represents the image calculated at the j-th angle
  • b j (u, v) represents the image obtained experimentally at the j-th angle.
  • the reconstruction module includes: a detection unit, used to detect the parameter deletion unit of all local orbit functions after the parameters are iteratively updated in each step, and a unit used to detect the parameter deletion unit of all local orbit functions after detecting
  • a detection unit used to detect the parameter deletion unit of all local orbit functions after the parameters are iteratively updated in each step
  • a unit used to detect the parameter deletion unit of all local orbit functions after detecting When the parameters of any local orbit function are less than the threshold obtained from all the parameters of the local orbit function, while deleting the any local orbit function, a binary tree is established to obtain the center distance of the local orbit function less than the preset pixel.
  • the local orbit function pair index is used to delete any local orbit function in the local orbit function pair; the protection unit is used to update each local orbit function after iteratively updating the parameters and deleting the local orbit function at each step.
  • the parameters of the orbital function are all reduced to a preset multiple with a preset probability, and a protection time is set so that filtering operations and deletion operations
  • the parameters to be optimized include the three-dimensional spatial coordinates of the center of each local orbit function, parameters describing its shape, three Euler angles corresponding to each corner, and each At least one of the drift of the sample under the angle and the mechanical tilt deviation of the sample stage.
  • a third embodiment of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor.
  • the processor executes the program to implement The three-dimensional reconstruction method of local orbital function as described in the above embodiment.
  • a fourth embodiment of the present application provides a computer-readable storage medium that stores computer instructions, and the computer instructions are used to cause the computer to execute the local orbit function as described in the above embodiment.
  • Three-dimensional reconstruction method Three-dimensional reconstruction method.
  • the embodiment of the present application obtains calculated images at multiple tilt angles based on the collected image data of the sample at multiple tilt angles, and then calculates the loss function of the calculated image, and obtains the gradient of the optimized parameters, thereby obtaining the optimized parameters. , after repeated screening and calculation, until the loss function meets the convergence conditions, the three-dimensional reconstruction result is obtained, which can streamline the three-dimensional coordinate reconstruction process, not only reducing the requirements for hardware, but also reducing tedious human intervention and saving labor. At the same time, during the iterative process, sample drift and mechanical tilt error of the sample stage can also be corrected, thereby improving the accuracy of three-dimensional coordinate reconstruction.
  • Figure 1 is a flow chart of a three-dimensional reconstruction method of local orbital functions provided according to an embodiment of the present application
  • Figure 2 is a flow chart of a three-dimensional reconstruction method of local orbital functions according to an embodiment of the present application
  • Figure 3 is a schematic diagram of simulations at 25°, 0° and -25° of small particles composed of 10,000 atoms to be reconstructed according to an embodiment of the present application;
  • Figure 4 is a schematic diagram of scatter points composed of the initial input atomic coordinates of the three-dimensional reconstruction method of the local orbital function according to an embodiment of the present application;
  • Figure 5 is a schematic diagram of the calculation of the initial input local orbit function at tilt angles of 25°, 0° and -25° respectively according to the three-dimensional reconstruction method of the local orbit function according to an embodiment of the present application;
  • Figure 6 is a schematic polyline diagram of the value of the loss function during the iterative process of the three-dimensional reconstruction method of the local orbit function according to an embodiment of the present application;
  • Figure 7 is a schematic diagram of an atomic model obtained after convergence of the three-dimensional reconstruction method of local orbital functions according to an embodiment of the present application
  • Figure 8 shows the difference between the calculated image and the experimental image during convergence of the three-dimensional reconstruction method of the local orbit function according to one embodiment of the present application
  • Figure 9 is a histogram of the distance between the atomic coordinates and the real coordinates obtained by the convergence calculation of the three-dimensional reconstruction method of the local orbital function according to an embodiment of the present application;
  • Figure 10 is a schematic structural diagram of a three-dimensional reconstruction device of a local orbit function according to an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
  • this application provides a three-dimensional reconstruction method of local orbit function.
  • this method multiple tilt angles are obtained based on the collected image data of the sample at multiple tilt angles. Calculate the image under the calculation image, then calculate the loss function of the calculation image, obtain the gradient of the optimization parameters, and obtain the optimization parameters.
  • FIG. 1 is a schematic flowchart of a three-dimensional reconstruction method of a local orbital function provided by an embodiment of the present application.
  • the three-dimensional reconstruction method of the local orbit function includes the following steps:
  • step S101 image data of the sample at multiple tilt angles is collected.
  • the embodiment of the present application can capture HAADF (High-Angle Annular Dark-Field imaging) images of the sample at a series of different rotation angles, and then obtain the sample at multiple tilt angles.
  • HAADF High-Angle Annular Dark-Field imaging
  • the image data is convenient for subsequent processing of image data and three-dimensional coordinate reconstruction.
  • collecting image data of samples at multiple tilt angles includes: acquiring initial image data of samples at multiple tilt angles; centering and combining the initial image data. axis and noise reduction processing, and normalized by the processed image to obtain image data.
  • the initial image data can be subjected to centering and axis and noise reduction processing. While facilitating subsequent calculations, it avoids the impact of noise on three-dimensional coordinate reconstruction, and then through linear normalization, image data that can be used for subsequent calculations is obtained.
  • M represents the total number of tilt angles
  • j represents the serial number of the tilt angle
  • u represents the abscissa of each pixel in the image
  • v represents the ordinate of each pixel in the image
  • b j (u, v) represents the jth Images obtained from experiments at several angles.
  • step S102 based on the image data, points are scattered at equal intervals in the real space and linear accumulation is used to obtain calculated images at each tilt angle of multiple tilt angles.
  • the embodiment of the present application can first set up a discretized and limited number of local orbit functions in real space based on the image data obtained in the above steps as the initial input of the iterative process, where each The strength H of the local orbital function can be set to 1e -5 , the width can be set to 1.4, the sample drift at each angle (u j , v j ) and the offset of the Euler rotation angle Both can be set to 0.
  • the embodiment of the present application can express the three-dimensional coordinates of the local orbit function center at different angles as:
  • this embodiment of the application can calculate the coordinates of the local orbit center at each angle:
  • embodiments of the present application can represent the local orbit function as a three-dimensional Gaussian function based on the above coordinates, and calculate the value of each local orbit function at each angle:
  • D ij represents the value of the i-th local orbit function at the j-th angle in the real space (u, v, w)
  • H i and B i are the parameters to be optimized, representing the i-th local orbit function respectively.
  • the intensity and width of the orbit function are the three-dimensional position coordinates of the real space
  • (u ij , v ij , w ij ) are the center three-dimensional position coordinates of the i-th local orbit function at the j-th angle
  • (u j ,v j ,w j ) is the three-dimensional drift of the local orbit function center relative to the true position (u ij ,v ij ,w ij ) at the jth angle.
  • N represents the total number of local orbit functions
  • D ij represents the value of the i-th local orbit function at the j-th angle in the real space (u, v, w).
  • step S103 the loss function is calculated based on the calculation image at each tilt angle, and the gradient of the loss function with respect to the parameters to be optimized is obtained, and the parameters to be optimized are optimized according to the gradient, atoms that meet the preset conditions are screened out, and new atoms are recalculated.
  • the embodiments of the present application can use the gradient optimization algorithm to solve the parameters, and reconstruct the three-dimensional space coordinates of the center of the local orbit function and the shape of the local orbit function in real space through multiple calculations and screenings. Obtain three-dimensional reconstruction results.
  • embodiments of the present application can use the calculation images calculated in the above steps to calculate the loss function, and then obtain the gradient of the loss function with respect to the parameters to be optimized, and use the gradient optimization algorithm to optimize the parameters to be optimized, and then filter out the parameters that satisfy the preset
  • the atom of the condition is calculated repeatedly until the convergence condition is met, thereby reconstructing the three-dimensional space coordinates of the center of the local orbital function and the shape of the local orbital function in real space, and obtaining the three-dimensional reconstruction result.
  • the embodiments of the present application can directly obtain the three-dimensional coordinates of the sample atoms, skipping the process of first obtaining the three-dimensional density matrix and then searching for peaks to obtain the three-dimensional coordinates. This not only reduces the requirements for hardware, but also eliminates cumbersome human intervention in the process of peak searching. This saves labor costs, and at the same time, during the iterative process, sample drift and mechanical tilt errors of the sample stage can be corrected, thereby improving the accuracy of three-dimensional coordinate reconstruction.
  • preset conditions can be set by those skilled in the art according to actual conditions, and are not specifically limited here.
  • the calculation formula of the loss function is:
  • W represents the loss function
  • M represents the total number of tilt angles
  • j represents the serial number of the tilt angle
  • i represents the local orbit function serial number
  • P represents the side length of the image
  • u represents the abscissa of each pixel in the image
  • v represents the ordinate of each pixel in the image
  • f j (u, v) represents the image calculated at the j-th angle
  • b j (u, v) represents the image obtained experimentally at the j-th angle.
  • the embodiment of the present application can calculate the loss function through the calculation image in the above steps, and write the loss function as a function about the calculation image and the experimental image:
  • W represents the loss function
  • M represents the total number of tilt angles
  • j represents the serial number of the tilt angle
  • i represents the local orbit function serial number
  • P represents the side length of the image
  • u represents the abscissa of each pixel in the image
  • v represents the ordinate of each pixel in the image
  • f j (u, v) represents the image calculated at the j-th angle
  • b j (u, v) represents the experimental image at the j-th angle
  • m j (u ,v) is the difference between the calculated image and the experimental image at the jth angle.
  • the parameters to be optimized include the three-dimensional spatial coordinates of the center of each local orbit function, parameters describing its shape, three Euler angles corresponding to each corner, and At least one of the drift of the sample and the mechanical tilt deviation of the sample stage.
  • the embodiment of the present application can calculate the loss function with respect to the three-dimensional coordinates (x i , y i , z i ) of the center of the local orbit function, the intensity Hi , the width Bi , and the sample drift amount at each angle ( u j ,v j ), sample stage angle deviation gradient of equal parameters.
  • this embodiment of the present application can use the gradient obtained by calculation to update the target parameters:
  • the gradient can be obtained using a software library with automatic derivation function or through analytical expressions.
  • screening out atoms that meet preset conditions includes: after iteratively updating the parameters in each step, detecting the parameters of all local orbital functions and detecting any local orbital function.
  • the parameters of are less than the threshold obtained from all local orbit function parameters, while deleting any local orbit function, establish a binary tree to obtain the local orbit function pair index whose center distance is less than the preset pixel, and delete the local orbit function Any local orbital function in the orbital function pair; after each step iteratively updates the parameters and deletes the local orbital function, the parameters of each local orbital function are reduced to a preset multiple with a preset probability, and set Protection time, so that filtering operations and deletion operations are not allowed during the protection time.
  • screening out atoms that meet preset conditions includes deleting redundant local orbital functions and screening local orbital functions.
  • the method of deleting redundant local orbital functions is: after updating the parameters iteratively at each step, check the parameters H of all local orbital functions. If the parameter H of a certain local orbital function is less than the parameter H of all local orbital functions. When the threshold value is obtained, the local orbit function is deleted. At the same time, by establishing a binary tree, the local orbit function pair index of the local orbit function center distance from the preset pixel is obtained, and any one of the local orbit function pairs is deleted. Orbital function.
  • the threshold can be set by those skilled in the art according to the actual situation, or can be set as a reference value, such as 0.01 times the maximum value of all local orbit function parameters H; the preset pixel can be set by those skilled in the art according to Set according to the actual situation, or set to a reference value, such as 2 pixels.
  • the method of screening local orbital functions is as follows: after iteratively updating the parameters and deleting the local orbital functions at each step, the preset probability of the parameter H of each local orbital function is reduced to a preset multiple, and a protection time is set. Local orbital functions cannot be filtered and deleted during the protection time.
  • the preset probability and preset multiple can be set by those skilled in the art according to the actual situation, or can be set as a reference value.
  • the parameter H of each local orbit function is reduced to 0.1 times with a probability of 0.02. ;
  • the embodiment of the present application includes the following steps:
  • Step S201 Collect image data.
  • the embodiment of this application needs to reconstruct a small particle composed of 10,000 atoms, and obtain its simulation by tilting at angles of ⁇ 25°, ⁇ 20°, ⁇ 15°, ⁇ 5°, and 0°.
  • Image, columns 1, 2, and 3 in Figure 3 correspond to images of 25°, 0°, and -25° respectively.
  • Step S202 Calculate the image.
  • Embodiments of the present application can scatter points at equal intervals in real space based on image data and use linear accumulation to obtain calculated images at each tilt angle of multiple tilt angles.
  • the initial scatter point set is a cylinder composed of equally spaced atoms.
  • the calculated images at various angles at the initial iteration are obtained, among which the 1st, 2nd, and 3rd columns correspond to images of 25°, 0°, and -25° respectively.
  • Step S203 Calculate the loss function and iteratively update the target parameters.
  • the embodiment of this application can calculate the loss function and the gradient of the target parameters, and use the following formula to iteratively update the target parameters:
  • the line chart of the value of the loss function with the iterative process is shown in Figure 6; the schematic diagram of the atomic model obtained during final convergence is shown in Figure 7; the difference reference diagram between the calculated image and the experimental image is shown in Figure 8, where , the first column is the calculation diagram, the second column is the experimental diagram, the third column is the difference between the two, the first row is the data when the tilt angle is 25°, the second row is the data when the tilt angle is 20°;
  • the distance histogram between the calculated atomic coordinates and the real coordinates is shown in Figure 9.
  • the three-dimensional reconstruction method of the local orbit function proposed in the embodiment of the present application, based on the collected image data of the sample at multiple tilt angles, calculated images at multiple tilt angles are obtained, and then the loss of the calculated image is calculated function to obtain the gradient of the optimization parameters, thereby obtaining the optimization parameters.
  • the three-dimensional reconstruction result is obtained, which can streamline the three-dimensional coordinate reconstruction process, not only reducing the need for hardware requirements, it can also reduce tedious human intervention and save labor costs.
  • iteration process it can also correct sample drift and mechanical tilt errors of the sample stage, thereby improving the accuracy of three-dimensional coordinate reconstruction.
  • Figure 10 is a block schematic diagram of a three-dimensional reconstruction device of a local orbit function according to an embodiment of the present application.
  • the three-dimensional reconstruction device 10 of the local orbit function includes: an acquisition module 100, an accumulation module 200 and a reconstruction module 300.
  • the acquisition module 100 is used to acquire image data of samples at multiple tilt angles.
  • the accumulation module 200 is used to scatter points at equal intervals in the real space based on the image data and use linear accumulation to obtain calculated images at each tilt angle of multiple tilt angles.
  • the reconstruction module 300 is used to calculate the loss function based on the calculated image at each tilt angle, obtain the gradient of the loss function with respect to the parameters to be optimized, and optimize the parameters to be optimized based on the gradient, screen out atoms that meet the preset conditions, and re- count Calculate a new loss function until the convergence conditions are met, reconstruct the three-dimensional space coordinates of the center of the local orbit function and the shape of the local orbit function in real space, and obtain the three-dimensional reconstruction result.
  • the collection module 100 includes: an acquisition unit and a noise reduction unit.
  • the acquisition unit is used to acquire initial image data of the sample at multiple tilt angles.
  • the noise reduction unit is used to perform centering and noise reduction processing on the initial image data, and normalizes the processed image to obtain image data.
  • the calculation formula of the loss function is:
  • W represents the loss function
  • M represents the total number of tilt angles
  • j represents the serial number of the tilt angle
  • i represents the local orbit function serial number
  • P represents the side length of the image
  • u represents the abscissa of each pixel in the image
  • v represents the ordinate of each pixel in the image
  • f j (u, v) represents the image calculated at the j-th angle
  • b j (u, v) represents the image obtained experimentally at the j-th angle.
  • the reconstruction module 300 includes: a detection unit and a protection unit.
  • the detection unit is used to detect the parameter deletion unit of all local orbit functions after iteratively updating the parameters in each step, and is used to detect that the parameters of any local orbit function are smaller than those obtained from all the local orbit function parameters.
  • the threshold while deleting any local orbit function, establish a binary tree to obtain the local orbit function pair index whose center distance is smaller than the preset pixel, and delete any local orbit function in the local orbit function pair.
  • the protection unit is used to reduce the parameters of each local orbit function to a preset multiple with a preset probability after iteratively updating the parameters and deleting the local orbit function at each step, and sets the protection time so that during the protection During this time, filtering operations and deletion operations are not allowed.
  • the parameters to be optimized include the three-dimensional spatial coordinates of the center of each local orbit function, parameters describing its shape, three Euler angles corresponding to each corner, and At least one of the drift of the sample and the mechanical tilt deviation of the sample stage.
  • the three-dimensional reconstruction device of the local orbit function proposed in the embodiment of the present application, based on the collected image data of the sample at multiple tilt angles, calculated images at multiple tilt angles are obtained, and then the loss of the calculated image is calculated function to obtain the gradient of the optimization parameters, thereby obtaining the optimization parameters.
  • the three-dimensional reconstruction result is obtained, which can streamline the three-dimensional coordinate reconstruction process, not only reducing the need for hardware requirements, it can also reduce tedious human intervention and save labor costs.
  • iteration process it can also correct sample drift and mechanical tilt errors of the sample stage, thereby improving the accuracy of three-dimensional coordinate reconstruction.
  • FIG 11 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • the electronic device may include:
  • the processor 1102 executes the program, it implements the three-dimensional reconstruction method of the local orbit function provided in the above embodiment.
  • electronic equipment also includes:
  • Communication interface 1103 is used for communication between the memory 1101 and the processor 1102.
  • Memory 1101 is used to store computer programs that can run on the processor 1102.
  • the memory 1101 may include high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
  • the bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc.
  • ISA Industry Standard Architecture
  • PCI Peripheral Component
  • EISA Extended Industry Standard Architecture
  • the bus can be divided into address bus, data bus, control bus, etc. For ease of presentation, only one thick line is used in Figure 11, but it does not mean that there is only one bus or one type of bus.
  • the memory 1101, the processor 1102 and the communication interface 1103 are integrated on one chip, the memory 1101, the processor 1102 and the communication interface 1103 can communicate with each other through the internal interface.
  • the processor 1102 may be a central processing unit (Central Processing Unit, CPU for short), or an Application Specific Integrated Circuit (ASIC for short), or one or more processors configured to implement the embodiments of the present application. integrated circuit.
  • CPU Central Processing Unit
  • ASIC Application Specific Integrated Circuit
  • This embodiment also provides a computer-readable storage medium on which a computer program is stored.
  • the program is executed by a processor, the above three-dimensional reconstruction method of the local orbital function is implemented.
  • first and second are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Therefore, features defined as “first” and “second” may explicitly or implicitly include at least one of these features. In the description of this application, “N” means at least two, such as two, three, etc., unless otherwise clearly and specifically limited.
  • a "computer-readable medium” may be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
  • Non-exhaustive list of computer readable media include the following: electrical connections with one or N wires (electronic device), portable computer disk cartridge (magnetic device), random access memory (RAM), Read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable compact disc read-only memory (CDROM).
  • the computer-readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, and subsequently edited, interpreted, or otherwise suitable as necessary. process to obtain the program electronically and then store it in computer memory.
  • N steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system.
  • a suitable instruction execution system For example, if it is implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: discrete logic gate circuits with logic functions for implementing data signals; Logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.
  • each functional unit in various embodiments of the present application can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or software function modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
  • the storage media mentioned above can be read-only memory, magnetic disks or optical disks, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Theoretical Computer Science (AREA)
  • Pure & Applied Mathematics (AREA)
  • Software Systems (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Data Mining & Analysis (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Processing (AREA)

Abstract

本申请公开了一种局域轨道函数的三维重构方法及装置，其中，方法包括：采集多个倾转角度下的样品的图像数据，在实空间等间隔进行撒点并利用线性累加，得到多个倾转角度的每个倾转角度下的计算图像，进而获取损失函数关于待优化参数的梯度，且根据梯度优化待优化参数，筛选出满足预设条件的原子，重新计算新的损失函数，直至满足收敛条件，在实空间重构局域轨道函数的中心的三维空间坐标和局域轨道函数的形状，得到三维重构结果。由此，解决了相关技术中只能从重构出的三维密度矩阵中获取原子的三维坐标，且无法对误差进行校正，导致重构的过程对硬件需求较高，且重构的三维坐标精度较差的技术问题。

Description

局域轨道函数的三维重构方法及装置
相关申请的交叉引用
本申请基于申请号为202210381325.3，申请日为2022年04月12日申请的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本申请涉及三维成像技术领域，特别涉及一种局域轨道函数的三维重构方法及装置。
背景技术
通常情况下样品的电子显微镜照片中蕴含着样品丰富的、难以直观获取的信息，应用三维重构算法从样品一系列电子显微镜投影照片中获取样品的三维结构信息对在根本的层面上理解材料成分和性能的关系有很大帮助。
近年来，随着数据采集方法、迭代三维重建算法和后处理方法的发展，AET(Atomic Electrical Tomography，原子尺度电子层析成像)已成为三维和四维原子尺度结构表征的有力工具，它提供了在原子水平上关联材料结构和性质的能力，相关技术中的AET能够以亚埃精度确定三维原子坐标和元素种类，并揭示其在动态过程中的原子尺度时间演化。
然而，相关技术中，AET算法从重构出的三维密度矩阵中通过寻峰来获取原子的三维坐标，对硬件的需求较高，且在寻峰过程中，需要进行繁琐的人为干预，难以避免因人为干预造成的误差，同时，在直接对原子的三维坐标进行重构的同时，无法对采集图像时存在的样品漂移、样品台的机械倾转误差进行矫正，准确度较差，有待改善。
发明内容
本申请提供一种局域轨道函数的三维重构方法及装置，以解决相关技术中只能从重构出的三维密度矩阵中获取原子的三维坐标，且无法对误差进行校正，导致重构的过程对硬件需求较高，且重构的三维坐标精度较差的技术问题。
本申请第一方面实施例提供一种局域轨道函数的三维重构方法，包括以下步骤：采集多个倾转角度下的样品的图像数据；基于所述图像数据，在实空间等间隔进行撒点并利用线性累加，得到所述多个倾转角度的每个倾转角度下的计算图像；以及根据每个倾转角度下的计算图像计算损失函数，并获取所述损失函数关于待优化参数的梯度，且根据所述梯度优化所述待优化参数，筛选出满足预设条件的原子，重新计算新的损失函数，直至满足 收敛条件，在所述实空间重构局域轨道函数的中心的三维空间坐标和局域轨道函数的形状，得到三维重构结果。
可选地，在本申请的一个实施例中，采集所述多个倾转角度下的样品的图像数据，包括：获取所述多个倾转角度下的样品的初始图像数据；对所述初始图像数据进行对中合轴和降噪处理，并由处理后的图像归一化，得到所述图像数据。
可选地，在本申请的一个实施例中，所述损失函数的计算公式为：
其中，W代表损失函数，M代表倾转角度的总数，j代表倾转角度的序号，i代表局域轨道函数序号，P代表图像的边长，u代表图像中每个像素的横坐标，v代表图像中每个像素的纵坐标，fj(u,v)代表第j个角度下计算得到的图像，bj(u,v)代表第j个角度下实验所得的图像。
可选地，在本申请的一个实施例中，所述筛选出满足预设条件的原子，包括：在每一步迭代更新完参数后，检测所有局域轨道函数的参数在检测到任一局域轨道函数的参数小于由所述所有局域轨道函数参数得到的阈值时，删除所述任一局域轨道函数的同时，通过建立二叉树得到局域轨道函数中心距离小于预设像素的局域轨道函数对索引，删除局域轨道函数对中的任意一个局域轨道函数；每一步迭代更新完参数并删除完所述局域轨道函数后，将每个局域轨道函数的参数均以预设概率减小为预设倍数，且设置保护时间，使得在所述保护时间内，不允许执行筛选操作和删除操作。
可选地，在本申请的一个实施例中，所述待优化参数包括每个局域轨道函数中心的三维空间坐标、描述其形状的参数、每个转角对应的三个欧拉角、每个角度下样品的漂移、样品台的机械倾转偏差中的至少一项。
本申请第二方面实施例提供一种局域轨道函数的三维重构装置，包括：采集模块，用于采集多个倾转角度下的样品的图像数据；累加模块，用于基于所述图像数据，在实空间等间隔进行撒点并利用线性累加，得到所述多个倾转角度的每个倾转角度下的计算图像；以及重构模块，用于根据每个倾转角度下的计算图像计算损失函数，并获取所述损失函数关于待优化参数的梯度，且根据所述梯度优化所述待优化参数，筛选出满足预设条件的原子，重新计算新的损失函数，直至满足收敛条件，在所述实空间重构局域轨道函数的中心的三维空间坐标和局域轨道函数的形状，得到三维重构结果。
可选地，在本申请的一个实施例中，所述采集模块包括：获取单元，用于获取所述多个倾转角度下的样品的初始图像数据；降噪单元，用于对所述初始图像数据进行对中合轴和降噪处理，并由处理后的图像归一化，得到所述图像数据。
可选地，在本申请的一个实施例中，所述损失函数的计算公式为：
其中，W代表损失函数，M代表倾转角度的总数，j代表倾转角度的序号，i代表局域轨道函数序号，P代表图像的边长，u代表图像中每个像素的横坐标，v代表图像中每个像素的纵坐标，fj(u,v)代表第j个角度下计算得到的图像，bj(u,v)代表第j个角度下实验所得的图像。
可选地，在本申请的一个实施例中，所述重构模块包括：检测单元，用于在每一步迭代更新完参数后，检测所有局域轨道函数的参数删除单元，用于在检测到任一局域轨道函数的参数小于由所述所有局域轨道函数参数得到的阈值时，删除所述任一局域轨道函数的同时，通过建立二叉树得到局域轨道函数中心距离小于预设像素的局域轨道函数对索引，删除局域轨道函数对中的任意一个局域轨道函数；保护单元，用于在每一步迭代更新完参数并删除完所述局域轨道函数后，将每个局域轨道函数的参数均以预设概率减小为预设倍数，且设置保护时间，使得在所述保护时间内，不允许执行筛选操作和删除操作。
可选地，在本申请的一个实施例中，所述待优化参数包括每个局域轨道函数中心的三维空间坐标、描述其形状的参数、每个转角对应的三个欧拉角、每个角度下样品的漂移、样品台的机械倾转偏差中的至少一项。
本申请第三方面实施例提供一种电子设备，包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述程序，以实现如上述实施例所述的局域轨道函数的三维重构方法。
本申请第四方面实施例提供一种计算机可读存储介质，所述计算机可读存储介质存储计算机指令，所述计算机指令用于使所述计算机执行如上述实施例所述的局域轨道函数的三维重构方法。
本申请实施例基于采集的多个倾转角度下的样品的图像数据，获得多个倾转角度下的计算图像，进而计算出计算图像的损失函数，求取优化参数的梯度，从而获得优化参数，在经过反复筛选和计算后，直至损失函数满足收敛条件，进而获得三维重构结果，可以精简三维坐标的重构过程，不仅降低了对硬件的要求，还能减少繁琐的人为干预，节约劳动力成本，同时在迭代过程中，还可以矫正样品漂移和样品台的机械倾转误差，进而提高三维坐标重构的准确性。由此，解决了相关技术中只能从重构出的三维密度矩阵中获取原子的三维坐标，且无法对误差进行校正，导致重构的过程对硬件需求较高，且重构的三维坐标精度较差的技术问题。
本申请附加的方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本申请的实践了解到。
附图说明
本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：
图1为根据本申请实施例提供的一种局域轨道函数的三维重构方法的流程图；
图2为根据本申请一个实施例的局域轨道函数的三维重构方法的流程图；
图3为根据本申请一个实施例的待重构的10000个原子构成的小颗粒在25°、0°和-25°的模拟示意图；
图4为根据本申请一个实施例的局域轨道函数的三维重构方法的初始输入的原子坐标构成的散点示意图；
图5为根据本申请一个实施例的局域轨道函数的三维重构方法的初始输入的局域轨道函数在倾转角度分别为25°、0°和-25°的计算示意图；
图6为根据本申请一个实施例的局域轨道函数的三维重构方法的迭代过程中损失函数的值的折线示意图；
图7为根据本申请一个实施例的局域轨道函数的三维重构方法的收敛后获得的原子模型示意图；
图8为根据本申请一个实施例的局域轨道函数的三维重构方法的收敛时计算图像与实验图像的差值；
图9为根据本申请一个实施例的局域轨道函数的三维重构方法的收敛计算所得原子坐标与真实坐标之间的距离直方图；
图10为根据本申请实施例提供的一种局域轨道函数的三维重构装置的结构示意图；
图11为根据本申请实施例提供的电子设备的结构示意图。
具体实施方式
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，旨在用于解释本申请，而不能理解为对本申请的限制。
下面参考附图描述本申请实施例的局域轨道函数的三维重构方法及装置。针对上述背景技术中心提到的相关技术只能从重构出的三维密度矩阵中获取原子的三维坐标，且无法对误差进行校正，导致重构的过程对硬件需求较高，且重构的三维坐标精度较差的技术问题，本申请提供了一种局域轨道函数的三维重构方法，在该方法中，基于采集的多个倾转角度下的样品的图像数据，获得多个倾转角度下的计算图像，进而计算出计算图像的损失函数，求取优化参数的梯度，从而获得优化参数，在经过反复筛选和计算后，直至损失函 数满足收敛条件，进而获得三维重构结果，可以精简三维坐标的重构过程，不仅降低了对硬件的要求，还能减少繁琐的人为干预，节约劳动力成本，同时在迭代过程中，还可以矫正样品漂移和样品台的机械倾转误差，进而提高三维坐标重构的准确性。由此，解决了相关技术中只能从重构出的三维密度矩阵中获取原子的三维坐标，且无法对误差进行校正，导致重构的过程对硬件需求较高，且重构的三维坐标精度较差的技术问题。
具体而言，图1为本申请实施例所提供的一种局域轨道函数的三维重构方法的流程示意图。
如图1所示，该局域轨道函数的三维重构方法包括以下步骤：
在步骤S101中，采集多个倾转角度下的样品的图像数据。
在实际执行过程中，本申请实施例可以拍摄样品在一系列不同转角下的HAADF(High-Angle Annular Dark-Field imaging，高角环形暗场像)图像，进而获得样品多个倾转角度下的样品的图像数据，便于后续对图像数据进行处理，并进行三维坐标重构。
可选地，在本申请的一个实施例中，采集多个倾转角度下的样品的图像数据，包括：获取多个倾转角度下的样品的初始图像数据；对初始图像数据进行对中合轴和降噪处理，并由处理后的图像归一化，得到图像数据。
具体地，本申请实施例通过拍摄获得多个倾转角度下的样品的初始图像数据后，为保证后续三维坐标重构的准确性，可以对初始图像数据进行对中和轴和降噪处理，在便于后续计算的同时，避免噪点对三维坐标重构的影响，再通过线性归一化，得到可用于后续计算的图像数据。
其中，线性归一化的具体过程如下：
其中，M代表倾转角度的总数，j代表倾转角度的序号，u代表图像中每个像素的横坐标，v代表图像中每个像素的纵坐标，bj(u,v)代表第j个角度下实验得到的图像。
在步骤S102中，基于图像数据，在实空间等间隔进行撒点并利用线性累加，得到多个倾转角度的每个倾转角度下的计算图像。
作为一种可能实现的方式，本申请实施例首先可以基于上述步骤获得的图像数据，在实空间以一定规律设置离散化、有限数目的局域轨道函数作为迭代过程的初始输入，其中，每个局域轨道函数的强度H可以设置为1e-5，宽度可以设置为1.4，各角度下的样品漂移量(uj,vj)和欧拉转角的偏移量皆可以设置为0。
具体地，本申请实施例可以将不同角度下的局域轨道函数中心的三维坐标表示为：
其中，为第j个角度对应的分布绕x轴、y轴、z轴的欧拉角，(xi,yi,zi)为第i个局域轨道函数在未旋转(三个欧拉角皆为0)时的中心三维位置坐标，(uij,vij,wij)为第i个局域轨道函数在第j个角度下的中心三维位置坐标。
进一步地，本申请实施例可以计算各角度下局域轨道中心的坐标：
进而，本申请实施例可以根据上述坐标，将局域轨道函数表示为三维高斯函数，计算每个局域轨道函数在各个角度下的值：
其中，Dij表示第i个局域轨道函数在第j个角度下在实空间(u,v,w)位置处的值，Hi和Bi为待优化参数，分别代表第i个局域轨道函数的强度与宽度，(u,v,w)为实空间三维位置坐标，(uij,vij,wij)为第i个局域轨道函数在第j个角度下的中心三维位置坐标，(uj,vj,wj)为第j个角度下局域轨道函数中心相对于真实位置(uij,vij,wij)的三维方向漂移。
再通过线性累加，本申请实施例可以得到计算图像：
其中，N代表局域轨道函数总个数，Dij表示第i个局域轨道函数在第j个角度下在实空间(u,v,w)位置处的值。
在步骤S103中，根据每个倾转角度下的计算图像计算损失函数，并获取损失函数关于待优化参数的梯度，且根据梯度优化待优化参数，筛选出满足预设条件的原子，重新计算新的损失函数，直至满足收敛条件，在实空间重构局域轨道函数的中心的三维空间坐标和局域轨道函数的形状，得到三维重构结果。
在实际执行过程中，本申请实施例可以利用梯度的优化算法求解参数，并通过多次计算和筛选，在实空间重构局域轨道函数的中心的三维空间坐标和局域轨道函数的形状，得到三维重构结果。
具体地，本申请实施例可以利用上述步骤中计算获得的计算图像，计算损失函数，进而获取损失函数关于待优化参数的梯度，利用梯度的优化算法，优化待优化参数，进而筛选出满足预设条件的原子，并重复计算损失函数，直至满足收敛条件，从而在实空间重构局域轨道函数的中心的三维空间坐标和局域轨道函数的形状，得到三维重构结果。
本申请实施例可以直接获得样品原子的三维坐标，跳过了先得到三维密度矩阵再寻峰得到三维坐标的过程，不仅降低了对硬件的要求，还去除了寻峰过程中的繁琐人为干预，节约了劳动成本，同时在迭代过程中，还可以矫正样品漂移和样品台的机械倾转误差，进而提高三维坐标重构的准确性。
需要注意的是，预设条件可以由本领域技术人员根据实际情况进行设置，在此不做具体限制。
可选地，在本申请的一个实施例中，损失函数的计算公式为：
其中，W代表损失函数，M代表倾转角度的总数，j代表倾转角度的序号，i代表局域轨道函数序号，P代表图像的边长，u代表图像中每个像素的横坐标，v代表图像中每个像素的纵坐标，fj(u,v)代表第j个角度下计算得到的图像，bj(u,v)代表第j个角度下实验所得的图像。
具体地，本申请实施例可以通过上述步骤中的计算图像计算损失函数，并将损失函数写为关于计算图像和实验图像的函数：
其中，W代表损失函数，M代表倾转角度的总数，j代表倾转角度的序号，i代表局域轨道函数序号，P代表图像的边长，u代表图像中每个像素的横坐标，v代表图像中每个像素的纵坐标，fj(u,v)代表第j个角度下计算得到的图像，bj(u,v)代表第j个角度下实验所得的图像，mj(u,v)为第j个角度下计算图像与实验图像之差。
可选地，在本申请的一个实施例中，待优化参数包括每个局域轨道函数中心的三维空间坐标、描述其形状的参数，每个转角对应的三个欧拉角，每个角度下样品的漂移、样品台的机械倾转偏差中的至少一项。
在获取损失函数后，本申请实施例可以求出损失函数关于局域轨道函数中心三维坐标(xi,yi,zi)、强度Hi、宽度Bi、每个角度下样品漂移量(uj,vj)、样品台角度偏差量等参数的梯度。
进一步地，本申请实施例可以利用计算获得的梯度对目标参数进行更新：









其中，和是各个参数的学习率。
需要注意的是，梯度的求取可以利用具有自动求导功能的软件库实现，也可以通过解析表达式实现。
可选地，在本申请的一个实施例中，筛选出满足预设条件的原子，包括：在每一步迭代更新完参数后，检测所有局域轨道函数的参数在检测到任一局域轨道函数的参数小于由所有局域轨道函数参数得到的阈值时，删除任一局域轨道函数的同时，通过建立二叉树得到局域轨道函数中心距离小于预设像素的局域轨道函数对索引，删除局域轨道函数对中的任意一个局域轨道函数；每一步迭代更新完参数并删除完局域轨道函数后，将每个局域轨道函数的参数均以预设概率减小为预设倍数，且设置保护时间，使得在保护时间内，不允许执行筛选操作和删除操作。
可以理解的是，筛选出满足预设条件的原子包括删除冗余局域轨道函数和筛选局域轨道函数。
其中，删除冗余局域轨道函数的方法为：在每一步迭代更新完参数后，检查所有局域轨道函数的参数H，若某局域轨道函数的参数H小于小于由所有局域轨道函数参数得到的阈值时，则删除该局域轨道函数，同时通过建立二叉树的方法，得到局域轨道函数中心距离预设像素的局域轨道函数对索引，删除局域轨道函数对中的任意一个局域轨道函数。
需要注意的是，阈值可以由本领域技术人员根据实际情况进行设置，也可以设置为参考值，如所有局域轨道函数参数H最大值的0.01倍；预设像素可以由本领域技术人员根据 实际情况进行设置，也可以设置为参考值，如2像素。
筛选局域轨道函数的方法为：在每一步迭代更新完参数并删除完局域轨道函数后，每个局域轨道函数的参数H预设概率减小为预设倍数，且设置保护时间，在保护时间内局域轨道函数不可被筛选和删除。
需要注意的是，预设概率和预设倍数可以由本领域技术人员根据实际情况进行设置，也可以设置为参考值，如每个局域轨道函数的参数H皆以0.02的概率减小为0.1倍；保护事件可以由本领域技术人员根据实际情况进行设置，也可以设置为参考值，如设置保护时间T＝50。
下面结合图2至图9所示，以一个具体实施例对本申请实施例的局域轨道函数的三维重构方法的工作原理进行详细阐述。
如图2所示，以重构10000个原子组成的小颗粒为例，本申请实施例包括以下步骤：
步骤S201：采集图像数据。如图3所示，本申请实施例需要重构10000个原子组成的小颗粒，通过在角度为±25°、±20°、±15°、±5°、0°时倾转，获得其模拟图像，图3中第1、2、3列分别对应25°、0°和-25°的图像。
步骤S202：计算图像。本申请实施例可以基于图像数据，在实空间等间隔进行撒点并利用线性累加，得到多个倾转角度的每个倾转角度下的计算图像。如图4所示，初始撒点集合为等间隔原子构成的圆柱形，并参考图5所示，利用公式得到迭代初始的各个角度下的计算图像，其中，第1、2、3列分别对应25°、0°和-25°的图像。
步骤S203：计算损失函数，并对目标参数进行迭代更新。本申请实施例可以计算损失函数及目标参数的梯度，并利用以下公式对目标参数进行迭代更新：









其中，和是各个参数的学习率。
步骤S204：删除、筛选局域轨道函数。进一步地，本申请实施例可以删除强度H小于最强局域轨道函数强度Hmax*0.01的局域轨道函数，同时以0.02的概率重置该局域轨道函数，使局域轨道函数强度H减小为原来的0.1倍，设置保护周期T＝50。
通过循环计算损失函数，并更新各目标参数，删除和筛选局域轨道函数的操作，直至局域轨道函数数目收敛，实现三维坐标重构。
其中，损失函数的值随迭代过程的折线图如图6所示；最终收敛时得到的原子模型的示意图如图7所示；计算图像与实验图像的差值参考图如图8所示，其中，第1列为计算图，第2列为实验图，第3列为二者差值，第1行为倾转角度为25°下的数据，第2行为倾转角度为20°下的数据；计算所得原子坐标与真实坐标距离直方图如图9所示。
根据本申请实施例提出的局域轨道函数的三维重构方法，基于采集的多个倾转角度下的样品的图像数据，获得多个倾转角度下的计算图像，进而计算出计算图像的损失函数，求取优化参数的梯度，从而获得优化参数，在经过反复筛选和计算后，直至损失函数满足收敛条件，进而获得三维重构结果，可以精简三维坐标的重构过程，不仅降低了对硬件的要求，还能减少繁琐的人为干预，节约劳动力成本，同时在迭代过程中，还可以矫正样品漂移和样品台的机械倾转误差，进而提高三维坐标重构的准确性。由此，解决了相关技术中只能从重构出的三维密度矩阵中获取原子的三维坐标，且无法对误差进行校正，导致重构的过程对硬件需求较高，且重构的三维坐标精度较差的技术问题。
其次参照附图描述根据本申请实施例提出的局域轨道函数的三维重构装置。
图10是本申请实施例的局域轨道函数的三维重构装置的方框示意图。
如图10所示，该局域轨道函数的三维重构装置10包括：采集模块100、累加模块200和重构模块300。
具体地，采集模块100，用于采集多个倾转角度下的样品的图像数据。
累加模块200，用于基于图像数据，在实空间等间隔进行撒点并利用线性累加，得到多个倾转角度的每个倾转角度下的计算图像。
重构模块300，用于根据每个倾转角度下的计算图像计算损失函数，并获取损失函数关于待优化参数的梯度，且根据梯度优化待优化参数，筛选出满足预设条件的原子，重新计 算新的损失函数，直至满足收敛条件，在实空间重构局域轨道函数的中心的三维空间坐标和局域轨道函数的形状，得到三维重构结果。
可选地，在本申请的一个实施例中，采集模块100包括：获取单元和降噪单元。
其中，获取单元，用于获取多个倾转角度下的样品的初始图像数据。
降噪单元，用于对初始图像数据进行对中合轴和降噪处理，并由处理后的图像归一化，得到图像数据。
可选地，在本申请的一个实施例中，损失函数的计算公式为：
其中，W代表损失函数，M代表倾转角度的总数，j代表倾转角度的序号，i代表局域轨道函数序号，P代表图像的边长，u代表图像中每个像素的横坐标，v代表图像中每个像素的纵坐标，fj(u,v)代表第j个角度下计算得到的图像，bj(u,v)代表第j个角度下实验所得的图像。
可选地，在本申请的一个实施例中，重构模块300包括：检测单元和保护单元。
其中，检测单元，用于在每一步迭代更新完参数后，检测所有局域轨道函数的参数删除单元，用于在检测到任一局域轨道函数的参数小于由所有局域轨道函数参数得到的阈值时，删除任一局域轨道函数的同时，通过建立二叉树得到局域轨道函数中心距离小于预设像素的局域轨道函数对索引，删除局域轨道函数对中的任意一个局域轨道函数。
保护单元，用于在每一步迭代更新完参数并删除完局域轨道函数后，将每个局域轨道函数的参数均以预设概率减小为预设倍数，且设置保护时间，使得在保护时间内，不允许执行筛选操作和删除操作。
可选地，在本申请的一个实施例中，待优化参数包括每个局域轨道函数中心的三维空间坐标、描述其形状的参数，每个转角对应的三个欧拉角，每个角度下样品的漂移、样品台的机械倾转偏差中的至少一项。
需要说明的是，前述对局域轨道函数的三维重构方法实施例的解释说明也适用于该实施例的局域轨道函数的三维重构装置，此处不再赘述。
根据本申请实施例提出的局域轨道函数的三维重构装置，基于采集的多个倾转角度下的样品的图像数据，获得多个倾转角度下的计算图像，进而计算出计算图像的损失函数，求取优化参数的梯度，从而获得优化参数，在经过反复筛选和计算后，直至损失函数满足收敛条件，进而获得三维重构结果，可以精简三维坐标的重构过程，不仅降低了对硬件的要求，还能减少繁琐的人为干预，节约劳动力成本，同时在迭代过程中，还可以矫正样品漂移和样品台的机械倾转误差，进而提高三维坐标重构的准确性。由此，解决了相关技术中只能从重构出的三维密度矩阵中获取原子的三维坐标，且无法对误差进行校正，导致重 构的过程对硬件需求较高，且重构的三维坐标精度较差的技术问题。
图11为本申请实施例提供的电子设备的结构示意图。该电子设备可以包括：
存储器1101、处理器1102及存储在存储器1101上并可在处理器1102上运行的计算机程序。
处理器1102执行程序时实现上述实施例中提供的局域轨道函数的三维重构方法。
进一步地，电子设备还包括：
通信接口1103，用于存储器1101和处理器1102之间的通信。
存储器1101，用于存放可在处理器1102上运行的计算机程序。
存储器1101可能包含高速RAM存储器，也可能还包括非易失性存储器(non-volatile memory)，例如至少一个磁盘存储器。
如果存储器1101、处理器1102和通信接口1103独立实现，则通信接口1103、存储器1101和处理器1102可以通过总线相互连接并完成相互间的通信。总线可以是工业标准体系结构(Industry Standard Architecture，简称为ISA)总线、外部设备互连(Peripheral Component，简称为PCI)总线或扩展工业标准体系结构(Extended Industry Standard Architecture，简称为EISA)总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示，图11中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
可选地，在具体实现上，如果存储器1101、处理器1102及通信接口1103，集成在一块芯片上实现，则存储器1101、处理器1102及通信接口1103可以通过内部接口完成相互间的通信。
处理器1102可能是一个中央处理器(Central Processing Unit，简称为CPU)，或者是特定集成电路(Application Specific Integrated Circuit，简称为ASIC)，或者是被配置成实施本申请实施例的一个或多个集成电路。
本实施例还提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如上的局域轨道函数的三维重构方法。
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或N个实施例或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技 术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。
此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本申请的描述中，“N个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更N个用于实现定制逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本申请的优选实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本申请的实施例所属技术领域的技术人员所理解。
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的定序列表，可以具体实现在任何计算机可读介质中，以供指令执行***、装置或设备(如基于计算机的***、包括处理器的***或其他可以从指令执行***、装置或设备取指令并执行指令的***)使用，或结合这些指令执行***、装置或设备而使用。就本说明书而言，"计算机可读介质"可以是任何可以包含、存储、通信、传播或传输程序以供指令执行***、装置或设备或结合这些指令执行***、装置或设备而使用的装置。计算机可读介质的更具体的示例(非穷尽性列表)包括以下：具有一个或N个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式光盘只读存储器(CDROM)。另外，计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在计算机存储器中。
应当理解，本申请的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，N个步骤或方法可以用存储在存储器中且由合适的指令执行***执行的软件或固件来实现。如，如果用硬件来实现和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。
本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中， 该程序在执行时，包括方法实施例的步骤之一或其组合。
此外，在本申请各个实施例中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。
上述提到的存储介质可以是只读存储器，磁盘或光盘等。尽管上面已经示出和描述了本申请的实施例，可以理解的是，上述实施例是示例性的，不能理解为对本申请的限制，本领域的普通技术人员在本申请的范围内可以对上述实施例进行变化、修改、替换和变型。

Claims (12)

  1. 一种局域轨道函数的三维重构方法，其特征在于，包括以下步骤：
    采集多个倾转角度下的样品的图像数据；
    基于所述图像数据，在实空间等间隔进行撒点并利用线性累加，得到所述多个倾转角度的每个倾转角度下的计算图像；以及
    根据每个倾转角度下的计算图像计算损失函数，并获取所述损失函数关于待优化参数的梯度，且根据所述梯度优化所述待优化参数，筛选出满足预设条件的原子，重新计算新的损失函数，直至满足收敛条件，在所述实空间重构局域轨道函数的中心的三维空间坐标和局域轨道函数的形状，得到三维重构结果。
  2. 根据权利要求1所述的方法，其特征在于，采集所述多个倾转角度下的样品的图像数据，包括：
    获取所述多个倾转角度下的样品的初始图像数据；
    对所述初始图像数据进行对中合轴和降噪处理，并由处理后的图像归一化，得到所述图像数据。
  3. 根据权利要求1所述的方法，其特征在于，所述损失函数的计算公式为：
    其中，W代表损失函数，M代表倾转角度的总数，j代表倾转角度的序号，i代表局域轨道函数序号，P代表图像的边长，u代表图像中每个像素的横坐标，v代表图像中每个像素的纵坐标，fj(u,v)代表第j个角度下计算得到的图像，bj(u,v)代表第j个角度下实验所得的图像。
  4. 根据权利要求1所述的方法，其特征在于，所述筛选出满足预设条件的原子，包括：
    在每一步迭代更新完参数后，检测所有局域轨道函数的参数
    在检测到任一局域轨道函数的参数小于由所述所有局域轨道函数参数得到的阈值时，删除所述任一局域轨道函数的同时，通过建立二叉树得到局域轨道函数中心距离小于预设像素的局域轨道函数对索引，删除局域轨道函数对中的任意一个局域轨道函数；
    每一步迭代更新完参数并删除完所述局域轨道函数后，将每个局域轨道函数的参数均以预设概率减小为预设倍数，且设置保护时间，使得在所述保护时间内，不允许执行筛选操作和删除操作。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述待优化参数包括每个局域轨道函数中心的三维空间坐标、描述其形状的参数、每个转角对应的三个欧拉角、每个角度下样品的漂移、样品台的机械倾转偏差中的至少一项。
  6. 一种局域轨道函数的三维重构装置，其特征在于，包括：
    采集模块，用于采集多个倾转角度下的样品的图像数据；
    累加模块，用于基于所述图像数据，在实空间等间隔进行撒点并利用线性累加，得到所述多个倾转角度的每个倾转角度下的计算图像；以及
    重构模块，用于根据每个倾转角度下的计算图像计算损失函数，并获取所述损失函数关于待优化参数的梯度，且根据所述梯度优化所述待优化参数，筛选出满足预设条件的原子，重新计算新的损失函数，直至满足收敛条件，在所述实空间重构局域轨道函数的中心的三维空间坐标和局域轨道函数的形状，得到三维重构结果。
  7. 根据权利要求6所述的装置，其特征在于，所述采集模块包括：
    获取单元，用于获取所述多个倾转角度下的样品的初始图像数据；
    降噪单元，用于对所述初始图像数据进行对中合轴和降噪处理，并由处理后的图像归一化，得到所述图像数据。
  8. 根据权利要求6所述的装置，其特征在于，所述损失函数的计算公式为：
    其中，W代表损失函数，M代表倾转角度的总数，j代表倾转角度的序号，i代表局域轨道函数序号，P代表图像的边长，u代表图像中每个像素的横坐标，v代表图像中每个像素的纵坐标，fj(u,v)代表第j个角度下计算得到的图像，bj(u,v)代表第j个角度下实验所得的图像。
  9. 根据权利要求6所述的装置，其特征在于，所述重构模块包括：
    检测单元，用于在每一步迭代更新完参数后，检测所有局域轨道函数的参数
    删除单元，用于在检测到任一局域轨道函数的参数小于由所述所有局域轨道函数参数得到的阈值时，删除所述任一局域轨道函数的同时，通过建立二叉树得到局域轨道函数中心距离小于预设像素的局域轨道函数对索引，删除局域轨道函数对中的任意一个局域轨道函数；
    保护单元，用于在每一步迭代更新完参数并删除完所述局域轨道函数后，将每个局域轨道函数的参数均以预设概率减小为预设倍数，且设置保护时间，使得在所述保护时间内，不允许执行筛选操作和删除操作。
  10. 根据权利要求6-9任一项所述的装置，其特征在于，所述待优化参数包括每个局域轨道函数中心的三维空间坐标、描述其形状的参数、每个转角对应的三个欧拉角、每个角度下样品的漂移、样品台的机械倾转偏差中的至少一项。
  11. 一种电子设备，其特征在于，包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述程序，以实现如权利要求1-5任 一项所述的局域轨道函数的三维重构方法。
  12. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，该程序被处理器执行，以用于实现如权利要求1-5任一项所述的局域轨道函数的三维重构方法。
PCT/CN2023/080253 2022-04-12 2023-03-08 局域轨道函数的三维重构方法及装置 WO2023197785A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210381325.3A CN114648611B (zh) 2022-04-12 2022-04-12 局域轨道函数的三维重构方法及装置
CN202210381325.3 2022-04-12

Publications (1)

Publication Number Publication Date
WO2023197785A1 true WO2023197785A1 (zh) 2023-10-19

Family

ID=81997728

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/080253 WO2023197785A1 (zh) 2022-04-12 2023-03-08 局域轨道函数的三维重构方法及装置

Country Status (2)

Country Link
CN (1) CN114648611B (zh)
WO (1) WO2023197785A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117635840A (zh) * 2023-12-05 2024-03-01 清华大学 基于扫描衍射图的局域轨道函数三维重构方法及装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114648611B (zh) * 2022-04-12 2023-07-18 清华大学 局域轨道函数的三维重构方法及装置
CN117392316B (zh) * 2023-10-13 2024-06-18 清华大学 基于系列欠焦图像的三维重构方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170178316A1 (en) * 2015-12-18 2017-06-22 Carestream Health, Inc. Accelerated statistical iterative reconstruction
CN111208513A (zh) * 2020-01-15 2020-05-29 西安电子科技大学 空间目标isar图像序列能量反向投影与三维重构方法
CN111862316A (zh) * 2020-07-28 2020-10-30 杭州深瞳科技有限公司 一种基于优化的imu紧耦合稠密直接rgbd的三维重建方法
CN113720865A (zh) * 2021-08-06 2021-11-30 清华大学 自动矫正样品带轴偏离的电子层叠成像方法及装置
CN114648611A (zh) * 2022-04-12 2022-06-21 清华大学 局域轨道函数的三维重构方法及装置

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9613285B2 (en) * 2012-03-22 2017-04-04 The Charles Stark Draper Laboratory, Inc. Compressive sensing with local geometric features
WO2014171988A2 (en) * 2013-01-29 2014-10-23 Andrew Robert Korb Methods for analyzing and compressing multiple images
CA2983880A1 (en) * 2015-05-05 2016-11-10 Kyndi, Inc. Quanton representation for emulating quantum-like computation on classical processors
WO2019246397A1 (en) * 2018-06-21 2019-12-26 The University Of Chicago A fully fourier space spherical convolutional neural network based on clebsch-gordan transforms
CN109508678B (zh) * 2018-11-16 2021-03-30 广州市百果园信息技术有限公司 人脸检测模型的训练方法、人脸关键点的检测方法和装置
US10607164B1 (en) * 2019-07-28 2020-03-31 Zichu Wang Architects space programming process that determines new classroom sizes and count for educational institutions
CN110874864B (zh) * 2019-10-25 2022-01-14 奥比中光科技集团股份有限公司 获取对象三维模型的方法、装置、电子设备及***
CN111179339B (zh) * 2019-12-13 2024-03-08 深圳市瑞立视多媒体科技有限公司 基于三角测量的坐标定位方法、装置、设备及存储介质
CN111507908B (zh) * 2020-03-11 2023-10-20 平安科技（深圳）有限公司 图像矫正处理方法、装置、存储介质及计算机设备
CN112424835B (zh) * 2020-05-18 2023-11-24 上海联影医疗科技股份有限公司 用于图像重建的***和方法
FR3111067A1 (fr) * 2020-06-06 2021-12-10 Olivier Querbes Prise d'empreinte optique de l’arcade dentaire d’un patient
US20210397108A1 (en) * 2020-06-18 2021-12-23 Konica Minolta, Inc. Image forming method and image forming system
CN112489102A (zh) * 2020-11-30 2021-03-12 北京百度网讯科技有限公司 三维重建方法、装置、设备以及存储介质
CN113034681B (zh) * 2021-04-07 2022-05-03 清华大学 空间平面关系约束的三维重建方法及装置
CN114186191A (zh) * 2021-11-19 2022-03-15 合肥联宝信息技术有限公司 坐标转换矩阵的计算方法、装置、设备及可读存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170178316A1 (en) * 2015-12-18 2017-06-22 Carestream Health, Inc. Accelerated statistical iterative reconstruction
CN111208513A (zh) * 2020-01-15 2020-05-29 西安电子科技大学 空间目标isar图像序列能量反向投影与三维重构方法
CN111862316A (zh) * 2020-07-28 2020-10-30 杭州深瞳科技有限公司 一种基于优化的imu紧耦合稠密直接rgbd的三维重建方法
CN113720865A (zh) * 2021-08-06 2021-11-30 清华大学 自动矫正样品带轴偏离的电子层叠成像方法及装置
CN114648611A (zh) * 2022-04-12 2022-06-21 清华大学 局域轨道函数的三维重构方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZHENG XING-RONG, YANG WEI, ZHANG XIN-DAN, DU CHEN-MIN, LI XIAO-LONG, TANG HUAN: "Visualization Research of the Spherical Harmonic Function and Atomic Orbitalbased on MATLAB Software", JOURNAL OF SICHUAN UNIVERSITY (NATURAL SCIENCE EDITION), vol. 57, no. 5, 1 September 2020 (2020-09-01), pages 968 - 974, XP093098621, DOI: 10.3969/j.issn.0490-6756.2020.05.022 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117635840A (zh) * 2023-12-05 2024-03-01 清华大学 基于扫描衍射图的局域轨道函数三维重构方法及装置

Also Published As

Publication number Publication date
CN114648611B (zh) 2023-07-18
CN114648611A (zh) 2022-06-21

Similar Documents

Publication Publication Date Title
WO2023197785A1 (zh) 局域轨道函数的三维重构方法及装置
JP5500242B2 (ja) 複数クラスの目標の検出装置および検出方法
US10614285B2 (en) Computing technologies for image operations
US9377546B2 (en) Automatic extraction and characterization of fault and fracture populations
Tuller et al. Segmentation of X‐ray CT data of porous materials: A review of global and locally adaptive algorithms
GB2463141A (en) Medical image segmentation
CN108154191B (zh) 文档图像的识别方法和***
Lianheng et al. A practical photogrammetric workflow in the field for the construction of a 3D rock joint surface database
US10957075B2 (en) Representation of a component using cross-sectional images
CN110728675A (zh) 肺结节分析装置、模型训练方法、装置及分析设备
CN109411056B (zh) 图像存储方法及***
WO2022134415A1 (zh) 室内机器人定位方法、装置、终端设备及存储介质
CN116415020A (zh) 一种图像检索的方法、装置、电子设备及存储介质
CN106558057B (zh) 一种医学图像分割方法
Tefera et al. 3DNOW: Image-based 3D reconstruction and modeling via WEB
WO2016204241A1 (ja) 画像処理装置、画像処理方法、プログラム及び記録媒体
US20230169637A1 (en) Method for grain size analysis
CN114359670A (zh) 非结构化数据标注方法、装置、计算机设备及存储介质
JP6745268B2 (ja) シンチグラフィー画像の正規化技術
Gaschen et al. MAMA User Guide v2. 0.1
CN112697658A (zh) 存储器、电子显微镜颗粒几何性质测定方法、设备和装置
CN113163101A (zh) 图像曝光调整方法、装置、设备和介质
US11481903B2 (en) Iterative branching structure segmentation method and system
URSU PROCESSING OF LOW-CONTRAST 3D TOMOGRAPHIC IMAGES OF COMPOSITE MATERIALS
WO2021166574A1 (ja) 画像処理装置、画像処理方法、及びコンピュータ読み取り可能な記録媒体

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23787433

Country of ref document: EP

Kind code of ref document: A1