CN118135120A

CN118135120A - Three-dimensional reconstruction and micromanipulation system for surface morphology of nano sample

Info

Publication number: CN118135120A
Application number: CN202410547691.0A
Authority: CN
Inventors: 肖晓晖; 吴志科; 黄�俊; 何政
Original assignee: Wuhan University WHU
Current assignee: Wuhan University WHU
Priority date: 2024-05-06
Filing date: 2024-05-06
Publication date: 2024-06-04
Anticipated expiration: 2044-05-06
Also published as: CN118135120B

Abstract

The invention provides a three-dimensional reconstruction and micromanipulation system for surface morphology of a nano sample, which comprises the steps that a sample fixing table, a temperature control platform and a probe are controlled to move along the axial direction of a microscope objective lens at equal steps, a multi-depth-of-field image sequence of the nano sample, namely a focusing stack, is shot by a microscope camera to serve as data input of three-dimensional reconstruction, depth characteristics of the focusing stack are extracted by using a self-encoder, a attention module is introduced to improve characteristic extraction capacity, a plurality of characteristic images with deep characteristics are output by the encoder through training and updating self-encoder parameters, the plurality of characteristic images output by the encoder are fused into 1 depth characteristic image through focusing evaluation, the processed focusing stack is further obtained to generate a 3D morphology image, and a micromanipulation system is arranged based on the method, so that the sample fixing table, the temperature control platform and the probe are visually controlled to move to realize operations such as welding, transferring, bending, twisting and the like of the nano sample under a vacuum constant-temperature environment, and multiple risks existing in the operation of the nano sample under the atmospheric environment are avoided.

Description

Three-dimensional reconstruction and micromanipulation system for surface morphology of nano sample

Technical Field

The invention relates to the technical field of nano material treatment, in particular to a three-dimensional reconstruction and micromanipulation system for the surface morphology of a nano sample.

Background

The nano material has special physical effects such as small size effect, surface effect, quantum size effect and the like, so that the nano material has mechanical, thermal, electrical and other properties which are quite different from those of common macroscopic materials. With the gradual development of various excellent properties of nano materials, nano materials are widely applied to various fields of chemistry, aerospace, medicine and the like. Therefore, the measurement and characterization of the special properties of the nano material are critical to the detection and design of the practical application of the nano device.

In the field of micromanipulation and assembly, operators can only judge the contact condition of an operating arm and a sample through experience due to limited depth of field of microscopic imaging. The three-dimensional morphology information of the sample surface is recovered and reconstructed in three dimensions, so that more accurate depth information can be provided for operators. The microscopic three-dimensional morphology reconstruction method can be divided into an active optical method and a passive optical method, active three-dimensional imaging equipment comprises a laser confocal microscope, a white light interferometer and the like, the passive three-dimensional reconstruction method comprises a monocular vision method based on a motion restoration structure, a binocular vision method based on a stereoscopic vision restoration morphology and the like, and the typical three-dimensional imaging technology needs long-time scanning or an additional optical-mechanical structure, so that time cost and hardware cost are greatly increased, and quick and efficient three-dimensional imaging is not facilitated.

At present, in the field of micromanipulation, because the depth of field of microscopic imaging is limited, an operator can only judge the contact condition of an operation arm and a sample through experience without additional measuring equipment, and the tip of the operation arm is extremely easy to bend or damage the sample. Meanwhile, most experimental equipment for carrying out lap joint operation on low-dimensional nano materials is provided with 2 operation arms, so that only nano particles and nano wires can be simply lap-jointed, the operation on two-dimensional film materials can not be realized, and the operation of bending, twisting and the like on the materials is not supported. Meanwhile, the sample is operated and transferred in the atmospheric environment, and a plurality of uncertain factors are generated during measurement, so that the in-situ measurement and characterization of the performance of the nano material are difficult to carry out by the current experimental equipment.

Disclosure of Invention

According to the defects of the prior art, the invention aims to provide a three-dimensional reconstruction and micromanipulation system for the surface morphology of a nano sample, compared with the traditional method for acquiring focusing information by using a gray level original image, the method acquires the focal power from depth characteristics, can create high-quality three-dimensional morphology without any preprocessing or post-processing technology, and has better characteristic extraction capability and lower calculation cost and complexity.

In order to solve the technical problems, the invention adopts the following technical scheme:

A three-dimensional reconstruction method of the surface morphology of a nano sample comprises the following steps:

And (3) data acquisition: controlling the sample fixing table, the temperature control platform and the probe to axially move along the microscope objective lens at a constant distance by setting a moving step, shooting a multi-depth-of-field image sequence of the sample by a microscope camera, acquiring a focusing stack, and inputting the focusing stack as three-dimensional reconstruction data of the microscopic morphology;

training and learning: extracting depth features of a focus stack by using a self-encoder, wherein the self-encoder comprises an encoder and a decoder, the encoder and the decoder are provided with a plurality of convolution blocks, no pooling layer is added, the convolution blocks comprise a convolution layer, a batch normalization layer and Relu activation functions, the loss function of the self-encoder is a mean square error, and a attention module is introduced after the convolution blocks of the encoder to improve the feature extraction capability, and the parameters of the self-encoder are updated through training, so that the encoder outputs a plurality of feature images with deep features;

And (3) image fusion: fusing a plurality of characteristic images output by the encoder into 1 depth characteristic image, calculating the focusing degree of the characteristic image through a focusing evaluation operator, and fusing the largest focusing degree value in each pixel into the depth characteristic image;

Three-dimensional reconstruction: and fusing the self-encoder and the focusing stack generated after focusing evaluation into a three-dimensional image, comparing the focal power of each pixel point according to an image sequence, wherein the height of a layer where the maximum focal power is positioned is the three-dimensional reconstruction result of the point, and generating a 3D morphology image after fusing.

Further, the attention module comprises a channel attention module and a spatial attention module;

The channel attention module respectively passes through a maximum pooling layer and an average pooling layer of the characteristic image output by the convolution block, and the output result is processed through a multi-layer perceptron comprising two hidden layers and summed according to elements, normalized by using a sigmoid function, and a channel attention weight function is obtained ：

In the method, in the process of the invention,Representing a sigmoid function, MLP representing a multi-layer perceptron,/>And/>Maximum pooling and average pooling, respectively,/>Is a feature image;

The spatial attention module carries out maximum pooling and average pooling on the characteristic images, and normalizes the characteristic images through a sigmoid function after the layers are fused to obtain a spatial attention weight function ：

。

Further, in the image fusion process, a Gaussian-like four-neighborhood gradient operator GFG is adopted to calculate the image gradient, and the expression is as follows:

In the method, in the process of the invention, Represents the output of the encoder/>, Of a feature imagePixel dot,/>Is made up of pixel points/>For the value of the center pixel convolved with the Gaussian-like convolution template,/>And/>Representing the gradient of the pixel point in the horizontal and vertical directions respectively,/>Representing a gaussian-like matrix template;

multiple characteristic images output by encoder Thereby transforming into by/>Multiple feature matrices of composition/>Comparing multiple feature matrices one by oneGradient values of all pixel points are fused into an image representing depth characteristics of the layer by taking the maximum valueIn/>Represents the/>, of the focus stackSheet image:

。

Further, in the three-dimensional reconstruction process, the focusing stack includes Image for each pixel/>Composition length is/>Is a feature matrix of (a):

the images are shot equidistantly and the serial numbers of the images are used Depth information characterizing the image, considered the feature matrix/>The maximum value in (2) is the focusing point of the pixel point, and is expressed by the value/>/>The value characterizes the height position of the pointFor each pixel/>Performing this operation, the 3D topography is then expressed as:

。

A micromanipulation system for three-dimensional reconstruction of surface topography of a nano-sample, comprising:

the top of the vacuum cavity is provided with an upper cover and an observation window, a microscope camera is arranged above the upper cover and the observation window to obtain a multi-depth-of-field image sequence of the sample, and the microscope camera is used for obtaining the multi-depth-of-field image sequence of the sample;

The displacement platform comprises four-degree-of-freedom displacement platforms and a three-dimensional displacement platform, wherein the four-degree-of-freedom displacement platforms are provided with translational degrees of freedom in the direction of X, Y, Z and rotational degrees of freedom rotating around the Z direction, the two four-degree-of-freedom displacement platforms are arranged in the vacuum cavity and are oppositely arranged at two sides of the upper cover and the observation window, the two four-degree-of-freedom displacement platforms are respectively provided with a sample fixing platform and a temperature control platform, the four-degree-of-freedom displacement platforms are used for controlling the sample fixing platform or the temperature control platform to move according to a set path, a plurality of three-dimensional displacement platforms are arranged on the vacuum cavity, the number of the three-dimensional displacement platforms is set according to requirements, the three-dimensional displacement platforms are provided with probes extending into the vacuum cavity, and the three-dimensional displacement platforms are used for controlling the probes to move according to the set path;

The control module comprises an upper computer and is used for controlling the microscopic camera, the two four-degree-of-freedom displacement platforms and the plurality of three-dimensional displacement platforms through the upper computer;

and the 3D morphology image acquisition module is used for processing the multi-depth image sequence of the sample acquired by the microscope camera by adopting the three-dimensional reconstruction method of the surface morphology of the nano sample to obtain a 3D morphology image.

Further, the three-dimensional displacement platforms all regulate the moving distance through the linear actuator, and the method for acquiring the mapping relation between the position of the probe in the microscopic image and the displacement value of the linear actuator comprises the following steps: the method for calibrating the position of the probe comprises the steps of dragging the probe on an upper computer to control the position, calibrating the transformation relation between a screen pane coordinate system and a displacement table plane rectangular coordinate system, wherein the calibration method is to control the probe to perform displacement with the length of s on an x axis, and the positions of linear actuators for driving the probe to move before and after the displacement are respectively as follows,/>The positions of the probe tips on the screen pane before and after displacement are recorded as/>, respectively,/>Mapping relation/>, of screen pane coordinate system and displacement platform rectangular coordinate systemThe method comprises the following steps:

Converting a screen pane coordinate system and a three-dimensional displacement platform plane rectangular coordinate system:

In the method, in the process of the invention, Representing the displacement distance of the linear actuator in the rectangular coordinate system of the plane of the displacement platform,/>Representing coordinates of the probe tip in a screen pane coordinate system;

Wherein the method comprises the steps of For the deflection angle of the plane rectangular coordinate system of the displacement platform and the screen pane coordinate system, the trigonometric function value is calculated by the following formula:

and obtaining the conversion relation between the screen pane coordinate system and the displacement platform plane rectangular coordinate system.

Further, the probe comprises a solid probe and a liquid probe, wherein the solid probe comprises a thermocouple temperature sensor, a ceramic heating pipe, a needle head clamp and a needle head body which are connected in sequence; the liquid probe includes a linear actuator and a syringe disposed at a distal end of the linear actuator.

Further, the temperature control platform comprises a water cooling head, a temperature control sheet, a thermosensitive temperature sensor and a temperature control platform shell which are arranged from bottom to top, wherein the temperature control sheet is a ceramic heating sheet or a semiconductor refrigerating sheet, the temperature control platform shell is provided with a resistance temperature sensor RTD, the RTD comprises two cantilever beams which are at a certain distance and are plated with platinum, the cantilever beams are used as heat transfer parts of the sensor, the two cantilever beams are connected to the surface of a silicon substrate plated with platinum, and a platinum plating area on the surface of the silicon substrate is used as a heat sink of the sensor.

Further, a measuring circuit is arranged to calibrate the RTD, and the RTD controller outputs an analog voltageThrough standard resistance/>And RTD resistance/>Serial voltage division to obtain standard resistor/>And RTD resistance/>The voltages of (2) are/>, respectivelyAnd/>Regulating the size of analog voltage, collecting and drawing RTD resistance/>And voltage/>Obtaining RTD resistance/>Voltage with value fluctuating in a small range/>A section within which the test voltage/>, is selectedTemperature change is controlled by using a temperature control platform, and test voltage/>, is obtained and drawnRTD resistance/>And temperature T, the slope of which is RTD temperature coefficient/>And then according to the temperature coefficient/>Obtaining temperature rise/>According to the temperature rise/>Obtaining thermal conductivity of RTD/>；

According to standard resistanceAnd RTD resistance/>Voltage/>And/>Obtain RTD resistance/>Numerical value:

Temperature rise The calculation formula of (2) is as follows:

Wherein, Is RTD resistance/>Resistance value variation amount of (a);

Thermal conductivity of RTD The calculation formula is as follows:

Wherein, Representing the length of the RTD cantilever,/>Indicating the cross-sectional area of the RTD cantilever.

Further, the four-degree-of-freedom displacement platform and the three-dimensional displacement platform are respectively used for adjusting the moving distance through a linear actuator;

the four-degree-of-freedom displacement platform is fixed with a sample fixing table, the top end of the sample fixing table is provided with low-melting-point metal, one end of the sample is welded on the sample fixing table when the nanowire and other samples are measured, the four-degree-of-freedom displacement platform connected with the sample fixing table is provided with one rotational degree of freedom and three translational degrees of freedom, and the bending and twisting of the nano sample are accurately controlled by controlling the displacement and rotation of the sample fixing table.

Compared with the prior art, the invention has the following advantages and beneficial effects:

1. Compared with the traditional method for acquiring focusing information by using a gray level original image, the method acquires the focusing power from the depth feature, can create a high-quality three-dimensional morphology without any preprocessing or post-processing technology, and has better feature extraction capability, lower calculation cost and complexity, higher reconstruction efficiency and lower hardware cost. By rapidly scanning and reconstructing the three-dimensional morphology of the microscopic sample, accurate depth information can be provided for operators, and bending of the probe tip and sample damage are effectively prevented.

2. The existing experimental equipment is only provided with 2 operation arms for operating the sample, so that simple transfer of nano particles and nano wires can be realized, and more complicated operation can not be performed. The invention is provided with a plurality of three-dimensional displacement platforms for operating the sample, the three-dimensional displacement platforms can perform independent movement or cooperative movement, the number of the three-dimensional displacement platforms can be expanded, and the requirements of overlapping and in-situ measurement characterization of the nano material sample are completely met.

3. The invention can realize full-automatic control, and a user can finish the operations of sample transfer, lap joint, welding, bending, torsion, stretching and the like by only operating in a man-machine interface carried by an upper computer, so that the manual operation difficulty is greatly reduced, and the problems of time and labor waste for manually carrying out in-situ measurement and characterization of the nano material properties are solved.

4. According to the invention, the linear actuator is used for replacing the manual spiral knob of the three-dimensional displacement platform, so that the difficulty of manual operation is reduced, the problem of return error of the spiral knob is avoided, and the sample operation is more stable.

5. The invention can provide vacuum and constant temperature environment for measuring and characterizing the performance of the nano material, and provides a sample contact point with different environment temperature, thereby facilitating thermodynamic measurement.

6. The invention can heat the probe and control the temperature of the probe, and can accurately finish the sample heating and welding operation; the liquid probe clamp can be used for dripping and transferring experimental liquid samples, accurately controlling the adding amount of the samples, and dripping silver colloid to assist welding operation.

Drawings

The accompanying drawings are included to provide a further understanding of the application, and are incorporated in and constitute a part of this specification. The exemplary embodiments of the present application and the descriptions thereof are for explaining the present application and do not constitute an undue limitation of the present application. In the drawings:

FIG. 1 is a flow chart of three-dimensional reconstruction of microscopic features according to the present invention.

FIG. 2 is a schematic diagram of the overall structure of the micro-manipulation system according to the present invention.

FIG. 3 is a schematic diagram of the internal structure of the micro-operation system according to the present invention.

FIG. 4 is a schematic view of a probe holder attachment of the present invention attached to a probe.

FIG. 5 is a schematic diagram showing the overall structure of the solid probe of the present invention.

FIG. 6 is a schematic diagram showing the internal structure of the solid probe according to the present invention.

FIG. 7 is a schematic diagram showing the overall structure of the liquid probe of the present invention.

FIG. 8 is a schematic view showing the internal structure of the liquid probe of the present invention.

Fig. 9 is a schematic diagram of the overall structure of the temperature control platform according to the present invention.

Fig. 10 is a schematic view of an internal structure of the temperature control platform according to the present invention.

FIG. 11 is a schematic diagram of a control module according to the present invention.

FIG. 12 is a schematic diagram of an RTD calibration measurement circuit according to the present invention.

Wherein, 1, a linear actuator; 2. a three-dimensional displacement platform; 3. a probe clamp connection; 4. KF flange assembly; 5. a bellows; 6. a vacuum chamber; 7. an upper cover and an observation window; 8. a probe; 81. a thermocouple temperature sensor; 82. a ceramic heating tube; 83. a needle holder; 84. a needle body; 9. a temperature control platform; 91. a temperature controlled platform housing, 92, RTD; 921. si ₃N₄/Pt cantilever; 922. a silicon substrate; 93. a temperature control element; 94. a red copper water-cooled head; 10. a temperature control platform connector; 11. a sample fixing table; 12. a syringe; 13. an upper computer; 14. a microscopic camera; 15. a linear actuator controller; 16. an RTD controller; 17. a microcomputer temperature control module; 18. a thermistor temperature sensor.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first", "a second", etc. may explicitly or implicitly include one or more such feature. In the description of the present invention, unless otherwise indicated, the meaning of "a plurality" is two or more.

Example 1

Embodiment 1 provides a three-dimensional reconstruction method of a surface morphology of a nano sample, as shown in fig. 1, comprising:

Step S1, data acquisition: the sample fixing table 11, the temperature control platform 9 and the probe 8 are controlled to move along the axial direction of the microscope objective lens at a constant distance by setting a moving step, a multi-depth-of-field image sequence of the sample is shot by the microscope camera 14, a focusing stack is obtained, and the focusing stack is used as data input for three-dimensional reconstruction of microscopic morphology;

Step S2, training and learning: the depth characteristics of a focusing stack are extracted by using a self-encoder, the self-encoder comprises an encoder and a decoder, the microscopic camera 14 is a fixed focus camera and the working distance of a microscopic object lens is fixed, so that the space distance and pixels on a focusing plane have a fixed mapping relation, the space position relation between a nano sample and a probe 8 can be well confirmed by using pictures with the same size as an original picture to perform three-dimensional modeling, the encoder and the decoder are provided with three convolution blocks, the convolution blocks comprise convolution layers, batch normalization layers and Relu activation functions, no pooling layer is added for ensuring that the size of an image is not changed, the loss function is mean square error, and the encoder is led to an attention module after the convolution blocks so as to improve the characteristic extraction capability, and the parameters of the self-encoder are updated through training, so that the deep characteristics output by the encoder are more definite, and further the encoder outputs a plurality of characteristic images with deep characteristics;

Step S3, image fusion: the 32 characteristic images output by the encoder are fused into 1 depth characteristic image, the focusing degree of the characteristic image is calculated through a focusing evaluation operator, and the value with the largest focusing degree in each pixel is fused into the depth characteristic image;

Step S4, three-dimensional reconstruction: and fusing the focusing stack generated after the self-encoder and the focusing evaluation into a three-dimensional image, comparing the focal power of each pixel point according to an image sequence, wherein the height of a layer where the maximum focal power is positioned is the three-dimensional reconstruction result of the point, and generating a 3D morphology image for processing the nano sample after the fusion.

The three-dimensional reconstruction method of the surface morphology of the nano sample is a focusing recovery morphology method based on unsupervised depth learning, a multi-depth image sequence of a sample is shot by a microscope camera 14 through controlling the movement of an objective table, a focusing stack is obtained and used as data input for three-dimensional reconstruction of the micro morphology, further, a 3D morphology image is obtained through training learning, image fusion and three-dimensional reconstruction, an operator can know the state of the nano sample in an omnibearing manner when processing the nano sample, the operator can accurately judge the contact condition of the probe and the sample, and compared with the traditional method for obtaining focusing information by using a gray level original image, the method can be used for obtaining focal power from depth characteristics, and can create high-quality three-dimensional morphology without any preprocessing or post-processing technology, so that the method has better feature extraction capability and lower calculation cost and complexity.

In embodiment 1, the attention module includes a channel attention module and a spatial attention module;

the channel attention module respectively passes through a maximum pooling layer and an average pooling layer of the characteristic image output by the convolution module, the output result passes through a multi-layer perceptron (MLP) comprising two hidden layers and is summed according to elements, and then a sigmoid function is used for normalization to obtain a channel attention weight function ：

The spatial attention module is used for carrying out maximum pooling and average pooling on the characteristic images, merging the image layers, and normalizing the image layers through a sigmoid function to obtain a spatial attention weight function ：

In the method, in the process of the invention,Representing a sigmoid function,/>And/>Maximum pooling and average pooling, respectively.

In embodiment 1, during image fusion, the gray value of the region where the image is in focus changes more than the out-of-focus region, and a focus evaluation operator is used to calculate the focus level of each pixel of the image. The applicable focusing evaluation operator can be selected according to different scenes, and the image gradient is calculated by adopting a Gaussian-like four-neighborhood gradient operator GFG, wherein the expression is as follows:

32 characteristic images output by encoder Thereby transforming into by/>Composed of 32 eigenvectors/>One by one, 32 feature matrices/>Gradient values of all pixel points are fused into an image representing depth characteristics of the layer by taking the maximum valueIn/>Represents the/>, of the focus stackSheet image:

。

in embodiment 1, the focus stack includes together during the three-dimensional reconstruction process Image for each pixel/>Can be formed into a length of/>Is a feature matrix of (a):

。

the images are shot equidistantly and the serial numbers of the images are used Depth information characterizing the image, considered the feature matrix/>The maximum value in (2) is the focusing point of the pixel point, and is expressed by the value/>/>The value characterizes the height position of the pointFor each pixel/>Performing this operation, the 3D topography map can be expressed as:

。

Example 2

Embodiment 2 provides a micromanipulation system for three-dimensional reconstruction of surface topography of a nano-sample, as shown in fig. 2-10, comprising:

The top of the vacuum cavity 6 is provided with an upper cover and an observation window 7, a microscope camera 14 is arranged above the upper cover and the observation window 7, and the microscope camera is used for 14 obtaining a multi-depth-of-field image sequence of a sample;

the displacement platform comprises four-degree-of-freedom displacement platforms and three-dimensional displacement platforms 2, the four-degree-of-freedom displacement platforms are provided with translational degrees of freedom in the direction of X, Y, Z and rotational degrees of freedom rotating around the Z direction, the two four-degree-of-freedom displacement platforms are arranged in the vacuum cavity 6 and are oppositely arranged at two sides of the upper cover and the observation window 7, the two four-degree-of-freedom displacement platforms are respectively provided with a sample fixing platform 11 and a temperature control platform 9, the four-degree-of-freedom displacement platforms are used for controlling the sample fixing platform 11 or the temperature control platform 9 to move according to a set path, the three-dimensional displacement platforms 2 are arranged on the vacuum cavity 6, the number of the three-dimensional displacement platforms 2 can be set according to requirements, probes 8 extending into the vacuum cavity 6 are arranged on the three-dimensional displacement platforms 2, and the three-dimensional displacement platforms 2 are used for controlling the probes 8 to move according to the set path;

the control module comprises an upper computer 13 and is used for controlling a microscopic camera 14, two four-degree-of-freedom displacement platforms and a plurality of three-dimensional displacement platforms 2 through the upper computer 13;

The 3D morphology image obtaining module is configured to process the multi-depth image sequence of the sample obtained by the microscope camera 14 by adopting the three-dimensional reconstruction method of the surface morphology of the nano sample, so as to obtain a 3D morphology image.

The micro-operation system for three-dimensional reconstruction of the surface morphology of the nano-sample provided in the embodiment 2 is provided with two first four-degree-of-freedom displacements and a plurality of three-dimensional displacement platforms 2 for operating the nano-materials, the displacement platforms can perform independent movement or cooperative movement, the number of the three-dimensional displacement platforms 2 can be expanded, the requirements of lap joint and in-situ measurement characterization of the nano-material samples are completely met, in addition, a vacuum and temperature-controllable measurement environment is provided, a plurality of risks existing in the operation and the transfer of the samples in the atmospheric environment are avoided, and the operations of welding, transferring, bending, twisting and the like of the nano-materials can be visually controlled by combining the upper cover, the observation window 7 and the micro-camera 14 with the control module.

In a specific embodiment of this embodiment, as shown in fig. 1 and fig. 2, the number of the three-dimensional displacement platforms 2 is 4, the number can be expanded according to the needs, every two three-dimensional displacement platforms 2 are in a group, two groups of three-dimensional displacement platforms 2 are respectively arranged at two ends of the vacuum cavity 6, and the nano materials can be operated omnidirectionally through the 4 three-dimensional displacement platforms 2, so as to realize the operations of welding, transferring, bending, twisting and the like of the nano materials.

As shown in fig. 11, the control module includes an upper computer 13, and in order to realize visual control of the probe 8, the upper computer 13 is required to be connected with a microscope camera 14 to acquire a real-time display microscopic image. In controlling the position of the probe 8, the position of the probe 8 is more convenient and accurate by using a mouse than the joystick or button control, but this requires the display image of the micro camera 14 to be incorporated into the control program of the host computer 13. Different from the development platform supported by the different types of microscopic industrial cameras, the development platform supported by the linear actuator 1 is not necessarily compatible, and there is a possibility of initializing camera display parameters by reading camera data streams. Therefore, the API function of Windows can be used to control the micro camera 14 software interface to achieve communication interaction with the micro operating system user interface. When the upper computer 13 is used for controlling a program, the interface of the micro camera 14 software is arranged above the control program, and a layered and transparent expansion window style is added for the micro camera 14 software interface, so that a mouse can pass through the window to operate the control program, and the micro camera 14 software interface can be normally displayed. If necessary, the window of the micro camera 14 software interface is initialized, and the shooting, storing and measuring functions are restored. The method can be adapted to most of the microscope cameras 14 without reprogramming, avoids the conflict between the microscope cameras 14 and the development platform of the linear actuator 1, and retains the original functions of the microscope cameras 14.

The upper computer 13 controls the software interface of the microscope camera 14 by using the API function of Windows, and the dynamic link library User32.DLL is a system DLL file, which is the main support of the Windows graphical user interface.

The method comprises the following steps:

retrieving a window handle of the microscope camera 14 software interface using FindWindowA functions;

The size, the position and the Z sequence of a software interface of the microscope camera 14 are changed by using a SetWindow Pos function, so that the software interface is placed at the top end of a control program of the upper computer 13;

Using SetWindowLongA function to change window attribute of microscope camera 14 software interface, adding extended window patterns WS_EX_ LAYERED and WS_EX_ TRANSPARENT, WS _EX_ LAYERED attribute to window to make window support TRANSPARENT, WS_EX_TRANSPARENT to make mouse penetrate the window, and make mouse penetrate the microscope camera 14 software interface to control upper computer 13 control program.

In embodiment 2, the four-degree-of-freedom displacement platform and the three-dimensional displacement platform 2 regulate the movement distance through the linear actuator 1, and the linear motion output by the linear actuator 1 is transmitted to the four-degree-of-freedom displacement platform or the three-dimensional displacement platform 2, so that nano-scale operation is realized. Compared with a manual screw joystick, the linear actuator 1 is used for controlling more conveniently and accurately, and the return error is reduced.

The sample fixing table 11 is fixed on the four-degree-of-freedom displacement platform, the low-melting-point metal is arranged at the top end of the fixing table, one end of the sample can be welded on the sample fixing table 11 when the sample such as the nanowire is measured, the four-degree-of-freedom displacement platform connected with the sample fixing table 11 has one rotational degree of freedom and three translational degrees of freedom, and the bending and twisting of the nano sample can be accurately controlled by controlling the displacement and rotation of the sample fixing table 11.

The control module is connected with a linear actuator controller 15 through a host computer 13 and controls the linear actuator 1.

When the system is used for the first time, the communication ports of the linear actuator 1 are required to be set in sequence, and the screen pane coordinate system and the displacement platform plane rectangular coordinate system are converted.

To ensure that the probe 8 can accurately follow the displacement of the mouse, it is necessary to obtain a mapping relationship between the position of the probe 8 in the microscopic image and the displacement value of the linear actuator 1. The position of the tip of the probe 8 in the microscopic image can be represented using a screen pane coordinate system; the actual position of the probe 8 is represented by using a rectangular planar coordinate system of the three-dimensional displacement platform 2, and each axis coordinate of the coordinate system is a displacement value of the corresponding linear actuator 1. To determine this mapping relationship, it is necessary to input a displacement command to the linear actuator controller 15 so that the tip of the probe 8 appears in the microscopic image taken by the microscopic camera 14, and to control the probe 8 to perform a linear displacement.

In example 2, the method for obtaining the mapping relation between the position of the probe in the microscopic image and the displacement value of the linear actuator is as follows: the position control is carried out by dragging the probe 8 through a mouse on the upper computer 13, the transformation relation between a screen pane coordinate system and a displacement table plane rectangular coordinate system is required to be calibrated, the calibration method is that the probe 8 is controlled to carry out displacement with the length of s on the x axis, and the positions of the linear actuators 1 driving the probe 8 to move before and after the displacement are respectively as follows,/>The position of the tip of the probe 8 on the screen pane before and after displacement is noted as/>, respectively,/>Mapping relation/>, of screen pane coordinate system and displacement platform rectangular coordinate systemThe method comprises the following steps:

。

And converting a screen pane coordinate system and a plane rectangular coordinate system of the three-dimensional displacement platform 2:

In the method, in the process of the invention, Represents the displacement distance of the linear actuator 1 in the rectangular coordinate system of the plane of the displacement platform,/>Representing the coordinates of the tip of the probe 8 in the screen pane coordinate system.

Wherein,The three-angle function value of the deflection angle of the plane rectangular coordinate system of the displacement platform and the screen pane coordinate system is calculated by the following formula:

and then the conversion relation between the screen pane coordinate system and the displacement platform plane rectangular coordinate system can be obtained.

After determining the mapping relationship of the two coordinate systems, the mouse can be used to control the precise movement of the probe 8. It should be noted that a will be different under different magnification factors, and it needs to be calibrated separately; the command input to the linear actuator 1 by the host computer 13 should be an absolute position, and if a relative displacement command is input, accumulation of displacement errors is caused, which affects the accuracy of control. Because the upper computer 13 can independently control each linear actuator 1, the three-dimensional displacement platforms 2 of the probes 8 can independently or cooperatively move, and the system allows any number of probes 8 to be installed at any position and drives the three-dimensional displacement platforms 2 of the probes 8 to move, so that the operation requirements of lapping, transferring and the like of nano samples can be completely met.

In example 2, as shown in fig. 4 to 8, the probe 8 comprises a solid probe and a liquid probe, the probe 8 is fixed on the probe holder connecting piece 3, the corrugated tube 5 is arranged in the middle of the probe holder connecting piece 3, and the corrugated tube 5 is fixed on the probe holder fixing piece 3 through the KF flange assembly 4.

As shown in fig. 4 to 6, the solid probe includes a thermocouple temperature sensor 81, a ceramic heating pipe 82, a needle holder 83 and a needle body 84 which are sequentially connected, and can heat the probe and control the temperature of the probe, thereby precisely completing the sample heating and welding operations.

As shown in fig. 7 and 8, the liquid probe comprises a linear actuator 1 and a syringe 12 arranged at the tail end of the linear actuator 1, so that the dripping and transferring of a liquid sample can be tested, the adding amount of the sample can be accurately controlled, and the silver colloid can be dripped to assist the welding operation.

Specifically, the front end of the needle head clamp 83 has the same structure with a common drill head clamp, the tip of the probe is easy to wear in the experimental process, the structure is convenient for replacing the needle head body 84, the rear end of the needle head clamp 83 is embedded with the ceramic heating pipe 82 and the thermocouple temperature sensor 81, the ceramic heating pipe 82 and the thermocouple temperature sensor 81 are connected with the microcomputer temperature control module 17, heating and temperature control of the probe can be realized, and sample welding is convenient to carry out.

If the probe is bent, defective, attached with contaminants, etc. during the experiment, it is necessary to manually replace the probe, specifically the needle body 84 or the syringe 12, and to install the needle holder 83 as needed. The ceramic heating pipe 82 in the probe is connected with the microcomputer temperature control module 17 to heat the needle body 84 after being electrified, the temperature of the needle body 84 is fed back in real time through the thermocouple temperature sensor 81, the output voltage of the microcomputer temperature control module 17 is regulated, and the temperature of the needle body 84 can be controlled. The heated needle body 84 may melt the low melting point metal and weld the sample.

In the embodiment, as shown in fig. 9-10, the temperature control platform 9 includes a temperature control platform shell 91, a red copper water cooling head 94, a heat-sensitive temperature sensor and a temperature control element 93, the red copper water cooling head 94, the temperature control element 93 and the heat-sensitive temperature sensor probe are clamped in the temperature control platform shell 91 from bottom to top, the temperature control element 93 is a ceramic heating plate or a semiconductor refrigerating plate, the red copper water cooling head 94 is arranged, the semiconductor refrigerating plate radiating surface is prevented from being burnt due to overhigh temperature through water circulation heat dissipation, and the temperature control platform shell 91, the temperature control element 93 and the red copper water cooling head 94 are assembled into a whole through interference fit. Lifting lugs on two sides of the temperature control platform shell 91 are used as temperature control platform connecting pieces 10, and are connected with the four-degree-of-freedom displacement platform through the temperature control platform connecting pieces 10, so that a sample is controlled to perform displacement focusing conveniently. In order to facilitate heat conduction, heat-conducting silicone grease needs to be coated on the contact surfaces of the temperature control element 93 and the inner surface of the temperature control platform shell 91 and the contact surfaces of the temperature control element 93 and the upper surface of the red copper water cooling head 94. The microcomputer temperature control module 17 is connected with the temperature control element 93 for controlling the heating and heat dissipation of the temperature control platform 9. The probe of the thermistor temperature sensor 18 is installed on the contact surface between the temperature control element 93 and the inner surface of the temperature control platform shell 91, and is used for feeding back the real-time temperature of the temperature control platform 9 and adjusting the voltage output of the microcomputer temperature control module 17.

In the present invention, as shown in fig. 9 to 10, an RTD92 is disposed on a temperature control element 93, where the RTD92 includes two Si ₃N₄/Pt cantilevers 921 with a spacing of 30 μm, a self width of 3 μm, a total thickness of 0.3 μm, and a platinized thickness of 0.1 μm, and two S ₃N₄/Pt cantilevers 921 are connected to the surface of a platinized silicon substrate 922.

Specifically, the RTD92 is installed on the upper surface of the temperature control platform shell 91, and the connection between the temperature control platform shell 91 and the RTD92 is fixed by conductive silver paste. The main part of the RTD92 is two S ₃N₄/Pt cantilever beams 921, and the shell of the S ₃N₄/Pt cantilever beams is platinized to serve as a heat transfer part; the plated area of platinum at the junction with RTD92 serves to dissipate heat and serves as a heat sink for the sensor. The heat sinks around RTD92 are fixedly connected to copper wires by thermally conductive silver paste and then to RTD controller 16. RTD controller 16 includes a PCB, a chassis, a voltage output module, a voltage input module, and is capable of analog output and acquisition of voltage.

When the system is first used, the RTD92 needs to be calibrated. When the platinized layer of RTD92 is energized, its resistance increases due to joule heat, and the resistance increases with increasing temperature, thus being useful as a temperature sensor. Before using the RTD92, it is necessary to obtain the relationship between the resistance and the temperature of the RTD92, i.e. to calibrate the R-T curve with a slope of the temperature coefficient of resistance。

The temperature of RTD92 is affected by two heat sources: the temperature control system controls the heating element to generate heat, and the heating element conducts the heat to the RTD92 to control the overall temperature of the heat transfer part of the RTD92 and the heat sink; when the RTD92 is energized, the joule heat generated by the excessive input voltage is not negligible, and the resistance of the RTD92 is significantly changed, while the excessively small input voltage generates a small resistance change, increasing the measurement error. Therefore, before calibrating the R-T curve, a suitable test voltage needs to be found to enable the RTD92 to achieve a good measurement effect at the test voltage, i.e., before calibrating the R-T curve, the R-U curve needs to be calibrated.

In the control module, as shown in fig. 11, the microcomputer temperature control module 17, the linear actuation controller 15 and the micro camera 14 are connected through the upper computer 13, so as to control the input voltages of the ceramic heating pipe 82, the ceramic heating plate and the semiconductor refrigerating plate and receive the data of the thermocouple temperature sensor 81 and the thermistor temperature sensor 18; the upper computer 13 is connected with the RTD controller 16 to control the input voltage of the RTD92 circuit and measure the voltage across the RTD 92.

As shown in FIG. 12, the measurement circuit is configured to calibrate RTD92 and output an analog voltage through RTD controller 16Through standard resistance/>And RTD92 resistance/>Serial voltage division to obtain standard resistor/>And RTD92 resistance/>The voltages of (a) are respectivelyAnd/>The analog voltage is regulated, and the RTD92 resistance/>' is acquired and drawnAnd voltage/>Obtaining RTD92 resistance/>Voltage with value fluctuating in a small range/>A section within which the test voltage/>, is selectedTemperature change is controlled by using a temperature control platform 9, and test voltage/>, is obtained and drawnRTD92 resistance/>And temperature T, the slope of which is RTD92 temperature coefficient/>And then according to the temperature coefficient/>Obtaining temperature rise/>According to the temperature rise/>Obtaining thermal conductivity of RTD92。

In particular, according to standard resistanceAnd RTD92 resistance/>Voltage/>And/>Obtain RTD92 resistance/>Numerical value:

。

Temperature rise The calculation formula of (2) is as follows:

。

Thermal conductivity of RTD92 The calculation formula is as follows:

Wherein, Representing the length of the RTD92 cantilever,/>Representing the cross-sectional area of the cantilever of RTD 92.

In the invention, the sample fixing table 11 is fixed on a four-degree-of-freedom displacement platform, the top end of the sample fixing table 11 is provided with low-melting-point metal, and one end of the sample can be welded on the sample fixing table 11 when the sample such as a nanowire is measured. The four-degree-of-freedom displacement platform connected with the sample fixing table 11 has one rotation degree of freedom and three translation degrees of freedom, and the bending and twisting of the nano sample can be accurately controlled by controlling the displacement and rotation of the sample fixing table 11.

Among them, low-melting point metals often refer to metals, alloys and metal derivatives thereof having a melting point lower than 300 ℃, and thus have different melting points due to the polycrystalline phase characteristics of alloy constituent elements and the difference in composition.

The control module of the system comprises a host computer 13 for displaying, processing data and issuing instructions, wherein the host computer 13 is used for controlling a microscopic camera 14, a linear actuator controller 15, an RTD controller 16 and a microcomputer temperature control module 17.

Specifically, the microscopic camera 14 is used for shooting microscopic images, and has functions of adjusting display parameters, measuring and the like; a linear actuator controller 15 for acquiring and modifying various parameters such as the position, speed, etc. of the linear actuator 1; the RTD controller 16 includes a PCB, a chassis, a voltage output module, and a voltage input module, and is configured to output and collect a voltage to the RTD92, and to prevent the RTD92 from being damaged by an excessive voltage, the voltage on two sides of the RTD92 needs to be controlled at mV level; the microcomputer temperature control module 17 is used for receiving the temperature fed back by the thermistor temperature sensor 18 and supplying power to the temperature control element 93 through a 24V direct current stabilized voltage supply.

In the experiment, the vacuum cavity 6 is connected with the vacuum pump to provide a vacuum measuring environment, so that a plurality of risks existing in transferring and operating samples in the atmospheric environment are avoided. The microcomputer temperature control module 17 is used for electrifying the ceramic heating plate or the semiconductor refrigerating plate, and the environmental temperature of the lofting platform and the sample is controlled according to the temperature regulation voltage fed back by the thermistor temperature sensor 18. Controlling RTD92 to heat by RTD controller 16, calculating the resistance wire temperature by the resistance of RTD92 may provide a point of contact that is different from ambient temperature. The RTD92 has two Si ₃N₄/Pt cantilevers 921, and a sample is transferred and soldered to the two cantilevers using a probe, and only one end of the SiN/Pt cantilever 921 is heated, the temperature of which is calculated from the resistance value of the RTD 92; the other end is not heated and its temperature is equal to the ambient temperature. The method can be matched with a Raman spectrometer to measure parameters such as the heat conductivity of a sample.

One end of the nanomaterial is welded on the temperature control platform 9, and the other end is welded on the sample fixing platform 11. The displacement platform connected with the sample fixing platform 11 is additionally provided with a rotary platform on the basis of the three-dimensional displacement platform 2, so that the sample can be controlled to realize bending, torsion, stretching and other operations, and parameter measurement under different states is realized.

In summary, the invention has the following advantages:

1. Compared with the traditional method for acquiring focus information by using a gray level original image, the method provided by the invention acquires the focus power from the depth feature, can create a high-quality three-dimensional morphology without any preprocessing or post-processing technology, and has the advantages of better feature extraction capability, lower calculation cost and complexity, higher reconstruction efficiency and lower hardware cost. By rapidly scanning and reconstructing the three-dimensional morphology of the microscopic sample, accurate depth information can be provided for operators, and bending of the probe tip and sample damage are effectively prevented.

2. The existing experimental equipment is only provided with 2 operation arms for operating the sample, so that simple transfer of nano particles and nano wires can be realized, and more complicated operation can not be performed. The system is provided with a plurality of three-dimensional displacement platforms 2 for operating samples, the three-dimensional displacement platforms 2 can perform independent movement or cooperative movement, the number of the three-dimensional displacement platforms 2 can be expanded, and the requirements of overlapping and in-situ measurement characterization of nano material samples are completely met.

3. The full-automatic control can be realized, and a user can finish the operations of sample transfer, lap joint, welding, bending, torsion, stretching and the like by only operating in a man-machine interface carried by the upper computer 13, so that the manual operation difficulty is greatly reduced, and the problems of time and labor waste for manually carrying out in-situ measurement and characterization of the nano material properties are solved.

4. The linear actuator 1 is used for replacing a manual spiral knob of the three-dimensional displacement platform 2, so that the difficulty of manual operation is reduced, the problem of return error of the spiral knob is avoided, and the sample operation is more stable.

5. The system can provide vacuum and constant temperature environment for measuring and characterizing the performance of the nano material, and provides a sample contact point with different environment temperature, so that thermodynamic measurement is facilitated.

6. The probe can be heated and the temperature of the probe can be controlled, so that the sample heating and welding operations can be accurately completed; the liquid probe clamp can be used for dripping and transferring experimental liquid samples, accurately controlling the adding amount of the samples, and dripping silver colloid to assist welding operation.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

1. The three-dimensional reconstruction method of the surface morphology of the nano sample is characterized by comprising the following steps of:

2. The method for three-dimensional reconstruction of the surface morphology of a nano-sample according to claim 1, wherein the method comprises the following steps:

The attention module comprises a channel attention module and a space attention module;

。

3. The method for three-dimensional reconstruction of the surface morphology of a nano-sample according to claim 1, wherein the method comprises the following steps:

In the image fusion process, a Gaussian-like four-neighborhood gradient operator GFG is adopted to calculate the image gradient, and the expression is as follows:

。

4. the method for three-dimensional reconstruction of the surface morphology of a nano-sample according to claim 1, wherein the method comprises the following steps:

in the three-dimensional reconstruction process, the focusing stack contains Image for each pixel/>Composition length is/>Is a feature matrix of (a):

the images are shot equidistantly and the serial numbers of the images are used Depth information characterizing the image, considered the feature matrix/>The maximum value in (2) is the focusing point of the pixel point, and is expressed by the value/>/>The value characterizes the height position/>, where the point is locatedFor each pixel/>Performing this operation, then 3D topography/>Expressed as:

。

5. a micromanipulation system for three-dimensional reconstruction of surface topography of a nano-sample, comprising:

the top of the vacuum cavity is provided with an upper cover and an observation window, and a microscope is arranged above the upper cover and the observation window and is used for acquiring a multi-depth-of-field image sequence of a sample;

The displacement platform comprises four-degree-of-freedom displacement platforms and a three-dimensional displacement platform, wherein the four-degree-of-freedom displacement platforms are provided with translational degrees of freedom in the direction of X, Y, Z and rotational degrees of freedom rotating around the Z direction, the two four-degree-of-freedom displacement platforms are arranged in the vacuum cavity and are oppositely arranged at two sides of the upper cover and the observation window, the two four-degree-of-freedom displacement platforms are respectively provided with a sample fixing platform and a temperature control platform, the four-degree-of-freedom displacement platforms are used for controlling the sample fixing platform or the temperature control platform to move according to a set path, a plurality of three-dimensional displacement platforms are arranged on the vacuum cavity, probes extending into the vacuum cavity are arranged on the three-dimensional displacement platforms, and the three-dimensional displacement platforms are used for controlling the probes to move according to the set path;

The 3D morphology image acquisition module is configured to process a multi-depth image sequence of a sample acquired by a microscope camera by adopting the three-dimensional reconstruction method of the surface morphology of the nano sample according to any one of claims 1 to 4, so as to obtain a 3D morphology image.

6. The micromanipulation system for three-dimensional reconstruction of surface topography of a nano-sample of claim 5, wherein:

The three-dimensional displacement platforms all regulate the moving distance through the linear actuator, and the method for acquiring the mapping relation between the position of the probe in the microscopic image and the displacement value of the linear actuator comprises the following steps: the method for calibrating the position of the probe comprises the steps of dragging the probe on an upper computer to control the position, calibrating the transformation relation between a screen pane coordinate system and a displacement table plane rectangular coordinate system, wherein the calibration method is to control the probe to perform displacement with the length of s on an x axis, and the positions of linear actuators for driving the probe to move before and after the displacement are respectively as follows ,/>The positions of the probe tips on the screen pane before and after displacement are recorded as/>, respectively,/>Mapping relation/>, of screen pane coordinate system and displacement platform rectangular coordinate systemThe method comprises the following steps:

7. The nanomaterial micromanipulation system suitable for use in multiple environments of claim 5, wherein:

the probe comprises a solid probe and a liquid probe, wherein the solid probe comprises a thermocouple temperature sensor, a ceramic heating pipe, a needle head clamp and a needle head body which are connected in sequence; the liquid probe includes a linear actuator and a syringe disposed at a distal end of the linear actuator.

8. The nanomaterial micromanipulation system suitable for use in multiple environments of claim 5, wherein:

The temperature control platform comprises a water cooling head, a temperature control sheet, a thermosensitive temperature sensor and a temperature control platform shell which are arranged from bottom to top, wherein the temperature control sheet is a ceramic heating sheet or a semiconductor refrigerating sheet, the temperature control platform shell is provided with a resistance temperature sensor RTD, the RTD comprises two cantilever beams which are at a certain distance and are plated with platinum, the cantilever beams are used as heat transfer parts of the sensor, the two cantilever beams are connected to the surface of a platinized silicon substrate, and a platinized area on the surface of the silicon substrate is used as a heat sink of the sensor.

9. The nanomaterial micromanipulation system suitable for use in multiple environments of claim 8, wherein:

the RTD is calibrated by setting a measuring circuit, and the analog voltage is output through an RTD controller Through standard resistance/>And RTD resistance/>Serial voltage division to obtain standard resistor/>And RTD resistance/>The voltages of (2) are/>, respectivelyAnd/>Regulating the size of analog voltage, collecting and drawing RTD resistance/>And voltage/>Obtaining RTD resistance/>Voltage with value fluctuating in a small range/>A section within which the test voltage/>, is selectedTemperature change is controlled by using a temperature control platform, and test voltage/>, is obtained and drawnRTD resistance/>And temperature T, the slope of which is RTD temperature coefficient/>According to temperature coefficientObtaining temperature rise/>According to the temperature rise/>Obtaining thermal conductivity of RTD/>；

Temperature rise The calculation formula of (2) is as follows:

Wherein, Is RTD resistance/>Resistance value variation amount of (a);

Thermal conductivity of RTD The calculation formula is as follows:

10. The nanomaterial micromanipulation system suitable for use in multiple environments of claim 7, wherein: the four-degree-of-freedom displacement platform and the three-dimensional displacement platform are respectively used for adjusting the moving distance through a linear actuator;