WO2024082284A1 - Orthodontic automatic tooth arrangement method and system based on mesh feature deep learning - Google Patents

Orthodontic automatic tooth arrangement method and system based on mesh feature deep learning Download PDF

Info

Publication number
WO2024082284A1
WO2024082284A1 PCT/CN2022/126784 CN2022126784W WO2024082284A1 WO 2024082284 A1 WO2024082284 A1 WO 2024082284A1 CN 2022126784 W CN2022126784 W CN 2022126784W WO 2024082284 A1 WO2024082284 A1 WO 2024082284A1
Authority
WO
WIPO (PCT)
Prior art keywords
tooth
feature
model
dentition
features
Prior art date
Application number
PCT/CN2022/126784
Other languages
French (fr)
Chinese (zh)
Inventor
夏泽洋
黄嘉伟
熊璟
Original Assignee
深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳先进技术研究院 filed Critical 深圳先进技术研究院
Priority to PCT/CN2022/126784 priority Critical patent/WO2024082284A1/en
Publication of WO2024082284A1 publication Critical patent/WO2024082284A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61C—DENTISTRY; APPARATUS OR METHODS FOR ORAL OR DENTAL HYGIENE
    • A61C7/00—Orthodontics, i.e. obtaining or maintaining the desired position of teeth, e.g. by straightening, evening, regulating, separating, or by correcting malocclusions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G06V10/422—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation for representing the structure of the pattern or shape of an object therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/30—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing

Definitions

  • the present invention relates to the field of oral medicine technology, and more specifically, to an automatic tooth arrangement method and system for orthodontics based on deep learning of grid features.
  • Orthodontics is a discipline that specializes in the etiology, diagnosis, treatment and prevention of malocclusion.
  • Clinical orthodontic treatment applies correction force to the teeth to move them, thereby restoring the normal arrangement and occlusion of the teeth by installing correction appliances inside and outside the mouth of patients with malocclusion.
  • Orthodontic tooth arrangement refers to arranging the teeth of patients with malocclusion into the expected ideal position after treatment, which serves as a reference for orthodontic treatment planning. It is a key step in digital orthodontic treatment. However, at present, in the clinical orthodontic treatment planning stage, doctors usually manually arrange the desired tooth position through the human-computer interaction interface. This manual tooth arrangement method generates excessively high labor and time costs.
  • the method based on deep learning regards the tooth arrangement task as a three-dimensional space six-degree-of-freedom posture regression problem, uses the dental posture data before and after treatment to train the deep neural network, learns the shape characteristics of malocclusion teeth, the spatial posture and the mapping relationship between the ideal tooth posture from a large number of cases, and then solves the movement amount from malocclusion teeth to ideal teeth.
  • computer-assisted tooth arrangement solutions mainly include tooth arrangement methods based on dental arch curve fitting and automatic tooth arrangement methods based on deep learning.
  • the method based on dental arch curve fitting it first uses the characteristic points on the teeth to fit the ideal dental arch curve, and then calculates the movement of each tooth based on the positional relationship between the current position of the tooth and the ideal dental arch curve, and then solves the tooth posture after arrangement.
  • the tooth arrangement effect of this method depends on the accuracy of the input characteristic points, which are usually manually selected, so it is difficult to achieve fully automatic tooth arrangement.
  • the original data used is non-perspective data obtained through scanning, which only has surface crown information but no root information, and cannot explicitly consider the root. Therefore, the root of the arranged teeth may be too close to or even move out of the alveolar bone, which is inconsistent with the clinical requirements of orthodontics.
  • the mainstream data types currently used to describe three-dimensional models include point clouds, voxels, multi-view images, and mesh models.
  • the mesh model not only contains the location information of the object, but also can extract the relative topological relationship between adjacent units on the surface of the object, and has a strong representation ability for the shape of the object.
  • the neural network that processes the mesh has achieved better results in object classification, segmentation and other tasks than the neural network that processes the point cloud.
  • the existing deep neural network for automatic tooth arrangement uses the point cloud processing network as an encoder to extract features from the tooth point cloud, compared with the mesh processing network, its ability to capture and process fine-grained features of the tooth is weaker, thus restricting the effect of automatic tooth arrangement.
  • the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a new automatic tooth arrangement method based on deep learning of mesh features, which can learn its feature representation from the tooth triangular mesh model, thereby improving the accuracy of the automatic tooth arrangement results based on the deep learning method.
  • a method for automatic tooth arrangement in orthodontics based on deep learning of grid features comprises the following steps:
  • tooth triangular mesh three-dimensional model for tooth arrangement, wherein the tooth triangular mesh three-dimensional model uses vertices, centroids, midlines, normal vectors and adjacent surface indices of triangular facets to represent shape information and spatial position information of the teeth;
  • the deep learning model includes a first feature encoder, a second feature encoder and a feature decoder and mapper.
  • the first feature encoder takes the tooth triangulated mesh three-dimensional model as input to obtain tooth shape features
  • the second feature encoder takes the dentition point cloud as input to obtain dentition global features.
  • the feature decoder and mapper obtains the predicted tooth arrangement results based on the tooth shape features and the dentition global features.
  • an automatic tooth arrangement system for orthodontics based on deep learning of grid features comprises:
  • a tooth model acquisition unit is used to acquire a tooth triangular mesh three-dimensional model for tooth arrangement, wherein the tooth triangular mesh three-dimensional model uses vertices, centroids, midlines, normal vectors and adjacent surface indexes of triangular facets to represent shape information and spatial position information of the tooth;
  • Prediction unit used for inputting the tooth triangular mesh model into the pre-trained deep learning model to obtain the predicted tooth position of the tooth arrangement result
  • the deep learning model includes a first feature encoder, a second feature encoder and a feature decoder and mapper.
  • the first feature encoder takes the tooth triangulated mesh three-dimensional model as input to obtain tooth shape features
  • the second feature encoder takes the dentition point cloud as input to obtain dentition global features.
  • the feature decoder and mapper obtains the predicted tooth arrangement results based on the tooth shape features and the dentition global features.
  • the advantage of the present invention is that it can automatically plan the ideal position of the teeth of patients with malocclusion based on the three-dimensional model of the tooth triangular mesh, saving the time spent by orthodontists in the treatment planning stage, and overcoming the shortcomings of the current automatic tooth arrangement method based on deep learning, which has a weak ability to extract fine-grained features of teeth and leads to low accuracy of results.
  • FIG1 is a flow chart of an automatic tooth arrangement method for orthodontics based on deep learning of grid features according to an embodiment of the present invention
  • FIG2 is a schematic diagram of a triangular mesh model of a tooth according to an embodiment of the present invention.
  • FIG3 is a structural diagram of an automatic tooth arrangement network model according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a tooth arrangement effect according to an embodiment of the present invention.
  • the provided automatic tooth arrangement method based on deep learning of grid features includes the following steps.
  • Step S110 constructing a training set using the tooth triangulated mesh three-dimensional model.
  • the training set is constructed according to the following steps:
  • Step S111 segmenting and reconstructing the oral bone tissue model.
  • a high-precision crown model and a three-dimensional model of complete teeth are obtained by segmenting and reconstructing the intraoral scan image (or laser scan image of a plaster model) and the oral CT image of a malocclusion patient, respectively, and then the root part of the high-precision crown model obtained from the dental laser scan image and the complete tooth model obtained from the oral CT image are registered and fused to obtain a three-dimensional triangular mesh model of teeth for tooth arrangement.
  • the model is simplified so that the number of triangular facets of all teeth is unified to N. In one embodiment, N is 1000.
  • Step S112 constructing a tooth arrangement data set.
  • the triangular mesh models of all teeth in the malocclusion dentition and the desired dentition are saved respectively.
  • the position of the desired dentition is the expected value of the tooth arrangement task, which is used as the supervision value of network training during the training process and as the gold standard during the test process.
  • the data set is randomly divided into training set, validation set and test set.
  • a three-dimensional model of the oral cavity established using image data after treatment can also be used as the expected value.
  • Step S113 enhancing the tooth arrangement data.
  • data enhancement is divided into individual movement and overall movement.
  • each tooth in the malocclusion dentition in the training set is randomly moved, and the malocclusion dentition before and after the movement corresponds to the same desired dentition.
  • For overall movement all malocclusion dentitions and desired dentitions in the training set are randomly moved synchronously to obtain the expanded training set.
  • the data set can be expanded, thereby improving the accuracy of subsequent model training.
  • Step S114 data preprocessing.
  • Point cloud and mesh data are extracted from the above tooth model as input for the subsequent tooth arrangement network model and loss function.
  • the point cloud is the original vertices of all triangular facets
  • the mesh data is calculated from the normalized tooth model, including the vertices, centroids, midlines, normal vectors, and adjacent face indices of the triangular facets, as shown in Figure 2.
  • normalizing the tooth model includes: scaling all teeth so that they are circumscribed to a unit sphere whose center of mass is the center of the sphere.
  • the shape information of the tooth and its absolute spatial position information are decoupled by normalization, and only the relative position information between vertices is retained. This processing method improves the robustness of the subsequent automatic tooth arrangement network model for teeth of different shapes.
  • Step S120 constructing a deep learning model as a tooth arrangement network model, the model includes a dentition global feature encoder, a tooth shape feature encoder, and a feature decoder and mapper.
  • a deep learning model is constructed as an automatic tooth arrangement network model.
  • the deep learning model and the tooth arrangement network model or the automatic tooth arrangement network model have the same meaning unless otherwise specified according to the context.
  • the deep learning model includes a dentition global feature encoder, a tooth shape feature encoder, and a feature decoder and mapper.
  • the dentition global feature encoder may include multiple convolutional layers, each of which is sequentially connected to batch normalization and Relu activation function.
  • the input of the dentition global feature encoder is the point cloud of the original malocclusion teeth. After convolution, the malocclusion features with global information are output.
  • the tooth shape feature encoder for example, includes a series of identical parallel-connected tooth shape feature extraction modules, each module corresponding to a single tooth one-to-one, so as to independently learn its shape representation from the data of a single tooth.
  • the tooth shape feature extraction module of the embodiment of the present invention learns feature representation from the triangular mesh model of the tooth to improve the ability to extract local fine-grained features of the tooth.
  • the tooth shape feature extraction module includes a space description module, a structure description module and a mesh convolution module.
  • the embodiment of the present invention first proposes to use a mesh feature learning network to extract features from the triangular mesh model of teeth.
  • MeshNet or other deep learning networks that process triangular mesh models can be used to achieve tooth-level feature extraction.
  • the spatial description module uses two one-dimensional convolution layers with a kernel size of 1 to map the centroid of the tooth in the local coordinate system to local spatial features, where the first convolution layer is used to extract local spatial features with a receptive field size of 1 from the local spatial position of the triangular facet.
  • the second convolution layer is used to learn the transformation of features to align the feature vectors of triangular facets with similar shapes but different postures, and learn more robust local spatial features, which are recorded as
  • the structure description module includes triangular patch rotation convolution and triangular patch kernel correlation.
  • the triangular patch rotation convolution can be described using a symmetric function f, expressed as:
  • h1 shares the weights of the combination of the three groups of midlines of the input
  • oa represents the three-dimensional vector formed by the centroid of the triangle and vertex A
  • ob represents the three-dimensional vector formed by the centroid of the triangle and vertex B
  • oc represents the three-dimensional vector formed by the centroid of the triangle and vertex C.
  • K1 32
  • K2 64.
  • the triangular patch rotation convolution operation has rotation invariance to eliminate the influence of the disorder of the midline on the extraction of unified triangular patch shape features.
  • Triangle patch kernel correlation is used to expand the receptive field of the structural feature vector to the adjacent patches of the triangle patch. Its input is the normal vector of the triangle patch and the adjacent face index. First, the normal vector of each face is combined with the normal vectors of the corresponding three adjacent faces through the adjacent face index to obtain the aggregated normal vector. Then, a kernel with learnable parameters is set to perform correlation operations with the aggregated normal vector.
  • results of the triangle patch rotation convolution operation and the triangle patch kernel correlation operation are spliced, and then two layers of one-dimensional convolution with channels of 131 are used to map the spliced vectors into structural features, which are recorded as
  • a grid convolution module is used to reorganize the outputs of the spatial description module and the structural description module at the tooth level. and To extract deeper features.
  • the local spatial features are concatenated with the corresponding structural feature vectors and then convolved to obtain the local spatial features that incorporate structural information, which can be expressed as:
  • W mc1 is a one-dimensional convolution with a convolution kernel size of 1 and 256 channels.
  • the structural feature vector of each triangular facet is first aggregated with the structural feature vector of the adjacent triangular facets through the adjacent face index, and then the aggregated structural feature vector is convolved and max pooled to obtain the aggregated structural features, which can be expressed as:
  • W mc2 is a two-dimensional convolution with a convolution kernel size of 1 ⁇ 1 and 256 channels.
  • the global features of the dentition with the spatial position information of the teeth and the shape features of all teeth are fused and reduced in dimension, and then mapped into the movement of the teeth as the output of the tooth arrangement network.
  • the decoder concatenates the input dentition global feature F dentition and the shape feature of all teeth F shape ⁇ N, and uses two one-dimensional convolutional layers with 28 and 64 channels and one fully connected layer to decode the concatenated features to obtain the movement to be transformed Then, the mobile representation transformation is performed, which may include two steps: constraint transformation and representation transformation.
  • sign( ⁇ ) represents a sign function.
  • the rotation and translation are transformed into a homogeneous transformation matrix in the world coordinate system, and the predicted tooth position is calculated.
  • the axis vector L A axis ′ is converted into a rotation matrix using the exponential mapping from so(3) to SO(3)
  • L T is converted into the homogeneous transformation matrix W T in the world coordinate system, and the original malocclusion teeth are Move the tooth to get the automatic tooth arrangement result, that is, the predicted position of the tooth, recorded as
  • the rotation amount is constrained by using the softmax function, which ensures that the output movement amount is a rigid body transformation matrix and at the same time can meet the numerical stability of the calculation process.
  • the loss function is set using the Euclidean distance measurement network output of the tooth predicted position and the distance between the corresponding points of the expected position, expressed as:
  • the maximum value of the Euclidean distance deviation can also be weighted and used as a separate item of the loss function.
  • the cosine similarity error between the predicted transformation matrix and the expected transformation matrix can be used as the loss function.
  • the constructed deep learning model was trained using the training set.
  • the Adam optimizer was used to optimize the model parameters by minimizing the loss function value, and a series of network models with different parameters were obtained.
  • the trained model was then verified using the validation set, and the network model with the best effect was saved as the final tooth arrangement network model, which was used to perform tooth arrangement operations on malocclusion samples that had not been tooth arranged.
  • Step S140 using the trained deep learning model to obtain the tooth arrangement result of the target patient.
  • the final tooth arrangement network model can be used for actual tooth arrangement prediction, that is, the triangular mesh three-dimensional model of the target patient's teeth to be arranged is input into the trained tooth arrangement network model to obtain the predicted tooth movement as the tooth arrangement result.
  • LAxis is the rotation axis vector in the local coordinate system
  • the present invention also provides an automatic tooth arrangement system for orthodontics based on deep learning of mesh features.
  • the system comprises: a tooth model acquisition unit, which is used to acquire a tooth triangular mesh three-dimensional model for tooth arrangement, wherein the tooth triangular mesh three-dimensional model uses the vertices, centroids, midlines, normal vectors and adjacent face indexes of triangular facets to characterize the shape information and spatial position information of the teeth; a prediction unit, which is used to input the tooth triangular mesh model into a pre-trained deep learning model to obtain the predicted tooth position of the tooth arrangement result, wherein the deep learning model comprises a dentition global feature encoder, a tooth shape feature encoder and a feature decoder and mapper, wherein the tooth shape feature encoder uses the tooth triangular mesh three-dimensional model as input to obtain tooth shape features, the dentition global feature encoder uses the dentition point cloud as input to obtain dentition global features, and the feature decoder and mapper obtains the predicted tooth arrangement result based on
  • the model training process involved in the present invention can be performed offline on a server or in the cloud, and the trained model can be embedded in an electronic device to achieve real-time automatic tooth arrangement result generation.
  • the electronic device can be a terminal device or a server, and the terminal device includes any terminal device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sale (POS), a car computer, an intelligent wearable device (smart watch, virtual reality glasses, virtual reality helmet, etc.).
  • the server includes but is not limited to an application server or a Web server, and can be an independent server, a cluster server or a cloud server.
  • the present invention can also be extended to the task of automatic denture arrangement in the field of oral prosthodontics.
  • the present invention designs a mesh feature learning network to extract shape features from the triangular mesh model of teeth, and on this basis learns the shape features and the underlying rules of the global spatial features and the expected positions of teeth to arrange teeth.
  • the present invention has the following advantages:
  • the existing deep learning-based automatic tooth arrangement method uses scanned images of the crown as input, which only contains crown information, but not the root and alveolar bone information inside the gums. Therefore, it is impossible to explicitly consider the root posture. In clinical use, the root may be too close to or even move out of the alveolar bone, leading to root absorption, bone fenestration and bone cracking, which does not meet clinical requirements.
  • the present invention considers the root posture, inputs the complete tooth 3D model data containing the root into the dentition global feature encoder and the tooth shape feature encoder, and the loss function also calculates the distance error of the tooth arrangement result containing the root accordingly, and then realizes the introduction of the root posture into the automatic tooth arrangement algorithm by optimizing the network model parameters.
  • the existing deep learning-based automatic tooth arrangement method uses a point cloud processing network as a tooth point cloud feature encoder, which has limited ability to extract fine-grained features.
  • the present invention uses a mesh processing network to extract tooth shape features, which can more fully extract features from triangular mesh data with object surface topological relationships, and designs a tooth shape encoder based on mesh feature learning to extract fine-grained shape features of the tooth triangular mesh model, thereby improving the accuracy of deep learning network automatic tooth arrangement.
  • the present invention may be a system, a method and/or a computer program product.
  • the computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.
  • Computer readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device.
  • Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.
  • a more specific example (non-exhaustive list) of computer readable storage medium includes: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove having instructions stored thereon, and any suitable combination thereof.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • SRAM static random access memory
  • CD-ROM compact disk read-only memory
  • DVD digital versatile disk
  • memory stick a floppy disk
  • mechanical encoding device for example, a punch card or a convex structure in a groove having instructions stored thereon, and any suitable combination thereof
  • the computer readable storage medium used here is not interpreted as an instantaneous signal itself, such as a radio wave or other freely propagating electromagnetic waves, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.
  • the computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing/processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and/or edge servers.
  • the network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device.
  • the computer program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, Python, etc., and conventional procedural programming languages such as "C" language or similar programming languages.
  • Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server.
  • the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet).
  • an electronic circuit such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.
  • These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions/actions specified in one or more boxes in the flowchart and/or block diagram is generated.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and/or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions/actions specified in one or more boxes in the flowchart and/or block diagram.
  • Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions/actions specified in one or more boxes in the flowchart and/or block diagram.
  • each box in the flowchart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the specified logical function.
  • the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
  • each box in the block diagram and/or flowchart, and the combination of the boxes in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Epidemiology (AREA)
  • Public Health (AREA)
  • Primary Health Care (AREA)
  • Computer Graphics (AREA)
  • Databases & Information Systems (AREA)
  • Animal Behavior & Ethology (AREA)
  • Computer Hardware Design (AREA)
  • Geometry (AREA)
  • Architecture (AREA)
  • Probability & Statistics with Applications (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Dentistry (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Veterinary Medicine (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Dental Tools And Instruments Or Auxiliary Dental Instruments (AREA)

Abstract

An orthodontic automatic tooth arrangement method and system based on mesh feature deep learning. The method comprises: acquiring a tooth triangular mesh three-dimensional model for tooth arrangement; and inputting the tooth triangular mesh model into a pre-trained deep learning model, so as to obtain a predicted tooth position of a tooth arrangement result. The deep learning model comprises a first feature encoder, a second feature encoder, and a feature decoding and mapper, wherein the first feature encoder takes a tooth triangular mesh three-dimensional model as an input, so as to obtain a tooth shape feature, the second feature encoder takes a dentition point cloud as an input, so as to obtain a global dentition feature, and the feature decoding and mapper obtains a predicted tooth arrangement result on the basis of the tooth shape feature and the global dentition feature.

Description

基于网格特征深度学习的口腔正畸自动排牙方法和***Orthodontic automatic tooth arrangement method and system based on deep learning of grid features 技术领域Technical Field
本发明涉及口腔医学技术领域，更具体地，涉及一种基于网格特征深度学习的口腔正畸自动排牙方法和***。The present invention relates to the field of oral medicine technology, and more specifically, to an automatic tooth arrangement method and system for orthodontics based on deep learning of grid features.
背景技术Background technique
牙齿错颌畸形发病率较高，对口腔健康的危害不容忽视。口腔正畸学是专门研究错颌畸形的病因、诊断、治疗和预防的学科。临床口腔正畸治疗通过在错颌畸形患者口内外安装矫治器械，对牙齿施加矫治力以使牙齿产生移动，从而恢复牙齿正常的排列和咬合关系。The incidence of malocclusion is high, and the harm to oral health cannot be ignored. Orthodontics is a discipline that specializes in the etiology, diagnosis, treatment and prevention of malocclusion. Clinical orthodontic treatment applies correction force to the teeth to move them, thereby restoring the normal arrangement and occlusion of the teeth by installing correction appliances inside and outside the mouth of patients with malocclusion.
正畸排牙是指将错颌畸形患者的牙列排列成预期的治疗后理想位姿，以作为正畸治疗规划的参考基础，是数字化口腔正畸治疗的关键步骤。但是目前在临床口腔正畸治疗规划阶段，医师通常通过人机交互界面手动排列出期望的牙齿位姿。这种人工排牙方式产生了过高的人工和时间成本。Orthodontic tooth arrangement refers to arranging the teeth of patients with malocclusion into the expected ideal position after treatment, which serves as a reference for orthodontic treatment planning. It is a key step in digital orthodontic treatment. However, at present, in the clinical orthodontic treatment planning stage, doctors usually manually arrange the desired tooth position through the human-computer interaction interface. This manual tooth arrangement method generates excessively high labor and time costs.
近年来，随着计算机图形学和人工智能技术的发展，学界展开了人工智能排牙算法的研究，主要分为早期基于牙弓曲线拟合的方法和现今基于深度学习的方法两大类。基于深度学习的方法是将排牙任务看作一个三维空间六自由度位姿回归问题，使用治疗前后的牙列位姿数据训练深度神经网络，从大量案例中学习错颌牙齿的形状特征、空间位姿与理想牙齿位姿的映射关系，进而求解出从错颌牙齿到理想牙齿的移动量。In recent years, with the development of computer graphics and artificial intelligence technology, the academic community has launched research on artificial intelligence tooth arrangement algorithms, which are mainly divided into two categories: the early method based on dental arch curve fitting and the current method based on deep learning. The method based on deep learning regards the tooth arrangement task as a three-dimensional space six-degree-of-freedom posture regression problem, uses the dental posture data before and after treatment to train the deep neural network, learns the shape characteristics of malocclusion teeth, the spatial posture and the mapping relationship between the ideal tooth posture from a large number of cases, and then solves the movement amount from malocclusion teeth to ideal teeth.
在现有技术中，计算机辅助排牙方案主要包括基于牙弓曲线拟合的排牙方法和基于深度学习的自动排牙方法。In the prior art, computer-assisted tooth arrangement solutions mainly include tooth arrangement methods based on dental arch curve fitting and automatic tooth arrangement methods based on deep learning.
对于基于牙弓曲线拟合的方法，其首先使用牙齿上的特征点拟合理想牙弓曲线，再根据牙齿当前位置和理想牙弓曲线的位置关系计算每颗牙齿的移动量，进而求解排列后的牙齿位姿。该方法的排牙效果取决于输入特征点的精度，特征点通常通过人工手动选取，因此难以实现全自动排牙。For the method based on dental arch curve fitting, it first uses the characteristic points on the teeth to fit the ideal dental arch curve, and then calculates the movement of each tooth based on the positional relationship between the current position of the tooth and the ideal dental arch curve, and then solves the tooth posture after arrangement. The tooth arrangement effect of this method depends on the accuracy of the input characteristic points, which are usually manually selected, so it is difficult to achieve fully automatic tooth arrangement.
对于基于深度学习的方法，其使用的原始数据是通过扫描得到的非透视数据，仅具有表面的牙冠信息，不具有牙根的信息，无法显式考虑牙根，因此可能出现排列后牙齿的牙根过于贴近甚至移出牙槽骨的情况，与正畸临床要求不符。此外，目前用于描述三维模型的主流数据类型包括点云、体素、多视角图片和网格模型，其中网格模型不仅包含了物体的位置信息，还能够提取物体表面相邻单元的相对拓扑关系，对物体的形状具有较强的表示能力。与之相应地，处理网格的神经网络相比于处理点云的神经网络，在物体分类、分割等任务上取得了更好的效果。由于现有自动排牙深度神经网络使用点云处理网络作为编码器对牙齿点云进行特征提取，与网格处理网络相比，其对于牙齿细粒度特征的捕捉和处理能力较弱，因此制约了自动排牙效果。For deep learning-based methods, the original data used is non-perspective data obtained through scanning, which only has surface crown information but no root information, and cannot explicitly consider the root. Therefore, the root of the arranged teeth may be too close to or even move out of the alveolar bone, which is inconsistent with the clinical requirements of orthodontics. In addition, the mainstream data types currently used to describe three-dimensional models include point clouds, voxels, multi-view images, and mesh models. The mesh model not only contains the location information of the object, but also can extract the relative topological relationship between adjacent units on the surface of the object, and has a strong representation ability for the shape of the object. Correspondingly, the neural network that processes the mesh has achieved better results in object classification, segmentation and other tasks than the neural network that processes the point cloud. Since the existing deep neural network for automatic tooth arrangement uses the point cloud processing network as an encoder to extract features from the tooth point cloud, compared with the mesh processing network, its ability to capture and process fine-grained features of the tooth is weaker, thus restricting the effect of automatic tooth arrangement.
发明内容Summary of the invention
本发明的目的是克服上述现有技术的缺陷，提供一种新的基于网格特征深度学习的自动排牙方法，能够从牙齿三角网格模型中学习其特征表示，提高了基于深度学习方法的自动排牙结果的准确度。The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a new automatic tooth arrangement method based on deep learning of mesh features, which can learn its feature representation from the tooth triangular mesh model, thereby improving the accuracy of the automatic tooth arrangement results based on the deep learning method.
根据本发明的第一方面，提供一种基于网格特征深度学习的口腔正畸自动排牙方法。该方法包括以下步骤：According to a first aspect of the present invention, a method for automatic tooth arrangement in orthodontics based on deep learning of grid features is provided. The method comprises the following steps:
获取用于排牙的牙齿三角网格三维模型，所述牙齿三角网格三维模型利用三角面片的顶点、形心、中线、法向量和邻接面索引表征牙齿的形状信息和空间位置信息；Acquire a tooth triangular mesh three-dimensional model for tooth arrangement, wherein the tooth triangular mesh three-dimensional model uses vertices, centroids, midlines, normal vectors and adjacent surface indices of triangular facets to represent shape information and spatial position information of the teeth;
将所述牙齿三角网格模型输入到预训练的深度学习模型，获得排牙结果的牙齿预测位置；Inputting the tooth triangular mesh model into a pre-trained deep learning model to obtain the predicted tooth position of the tooth arrangement result;
其中，所述深度学习模型包括第一特征编码器、第二特征编码器以及特征解码与映射器，第一特征编码器以所述牙齿三角网格三维模型作为输入，获得牙齿形状特征，第二特征编码器以牙列点云作为输入，获得牙列全局特征，所述特征解码与映射器基于所述牙齿形状特征和所述牙列全局特征获得预测的排牙结果。Among them, the deep learning model includes a first feature encoder, a second feature encoder and a feature decoder and mapper. The first feature encoder takes the tooth triangulated mesh three-dimensional model as input to obtain tooth shape features, and the second feature encoder takes the dentition point cloud as input to obtain dentition global features. The feature decoder and mapper obtains the predicted tooth arrangement results based on the tooth shape features and the dentition global features.
根据本发明的第二方面，提供一种基于网格特征深度学习的口腔正畸 自动排牙***。该***包括：According to a second aspect of the present invention, an automatic tooth arrangement system for orthodontics based on deep learning of grid features is provided. The system comprises:
牙齿模型获取单元：用于获取用于排牙的牙齿三角网格三维模型，所述牙齿三角网格三维模型利用三角面片的顶点、形心、中线、法向量和邻接面索引表征牙齿的形状信息和空间位置信息；A tooth model acquisition unit is used to acquire a tooth triangular mesh three-dimensional model for tooth arrangement, wherein the tooth triangular mesh three-dimensional model uses vertices, centroids, midlines, normal vectors and adjacent surface indexes of triangular facets to represent shape information and spatial position information of the tooth;
预测单元：用于将所述牙齿三角网格模型输入到预训练的深度学习模型，获得排牙结果的牙齿预测位置；Prediction unit: used for inputting the tooth triangular mesh model into the pre-trained deep learning model to obtain the predicted tooth position of the tooth arrangement result;
其中，所述深度学习模型包括第一特征编码器、第二特征编码器以及特征解码与映射器，第一特征编码器以所述牙齿三角网格三维模型作为输入，获得牙齿形状特征，第二特征编码器以牙列点云作为输入，获得牙列全局特征，所述特征解码与映射器基于所述牙齿形状特征和所述牙列全局特征获得预测的排牙结果。Among them, the deep learning model includes a first feature encoder, a second feature encoder and a feature decoder and mapper. The first feature encoder takes the tooth triangulated mesh three-dimensional model as input to obtain tooth shape features, and the second feature encoder takes the dentition point cloud as input to obtain dentition global features. The feature decoder and mapper obtains the predicted tooth arrangement results based on the tooth shape features and the dentition global features.
与现有技术相比，本发明的优点在于，能够基于牙齿三角网格三维模型自动规划错颌畸形患者牙齿的理想位姿，节省口腔正畸医师治疗规划阶段所用时间，克服目前基于深度学习的自动排牙方法提取牙齿细粒度特征的能力较弱导致结果准确度不高的缺点。Compared with the existing technology, the advantage of the present invention is that it can automatically plan the ideal position of the teeth of patients with malocclusion based on the three-dimensional model of the tooth triangular mesh, saving the time spent by orthodontists in the treatment planning stage, and overcoming the shortcomings of the current automatic tooth arrangement method based on deep learning, which has a weak ability to extract fine-grained features of teeth and leads to low accuracy of results.
通过以下参照附图对本发明的示例性实施例的详细描述，本发明的其它特征及其优点将会变得清楚。Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
被结合在说明书中并构成说明书的一部分的附图示出了本发明的实施例，并且连同其说明一起用于解释本发明的原理。The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
图1是根据本发明一个实施例的基于网格特征深度学习的口腔正畸自动排牙方法的流程图；FIG1 is a flow chart of an automatic tooth arrangement method for orthodontics based on deep learning of grid features according to an embodiment of the present invention;
图2是根据本发明一个实施例的牙齿三角网格模型的示意图；FIG2 is a schematic diagram of a triangular mesh model of a tooth according to an embodiment of the present invention;
图3是根据本发明一个实施例的自动排牙网络模型的结构图；FIG3 is a structural diagram of an automatic tooth arrangement network model according to an embodiment of the present invention;
图4是根据本发明一个实施例的排牙效果示意图。FIG. 4 is a schematic diagram of a tooth arrangement effect according to an embodiment of the present invention.
具体实施方式Detailed ways
现在将参照附图来详细描述本发明的各种示例性实施例。应注意到： 除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本发明的范围。Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本发明及其应用或使用的任何限制。The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.
在这里示出和讨论的所有例子中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它例子可以具有不同的值。In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
参见图1所示，所提供的基于网格特征深度学习的自动排牙方法包括以下步骤。As shown in FIG1 , the provided automatic tooth arrangement method based on deep learning of grid features includes the following steps.
步骤S110，利用牙齿三角网格三维模型构建训练集。Step S110, constructing a training set using the tooth triangulated mesh three-dimensional model.
在一个实施例中，根据以下步骤构建训练集：In one embodiment, the training set is constructed according to the following steps:
步骤S111，分割重构口腔骨组织模型。Step S111, segmenting and reconstructing the oral bone tissue model.
例如，分别从错颌畸形患者的口内扫描图像(或石膏模型的激光扫描图像)和口腔CT图像中分割重构得到高精度牙冠模型和完整牙齿的三维模型，再对从牙颌激光扫描图像得到的高精度牙冠模型和口腔CT图像得到的完整牙齿模型的牙根部分进行配准和融合，得到用于排牙的牙齿三角网格三维模型。对模型进行简化，使所有牙齿的三角面片数统一为N。在一个实施例中，N取1000。For example, a high-precision crown model and a three-dimensional model of complete teeth are obtained by segmenting and reconstructing the intraoral scan image (or laser scan image of a plaster model) and the oral CT image of a malocclusion patient, respectively, and then the root part of the high-precision crown model obtained from the dental laser scan image and the complete tooth model obtained from the oral CT image are registered and fused to obtain a three-dimensional triangular mesh model of teeth for tooth arrangement. The model is simplified so that the number of triangular facets of all teeth is unified to N. In one embodiment, N is 1000.
步骤S112，构建排牙数据集。Step S112, constructing a tooth arrangement data set.
例如，根据六项颌标准将错颌牙齿移动到期望位置后，分别保存错颌牙列和期望牙列的所有牙齿的三角网格模型。期望牙列的位置为排牙任务的期望值，其在训练过程中作为网络训练的监督值，在测试过程中作为金标准。对数据集进行随机划分，得到训练集、验证集和测试集。For example, after the malocclusion teeth are moved to the desired position according to the six jaw standards, the triangular mesh models of all teeth in the malocclusion dentition and the desired dentition are saved respectively. The position of the desired dentition is the expected value of the tooth arrangement task, which is used as the supervision value of network training during the training process and as the gold standard during the test process. The data set is randomly divided into training set, validation set and test set.
在另一个实施例中，也可以使用治疗后(牙齿已排齐)图像数据建立 的口腔三维模型作为期望值。In another embodiment, a three-dimensional model of the oral cavity established using image data after treatment (teeth are aligned) can also be used as the expected value.
步骤S113，对排牙数据进行增强。Step S113, enhancing the tooth arrangement data.
例如，数据增强分为个体移动和整体移动。对于个体移动，随机移动训练集中错颌牙列中的每颗牙齿，移动前后的错颌牙列对应同一期望牙列。对于整体移动，对训练集中所有错颌牙列和期望牙列进行同步的随机移动，得到扩充后的训练集。通过数据增强，可以扩充数据集，从而提升后续模型训练的精确度。For example, data enhancement is divided into individual movement and overall movement. For individual movement, each tooth in the malocclusion dentition in the training set is randomly moved, and the malocclusion dentition before and after the movement corresponds to the same desired dentition. For overall movement, all malocclusion dentitions and desired dentitions in the training set are randomly moved synchronously to obtain the expanded training set. Through data enhancement, the data set can be expanded, thereby improving the accuracy of subsequent model training.
步骤S114，数据预处理。Step S114: data preprocessing.
从上述牙齿模型中提取点云和网格数据，作为后续排牙网络模型及损失函数的输入。例如，点云为所有三角面片的原始顶点，网格数据从归一化后的牙齿模型中计算得到，包括三角面片的顶点、形心、中线、法向量和邻接面索引，如图2所示。Point cloud and mesh data are extracted from the above tooth model as input for the subsequent tooth arrangement network model and loss function. For example, the point cloud is the original vertices of all triangular facets, and the mesh data is calculated from the normalized tooth model, including the vertices, centroids, midlines, normal vectors, and adjacent face indices of the triangular facets, as shown in Figure 2.
在一个实施例中，对牙齿模型进行归一化包括：将所有牙齿进行缩放，使其分别外接于其质心为球心的单位球。通过归一化将牙齿的形状信息和其空间绝对位置信息解耦，只保留顶点之间的相对位置信息。这种处理方式提升了后续自动排牙网络模型对于不同形状牙齿的鲁棒性。In one embodiment, normalizing the tooth model includes: scaling all teeth so that they are circumscribed to a unit sphere whose center of mass is the center of the sphere. The shape information of the tooth and its absolute spatial position information are decoupled by normalization, and only the relative position information between vertices is retained. This processing method improves the robustness of the subsequent automatic tooth arrangement network model for teeth of different shapes.
步骤S120，构建深度学习模型作为排牙网络模型，模型包括牙列全局特征编码器、牙齿形状特征编码器以及特征解码与映射器。Step S120, constructing a deep learning model as a tooth arrangement network model, the model includes a dentition global feature encoder, a tooth shape feature encoder, and a feature decoder and mapper.
在此步骤中，构建深度学习模型作为自动排牙网络模型，在本文的描述中，深度学习模型和排牙网络模型或自动排牙网络模型具有相同含义，除非根据上下文另有所指。In this step, a deep learning model is constructed as an automatic tooth arrangement network model. In the description of this article, the deep learning model and the tooth arrangement network model or the automatic tooth arrangement network model have the same meaning unless otherwise specified according to the context.
参见图3所示，深度学习模型包括牙列全局特征编码器、牙齿形状特征编码器以及特征解码与映射器。As shown in Figure 3, the deep learning model includes a dentition global feature encoder, a tooth shape feature encoder, and a feature decoder and mapper.
(1)牙列全局特征编码器(1) Dental global feature encoder
在一个实施例中，牙列全局特征编码器可包括多层卷积层，每层卷积层依次连接批正则化和Relu激活函数。In one embodiment, the dentition global feature encoder may include multiple convolutional layers, each of which is sequentially connected to batch normalization and Relu activation function.
牙列全局特征编码器输入为原始错颌牙齿的点云
Figure PCTCN2022126784-appb-000001
经过卷积后输出带有全局信息的错颌牙列特征。 The input of the dentition global feature encoder is the point cloud of the original malocclusion teeth.
Figure PCTCN2022126784-appb-000001
After convolution, the malocclusion features with global information are output.
(2)牙齿形状特征编码器(2) Tooth shape feature encoder
例外，借鉴TANet(Wei G,Cui Z,Liu Y,et al.TANet:Towards Fully Automatic Tooth Arrangement[C]//European Conference on Computer Vision.Springer,Cham,2020:481-497.)和Li等(Li X,Bi L,Kim J,et al.Malocclusion treatment planning via pointnet based spatial transformation network[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Springer,Cham,2020:105-114)的自动排牙网络设计，牙齿形状特征编码器例如包含一系列相同的并行连接的牙齿形状特征提取模块，每个模块与单颗牙齿一一对应，以分别从单颗牙齿的数据中独立地学习其形状表示。与现有网络不同的是，本发明实施例的牙齿形状特征提取模块从牙齿的三角网格模型中学习特征表示，以提升对牙齿局部细粒度特征的提取能力。牙齿形状特征提取模块包括空间描述模块、结构描述模块和网格卷积模块。As an exception, referring to the automatic tooth arrangement network design of TANet (Wei G, Cui Z, Liu Y, et al. TANet: Towards Fully Automatic Tooth Arrangement [C] // European Conference on Computer Vision. Springer, Cham, 2020: 481-497.) and Li et al. (Li X, Bi L, Kim J, et al. Malocclusion treatment planning via pointnet based spatial transformation network [C] // International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2020: 105-114), the tooth shape feature encoder, for example, includes a series of identical parallel-connected tooth shape feature extraction modules, each module corresponding to a single tooth one-to-one, so as to independently learn its shape representation from the data of a single tooth. Different from the existing network, the tooth shape feature extraction module of the embodiment of the present invention learns feature representation from the triangular mesh model of the tooth to improve the ability to extract local fine-grained features of the tooth. The tooth shape feature extraction module includes a space description module, a structure description module and a mesh convolution module.
需说明的是，现有的基于深度学习的自动排牙方法使用的都是点云数据及网络，本发明实施例首次提出使用网格特征学习网络对牙齿的三角网格模型进行特征提取，可采用MeshNet或其他处理三角网格模型的深度学习网络实现牙齿级的特征提取。通过设计了基于网格特征学习的牙齿形状特征编码器，提升了对牙齿细粒度形状特征的提取能力。It should be noted that the existing automatic tooth arrangement methods based on deep learning all use point cloud data and networks. The embodiment of the present invention first proposes to use a mesh feature learning network to extract features from the triangular mesh model of teeth. MeshNet or other deep learning networks that process triangular mesh models can be used to achieve tooth-level feature extraction. By designing a tooth shape feature encoder based on mesh feature learning, the ability to extract fine-grained tooth shape features is improved.
在一个实施例中，空间描述模块使用两层核尺寸为1的一维卷积层,将牙齿局部坐标系下的形心映射为局部空间特征，其中第一层卷积用于从三角面片的局部空间位置中提取感受野大小为1的局部空间特征。第二层卷积用于学习特征的变换，以配准形状相似但位姿不同的三角面片的特征向量，学到更具有鲁棒性的局部空间特征，记为
Figure PCTCN2022126784-appb-000002
In one embodiment, the spatial description module uses two one-dimensional convolution layers with a kernel size of 1 to map the centroid of the tooth in the local coordinate system to local spatial features, where the first convolution layer is used to extract local spatial features with a receptive field size of 1 from the local spatial position of the triangular facet. The second convolution layer is used to learn the transformation of features to align the feature vectors of triangular facets with similar shapes but different postures, and learn more robust local spatial features, which are recorded as
Figure PCTCN2022126784-appb-000002
在一个实施例中，结构描述模块包括三角面片旋转卷积和三角面片核相关。三角面片旋转卷积可以使用对称函数f描述，表示为：In one embodiment, the structure description module includes triangular patch rotation convolution and triangular patch kernel correlation. The triangular patch rotation convolution can be described using a symmetric function f, expressed as:
Figure PCTCN2022126784-appb-000003
Figure PCTCN2022126784-appb-000003
其中
Figure PCTCN2022126784-appb-000004
分别为2个一维卷积层，h 1对输入的三组中线的组合进行了权值共享，oa表示三角面片的形心与顶点A所构成的三维向量，ob表示三角面片的形心与顶点B所构成的三维向 量，oc表示三角面片的形心与顶点C所构成的三维向量。在本实施例中，K 1＝32，K 2＝64。三角面片旋转卷积运算具有旋转不变性，以消除中线的无序性对于提取统一的三角面片形状特征的影响。 in
Figure PCTCN2022126784-appb-000004
There are two one-dimensional convolution layers respectively, h1 shares the weights of the combination of the three groups of midlines of the input, oa represents the three-dimensional vector formed by the centroid of the triangle and vertex A, ob represents the three-dimensional vector formed by the centroid of the triangle and vertex B, and oc represents the three-dimensional vector formed by the centroid of the triangle and vertex C. In this embodiment, K1 = 32, K2 = 64. The triangular patch rotation convolution operation has rotation invariance to eliminate the influence of the disorder of the midline on the extraction of unified triangular patch shape features.
三角面片核相关用于将结构特征向量的感受野扩大至三角面片的相邻面片，其输入为三角面片的法向量和邻接面索引。首先通过邻接面索引将每个面的法向量和对应的三个邻接面的法向量进行组合，得到聚合法向量。再设置参数可学习的核，与聚合法向量进行相关运算。Triangle patch kernel correlation is used to expand the receptive field of the structural feature vector to the adjacent patches of the triangle patch. Its input is the normal vector of the triangle patch and the adjacent face index. First, the normal vector of each face is combined with the normal vectors of the corresponding three adjacent faces through the adjacent face index to obtain the aggregated normal vector. Then, a kernel with learnable parameters is set to perform correlation operations with the aggregated normal vector.
进一步地，对三角面片旋转卷积运算和三角面片核相关运算的结果进行拼接，再使用通道均为131的两层一维卷积将拼接后的向量映射为结构特征，记为
Figure PCTCN2022126784-appb-000005
Furthermore, the results of the triangle patch rotation convolution operation and the triangle patch kernel correlation operation are spliced, and then two layers of one-dimensional convolution with channels of 131 are used to map the spliced vectors into structural features, which are recorded as
Figure PCTCN2022126784-appb-000005
为了增大局部空间特征和结构特征的感受野，使用了网格卷积模块，在牙齿水平上重组空间描述模块和结构描述模块的输出
Figure PCTCN2022126784-appb-000006
和
Figure PCTCN2022126784-appb-000007
以提取更深层的特征。 In order to increase the receptive field of local spatial features and structural features, a grid convolution module is used to reorganize the outputs of the spatial description module and the structural description module at the tooth level.
Figure PCTCN2022126784-appb-000006
and
Figure PCTCN2022126784-appb-000007
To extract deeper features.
对于局部空间特征，将其与相应的结构特征向量拼接后进行卷积，以获取融合了结构信息的局部空间特征，可表示为：For the local spatial features, they are concatenated with the corresponding structural feature vectors and then convolved to obtain the local spatial features that incorporate structural information, which can be expressed as:
Figure PCTCN2022126784-appb-000008
Figure PCTCN2022126784-appb-000008
其中
Figure PCTCN2022126784-appb-000009
表示向量的拼接，W mc1为一维卷积，其卷积核尺寸为1，通道数为256。 in
Figure PCTCN2022126784-appb-000009
represents the concatenation of vectors, W mc1 is a one-dimensional convolution with a convolution kernel size of 1 and 256 channels.
对于结构特征，类似于结构描述模块中三角面片的法向量聚合操作，首先通过邻接面索引，将每个三角面片的结构特征向量与相邻三角面片的结构特征向量进行聚合，再对聚合后的结构特征向量进行卷积和最大池化操作，以获取聚合后的结构特征，可表示为：For structural features, similar to the normal vector aggregation operation of triangular facets in the structural description module, the structural feature vector of each triangular facet is first aggregated with the structural feature vector of the adjacent triangular facets through the adjacent face index, and then the aggregated structural feature vector is convolved and max pooled to obtain the aggregated structural features, which can be expressed as:
Figure PCTCN2022126784-appb-000010
Figure PCTCN2022126784-appb-000010
其中gather表示相邻三角面片的结构特征向量的聚合，W mc2为二维卷积，其卷积核尺寸为1×1，通道数为256。 Where gather represents the aggregation of structural feature vectors of adjacent triangular faces, W mc2 is a two-dimensional convolution with a convolution kernel size of 1×1 and 256 channels.
接下来，在牙齿水平上对
Figure PCTCN2022126784-appb-000011
和
Figure PCTCN2022126784-appb-000012
进行拼接，得到牙齿形状特征
Figure PCTCN2022126784-appb-000013
可表示为： Next, at the tooth level
Figure PCTCN2022126784-appb-000011
and
Figure PCTCN2022126784-appb-000012
Splice to get the tooth shape characteristics
Figure PCTCN2022126784-appb-000013
It can be expressed as:
Figure PCTCN2022126784-appb-000014
Figure PCTCN2022126784-appb-000014
(3)特征解码与映射器(3) Feature Decoder and Mapper
在特征解码与映射器中，对带有牙齿空间位置信息的牙列全局特征和所有牙齿的形状特征进行融合和降维，再映射为牙齿的移动量，作为排牙网络的输出。In the feature decoder and mapper, the global features of the dentition with the spatial position information of the teeth and the shape features of all teeth are fused and reduced in dimension, and then mapped into the movement of the teeth as the output of the tooth arrangement network.
在图3中，解码器将输入的牙列全局特征F dentition和所有牙齿的形状特征F shape×N进行拼接，使用通道数分别为28，64的两层一维卷积层和一层全连接层对拼接的特征进行解码，得到待变换移动量
Figure PCTCN2022126784-appb-000015
然后，进行移动表示变换。移动表示变换可包括约束变换和表示变换两步。 In Figure 3, the decoder concatenates the input dentition global feature F dentition and the shape feature of all teeth F shape ×N, and uses two one-dimensional convolutional layers with 28 and 64 channels and one fully connected layer to decode the concatenated features to obtain the movement to be transformed
Figure PCTCN2022126784-appb-000015
Then, the mobile representation transformation is performed, which may include two steps: constraint transformation and representation transformation.
对于约束变换，为了使转轴向量满足单位向量且保证变换过程的数值稳定性，使用softmax函数对转轴向量 LA axis＝( LA 0， LA 1， LA 2)进行映射，得到新的转轴向量 LA axis′： For constrained transformation, in order to make the rotation axis vector satisfy the unit vector and ensure the numerical stability of the transformation process, the softmax function is used to map the rotation axis vector LAxis = ( LA0 , LA1 , LA2 ) to obtain a new rotation axis vector LAxis ′:
Figure PCTCN2022126784-appb-000016
Figure PCTCN2022126784-appb-000016
其中sign(·)表示符号函数。考虑到实际情况下治疗前后牙齿的移动量不超过20mm，因此可使用tanh函数对平移向量 Lt＝( LA 4， LA 5， LA 6)进行映射，得到限幅的平移向量 Lt′。 Wherein sign(·) represents a sign function. Considering that the movement of teeth before and after treatment does not exceed 20 mm in actual situations, the tanh function can be used to map the translation vector L t = ( L A 4 , L A 5 , L A 6 ) to obtain a limited translation vector L t′.
对于表示变换，将旋转量和平移量变换为世界坐标系下的齐次变换矩阵，并计算牙齿预测位置。首先使用从so(3)到SO(3)的指数映射将转轴向量 LA axis′转换为旋转矩阵
Figure PCTCN2022126784-appb-000017
For the representation transformation, the rotation and translation are transformed into a homogeneous transformation matrix in the world coordinate system, and the predicted tooth position is calculated. First, the axis vector L A axis ′ is converted into a rotation matrix using the exponential mapping from so(3) to SO(3)
Figure PCTCN2022126784-appb-000017
LR＝I+sin(θ) LA axis+(1-cos(θ)) LA axis  (6) LR ＝I+sin(θ) LA axis +(1-cos(θ)) LA axis (6)
其中θ＝ LA 3，
Figure PCTCN2022126784-appb-000018
为转轴向量 LA axis′的反对称矩阵(Skew-symmetric matrix)。将旋转矩阵 LR和平移向量 Lt′组合为牙齿局部坐标系下的齐次变换矩阵
Figure PCTCN2022126784-appb-000019
Where θ = L A 3 ,
Figure PCTCN2022126784-appb-000018
The rotation matrix LR and the translation vector Lt ′ are combined into a homogeneous transformation matrix in the tooth local coordinate system.
Figure PCTCN2022126784-appb-000019
Figure PCTCN2022126784-appb-000020
Figure PCTCN2022126784-appb-000020
然后，将 LT转换为世界坐标系下的齐次变换矩阵 WT，对原始错颌牙 齿
Figure PCTCN2022126784-appb-000021
进行移动，得到自动排牙结果，即牙齿预测位置，记为
Figure PCTCN2022126784-appb-000022
Then, L T is converted into the homogeneous transformation matrix W T in the world coordinate system, and the original malocclusion teeth are
Figure PCTCN2022126784-appb-000021
Move the tooth to get the automatic tooth arrangement result, that is, the predicted position of the tooth, recorded as
Figure PCTCN2022126784-appb-000022
Figure PCTCN2022126784-appb-000023
Figure PCTCN2022126784-appb-000023
在上述特征解码与映射器中，通过使用softmax函数对旋转量进行约束，保证了输出移动量为刚体变换矩阵，且同时能够满足计算过程的数值稳定性。In the above-mentioned feature decoder and mapper, the rotation amount is constrained by using the softmax function, which ensures that the output movement amount is a rigid body transformation matrix and at the same time can meet the numerical stability of the calculation process.
S130，基于设定的损失函数训练深度学习模型。S130, training a deep learning model based on a set loss function.
例如，使用欧氏距离度量网络输出的牙齿预测位置和期望位置的对应点的间距设置损失函数，表示为：For example, the loss function is set using the Euclidean distance measurement network output of the tooth predicted position and the distance between the corresponding points of the expected position, expressed as:
Figure PCTCN2022126784-appb-000024
Figure PCTCN2022126784-appb-000024
其中，
Figure PCTCN2022126784-appb-000025
表示期望的牙齿点云位置，avg表示平均。 in,
Figure PCTCN2022126784-appb-000025
represents the expected tooth point cloud position, and avg represents the average.
应理解的是，除了直接将排列后牙齿点云预测位置和期望位置的欧氏距离偏差的平均值作为损失函数，也可以将欧氏距离偏差的最大值进行加权后单独作为损失函数的一项。或者使用预测变换矩阵和期望变换矩阵的余弦相似性误差作为损失函数。It should be understood that in addition to directly using the average value of the Euclidean distance deviation between the predicted position and the expected position of the tooth point cloud after arrangement as the loss function, the maximum value of the Euclidean distance deviation can also be weighted and used as a separate item of the loss function. Alternatively, the cosine similarity error between the predicted transformation matrix and the expected transformation matrix can be used as the loss function.
进一步地，使用训练集对构建的深度学习模型进行训练。在训练过程中使用Adam优化器，通过最小化损失函数值优化模型的参数，得到一系列参数不同的网络模型。进而利用验证集对训练后的模型进行验证，保存效果最优的网络模型作为最终的排牙网络模型，使用该排牙网络模型对未排牙的错颌畸形样本进行排牙操作。Furthermore, the constructed deep learning model was trained using the training set. During the training process, the Adam optimizer was used to optimize the model parameters by minimizing the loss function value, and a series of network models with different parameters were obtained. The trained model was then verified using the validation set, and the network model with the best effect was saved as the final tooth arrangement network model, which was used to perform tooth arrangement operations on malocclusion samples that had not been tooth arranged.
步骤S140，利用经训练的深度学习模型获取目标患者的排牙结果。Step S140, using the trained deep learning model to obtain the tooth arrangement result of the target patient.
在完成模型训练和验证后，即可将获得的最终排牙网络模型用于实际的排牙预测，即将目标患者的待排牙的牙齿三角网格三维模型输入到训练好的排牙网络模型，获得预测的牙齿移动量作为排牙结果。After completing the model training and verification, the final tooth arrangement network model can be used for actual tooth arrangement prediction, that is, the triangular mesh three-dimensional model of the target patient's teeth to be arranged is input into the trained tooth arrangement network model to obtain the predicted tooth movement as the tooth arrangement result.
需要说明的是，在不违背本发明精神和范围的前提下，本领域技术人员可对上述实施例进行适当的改变或变型。例如。在特征解码与映射器中，除使用softmax函数对旋转量进行约束外，也可以使用其他方法，以使用于表示旋转的三维转轴向量恒为单位向量，且在计算时满足数值稳定性。It should be noted that, without violating the spirit and scope of the present invention, those skilled in the art may make appropriate changes or modifications to the above embodiments. For example, in the feature decoder and mapper, in addition to using the softmax function to constrain the rotation amount, other methods may also be used to ensure that the three-dimensional axis vector used to represent the rotation is always a unit vector and meets numerical stability during calculation.
例如：For example:
Figure PCTCN2022126784-appb-000026
Figure PCTCN2022126784-appb-000026
其中 LA axis为局部坐标系下的转轴向量，k＝0，1，2，分别表示转轴向量的三维分量。 Wherein LAxis is the rotation axis vector in the local coordinate system, and k=0, 1, 2 respectively represent the three-dimensional components of the rotation axis vector.
相应地，本发明还提供一种基于网格特征深度学习的口腔正畸自动排牙***。该***包括：牙齿模型获取单元，其用于获取用于排牙的牙齿三角网格三维模型，所述牙齿三角网格三维模型利用三角面片的顶点、形心、中线、法向量和邻接面索引表征牙齿的形状信息和空间位置信息；预测单元，其用于将所述牙齿三角网格模型输入到预训练的深度学习模型，获得排牙结果的牙齿预测位置，其中所述深度学习模型包括牙列全局特征编码器、牙齿形状特征编码器以及特征解码与映射器，牙齿形状特征编码器以所述牙齿三角网格三维模型作为输入，获得牙齿形状特征，牙列全局特征编码器以牙列点云作为输入，获得牙列全局特征，所述特征解码与映射器基于所述牙齿形状特征和所述牙列全局特征获得预测的排牙结果。该***中的各单元可采用DSP、FPGA、处理器或专用硬件实现。Accordingly, the present invention also provides an automatic tooth arrangement system for orthodontics based on deep learning of mesh features. The system comprises: a tooth model acquisition unit, which is used to acquire a tooth triangular mesh three-dimensional model for tooth arrangement, wherein the tooth triangular mesh three-dimensional model uses the vertices, centroids, midlines, normal vectors and adjacent face indexes of triangular facets to characterize the shape information and spatial position information of the teeth; a prediction unit, which is used to input the tooth triangular mesh model into a pre-trained deep learning model to obtain the predicted tooth position of the tooth arrangement result, wherein the deep learning model comprises a dentition global feature encoder, a tooth shape feature encoder and a feature decoder and mapper, wherein the tooth shape feature encoder uses the tooth triangular mesh three-dimensional model as input to obtain tooth shape features, the dentition global feature encoder uses the dentition point cloud as input to obtain dentition global features, and the feature decoder and mapper obtains the predicted tooth arrangement result based on the tooth shape features and the dentition global features. Each unit in the system can be implemented using DSP, FPGA, processor or dedicated hardware.
为进一步验证本发明的效果，在相同数据集上，对本发明所提出的方法和现有基于深度学习的同类方法TANet进行了对比实验。结果表明，本发明的自动排牙方法能够从排牙前后的数据中学习到错颌牙列和理想牙列位姿的映射关系，所得自动排牙结果的准确度高于该同类方法。参见图4所示，其中图4(a)对应错颌牙列，图4(b)是现有技术TANet的排牙结果，图4(c)是本发明的排牙结果，图4(d)是期望牙列，图4中每一行代表一例样本。由图4可以看出，对于第一例样本，中切牙和侧切牙的位置符合期望，但TANet输出结果的前磨牙间隙偏大；对于第二例样本，右上颌前磨牙的拥挤状况均得到改善；对于第三例样本，排牙后牙列明显更为整齐，但与人工排牙得到的期望牙列相比，都出现了牙齿间隙偏小甚至穿模的情况。To further verify the effect of the present invention, a comparative experiment was conducted on the method proposed in the present invention and the existing similar method TANet based on deep learning on the same data set. The results show that the automatic tooth arrangement method of the present invention can learn the mapping relationship between the malocclusion dentition and the ideal dentition posture from the data before and after tooth arrangement, and the accuracy of the obtained automatic tooth arrangement result is higher than that of the similar method. See Figure 4, where Figure 4 (a) corresponds to the malocclusion dentition, Figure 4 (b) is the tooth arrangement result of the prior art TANet, Figure 4 (c) is the tooth arrangement result of the present invention, and Figure 4 (d) is the expected dentition, and each row in Figure 4 represents a sample. It can be seen from Figure 4 that for the first sample, the positions of the central incisor and the lateral incisor meet the expectations, but the premolar gap of the TANet output result is too large; for the second sample, the crowding of the right maxillary premolars is improved; for the third sample, the dentition is obviously more neat after tooth arrangement, but compared with the expected dentition obtained by manual tooth arrangement, the tooth gap is too small or even penetrates the mold.
本发明涉及的模型训练过程可在服务器或云端离线进行，将经训练的 模型嵌入到电子设备即可实现实时的自动排牙结果生成。该电子设备可以是终端设备或者服务器，终端设备包括手机、平板电脑、个人数字助理(PDA)、销售终端(POS)、车载电脑、智能可穿戴设备(智能手表、虚拟现实眼镜、虚拟现实头盔等)等任意终端设备。服务器包括但不限于应用服务器或Web服务器，可以为独立服务器、集群服务器或云服务器。本发明除应用于口腔正畸领域的自动排牙外，还可以推广到口腔修复学领域的义齿自动排列任务。The model training process involved in the present invention can be performed offline on a server or in the cloud, and the trained model can be embedded in an electronic device to achieve real-time automatic tooth arrangement result generation. The electronic device can be a terminal device or a server, and the terminal device includes any terminal device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sale (POS), a car computer, an intelligent wearable device (smart watch, virtual reality glasses, virtual reality helmet, etc.). The server includes but is not limited to an application server or a Web server, and can be an independent server, a cluster server or a cloud server. In addition to being applied to the automatic tooth arrangement in the field of orthodontics, the present invention can also be extended to the task of automatic denture arrangement in the field of oral prosthodontics.
综上所述，本发明设计了网格特征学习网络对牙齿的三角网格模型进行形状特征提取，在此基础上学习形状特征及全局空间特征与牙齿期望位置的潜在规律进行排牙。相对于现有技术，本发明具有以下优势：In summary, the present invention designs a mesh feature learning network to extract shape features from the triangular mesh model of teeth, and on this basis learns the shape features and the underlying rules of the global spatial features and the expected positions of teeth to arrange teeth. Compared with the prior art, the present invention has the following advantages:
1)现有基于深度学习的自动排牙方法输入为牙冠的口扫图像，仅包含牙冠信息，不包含牙龈内部的牙根和牙槽骨的信息，因此无法显式考虑牙根位姿，在临床使用中可能出现由于牙根过于贴近甚至移出牙槽骨，导致牙根吸收、骨开窗和骨开裂等情况，不符合临床要求。本发明考虑牙根位姿，将含有牙根的完整牙齿三维模型数据输入牙列全局特征编码器和牙齿形状特征编码器，并且损失函数也相应地对包含了牙根的排牙结果的距离误差进行计算，进而通过优化网络模型参数，实现了将牙根位姿引入自动排牙算法。1) The existing deep learning-based automatic tooth arrangement method uses scanned images of the crown as input, which only contains crown information, but not the root and alveolar bone information inside the gums. Therefore, it is impossible to explicitly consider the root posture. In clinical use, the root may be too close to or even move out of the alveolar bone, leading to root absorption, bone fenestration and bone cracking, which does not meet clinical requirements. The present invention considers the root posture, inputs the complete tooth 3D model data containing the root into the dentition global feature encoder and the tooth shape feature encoder, and the loss function also calculates the distance error of the tooth arrangement result containing the root accordingly, and then realizes the introduction of the root posture into the automatic tooth arrangement algorithm by optimizing the network model parameters.
2)现有基于深度学习的自动排牙方法中使用点云处理网络作为牙齿点云特征编码器，对于细粒度特征的提取能力有限。本发明使用网格处理网络提取牙齿形状特征，能更充分地从带有物体表面拓扑关系的三角网格数据中提取特征，并设计了基于网格特征学习的牙齿形状编码器，以提取牙齿三角网格模型的细粒度形状特征，从而提升深度学习网络自动排牙的准确度。2) The existing deep learning-based automatic tooth arrangement method uses a point cloud processing network as a tooth point cloud feature encoder, which has limited ability to extract fine-grained features. The present invention uses a mesh processing network to extract tooth shape features, which can more fully extract features from triangular mesh data with object surface topological relationships, and designs a tooth shape encoder based on mesh feature learning to extract fine-grained shape features of the tooth triangular mesh model, thereby improving the accuracy of deep learning network automatic tooth arrangement.
本发明可以是***、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本发明的各个方面的计算机可读程序指令。The present invention may be a system, a method and/or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是但不限于电存储设备、 磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。Computer readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A more specific example (non-exhaustive list) of computer readable storage medium includes: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove having instructions stored thereon, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as an instantaneous signal itself, such as a radio wave or other freely propagating electromagnetic waves, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing/processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and/or edge servers. The network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device.
用于执行本发明操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++、Python等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、 现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本发明的各个方面。The computer program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, Python, etc., and conventional procedural programming languages such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.
这里参照根据本发明实施例的方法、装置(***)和计算机程序产品的流程图和/或框图描述了本发明的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。Various aspects of the present invention are described herein with reference to the flow charts and/or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and/or block diagram and the combination of each box in the flow chart and/or block diagram can be implemented by computer-readable program instructions.
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions/actions specified in one or more boxes in the flowchart and/or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and/or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions/actions specified in one or more boxes in the flowchart and/or block diagram.
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions/actions specified in one or more boxes in the flowchart and/or block diagram.
附图中的流程图和框图显示了根据本发明的多个实施例的***、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的***来实现，或者可以用专用硬件与计算机指令的组合来实现。对于本领域技术人 员来说公知的是，通过硬件方式实现、通过软件方式实现以及通过软件和硬件结合的方式实现都是等价的。The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and/or flowchart, and the combination of the boxes in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。本发明的范围由所附权利要求来限定。Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims (10)

  1. 一种基于网格特征深度学习的口腔正畸自动排牙方法，包括以下步骤：An automatic tooth arrangement method for orthodontics based on deep learning of grid features comprises the following steps:
    获取用于排牙的牙齿三角网格三维模型，所述牙齿三角网格三维模型利用三角面片的顶点、形心、中线、法向量和邻接面索引表征牙齿的形状信息和空间位置信息；Acquire a tooth triangular mesh three-dimensional model for tooth arrangement, wherein the tooth triangular mesh three-dimensional model uses vertices, centroids, midlines, normal vectors and adjacent surface indices of triangular facets to represent shape information and spatial position information of the teeth;
    将所述牙齿三角网格模型输入到预训练的深度学习模型，获得排牙结果的牙齿预测位置；Inputting the tooth triangular mesh model into a pre-trained deep learning model to obtain the predicted tooth position of the tooth arrangement result;
    其中，所述深度学习模型包括第一特征编码器、第二特征编码器以及特征解码与映射器，第一特征编码器以所述牙齿三角网格三维模型作为输入，获得牙齿形状特征，第二特征编码器以牙列点云作为输入，获得牙列全局特征，所述特征解码与映射器基于所述牙齿形状特征和所述牙列全局特征获得预测的排牙结果。Among them, the deep learning model includes a first feature encoder, a second feature encoder and a feature decoder and mapper. The first feature encoder takes the tooth triangulated mesh three-dimensional model as input to obtain tooth shape features, and the second feature encoder takes the dentition point cloud as input to obtain dentition global features. The feature decoder and mapper obtains the predicted tooth arrangement results based on the tooth shape features and the dentition global features.
  2. 根据权利要求1所述的方法，其特征在于，所述牙齿三角网格三维模型根据以下步骤获得：The method according to claim 1, characterized in that the tooth triangulated mesh three-dimensional model is obtained according to the following steps:
    从目标的口内扫描图像和口腔图像中分割重构得到牙冠模型和完整的牙齿三维模型；The crown model and the complete three-dimensional tooth model are obtained by segmenting and reconstructing the target intraoral scan image and oral cavity image;
    对所述牙冠模型和所述牙齿三维模型的牙根部分进行配准和融合，得到用于排牙的所述牙齿三角网格三维模型。The crown model and the root part of the three-dimensional tooth model are registered and fused to obtain the three-dimensional tooth triangulated mesh model for tooth arrangement.
  3. 根据权利要求1所述的方法，其特征在于，第一特征编码器包括多个的并行连接的牙齿形状特征提取模块，每个牙齿形状特征提取模块与单颗牙齿一一对应，用于从单颗牙齿的数据中独立学习其形状表示。The method according to claim 1 is characterized in that the first feature encoder comprises a plurality of tooth shape feature extraction modules connected in parallel, each tooth shape feature extraction module corresponds one-to-one to a single tooth, and is used to independently learn the shape representation of a single tooth from its data.
  4. 根据权利要求3所述的方法，其特征在于，所述牙齿形状特征提取模块包括空间描述模块、结构描述模块和网格卷积模块，所述空间描述模块以牙齿的形心作为输入，用于将牙齿局部坐标系下的形心映射为局部空间特征并学习局部空间特征之间的变换；所述结构描述模块以牙齿的中线、法向量和邻接面索引作为输入，通过旋转卷积与核相关操作，获得结构特征；所述网格卷积模块通过融合所述局部空间特征和所述结构特征，获得所述牙齿形状特征。The method according to claim 3 is characterized in that the tooth shape feature extraction module includes a space description module, a structure description module and a grid convolution module, the space description module takes the centroid of the tooth as input, and is used to map the centroid in the local coordinate system of the tooth to a local space feature and learn the transformation between local space features; the structure description module takes the midline, normal vector and adjacent surface index of the tooth as input, and obtains the structure feature through rotational convolution and kernel-related operations; the grid convolution module obtains the tooth shape feature by fusing the local space feature and the structure feature.
  5. 根据权利要求4所述的方法，其特征在于，所述牙齿形状特征表示为：The method according to claim 4, characterized in that the tooth shape feature is expressed as:
    Figure PCTCN2022126784-appb-100001
    Figure PCTCN2022126784-appb-100001
    其中：in:
    Figure PCTCN2022126784-appb-100002
    Figure PCTCN2022126784-appb-100002
    Figure PCTCN2022126784-appb-100003
    Figure PCTCN2022126784-appb-100003
    其中，
    Figure PCTCN2022126784-appb-100004
    表示向量的拼接，W mc1为一维卷积，gather表示相邻三角面片的结构特征向量的聚合，W mc2为二维卷积，
    Figure PCTCN2022126784-appb-100005
    是空间描述模块的输出，和
    Figure PCTCN2022126784-appb-100006
    是结构描述模块的输出。     in,
    Figure PCTCN2022126784-appb-100004
    represents the concatenation of vectors, W mc1 is a one-dimensional convolution, gather represents the aggregation of structural feature vectors of adjacent triangular faces, and W mc2 is a two-dimensional convolution.
    Figure PCTCN2022126784-appb-100005
    is the output of the spatial description module, and
    Figure PCTCN2022126784-appb-100006
    is the output of the structure description module.
  6. 根据权利要求1所述的方法，其特征在于，所述特征解码与映射器执行：The method according to claim 1, characterized in that the feature decoder and mapper performs:
    将牙列全局特征和所有牙齿的形状特征进行拼接，并对拼接特征进行解码，得到待变换移动量；The global features of the dentition and the shape features of all teeth are spliced, and the spliced features are decoded to obtain the movement amount to be transformed;
    对于所述待变换移动量，使用softmax函数对相应的转轴向量进行映射，得到新的转轴向量；For the movement amount to be transformed, a softmax function is used to map the corresponding rotation axis vector to obtain a new rotation axis vector;
    将旋转量和平移量变换为世界坐标系下的齐次变换矩阵，并计算牙齿预测位置。The rotation and translation are transformed into a homogeneous transformation matrix in the world coordinate system, and the predicted tooth position is calculated.
  7. 根据权利要求1所述的方法，其特征在于，训练所述深度学习模型的损失函数表示为：The method according to claim 1, characterized in that the loss function for training the deep learning model is expressed as:
    Figure PCTCN2022126784-appb-100007
    Figure PCTCN2022126784-appb-100007
    其中，
    Figure PCTCN2022126784-appb-100008
    表示排列后牙齿点云预测位置，
    Figure PCTCN2022126784-appb-100009
    表示期望的牙齿点云位置，avg表示平均。     in,
    Figure PCTCN2022126784-appb-100008
    Indicates the predicted position of the tooth point cloud after arrangement,
    Figure PCTCN2022126784-appb-100009
    represents the expected tooth point cloud position, and avg represents the average.
  8. 根据权利要求1所述的方法，其特征在于，训练所述深度学习模型的训练集根据以下步骤获得：The method according to claim 1, characterized in that the training set for training the deep learning model is obtained according to the following steps:
    采集多个病例样本构建排牙数据集，该排牙数据集中的每条数据反映排牙的牙齿三角网格三维模型与期望的牙齿三角网格三维模型之间的对应关系；Collecting multiple case samples to construct a tooth arrangement data set, each data in the tooth arrangement data set reflects the corresponding relationship between the tooth triangular mesh three-dimensional model of the tooth arrangement and the expected tooth triangular mesh three-dimensional model;
    对所述排牙数据集进行数据增强，获得扩充的排牙数据集，所述数据增强包括：Performing data enhancement on the tooth arrangement data set to obtain an expanded tooth arrangement data set, wherein the data enhancement includes:
    随机移动所述排牙数据集中错颌牙列中的每颗牙齿，移动前后的错颌牙列对应同一期望牙列；Randomly moving each tooth in the malocclusion dentition in the tooth arrangement data set, so that the malocclusion dentition before and after the movement corresponds to the same desired dentition;
    对所述排牙数据集中所有错颌牙列和期望牙列进行同步的随机移动。All malocclusion dentition and desired dentition in the tooth arrangement data set are subjected to synchronous random movement.
  9. 一种基于网格特征深度学习的口腔正畸自动排牙***，包括：An orthodontic automatic tooth arrangement system based on deep learning of grid features, comprising:
    牙齿模型获取单元：用于获取用于排牙的牙齿三角网格三维模型，所述牙齿三角网格三维模型利用三角面片的顶点、形心、中线、法向量和邻接面索引表征牙齿的形状信息和空间位置信息；A tooth model acquisition unit is used to acquire a tooth triangular mesh three-dimensional model for tooth arrangement, wherein the tooth triangular mesh three-dimensional model uses vertices, centroids, midlines, normal vectors and adjacent surface indexes of triangular facets to represent shape information and spatial position information of the tooth;
    预测单元：用于将所述牙齿三角网格模型输入到预训练的深度学习模型，获得排牙结果的牙齿预测位置；Prediction unit: used for inputting the tooth triangular mesh model into the pre-trained deep learning model to obtain the predicted tooth position of the tooth arrangement result;
    其中，所述深度学习模型包括第一特征编码器、第二特征编码器以及特征解码与映射器，第一特征编码器以所述牙齿三角网格三维模型作为输入，获得牙齿形状特征，第二特征编码器以牙列点云作为输入，获得牙列全局特征，所述特征解码与映射器基于所述牙齿形状特征和所述牙列全局特征获得预测的排牙结果。Among them, the deep learning model includes a first feature encoder, a second feature encoder and a feature decoder and mapper. The first feature encoder takes the tooth triangulated mesh three-dimensional model as input to obtain tooth shape features, and the second feature encoder takes the dentition point cloud as input to obtain dentition global features. The feature decoder and mapper obtains the predicted tooth arrangement results based on the tooth shape features and the dentition global features.
  10. 一种计算机可读存储介质，其上存储有计算机程序，其中，该计算机程序被处理器执行时实现根据权利要求1至8中任一项所述方法的步骤。A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.
PCT/CN2022/126784 2022-10-21 2022-10-21 Orthodontic automatic tooth arrangement method and system based on mesh feature deep learning WO2024082284A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/126784 WO2024082284A1 (en) 2022-10-21 2022-10-21 Orthodontic automatic tooth arrangement method and system based on mesh feature deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/126784 WO2024082284A1 (en) 2022-10-21 2022-10-21 Orthodontic automatic tooth arrangement method and system based on mesh feature deep learning

Publications (1)

Publication Number Publication Date
WO2024082284A1 true WO2024082284A1 (en) 2024-04-25

Family

ID=90736679

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/126784 WO2024082284A1 (en) 2022-10-21 2022-10-21 Orthodontic automatic tooth arrangement method and system based on mesh feature deep learning

Country Status (1)

Country Link
WO (1) WO2024082284A1 (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW201248446A (en) * 2011-05-31 2012-12-01 Eped Inc Evaluation system for tooth shape teaching and training score
CN105488849A (en) * 2015-11-24 2016-04-13 嘉兴学院 Hybrid level set based three-dimensional tooth modeling method
CN108986111A (en) * 2018-07-02 2018-12-11 西安增材制造国家研究院有限公司 A kind of three-dimensional dentognathic model dividing method for area of computer aided stealth correction
WO2020218560A1 (en) * 2019-04-26 2020-10-29 株式会社カイ Tooth position analysis device, tooth region extraction model generation method, tooth position analysis method, program, and recording medium
CN112120810A (en) * 2020-09-29 2020-12-25 深圳市深图医学影像设备有限公司 Three-dimensional data generation method of tooth orthodontic concealed appliance
CN112690914A (en) * 2021-03-24 2021-04-23 汉斯夫(杭州)医学科技有限公司 Tooth and gum modeling method suitable for digital orthodontic application
CN113052902A (en) * 2020-12-29 2021-06-29 上海银马科技有限公司 Dental treatment monitoring method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW201248446A (en) * 2011-05-31 2012-12-01 Eped Inc Evaluation system for tooth shape teaching and training score
CN105488849A (en) * 2015-11-24 2016-04-13 嘉兴学院 Hybrid level set based three-dimensional tooth modeling method
CN108986111A (en) * 2018-07-02 2018-12-11 西安增材制造国家研究院有限公司 A kind of three-dimensional dentognathic model dividing method for area of computer aided stealth correction
WO2020218560A1 (en) * 2019-04-26 2020-10-29 株式会社カイ Tooth position analysis device, tooth region extraction model generation method, tooth position analysis method, program, and recording medium
CN112120810A (en) * 2020-09-29 2020-12-25 深圳市深图医学影像设备有限公司 Three-dimensional data generation method of tooth orthodontic concealed appliance
CN113052902A (en) * 2020-12-29 2021-06-29 上海银马科技有限公司 Dental treatment monitoring method
CN112690914A (en) * 2021-03-24 2021-04-23 汉斯夫(杭州)医学科技有限公司 Tooth and gum modeling method suitable for digital orthodontic application

Similar Documents

Publication Publication Date Title
US20220218449A1 (en) Dental cad automation using deep learning
US11232573B2 (en) Artificially intelligent systems to manage virtual dental models using dental images
US11735306B2 (en) Method, system and computer readable storage media for creating three-dimensional dental restorations from two dimensional sketches
US20210267730A1 (en) Dental cad automation using deep learning
JP7451406B2 (en) Automatic 3D root shape prediction using deep learning methods
US20210322136A1 (en) Automated orthodontic treatment planning using deep learning
US20210358604A1 (en) Interface For Generating Workflows Operating On Processing Dental Information From Artificial Intelligence
US20230196570A1 (en) Computer-implemented method and system for predicting orthodontic results based on landmark detection
CN111784754A (en) Tooth orthodontic method, device, equipment and storage medium based on computer vision
WO2023242757A1 (en) Geometry generation for dental restoration appliances, and the validation of that geometry
CN113554607A (en) Tooth body detection model, generation method and tooth body segmentation method
WO2024082284A1 (en) Orthodontic automatic tooth arrangement method and system based on mesh feature deep learning
TW202409874A (en) Dental restoration automation
WO2024119612A1 (en) Digital oral cavity model mark point recognition method and apparatus, and electronic device
CN115938544A (en) Automatic orthodontic tooth arrangement method and system based on grid feature deep learning
US20230087800A1 (en) Automated tooth administration in a dental restoration workflow
Li et al. A fine-grained orthodontics segmentation model for 3D intraoral scan data
US20240029380A1 (en) Integrated Dental Restoration Design Process and System
WO2024127311A1 (en) Machine learning models for dental restoration design generation
WO2024127308A1 (en) Classification of 3d oral care representations
CN118000934B (en) Dental arch curve generation method and device, electronic equipment and storage medium
TW202416910A (en) Integrated dental restoration design process and system
WO2024127304A1 (en) Transformers for final setups and intermediate staging in clear tray aligners
WO2024127313A1 (en) Metrics calculation and visualization in digital oral care
WO2024013282A1 (en) Method and system for tooth pose estimation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22962467

Country of ref document: EP

Kind code of ref document: A1