WO2024071943A1 - Artificial intelligence-based method for providing information on scoliosis - Google Patents

Artificial intelligence-based method for providing information on scoliosis Download PDF

Info

Publication number
WO2024071943A1
WO2024071943A1 PCT/KR2023/014735 KR2023014735W WO2024071943A1 WO 2024071943 A1 WO2024071943 A1 WO 2024071943A1 KR 2023014735 W KR2023014735 W KR 2023014735W WO 2024071943 A1 WO2024071943 A1 WO 2024071943A1
Authority
WO
WIPO (PCT)
Prior art keywords
scoliosis
confirmed
artificial intelligence
image
type
Prior art date
Application number
PCT/KR2023/014735
Other languages
French (fr)
Korean (ko)
Inventor
이정섭
이상학
박강현
고태식
Original Assignee
부산대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 부산대학교 산학협력단 filed Critical 부산대학교 산학협력단
Publication of WO2024071943A1 publication Critical patent/WO2024071943A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/50—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1071—Measuring physical dimensions, e.g. size of the entire body or parts thereof measuring angles, e.g. using goniometers
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1079—Measuring physical dimensions, e.g. size of the entire body or parts thereof using optical or photographic means
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/45—For evaluating or diagnosing the musculoskeletal system or teeth
    • A61B5/4538—Evaluating a particular part of the muscoloskeletal system or a particular medical condition
    • A61B5/4561—Evaluating static posture, e.g. undesirable back curvature
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

  • the present invention relates to convergence technology that combines the fields of medicine and artificial intelligence.
  • Scoliosis is a disease in which the spine is curved in three dimensions. The prevalence is reported to be 2-3%, and it is especially reported in children and adolescent girls aged 10 or older. If it occurs in children and adolescents, the spinal curve continues to progress along with growth, and in this process, the patient may end up living in a bent posture to make himself comfortable, which often worsens the condition. In hospitals, it is treated using braces or surgically. If the condition worsens, it may lead to death due to respiratory failure.
  • the patient only comes to the hospital after the scoliosis has progressed to the point where the curvature can be seen with the naked eye or pain is felt. Moreover, since there is a significant shortage of pediatric spine orthopedic specialists and specialized hospitals in Korea, the timing of visit to the hospital is further delayed. If you come to the hospital late, the treatment period will be longer, the effects will be delayed, and the process will be painful.
  • the person or their parents must have the ability and will to recognize it with the naked eye, or visit the hospital regularly and receive a diagnosis through professional medical images such as X-ray images or CT scan images. Even if you visit a hospital, if the professional medical staff lacks experience, it is not easy to diagnose scoliosis using only external images, so professional medical images such as X-ray images or CT imaging images are required. In this process, children/adolescents may be exposed to unnecessary radiation, and the burden of personal and national medical costs increases.
  • An artificial intelligence model can be considered as a technology that determines whether there is scoliosis by inputting only an external image.
  • medical-related image analysis is a field in which significant progress is being made.
  • Patents that use artificial intelligence to analyze images and provide medical-related information can be classified into two criteria.
  • the first classification standard is to distinguish between technology based on professional medical images and technology based on general external images (images taken with smartphones, etc.), and the second classification standard is technology that calculates information related to diseases other than scoliosis and technology that calculates information related to diseases other than scoliosis. It is classified using technology to calculate information related to scoliosis. Accordingly, patents in a total of four fields are divided, and patents are reviewed accordingly.
  • Korean Patent No. 10-2181701 discloses a technology for detecting certain diseases using images of nail wrinkle capillaries.
  • Korean Patent Publication No. 10-2020-0110878 discloses a technology for confirming the gum area and performing early diagnosis of cavities by using the specular reflection image of the oral cavity image.
  • U.S. Patent No. 10,468,142 discloses a technology for predicting disease using corneal photographs.
  • Korean Patent Publication No. 10-2022-0057793 discloses a technology for checking atopic dermatitis using images taken of the skin.
  • Korean Patent No. 10-2388337 discloses a technology for checking the accuracy of temporomandibular joint movement based on video footage of the jaw movement situation.
  • Korean Patent No. 10-2354980 discloses a technology for confirming the presence or absence of anterior segment disease based on images taken of the anterior segment.
  • Korean Patent No. 10-2274330 discloses a technology for determining whether a person has a stroke using an image of the face.
  • Korean Patent No. 10-2251925 discloses a technology for recording abnormal walking based on videos of walking situations.
  • Korean Patent No. 10-2214756 discloses a technology for checking the location and progress of cavities using oral imaging images.
  • 10-2047237 discloses a technology for predicting health status and likelihood of disease using images representing skin color.
  • U.S. Patent Publication No. 2022-0133215 discloses a technology for analyzing lesions by checking skin color in an image.
  • U.S. Patent Publication No. 2021-0345971 discloses a technology for diagnosing Lyme disease using digital images of the skin.
  • Korean Patent No. 10-2389067 discloses a technology that automatically determines the tilt of each vertebra using X-ray images, which are professional medical images, and determines whether there is scoliosis based on this.
  • Korean Patent No. 10-2383857 also discloses a technology for estimating Cobb angle and diagnosing scoliosis using X-ray images, which are professional medical images.
  • Japanese Patent No. 3234668 discloses a technology for recognizing scoliosis by segmenting the vertebral body, prevertebral body, and iliac regions in an X-ray image.
  • the above technologies commonly identify vertebrae individually and calculate each tilt or estimate the shape of each vertebrae itself. This method is highly accurate because it utilizes professional medical images, but it requires professional medical images such as X-ray photographs. Therefore, the patented technologies cannot be used before visiting the hospital, and the problem of lack of early diagnosis described above cannot be solved.
  • Japanese Patent Publication No. 2021-115471 discloses a technology for generating skeletal data based on images or videos taken of the subject's body, detecting skeletal deformation over time, diagnosing posture, and diagnosing diseases such as scoliosis.
  • This prior art is a technology that detects changes between images by securing multiple images with different shooting times (e.g., 1 week, etc.) with time set as a variable, so the information related to scoliosis at the time when only one image was captured is cannot be provided.
  • shooting times e.g., 1 week, etc.
  • Japanese Patent Publication No. 2020-040763 discloses a technology for estimating the scoliotic angle through an external image and its mirror image. Although this technology helps identify areas of asymmetry in images of a subject's back, the fact that an asymmetric back was imaged does not directly lead to scoliosis. For example, if the photograph is taken in a crooked position, there is a high possibility of being misdiagnosed as having scoliosis.
  • Japanese Patent No. 6280676 discloses a technology for estimating the spine alignment and calculating the Cobb angle, turning angle, etc., using Moire images after photographing the back.
  • moiré images there is a problem that only the extent to which the spine protrudes outside the skin is used as the basis for judgment. For example, if the spine does not protrude beyond the skin due to obesity or other reasons, the alignment of the spine is not correctly determined, and the portion of the spine that protrudes outside the skin is symmetrical, but scoliosis actually progresses, or vice versa. Since it is significant, diagnostic accuracy is greatly reduced and the possibility of misdiagnosis is high.
  • Korean Patent Publication No. 10-2021-0157684 discloses a technology for 3D modeling individual vertebrae using X-rays or MRI images, which are professional medical images.
  • Korean Patent No. 10-2062539 discloses a technology to model each lumbar vertebrae and check for lumbar disease using the center point and rotation angle.
  • Korean Patent No. 10-1968144 discloses a method of extracting professional medical images from PACS (medical image storage system), extracting images of the spine or cervical spine, and diagnosing the inclination angle through contour processing and inclination calculation of specific bones.
  • PACS medical image storage system
  • Patent Document 1 Korean Patent No. 10-2181701
  • Patent Document 2 Korean Patent Publication No. 10-2020-0110878
  • Patent Document 3 US Patent No. 10,468,142
  • Patent Document 4 Korean Patent Publication No. 10-2022-0057793
  • Patent Document 5 Korean Patent No. 10-2388337
  • Patent Document 6 Korean Patent No. 10-2354980
  • Patent Document 7 Korean Patent No. 10-2274330
  • Patent Document 8 Korean Patent No. 10-2251925
  • Patent Document 9 Korean Patent No. 10-2214756
  • Patent Document 10 Korean Patent No. 10-2047237
  • Patent Document 11 U.S. Patent Publication No. 2022-0133215
  • Patent Document 12 U.S. Patent Publication No. 2021-0345971
  • Patent Document 13 Korean Patent No. 10-2389067
  • Patent Document 14 Korean Patent No. 10-2383857
  • Patent Document 15 Japanese Patent No. 3234668
  • Patent Document 16 Japanese Patent Publication No. 2021-115471
  • Patent Document 17 Japanese Patent Publication No. 2020-040763
  • Patent Document 18 Japanese Patent No. 6280676
  • Patent Document 19 Korean Patent Publication No. 10-2021-0157684
  • Patent Document 20 Korean Patent No. 10-2062539
  • Patent Document 21 Korean Patent No. 10-1968144
  • the present invention was created to solve the above problems.
  • One embodiment of the present invention to solve the above problems includes the steps of: (a) collecting an external image in the image collection unit 110; -
  • the external image includes an image of the back, (b) inputting the collected external image to the artificial intelligence learning unit 100; (c) adding a guideline to the input external image and inputting it into the artificial intelligence learning unit 100; -
  • the guideline includes a line indicating the protrusion of one of the left and right sides of the back, the outline of both shoulders and the outline of both sides,
  • the professional diagnosis information includes any one type of scoliosis among normal and multiple types.
  • the artificial intelligence learning unit 100 uses the external image with the guideline added as input data. and generating an artificial intelligence model by learning a learning dataset using the professional diagnosis information as output data.
  • step (e) a step of inputting an appearance image and a guideline corresponding to the input appearance image into the artificial intelligence model generated in step (d) by the image input unit 210. ; (g) calculating scoliosis information indicating normal or one type of scoliosis among multiple types by the artificial intelligence model; and (h) outputting the calculated scoliosis information by the output unit 220. It is preferable to further include.
  • steps (f) to (h) are performed by a terminal, the terminal includes a camera, and the external image input in step (f) is an image captured by the camera of the terminal. desirable.
  • Step (i) transmitting the external appearance image input in step (f) and the scoliosis information calculated in step (g) to a preset hospital medical management system 300; and (j) confirming professional diagnosis information corresponding to the external image transmitted in step (i) in the hospital medical management system 300, wherein the terminal further includes a communication module, Step (i) is preferably a step in which the information is transmitted to the hospital medical management system 300 by the communication module of the terminal.
  • step (k) the hospital medical management system 300 uses the external image transmitted in step (i) and the professional diagnosis information confirmed in step (j) to the artificial intelligence learning unit ( 100) transmitting; and (l) the artificial intelligence learning unit 100 reinforcing the artificial intelligence model using the appearance image and professional diagnosis information transmitted in step (k). It is preferable to further include.
  • step (j) (m) generating a changed appearance image different from the appearance image input in step (f) by the camera of the terminal; (o) inputting the changed appearance image generated in step (m) by the image input unit 210 and the guideline corresponding to the input changed appearance image into the artificial intelligence model; (p) calculating altered scoliosis information indicating normal or one type of scoliosis among multiple types by the artificial intelligence model; and (q) outputting the calculated changed scoliosis information by the output unit 220.
  • scoliosis examples include: Type 1 - Thoracic scoliosis, Type 2 - Double Thoracic scoliosis, and Type 3 - Double Major-Thoracic/Lumbar.
  • it includes scoliosis, Type 4 - Triple Curve scoliosis, and Type 5 - Lumbar/Thoracolumbar scoliosis.
  • step (c) identifies the back portion of the human body in the input appearance image, confirms the left and right central axes in the identified back portion, and pixels of the back portion based on the left and right central axes in the input appearance image. Check the saturation difference, and if it is more than the preset saturation difference, the side with low saturation is judged to be a case of protrusion of the left and right sides of the back, and that part is marked with a line, and the upper outline of the back is used to mark the two shoulders. It is preferable to further include the step of marking the outline and marking the outline of both sides using the left and right outlines in the back.
  • one side of the back is protruding through the protruding lines on one of the left and right sides of the back, and the other shoulder is not raised through the outlines of both shoulders, and one side is retracted through the outlines of both sides. If this is not confirmed, it is confirmed as type 1 scoliosis, and one side of the back is protruded by the protrusion line on one of the left and right sides of the back, the other shoulder is raised by the outline of both shoulders, and both sides are confirmed to be raised.
  • one side is not confirmed through the outline, it is confirmed as type 2 scoliosis, one side of the back is protruded by the protrusion line on one of the left and right sides of the back, and the other shoulder is raised by the outline of both shoulders. If it is not confirmed and one side is confirmed through the outlines of both sides, type 3 scoliosis is confirmed, and one-sided back protrusion is confirmed by the protrusion lines on one of the left and right sides of the back, and the outlines of both shoulders are confirmed.
  • Another embodiment of the present invention to solve the above problems provides a computer program stored in a computer recording medium on which the above-described method is performed, or a computer recording medium in which the program is stored.
  • the accuracy is high.
  • a common problem with patents that calculate scoliosis-related information using external images was the possibility of misdiagnosis due to the limitations of each technique.
  • the present invention sets new guidelines and provides learning based on them. By performing this, the possibility of diagnosing scoliosis was increased. Accuracy is high because actual diagnostic information from professional medical staff is used during learning. Excellent accuracy was confirmed in the verification experiment described later.
  • diagnostic accuracy can be improved through data reinforcement, while patients can easily set visit schedules, accurately inform hospitals of their current situation, and treatment effects can be continuously monitored.
  • the problem with artificial intelligence technology using medical images is the lack of data containing diagnostic information by professional medical staff, and the present invention can fundamentally solve this problem.
  • the effect can be monitored by the patient and professional medical staff together, and in some cases, the need to change the orthosis or exercise method can be quickly confirmed and the treatment effect can be improved.
  • diagnosis results of professional medical staff are managed, they can be used to strengthen an artificial intelligence model.
  • Figure 1 discloses a system in which the method according to the invention is performed.
  • Figure 2 is a diagram for explaining scoliosis types in the method according to the present invention, and for each type, an actual external image and a professional medical image are disclosed together.
  • Figure 3 is a conceptual diagram illustrating elements for determining the type of scoliosis in the method according to the present invention.
  • Figure 4 is a flow chart for explaining the method according to the present invention.
  • Figure 5 is a conceptual diagram showing an example in which the method according to the present invention is applied mainly to schools and hospitals.
  • Figure 6 shows the results of verifying the importance of each guideline element after executing the method according to the present invention.
  • “appearance image” refers to an image of the patient's back observed with the naked eye through an image taken with a general camera (for example, a smartphone camera). In other words, it is an image, not a professional medical video.
  • professional medical images refers to images, such as X-ray images and CT images, that are accessible in hospitals and must be captured by professional medical staff.
  • professional diagnosis information refers to information determined by a professional medical staff as to whether the patient has scoliosis and, if so, what type of scoliosis it is.
  • guideline means a line that overlaps on the external image.
  • FIG. 2 it is shown as a circle and a straight red line.
  • scoliosis information is information calculated by an artificial intelligence model, and refers to information determined as to whether scoliosis is present and, if so, what type of scoliosis it is. That is, it includes normal scoliosis or any one type of scoliosis among multiple scoliosis types.
  • scoliosis type refers to classification according to the type of curve.
  • the present invention proposes five types of scoliosis. This is a type that can be classified only based on external images, but has been decided to be a classification that will help establish a quick and accurate professional diagnosis and treatment method when the information is delivered to professional medical staff.
  • the system for performing the method according to the present invention includes an artificial intelligence learning unit 100 and a scoliosis information calculation unit 200.
  • the artificial intelligence learning unit 100 collects data for learning, sets a learning dataset, and performs the function of creating an artificial intelligence model through learning.
  • the type of artificial intelligence used here is not limited.
  • the artificial intelligence learning unit 100 includes an image collection unit 110 that collects images, a guideline provision unit 120 that includes guidelines in the image, and professional diagnosis information that inputs professional diagnosis information from a professional medical staff into the image. Includes an input unit 130.
  • the image collection unit 110 collects multiple external images. Appearance images showing the presence and type of scoliosis must be collected by professional medical staff.
  • the guideline providing unit 120 manually or automatically includes guidelines in the external image. The specific method will be described later.
  • the professional diagnosis information input unit 130 performs a function of inputting professional diagnosis information, which is information about the presence and type of scoliosis diagnosed by a professional medical staff, to the external image for each collected external image.
  • the artificial intelligence learning unit 100 is data confirmed by the image collection unit 110, the guideline provision unit 120, and the professional diagnosis information input unit 130, and uses the external image with the guideline added as input data and the professional diagnosis information input unit 130. Set up a learning dataset with diagnostic information as output data and learn it to create an artificial intelligence model.
  • the scoliosis information calculation unit 200 includes a generated artificial intelligence model and includes an image input unit 210 and an output unit 220.
  • the external image to be input to the artificial intelligence model created through the image input unit 210 is confirmed.
  • the output unit 220 outputs scoliosis information in which the external image input by the image input unit 210 is the result of an artificial intelligence model.
  • the scoliosis information calculation unit 200 is preferably a program or application that can be executed on a terminal that the user can easily access.
  • the terminal is preferably a smartphone that includes a camera and a communication module. In this case, if the user's back is photographed using the smartphone's camera, it is recognized as an external image, and the image can be easily input into the artificial intelligence model by uploading it to the application, and the results can also be easily confirmed.
  • system for performing the method according to the present invention may be further linked to the hospital medical management system 300.
  • the external image and scoliosis information (i.e., presence and type of scoliosis) confirmed by the scoliosis information calculation unit 200 may be automatically transmitted to the hospital medical management system 300.
  • the hospital medical management system 300 identifies patients using a separate method, stores and uploads the relevant information in a digital chart, and makes it accessible to professional medical staff.
  • scoliosis information calculation unit 200 that is, the patient's smartphone, and is integrated and managed. It may be possible.
  • the patient continuously creates a modified appearance image by taking pictures of his/her back using a smartphone, and this information is entered into the scoliosis information calculation unit 200 to change the scoliosis. Information is generated, and the generated information is transmitted back to the hospital medical management system 300. Through this, patients can check the treatment progress themselves, and professional medical staff can also monitor it.
  • professional diagnosis information which is information about the diagnosis by professional medical staff, is generated, and the external image input to the image input unit 210 and generated by the hospital medical management system 300
  • the received professional diagnosis information can form an additional learning dataset, enabling reinforcement learning of the artificial intelligence model.
  • the present invention is a form that can be classified only by external image, and at the same time, when the information is delivered to professional medical staff, it helps in setting up a quick and accurate professional diagnosis and treatment method. It divides scoliosis into five types according to the curvature of the spine. We suggest a way to differentiate.
  • type 1 is thoracic scoliosis
  • type 2 is double thoracic scoliosis
  • type 3 is double major-thoracic/lumbar.
  • Scoliosis Type 4 is Triple Curve scoliosis
  • Type 5 is Lumbar/Thoracolumbar scoliosis.
  • This classification can be performed by professional medical staff, and is largely based on three factors as shown in FIG. 3. This includes (A) whether one side of the back (either left or right) sticks out, (B) whether the shoulder on the other side (i.e., the opposite side if the back sticks out) rises, and (C) whether the side of the side goes in. These may appear in combinations of one or more. Therefore, the scoliosis type is determined through the combination of these. A combination not shown (for example, a case where one side of the back sticks out and the other shoulder goes up but one side does not go in) can be understood clinically as an extremely exceptional case.
  • the external image includes guidelines for identifying them. Therefore, the guideline includes a line indicating the protrusion of one of the left and right sides of the back, the outlines of both shoulders, and the outlines of both sides.
  • An example of the guidelines can also be found in Figure 2.
  • the saturation difference between the pixels in the back area based on the left and right central axes in the external image. If the saturation difference is more than the preset, the side with low saturation is judged to be a case where one of the left and right sides of the back protrudes, and that part is marked with a line. Display. Additionally, the outline of both shoulders is marked using the upper outline of the back. Additionally, the outlines of both sides are marked using the left and right outlines on the back.
  • the part of the shoulder that is more raised is confirmed as shoulder rise, and if the difference in the degree of indentation toward the left and right central axes of the outlines of both sides is more than the preset inclination, the part that is more indented is confirmed as the side indentation. You can check by entering.
  • the criteria for determining these may not be specified and may vary depending on artificial intelligence learning using a learning dataset containing professional diagnosis information. It may vary depending on reinforcement learning, which will be described later. In either case, there will be a change in the direction of increasing the accuracy of professional diagnosis information.
  • the protrusion of one side of the back is confirmed by the protrusion line on one of the left and right sides of the back, the raising of the other shoulder is not confirmed through the outlines of both shoulders, and the retraction of one side is confirmed through the outlines of both sides. If this is not the case, it is confirmed as type 1 scoliosis, and one-sided back protrusion is confirmed by the protruding line on one of the left and right sides of the back, the other shoulder is raised through the outline of both shoulders, and one side is confirmed by the outline of both sides.
  • the retraction is not confirmed, it is confirmed as type 2 scoliosis, and the protrusion of one side of the back is confirmed by the protrusion line on one of the left and right sides of the back, and the protrusion of the other shoulder is not confirmed by the outline of both shoulders and the outline of both sides. If one side is confirmed to be indented, it is confirmed as type 3 scoliosis, one side of the back is protruded by the protruding line on one of the left and right sides of the back, the other shoulder is raised through the outlines of both shoulders, and both sides are confirmed to be raised.
  • Type 4 scoliosis is confirmed when one side is confirmed through the outline of the back, and when one side of the back is not protruded by the protrusion line on one side of the left or right side of the back, the other shoulder is raised through the outline of both shoulders. If it is not confirmed and unilateral indentation is confirmed through the outlines of both sides, it can be confirmed as type 5 scoliosis.
  • the guidelines included in the image are automatically recognized and used as one of the input data.
  • the external image collected in the image collection unit 110 is input to the artificial intelligence learning unit 100, and a guideline is added to the input external image and input to the artificial intelligence learning unit 100. Additionally, professional diagnosis information corresponding to the input external image is further input.
  • the artificial intelligence learning unit 100 generates an artificial intelligence model by learning a learning dataset using the appearance image with the guideline added as input data and the professional diagnosis information as output data (S110).
  • an external image is input to the artificial intelligence model by the image input unit 210 (S210), and when a guideline corresponding to the input external image is input together (S220), the artificial intelligence model determines whether the image is normal or one of multiple types.
  • Scoliosis information indicating one type of scoliosis is calculated and output by the output unit 220 (S230).
  • the hospital that developed the present invention can introduce the present invention to schools or related government departments [A1], and it can be introduced to students or parents through schools, etc. [A2].
  • Diagnosed information can be transmitted to the hospital.
  • the external image input by the user and the scoliosis information calculated by the artificial intelligence model are transmitted to the preset hospital medical management system 300 (S310) [C1]. This transmission can be easily accomplished through the communication module of the smartphone.
  • the hospital can check this and set a visit schedule [C2], and when the patient visits the hospital and receives a direct diagnosis by a professional medical staff, professional diagnosis information is generated and uploaded to the hospital medical management system 300 [C3]. Therefore, the hospital medical management system 300 confirms the external image and the corresponding professional diagnosis information, sets it as an additional learning dataset, and transmits it to the artificial intelligence learning unit 100. Accordingly, the artificial intelligence model is reinforced learning (S320)[C4].
  • the patient i.e., the user of the present invention
  • a terminal such as a smartphone
  • This is input into the artificial intelligence model through a program or application installed on the terminal, and guidelines are also input.
  • information on the changed scoliosis according to the changed external image, that is, scoliosis of either normal or multiple types is confirmed (S410) [C5]. If the patient confirms this, he or she can self-diagnose the progress of the treatment, and it is transmitted to the hospital medical management system (300) to be confirmed by the medical staff, allowing the patient to track the progress of the treatment and, if necessary, change the brace or exercise method or set a visit schedule again. You can.
  • the applicant entered and executed some data into the program and performed a verification experiment comparing the data with the professional diagnosis results of professional medical staff on patients representing the data, and found that the accuracy was about 80%. Appearance was confirmed.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Molecular Biology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Dentistry (AREA)
  • Theoretical Computer Science (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Evolutionary Computation (AREA)
  • Physical Education & Sports Medicine (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Rheumatology (AREA)
  • Orthopedic Medicine & Surgery (AREA)
  • Image Analysis (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

The present invention relates to a method for creating an artificial intelligence model and a method for providing information using same, wherein users upload images of their backs taken with smartphones or similar devices to a program or application installed on the smartphone to obtain scoliosis-related information.

Description

인공지능 기반의 척추측만증 정보 제공 방법How to provide scoliosis information based on artificial intelligence
본 발명은 의학 분야와 인공지능 분야가 융합된 융합 기술에 관한 것이다.The present invention relates to convergence technology that combines the fields of medicine and artificial intelligence.
척추측만증(scoliosis)은 척추가 3차원으로 휘어지는 병이다. 유병율은 2~3%로 보고되며, 10세 이후인 소아/청소년기 여아에게서 특히 많이 보고된다. 소아 청소년기에 발병하면 성장과 함께 척추 휘어짐이 계속 진행되며, 이 과정에서 환아 스스로가 몸을 편하게 하고자 구부정한 자세로 생활하게 될 수 있어 악화되는 경우가 많다. 병원에서는 보조기를 통해 치료하거나 수술적으로 치료한다. 정도가 심해지는 경우 호흡 부전으로 인해 사망에 이를 수도 있다. Scoliosis is a disease in which the spine is curved in three dimensions. The prevalence is reported to be 2-3%, and it is especially reported in children and adolescent girls aged 10 or older. If it occurs in children and adolescents, the spinal curve continues to progress along with growth, and in this process, the patient may end up living in a bent posture to make himself comfortable, which often worsens the condition. In hospitals, it is treated using braces or surgically. If the condition worsens, it may lead to death due to respiratory failure.
많은 경우 육안으로 휘어짐이 확인될 수 있을 정도 또는 고통이 느껴질 정도로 척추측만증이 상당히 진행된 후 비로소 환아가 내원하게 된다. 더욱이 국내 소아 척추 정형외과 전문의와 전문병원이 상당히 부족한 실정이기에, 내원 시점은 더욱 크게 늦어진다. 늦은 시점에 병원에 오게 되면 치료 기간이 길어져 효과가 늦게 나타나며 그 과정도 고통스럽다. In many cases, the patient only comes to the hospital after the scoliosis has progressed to the point where the curvature can be seen with the naked eye or pain is felt. Moreover, since there is a significant shortage of pediatric spine orthopedic specialists and specialized hospitals in Korea, the timing of visit to the hospital is further delayed. If you come to the hospital late, the treatment period will be longer, the effects will be delayed, and the process will be painful.
따라서, 소아/청소년기에 발생하기 시작하는 척추측만증의 조기 진단이 매우 중요하다. 변형이 덜 심한 상태에서 진료에 착수하여야 치료 효과가 상승하고 환자 만족도가 개선된다. 그러나 현재의 소아/청소년을 대상으로 의무적 또는 정기적인 척추측만증 조기 선별 검사는 별도로 진행되지 않는 실정이다. Therefore, early diagnosis of scoliosis, which begins to occur in childhood/adolescence, is very important. Treatment effectiveness increases and patient satisfaction improves when treatment is started with less severe deformation. However, there is currently no separate mandatory or regular scoliosis early screening test for children/adolescents.
결국, 조기 진단을 위해서는 본인 또는 부모가 육안으로 인지할 수 있을 정도의 능력과 의지를 갖거나, 병원에 정기적으로 방문하여 X레이 영상이나 CT촬영 영상과 같은 전문의료영상을 통해 진단을 받아야 한다. 병원에 방문하여도 전문의료진의 경험이 부족하다면 외견이미지만 이용하여 척추측만증 여부를 진단하는 것은 쉽지 않기에 X레이 영상이나 CT촬영 영상과 같은 전문의료영상을 필요로 한다. 이 과정에서 소아/청소년에게 불필요한 피폭이 이루어질 수 있으며, 개인적인 의료비용과 국가적인 의료비용 부담이 증가한다.Ultimately, for early diagnosis, the person or their parents must have the ability and will to recognize it with the naked eye, or visit the hospital regularly and receive a diagnosis through professional medical images such as X-ray images or CT scan images. Even if you visit a hospital, if the professional medical staff lacks experience, it is not easy to diagnose scoliosis using only external images, so professional medical images such as X-ray images or CT imaging images are required. In this process, children/adolescents may be exposed to unnecessary radiation, and the burden of personal and national medical costs increases.
이에, 비전문 의료인인 소아/청소년 본인 또는 그 부모가, 스마트폰 카메라와 같은 일반적 카메라로 촬영한 외견이미지만 이용하여, 척추측만증이 의심되는지 여부를 높은 정확도로 확인할 수 있는 기술이 필요한 실정이다. 이러한 기술이 개발된다면, 조기 진단은 물론 치료 과정에서 경과 관찰에도 큰 도움이 될 수 있다. Accordingly, there is a need for technology that allows children/adolescents or their parents, who are non-professional medical professionals, to confirm with high accuracy whether scoliosis is suspected using only external images taken with a general camera such as a smartphone camera. If such technology is developed, it could be of great help not only in early diagnosis but also in monitoring progress during treatment.
외견이미지만 입력하면 척추측만증 여부가 판단되는 기술로서, 인공지능 모델을 고려할 수 있다. 실제, 인공지능이 적용되는 기술분야 중 의료 관련 영상 분석 분야는 상당한 진척을 이루는 분야이다. An artificial intelligence model can be considered as a technology that determines whether there is scoliosis by inputting only an external image. In fact, among the technological fields where artificial intelligence is applied, medical-related image analysis is a field in which significant progress is being made.
관련된 특허문헌을 검토한다. Review related patent documents.
인공지능을 이용하여 영상을 분석하고 의료 관련 정보를 제공하는 특허는 두 개의 기준으로 분류할 수 있다. 첫번째 분류 기준은 전문의료영상을 기초로 하는 기술과 일반적 외견이미지(스마트폰 등으로 촬영한 이미지)를 기초로 하는 기술로 구분하는 것이고, 두번째 분류 기준은 척추측만증 외의 질환 관련 정보를 연산하는 기술과 척추측만증과 관련된 정보를 연산하는 기술로 구분하는 것이다. 이에 따라 총 4개 분야의 특허가 구분되는바, 이에 따라 특허를 구분하여 검토한다. Patents that use artificial intelligence to analyze images and provide medical-related information can be classified into two criteria. The first classification standard is to distinguish between technology based on professional medical images and technology based on general external images (images taken with smartphones, etc.), and the second classification standard is technology that calculates information related to diseases other than scoliosis and technology that calculates information related to diseases other than scoliosis. It is classified using technology to calculate information related to scoliosis. Accordingly, patents in a total of four fields are divided, and patents are reviewed accordingly.
첫째, 전문의료영상을 기초로 척추측만증 외의 질환 관련 정보를 연산하는 특허이다.First, it is a patent that calculates information related to diseases other than scoliosis based on professional medical images.
한국등록특허 제10-2181701호는 손톱주름 모세혈관의 이미지를 활용하여 소정의 질병을 판독하는 기술을 개시한다. 한국공개특허 제10-2020-0110878호는 구강 이미지의 경면 반사 이미지를 활용하여 잇몸 영역을 확인하고 충치 조기진단을 시행하는 기술을 개시한다. 미국등록특허 제10,468,142호는 각막 사진을 이용하여 질병을 예측하는 기술을 개시한다.Korean Patent No. 10-2181701 discloses a technology for detecting certain diseases using images of nail wrinkle capillaries. Korean Patent Publication No. 10-2020-0110878 discloses a technology for confirming the gum area and performing early diagnosis of cavities by using the specular reflection image of the oral cavity image. U.S. Patent No. 10,468,142 discloses a technology for predicting disease using corneal photographs.
둘째, 일반적인 외견이미지를 기초로 척추측만증 외의 질환 관련 정보를 연산하는 특허이다. Second, it is a patent that calculates information related to diseases other than scoliosis based on general external images.
한국공개특허 제10-2022-0057793호는 피부를 촬영한 이미지를 이용하여 아토피 피부염 여부를 확인하는 기술을 개시한다. 한국등록특허 제10-2388337호는 턱운동 상황을 촬영한 동영상을 기반으로 턱관절 운동의 정확성 여부를 확인하는 기술을 개시한다. 한국등록특허 제10-2354980호는 전안부를 촬영한 이미지를 기초로 전안부 질환 발생 유무를 확인하는 기술을 개시한다. 한국등록특허 제10-2274330호는 얼굴에 대한 이미지를 이용하여 뇌졸증 여부를 판단하는 기술을 개시한다. 한국등록특허 제10-2251925호는 보행 상황을 촬영한 동영상을 기반으로 이상보행여부를 촬영하는 기술을 개시한다. 한국등록특허 제10-2214756호는 구강 촬영 영상을 이용하여 충치 위치 및 진행 정도를 확인하는 기술을 개시한다. 한국등록특허 제10-2047237호는 피부 색상을 나타내는 이미지를 이용하여 건강 상태 및 질병 가능성을 예측하는 기술을 개시한다. 미국공개특허 제2022-0133215호는 이미지에서 피부 색상을 확인하여 병변을 분석하는 기술을 개시한다. 미국공개특허 제2021-0345971호는 피부에 대한 디지털 이미지를 이용하여 라임병을 진단하는 기술을 개시한다. Korean Patent Publication No. 10-2022-0057793 discloses a technology for checking atopic dermatitis using images taken of the skin. Korean Patent No. 10-2388337 discloses a technology for checking the accuracy of temporomandibular joint movement based on video footage of the jaw movement situation. Korean Patent No. 10-2354980 discloses a technology for confirming the presence or absence of anterior segment disease based on images taken of the anterior segment. Korean Patent No. 10-2274330 discloses a technology for determining whether a person has a stroke using an image of the face. Korean Patent No. 10-2251925 discloses a technology for recording abnormal walking based on videos of walking situations. Korean Patent No. 10-2214756 discloses a technology for checking the location and progress of cavities using oral imaging images. Korean Patent No. 10-2047237 discloses a technology for predicting health status and likelihood of disease using images representing skin color. U.S. Patent Publication No. 2022-0133215 discloses a technology for analyzing lesions by checking skin color in an image. U.S. Patent Publication No. 2021-0345971 discloses a technology for diagnosing Lyme disease using digital images of the skin.
셋째, 전문의료영상을 기초로 척추측만증과 관련된 정보를 연산하는 특허이다.Third, it is a patent that calculates information related to scoliosis based on professional medical images.
한국등록특허 제10-2389067호는 전문 의료영상인 X레이 이미지를 이용하여 척추뼈 각각의 기울기를 자동으로 결정하고 이를 기반으로 척추측만증 여부를 판단하는 기술을 개시한다. 한국등록특허 제10-2383857호 역시 전문 의료영상인 X레이 이미지를 이용하여 콥 각도를 추정하고 척추측만증을 진단하는 기술을 개시한다. 일본등록특허 제3234668호는 X레이 이미지에서 추체, 선추, 장골 영역을 세그먼트하여 척추측만증을 인지하는 기술을 개시한다. 위 기술들은 공통적으로 척추뼈를 개별적으로 식별하고 각각의 기울기를 연산하거나 각각의 형상 자체를 추정한다. 이러한 방식은 전문의료영상을 활용하기에 정확성이 높지만, X레이 사진과 같은 전문의료영상을 반드시 필요로 한다. 따라서, 내원하기 전에는 본 특허기술들을 활용할 수 없어서, 전술한 조기 진단 부재의 문제점을 해결할 수 없다. Korean Patent No. 10-2389067 discloses a technology that automatically determines the tilt of each vertebra using X-ray images, which are professional medical images, and determines whether there is scoliosis based on this. Korean Patent No. 10-2383857 also discloses a technology for estimating Cobb angle and diagnosing scoliosis using X-ray images, which are professional medical images. Japanese Patent No. 3234668 discloses a technology for recognizing scoliosis by segmenting the vertebral body, prevertebral body, and iliac regions in an X-ray image. The above technologies commonly identify vertebrae individually and calculate each tilt or estimate the shape of each vertebrae itself. This method is highly accurate because it utilizes professional medical images, but it requires professional medical images such as X-ray photographs. Therefore, the patented technologies cannot be used before visiting the hospital, and the problem of lack of early diagnosis described above cannot be solved.
넷째, 일반적인 외견이미지를 기초로 척추측만증과 관련된 정보를 연산하는 특허이다. Fourth, this is a patent that calculates information related to scoliosis based on general external images.
일본공개특허 제2021-115471호는 피측정자의 신체를 촬영한 이미지 내지 동영상을 기초로 골격 데이터를 생성하고, 시간에 따른 골격 변형을 검출하여 자세를 진단하고 척추측만증 등의 질환을 진단하는 기술을 개시한다. 본 종래기술은 시간이 변수로 설정되어 촬영시간이 다른(예컨대, 1주일 등) 다수의 이미지를 확보하여 이미지 사이의 변화를 검출하는 기술이기에 이미지를 1장만 촬영한 현재 시점의 척추측만증 관련 정보는 제공할 수 없다. 또한, 골격 데이터를 추정하는 단계가 포함되기에 촬영 자세가 조금만 바뀌어도 척추측만증이라고 오진단할 가능성이 높다. Japanese Patent Publication No. 2021-115471 discloses a technology for generating skeletal data based on images or videos taken of the subject's body, detecting skeletal deformation over time, diagnosing posture, and diagnosing diseases such as scoliosis. Begin. This prior art is a technology that detects changes between images by securing multiple images with different shooting times (e.g., 1 week, etc.) with time set as a variable, so the information related to scoliosis at the time when only one image was captured is cannot be provided In addition, since a step of estimating skeletal data is included, there is a high possibility of misdiagnosing scoliosis even if the shooting posture changes slightly.
일본공개특허 제2020-040763호는 외견이미지와 그 거울역상 이미지를 통해 측만각을 추정하는 기술을 개시한다. 본 기술은 피험자의 등 이미지에서 비대칭 영역을 확인하는 것에 도움을 주지만 비대칭 등이 촬영되었다는 사실이 바로 척추측만증으로 연결되지 않는다. 예컨대, 비뚤어진 자세로 촬영한다면 척추측만증이 있는 것으로 오진단할 가능성이 높다. Japanese Patent Publication No. 2020-040763 discloses a technology for estimating the scoliotic angle through an external image and its mirror image. Although this technology helps identify areas of asymmetry in images of a subject's back, the fact that an asymmetric back was imaged does not directly lead to scoliosis. For example, if the photograph is taken in a crooked position, there is a high possibility of being misdiagnosed as having scoliosis.
일본등록특허 제6280676호는 등을 촬영한 후 모아레(Moire) 이미지를 이용하여 척추 배열을 추정하고 코브각, 선회각 등을 산출하는 기술을 개시한다. 모아레 이미지를 활용하기에 척추가 피부 외부로 돌출된 정도만을 판단 근거로 활용한다는 문제점이 있다. 예컨대, 비만 등의 이유로 척추가 피부 외부로 돌출된 정도가 부족하다면 척추의 배열이 정확하게 정되지 않으며, 척추가 피부 외부로 돌출된 부분은 대칭이지만 실제로는 척추측만증이 진행되는 경우 또는 그 반대의 경우도 상당하므로 진단 정확도가 크게 떨어지고 오진단 가능성이 높다.Japanese Patent No. 6280676 discloses a technology for estimating the spine alignment and calculating the Cobb angle, turning angle, etc., using Moire images after photographing the back. When using moiré images, there is a problem that only the extent to which the spine protrudes outside the skin is used as the basis for judgment. For example, if the spine does not protrude beyond the skin due to obesity or other reasons, the alignment of the spine is not correctly determined, and the portion of the spine that protrudes outside the skin is symmetrical, but scoliosis actually progresses, or vice versa. Since it is significant, diagnostic accuracy is greatly reduced and the possibility of misdiagnosis is high.
이와 같이, 외견이미지를 이용하여 척추측만증과 관련된 정보를 확인하는 특허가 존재하며 다양한 기법을 적용하고 있지만, 공통적으로 정확성이 낮고 오진단의 가능성이 높다는 문제점을 갖고 있다. Likewise, there are patents that use external images to confirm information related to scoliosis and various techniques are applied, but they have the common problem of low accuracy and high possibility of misdiagnosis.
위 분류에 포함되지 않지만 참조할 수 있는 관련 특허들은 다음과 같다. Related patents that are not included in the above classification but can be referenced are as follows.
한국공개특허 제10-2021-0157684호는 전문 의료영상인 X레이 내지 MRI 사진을 이용하여 개별적인 척추뼈를 3D 모델링하는 기술을 개시한다. 한국등록특허 제10-2062539호는 각각의 요추를 모델링하고 중심점과 회전 각도를 이용하여 요추 질환 여부를 확인하는 기술을 개시한다. 한국등록특허 제10-1968144호는 PACS(의료영상저장시스템)에서 전문 의료영상을 추출한 후 척추 내지 경추 이미지를 추출하고, 특정 뼈의 윤곽선 처리 및 경사도 산출을 통해 경사각을 진단하는 방법을 개시한다. Korean Patent Publication No. 10-2021-0157684 discloses a technology for 3D modeling individual vertebrae using X-rays or MRI images, which are professional medical images. Korean Patent No. 10-2062539 discloses a technology to model each lumbar vertebrae and check for lumbar disease using the center point and rotation angle. Korean Patent No. 10-1968144 discloses a method of extracting professional medical images from PACS (medical image storage system), extracting images of the spine or cervical spine, and diagnosing the inclination angle through contour processing and inclination calculation of specific bones.
(선행기술문헌)(Prior art literature)
(특허문헌 1) 한국등록특허 제10-2181701호(Patent Document 1) Korean Patent No. 10-2181701
(특허문헌 2) 한국공개특허 제10-2020-0110878호(Patent Document 2) Korean Patent Publication No. 10-2020-0110878
(특허문헌 3) 미국등록특허 제10,468,142호 (Patent Document 3) US Patent No. 10,468,142
(특허문헌 4) 한국공개특허 제10-2022-0057793호(Patent Document 4) Korean Patent Publication No. 10-2022-0057793
(특허문헌 5) 한국등록특허 제10-2388337호(Patent Document 5) Korean Patent No. 10-2388337
(특허문헌 6) 한국등록특허 제10-2354980호(Patent Document 6) Korean Patent No. 10-2354980
(특허문헌 7) 한국등록특허 제10-2274330호(Patent Document 7) Korean Patent No. 10-2274330
(특허문헌 8) 한국등록특허 제10-2251925호(Patent Document 8) Korean Patent No. 10-2251925
(특허문헌 9) 한국등록특허 제10-2214756호(Patent Document 9) Korean Patent No. 10-2214756
(특허문헌 10) 한국등록특허 제10-2047237호(Patent Document 10) Korean Patent No. 10-2047237
(특허문헌 11) 미국공개특허 제2022-0133215호(Patent Document 11) U.S. Patent Publication No. 2022-0133215
(특허문헌 12) 미국공개특허 제2021-0345971호(Patent Document 12) U.S. Patent Publication No. 2021-0345971
(특허문헌 13) 한국등록특허 제10-2389067호(Patent Document 13) Korean Patent No. 10-2389067
(특허문헌 14) 한국등록특허 제10-2383857호(Patent Document 14) Korean Patent No. 10-2383857
(특허문헌 15) 일본등록특허 제3234668호(Patent Document 15) Japanese Patent No. 3234668
(특허문헌 16) 일본공개특허 제2021-115471호(Patent Document 16) Japanese Patent Publication No. 2021-115471
(특허문헌 17) 일본공개특허 제2020-040763호(Patent Document 17) Japanese Patent Publication No. 2020-040763
(특허문헌 18) 일본등록특허 제6280676호(Patent Document 18) Japanese Patent No. 6280676
(특허문헌 19) 한국공개특허 제10-2021-0157684호(Patent Document 19) Korean Patent Publication No. 10-2021-0157684
(특허문헌 20) 한국등록특허 제10-2062539호(Patent Document 20) Korean Patent No. 10-2062539
(특허문헌 21) 한국등록특허 제10-1968144호 (Patent Document 21) Korean Patent No. 10-1968144
본 발명은 상기와 같은 문제점을 해결하기 위하여 안출된 것이다. The present invention was created to solve the above problems.
스마트폰 등을 활용하여 누구라도 간단하게 촬영할 수 있는 외견이미지만으로, 전문의료영상 없이도, 척추측만증 여부를 확인할 수 있는 기술을 제안하고자 한다. 특히, 높은 정확도로 척추측만증 여부를 확인할 수 있으며, 구체적으로 척추측만증 타입(즉, 만곡의 형태)인지 여부까지 확인 가능한 기술을 제안하고자 한다. We would like to propose a technology that can confirm the presence of scoliosis without professional medical imaging using only an external image that anyone can easily capture using a smartphone, etc. In particular, we would like to propose a technology that can confirm the presence of scoliosis with high accuracy and, specifically, whether it is the type of scoliosis (i.e., the shape of the curve).
상기와 같은 과제를 해결하기 위한 본 발명의 일 실시예는, (a) 이미지 수집부(110)에서 외견이미지가 수집되는 단계; - 여기서, 외견이미지는 등의 이미지를 포함함, (b) 수집된 외견이미지가 인공지능 학습부(100)에 입력되는 단계; (c) 상기 입력된 외견이미지 상에 가이드라인이 추가되어 상기 인공지능 학습부(100)에 입력되는 단계; - 여기서, 상기 가이드라인은 등의 좌우측 중 일측의 튀어나옴이 있는 경우 이를 표시하는 선, 양측 어깨의 윤곽선 및 양측 옆구리의 윤곽선을 포함함, (d) 상기 입력된 외견이미지에 대응되는 전문진단정보가 상기 인공지능 학습부(100)에 입력되는 단계; 및 - 여기서, 상기 전문진단정보는, 정상 및 다수의 타입 중 어느 하나의 타입의 척추측만증임을 포함함, (e) 상기 인공지능 학습부(100)가 상기 가이드라인이 추가된 외견이미지를 입력데이터로 하고 상기 전문진단정보를 출력데이터로 한 학습데이터세트를 학습하여 인공지능 모델을 생성하는 단계;를 포함하는 방법을 제공한다. One embodiment of the present invention to solve the above problems includes the steps of: (a) collecting an external image in the image collection unit 110; - Here, the external image includes an image of the back, (b) inputting the collected external image to the artificial intelligence learning unit 100; (c) adding a guideline to the input external image and inputting it into the artificial intelligence learning unit 100; - Here, the guideline includes a line indicating the protrusion of one of the left and right sides of the back, the outline of both shoulders and the outline of both sides, (d) Professional diagnosis information corresponding to the input external image. inputting into the artificial intelligence learning unit 100; and - Here, the professional diagnosis information includes any one type of scoliosis among normal and multiple types. (e) The artificial intelligence learning unit 100 uses the external image with the guideline added as input data. and generating an artificial intelligence model by learning a learning dataset using the professional diagnosis information as output data.
또한, 상기 (e) 단계 이후, (f) 이미지 입력부(210)에 의해 상기 (d) 단계에서 생성된 인공지능 모델에 외견이미지와, 상기 입력된 외견이미지에 해당하는 가이드라인이 함께 입력되는 단계; (g) 상기 인공지능 모델에 의해 정상 또는 다수의 타입 중 어느 하나의 타입의 척추측만증을 나타내는 척추측만증 정보가 연산되는 단계; 및 (h) 연산된 상기 척추측만증 정보가 출력부(220)에 의해 출력되는 단계;를 더 포함하는 것이 바람직하다.In addition, after step (e), (f) a step of inputting an appearance image and a guideline corresponding to the input appearance image into the artificial intelligence model generated in step (d) by the image input unit 210. ; (g) calculating scoliosis information indicating normal or one type of scoliosis among multiple types by the artificial intelligence model; and (h) outputting the calculated scoliosis information by the output unit 220. It is preferable to further include.
또한, 상기 (f) 단계 내지 상기 (h) 단계는 단말기에 의해 수행되며, 상기 단말기는 카메라를 포함하며, 상기 (f) 단계에서 입력되는 외견이미지는 상기 단말기의 카메라에 의해 촬영된 이미지인 것이 바람직하다.In addition, steps (f) to (h) are performed by a terminal, the terminal includes a camera, and the external image input in step (f) is an image captured by the camera of the terminal. desirable.
또한, 상기 (h) 단계 이후, (i) 상기 (f) 단계에서 입력된 외견이미지 및 상기 (g) 단계에서 연산된 척추측만증 정보가 미리 설정된 병원 의료관리 시스템(300)에 전송되는 단계; 및 (j) 상기 병원 의료관리 시스템(300)에서 상기 (i) 단계에서 전송된 외견이미지에 대응되는 전문진단정보가 확인되는 단계;를 더 포함하며, 상기 단말기는 통신 모듈을 더 포함하며, 상기 (i) 단계는 상기 단말기의 통신 모듈에 의해 상기 병원 의료관리 시스템(300)에 전송되는 단계인 것이 바람직하다.In addition, after step (h), (i) transmitting the external appearance image input in step (f) and the scoliosis information calculated in step (g) to a preset hospital medical management system 300; and (j) confirming professional diagnosis information corresponding to the external image transmitted in step (i) in the hospital medical management system 300, wherein the terminal further includes a communication module, Step (i) is preferably a step in which the information is transmitted to the hospital medical management system 300 by the communication module of the terminal.
또한, 상기 (j) 단계 이후, (k) 상기 병원 의료관리 시스템(300)이 상기 (i) 단계에서 전송된 외견이미지 및 상기 (j) 단계에서 확인되는 전문진단정보를 상기 인공지능 학습부(100)에 전송하는 단계; 및 (l) 상기 인공지능 학습부(100)가 상기 (k) 단계에서 전송된 외견이미지 및 전문진단정보를 이용하여 상기 인공지능 모델을 강화학습하는 단계;를 더 포함하는 것이 바람직하다.In addition, after step (j), (k) the hospital medical management system 300 uses the external image transmitted in step (i) and the professional diagnosis information confirmed in step (j) to the artificial intelligence learning unit ( 100) transmitting; and (l) the artificial intelligence learning unit 100 reinforcing the artificial intelligence model using the appearance image and professional diagnosis information transmitted in step (k). It is preferable to further include.
또한, 상기 (j) 단계 이후, (m) 상기 단말기의 카메라에 의해, 상기 (f) 단계에서 입력된 외견이미지와 상이한 변경 외견이미지가 생성되는 단계; (o) 상기 이미지 입력부(210)에 의해 상기 (m) 단계에서 생성된 변경 외견이미지와, 상기 입력된 변경 외견이미지에 해당하는 가이드라인이 상기 인공지능 모델에 함께 입력되는 단계; (p) 상기 인공지능 모델에 의해 정상 또는 다수의 타입 중 어느 하나의 타입의 척추측만증을 나타내는 변경 척추측만증 정보가 연산되는 단계; 및 (q) 연산된 상기 변경 척추측만증 정보가 상기 출력부(220)에 의해 출력되는 단계;를 더 포함하는 것이 바람직하다.In addition, after step (j), (m) generating a changed appearance image different from the appearance image input in step (f) by the camera of the terminal; (o) inputting the changed appearance image generated in step (m) by the image input unit 210 and the guideline corresponding to the input changed appearance image into the artificial intelligence model; (p) calculating altered scoliosis information indicating normal or one type of scoliosis among multiple types by the artificial intelligence model; and (q) outputting the calculated changed scoliosis information by the output unit 220.
또한, 상기 다수의 타입의 척추측만증은, 제1형 - 흉부(Thoracic) 척추측만증, 제2형 - 이중 흉부(Double Thoracic) 척추측만증, 제3형 - 이중 메이저-흉부/요추(Thoracic/Lumbar) 척추측만증, 제4형 - 삼중 커브(Triple Curve) 척추측만증, 및 제5형 - 요추/흉요(Lumbar/Thoracolumbar) 척추측만증을 포함하는 것이 바람직하다.In addition, the multiple types of scoliosis include: Type 1 - Thoracic scoliosis, Type 2 - Double Thoracic scoliosis, and Type 3 - Double Major-Thoracic/Lumbar. Preferably, it includes scoliosis, Type 4 - Triple Curve scoliosis, and Type 5 - Lumbar/Thoracolumbar scoliosis.
또한, 상기 (c) 단계는, 상기 입력된 외견이미지에서 인체 중 등 부분을 식별하고, 식별된 등 부분에서 좌우 중심축을 확인하고, 상기 입력된 외견이미지에서 상기 좌우 중심축을 기준으로 등 부분의 픽셀의 채도 차이를 확인하여 기 설정된 채도 차이 이상인 경우 채도가 낮은 일측을 등의 좌우측 중 일측의 튀어나옴이 있는 경우로 판단하여 해당 부분을 선으로 표시하고, 등 부분의 상측 윤곽선을 이용하여 양측 어깨의 윤곽선을 표시하고, 등 부분에서 좌우측 윤곽선을 이용하여 양측 옆구리의 윤곽선을 표시하는 단계;를 더 포함하는 것이 바람직하다.In addition, step (c) identifies the back portion of the human body in the input appearance image, confirms the left and right central axes in the identified back portion, and pixels of the back portion based on the left and right central axes in the input appearance image. Check the saturation difference, and if it is more than the preset saturation difference, the side with low saturation is judged to be a case of protrusion of the left and right sides of the back, and that part is marked with a line, and the upper outline of the back is used to mark the two shoulders. It is preferable to further include the step of marking the outline and marking the outline of both sides using the left and right outlines in the back.
또한, 양측 어깨의 윤곽선의 경사도 차이가 기 설정된 경사도 이상인 경우 더 올라온 어깨 부분을 어깨 올라옴으로 확인하고, 그리고 양측 옆구리 윤곽선의 좌우 중심축을 향하여 들어감 정도 차이가 기 설정된 들어감 이상인 경우 더 들어간 부분을 옆구리 들어감으로 확인하는 것이 바람직하다.In addition, if the difference in inclination between the outlines of both shoulders is more than the preset inclination, the part of the shoulder that is more raised is confirmed as shoulder elevation, and if the difference in the degree of indentation toward the left and right central axes of the outlines of both sides is more than the preset inclination, the part that is more inflated is checked in to the side. It is advisable to check with .
또한, 상기 가이드라인에 의해, 등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되고, 양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되지 않고, 양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되지 않는 경우, 제1형 척추측만증으로 확인되고, 등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되고, 양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되고, 양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되지 않는 경우, 제2형 척추측만증으로 확인되고, 등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되고, 양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되지 않고 양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되는 경우, 제3형 척추측만증으로 확인되고, 등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되고, 양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되고, 양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되는 경우, 제4형 척추측만증으로 확인되고, 그리고 등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되지 않고, 양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되지 않고, 양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되는 경우, 제5형 척추측만증으로 확인되는 것이 바람직하다.In addition, according to the above guidelines, one side of the back is protruding through the protruding lines on one of the left and right sides of the back, and the other shoulder is not raised through the outlines of both shoulders, and one side is retracted through the outlines of both sides. If this is not confirmed, it is confirmed as type 1 scoliosis, and one side of the back is protruded by the protrusion line on one of the left and right sides of the back, the other shoulder is raised by the outline of both shoulders, and both sides are confirmed to be raised. If one side is not confirmed through the outline, it is confirmed as type 2 scoliosis, one side of the back is protruded by the protrusion line on one of the left and right sides of the back, and the other shoulder is raised by the outline of both shoulders. If it is not confirmed and one side is confirmed through the outlines of both sides, type 3 scoliosis is confirmed, and one-sided back protrusion is confirmed by the protrusion lines on one of the left and right sides of the back, and the outlines of both shoulders are confirmed. If the other shoulder is raised and one side is confirmed to be retracted through the outlines of both sides, type 4 scoliosis is confirmed, and one side of the back is not protruded by the protrusion line on one of the left and right sides of the back. If the other shoulder is not raised through the outlines of both shoulders, and one side is depressed through the outlines of both sides, it is desirable to confirm it as type 5 scoliosis.
그리고, 상기와 같은 과제를 해결하기 위한 본 발명의 다른 실시예는 전술한 방법이 수행되는 컴퓨터 기록매체에 저장되는 컴퓨터 프로그램 내지 해당 프로그램이 저장되는 컴퓨터 기록매체를 제공한다.Another embodiment of the present invention to solve the above problems provides a computer program stored in a computer recording medium on which the above-described method is performed, or a computer recording medium in which the program is stored.
본 발명에 따라, 다음과 같은 효과가 달성된다. According to the present invention, the following effects are achieved.
첫째, 누구라도 손쉽게 원하는 시점에 척추측만증 여부를 조기 진단할 수 있어서 질병 접근성이 향상된다. 척추측만증의 타입은 물론 진행 양상과 치료 효과에 따른 변화까지도 원하는 시점에 손쉽게 확인할 수 있다. First, anyone can easily diagnose scoliosis at an early stage at any time, improving accessibility to the disease. You can easily check the type of scoliosis, as well as the progression pattern and changes according to treatment effects, at any desired time.
둘째, 정확도가 높다. 종래기술로서 전술한 특허들 중 외견이미지로 척추측만증 관련 정보를 연산하는 특허들의 공통적 문제점은 각각의 기법의 한계로 인한 오진단 가능성이었으나, 본 발명은 가이드라인을 새롭게 설정하고, 이를 기반으로 학습을 수행함으로써, 척추측만증 진단의 가능성을 높였다. 학습시 전문의료진의 실제 진단정보가 활용되어 정확도가 높다. 후술하는 검증 실험에서도 정확도가 우수함이 확인되었다. Second, the accuracy is high. Among the patents described above as prior art, a common problem with patents that calculate scoliosis-related information using external images was the possibility of misdiagnosis due to the limitations of each technique. However, the present invention sets new guidelines and provides learning based on them. By performing this, the possibility of diagnosing scoliosis was increased. Accuracy is high because actual diagnostic information from professional medical staff is used during learning. Excellent accuracy was confirmed in the verification experiment described later.
셋째, 불필요한 전문의료영상 촬영을 방지함으로써, 개인적 국가적 의료비용을 절감하고 소아/청소년의 불필요한 피폭을 감소시킨다. Third, by preventing unnecessary professional medical imaging, it reduces personal and national medical costs and reduces unnecessary radiation exposure of children and adolescents.
넷째, 학교 및 병원과 연계되어 데이터 보강을 통해 진단 정확도를 높이는 한편, 환자에게는 내원 스케줄을 손쉽게 설정하고 현재의 상황을 병원에 정확히 알려줄 수 있으며 치료 효과가 지속적으로 모니터링될 수 있다. 의료영상을 활용한 인공지능 기술의 문제점은 전문의료진에 의한 진단정보가 포함된 데이터가 부족하다는 것인데, 본 발명은 이를 원천적으로 해결할 수 있다. 또한, 보조기를 활용하거나 병원에서 안내하는 운동을 수행하는 경우 그 효과를 환자와 전문의료진이 함께 모니터링할 수 있으며, 경우에 따라 보조기 내지 운동 방식의 변경 필요성을 신속히 확인하고 치료 효과를 높일 수 있다. 또한, 전문의료진의 진단 결과가 관리되므로, 이를 이용하여 인공지능 모델을 강화학습할 수도 있다. Fourth, by linking with schools and hospitals, diagnostic accuracy can be improved through data reinforcement, while patients can easily set visit schedules, accurately inform hospitals of their current situation, and treatment effects can be continuously monitored. The problem with artificial intelligence technology using medical images is the lack of data containing diagnostic information by professional medical staff, and the present invention can fundamentally solve this problem. In addition, when using an orthosis or performing exercise guided by a hospital, the effect can be monitored by the patient and professional medical staff together, and in some cases, the need to change the orthosis or exercise method can be quickly confirmed and the treatment effect can be improved. In addition, since the diagnosis results of professional medical staff are managed, they can be used to strengthen an artificial intelligence model.
도 1은 본 발명에 따른 방법이 수행되는 시스템을 개시한다.Figure 1 discloses a system in which the method according to the invention is performed.
도 2는 본 발명에 따른 방법에서 척추측만증 타입을 설명하기 위한 도면으로, 각각의 타입에서 실제 외견이미지와 전문의료영상을 함께 개시한다. Figure 2 is a diagram for explaining scoliosis types in the method according to the present invention, and for each type, an actual external image and a professional medical image are disclosed together.
도 3은 본 발명에 따른 방법에서 척추측만증 타입을 결정하기 위한 요소들을 설명하기 위한 개념도이다.Figure 3 is a conceptual diagram illustrating elements for determining the type of scoliosis in the method according to the present invention.
도 4는 본 발명에 따른 방법을 설명하기 위한 순서도이다.Figure 4 is a flow chart for explaining the method according to the present invention.
도 5는 본 발명에 따른 방법이 학교와 병원을 중심으로 적용되는 일례를 도시하는 개념도이다. Figure 5 is a conceptual diagram showing an example in which the method according to the present invention is applied mainly to schools and hospitals.
도 6은 본 발명에 따른 방법을 실행한 후 각 가이드라인 요소들의 중요도를 검증한 결과를 나타낸다. Figure 6 shows the results of verifying the importance of each guideline element after executing the method according to the present invention.
이하에서, "외견이미지"는, 일반적인 카메라(예를 들어, 스마트폰 카메라)로 촬영한 영상으로 육안으로 관찰되는 환자의 등 이미지를 의미한다. 즉, 전문의료영상이 아닌 이미지이다. Hereinafter, “appearance image” refers to an image of the patient's back observed with the naked eye through an image taken with a general camera (for example, a smartphone camera). In other words, it is an image, not a professional medical video.
이하에서, "전문의료영상"은, X레이 영상, CT 영상과 같이 병원에서 접근 가능하며 전문의료진에 의해 촬영되어야 하는 영상을 의미한다. Hereinafter, “professional medical images” refers to images, such as X-ray images and CT images, that are accessible in hospitals and must be captured by professional medical staff.
이하에서, "전문진단정보"는, 전문의료진의 결정에 의한 것으로, 척추측만증인지 여부와 척추측만증인 경우 어떠한 타입인지 결정된 정보를 의미한다. Hereinafter, “professional diagnosis information” refers to information determined by a professional medical staff as to whether the patient has scoliosis and, if so, what type of scoliosis it is.
이하에서, "가이드라인"은 외견이미지 상에 오버랩되는 선을 의미한다. 예시적으로, 도 2에서 원과 직선의 적색선으로 도시된다. Hereinafter, “guideline” means a line that overlaps on the external image. By way of example, in FIG. 2, it is shown as a circle and a straight red line.
이하에서, "척추측만증 정보"는, 인공지능 모델에 의해 연산된 정보로서, 척추측만증인지 여부와 척추측만증인 경우 어떠한 타입인지 결정된 정보를 의미한다. 즉, 정상이거나, 다수의 척추측만증 타입 중 어느 하나의 타입의 척추측만증인 것을 포함한다.Hereinafter, “scoliosis information” is information calculated by an artificial intelligence model, and refers to information determined as to whether scoliosis is present and, if so, what type of scoliosis it is. That is, it includes normal scoliosis or any one type of scoliosis among multiple scoliosis types.
이하에서, "척추측만증 타입"은 만곡의 형태에 따른 분류를 의미한다. 본 발명은 5개 타입의 척추측만증 타입을 제안한다. 이는, 외견이미지만으로 분류할 수 있는 형태이지 해당 정보가 전문의료진에게 전달될 경우 신속하고 정확한 전문 진단 및 치료 방식 설정에 도움을 주는 분류로 결정한 것이다. Hereinafter, “scoliosis type” refers to classification according to the type of curve. The present invention proposes five types of scoliosis. This is a type that can be classified only based on external images, but has been decided to be a classification that will help establish a quick and accurate professional diagnosis and treatment method when the information is delivered to professional medical staff.
이하, 도면을 참조하여 본 발명을 보다 상세히 설명한다.Hereinafter, the present invention will be described in more detail with reference to the drawings.
1. 시스템의 설명1. Description of the system
*본 발명에 따른 방법을 수행하기 위한 시스템은, 인공지능 학습부(100)와 척추측만증 정보 연산부(200)를 포함한다. *The system for performing the method according to the present invention includes an artificial intelligence learning unit 100 and a scoliosis information calculation unit 200.
인공지능 학습부(100)는 학습을 위한 데이터를 수집하고 학습데이터세트를 설정하여 학습을 통해 인공지능 모델을 생성하는 기능을 수행한다. 여기에서 사용되는 인공지능의 종류는 제한되지 않는다. The artificial intelligence learning unit 100 collects data for learning, sets a learning dataset, and performs the function of creating an artificial intelligence model through learning. The type of artificial intelligence used here is not limited.
인공지능 학습부(100)는 이미지를 수집하는 이미지 수집부(110), 해당 이미지에 가이드라인을 포함시키는 가이드라인 제공부(120) 및 해당 이미지에 전문의료진의 전문진단정보를 입력하는 전문진단정보 입력부(130)를 포함한다. The artificial intelligence learning unit 100 includes an image collection unit 110 that collects images, a guideline provision unit 120 that includes guidelines in the image, and professional diagnosis information that inputs professional diagnosis information from a professional medical staff into the image. Includes an input unit 130.
이미지 수집부(110)는 다수의 외견이미지를 수집한다. 전문의료진에 의한 척추측만증 여부 및 타입이 대응되어 있는 외견이미지를 수집하여야 한다. The image collection unit 110 collects multiple external images. Appearance images showing the presence and type of scoliosis must be collected by professional medical staff.
가이드라인 제공부(120)는 수동 또는 자동으로 외견이미지에 가이드라인을 포함시킨다. 구체적 방법은 후술한다. The guideline providing unit 120 manually or automatically includes guidelines in the external image. The specific method will be described later.
전문진단정보 입력부(130)는 수집된 외견이미지마다 전문의료진에 의해 진단된 척추측만증 여부 및 타입에 관한 정보인 전문진단정보를 외견이미지에 매칭하여 입력하는 기능을 수행한다.The professional diagnosis information input unit 130 performs a function of inputting professional diagnosis information, which is information about the presence and type of scoliosis diagnosed by a professional medical staff, to the external image for each collected external image.
인공지능 학습부(100)는 이미지 수집부(110), 가이드라인 제공부(120) 및 전문진단정보 입력부(130)에 의해 확인된 데이터로서, 가이드라인이 추가된 외견이미지를 입력데이터로 하고 전문진단정보를 출력데이터로 하는 학습데이터세트를 설정하고 이를 학습시켜 인공지능 모델을 생성한다.The artificial intelligence learning unit 100 is data confirmed by the image collection unit 110, the guideline provision unit 120, and the professional diagnosis information input unit 130, and uses the external image with the guideline added as input data and the professional diagnosis information input unit 130. Set up a learning dataset with diagnostic information as output data and learn it to create an artificial intelligence model.
척추측만증 정보 연산부(200)는 생성된 인공지능 모델을 포함하며 이미지 입력부(210)와 출력부(220)를 포함한다. The scoliosis information calculation unit 200 includes a generated artificial intelligence model and includes an image input unit 210 and an output unit 220.
이미지 입력부(210)를 통해 생성된 인공지능 모델에 입력될 외견이미지가 확인된다. The external image to be input to the artificial intelligence model created through the image input unit 210 is confirmed.
출력부(220)는, 이미지 입력부(210)에 의해 입력된 외견이미지가 인공지능 모델의 결과인 척추측만증 정보를 출력한다. The output unit 220 outputs scoliosis information in which the external image input by the image input unit 210 is the result of an artificial intelligence model.
척추측만증 정보 연산부(200)는 사용자가 쉽게 접근할 수 있는 단말기에서 수행 가능한 프로그램 내지 어플리케이션인 것이 바람직하며, 특히 단말기는 카메라와 통신 모듈을 포함한 스마트폰인 것이 바람직하다. 이 경우, 스마트폰의 카메라를 이용하여 사용자의 등을 촬영하면 외견이미지로 인식되고, 어플리케이션에 해당 이미지를 업로드함으로써 손쉽게 인공지능 모델에 입력할 수 있으며, 그 결과 역시 손쉽게 확인될 수 있다. The scoliosis information calculation unit 200 is preferably a program or application that can be executed on a terminal that the user can easily access. In particular, the terminal is preferably a smartphone that includes a camera and a communication module. In this case, if the user's back is photographed using the smartphone's camera, it is recognized as an external image, and the image can be easily input into the artificial intelligence model by uploading it to the application, and the results can also be easily confirmed.
한편, 본 발명에 따른 방법을 수행하기 위한 시스템은, 병원 의료관리 시스템(300)과 더 연계될 수 있다. Meanwhile, the system for performing the method according to the present invention may be further linked to the hospital medical management system 300.
척추측만증 정보 연산부(200)에서 확인한 외견이미지와 척추측만증 정보(즉, 척추측만증 여부 및 타입)는 병원 의료관리 시스템(300)에 자동으로 전달될 수 있다. 병원 의료관리 시스템(300)은 별도의 방법으로 환자를 식별한 후 해당 정보들을 디지털 차트에 저장하고 업로드하여 전문의료진이 접근 가능하게 한다. The external image and scoliosis information (i.e., presence and type of scoliosis) confirmed by the scoliosis information calculation unit 200 may be automatically transmitted to the hospital medical management system 300. The hospital medical management system 300 identifies patients using a separate method, stores and uploads the relevant information in a digital chart, and makes it accessible to professional medical staff.
환자가 내원하여 진단을 받고 치료를 받으면 병원 의료관리 시스템(300)에 진단 정보, 치료 정보 등이 저장되는데, 이러한 정보들은 척추측만증 정보 연산부(200), 즉 환자의 스마트폰으로 전달되어 통합 관리될 수도 있다. When a patient visits the hospital, is diagnosed, and receives treatment, diagnosis information, treatment information, etc. are stored in the hospital medical management system 300. This information is transmitted to the scoliosis information calculation unit 200, that is, the patient's smartphone, and is integrated and managed. It may be possible.
보조기, 운동, 수술 등을 통한 치료 동안에도 사용자인 환자는 스마트폰을 이용하여 자신의 등을 촬영하여 변경 외견이미지를 지속 생성하고, 이러한 정보는 척추측만증 정보 연산부(200)에 입력되어 변경 척추측만증 정보가 생성되며, 생성된 정보들이 다시 병원 의료관리 시스템(300)으로 전송된다. 이를 통해, 치료 경과 과정을 환자 스스로 확인할 수도 있고, 전문의료진 역시 이를 모니터링할 수 있다. Even during treatment through braces, exercise, surgery, etc., the user, the patient, continuously creates a modified appearance image by taking pictures of his/her back using a smartphone, and this information is entered into the scoliosis information calculation unit 200 to change the scoliosis. Information is generated, and the generated information is transmitted back to the hospital medical management system 300. Through this, patients can check the treatment progress themselves, and professional medical staff can also monitor it.
또한, 병원 의료관리 시스템(300)과 연계됨으로써, 전문의료진의 진단에 대한 정보인 전문진단정보가 생성되는바, 이미지 입력부(210)에 입력된 외견이미지와, 병원 의료관리 시스템(300)에서 생성된 전문진단정보가 추가 학습데이터세트를 형성할 수 있어서, 인공지능 모델을 강화학습할 수 있다. In addition, by linking with the hospital medical management system 300, professional diagnosis information, which is information about the diagnosis by professional medical staff, is generated, and the external image input to the image input unit 210 and generated by the hospital medical management system 300 The received professional diagnosis information can form an additional learning dataset, enabling reinforcement learning of the artificial intelligence model.
2. 방법의 설명2. Description of method
2.1 척추측만증 타입과 이를 결정하는 요소의 설명2.1 Description of scoliosis types and factors that determine them
도 2 및 도 3을 참조하여 본 발명에 따른 방법에서 식별하는 척추측만증 타입과 이를 결정하는 요소들을 설명한다. 2 and 3, the type of scoliosis identified in the method according to the present invention and the factors that determine it will be described.
본 발명은, 외견이미지만으로 분류할 수 있는 형태인 동시에, 해당 정보가 전문의료진에게 전달될 경우 신속하고 정확한 전문 진단 및 치료 방식 설정에 도움을 주는 것으로서, 척추 만곡에 따라 척추측만증을 5개의 타입으로 구분하는 방식을 제안한다.The present invention is a form that can be classified only by external image, and at the same time, when the information is delivered to professional medical staff, it helps in setting up a quick and accurate professional diagnosis and treatment method. It divides scoliosis into five types according to the curvature of the spine. We suggest a way to differentiate.
도 2에 도시된 바와 같이, 제1형은 흉부(Thoracic) 척추측만증이고, 제2형은이중 흉부(Double Thoracic) 척추측만증이고, 제3형은 이중 메이저-흉부/요추(Thoracic/Lumbar) 척추측만증이고, 제4형은 삼중 커브(Triple Curve) 척추측만증이고, 제5형은 요추/흉요(Lumbar/Thoracolumbar) 척추측만증이다. As shown in Figure 2, type 1 is thoracic scoliosis, type 2 is double thoracic scoliosis, and type 3 is double major-thoracic/lumbar. Scoliosis, Type 4 is Triple Curve scoliosis, and Type 5 is Lumbar/Thoracolumbar scoliosis.
이러한 구분은 전문의료진에 의해 수행될 수 있는데, 도 3에 도시된 바와 같이 크게 3가지 요소를 기반으로 판단된다. 이는, (A)등의 일측(좌우측 중 일측) 튀어나옴 여부, (B)타측(즉, 등이 튀어나온 경우 그 반대측) 어깨 올라옴 여부, (C)상기 일측 옆구리 들어감 여부이다. 이들은 하나 이상이 조합되어 나타날 수 있다. 따라서, 이들의 조합을 통하여 척추측만증 타입이 결정된다. 도시되지 않는 조합의 경우(예컨대, 등의 일측이 튀어나오고 타측 어깨가 올라가지만 일측 옆구리가 들어가지 않는 경우)는 임상적으로 극히 예외적인 경우로 이해될 수 있다. This classification can be performed by professional medical staff, and is largely based on three factors as shown in FIG. 3. This includes (A) whether one side of the back (either left or right) sticks out, (B) whether the shoulder on the other side (i.e., the opposite side if the back sticks out) rises, and (C) whether the side of the side goes in. These may appear in combinations of one or more. Therefore, the scoliosis type is determined through the combination of these. A combination not shown (for example, a case where one side of the back sticks out and the other shoulder goes up but one side does not go in) can be understood clinically as an extremely exceptional case.
이를 자동으로 구분하기 위해 외견이미지에는 이들을 식별하는 가이드라인이 포함된다. 따라서, 가이드라인은 등의 좌우측 중 일측의 튀어나옴이 있는 경우 이를 표시하는 선, 양측 어깨의 윤곽선 및 양측 옆구리의 윤곽선을 포함한다. 가이드라인의 예시는 도 2에서도 확인할 수 있다. In order to automatically distinguish them, the external image includes guidelines for identifying them. Therefore, the guideline includes a line indicating the protrusion of one of the left and right sides of the back, the outlines of both shoulders, and the outlines of both sides. An example of the guidelines can also be found in Figure 2.
가이드라인은 해당 정보를 안내할 뿐이다. 도 2에서와 같이 가이드라인이 도시되었다고 해당 요소가 반드시 해당하는 것으로 체크되는 것은 아니다. 예컨대, 어깨선 가이드라인이 도시되었으나 좌우 경사도 차이가 후술하는 기 설정된 경사도 차이보다 크지 않을 수 있으며 이 경우 (B)의 타측 어깨 올라옴은 아닌 것으로 확인될 수 있다. Guidelines only provide information. Just because a guideline is shown as in Figure 2, the corresponding element is not necessarily checked as applicable. For example, although the shoulder line guideline is shown, the left and right slope difference may not be greater than the preset slope difference described later, and in this case, it can be confirmed that the other shoulder of (B) is not raised.
가이드라인은 자동 또는 수동으로 확인될 수 있다. Guidelines can be checked automatically or manually.
수동 확인의 경우, 전문의료진에 의해 위 가이드라인들이 이미지 상에서 직접 체크되고 입력된다.In the case of manual confirmation, the above guidelines are checked and entered directly on the image by professional medical staff.
자동 확인의 경우, 외견이미지가 입력되면 자동으로 가이드라인이 생성되는 방식이다. 이를 위해 외견이미지에서 인체 중 등 부분이 자동으로 식별되고, 식별된 등 부분에서 좌우 중심축을 확인한다. 이러한 기술은 널리 사용되는 종래기술인바 상세한 설명은 생략한다. In the case of automatic confirmation, guidelines are automatically created when an external image is input. For this purpose, the back part of the human body is automatically identified in the external image, and the left and right central axes are confirmed in the identified back part. Since this technology is a widely used conventional technology, detailed description is omitted.
다음, 외견이미지에서 좌우 중심축을 기준으로 등 부분의 픽셀의 채도 차이를 확인하여 기 설정된 채도 차이 이상인 경우 채도가 낮은 일측을 등의 좌우측 중 일측의 튀어나옴이 있는 경우로 판단하여 해당 부분을 선으로 표시한다. 또한, 등 부분의 상측 윤곽선을 이용하여 양측 어깨의 윤곽선을 표시한다. 또한, 등 부분에서 좌우측 윤곽선을 이용하여 양측 옆구리의 윤곽선을 표시한다. Next, check the saturation difference between the pixels in the back area based on the left and right central axes in the external image. If the saturation difference is more than the preset, the side with low saturation is judged to be a case where one of the left and right sides of the back protrudes, and that part is marked with a line. Display. Additionally, the outline of both shoulders is marked using the upper outline of the back. Additionally, the outlines of both sides are marked using the left and right outlines on the back.
여기서, 양측 어깨의 윤곽선의 경사도 차이가 기 설정된 경사도 이상인 경우 더 올라온 어깨 부분을 어깨 올라옴으로 확인하고, 그리고 양측 옆구리 윤곽선의 좌우 중심축을 향하여 들어감 정도의 차이가 기 설정된 들어감 이상인 경우 더 들어간 부분을 옆구리 들어감으로 확인할 수 있다. Here, if the difference in slope between the outlines of both shoulders is more than the preset slope, the part of the shoulder that is more raised is confirmed as shoulder rise, and if the difference in the degree of indentation toward the left and right central axes of the outlines of both sides is more than the preset inclination, the part that is more indented is confirmed as the side indentation. You can check by entering.
다만, 언급한 기 설정된 채도 차이, 기 설정된 경사도, 기 설정된 들어감 여부 등 이들을 결정하는 기준은, 전문진단정보가 포함된 학습데이터세트를 활용한 인공지능 학습에 따라 특정되지 않고 달라질 수 있다. 후술하는 강화학습에 따라 달라질 수도 있다. 어느 경우이든 전문진단정보와의 정확도를 높이는 방향으로 달라질 것이다. However, the criteria for determining these, such as the preset saturation difference, preset slope, and preset entry, may not be specified and may vary depending on artificial intelligence learning using a learning dataset containing professional diagnosis information. It may vary depending on reinforcement learning, which will be described later. In either case, there will be a change in the direction of increasing the accuracy of professional diagnosis information.
이제, 표시된 1개 내지 3개의 가이드라인은 인공지능 모델의 입력데이터로 활용된다. Now, one to three displayed guidelines are used as input data for the artificial intelligence model.
제한되지 않는 예시를 든다면, 등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되고 양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되지 않고 양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되지 않는 경우 제1형 척추측만증으로 확인되고, 등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되고 양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되고 양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되지 않는 경우 제2형 척추측만증으로 확인되고, 등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되고 양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되지 않고 양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되는 경우 제3형 척추측만증으로 확인되고, 등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되고 양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되고 양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되는 경우 제4형 척추측만증으로 확인되고, 그리고 등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되지 않고 양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되지 않고 양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되는 경우 제5형 척추측만증으로 확인될 수 있다. To give a non-limiting example, the protrusion of one side of the back is confirmed by the protrusion line on one of the left and right sides of the back, the raising of the other shoulder is not confirmed through the outlines of both shoulders, and the retraction of one side is confirmed through the outlines of both sides. If this is not the case, it is confirmed as type 1 scoliosis, and one-sided back protrusion is confirmed by the protruding line on one of the left and right sides of the back, the other shoulder is raised through the outline of both shoulders, and one side is confirmed by the outline of both sides. If the retraction is not confirmed, it is confirmed as type 2 scoliosis, and the protrusion of one side of the back is confirmed by the protrusion line on one of the left and right sides of the back, and the protrusion of the other shoulder is not confirmed by the outline of both shoulders and the outline of both sides. If one side is confirmed to be indented, it is confirmed as type 3 scoliosis, one side of the back is protruded by the protruding line on one of the left and right sides of the back, the other shoulder is raised through the outlines of both shoulders, and both sides are confirmed to be raised. Type 4 scoliosis is confirmed when one side is confirmed through the outline of the back, and when one side of the back is not protruded by the protrusion line on one side of the left or right side of the back, the other shoulder is raised through the outline of both shoulders. If it is not confirmed and unilateral indentation is confirmed through the outlines of both sides, it can be confirmed as type 5 scoliosis.
한편, 인공지능 모델 학습시, 이미지에 포함된 가이드라인은 자동 인식되고, 이는 입력데이터의 하나로서 활용된다. Meanwhile, when learning an artificial intelligence model, the guidelines included in the image are automatically recognized and used as one of the input data.
2.2 척추측만증 정보 제공 방법의 설명2.2 Description of how to provide scoliosis information
이하, 도 4를 참조하여 본 발명에 따른 척추측만증 정보 제공 방법을 설명한다. Hereinafter, a method for providing scoliosis information according to the present invention will be described with reference to FIG. 4.
먼저, 이미지 수집부(110)에서 수집된 외견이미지가 인공지능 학습부(100)에 입력되고, 입력된 외견이미지 상에 가이드라인이 추가되어 인공지능 학습부(100)에 입력된다. 또한, 입력된 외견이미지에 대응되는 전문진단정보가 더 입력된다. 인공지능 학습부(100)는 가이드라인이 추가된 외견이미지를 입력데이터로 하고 상기 전문진단정보를 출력데이터로 한 학습데이터세트를 학습하여 인공지능 모델을 생성한다(S110).First, the external image collected in the image collection unit 110 is input to the artificial intelligence learning unit 100, and a guideline is added to the input external image and input to the artificial intelligence learning unit 100. Additionally, professional diagnosis information corresponding to the input external image is further input. The artificial intelligence learning unit 100 generates an artificial intelligence model by learning a learning dataset using the appearance image with the guideline added as input data and the professional diagnosis information as output data (S110).
이제, 스마트폰 등에 해당 프로그램 내지 어플리케이션이 설치되면 사용자는 손쉽게 이에 접근할 수 있다. 즉, 이미지 입력부(210)에 의해 인공지능 모델에 외견이미지가 입력되되(S210), 입력된 외견이미지에 해당하는 가이드라인이 함께 입력되면(S220), 인공지능 모델에 의해 정상 또는 다수의 타입 중 어느 하나의 타입의 척추측만증을 나타내는 척추측만증 정보가 연산되어 출력부(220)에 의해 출력된다(S230). Now, once the program or application is installed on a smartphone, etc., users can easily access it. That is, an external image is input to the artificial intelligence model by the image input unit 210 (S210), and when a guideline corresponding to the input external image is input together (S220), the artificial intelligence model determines whether the image is normal or one of multiple types. Scoliosis information indicating one type of scoliosis is calculated and output by the output unit 220 (S230).
3. 병원과 학교에서 본 발명의 적용되는 방식의 설명3. Description of how the present invention is applied in hospitals and schools
이하, 도 4와 도 5를 함께 참조하여 본 발명에 따른 방법이 학교 및 병원과 연계되어 제공되는 방식을 설명한다. Hereinafter, with reference to FIGS. 4 and 5, it will be described how the method according to the present invention is provided in connection with schools and hospitals.
본 발명을 개발한 병원은 학교 또는 관련 정부부처에 본 발명을 소개할 수 있으며[A1], 이는 학교 등을 통해 학생 또는 부모에게 안내될 수 있다[A2]. The hospital that developed the present invention can introduce the present invention to schools or related government departments [A1], and it can be introduced to students or parents through schools, etc. [A2].
학생 또는 부모는 스마트폰 등에 프로그램 내지 어플리케이션을 설치하고[B1], 카메라를 이용하여 외견이미지를 생성하여 입력하면[B2], 언제라도 손쉽게 척추측만증 여부가 조기진단되어 출력된다[B3]. Students or parents install a program or application on a smartphone, etc. [B1], use a camera to create and input an external image [B2], and early diagnosis of scoliosis is easily diagnosed and printed at any time [B3].
진단된 정보는 병원으로 전송될 수 있다. Diagnosed information can be transmitted to the hospital.
병원에 전송된 경우 다음의 두 가지 방식으로 활용될 수 있다. When transmitted to a hospital, it can be used in the following two ways.
첫째, 강화학습이다. First, it is reinforcement learning.
구체적으로, 사용자가 입력한 외견이미지와 인공지능 모델에 의해 연산된 척추측만증 정보가 미리 설정된 병원 의료관리 시스템(300)에 전송된다(S310)[C1]. 이러한 전송은 스마트폰의 통신 모듈을 통해 손쉽게 이루어질 수 있다. Specifically, the external image input by the user and the scoliosis information calculated by the artificial intelligence model are transmitted to the preset hospital medical management system 300 (S310) [C1]. This transmission can be easily accomplished through the communication module of the smartphone.
병원에서는 이를 확인하여 내원 스케쥴을 설정할 수 있으며[C2], 환자가 내원하여 전문의료진에 의한 직접 진단을 받음으로써 전문진단정보가 생성되고 병원 의료관리 시스템(300)에 업로드된다[C3]. 따라서, 병원 의료관리 시스템(300)에서는 외견이미지와 이에 대응되는 전문진단정보가 확인하여 추가 학습데이터세트로 설정하고 인공지능 학습부(100)에 전송한다. 이에 따라 인공지능 모델은 강화학습된다(S320)[C4]. The hospital can check this and set a visit schedule [C2], and when the patient visits the hospital and receives a direct diagnosis by a professional medical staff, professional diagnosis information is generated and uploaded to the hospital medical management system 300 [C3]. Therefore, the hospital medical management system 300 confirms the external image and the corresponding professional diagnosis information, sets it as an additional learning dataset, and transmits it to the artificial intelligence learning unit 100. Accordingly, the artificial intelligence model is reinforced learning (S320)[C4].
둘째, 치료 경과에 따른 자가진단 및 전문의료진의 경과 추적이다. Second, self-diagnosis and follow-up by professional medical staff according to the progress of treatment.
치료가 시작되면 환자(즉, 본 발명의 사용자)는 스마트폰과 같은 단말기를 활용하여 종종 변경 외견이미지를 생성할 수 있다. 이는 해당 단말기에 설치된 프로그램 또는 어플리케이션을 통하여 인공지능 모델에 입력되며 마찬가지로 가이드라인이 함께 입력된다. 이에 따라 변경 외견이미지에 따른 변경 척추측만증 정보, 즉 정상 또는 다수의 타입 중 어느 하나의 타입의 척추측만증이 확인된다(S410)[C5]. 환자가 이를 확인하면 치료 경과를 자가진단할 수 있으며, 병원 의료관리 시스템(300)에 전송되어 전문의료진이 확인함으로써 치료 경과를 추적하고 필요시 보조기 또는 운동 방식을 변경하거나 다시 내원 스케쥴을 설정하도록 할 수 있다. Once treatment begins, the patient (i.e., the user of the present invention) can often create an altered appearance image using a terminal such as a smartphone. This is input into the artificial intelligence model through a program or application installed on the terminal, and guidelines are also input. Accordingly, information on the changed scoliosis according to the changed external image, that is, scoliosis of either normal or multiple types is confirmed (S410) [C5]. If the patient confirms this, he or she can self-diagnose the progress of the treatment, and it is transmitted to the hospital medical management system (300) to be confirmed by the medical staff, allowing the patient to track the progress of the treatment and, if necessary, change the brace or exercise method or set a visit schedule again. You can.
4. 검증4. Verification
본 출원인은 설명한 방법의 프로그램을 제작한 후 일부 데이터를 해당 프로그램에 입력하여 실행하는 한편 해당 데이터를 나타내는 환자에 대한 전문의료진의 전문진단결과와 비교하는 검증 실험을 수행하였으며, 약 80%의 정확도가 나타남을 확인하였다. After creating a program using the described method, the applicant entered and executed some data into the program and performed a verification experiment comparing the data with the professional diagnosis results of professional medical staff on patients representing the data, and found that the accuracy was about 80%. Appearance was confirmed.
검증 실험 동안, (A)등의 일측(좌우측 중 일측) 튀어나옴 여부, (B)타측(즉, 등이 튀어나온 경우 그 반대측) 어깨 올라옴 여부, (C)상기 일측 옆구리 들어감 여부의 세 개의 요소들 중 어느 요소가 척추측만증 타입 결정에 더 많은 영향을 주었는지 여부를 분석함으로써 중요도를 확인하였다. 그 결과 날개뼈의 돌출 여부(Scapula), 즉 (A)등의 일측 튀어나옴 여부가 다른 요소들 대비 3배 내지 10배의 차이를 두고 중요한 요소임이 더 확인되었다. During the verification experiment, three factors were determined: (A) whether one side of the back (either left or right) sticks out, (B) whether the shoulder on the other side (i.e., the opposite side if the back sticks out) rises, and (C) whether the side of the side goes in. The importance was confirmed by analyzing which of these factors had more influence in determining the type of scoliosis. As a result, it was further confirmed that the protrusion of the wing bones (scapula), that is, whether one side of the back (A) protrudes, is an important factor with a difference of 3 to 10 times compared to other factors.
(도면부호의 설명)(Explanation of drawing symbols)
100: 인공지능 학습부100: Artificial Intelligence Learning Department
110: 이미지 수집부110: Image collection unit
120: 가이드라인 제공부120: Guideline provision department
130: 전문진단정보 입력부130: Professional diagnosis information input unit
200: 척추측만증 정보 입력부200: Scoliosis information input unit
210: 이미지 입력부210: Image input unit
220: 출력부220: output unit
300: 병원 의료관리 시스템300: Hospital medical management system

Claims (12)

  1. (a) 이미지 수집부(110)에서 외견이미지가 수집되는 단계; - 여기서, 외견이미지는 등의 이미지를 포함함(a) collecting an external image in the image collection unit 110; - Here, the external image includes images of the back.
    (b) 수집된 외견이미지가 인공지능 학습부(100)에 입력되는 단계;(b) inputting the collected appearance images into the artificial intelligence learning unit 100;
    (c) 상기 입력된 외견이미지 상에 가이드라인이 추가되어 상기 인공지능 학습부(100)에 입력되는 단계; - 여기서, 상기 가이드라인은 등의 좌우측 중 일측의 튀어나옴이 있는 경우 이를 표시하는 선, 양측 어깨의 윤곽선 및 양측 옆구리의 윤곽선을 포함함(c) adding a guideline to the input external image and inputting it into the artificial intelligence learning unit 100; - Here, the guideline includes the line indicating the protrusion of one of the left and right sides of the back, the outline of both shoulders, and the outline of both sides.
    (d) 상기 입력된 외견이미지에 대응되는 전문진단정보가 상기 인공지능 학습부(100)에 입력되는 단계; 및 - 여기서, 상기 전문진단정보는, 정상 및 다수의 타입 중 어느 하나의 타입의 척추측만증임을 포함함(d) inputting professional diagnosis information corresponding to the input external image into the artificial intelligence learning unit 100; And - Here, the professional diagnosis information includes any one type of scoliosis among normal and multiple types.
    (e) 상기 인공지능 학습부(100)가 상기 가이드라인이 추가된 외견이미지를 입력데이터로 하고 상기 전문진단정보를 출력데이터로 한 학습데이터세트를 학습하여 인공지능 모델을 생성하는 단계;를 포함하는, (e) the artificial intelligence learning unit 100 generates an artificial intelligence model by learning a learning dataset using the appearance image to which the guideline is added as input data and the professional diagnosis information as output data; including; doing,
    방법.method.
  2. 제 1 항에 있어서, According to claim 1,
    상기 (e) 단계 이후, After step (e) above,
    (f) 이미지 입력부(210)에 의해 상기 (d) 단계에서 생성된 인공지능 모델에 외견이미지와, 상기 입력된 외견이미지에 해당하는 가이드라인이 함께 입력되는 단계;(f) inputting an external image and a guideline corresponding to the input external image into the artificial intelligence model generated in step (d) by the image input unit 210;
    (g) 상기 인공지능 모델에 의해 정상 또는 다수의 타입 중 어느 하나의 타입의 척추측만증을 나타내는 척추측만증 정보가 연산되는 단계; 및(g) calculating scoliosis information indicating normal or one type of scoliosis among multiple types by the artificial intelligence model; and
    (h) 연산된 상기 척추측만증 정보가 출력부(220)에 의해 출력되는 단계;를 더 포함하는,(h) outputting the calculated scoliosis information by the output unit 220; further comprising,
    방법.method.
  3. 제 2 항에 있어서, According to claim 2,
    상기 (f) 단계 내지 상기 (h) 단계는 단말기에 의해 수행되며, Steps (f) to (h) are performed by the terminal,
    상기 단말기는 카메라를 포함하며, The terminal includes a camera,
    상기 (f) 단계에서 입력되는 외견이미지는 상기 단말기의 카메라에 의해 촬영된 이미지인, The external image input in step (f) is an image captured by the camera of the terminal,
    방법.method.
  4. 제 3 항에 있어서, According to claim 3,
    상기 (h) 단계 이후, After step (h) above,
    (i) 상기 (f) 단계에서 입력된 외견이미지 및 상기 (g) 단계에서 연산된 척추측만증 정보가 미리 설정된 병원 의료관리 시스템(300)에 전송되는 단계; 및(i) transmitting the external image input in step (f) and the scoliosis information calculated in step (g) to a preset hospital medical management system 300; and
    (j) 상기 병원 의료관리 시스템(300)에서 상기 (i) 단계에서 전송된 외견이미지에 대응되는 전문진단정보가 확인되는 단계;를 더 포함하며, (j) confirming professional diagnosis information corresponding to the external image transmitted in step (i) in the hospital medical management system 300;
    상기 단말기는 통신 모듈을 더 포함하며, The terminal further includes a communication module,
    상기 (i) 단계는 상기 단말기의 통신 모듈에 의해 상기 병원 의료관리 시스템(300)에 전송되는 단계인, The step (i) is a step transmitted to the hospital medical management system 300 by the communication module of the terminal,
    방법.method.
  5. 제 4 항에 있어서, According to claim 4,
    상기 (j) 단계 이후, After step (j) above,
    (k) 상기 병원 의료관리 시스템(300)이 상기 (i) 단계에서 전송된 외견이미지 및 상기 (j) 단계에서 확인되는 전문진단정보를 상기 인공지능 학습부(100)에 전송하는 단계; 및(k) the hospital medical management system 300 transmitting the external appearance image transmitted in step (i) and the professional diagnosis information confirmed in step (j) to the artificial intelligence learning unit 100; and
    (l) 상기 인공지능 학습부(100)가 상기 (k) 단계에서 전송된 외견이미지 및 전문진단정보를 이용하여 상기 인공지능 모델을 강화학습하는 단계;를 더 포함하는, (l) the artificial intelligence learning unit 100 reinforcing the artificial intelligence model using the appearance image and professional diagnosis information transmitted in step (k); further comprising,
    방법.method.
  6. 제 5 항에 있어서, According to claim 5,
    상기 (j) 단계 이후, After step (j) above,
    (m) 상기 단말기의 카메라에 의해, 상기 (f) 단계에서 입력된 외견이미지와 상이한 변경 외견이미지가 생성되는 단계; (m) generating, by the camera of the terminal, a modified appearance image different from the appearance image input in step (f);
    (o) 상기 이미지 입력부(210)에 의해 상기 (m) 단계에서 생성된 변경 외견이미지와, 상기 입력된 변경 외견이미지에 해당하는 가이드라인이 상기 인공지능 모델에 함께 입력되는 단계;(o) inputting the changed appearance image generated in step (m) by the image input unit 210 and the guideline corresponding to the input changed appearance image into the artificial intelligence model;
    (p) 상기 인공지능 모델에 의해 정상 또는 다수의 타입 중 어느 하나의 타입의 척추측만증을 나타내는 변경 척추측만증 정보가 연산되는 단계; 및(p) calculating altered scoliosis information indicating normal or one type of scoliosis among multiple types by the artificial intelligence model; and
    (q) 연산된 상기 변경 척추측만증 정보가 상기 출력부(220)에 의해 출력되는 단계;를 더 포함하는, (q) outputting the calculated altered scoliosis information by the output unit 220; further comprising,
    방법.method.
  7. 제 1 항에 있어서, According to claim 1,
    상기 다수의 타입의 척추측만증은 다음을 포함하는, The multiple types of scoliosis include:
    제1형 - 흉부(Thoracic) 척추측만증,Type 1 - Thoracic scoliosis,
    제2형 - 이중 흉부(Double Thoracic) 척추측만증,Type 2 - Double Thoracic scoliosis,
    제3형 - 이중 메이저-흉부/요추(Thoracic/Lumbar) 척추측만증,Type 3 - Double Major-Thoracic/Lumbar Scoliosis,
    제4형 - 삼중 커브(Triple Curve) 척추측만증, 및Type 4 - Triple Curve Scoliosis, and
    제5형 - 요추/흉요(Lumbar/Thoracolumbar) 척추측만증Type 5 - Lumbar/Thoracolumbar scoliosis
    방법.method.
  8. 제 7 항에 있어서, According to claim 7,
    상기 (c) 단계는, In step (c),
    상기 입력된 외견이미지에서 인체 중 등 부분을 식별하고, 식별된 등 부분에서 좌우 중심축을 확인하고, Identify the back part of the human body from the input external image, and confirm the left and right central axes in the identified back part,
    상기 입력된 외견이미지에서 상기 좌우 중심축을 기준으로 등 부분의 픽셀의 채도 차이를 확인하여 기 설정된 채도 차이 이상인 경우 채도가 낮은 일측을 등의 좌우측 중 일측의 튀어나옴이 있는 경우로 판단하여 해당 부분을 선으로 표시하고, In the input external image, the saturation difference between the pixels of the back is checked based on the left and right central axes, and if the saturation difference is more than a preset value, the side with low saturation is judged to be a case where one of the left and right sides of the back protrudes, and the corresponding part is removed. Mark with a line,
    등 부분의 상측 윤곽선을 이용하여 양측 어깨의 윤곽선을 표시하고, Mark the outline of both shoulders using the upper outline of the back,
    등 부분에서 좌우측 윤곽선을 이용하여 양측 옆구리의 윤곽선을 표시하는 단계를 더 포함하는, Further comprising the step of marking the outlines of both sides using the left and right outlines in the back portion,
    방법.method.
  9. 제 8 항에 있어서, According to claim 8,
    양측 어깨의 윤곽선의 경사도 차이가 기 설정된 경사도 이상인 경우 더 올라온 어깨 부분을 어깨 올라옴으로 확인하고, 그리고If the difference in slope between the outlines of both shoulders is greater than the preset slope, the part of the shoulder that is raised more is confirmed as shoulder elevation, and
    양측 옆구리 윤곽선의 좌우 중심축을 향하여 들어감 정도 차이가 기 설정된 들어감 이상인 경우 더 들어간 부분을 옆구리 들어감으로 확인하는, If the difference in the degree of indentation toward the left and right central axes of the flank outlines on both sides is greater than the preset indentation, the further indentation is confirmed as side indentation.
    방법.method.
  10. 제 9 항에 있어서, According to clause 9,
    상기 가이드라인에 의해, According to the above guidelines,
    등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되고, Protrusion of one side of the back is confirmed by the protrusion line on one of the left and right sides of the back,
    양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되지 않고 The elevation of the other shoulder is not confirmed through the outline of both shoulders.
    양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되지 않는 경우, 제1형 척추측만증으로 확인되고, If one side is not confirmed through the outline of both sides, it is confirmed as type 1 scoliosis.
    등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되고, Protrusion of one side of the back is confirmed by the protrusion line on one of the left and right sides of the back,
    양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되고 The elevation of the other shoulder is confirmed through the outlines of both shoulders.
    양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되지 않는 경우, 제2형 척추측만증으로 확인되고, If one side is not confirmed through the outline of both sides, it is confirmed as type 2 scoliosis.
    등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되고, Protrusion of one side of the back is confirmed by the protrusion line on one of the left and right sides of the back,
    양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되지 않고 The elevation of the other shoulder is not confirmed through the outline of both shoulders.
    양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되는 경우, 제3형 척추측만증으로 확인되고, If unilateral side depression is confirmed through the outlines of both sides, it is confirmed as type 3 scoliosis.
    등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되고, Protrusion of one side of the back is confirmed by the protrusion line on one of the left and right sides of the back,
    양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되고 The elevation of the other shoulder is confirmed through the outlines of both shoulders.
    양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되는 경우, 제4형 척추측만증으로 확인되고, 그리고If unilateral side depression is confirmed through the outlines of both sides, it is confirmed as type 4 scoliosis, and
    등의 좌우측 중 일측의 튀어나옴 선에 의해 일측 등 튀어나옴이 확인되지 않고, Protrusion of one side of the back is not confirmed by the protrusion line on one side of the left and right sides of the back,
    양측 어깨의 윤곽선을 통해 타측 어깨 올라옴이 확인되지 않고 The elevation of the other shoulder is not confirmed through the outline of both shoulders.
    양측 옆구리의 윤곽선을 통해 일측 옆구리 들어감이 확인되는 경우, 제5형 척추측만증으로 확인되는, If one side is confirmed through the outline of both sides, it is confirmed as type 5 scoliosis.
    방법.method.
  11. 컴퓨터에 의해 제 1 항 내지 제 10 항 중 어느 한 항에 따른 방법이 수행되며, 컴퓨터 기록매체에 저장되는, 컴퓨터 프로그램.A computer program, wherein the method according to any one of claims 1 to 10 is performed by a computer and stored on a computer recording medium.
  12. 컴퓨터에 의해 제 1 항 내지 제 10 항 중 어느 한 항에 따른 방법이 수행되는 컴퓨터 프로그램이 저장되는, 컴퓨터 기록매체.A computer recording medium storing a computer program through which the method according to any one of claims 1 to 10 is performed by a computer.
PCT/KR2023/014735 2022-09-26 2023-09-26 Artificial intelligence-based method for providing information on scoliosis WO2024071943A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2022-0121666 2022-09-26
KR1020220121666A KR20240042866A (en) 2022-09-26 2022-09-26 Method for providing information on scoliosis based on artificial intelligence

Publications (1)

Publication Number Publication Date
WO2024071943A1 true WO2024071943A1 (en) 2024-04-04

Family

ID=90478727

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2023/014735 WO2024071943A1 (en) 2022-09-26 2023-09-26 Artificial intelligence-based method for providing information on scoliosis

Country Status (2)

Country Link
KR (1) KR20240042866A (en)
WO (1) WO2024071943A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150014226A (en) * 2013-07-29 2015-02-06 삼성전자주식회사 Electronic Device And Method For Taking Images Of The Same
KR102062539B1 (en) * 2019-03-06 2020-01-06 주식회사 딥노이드 Assistance diagnosis method for lumbar disease based on deep learning
KR102163701B1 (en) * 2020-04-17 2020-10-12 주식회사 딥노이드 Apparatus and method for classifying spine

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS6280676U (en) 1985-11-11 1987-05-23
KR100468142B1 (en) 2001-12-31 2005-01-26 장병욱 Carrier Mechanism
KR102181701B1 (en) 2017-10-24 2020-11-24 울산대학교 산학협력단 Apparatus for interpreting medical image and method for providing disease information
KR102047237B1 (en) 2017-12-13 2019-12-02 (주)엔텔스 Disease diagnosis method and system based on artificial intelligence analyzing image data
KR102354980B1 (en) 2018-06-11 2022-01-24 사회복지법인 삼성생명공익재단 The system for diagnosis of anterior eye diseases and method for diagnosis by the system
JP6711877B2 (en) 2018-09-07 2020-06-17 東芝エレベータ株式会社 Elevator inspection equipment
KR101968144B1 (en) 2018-10-11 2019-08-13 가톨릭대학교 산학협력단 Apparatus and method for diagnosising slant angle of vertebra and cervical vertebra
KR20200110878A (en) 2019-03-18 2020-09-28 민경준 A system and method for early diagnosing dental caries based on the deep learning
KR102251925B1 (en) 2019-07-18 2021-05-13 경상국립대학교 산학협력단 Apparatus and application for predicting musculoskeletal disorders
KR102214756B1 (en) 2019-12-20 2021-02-09 서울대학교치과병원 Mouth diagnosis system
JP2021115471A (en) 2020-01-21 2021-08-10 Ｐｏｓｅｎ株式会社 Posture diagnosis system, posture diagnosis method and data set for posture diagnosis
WO2021156645A1 (en) 2020-02-07 2021-08-12 James Young Solar panel and rail with edge connectors
KR102389067B1 (en) 2020-03-27 2022-04-20 연세대학교 산학협력단 Method for evaluating scoliosis and device evaluating scoliosis using the same
KR102383857B1 (en) 2020-05-14 2022-04-06 가천대학교 산학협력단 The method and the system for measuring Cobb's angle and the computer program thereof
KR102610915B1 (en) 2020-06-22 2023-12-06 한국전자통신연구원 Apparatus and method for analyzing medical image
KR102274330B1 (en) 2020-09-18 2021-07-07 가천대학교 산학협력단 Face Image Analysis Method and System for Diagnosing Stroke
KR102427505B1 (en) 2020-10-30 2022-08-01 이후경 User terminal configured to perform atopic dermatitis diagnosis based on deep learning
JP3234668U (en) 2021-08-17 2021-10-28 青島大学附属医院Ｔｈｅ Ａｆｆｉｌｉａｔｅｄ Ｈｏｓｐｉｔａｌ Ｏｆ Ｑｉｎｇｄａｏ Ｕｎｉｖｅｒｓｉｔｙ Image recognition system for scoliosis by X-ray
KR102388337B1 (en) 2022-01-25 2022-04-20 (주)힐링사운드 Service provision method of the application for temporomandibular joint disease improvement service

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150014226A (en) * 2013-07-29 2015-02-06 삼성전자주식회사 Electronic Device And Method For Taking Images Of The Same
KR102062539B1 (en) * 2019-03-06 2020-01-06 주식회사 딥노이드 Assistance diagnosis method for lumbar disease based on deep learning
KR102163701B1 (en) * 2020-04-17 2020-10-12 주식회사 딥노이드 Apparatus and method for classifying spine

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"Department of Medical Engineering, Konyang University Graduate School", 1 February 2022, DEPARTMENT OF MEDICAL ENGINEERING, KONYANG UNIVERSITY GRADUATE SCHOOL, KR, article KIM, JONG UN: "Evaluation of a deep learning model for scoliosis screening using preprocessing images of chest X-rays", pages: 1 - 48, XP009554399 *
RELIGHT: "Deepnoid - Artificial Intelligence Solutions for Medical and Industrial Use", NAVER BLOG, KR, KR, pages 1 - 5, XP009554393, Retrieved from the Internet <URL:https://blog.naver.com/relight0222/222881427739> [retrieved on 20240528] *

Also Published As

Publication number Publication date
KR20240042866A (en) 2024-04-02

Similar Documents

Publication Publication Date Title
WO2018056544A1 (en) Augmented reality system for dental surgery and implementation method therefor
JP6830082B2 (en) Dental analysis system and dental analysis X-ray system
McKenna et al. A method of matching skulls with photographic portraits using landmarks and measurements of the dentition
WO2021060700A1 (en) Apparatus and method for confirming videofluoroscopic swallowing study
US20030190067A1 (en) Apparatus, method, program, and system for displaying motion image, apparatus, method, program, and system for processing motion image, computer-readable storage medium, and method and system for assisting image diagnosis
WO2012033244A1 (en) Self-examination method and self-examination device
WO2013105815A1 (en) Fetus modeling method and image processing apparatus therefor
WO2017051944A1 (en) Method for increasing reading efficiency by using gaze information of user in medical image reading process and apparatus therefor
Islam et al. Deep learning of facial depth maps for obstructive sleep apnea prediction
WO2014077613A1 (en) Robot for repositioning procedure, and method for controlling operation thereof
WO2015186963A1 (en) Biological brain age calculation device and calculation method therefor
WO2020013642A1 (en) Oral care device and oral care service system including same
WO2017171295A1 (en) Augmented reality system in which estimation of jaw movement of patient is reflected and augmented reality providing method therefor
Rosati et al. The occlusal plane in the facial context: inter-operator repeatability of a new three-dimensional method
KR102388337B1 (en) Service provision method of the application for temporomandibular joint disease improvement service
Liu et al. Reliability of a three-dimensional facial camera for dental and medical applications: A pilot study
WO2022191575A1 (en) Simulation device and method based on face image matching
WO2021010777A1 (en) Apparatus and method for precise analysis of severity of arthritis
WO2023013805A1 (en) Method for deriving head measurement parameters for tooth correction diagnosis based on machine learning from three-dimensional cbct image captured at natural head position
WO2015147595A1 (en) Method for distinguishing artifact and diseased area in medical images
WO2024071943A1 (en) Artificial intelligence-based method for providing information on scoliosis
WO2019221586A1 (en) Medical image management system, method, and computer-readable recording medium
KR101801376B1 (en) Skull deformity analyzing system using a 3d topological descriptor and a method for analyzing skull deformity using the same
US5318441A (en) Method of cephalometric evaluation of dental radiographs
WO2023058994A1 (en) Method and device for orthodontic treatment result prediction based on deep learning

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23873058

Country of ref document: EP

Kind code of ref document: A1