CN116228727A

CN116228727A - Image-based human back epidermis spinal cord detection method and device

Info

Publication number: CN116228727A
Application number: CN202310246299.8A
Authority: CN
Inventors: 邓皓; 张雨新; 柳丛金; 张亮; 景富军; 刘勇
Original assignee: Yibin Micro Intelligent Technology Co ltd
Current assignee: Yibin Micro Intelligent Technology Co ltd
Priority date: 2023-03-15
Filing date: 2023-03-15
Publication date: 2023-06-06

Abstract

The application relates to the field of scoliosis detection, in particular to a detection method and a detection device of human back epidermis spinal cord based on images, wherein the method comprises the steps of obtaining back depth images of detected persons, and determining target point cloud data according to the back depth images; determining the preset number of characteristic point coordinates of the spine of the detected person in the back depth image according to the target point cloud data; performing curve fitting according to all the characteristic point coordinates to generate a target epidermis spine curve; according to the target epidermis spine curve, a target scoliosis angle is determined, whether the spine of the tested person is bent sideways is determined according to the angle value of the target scoliosis angle, the method is non-radiative in the whole process, has low requirements on proficiency of operators, can rapidly and efficiently obtain a result of whether the spine is bent sideways through deep learning and curve fitting, and is suitable for detecting scoliosis of large groups, particularly groups of students in middle and primary schools.

Description

Image-based human back epidermis spinal cord detection method and device

Technical Field

The application relates to the field of scoliosis detection, in particular to a detection method and device for human back epidermis spinal cord based on images.

Background

The human back epidermis spine is an important representation for judging the scoliosis condition of the human back, the scoliosis is a three-dimensional deformity of the spine, the three-dimensional deformity comprises a crown position, a sagittal position and an axial position, the abnormal sequences are frequently generated in children and teenagers, about 2% -4% of teenagers have scoliosis worldwide, at present, the scoliosis patient in China exceeds 300 ten thousand, the scoliosis patient increases at the speed of 30 ten thousand people per year, the national policy definitely indicates that the scoliosis of the primary and secondary school students is concerned with and ensured to develop healthily, and the scoliosis examination is listed in physical examination projects of the primary and secondary school students, so the human back epidermis spine is efficiently detected by utilizing an artificial intelligence technology, and the research on the scoliosis condition of the primary and secondary school students is judged to be urgent.

At present, the scoliosis detection in the physical examination of students in middle and primary schools is mainly carried out by using a scoliosis measuring instrument (scoliosis ruler) to carry out scoliosis screening through professional detection personnel. The existing home and abroad detection methods of scoliosis mainly comprise X-ray, EOS, ultrasound and moire patterns.

Although the X-ray has low cost, is simple, convenient and easy to operate, has more accurate measurement, has large radiation dose and has larger harm to the human body of middle and primary school people; EOS is costly and still has a small amount of radiation; the ultrasonic has high requirements on the doctor's manipulation, large error and low efficiency, and is not beneficial to large-scale screening; the moire map can only roughly judge the condition of the target.

Disclosure of Invention

In view of the problems, the present application has been developed in order to provide an image-based method and apparatus for detecting human back epidermis spinal lines that overcomes or at least partially solves the problems.

A method for detecting human back epidermis spinal cord based on an image, the method comprising:

acquiring a back depth image of a detected person, and determining target point cloud data according to the back depth image;

determining the preset number of characteristic point coordinates of the spine of the detected person in the back depth image according to the target point cloud data;

performing curve fitting according to all the characteristic point coordinates to generate a target epidermis spine curve;

and determining a target scoliosis angle according to the target epidermis spine curve, and determining whether the spine of the tested person is bent sideways according to the angle value of the target scoliosis angle.

Preferably, the acquiring a back depth image of the detected person, determining target point cloud data according to the back depth image, includes:

acquiring a back depth image of the detected person through a depth camera, and acquiring the back depth image from the depth camera;

obtaining calibrated internal parameters of the depth image, determining the three-dimensional coordinates of the point cloud corresponding to each pixel point in the back depth image according to the internal parameters,

and determining three-dimensional point cloud data according to all the three-dimensional coordinates of the point cloud, and carrying out noise reduction processing on the three-dimensional point cloud data by using a preset filtering algorithm to obtain the target point cloud data.

Preferably, the determining, according to the target point cloud data, a preset number of feature point coordinates of the spine of the detected person in the back depth image includes:

invoking a preset stacking hourglass network model, inputting the target point cloud data into the stacking hourglass network model, and training the stacking hourglass network model;

and outputting the preset number of feature point coordinates through the trained stacked hourglass network model.

Preferably, the calling the preset stacking hourglass network model includes:

determining the number of output neurons of each layer in the stacked hourglass network model and a proportionality coefficient after weight normalization;

and initializing the weight in the stacked hourglass network model by using a Kaiming network weight initialization method according to the number of the neurons and the proportionality coefficient.

Preferably, the generating the target epidermis spine curve according to curve fitting of all the feature point coordinates includes:

and performing polynomial interpolation fitting treatment on all the characteristic point coordinates by adopting an intersarc curve interpolation method to generate the target epidermis spine curve.

Preferably, the performing polynomial interpolation fitting processing on all the feature point coordinates by using an intersarc curve interpolation method includes:

obtaining an initial epidermis spine curve according to the intersarc curve interpolation method and the coordinate fitting of all the characteristic points;

determining a length value of the initial epidermis spinal curve, and equally dividing the initial epidermis spinal curve according to the length value;

performing linear interpolation on the initial epidermis spine curve subjected to the equipartition treatment to obtain all target characteristic point coordinates;

and fitting and generating the target epidermis spine curve according to all the target characteristic point coordinates and an interpolation curve interpolation method.

Preferably, the determining a target scoliosis angle according to the target epidermis spine curve, determining whether the spine of the tested person is scoliosis according to the angle value of the target scoliosis angle, includes:

determining a target normal vector according to the target epidermis spine curve;

determining a target scoliosis angle according to the target normal vector;

when the angle value of the target spine side bending angle is smaller than a preset side bending angle value, the spine of the detected person is normal; and when the angle value of the target scoliosis angle is larger than or equal to a preset scoliosis angle value, representing that the spine of the detected person is scoliosis.

Also provided is an image-based detection device for human back epidermis spinal cord, the device comprising:

the point cloud determining module is used for acquiring a back depth image of the detected person and determining target point cloud data according to the back depth image;

the coordinate determining module is used for determining the preset number of characteristic point coordinates of the spine of the detected person in the back depth image according to the target point cloud data;

the curve generation module is used for performing curve fitting according to all the characteristic point coordinates to generate a target epidermis spine curve;

and the scoliosis judging module is used for determining a target scoliosis angle according to the target epidermis spine curve and determining whether the spine of the tested person is scoliosis according to the angle value of the target scoliosis angle.

An apparatus comprising a processor, a memory and a computer program stored on the memory and capable of running on the processor, which when executed by the processor performs the steps of a method for image-based detection of human back epidermis spinal lines as described above.

A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of a method for detecting human back epidermis spinal cord based on an image as described above.

The application has the following advantages:

in the embodiment of the application, by acquiring a back depth image of a detected person, determining target point cloud data according to the back depth image, and determining a preset number of feature point coordinates of the spine of the detected person in the back depth image according to the target point cloud data; performing curve fitting according to all the characteristic point coordinates to generate a target epidermis spine curve; determining a target scoliosis angle according to the target epidermis spine curve, and determining whether the spine of the tested person is bent sideways according to the angle value of the target scoliosis angle; determining spine characteristic points and coordinates of a detected person in point cloud data of a back depth image, and further performing curve fitting on all the characteristic point coordinates to directly determine a target scoliosis angle from a target epidermis spine curve generated by fitting, so as to judge whether the spine of the detected person is bent sideways according to the target scoliosis angle; the scheme is non-radiative in the whole process, has low requirements on proficiency of operators, can quickly and efficiently obtain a result of whether lateral curvature is formed or not through deep learning and curve fitting, and is suitable for detecting the lateral curvature of the spine of a large group, particularly a group of students in middle and primary schools.

Drawings

In order to more clearly illustrate the technical solutions of the present application, the drawings that are needed in the description of the present application will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.

Fig. 1 is a flowchart illustrating steps of a method for detecting a human back epidermis spine line based on an image according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of a detection device for human back epidermis spine line based on an image according to an embodiment of the present application;

fig. 3 shows a schematic structural diagram of a computer device of a method for detecting a human back epidermis spine line based on an image according to the present invention.

Detailed Description

In order to make the objects, features and advantages of the present application more comprehensible, the present application is described in further detail below with reference to the accompanying drawings and detailed description. It will be apparent that the embodiments described are some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

Aiming at the high-incidence condition of scoliosis of the current students in middle and primary schools, the existing detection method such as X-ray, EOS, ultrasound, moire patterns and the like has application limitations, and the application provides a scoliosis condition to be detected based on a computer vision image and combined with a deep learning model so as to analyze the scoliosis condition to be detected, thereby achieving the purposes of zero radiation, low error and high efficiency of scoliosis detection.

Referring to fig. 1, a flowchart of steps of a method for detecting a human back epidermis spine line based on an image according to an embodiment of the present application is shown. The method comprises the following steps:

s110, acquiring a back depth image of a detected person, and determining target point cloud data according to the back depth image;

s120, determining the preset number of characteristic point coordinates of the spine of the detected person in the back depth image according to the target point cloud data;

s130, performing curve fitting according to all the characteristic point coordinates to generate a target epidermis spine curve;

and S140, determining a target scoliosis angle according to the target epidermis spinal curve, and determining whether the spinal column of the tested person is bent sideways according to the angle value of the target scoliosis angle.

The image features of the back of the human body are diversified, and the coordinates of the feature points change along with the change of the standing posture of the human body. For example, the actions of shrugging, bending over or twisting over the crotch of the human body can cause the fitted epidermis spinal curve to assume different morphologies.

Therefore, in order to reduce the difficulty of feature point prediction, when the back depth image of the detected person is acquired, the two arms need to naturally droop when the detected person stands; because the application relates to depth images, the depth image is sensitive to shooting distance, a detector needs to stand at a position 70cm plus or minus 5cm away from the front of a camera, and the back plane is kept parallel to the shooting plane of the camera.

Next, a method for detecting a human back epidermis spinal line based on an image as described above will be further described by the following examples.

In step S110, a back depth image of a person to be detected is acquired, and target point cloud data is determined according to the back depth image.

In one embodiment of the present invention, the specific process of determining the target point cloud data from the back depth image in step S110 may be further described in conjunction with the following description.

As described in the following steps:

As an example, a Kinect depth camera may be used to capture depth images of a person under test when the back is exposed, and calibration may be performed on the Kinect depth camera to obtain internal parameters of the camera before the images are captured. The two arms of the detected person naturally drop, stand at the position 70cm in front of the shooting of the camera, and the lower corner of the scapula of the detected person corresponds to the level of the depth camera.

Because the three-dimensional point cloud data is influenced by environmental conditions and camera quality, more noise points often exist, and the three-dimensional point cloud data is more remarkable at the edge of the human body contour; the filtering algorithm can adopt a bilateral filtering algorithm, can combine the distance and the space structure to remove noise, and has good effect.

In step S120, a preset number of feature point coordinates of the spine of the detected person in the back depth image are determined according to the target point cloud data.

In an embodiment of the present invention, the specific process of "determining the preset number of feature point coordinates of the spine of the subject in the back depth image" in step S120 may be further described in conjunction with the following description.

As described in the following steps:

In one embodiment, the model effect is verified by the test set during training of the stacked hourglass network model. After the stacking hourglass network model is trained, 20 characteristic points of the epidermis spinal lines can be output in the horizontal coordinate and the vertical coordinate, and 40 data are obtained in total.

It should be noted that, the stacked hourglass network model (Stacked Hourglass Networks) is a deep neural network model that is important in pose estimation in the field of computer vision, and in order to prevent the situations of variance attenuation and gradient disappearance of the activation values in the forward propagation process of the network, a Kaiming network weight initialization method may be used to initialize the network weights in the stacked hourglass network model.

As an example, the invoking the preset stacked hourglass network model includes:

The calculation formula for initializing the weight in the stacked hourglass network model is as follows:

wherein W is _ij And n is the number of output neurons of each layer, and U is the proportionality coefficient after weight normalization.

Through the calculation formula, the gradient disappearance of the stacked hourglass network model is solved, the variance attenuation of the activation values is avoided, and the activation values of each layer are kept to be in Gaussian distribution.

And step S130, performing curve fitting according to all the characteristic point coordinates to generate a target epidermis spine curve.

In an embodiment of the present invention, the specific process of "generating the target epidermis spine curve according to all the feature point coordinates" in step S130 may be further described in conjunction with the following description.

And (3) performing polynomial interpolation fitting processing on all the characteristic point coordinates by adopting an intersarc curve interpolation method to generate the target epidermis spine curve.

As an example, the above-mentioned characteristic point coordinates of the 20 epidermis spinal lines are subjected to a polynomial interpolation fitting process, and the calculation formula is as follows:

wherein x is the feature point coordinates, and w is the polynomial coefficient.

In this embodiment of the present application, performing polynomial interpolation fitting processing on all the feature point coordinates by using an Interparc curve interpolation method includes:

As an example, the calculation formula for performing the equipartition processing on the initial epidermis spine curve is as follows:

in step S140, a target scoliosis angle is determined according to the target epidermis spine curve, and whether the spine of the tested person is scoliosis is determined according to the angle value of the target scoliosis angle.

In an embodiment of the present invention, the specific process of "determining a target scoliosis angle according to the target epidermis spine curve, determining whether the tested person is scoliosis according to the angle value of the target scoliosis angle" in step S140 may be further described in conjunction with the following description.

As will be described in the following steps,

determining a target scoliosis angle according to the target normal vector;

As an example, the calculation formula for determining the target scoliosis angle by the target normal vector is as follows:

wherein v is _i ,v _j And theta is the angle of the target scoliosis angle and is the target normal vector.

It should be noted that the target scoliosis angle may be a Cobb angle, and the preset scoliosis angle value may be 12 °. When the Cobb angle is smaller than 12 degrees, the spine of the detected person is normal; when the Cobb angle is greater than or equal to 12 degrees, the spine of the tested person is lateral bending.

For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.

Referring to fig. 2, a schematic structural diagram of a detection device for human back epidermis spine line based on an image according to an embodiment of the present application is shown;

the device comprises:

the point cloud determining module 110 is configured to obtain a back depth image of a person to be detected, and determine target point cloud data according to the back depth image;

the coordinate determining module 120 is configured to determine a preset number of feature point coordinates of the spine of the detected person in the back depth image according to the target point cloud data;

the curve generating module 130 is configured to perform curve fitting according to all the feature point coordinates, and generate a target epidermis spine curve;

the scoliosis judging module 140 is configured to determine a target scoliosis angle according to the target epidermis spine curve, and determine whether the spine of the tested person is scoliosis according to an angle value of the target scoliosis angle.

Referring to fig. 3, a schematic structural diagram of a computer device of the method for detecting a human back epidermis spine line based on an image according to the present invention may specifically include the following steps:

the computer device 12 described above is embodied in the form of a general purpose computing device, and the components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, a bus 18 that connects the various system components, including the system memory 28 and the processing units 16.

Bus 18 represents one or more of several types of bus 18 structures, including a memory bus 18 or memory controller, a peripheral bus 18, an accelerated graphics port, a processor, or a local bus 18 using any of a variety of bus 18 architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus 18, micro channel architecture (MAC) bus 18, enhanced ISA bus 18, video Electronics Standards Association (VESA) local bus 18, and Peripheral Component Interconnect (PCI) bus 18.

Computer device 12 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by computer device 12 and includes both volatile and nonvolatile media, removable and non-removable media.

The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and/or cache memory 32. The computer device 12 may further include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only, storage system 34 may be used to read from or write to non-removable, nonvolatile magnetic media (commonly referred to as a "hard disk drive"). Although not shown in fig. 3, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable non-volatile optical disk such as a CD-ROM, DVD-ROM, or other optical media may be provided. In such cases, each drive may be coupled to bus 18 through one or more data medium interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 42, the program modules 42 being configured to carry out the functions of embodiments of the invention.

A program/utility 40 having a set (at least one) of program modules 42 may be stored in, for example, a memory, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules 42, and program data, each or some combination of which may include an implementation of a network environment. Program modules 42 generally perform the functions and/or methods of the embodiments described herein.

The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, camera, etc.), one or more devices that enable a user to interact with the computer device 12, and/or any devices (e.g., network card, modem, etc.) that enable the computer device 12 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 22. Moreover, computer device 12 may also communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet, through network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be appreciated that although not shown in fig. 3, other hardware and/or software modules may be used in connection with computer device 12, including, but not limited to: microcode, device drivers, redundant processing units 16, external disk drive arrays, RAID systems, tape drives, data backup storage systems 34, and the like.

The processing unit 16 executes programs stored in the system memory 28 to perform various functional applications and data processing, for example, to implement an image-based method for detecting a human back epidermis spinal line according to an embodiment of the present invention.

That is, the processing unit 16 realizes when executing the program: acquiring a back depth image of a detected person, and determining target point cloud data according to the back depth image; determining the preset number of characteristic point coordinates of the spine of the detected person in the back depth image according to the target point cloud data; performing curve fitting according to all the characteristic point coordinates to generate a target epidermis spine curve; and determining a target scoliosis angle according to the target epidermis spine curve, and determining whether the spine of the tested person is bent sideways according to the angle value of the target scoliosis angle.

In an embodiment of the present invention, the present invention further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method for detecting a human back epidermis spinal line based on an image as provided in all embodiments of the present application:

that is, the program is implemented when executed by a processor: acquiring a back depth image of a detected person, and determining target point cloud data according to the back depth image; determining the preset number of characteristic point coordinates of the spine of the detected person in the back depth image according to the target point cloud data; performing curve fitting according to all the characteristic point coordinates to generate a target epidermis spine curve; and determining a target scoliosis angle according to the target epidermis spine curve, and determining whether the spine of the tested person is bent sideways according to the angle value of the target scoliosis angle.

Any combination of one or more computer readable media may be employed. The computer readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.

While preferred embodiments of the present embodiments have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the present application.

Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.

The above description is made in detail on the detection method and device of human back epidermis spine line based on the image provided in the present application, and specific examples are applied herein to illustrate the principles and embodiments of the present application, and the description of the above examples is only used to help understand the method and core ideas of the present application; meanwhile, as those skilled in the art will have modifications in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.

Claims

1. An image-based detection method for human back epidermis spinal lines, which is characterized by comprising the following steps:

2. The method of claim 1, wherein the acquiring the back depth image of the subject, determining target point cloud data from the back depth image, comprises:

3. The method of claim 1, wherein the determining a preset number of feature point coordinates of the subject's spine in the back depth image from the target point cloud data comprises:

4. The method of claim 3, wherein the invoking the pre-set stacked hourglass network model, thereafter comprises:

5. The method of claim 1, wherein said generating a target epidermis spine curve by curve fitting based on all of said feature point coordinates comprises:

6. The method of claim 5, wherein performing polynomial interpolation fitting on all the feature point coordinates using an interpolation curve method comprises:

7. The method of claim 1, wherein determining a target scoliosis angle from the target epidermis spine curve, determining whether the subject's spine is scoliotic from an angle value of the target scoliosis angle, comprises:

determining a target scoliosis angle according to the target normal vector;

8. An image-based detection device for human back epidermis spinal cord, the device comprising:

9. An apparatus comprising a processor, a memory, and a computer program stored on the memory and capable of running on the processor, which when executed by the processor, implements the method of any one of claims 1 to 7.

10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the method according to any of claims 1 to 7.