CN113487626A

CN113487626A - Mirror image identification method and device, electronic equipment and storage medium

Info

Publication number: CN113487626A
Application number: CN202110745864.6A
Authority: CN
Inventors: 李先红; 沈丽萍; 李明; 陈汉清; 徐琦
Original assignee: Hangzhou Santan Medical Technology Co Ltd
Current assignee: Hangzhou Santan Medical Technology Co Ltd
Priority date: 2021-07-01
Filing date: 2021-07-01
Publication date: 2021-10-08
Anticipated expiration: 2041-07-01
Also published as: CN113487626B

Abstract

The embodiment of the invention provides a method and a device for identifying a mirror image, electronic equipment and a storage medium, relates to the technical field of image processing, and can automatically acquire an X-ray image which is not mirrored. The embodiment of the invention comprises the following steps: and determining the sphere center pixel point of the sphere center of each Mark sphere in the X-ray image to be identified. And then projecting the position of the sphere center of each Mark ball under the flange coordinate system into an X-ray image coordinate system to determine the sphere center projection point of each Mark ball. Then, a first matching error between the sphere center pixel point set and the sphere center projection point set is determined. And then carrying out mirror image transformation on the X-ray image to be recognized according to a preset mirror image mode to obtain a mirror image X-ray image, and determining a mirror image sphere center pixel point of the sphere center of each Mark ball in the mirror image X-ray image. And simultaneously determining a second matching error between the mirror image sphere center pixel point set and the sphere center projection point set. The image corresponding to the smallest matching error is then determined to be the X-ray image that has not been mirrored.

Description

Mirror image identification method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to a method and an apparatus for identifying a mirror image, an electronic device, and a storage medium.

Background

The C-arm X-ray machine is also called C-arm, is an X-ray imaging device with the shape similar to English letter C and used for interventional radiology and orthopedic surgery. The C-shaped arm can provide basis for diagnosis of doctors and decision of the surgical robot in the surgical process aiming at X-ray images shot by human bodies.

If the doctor can click the mirror image button of the C-shaped arm for conveniently observing the X-ray image, and accordingly, the C-shaped arm converts the shot X-ray image into a mirror image. However, the surgical robot cannot determine whether the acquired X-ray image captured by the C-arm is a mirror image, which results in inaccurate positioning of the surgical robot for the surgical position.

Currently, it can only be determined whether the X-ray image is a mirror image by means of manual identification. For example, a doctor who takes an X-ray image is asked whether a mirror button is pressed when taking an X-ray image, or whether an X-ray image is a mirror image is manually judged based on experience by comparing the X-ray image with an actual object. After the X-ray is determined to be a mirror image, the X-ray image is converted into an un-mirrored X-ray image. As can be seen, this way of obtaining an un-mirrored X-ray image relies on manual judgment and is inefficient.

Disclosure of Invention

The embodiment of the invention aims to provide a mirror image identification method, a mirror image identification device, electronic equipment and a storage medium, so as to automatically acquire an X-ray image which is not mirrored, and further improve acquisition efficiency. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a method for identifying a mirror image, where the method includes:

acquiring an X-ray image to be identified, wherein the X-ray image to be identified is an image obtained by transmitting X-rays emitted by a C-shaped arm through a plurality of Mark balls;

determining a spherical center pixel point of the spherical center of each Mark ball in the X-ray image to be recognized to obtain a spherical center pixel point set;

projecting the position of the sphere center of each Mark ball under a flange coordinate system into an X-ray image coordinate system, and determining the sphere center projection point of each Mark ball to obtain a sphere center projection point set;

determining a first match error between the set of sphere center pixel points and the set of sphere center projection points;

carrying out mirror image transformation on the X-ray image to be identified according to a preset mirror image mode to obtain a mirror image X-ray image, and determining mirror image sphere center pixel points of the sphere center of each Mark ball in the mirror image X-ray image to obtain a mirror image sphere center pixel point set;

determining a second matching error between the mirror image sphere center pixel point set and the sphere center projection point set;

and determining the image corresponding to the minimum matching error in the first matching error and the second matching error as the X-ray image which is shot by the C-shaped arm and is not mirrored.

Optionally, the determining a center pixel point of the center of each Mark ball in the X-ray image to be recognized includes:

carrying out contrast enhancement on the X-ray image to be identified to obtain an enhanced image;

performing binary segmentation on the enhanced image to obtain a binary image;

and performing circle detection on the binary image to obtain the circle center of each circle in the binary image, and taking the circle center of each circle as a sphere center pixel point of the sphere center of a Mark ball in the X-ray image to be recognized.

Optionally, the determining a first matching error between the set of sphere center pixel points and the set of sphere center projection points includes:

aiming at each spherical center pixel point, calculating the distance between the spherical center pixel point and each spherical center projection point to obtain the error corresponding to the spherical center pixel point;

determining a weighted sum of errors corresponding to the pixel points of the spherical centers as the first matching error through the specified weight parameters;

said determining a second match error between said set of sphere center pixel points and said set of sphere center projection points comprises:

calculating the distance between the pixel point of the sphere center of the mirror image and the projection point of each sphere center aiming at the pixel point of the sphere center of the mirror image to obtain the corresponding error of the pixel point of the sphere center of the mirror image;

and determining the weighted sum of the errors corresponding to the pixel points of the spherical centers of the mirror images through the appointed weight parameters as the second matching error.

Optionally, performing mirror image transformation on the X-ray image to be identified according to a preset mirror image mode to obtain a mirror image X-ray image, including:

carrying out mirror image transformation on the X-ray image to be identified by taking an X-axis as a rotating shaft to obtain a first mirror image X-ray image, wherein the X-axis is a horizontal axis passing through the center point of the X-ray image to be identified;

and/or carrying out mirror image transformation on the X-ray image to be recognized by taking a Y axis as a rotating shaft to obtain a second mirror image X-ray image, wherein the Y axis is a vertical axis passing through the center point of the X-ray image to be recognized.

Optionally, the determining a mirror image spherical center pixel point of the spherical center of each Mark ball in the mirror image X-ray image to obtain a mirror image spherical center pixel point set includes:

determining a first mirror image spherical center pixel point of the spherical center of each Mark ball in the first mirror image X-ray image to obtain a first mirror image spherical center pixel point set;

and/or determining a second mirror image sphere center pixel point of the sphere center of each Mark ball in the second mirror image X-ray image to obtain a second mirror image sphere center pixel point set;

the determining a second matching error between the mirror image sphere center pixel point set and the sphere center projection point set includes:

determining a second X-axis mirror image matching error between the first mirror image center pixel point set and the center projection point set;

and/or determining a second Y-axis mirror image matching error between the second mirror image sphere center pixel point set and the sphere center projection point set.

The determining that the image corresponding to the smallest matching error of the first matching error and the second matching error is an X-ray image that is captured by the C-arm and is not mirrored includes:

and taking the image corresponding to the minimum matching error in the first matching error, the second X-axis mirror image matching error and the second Y-axis mirror image matching error as the X-ray image which is not mirrored.

Optionally, after the acquiring the X-ray image to be recognized, the method further includes:

rotating the to-be-identified X-ray image along an image central point by a preset angle to obtain a rotating X-ray image, and determining a rotating spherical center pixel point of the spherical center of each Mark ball in the rotating X-ray image to obtain a rotating spherical center pixel point set;

determining a third match error between the set of rotational centroid pixel points and the set of centroid projection points;

and determining the image corresponding to the minimum matching error in the first matching error and the third matching error as the X-ray image which is not rotated and shot by the C-shaped arm.

In a second aspect, an embodiment of the present invention provides a mirror image recognition apparatus, where the apparatus includes:

the device comprises an acquisition module, a detection module and a display module, wherein the acquisition module is used for acquiring an X-ray image to be identified, and the X-ray image to be identified is an image obtained by transmitting X-rays emitted by a C-shaped arm through a plurality of Mark balls;

the determining module is used for determining the center of sphere pixel points of the center of sphere of each Mark ball in the X-ray image to be recognized to obtain a center of sphere pixel point set;

the determining module is further configured to project the position of the center of sphere of each Mark ball in the flange coordinate system into an X-ray image coordinate system, and determine a center projection point of each Mark ball to obtain a center projection point set;

the determining module is further configured to determine a first matching error between the set of sphere center pixel points and the set of sphere center projection points;

the mirror image module is used for carrying out mirror image transformation on the X-ray image to be identified according to a preset mirror image mode to obtain a mirror image X-ray image, and determining mirror image sphere center pixel points of the sphere center of each Mark ball in the mirror image X-ray image to obtain a mirror image sphere center pixel point set;

the determining module is further configured to determine a second matching error between the mirror image centroid pixel point set and the centroid projection point set;

the determining module is further configured to determine that an image corresponding to a minimum matching error of the first matching error and the second matching error is an X-ray image that is captured by the C-arm and is not mirrored.

Optionally, the determining module is specifically configured to:

performing binary segmentation on the enhanced image to obtain a binary image;

Optionally, the determining module is specifically configured to:

the determining module is specifically configured to:

Optionally, the mirror module is specifically configured to:

the determining module is specifically configured to:

The determining module is specifically configured to:

Optionally, the apparatus further comprises: a rotation module;

the rotation module is used for rotating the X-ray image to be identified by a preset angle along an image central point after the X-ray image to be identified is obtained, so as to obtain a rotation X-ray image, and determining a rotation center pixel point of the center of each Mark ball in the rotation X-ray image, so as to obtain a rotation center pixel point set;

the determining module is further configured to determine a third matching error between the set of rotational centroid pixel points and the set of centroid projection points;

the determining module is further configured to determine that an image corresponding to a minimum matching error of the first matching error and the third matching error is an X-ray image captured by the C-arm and not rotated.

In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;

a memory for storing a computer program;

and the processor is used for realizing the steps of any image identification method when executing the program stored in the memory.

In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements the steps of any of the image recognition methods described above.

In a fifth aspect, embodiments of the present invention further provide a computer program product containing instructions, which when run on a computer, cause the computer to perform any of the above-mentioned mirror image recognition methods.

The mirror image identification method, the mirror image identification device, the electronic equipment and the storage medium provided by the embodiment of the invention can perform mirror image transformation on an X-ray image to be identified to obtain a mirror image X-ray image. Respectively determining a first matching error between a central pixel point set of a Mark ball and a central projection point set of the Mark ball in an X-ray image to be identified and a second matching error between a mirror image central pixel point set of the Mark ball and a central projection point set of the Mark ball in a mirror image X-ray image. Since the center of sphere projection point is determined based on the position of the Mark ball in the flange coordinate system, and the flange coordinate system is the world coordinate system, the actual physical position of the Mark ball can be represented, so the smaller the matching error, the higher the possibility that the image is an X-ray image that is not mirrored. Therefore, the embodiment of the invention takes the image with the minimum matching error as the X-ray image which is shot by the C-shaped arm and is not mirrored. Therefore, the X-ray image which is not mirrored can be automatically acquired, and the acquisition efficiency is improved.

Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by referring to these drawings.

Fig. 1 is a flowchart of a method for identifying a mirror image according to an embodiment of the present invention;

FIG. 2 is an exemplary schematic diagram of an X-ray image provided by an embodiment of the present invention;

FIG. 3 is a flowchart of a method for identifying a rotated image according to an embodiment of the present invention;

FIG. 4 is an exemplary schematic diagram of another X-ray image provided by embodiments of the present invention;

FIG. 5 is a flowchart of another method for identifying a mirror image according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of a mirror image recognition apparatus according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived from the embodiments given herein by one of ordinary skill in the art, are within the scope of the invention.

In order to automatically acquire an X-ray image that is not mirrored, so as to improve acquisition efficiency, embodiments of the present invention provide a mirrored image identification method, which may be applied to an electronic device. Wherein the electronic device may be: a computer, server, tablet, or other device with image processing capabilities.

As shown in fig. 1, the mirror image identification method provided in the embodiment of the present invention includes the following steps:

and S101, acquiring an X-ray image to be identified.

The X-ray image to be identified is an image obtained by transmitting X-rays emitted by the C-shaped arm through a plurality of Mark balls.

S101, the obtained to-be-identified X-ray image is an X-ray image sent by the C-shaped arm to the electronic equipment, and if mirror image shooting is set on the C-shaped arm, the to-be-identified X-ray image is a mirror image of an original X-ray image shot by the C-shaped arm. If the C-type arm is not set to mirror image shooting, the image to be recognized is an original X-ray image which is shot by the C-type arm and is not mirrored.

S102, determining the center of sphere pixel points of the center of sphere of each Mark ball in the X-ray image to be recognized to obtain a center of sphere pixel point set.

In one embodiment, contrast enhancement may be performed on an X-ray image to be identified to obtain an enhanced image, and then binary segmentation may be performed on the enhanced image to obtain a binary image. And then, performing circle detection on the binary image to obtain the circle center of each circle in the binary image, and taking the circle center of each circle as a sphere center pixel point of the center of the Mark sphere in the X-ray image to be recognized.

Optionally, an Automatic Color Equalization (ACE) algorithm may be used to enhance the contrast of the image to be recognized. Alternatively, other contrast enhancement algorithms may be used, and the embodiment of the present invention is not limited in this respect. Contrast enhancement is carried out on the X-ray image to be recognized, so that the difference between the color value of the Mark ball area in the X-ray image to be recognized and the color value of other areas is larger, namely the contrast is more obvious.

The way of performing binary segmentation on the enhanced image can be implemented as follows: and determining the pixel points with the gray value smaller than the segmentation threshold value in the enhanced image as foreground pixel points, and updating the gray value of the foreground pixel points to be 0. And determining the pixel points with the gray value larger than or equal to the segmentation threshold value in the enhanced image as background pixel points, and updating the gray value of the background pixel points to be 255. For example, the grayscale threshold may be 128. The gray threshold may be other values, specifically set based on actual needs.

Optionally, a hough circle detection algorithm may be adopted, and based on a preset threshold, circle detection may be performed on the binary image. Alternatively, other circle detection algorithms, such as a circle fitting algorithm, may also be used, and the embodiment of the present invention is not limited in this respect.

Optionally, when the spherical center pixel point set is determined, the to-be-identified X-ray image may be copied first to obtain a copied image of the to-be-identified X-ray image, and contrast enhancement, binary segmentation, and circle detection are performed based on the copied image, so as to avoid damaging the obtained to-be-identified X-ray image.

S103, projecting the position of the sphere center of each Mark ball under the flange coordinate system into an X-ray image coordinate system, and determining the sphere center projection point of each Mark ball to obtain a sphere center projection point set.

The flange coordinate system in S103 is a flange coordinate system of a robot arm of the C-arm, and the robot arm is used for performing an operation on a patient. The X-ray image coordinate system in S103 refers to a coordinate system of an original X-ray image theoretically captured by the C-arm.

S104, determining a first matching error between the sphere center pixel point set and the sphere center projection point set.

In an embodiment of the present invention, the first matching error represents an error between each of the center pixel points and each of the center projection points.

S105, carrying out mirror image transformation on the X-ray image to be recognized according to a preset mirror image mode to obtain a mirror image X-ray image, and determining mirror image sphere center pixel points of the sphere center of each Mark ball in the mirror image X-ray image to obtain a mirror image sphere center pixel point set.

Optionally, a mode of determining a mirror image sphere center pixel point of the sphere center of each Mark sphere in the mirror image X-ray image is the same as the mode of determining the sphere center pixel point in S102, and reference may be made to the description in S102, which is not described herein again.

And S106, determining a second matching error between the mirror image sphere center pixel point set and the sphere center projection point set.

In an embodiment of the present invention, the second matching error represents an error between each mirror image centroid pixel point and each centroid projection point.

S107, determining the image corresponding to the minimum matching error in the first matching error and the second matching error as the X-ray image which is shot by the C-shaped arm and is not mirrored.

The image identification method provided by the embodiment of the invention can perform image transformation on the X-ray image to be identified to obtain the image X-ray image. Respectively determining a first matching error between a central pixel point set of a Mark ball and a central projection point set of the Mark ball in an X-ray image to be identified and a second matching error between a mirror image central pixel point set of the Mark ball and a central projection point set of the Mark ball in a mirror image X-ray image. Since the center of sphere projection point is determined based on the position of the Mark ball in the flange coordinate system, and the flange coordinate system is the world coordinate system, the actual physical position of the Mark ball can be represented, so the smaller the matching error, the higher the possibility that the image is an X-ray image that is not mirrored. Therefore, the embodiment of the invention takes the image with the minimum matching error as the X-ray image which is shot by the C-shaped arm and is not mirrored. Therefore, the X-ray image which is not mirrored can be automatically acquired, and the acquisition efficiency is improved.

The following describes, by way of an example, a manner in which the circle detection is performed on the binary image in S102 to obtain the center of each circle in the binary image:

the center of each circle in the binary image is determined by the following function:

void HoughCircles(InputArray image,OutputArray circles,int method,double dp,double minDist,double param1＝100,double param2＝100,int minRadius＝0,int maxRadius＝0)；

where void represents no type, HoughCircles is a function name of the hough circle detection algorithm, inputcurray image represents an input image (the binary image in the embodiment of the present invention), outputcurray circuits represent circular output vectors, each of which is represented by a floating point vector (x, y, radius) including 3 elements, where (x, y) represents coordinates of a center of a detected circle, and radius represents a radius of the circle. int method represents a detection method used, for example, HOUGH GRADIENT method in OpenCV, and an identifier of HOUGH GRADIENT method is HOUGH _ GRADIENT. The double dp indicates the size of the cumulative plane resolution, and when the size of the cumulative plane resolution is equal to the size × 1/dp of the original input image resolution, the resolutions of the cumulative plane resolution and the original input image resolution are the same when the default dp is 1. double minDist represents the minimum distance between two circle centers. And if the distance between the two circle centers is less than minDist, determining that the two circles are the same circle. double param1 is 100 indicating a high threshold for Canny edge detection, with a default high threshold of 100, wherein a low threshold for Canny edge detection is half of the high threshold by default, i.e., a low threshold of 50. The double param2 is a determination threshold value indicating whether or not a point on the accumulation plane is a center of a circle, the determination threshold value is default to 100, and the larger the determination threshold value is, the closer the detected circle is to the normative circle. int minus radius ═ 0 denotes the minimum value of the circle radius, which defaults to 0. int maxRadius ═ 0 denotes the maximum value of the circle radius, which defaults to 0. When both the maximum value and the minimum value of the circle radius are 0, it means that the radius of the detected circle is not limited.

The function can detect the circle center of a circle in the binary image, and the image of the Mark ball in the binary image is a circle and other circles can exist, so that when the binary image is subjected to circle detection, the radius range of the detected circle can be limited by setting values of two parameters, namely minus radius and maxRadius, and the center pixel point of the Mark ball can be determined more accurately.

Or, the circle of the binary image may be detected by using the function based on default values of minRadius and maxRadius, and then the center pixel point of the Mark ball may be further obtained by screening based on the radius of the circle output by the circle detection.

In the embodiment of the present invention, the manner of determining the center projection point of each Mark ball in S103 may be implemented by the following formulas (1) to (9):

x′＝x/z (3)

y′＝y/z (4)

where r²＝x²+y² (7)

u＝f_x*x″+c_x (8)

v＝f_y*y″+c_y (9)

where s is a predetermined scale factor, u and v denote coordinates on two coordinate axes of the X-ray image coordinate system, X, Y and Z denote coordinates on three coordinate axes of the flange coordinate system, and f_x、f_y、c_xAnd c_yIs the internal reference of C-shaped arm, r₁₁～r₃₃For the rotation parameters from the flange coordinate system to the C-arm camera coordinate system, t₁～t₃For the translation parameters from the flange coordinate system to the C-arm camera coordinate system, r₁₁～r₃₃And t₁～t₃The formed matrix can be called an external reference matrix of the C-type arm, x, y and z are coordinates on three coordinate axes of a C-type arm camera coordinate system, and R is from a flange coordinate system toRotation parameters of the C-arm camera coordinate system, t is translation parameters from the flange coordinate system to the C-arm camera coordinate system, k₁～k₆And p₁And p₂For the distortion parameter of the C-arm, where represents the constraint, that is, equation (7) is the constraint of equation (5) and equation (6).

In an embodiment of the present invention, the manner of determining the first matching error between the set of spherical center pixel points and the set of spherical center projection points in S104 may be implemented as follows: and calculating the distance between each spherical center pixel point and each spherical center projection point aiming at each spherical center pixel point to obtain the error corresponding to the spherical center pixel point, and determining the weighted sum of the errors corresponding to the spherical center pixel points through the specified weight parameters to be used as a first matching error.

In one embodiment, the first match error may be calculated by the following equation (10):

where E is the first match error, n_xThe number of center-of-sphere pixels included for a set of center-of-sphere pixels, n_yThe number of centre of sphere projection points, m, included for the set of centre of sphere projection points_ijWeight of error between ith sphere pixel point and jth sphere projection point, x_iIs the coordinate of the ith sphere center pixel point, y_jAnd (C) the coordinates of the jth sphere center projection point, R is a rotation parameter from the flange coordinate system to the C-arm camera coordinate system, t is a translation parameter from the flange coordinate system to the C-arm camera coordinate system, pro is a projection function, k is the number of iterations, | · |, represents a norm operation.

Determining a specified error parameter m_ijThe method of (3) can be realized as: calculating m by equation (11)_i,jWhere equation (12) is a constraint of equation (11), then m is paired by equation (13)_i,jIs normalized and m is represented by the formula (14)_i,jNormalizing the parameters of each column to obtain normalized m_i,j. M after normalization_i,jSubstituting into the formula (10), obtainE of the previous calculation judges whether E of the current calculation meets the iteration termination condition, if not, the formula (11) is continuously calculated until m of the current normalization is obtained when E of the current calculation meets the iteration termination condition_i,jAs specified error parameters.

Wherein the iteration termination condition comprises: and the difference value between the currently calculated E and the last calculated E in the iteration process is smaller than a preset difference value, and/or the number of times of currently calculating E exceeds the preset number of times.

Where E is the first match error, n_xThe number of center-of-sphere pixels included for a set of center-of-sphere pixels, n_yThe number of centre of sphere projection points, m, included for the set of centre of sphere projection points_ijWeight of error between ith sphere pixel point and jth sphere projection point, x_iIs the coordinate of the ith sphere center pixel point, y_jIs the coordinate of the jth sphere center projection point, R is the rotation parameter from the flange coordinate system to the C-arm camera coordinate system, t is the translation parameter from the flange coordinate system to the C-arm camera coordinate system, k is the iteration number, and exp represents an exponential function with e as the base.

In the embodiment of the invention, the specified error parameter m is obtained_ijThereafter, can be for m_ijAdding the spherical center pixel point i corresponding to the maximum parameter in the row of parameters into the point set X, and adding the row of parametersAnd adding a point set Y into the sphere center projection point j corresponding to the medium and maximum parameters, wherein the sphere center pixel point i and the sphere center projection point j form a pair of matching points. And (3) updating the external parameter matrix of the C-shaped arm by using a formula (1) based on each pair of matched points, and taking the updated external parameter matrix as the external parameter matrix corresponding to the first matching error. When the formula (1) is calculated for the first time to determine the center projection point, the method may be based on a preset external parameter matrix.

For example, suppose

For m_ijFirst column of 3, 3 being m₂₁Thus, the sphere center pixel point 2 is added to the point set X, the sphere center projection point 1 is added to the point set Y, and the sphere center pixel point 2 and the sphere center projection point 1 are a pair of matching points. Similarly, adding the sphere center pixel point 1 into the point set X, adding the sphere center projection point 2 into the point set Y, and simultaneously enabling the sphere center pixel point 1 and the sphere center projection point 2 to be a pair of matching points; adding the sphere center pixel point 3 into a point set X, adding the sphere center projection point 3 into a point set Y, and simultaneously enabling the sphere center pixel point 3 and the sphere center projection point 3 to be a pair of matching points.

For the above S105, performing mirror image transformation on the X-ray image to be recognized according to the preset mirror image mode, and obtaining the mirror image X-ray image may be implemented as follows: and carrying out mirror image transformation on the X-ray image to be identified by taking the X axis as a rotating axis to obtain a first mirror image X-ray image. Wherein, the X-axis is a horizontal axis passing through the center point of the X-ray image to be identified. And/or carrying out mirror image transformation on the X-ray image to be identified by taking the Y axis as a rotating axis to obtain a second mirror image X-ray image. And the Y axis is a vertical axis passing through the center point of the X-ray image to be identified.

For example, as shown in FIG. 2, FIG. 2 is an X-ray image, the vertical dashed line is the Y-axis of the X-ray image, and the horizontal dashed line is the X-axis of the X-ray image.

In the embodiment of the present invention, in the step S105, a mirror image center pixel point of the center of sphere of each Mark ball in the mirror image X-ray image is determined, and a manner of obtaining a mirror image center pixel point set may be implemented as follows: determining a first mirror image spherical center pixel point of the spherical center of each Mark ball in a first mirror image X-ray image to obtain a first mirror image spherical center pixel point set; and/or determining a second mirror image spherical center pixel point of the spherical center of each Mark ball in the second mirror image X-ray image to obtain a second mirror image spherical center pixel point set.

Optionally, the manner of determining the first mirror image sphere center pixel point and the second mirror image sphere center pixel point may refer to the manner of determining the sphere center pixel point, and details are not repeated here.

In the above S106, the manner of determining the second matching error between the mirror image sphere center pixel point set and the sphere center projection point set may be implemented as follows: determining a second X-axis mirror image matching error between the first mirror image sphere center pixel point set and the sphere center projection point set; and/or determining a second Y-axis mirror image matching error between the second mirror image sphere center pixel point set and the sphere center projection point set.

Optionally, the manner of determining the second X-axis mirror image matching error and the second Y-axis mirror image matching error may refer to the manner of determining the first matching error, which is not described herein again.

On this basis, the manner of determining that the image corresponding to the minimum matching error of the first matching error and the second matching error in S107 is the X-ray image captured by the C-arm and not mirrored can be implemented as follows: and taking the image corresponding to the minimum matching error in the first matching error, the second X-axis mirror image matching error and the second Y-axis mirror image matching error as the X-ray image which is not mirrored.

In the embodiment of the present invention, the X-ray image to be recognized in S101 may also be a rotated image, based on which, referring to fig. 3, the embodiment of the present invention may further automatically acquire an unrotated X-ray image, including the following steps:

s301, rotating the X-ray image to be recognized along the image center point by a preset angle to obtain a rotating X-ray image, and determining a rotating center pixel point of the center of each Mark ball in the rotating X-ray image to obtain a rotating center pixel point set.

In an embodiment, the X-ray image to be recognized may be rotated by different preset angles along an image center point, respectively, to obtain a plurality of rotational X-ray images, and a rotational center pixel point in each rotational X-ray image is determined, respectively, to obtain a plurality of sets of rotational center pixel points, where a rotational center pixel point included in each set of rotational center pixel points belongs to one rotational X-ray image.

For example, as shown in fig. 4, each rectangle in fig. 4 is an X-ray image, each black circle in the X-ray image corresponds to a Mark ball, and the "a" in the X-ray image is only used to facilitate viewing the rotation direction of the X-ray image. In the sequence from left to right, the 1 st X-ray image in fig. 4 is the X-ray image to be recognized, and when S301 is executed, the X-ray image to be recognized is rotated clockwise by 90 degrees along the central point of the image, so as to obtain a rotated X-ray image 1, that is, the 2 nd image in fig. 4. And the X-ray image to be identified is rotated 180 degrees clockwise along the image center point to obtain a rotated X-ray image 2, i.e. the 3 rd image in fig. 4. And the X-ray image to be identified is rotated clockwise 270 degrees along the image center point to obtain a rotated X-ray image 3, i.e. the 4 th image in fig. 4.

In the embodiment of the present invention, the manner of determining the rotation center pixel point in the rotation X-ray image may refer to the manner of determining the center pixel point in the X-ray image to be identified, and details are not repeated here.

S302, determining a third matching error between the rotating sphere center pixel point set and the sphere center projection point set.

In the embodiment of the present invention, the manner of determining the third matching error may refer to the manner of determining the first matching error, which is not described herein again.

Alternatively, when there are a plurality of rotated X-ray images, a third matching error may be determined based on each rotated X-ray image.

And S303, determining that the image corresponding to the minimum matching error in the first matching error and the third matching error is an X-ray image which is shot by the C-shaped arm and is not rotated.

Compared with the mode that whether the X-ray image to be identified is the rotated X-ray image or not is manually identified, and when the X-ray image to be identified is judged to be the rotated X-ray image, the X-ray image to be identified is rotated so as to obtain the non-rotated X-ray image, the embodiment of the invention can automatically obtain the non-rotated X-ray image, thereby improving the obtaining efficiency and accuracy.

In a practical application scenario, there are four cases as follows:

case one, the C-arm only has a mirror function.

At this time, the first matching error and the second matching error may be calculated, and an image corresponding to the smallest matching error of the first matching error and the second matching error may be used as an X-ray image that is not mirrored.

Furthermore, when the C-shaped arm only has the function of mirroring along the X axis, the first matching error and the second X axis mirror matching error can be calculated, and the image corresponding to the minimum matching error in the first matching error and the second X axis mirror matching error is used as the X-ray image which is not mirrored.

When the C-shaped arm only has the function of mirroring along the Y axis, the first matching error and the second Y-axis mirroring matching error can be calculated, and the image corresponding to the minimum matching error in the first matching error and the second Y-axis mirroring matching error is used as the X-ray image which is not mirrored.

Case two, the C-arm only has a swivel function.

At this time, the first matching error and the third matching error may be calculated, and an image corresponding to the smallest matching error of the first matching error and the third matching error may be used as an X-ray image that has not been rotated.

Optionally, the third matching errors may include one or more, and each third matching error is: and determining a matching error based on a rotating spherical center pixel point and a spherical center projection point in the rotating X-ray image obtained after the X-ray image to be recognized rotates by an angle.

The number of third matching errors is the number of angles that the C-arm can rotate for the X-ray image.

And in the third case, the C-shaped arm has a mirror image function and a rotation function, and the mirror image function and the rotation function cannot be simultaneously effective.

At this time, the first matching error, the second matching error and the third matching error may be calculated, and an image corresponding to the smallest matching error among the first matching error, the second matching error and the third matching error may be used as an X-ray image that is not mirrored and is not rotated.

Case four, the C-arm has a mirror function and a rotation function, and the mirror function and the rotation function can be simultaneously effective.

At this time, the first matching error, the second matching error, the third matching error and the fourth matching error may be calculated, and an image corresponding to the smallest matching error among the first matching error, the second matching error, the third matching error and the fourth matching error is taken as an X-ray image that is not mirrored and is not rotated.

Wherein, the manner of calculating the fourth matching error comprises: and carrying out mirror image transformation on the X-ray image to be recognized according to a preset mirror image mode to obtain a mirror image X-ray image, and then carrying out rotation of a preset angle on the mirror image X-ray image along the image central point to obtain a comprehensive X-ray image. And then determining a comprehensive sphere center pixel point of the sphere center of each Mark sphere in the comprehensive X-ray image to obtain a comprehensive sphere center pixel point set. And determining a fourth matching error between the comprehensive sphere center pixel point set and the sphere center projection point set.

The calculation method of the fourth matching error is the same as the calculation method of the first matching error, and reference may be made to the above description, which is not repeated herein.

According to the embodiment of the invention, not only can the X-ray image which is not mirrored, but also the X-ray image which is not rotated can be obtained, and the X-ray image which is not mirrored and is not rotated can also be obtained, so that the application range of the embodiment of the invention is expanded.

Referring to fig. 5, the following takes the example that the C-shaped arm can mirror the X-ray image along the X-axis, mirror the X-ray image along the Y-axis, or rotate 90 degrees clockwise, and describes the overall flow of the mirror image identification method provided by the embodiment of the present invention:

s501, the electronic equipment acquires the position of the center of each Mark ball in a flange coordinate system.

In one embodiment, the position of the center of each Mark ball in the flange coordinate system can be calibrated by a high-precision calibration instrument.

S502, the electronic equipment acquires the internal parameters of the C-shaped arm and the distortion parameters of the C-shaped arm.

In one embodiment, the internal parameter and the distortion parameter of the debugged C-arm may be obtained during the debugging process of the C-arm.

S503, placing the Mark plate embedded with the Mark ball between the patient and the C-shaped arm shadow through a mechanical arm of the C-shaped arm, and shooting an X-ray image to be recognized.

It should be noted that, in the process of moving the Mark plate by the mechanical arm in S503, the position of the center of sphere of each Mark ball under the flange coordinate system of the mechanical arm is not changed.

Wherein the shadow is arranged in front of the X-ray receiver of the C-shaped arm and used for enhancing the received X-ray. The Mark plate can be arranged perpendicular to the X-ray emitted by the X-ray emitter of the C-shaped arm so as to avoid the Mark balls from blocking each other on an X-ray image.

S504, the electronic equipment obtains the X-ray image to be identified sent by the C-shaped arm, and determines a spherical center pixel point set in the X-ray image to be identified.

The manner of determining the set of spherical center pixel points in S504 can refer to the related description in S102, and is not described herein again.

And S505, the electronic equipment projects the position of the center of the ball of each Mark ball under the flange coordinate system to an X-ray image coordinate system based on the internal reference of the C-shaped arm and the distortion parameter of the C-shaped arm to obtain a center projection point set.

The manner of obtaining the set of center projection points in S505 may refer to the related description in S103, and is not described herein again.

S506, the electronic equipment determines a first matching error between the sphere center pixel point set and the sphere center projection point set and an external parameter matrix corresponding to the first matching error.

The manner of determining the first matching error in S506 may refer to the related description in S104, and is not described herein again. The manner of determining the external reference matrix corresponding to the first matching error in S506 may refer to the above description, and is not repeated here.

S507, the electronic equipment performs mirror image transformation on the X-ray image to be recognized by taking the X axis as a rotating axis to obtain a first mirror image X-ray image, and performs mirror image transformation on the X-ray image to be recognized by taking the Y axis as the rotating axis to obtain a second mirror image X-ray image.

The manner of determining the first mirror image X-ray image and the second mirror image X-ray image in S507 may refer to the related description of S105, and details are not repeated here.

S508, the electronic equipment determines a first mirror image center pixel point of the center of sphere of each Mark ball in the first mirror image X-ray image, determines a second X-axis mirror image matching error between the first mirror image center pixel point set and the center projection point set and an external parameter matrix corresponding to the second X-axis mirror image matching error, determines a second mirror image center pixel point of the center of sphere of each Mark ball in the second mirror image X-ray image, determines a second Y-axis mirror image matching error between the second mirror image center pixel point set and the center projection point set and an external parameter matrix corresponding to the second Y-axis mirror image matching error.

S508 determines the second X-axis mirror image matching error and the second Y-axis mirror image matching error, which can refer to the related description in S106, and is not described herein again. The determination method of the external reference matrix corresponding to the second X-axis mirror image matching error and the external reference matrix corresponding to the second Y-axis mirror image matching error is the same as the determination method of the external reference matrix corresponding to the first matching error, and reference may be made to the above description, which is not repeated here.

S509, the electronic device rotates the X-ray image to be recognized 90 degrees clockwise along the central point of the image to obtain a rotating X-ray image, and determines a rotating center pixel point of the center of each Mark ball in the rotating X-ray image to obtain a rotating center pixel point set.

The manner of obtaining the rotation centroid pixel point set in S509 may refer to the description in S301, and is not described herein again.

S510, determining a third matching error between the rotating sphere center pixel point set and the sphere center projection point set.

The manner of determining the third matching error in S510 may refer to the description in S302, and is not described herein again.

And S511, determining an image corresponding to the minimum matching error in the first matching error, the second X-axis mirror image matching error, the second Y-axis mirror image matching error and the third matching error as an X-ray image which is shot by the C-shaped arm and is not mirrored and rotated, and taking an external parameter matrix corresponding to the minimum matching error as the external parameter matrix of the C-shaped arm.

In the related art, it is difficult to judge whether an X-ray image is mirrored and rotated if a mirror button and a rotation button are clicked when a doctor who photographs the X-ray image is asked, and if the doctor who photographs the X-ray image cannot be contacted or if the doctor who photographs the X-ray image forgets to click the mirror button and the rotation button when photographing.

Or, in the related art, the X-ray image is compared with the actual object manually, so as to judge whether the X-ray image is mirrored and rotated according to experience. However, characteristic points in the X-ray images captured during the operation are easily blocked, so that it is difficult for a human to distinguish whether the X-ray images are mirrored and rotated.

If the X-ray image cannot be distinguished manually whether the X-ray image is mirrored or rotated, the X-ray image needs to be shot again for the patient, the times of shooting the X-ray image for the patient are increased, and the operation efficiency is influenced.

The embodiment of the invention can automatically determine the X-ray image which is not mirrored and is not rotated based on the X-ray image to be identified, does not depend on manual judgment, improves the efficiency of acquiring the X-ray image which is not mirrored and is not rotated, reduces the times of shooting the X-ray image by the patient and improves the operation efficiency.

Based on the same inventive concept, corresponding to the above method embodiment, an embodiment of the present invention provides a mirror image recognition apparatus, as shown in fig. 6, the apparatus including: an acquisition module 601, a determination module 602 and a mirror image module 603;

the acquiring module 601 is configured to acquire an X-ray image to be identified, where the X-ray image to be identified is an image obtained by transmitting X-rays emitted by a C-shaped arm through a plurality of Mark balls;

a determining module 602, configured to determine a center pixel point of the center of each Mark ball in the X-ray image to be identified, to obtain a center pixel point set;

the determining module 602 is further configured to project the position of the center of sphere of each Mark ball in the flange coordinate system into an X-ray image coordinate system, and determine a center projection point of each Mark ball to obtain a center projection point set;

a determining module 602, further configured to determine a first matching error between the set of sphere center pixel points and the set of sphere center projection points;

the mirror image module 603 is configured to perform mirror image transformation on the X-ray image to be recognized according to a preset mirror image mode to obtain a mirror image X-ray image, and determine a mirror image center pixel point of the center of each Mark ball in the mirror image X-ray image to obtain a mirror image center pixel point set;

a determining module 602, further configured to determine a second matching error between the mirror image sphere center pixel point set and the sphere center projection point set;

the determining module 602 is further configured to determine that an image corresponding to a minimum matching error of the first matching error and the second matching error is an X-ray image captured by the C-arm and not mirrored.

Optionally, the determining module 602 is specifically configured to:

carrying out contrast enhancement on an X-ray image to be identified to obtain an enhanced image;

performing binary segmentation on the enhanced image to obtain a binary image;

and performing circle detection on the binary image to obtain the circle center of each circle in the binary image, and taking the circle center of each circle as a sphere center pixel point of the center of the Mark sphere in the X-ray image to be recognized.

Optionally, the determining module 602 is specifically configured to:

determining the weighted sum of the errors corresponding to the pixel points of the spherical centers as a first matching error through the specified weight parameters;

the determining module 602 is specifically configured to:

and determining the weighted sum of the errors corresponding to the pixel points of the spherical centers of the mirror images through the designated weight parameters as a second matching error.

Optionally, the mirroring module 603 is specifically configured to:

carrying out mirror image transformation on an X-ray image to be identified by taking an X-axis as a rotating shaft to obtain a first mirror image X-ray image, wherein the X-axis is a horizontal axis passing through the center point of the X-ray image to be identified;

Optionally, the mirroring module 603 is specifically configured to:

determining a first mirror image spherical center pixel point of the spherical center of each Mark ball in a first mirror image X-ray image to obtain a first mirror image spherical center pixel point set;

and/or determining a second mirror image spherical center pixel point of the spherical center of each Mark ball in a second mirror image X-ray image to obtain a second mirror image spherical center pixel point set;

the determining module 602 is specifically configured to:

determining a second X-axis mirror image matching error between the first mirror image sphere center pixel point set and the sphere center projection point set;

The determining module 602 is specifically configured to:

Optionally, the apparatus further comprises: a rotation module;

the rotating module is used for rotating the X-ray image to be recognized by a preset angle along the central point of the image after the X-ray image to be recognized is obtained, so as to obtain a rotating X-ray image, and determining a rotating spherical center pixel point of the spherical center of each Mark ball in the rotating X-ray image, so as to obtain a rotating spherical center pixel point set;

a determining module 602, further configured to determine a third matching error between the set of rotation centroid pixel points and the set of sphere centroid projection points;

the determining module 602 is further configured to determine that an image corresponding to a minimum matching error of the first matching error and the third matching error is an X-ray image captured by the C-arm and not rotated.

An embodiment of the present invention further provides an electronic device, as shown in fig. 7, including a processor 701, a communication interface 702, a memory 703 and a communication bus 704, where the processor 701, the communication interface 702, and the memory 703 complete mutual communication through the communication bus 704,

a memory 703 for storing a computer program;

the processor 701 is configured to implement the method steps in the above-described method embodiments when executing the program stored in the memory 703.

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

In yet another embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program realizes the steps of any one of the above mirror image recognition methods when executed by a processor.

In yet another embodiment, a computer program product containing instructions is provided, which when run on a computer causes the computer to perform any of the mirror image recognition methods of the above embodiments.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for identifying a mirror image, the method comprising:

2. The method according to claim 1, wherein the determining of the center pixel point of the center of each Mark ball in the X-ray image to be recognized comprises:

performing binary segmentation on the enhanced image to obtain a binary image;

3. The method of claim 1 or 2, wherein said determining a first match error between said set of sphere center pixel points and said set of sphere center projection points comprises:

4. The method according to claim 1, wherein performing mirror image transformation on the X-ray image to be recognized according to a preset mirror image manner to obtain a mirror image X-ray image comprises:

5. The method of claim 4, wherein the determining a mirror image centroid pixel point of the centroid of each Mark ball in the mirror image X-ray image to obtain a set of mirror image centroid pixel points comprises:

and/or determining a second Y-axis mirror image matching error between the second mirror image sphere center pixel point set and the sphere center projection point set;

6. The method of claim 1, wherein after said acquiring an X-ray image to be identified, the method further comprises:

7. A mirror image recognition apparatus, characterized in that the apparatus comprises:

8. The apparatus of claim 7, wherein the determining module is specifically configured to:

performing binary segmentation on the enhanced image to obtain a binary image;

9. The apparatus according to claim 7 or 8, wherein the determining module is specifically configured to:

the determining module is specifically configured to:

10. The apparatus of claim 7, wherein the mirroring module is specifically configured to:

11. The apparatus of claim 10, wherein the mirroring module is specifically configured to:

the determining module is specifically configured to:

12. The apparatus of claim 7, further comprising: a rotation module;

13. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;

a memory for storing a computer program;

a processor for implementing the method steps of any of claims 1-6 when executing a program stored in the memory.

14. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 6.