WO2021196955A1 - Image recognition method and related apparatus, and device - Google Patents

Image recognition method and related apparatus, and device Download PDF

Info

Publication number
WO2021196955A1
WO2021196955A1 PCT/CN2021/078748 CN2021078748W WO2021196955A1 WO 2021196955 A1 WO2021196955 A1 WO 2021196955A1 CN 2021078748 W CN2021078748 W CN 2021078748W WO 2021196955 A1 WO2021196955 A1 WO 2021196955A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
recognized
medical image
feature representation
medical
Prior art date
Application number
PCT/CN2021/078748
Other languages
French (fr)
Chinese (zh)
Inventor
叶宇翔
陈翼男
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Priority to KR1020227004540A priority Critical patent/KR20220031695A/en
Priority to JP2021577453A priority patent/JP2022539162A/en
Publication of WO2021196955A1 publication Critical patent/WO2021196955A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images
    • G06V2201/031—Recognition of patterns in medical or anatomical images of internal organs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • This application relates to the field of artificial intelligence technology, in particular to an image recognition method and related devices and equipment.
  • scan image categories often include timing-related pre-contrast scan, early arterial phase, late arterial phase, portal phase, delay phase, etc.
  • scan image categories can also include scan parameters related T1-weighted inverse imaging, T1-weighted in-phase imaging, T2-weighted imaging, diffusion-weighted imaging, surface diffusion coefficient imaging, etc.
  • the radiologist is usually required to identify the scanned image category of the scanned medical image to ensure that the required medical image is obtained; or, during hospitalization or outpatient treatment, the doctor is usually required to review the scanned medical image Recognize, determine the scanned image category of each medical image, and then read the image.
  • the above-mentioned method of manually identifying the scanned image category of the medical image has low efficiency, and is subject to subjective influence and is difficult to ensure accuracy. Therefore, how to improve the efficiency and accuracy of image recognition has become an urgent problem to be solved.
  • This application provides an image recognition method and related devices and equipment.
  • the first aspect of the present application provides an image recognition method, including: acquiring at least one scanned medical image to be recognized, and respectively determining the target area corresponding to the target organ in each medical image to be recognized; Perform feature extraction on regional image data to obtain the individual feature representation of each medical image to be recognized; fuse the individual feature representations of at least one medical image to be recognized to obtain a global feature representation; use the individual feature representation of each medical image to be recognized And the global feature representation to determine the scanned image category to which each medical image to be recognized belongs.
  • the feature extraction of the image data of each target area is performed to obtain each
  • the individual feature representation of the medical image to be recognized can eliminate interference from other organs, which is conducive to improving the accuracy of recognition, and the individual feature representations of at least one medical image to be recognized are merged to obtain a global feature representation, and then each medical image to be recognized
  • the individual feature representation and the global feature representation of the image can not only represent the characteristics of the medical image to be recognized, but also the difference of other medical images to be recognized, so that the individual feature representation and global feature representation of each medical image to be recognized can be used.
  • fusing the individual feature representations of at least one medical image to be identified to obtain a global feature representation includes: performing global pooling processing on the individual feature representations of at least one medical image to be identified to obtain a global feature representation.
  • the global feature representation can be obtained quickly and conveniently, which can help improve the efficiency of subsequent image recognition.
  • subjecting at least one individual feature representation of the medical image to be identified to global pooling processing to obtain the global feature representation includes: subjecting at least one individual feature representation of the medical image to be identified to global maximum pooling processing to obtain the first global feature representation; And, performing global average pooling processing on the individual feature representations of at least one medical image to be recognized to obtain a second global feature representation; and performing stitching processing on the first global feature representation and the second global feature representation to obtain a global feature representation.
  • the first global feature representation is obtained, and performing global average pooling processing on the individual feature representation of at least one medical image to be recognized to obtain the second Global feature representation, so that the first global feature representation and the second global feature representation are stitched together to obtain the global feature representation, so it can help to accurately represent the difference between each medical image to be recognized and other medical images to be recognized. , Which can help improve the accuracy of subsequent image recognition.
  • using the individual feature representation and global feature representation of each medical image to be recognized to determine the scanned image category to which each medical image to be recognized belongs includes: using the individual feature representation and global feature representation of each medical image to be recognized to obtain each A final feature representation of the medical image to be recognized, using the final feature representation of each medical image to be recognized to determine the scanned image category to which each medical image to be recognized belongs.
  • the final feature representation obtained by using the individual feature representation and the global feature representation of each medical image to be recognized can not only represent the characteristics of the medical image to be recognized, but also the difference of other medical images to be recognized, so as to use each
  • the final feature representation of each medical image to be recognized can improve the accuracy of image recognition when determining the scanned image category to which each medical image to be recognized belongs.
  • using the individual feature representation and the global feature representation of each medical image to be recognized to obtain the final feature representation of each medical image to be recognized includes: stitching the individual feature representation and the global feature representation of each medical image to be recognized respectively , Get the final feature representation corresponding to the medical image to be recognized.
  • the final feature representation corresponding to the medical image to be recognized can be quickly obtained, which can help improve the efficiency of subsequent image recognition.
  • performing feature extraction on the image data of each target area to obtain the individual feature representation of each medical image to be recognized includes: using the feature extraction sub-network of the recognition network to perform feature extraction on the image data of each target area to obtain each The individual feature representations of the medical images to be recognized; the individual feature representations of at least one medical image to be recognized are fused to obtain the global feature representation, and the individual feature representation and the global feature representation of each medical image to be recognized are used to obtain each
  • the final feature representation of the medical image includes: using the fusion module of the recognition network to fuse the individual feature representation of at least one medical image to be recognized to obtain a global feature representation, and use the individual feature representation and global feature representation of each medical image to be recognized, Obtain the final feature representation of each medical image to be recognized; use the final feature expression of each medical feature to be recognized to determine the scanned image category to which each medical image to be recognized belongs, including: using the classification sub-network of the recognition network to Recognizing the final feature of the medical image means performing predictive classification to obtain the scanned image category to
  • the individual feature representation of each medical image to be recognized is obtained, and the fusion module of the recognition network is used to extract the features of at least one medical image to be recognized.
  • the individual feature representations are fused to obtain a global feature representation.
  • the individual feature representation and global feature representation of each medical image to be recognized are used to obtain the final feature representation of each medical image to be recognized, so that the classification sub-network of the recognition network is used for each
  • the final feature of the medical image to be recognized indicates that the predicted classification is performed to obtain the scanned image category to which each medical image to be recognized belongs. Therefore, the scanned image category to which the medical image to be recognized belongs can be finally obtained through the recognition network, which can further improve the efficiency of image recognition .
  • the recognition network is obtained by training with sample medical images, and the number of sample medical images used for each training of the recognition network is not fixed.
  • the number of sample medical images used in each training recognition network is not fixed, and the number of sample medical images can be randomized, which can help to accurately scan image categories under different scanning protocols in different institutions.
  • Image recognition can improve the accuracy of image recognition.
  • the feature extraction sub-network includes at least one set of sequentially connected dense convolution blocks and pooling layers; and/or, the recognition network includes a preset number of feature extraction sub-networks; the feature extraction sub-network of the recognition network is used for each target Performing feature extraction on the image data of the region to obtain the individual feature representation of each medical image to be recognized includes: inputting the image data of each target region into a corresponding feature extraction sub-network for feature extraction, and obtaining the individual of each medical image to be recognized Feature representation.
  • the feature extraction sub-network includes at least one set of dense convolutional blocks and pooling layers that are connected in sequence. Therefore, the dense convolutional block connection strategy is adopted, that is, the features under each convolutional layer are tightly spliced with the next layer and transmitted Each subsequent layer can effectively alleviate the problem of gradient disappearance, strengthen feature propagation and feature reuse, and can greatly reduce the number of parameters; and the recognition network is set to include a preset number of feature extraction sub-networks, and The image data of each target area is input into a corresponding feature extraction sub-network for feature extraction, and the individual feature representation of each medical image to be recognized is obtained. The feature extraction operation of the image data of at least one target area can be processed in parallel. Conducive to improving the efficiency of image recognition.
  • respectively determining the target area corresponding to the target organ in each medical image to be recognized includes: using an organ detection network to detect at least one medical image to be recognized to obtain first position information of the target organ and information about the target organ.
  • the second position information of the adjacent organ; the first position information and the second position information are used to determine the target area corresponding to the target organ.
  • the organ detection network is used to detect at least one medical image to be recognized to obtain the first position information of the target organ and the second position information of the adjacent organs of the target organ, so that not only the target to be recognized can be considered Organs can also consider the surrounding organs, so that the first position information and the second position information can be used to determine the target area corresponding to the target organ, which can ensure that the shape of the organ changes under surgical treatment, etc.
  • the target area corresponding to the target organ is obtained by positioning, so the robustness of image recognition can be improved.
  • the medical image to be recognized is a three-dimensional image
  • the organ detection network is used to detect at least one medical image to be recognized to obtain the first position information of the target organ and the second position information of the adjacent organs of the target organ. It also includes: dividing each medical image to be identified along the coronal plane to obtain multiple three-dimensional sub-images; projecting each sub-image in a direction perpendicular to the coronal plane to obtain the corresponding two-dimensional sub-image; using organ detection
  • the network detects at least one medical image to be identified, and obtains the first position information of the target organ and the second position information of the adjacent organs of the target organ. Dimension sub-images are detected to obtain first position information and second position information.
  • each medical image to be recognized is divided along the coronal plane to obtain multiple three-dimensional sub-images, and each sub-image is projected in a direction perpendicular to the coronal plane to obtain the corresponding Therefore, the organ detection network is used to detect the two-dimensional sub-image corresponding to at least one medical image to be recognized, and the first position information and the second position information are obtained, which can further improve the target area location corresponding to the target organ Accuracy.
  • the target organ is the liver
  • the adjacent organs include at least one of the kidney and the spleen
  • the first position information includes at least one set of diagonal vertex positions of the corresponding area of the target organ and the size of the corresponding area.
  • the second position information includes at least one vertex position of the corresponding area adjacent to the organ.
  • setting the target organ as the liver, and setting the adjacent organ to include at least one of the kidney and the spleen can help locate the target area corresponding to the liver; setting the first position information to include the area corresponding to the target organ At least one set of diagonal vertex positions and the size of the corresponding area, and the second position information is set to include at least one vertex position adjacent to the corresponding area of the organ, which can facilitate accurate positioning of the target area corresponding to the target organ.
  • the method further includes at least one of the following: scanning at least one medical image to be recognized according to its scan Sort the image categories; if the scanned image categories of the medical images to be recognized are repeated, the first warning information is output to remind the scanner; if there is no preset scanned image category in the scanned image categories of at least one medical image to be recognized, then Output the second warning message to remind the scanner.
  • the scanned image category to which each medical image to be recognized belongs execute sorting of at least one medical image to be recognized according to its scanned image category, which can improve the convenience of doctors in reading the image;
  • the first warning information is output to remind the scanner
  • the preset scanned image category does not exist in the scan image category of at least one medical image to be recognized
  • the second warning information is output to remind the scanner.
  • the method further includes: preprocessing the image data of each target region, wherein the preprocessing includes the following At least one: adjusting the image size of the target area to a preset size, and normalizing the image intensity of the target area to a preset range.
  • the image data of each target area is preprocessed, and the preprocessing includes at least one of the following: adjusting the image size of the target area to a preset size, and normalizing the image intensity of the target area to The preset range can help improve the accuracy of subsequent image recognition.
  • the second aspect of the present application provides an image recognition device, including: a region acquisition module, a feature extraction module, a fusion processing module, and a category determination module.
  • the region acquisition module is configured to acquire at least one scanned medical image to be identified, and respectively determine The target region corresponding to the target organ in each medical image to be recognized;
  • the feature extraction module is configured to extract features from the image data of each target region to obtain the individual feature representation of each medical image to be recognized; configuration of the fusion processing module In order to fuse the individual feature representations of at least one medical image to be identified to obtain a global feature representation;
  • the category determination module is configured to use the individual feature representation and the global feature representation of each medical image to be identified to determine which medical image belongs to Scanned image category.
  • a third aspect of the present application provides an electronic device including a memory and a processor coupled to each other, and the processor is configured to execute program instructions stored in the memory to implement the image recognition method in the first aspect.
  • the fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, and the program instructions implement the image recognition method in the first aspect when the program instructions are executed by a processor.
  • the feature extraction is performed on the image data of each target area to obtain each
  • the individual feature representations of a medical image to be recognized can eliminate interference from other organs, which is beneficial to improve the accuracy of recognition, and the individual feature representations of at least one medical image to be recognized are merged to obtain a global feature representation, and then each to be recognized
  • the individual feature representation and global feature representation of medical images can not only represent the characteristics of the medical image to be recognized, but also the differences of other medical images to be recognized, so that the individual feature representation and global feature representation of each medical image to be recognized can be used.
  • FIG. 1 is a schematic flowchart of an embodiment of an image recognition method according to the present application.
  • FIG. 2 is a schematic diagram of the state of the process of determining the scanned image category to which the medical image to be recognized belongs;
  • FIG. 3 is a schematic flowchart of an embodiment of step S11 in FIG. 1;
  • FIG. 4 is a schematic diagram of the framework of an embodiment of the image recognition device of the present application.
  • FIG. 5 is a schematic diagram of the framework of an embodiment of the electronic device of the present application.
  • Fig. 6 is a schematic diagram of a framework of an embodiment of a computer-readable storage medium according to the present application.
  • system and "network” in this article are often used interchangeably in this article.
  • the term “and/or” in this article is only an association relationship describing the associated objects, which means that there can be three relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, exist alone B these three situations.
  • the character "/” in this text generally indicates that the associated objects before and after are in an "or” relationship.
  • "many” in this document means two or more than two.
  • the execution subject of the image recognition method may be an image recognition device.
  • the image recognition method may be executed by a terminal device or a server or other processing equipment.
  • the terminal device may be a user equipment (User Equipment, UE). ), mobile devices, user terminals, terminals, cellular phones, cordless phones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc.
  • the image recognition method can be implemented by a processor calling computer-readable instructions stored in the memory.
  • FIG. 1 is a schematic flowchart of an embodiment of an image recognition method according to the present application. Specifically, it can include the following steps:
  • Step S11 Obtain at least one scanned medical image to be recognized, and respectively determine the target area corresponding to the target organ in each medical image to be recognized.
  • the medical images to be recognized may include CT images and MR images, which are not limited here.
  • the medical image to be recognized may be obtained by scanning the abdomen, chest and other areas, and may be specifically set according to actual application conditions, which is not limited here. For example, when the liver, spleen, and kidney are the target organs that need diagnosis and treatment, the abdomen can be scanned to obtain medical images to be identified; or, when the heart and lungs are the target organs that need diagnosis and treatment, the chest can be scanned. Obtain the medical image to be recognized, and other situations can be deduced by analogy, so we will not give examples one by one here.
  • the scanning mode may be plain scanning, enhanced scanning, etc., which are not limited here.
  • the medical image to be recognized may be a three-dimensional image, and the target area corresponding to the target organ in the medical image to be recognized may be a three-dimensional area, which is not limited here.
  • the target organ can be set according to the actual application.
  • the target organ can be the liver; or when the doctor needs to judge whether the kidney has lesions and the extent of the disease, the target organ
  • the device can be the kidney, and other conditions can be set according to the actual application, so we will not give examples one by one here.
  • an organ detection network for detecting target organs can be pre-trained, so that the organ detection network can be directly used to detect each medical image to be identified to obtain the corresponding target area.
  • Step S12 Perform feature extraction on the image data of each target area respectively to obtain the individual feature representation of each medical image to be recognized.
  • the image data of each target area may also be preprocessed.
  • the preprocessing may include The image size of the area is adjusted to a preset size (for example, 32*256*256).
  • the preprocessing may also include normalizing the image intensity of the target area to a preset range (for example, the range of 0 to 1).
  • the preset ratio For example, the gray value corresponding to 99.9%
  • the normalized clamp value is used as the normalized clamp value, so that the contrast of the image data of the target area can be enhanced, which is beneficial to improve the accuracy of subsequent image recognition.
  • a recognition network in order to improve the convenience of feature extraction, can also be pre-trained.
  • the recognition network can include a feature extraction sub-network for feature extraction, so that the feature extraction sub-network can be used to analyze the image of each target area.
  • the data is feature-extracted, and the individual feature representation of each medical image to be recognized is obtained.
  • the feature extraction sub-network includes at least a set of sequentially connected dense convolution blocks (Dense Block) and a pooling layer.
  • the features of each layer of the dense convolution block are closely connected to the next layer. Splicing and transferring each layer afterwards makes the transfer of features and gradients more effective.
  • the feature extraction sub-network may include three sets of sequentially connected dense convolution blocks and pooling layers, where, except for the pooling layer contained in the last set of adaptive pooling, the pooling layers contained in other groups It is the maximum pooling; in addition, the feature extraction sub-network may also include a group, two groups, four groups, and other groups of dense convolution blocks (Dense Block) and a pooling layer connected in sequence, which are not limited here.
  • the recognition network may specifically include a preset number of feature extraction sub-networks, so that the image data of each target area can be input into a corresponding feature extraction sub-network for feature extraction, and each target area can be extracted. Recognize the individual feature representations of medical images, and then the feature extraction operations of the image data of each target area can be processed in parallel, so the efficiency of feature extraction can be improved, and the efficiency of subsequent image recognition can be improved.
  • the preset number can be greater than Or equal to the category of the scanned image.
  • the preset number can be set to an integer greater than or equal to 5 , For example, 5, 6, 7, etc., which are not limited here; or, when the scanned image category includes T1-weighted inverse imaging, T1-weighted in-phase imaging, T2-weighted imaging, diffusion-weighted imaging, surface diffusion related to tracing parameters In coefficient imaging, the preset number can be set to an integer greater than or equal to 5, for example, 5, 6, 7, etc., which are not limited here; or, when the scanned image category includes both T1 weighted inversion related to the tracing parameter Imaging, T1-weighted in-phase imaging, T2-weighted imaging, diffusion-weighted imaging, surface diffusion coefficient imaging, as well as timing-related pre-contrast scan, early arterial, late arterial, portal phase, and delay periods.
  • the preset number can be set It is an integer greater than or equal to 10, for example, 10, 11, 12, and so on.
  • the early arterial phase can indicate that the portal vein has not been enhanced
  • the late arterial phase can indicate that the portal vein has been enhanced
  • the portal phase can indicate that the portal vein has been fully enhanced and the liver vessels have been enhanced by forward blood flow, and the liver soft cell tissue has been under the markers.
  • the delay period can indicate that the portal vein and arteries are in an enhanced state and weaker than the portal stage, and the liver soft cell tissue is in an enhanced state and weaker than the portal stage.
  • Other scan image categories will not be illustrated here.
  • Figure 2 is a schematic diagram of the state of the process of determining the scanned image category to which the medical image to be recognized belongs. As shown in Figure 2, rectangular boxes filled with different gray levels represent medical image 1 to medical image to be recognized The individual feature representation 1, individual feature representation 2, individual feature representation 3, ..., individual feature representation n extracted from the image data of the target region corresponding to the target organ in n.
  • Step S13 Fusion of individual feature representations of at least one medical image to be identified to obtain a global feature representation.
  • the recognition network may also include a fusion module, so that the fusion module can be used to fuse the individual feature representations of at least one medical image to be recognized to obtain a global feature representation.
  • the individual feature representation of at least one medical image to be recognized may be subjected to global pooling processing to obtain a global feature representation.
  • the individual feature representation of at least one medical image to be recognized may be subjected to global maximum pooling (Global Max Pooling, GMP) processing to obtain a first global feature representation
  • GMP global maximum pooling
  • the individual feature representation of at least one medical image to be recognized can be global
  • GAP global average pooling
  • individual feature representation 1, individual feature representation 2, individual feature representation 3, ..., individual feature representation n are respectively subjected to global maximum pooling and global average pooling to obtain the first global feature representation ( The oblique line in FIG. 2 fills the rectangular frame) and the second global feature representation (the grid line fills the rectangular frame in FIG. 2), and the first global feature representation and the second global feature representation are spliced to obtain the global feature representation.
  • Step S14 Use the individual feature representation and the global feature representation of each medical image to be recognized to determine the scanned image category to which each medical image to be recognized belongs.
  • the individual feature representation and global feature representation of each medical image to be recognized can be used to obtain the final feature representation of each medical image to be recognized, and then the final feature representation of each medical image to be recognized can be used to determine each medical image to be recognized.
  • the category of the scanned image to which the medical image belongs so that the final feature representation can not only represent the characteristics of the medical image to be recognized, but also the difference of other medical images to be recognized, and then use the final feature representation of each medical image to be recognized to determine each
  • the accuracy of image recognition can be improved.
  • the fusion module in the recognition network can be used to use the individual feature representation and global feature representation of each medical image to be recognized to obtain each to be recognized The final feature representation of the medical image.
  • the individual feature representation and the global feature representation of each medical image to be recognized can also be spliced to obtain the final feature representation corresponding to the medical image to be recognized.
  • Figure 2 Please refer to Figure 2 in combination. As shown in Figure 2, the first global feature represented by a rectangular box filled with diagonal lines is represented and the second global feature represented by a rectangular box filled with grid lines is represented respectively and represented by a rectangular box filled with different gray levels. The individual feature representations of is spliced, and the final feature representation corresponding to each medical image to be recognized can be obtained.
  • the recognition network may also include a classification sub-network, so that the classification sub-network can be used to predict and classify the final feature representation of each medical image to be recognized, and obtain the scanned image category to which each medical image to be recognized belongs.
  • the classification sub-network can include a fully connected layer and a softmax layer, so that the fully connected layer can be used to connect the final feature representation of each medical image to be recognized, and the softmax layer can be used for probability normalization Therefore, the probability value that each medical image to be recognized belongs to each scanned image category is obtained, so the scanned image category corresponding to the maximum probability value can be used as the scanned image category to which the medical image to be recognized belongs.
  • the recognition network including the feature extraction sub-network, the fusion module and the classification sub-network may be obtained by training using sample medical images.
  • the feature extraction sub-network can be used to perform feature extraction on the image data of the target area annotated in each sample medical image to obtain the individual feature representation of each sample medical image
  • the fusion module can be used to extract the individual characteristics of at least one sample medical image.
  • Feature representations are fused to obtain a global feature representation.
  • the individual feature representation and global feature representation of each sample medical image are used to obtain the final feature representation of each sample medical image, and then the classification sub-network is used to determine the final feature of each sample medical image.
  • the number of sample medical images used for each training of the recognition network may not be fixed.
  • the sample medical images used for each training of the recognition network may belong to the same object, and the number of scanned image categories to which the sample medical images used for each training of the recognition network belongs may not be fixed.
  • the sample medical images used in a certain training recognition network belong to T1-weighted inverse imaging, T1-weighted in-phase imaging, and T2-weighted imaging
  • the sample medical images used in another training recognition network belong to diffusion-weighted imaging and surface diffusion coefficient imaging.
  • the specific settings can be set according to the actual application situation. I will not give examples one by one here, so that the number of sample medical images can be randomized, which can help to accurately scan image categories when different institutions and different scanning protocols are missing. Image recognition can improve the robustness of the recognition network.
  • the above-mentioned trained recognition network can be set in an image post-processing workstation, a filming workstation, a computer-aided image reading system, etc., so as to realize automatic recognition of medical images to be recognized and improve recognition efficiency.
  • all medical images to be recognized that belong to the same object in a scan process can be recognized in one recognition process, so that the performance of the recognition network can be fully verified;
  • all medical images to be recognized that belong to the same object in one scan can be recognized in one recognition process, so that the difference between each medical image to be recognized and all other medical images to be recognized can be considered , which in turn can help improve the accuracy of recognition.
  • At least one medical image to be recognized is obtained by scanning the same object. Therefore, in order to facilitate the doctor to read the image, after obtaining the scanned image category to which each medical image to be recognized belongs, at least one medical image to be recognized may be The images are sorted according to their scan image category, for example, T1-weighted inverse imaging, T1-weighted in-phase imaging, plain scan before angiography, early arterial, late arterial, portal phase, delay phase, T2-weighted imaging, diffusion-weighted imaging, The preset order of surface diffusion coefficient imaging is sorted. In addition, the preset order can also be set according to the doctor’s habits, which is not limited here, so as to improve the convenience of doctors in reading the film.
  • the sorted at least one medical image to be recognized can also be displayed in a window corresponding to the number of medical images to be recognized. For example, if the number of medical images to be recognized is 5, it can be displayed in 5 display windows. Medical images to be recognized. Therefore, it is possible to reduce the time for doctors to look through multiple medical images to be identified for comparison back and forth, and to improve the efficiency of image reading.
  • At least one medical image to be recognized is obtained by scanning the same object. Therefore, in order to perform quality control during the scanning process, after obtaining the scanned image category to which each medical image to be recognized belongs, it can also be determined Identify whether there is a repetition in the scanned image category of the medical image, and when there is a repetition, output first warning information to remind the scanner. For example, if there are two medical images to be recognized whose scan image categories are both "delay period", it can be considered that the scan quality is out of compliance during the scanning process. Therefore, in order to remind the scanner, the first warning message can be output. Therefore, it is possible to output the warning reason (for example, there are medical images to be recognized with repeated scan image categories, etc.).
  • the preset scanned image category is "portal phase"
  • the second warning message can be output, specifically, the reason for the warning can be output (for example, there is no portal vein image in the medical image to be identified, etc.). Therefore, the image quality control can be realized during the scanning process, so that when it is contrary to the actual situation, the error can be corrected in time, and the second registration of the patient can be avoided.
  • the feature extraction is performed on the image data of each target area to obtain each
  • the individual feature representations of a medical image to be recognized can eliminate interference from other organs, which is beneficial to improve the accuracy of recognition, and the individual feature representations of at least one medical image to be recognized are merged to obtain a global feature representation, and then each to be recognized
  • the individual feature representation and global feature representation of medical images can not only represent the characteristics of the medical image to be recognized, but also the differences of other medical images to be recognized, so that the individual feature representation and global feature representation of each medical image to be recognized can be used.
  • FIG. 3 is a schematic flowchart of an embodiment of step S11 in FIG. 1.
  • FIG. 3 is a schematic flowchart of an embodiment of determining the target region corresponding to the target organ in each medical image to be recognized, which may specifically include the following steps:
  • Step S111 Use the organ detection network to detect at least one medical image to be identified, to obtain first position information of the target organ and second position information of the adjacent organs of the target organ.
  • the backbone network of the organ detection network can adopt an efficient net. In other implementation scenarios, the backbone network of the organ detection network can also adopt other networks, which is not limited here.
  • the target organ may be set according to actual conditions.
  • the target organ may be the liver, and the adjacent organs of the target organ may include at least one of the kidney and the spleen.
  • the first position information of the target organ may include at least one set of diagonal vertex positions (for example, position coordinates) of the corresponding area of the target organ and the size (for example, length, width, etc.) of the corresponding area.
  • the second position information may at least include at least one vertex position (for example, position coordinates) of the corresponding region of the adjacent organ.
  • the medical image to be recognized can be a three-dimensional image.
  • each medical image to be recognized can be divided along the coronal plane to obtain multiple three-dimensional sub-images, and Project each sub-image in the direction perpendicular to the coronal plane to obtain the corresponding two-dimensional sub-image, so that subsequent identification and detection can be performed based on multiple two-dimensional sub-images obtained by the projection.
  • the organ detection network can be used to At least one two-dimensional sub-image corresponding to the medical image to be recognized is detected to obtain the first position information and the second position information, so that it can be accurately when the size of the target organ is abnormal or the shape of the target organ changes after surgical intervention
  • the first location information of the target organ and the second location information of the adjacent organ of the target organ are determined. For example, when the target organ is the liver, when the liver size is abnormal or the liver morphology changes (such as partial loss) after surgical intervention, the positions of the liver apex and liver apex cannot be stably represented.
  • Organs detection can be performed on two two-dimensional sub-images, and the detection results on multiple two-dimensional sub-images can be combined to obtain the first position information of the liver and the second position information of the kidney, spleen, etc., which can effectively avoid the apex and top of the liver. Possible detection error due to unstable position.
  • Step S112 Use the first position information and the second position information to determine the target area corresponding to the target organ.
  • the geographic correlation between the target organ and the adjacent organs in the anatomical structure can be considered, so the first location information and the second location information are used, It can accurately determine the target area corresponding to the target organ.
  • the first position information may include the upper left and lower left vertices of the corresponding area of the liver, the height and width of the corresponding area
  • the second position information may include the right side of the corresponding area of adjacent organs such as the spleen and kidney.
  • the lower vertex therefore, the target area corresponding to the liver can be obtained by cropping the medical image to be recognized according to the first position information and the second position information.
  • Other scenes can be deduced by analogy, so I won't give examples one by one here.
  • the organ detection network is used to detect at least one medical image to be recognized, and the first position information of the target organ and the second position information of the adjacent organs of the target organ are obtained.
  • the target organ that needs to be identified can also consider the surrounding adjacent organs, so that the first location information and the second location information can be used to determine the target area corresponding to the target organ, which can ensure that the morphology of the organ changes after surgical treatment, etc.
  • the target area corresponding to the target organ can also be located, so the robustness of image recognition can be improved.
  • FIG. 4 is a schematic diagram of a framework of an embodiment of an image recognition device 40 of the present application.
  • the image recognition device 40 includes a region acquisition module 41, a feature extraction module 42, a fusion processing module 43, and a category determination module 44.
  • the region acquisition module 41 is configured to acquire at least one scanned medical image to be recognized, and to determine each medical image to be recognized.
  • the target region in the image corresponding to the target organ is configured to extract features from the image data of each target region to obtain the individual feature representation of each medical image to be recognized;
  • the fusion processing module 43 is configured to at least The individual feature representations of a medical image to be recognized are fused to obtain a global feature representation;
  • the category determination module 44 is configured to use the individual feature representation and the global feature representation of each medical image to be recognized to determine the scanned image to which each medical image to be recognized belongs category.
  • the feature extraction is performed on the image data of each target area to obtain each
  • the individual feature representations of a medical image to be recognized can eliminate interference from other organs, which is beneficial to improve the accuracy of recognition, and the individual feature representations of at least one medical image to be recognized are merged to obtain a global feature representation, and then each to be recognized
  • the individual feature representation and global feature representation of medical images can not only represent the characteristics of the medical image to be recognized, but also the differences of other medical images to be recognized, so that the individual feature representation and global feature representation of each medical image to be recognized can be used.
  • the fusion processing module 43 is configured to perform global pooling processing on the individual feature representation of at least one medical image to be recognized to obtain a global feature representation.
  • the global feature representation can be obtained quickly and conveniently, which can help improve the efficiency of subsequent image recognition.
  • the fusion processing module 43 includes a first pooling sub-module configured to perform global maximum pooling processing on the individual feature representation of at least one medical image to be recognized to obtain the first global feature representation.
  • the fusion processing module 43 includes The second pooling sub-module is configured to perform global average pooling processing on the individual feature representations of at least one medical image to be recognized to obtain a second global feature representation.
  • the fusion processing module 43 includes a splicing processing sub-module configured to combine the first global The feature representation and the second global feature representation are spliced to obtain a global feature representation.
  • the first global feature representation is obtained by subjecting at least one individual feature representation of the medical image to be identified to global maximum pooling processing, and the individual feature representation of at least one medical image to be identified is subject to global average pooling processing ,
  • the second global feature representation so that the first global feature representation and the second global feature representation are stitched together to obtain the global feature representation, so it can help to accurately represent each medical image to be recognized and other medical images to be recognized. The difference between them can help improve the accuracy of subsequent image recognition.
  • the category determination module 44 includes a feature processing sub-module and a category determination sub-module.
  • the feature processing sub-module is configured to use the individual feature representation and the global feature representation of each medical image to be recognized to obtain each medical image to be recognized.
  • the final feature representation of the category determination sub-module is configured to use the final feature representation of each medical image to be recognized to determine the scanned image category to which each medical image to be recognized belongs.
  • the final feature representation obtained by using the individual feature representation and the global feature representation of each medical image to be recognized can not only represent the characteristics of the medical image to be recognized, but also the differences of other medical images to be recognized. Therefore, when the final feature representation of each medical image to be recognized is used to determine the scanned image category to which each medical image to be recognized belongs, the accuracy of image recognition can be improved.
  • the feature processing sub-module is configured to respectively perform stitching processing on the individual feature representation and the global feature representation of each medical image to be recognized to obtain the final feature representation corresponding to the medical image to be recognized.
  • the final feature representation corresponding to the medical image to be recognized can be quickly obtained, which can help improve the efficiency of subsequent image recognition.
  • the feature extraction module 42 is configured to use the feature extraction sub-network of the recognition network to perform feature extraction on the image data of each target area to obtain the individual feature representation of each medical image to be recognized
  • the fusion processing module 43 is configured to The fusion module of the recognition network is used to fuse the individual feature representations of at least one medical image to be recognized to obtain a global feature representation.
  • the feature processing sub-module is configured to use the fusion module of the recognition network to use the individual feature representation and the global feature of each medical image to be recognized Feature representation, the final feature representation of each medical image to be recognized is obtained, and the category determination sub-module is configured to use the classification sub-network of the recognition network to predict and classify the final feature representation of each medical image to be recognized to obtain each medical image to be recognized The category of the scanned image to which it belongs.
  • the feature extraction sub-network of the recognition network is used to perform feature extraction on the image data of each target area to obtain the individual feature representation of each medical image to be recognized
  • the fusion module of the recognition network is used to combine at least one
  • the individual feature representations of the recognized medical images are fused to obtain a global feature representation
  • the individual feature representations and global feature representations of each medical image to be recognized are used to obtain the final feature representation of each medical image to be recognized, thereby using the classifier of the recognition network
  • the network predicts and classifies the final feature representation of each medical image to be recognized, and obtains the scanned image category to which each medical image to be recognized belongs. Therefore, the scanned image category to which the medical image to be recognized belongs can be finally obtained through the recognition network, which can further improve The efficiency of image recognition.
  • the recognition network is obtained by training with sample medical images, and the number of sample medical images used for each training of the recognition network is not fixed.
  • the number of sample medical images used for each training recognition network is not fixed, and the number of sample medical images can be randomized, which can help when the types of scanned images are missing under different institutions and different scanning protocols.
  • the image recognition can also be performed accurately, and the accuracy of the image recognition can be improved.
  • the feature extraction sub-network includes at least one set of dense convolutional blocks and pooling layers connected in sequence; and/or, the recognition network includes a preset number of feature extraction sub-networks, and the feature extraction module 42 is configured to The image data of a target area are respectively input into a corresponding feature extraction sub-network for feature extraction, and the individual feature representation of each medical image to be recognized is obtained.
  • the feature extraction sub-network includes at least one set of dense convolutional blocks and pooling layers that are connected in sequence. Therefore, the dense convolutional block connection strategy is adopted, that is, the features under each convolutional layer are close to the next layer. After splicing and transferring each layer, it can effectively alleviate the problem of gradient disappearance, strengthen feature propagation and feature reuse, and can greatly reduce the number of parameters; and the recognition network is set to include a preset number of feature extractors Network, and input the image data of each target area into a corresponding feature extraction sub-network for feature extraction, and obtain the individual feature representation of each medical image to be recognized.
  • the feature extraction operation of the image data of at least one target area can be processed in parallel , It can help improve the efficiency of image recognition.
  • the area acquisition module 41 includes an organ detection sub-module configured to detect at least one medical image to be identified using an organ detection network to obtain first position information of the target organ and adjacent organs of the target organ.
  • the area acquisition module 41 includes an area determination sub-module configured to use the first location information and the second location information to determine the target area corresponding to the target organ.
  • the organ detection network is used to detect at least one medical image to be recognized, and the first position information of the target organ and the second position information of the adjacent organs of the target organ are obtained.
  • the target organ that needs to be identified can also consider the surrounding adjacent organs, so that the first location information and the second location information can be used to determine the target area corresponding to the target organ, which can ensure that the morphology of the organ changes after surgical treatment, etc.
  • the target area corresponding to the target organ can also be located, so the robustness of image recognition can be improved.
  • the medical image to be recognized is a three-dimensional image
  • the region acquisition module 41 further includes an image division sub-module configured to divide each medical image to be recognized along the coronal plane to obtain multiple three-dimensional sub-images.
  • the region acquisition module 41 also includes an image projection sub-module, configured to project each sub-image in a direction perpendicular to the coronal plane to obtain a corresponding two-dimensional sub-image, and the organ detection sub-module is configured to use the organ detection network to identify at least one The two-dimensional sub-image corresponding to the medical image is detected to obtain the first position information and the second position information.
  • each medical image to be recognized is divided along the coronal plane to obtain multiple three-dimensional sub-images, and each sub-image is performed in a direction perpendicular to the coronal plane. Projection to obtain the corresponding two-dimensional sub-image, so that the organ detection network is used to detect the two-dimensional sub-image corresponding to at least one medical image to be recognized, and the first position information and the second position information are obtained, which can further improve the correspondence of the target organ The accuracy of the target area positioning.
  • the target organ is the liver
  • the adjacent organs include at least one of the kidney and the spleen
  • the first position information includes at least one set of diagonal vertex positions and corresponding areas of the corresponding area of the target organ
  • the second position information includes at least one vertex position of the corresponding area adjacent to the organ.
  • setting the target organ as the liver, and setting the adjacent organ to include at least one of the kidney and the spleen can help locate the target area corresponding to the liver; and set the first position information to include the target organ. At least one set of diagonal vertex positions of the organ corresponding area and the size of the corresponding area, and the second position information is set to include at least one vertex position adjacent to the organ corresponding area, which can facilitate accurate positioning of the target area corresponding to the target organ.
  • the image recognition device 40 further includes an image sorting module configured to sort the at least one medical image to be recognized according to its scanned image category; the image recognition device 40 further includes a first output module configured to When the scanned image category of the image is repeated, the first warning information is output to remind the scanner; the image recognition device 40 further includes a second output module configured to have no preset scan in the scanned image category of the at least one medical image to be recognized In the image category, the second warning message is output to remind the scanner.
  • the scan image category to which each medical image to be recognized belongs is determined, it is executed to sort at least one medical image to be recognized according to its scan image category, which can improve the convenience of doctor reading;
  • the first warning information is output to remind the scanner, and when the preset scanned image category does not exist in the scanned image category of the at least one medical image to be recognized, the second warning information is output to remind Scanners can achieve image quality control during the scanning process, so that when it is contrary to reality, they can correct errors in time to avoid the second registration of patients.
  • the image recognition device 40 further includes a preprocessing module configured to preprocess the image data of each target area, wherein the preprocessing includes at least one of the following: adjusting the image size of the target area to a preset Size, normalize the image intensity of the target area to the preset range.
  • the image data of each target area is preprocessed, and the preprocessing includes at least one of the following: adjusting the image size of the target area to a preset size, and adjusting the image intensity of the target area It is normalized to the preset range, so it can help improve the accuracy of subsequent image recognition.
  • FIG. 5 is a schematic diagram of a framework of an embodiment of an electronic device 50 of the present application.
  • the electronic device 50 includes a memory 51 and a processor 52 coupled to each other, and the processor 52 is configured to execute program instructions stored in the memory 51 to implement the steps of any of the above-mentioned image recognition method embodiments.
  • the electronic device 50 may include but is not limited to a microcomputer and a server.
  • the electronic device 50 may also include mobile devices such as a notebook computer and a tablet computer, which are not limited herein.
  • the processor 52 is configured to control itself and the memory 51 to implement the steps of any of the above-mentioned image recognition method embodiments.
  • the processor 52 may also be referred to as a CPU (Central Processing Unit, central processing unit).
  • the processor 52 may be an integrated circuit chip with signal processing capabilities.
  • the processor 52 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA), or other Programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the processor 52 may be jointly implemented by an integrated circuit chip.
  • the above solution can improve the efficiency and accuracy of image recognition.
  • FIG. 6 is a schematic diagram of a framework of an embodiment of the computer-readable storage medium 60 of the present application.
  • the computer-readable storage medium 60 stores program instructions 601 that can be executed by the processor, and the program instructions 601 are used to implement the steps of any of the foregoing image recognition method embodiments.
  • the above solution can improve the efficiency and accuracy of image recognition.
  • the disclosed method and device can be implemented in other ways.
  • the device implementation described above is only illustrative, for example, the division of modules or units is only a logical function division, and there may be other divisions in actual implementation, for example, units or components can be combined or integrated. To another system, or some features can be ignored, or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of this embodiment.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of the present application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor execute all or part of the steps of the methods in the various embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Evolutionary Biology (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Image Analysis (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Hardware Redundancy (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

The present application discloses an image recognition method and a related apparatus, and a device. The image recognition method comprises: acquiring at least one medical image to be recognized which is obtained by scanning, and determining a target region, corresponding to a target organ, in each of said medical image; performing feature extraction on image data of each target region, so as to obtain an individual feature representation of each of said medical image; fusing individual feature representations of said at least one medical image, so as to obtain a global feature representation; and by using the individual feature representation and the global feature representation of each of said medical image, determining the category, to which each of said medical image belongs, of a scanned image. The described solution can increase the efficiency and accuracy of image recognition.

Description

图像识别方法及相关装置、设备Image recognition method and related device and equipment
相关申请的交叉引用Cross-references to related applications
本申请基于申请号为202010246970.5、申请日为2020年03月31日，申请名称为“图像识别方法及相关装置、设备”的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此以引入方式结合在本申请中。This application is filed based on a Chinese patent application with the application number 202010246970.5, the application date being March 31, 2020, and the application name "Image recognition method and related devices and equipment", and the priority of the Chinese patent application is requested. The Chinese patent The entire content of the application is incorporated into this application by way of introduction.
技术领域Technical field
本申请涉及人工智能技术领域，特别是涉及一种图像识别方法及相关装置、设备。This application relates to the field of artificial intelligence technology, in particular to an image recognition method and related devices and equipment.
背景技术Background technique
CT(Computed Tomography，计算机断层扫描)和MRI(Magnetic Resonance Imaging，核磁共振扫描)等医学图像在临床具有重要意义。为了使医学图像应用于临床，一般需要扫描得到至少一种扫描图像类别的医学图像。以与肝脏相关的临床为例，扫描图像类别往往包括与时序有关的造影前平扫、动脉早期、动脉晚期、门脉期、延迟期等等，此外，扫描图像类别还可以包含与扫描参数有关的T1加权反相成像、T1加权同相成像、T2加权成像、扩散加权成像、表面扩散系数成像等等。Medical images such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging, MRI scan) have important clinical significance. In order for medical images to be used in clinics, it is generally necessary to scan medical images of at least one type of scanned image. Taking liver-related clinics as an example, scan image categories often include timing-related pre-contrast scan, early arterial phase, late arterial phase, portal phase, delay phase, etc. In addition, scan image categories can also include scan parameters related T1-weighted inverse imaging, T1-weighted in-phase imaging, T2-weighted imaging, diffusion-weighted imaging, surface diffusion coefficient imaging, etc.
目前，在扫描过程中，通常需要放射科医师鉴别扫描得到的医学图像的扫描图像类别，以确保获取所需要的医学图像；或者，在住院或门诊诊疗时，通常需要医生对扫描得到的医学图像进行识别，判断每一医学图像的扫描图像类别，再进行阅片。然而，上述通过人工识别医学图像的扫描图像类别的方式，效率较低，且易受主观影响而难以确保准确性。故此，如何提高图像识别的效率和准确性成为亟待解决的问题。At present, during the scanning process, the radiologist is usually required to identify the scanned image category of the scanned medical image to ensure that the required medical image is obtained; or, during hospitalization or outpatient treatment, the doctor is usually required to review the scanned medical image Recognize, determine the scanned image category of each medical image, and then read the image. However, the above-mentioned method of manually identifying the scanned image category of the medical image has low efficiency, and is subject to subjective influence and is difficult to ensure accuracy. Therefore, how to improve the efficiency and accuracy of image recognition has become an urgent problem to be solved.
发明内容Summary of the invention
本申请提供一种图像识别方法及相关装置、设备。This application provides an image recognition method and related devices and equipment.
本申请第一方面提供了一种图像识别方法，包括：获取至少一个扫描得到的待识别医学图像，并分别确定每个待识别医学图像中与目标脏器对应的目标区域；分别对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示；将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示；利用每一待识别医学图像的个体特征表示和全局特征表示，确定每一待识别医学图像所属的扫描图像类别。The first aspect of the present application provides an image recognition method, including: acquiring at least one scanned medical image to be recognized, and respectively determining the target area corresponding to the target organ in each medical image to be recognized; Perform feature extraction on regional image data to obtain the individual feature representation of each medical image to be recognized; fuse the individual feature representations of at least one medical image to be recognized to obtain a global feature representation; use the individual feature representation of each medical image to be recognized And the global feature representation to determine the scanned image category to which each medical image to be recognized belongs.
因此，通过获取至少一个扫描得到的待识别医学图像，并分别确定每个待识别医学图像中与目标脏器对应的目标区域，从而分别对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示，能够排除其他脏器的干扰，有利于提高识别准确性，并将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示，进而每一待识别医学图像的个体特征表示和全局特征表示，不仅能够表示待识别医学图像自身的特征，还能够表示其他待识别医学图像的差异，从而在利用每一待识别医学图像的个体特征表示和全局特征表示，确定每一待识别医学图像所属的扫描图像类别时，能够提高图像识别的准确性，且通过特征表示来进行图像识别，能够免于人工参与，故能够提高图像识别的效率。Therefore, by acquiring at least one scanned medical image to be recognized, and respectively determining the target area corresponding to the target organ in each medical image to be recognized, the feature extraction of the image data of each target area is performed to obtain each The individual feature representation of the medical image to be recognized can eliminate interference from other organs, which is conducive to improving the accuracy of recognition, and the individual feature representations of at least one medical image to be recognized are merged to obtain a global feature representation, and then each medical image to be recognized The individual feature representation and the global feature representation of the image can not only represent the characteristics of the medical image to be recognized, but also the difference of other medical images to be recognized, so that the individual feature representation and global feature representation of each medical image to be recognized can be used. When determining the scanned image category to which each medical image to be recognized belongs, the accuracy of image recognition can be improved, and image recognition can be performed through feature representation, which can avoid manual participation, so the efficiency of image recognition can be improved.
其中，将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示包 括：将至少一个待识别医学图像的个体特征表示进行全局池化处理，得到全局特征表示。Wherein, fusing the individual feature representations of at least one medical image to be identified to obtain a global feature representation includes: performing global pooling processing on the individual feature representations of at least one medical image to be identified to obtain a global feature representation.
因此，通过将至少一个待识别医学图像的个体特征表示进行全局池化处理，能够快速方便地得到全局特征表示，故能够有利于提高后续图像识别的效率。Therefore, by performing global pooling processing on the individual feature representation of at least one medical image to be recognized, the global feature representation can be obtained quickly and conveniently, which can help improve the efficiency of subsequent image recognition.
其中，将至少一个待识别医学图像的个体特征表示进行全局池化处理，得到全局特征表示包括：将至少一个待识别医学图像的个体特征表示进行全局最大池化处理，得到第一全局特征表示；以及，将至少一个待识别医学图像的个体特征表示进行全局平均池化处理，得到第二全局特征表示；将第一全局特征表示和第二全局特征表示进行拼接处理，得到全局特征表示。Wherein, subjecting at least one individual feature representation of the medical image to be identified to global pooling processing to obtain the global feature representation includes: subjecting at least one individual feature representation of the medical image to be identified to global maximum pooling processing to obtain the first global feature representation; And, performing global average pooling processing on the individual feature representations of at least one medical image to be recognized to obtain a second global feature representation; and performing stitching processing on the first global feature representation and the second global feature representation to obtain a global feature representation.
因此，通过将至少一个待识别医学图像的个体特征表示进行全局最大池化处理，得到第一全局特征表示，并将至少一个待识别医学图像的个体特征表示进行全局平均池化处理，得到第二全局特征表示，从而将第一全局特征表示和第二全局特征表示进行拼接处理，得到全局特征表示，故能够有利于后续准确地表示每一待识别医学图像与其他待识别医学图像之间的差异，从而能够有利于提高后续图像识别的准确性。Therefore, by subjecting at least one individual feature representation of the medical image to be recognized to global maximum pooling processing, the first global feature representation is obtained, and performing global average pooling processing on the individual feature representation of at least one medical image to be recognized to obtain the second Global feature representation, so that the first global feature representation and the second global feature representation are stitched together to obtain the global feature representation, so it can help to accurately represent the difference between each medical image to be recognized and other medical images to be recognized. , Which can help improve the accuracy of subsequent image recognition.
其中，利用每一待识别医学图像的个体特征表示和全局特征表示，确定每一待识别医学图像所属的扫描图像类别包括：利用每一待识别医学图像的个体特征表示和全局特征表示，得到每一待识别医学图像的最终特征表示，利用每个待识别医学图像的最终特征表示，确定每一待识别医学图像所属的扫描图像类别。Among them, using the individual feature representation and global feature representation of each medical image to be recognized to determine the scanned image category to which each medical image to be recognized belongs includes: using the individual feature representation and global feature representation of each medical image to be recognized to obtain each A final feature representation of the medical image to be recognized, using the final feature representation of each medical image to be recognized to determine the scanned image category to which each medical image to be recognized belongs.
因此，利用每一待识别医学图像的个体特征表示和全局特征表示所得到的最终特征表示，不仅能够表示待识别医学图像自身的特征，还能够表示其他待识别医学图像的差异，从而在利用每个待识别医学图像的最终特征表示，确定每一待识别医学图像所属的扫描图像类别时，能够提高图像识别的准确性。Therefore, the final feature representation obtained by using the individual feature representation and the global feature representation of each medical image to be recognized can not only represent the characteristics of the medical image to be recognized, but also the difference of other medical images to be recognized, so as to use each The final feature representation of each medical image to be recognized can improve the accuracy of image recognition when determining the scanned image category to which each medical image to be recognized belongs.
其中，利用每一待识别医学图像的个体特征表示和全局特征表示，得到每一待识别医学图像的最终特征表示包括：分别将每一待识别医学图像的个体特征表示和全局特征表示进行拼接处理，得到待识别医学图像对应的最终特征表示。Among them, using the individual feature representation and the global feature representation of each medical image to be recognized to obtain the final feature representation of each medical image to be recognized includes: stitching the individual feature representation and the global feature representation of each medical image to be recognized respectively , Get the final feature representation corresponding to the medical image to be recognized.
因此，通过分别将每一待识别医学图像的个体特征表示和全局特征表示进行拼接处理，能够快速得到待识别医学图像对应的最终特征表示，故能够有利于提高后续图像识别的效率。Therefore, by separately stitching the individual feature representation and the global feature representation of each medical image to be recognized, the final feature representation corresponding to the medical image to be recognized can be quickly obtained, which can help improve the efficiency of subsequent image recognition.
其中，分别对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示包括：利用识别网络的特征提取子网络对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示；将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示，利用每一待识别医学图像的个体特征表示和全局特征表示，得到每一待识别医学图像的最终特征表示包括：利用识别网络的融合模块将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示，并利用每一待识别医学图像的个体特征表示和全局特征表示，得到每一待识别医学图像的最终特征表示；利用每个待识别医学特征的最终特征表达，确定每一待识别医学图像所属的扫描图像类别，包括：利用识别网络的分类子网络对每一待识别医学图像的最终特征表示进行预测分类，得到每一待识别医学图像所属的扫描图像类别。Among them, performing feature extraction on the image data of each target area to obtain the individual feature representation of each medical image to be recognized includes: using the feature extraction sub-network of the recognition network to perform feature extraction on the image data of each target area to obtain each The individual feature representations of the medical images to be recognized; the individual feature representations of at least one medical image to be recognized are fused to obtain the global feature representation, and the individual feature representation and the global feature representation of each medical image to be recognized are used to obtain each The final feature representation of the medical image includes: using the fusion module of the recognition network to fuse the individual feature representation of at least one medical image to be recognized to obtain a global feature representation, and use the individual feature representation and global feature representation of each medical image to be recognized, Obtain the final feature representation of each medical image to be recognized; use the final feature expression of each medical feature to be recognized to determine the scanned image category to which each medical image to be recognized belongs, including: using the classification sub-network of the recognition network to Recognizing the final feature of the medical image means performing predictive classification to obtain the scanned image category to which each medical image to be recognized belongs.
因此，通过利用识别网络的特征提取子网络对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示，并利用识别网络的融合模块将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示，利用每一待识别医学图像的个体特征表示和全局特征表示，得到每一待识别医学图像的最终特征表示，从而利用识别网络的分类子网络对每一待识别医学图像的最终特征表示进行预测分类，得到每一待识别医学图像所属的扫描图像类别，故能够通过识别网络最终获得待识别医学图像所属的扫描图像类别，从而能够进一步提高图像识别的效率。Therefore, by using the feature extraction sub-network of the recognition network to perform feature extraction on the image data of each target area, the individual feature representation of each medical image to be recognized is obtained, and the fusion module of the recognition network is used to extract the features of at least one medical image to be recognized. The individual feature representations are fused to obtain a global feature representation. The individual feature representation and global feature representation of each medical image to be recognized are used to obtain the final feature representation of each medical image to be recognized, so that the classification sub-network of the recognition network is used for each The final feature of the medical image to be recognized indicates that the predicted classification is performed to obtain the scanned image category to which each medical image to be recognized belongs. Therefore, the scanned image category to which the medical image to be recognized belongs can be finally obtained through the recognition network, which can further improve the efficiency of image recognition .
其中，识别网络是利用样本医学图像训练得到的，每次训练识别网络所使用的样本医学图像数量不固定。Among them, the recognition network is obtained by training with sample medical images, and the number of sample medical images used for each training of the recognition network is not fixed.
因此，每次训练识别网络采用的样本医学图像的数量并不固定，能够随机化样本医学图像的数量，从而能够有利于在不同机构不同扫描协议下扫描图像类别有所缺失时，也能够准确地进行图像识别，进而能够提高图像识别准确性。Therefore, the number of sample medical images used in each training recognition network is not fixed, and the number of sample medical images can be randomized, which can help to accurately scan image categories under different scanning protocols in different institutions. Image recognition can improve the accuracy of image recognition.
其中，特征提取子网络包括至少一组顺序连接的稠密卷积块和池化层；和/或，识别网络包括预设数量个特征提取子网络；利用识别网络的特征提取子网络对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示包括：将每一目标区域的图像数据分别输入对应一个特征提取子网络进行特征提取，得到每个待识别医学图像的个体特征表示。Wherein, the feature extraction sub-network includes at least one set of sequentially connected dense convolution blocks and pooling layers; and/or, the recognition network includes a preset number of feature extraction sub-networks; the feature extraction sub-network of the recognition network is used for each target Performing feature extraction on the image data of the region to obtain the individual feature representation of each medical image to be recognized includes: inputting the image data of each target region into a corresponding feature extraction sub-network for feature extraction, and obtaining the individual of each medical image to be recognized Feature representation.
因此，特征提取子网络包括至少一组顺序连接的稠密卷积块和池化层，故通过稠密卷积块的连接策略，即每一卷积层下的特征与下一层紧密拼接，并传递后后面的每一层，能够有效的缓解梯度消失问题，且加强特征传播以及特征复用，并能够极大地减少参数数量；而将识别网络设置为包括预设数量个特征提取子网络，并将每一目标区域的图像数据分别输入对应一个特征提取子网络进行特征提取，得到每个待识别医学图像的个体特征表示，能够将至少一个目标区域的图像数据的特征提取操作并行处理，故能够有利于提高图像识别的效率。Therefore, the feature extraction sub-network includes at least one set of dense convolutional blocks and pooling layers that are connected in sequence. Therefore, the dense convolutional block connection strategy is adopted, that is, the features under each convolutional layer are tightly spliced with the next layer and transmitted Each subsequent layer can effectively alleviate the problem of gradient disappearance, strengthen feature propagation and feature reuse, and can greatly reduce the number of parameters; and the recognition network is set to include a preset number of feature extraction sub-networks, and The image data of each target area is input into a corresponding feature extraction sub-network for feature extraction, and the individual feature representation of each medical image to be recognized is obtained. The feature extraction operation of the image data of at least one target area can be processed in parallel. Conducive to improving the efficiency of image recognition.
其中，分别确定每个待识别医学图像中与目标脏器对应的目标区域包括：利用脏器检测网络分别对至少一个待识别医学图像进行检测，得到目标脏器第一位置信息和目标脏器的毗邻脏器的第二位置信息；利用第一位置信息和第二位置信息，确定目标脏器对应的目标区域。Wherein, respectively determining the target area corresponding to the target organ in each medical image to be recognized includes: using an organ detection network to detect at least one medical image to be recognized to obtain first position information of the target organ and information about the target organ. The second position information of the adjacent organ; the first position information and the second position information are used to determine the target area corresponding to the target organ.
因此，利用脏器检测网络分别对至少一个待识别医学图像进行检测，得到目标脏器的第一位置信息和目标脏器的毗邻脏器的第二位置信息，故不仅能够考虑所需识别的目标脏器，还能够考虑周边毗邻脏器，从而利用第一位置信息和第二位置信息，确定目标脏器对应的目标区域，能够确保在经手术治疗等情况下脏器形态发生改变时，也能够定位得到目标脏器对应的目标区域，故能够提高图像识别的鲁棒性。Therefore, the organ detection network is used to detect at least one medical image to be recognized to obtain the first position information of the target organ and the second position information of the adjacent organs of the target organ, so that not only the target to be recognized can be considered Organs can also consider the surrounding organs, so that the first position information and the second position information can be used to determine the target area corresponding to the target organ, which can ensure that the shape of the organ changes under surgical treatment, etc. The target area corresponding to the target organ is obtained by positioning, so the robustness of image recognition can be improved.
其中，待识别医学图像为三维图像，利用脏器检测网络分别对至少一个待识别医学图像进行检测，得到目标脏器第一位置信息和目标脏器的毗邻脏器的第二位置信息之前，方法还包括：将每一待识别医学图像沿冠状面进行划分，得到多个三维子图像；将每一子图像沿垂直于冠状面的方向进行投影，得到对应的二维子图像；利用脏器检测网络分别对至少一个待识别医学图像进行检测，得到目标脏器第一位置信息和目标脏器的毗邻脏器的第二位置信息包括：利用脏器检测网络对至少一个待识别医学图像对应的二维子图像进行检测，得到第一位置信息和第二位置信息。Wherein, the medical image to be recognized is a three-dimensional image, and the organ detection network is used to detect at least one medical image to be recognized to obtain the first position information of the target organ and the second position information of the adjacent organs of the target organ. It also includes: dividing each medical image to be identified along the coronal plane to obtain multiple three-dimensional sub-images; projecting each sub-image in a direction perpendicular to the coronal plane to obtain the corresponding two-dimensional sub-image; using organ detection The network detects at least one medical image to be identified, and obtains the first position information of the target organ and the second position information of the adjacent organs of the target organ. Dimension sub-images are detected to obtain first position information and second position information.
因此，在待识别医学图像为三维图像时，将每一待识别医学图像沿冠状面进行划分，得到多个三维子图像，并将每一子图像沿垂直于冠状面的方向进行投影，得到对应的二维子图像，从而利用脏器检测网络对至少一个待识别医学图像对应的二维子图像进行检测，得到第一位置信息和第二位置信息，能够进一步提高目标脏器对应的目标区域定位的准确性。Therefore, when the medical image to be recognized is a three-dimensional image, each medical image to be recognized is divided along the coronal plane to obtain multiple three-dimensional sub-images, and each sub-image is projected in a direction perpendicular to the coronal plane to obtain the corresponding Therefore, the organ detection network is used to detect the two-dimensional sub-image corresponding to at least one medical image to be recognized, and the first position information and the second position information are obtained, which can further improve the target area location corresponding to the target organ Accuracy.
其中，目标脏器为肝脏，毗邻脏器包括肾脏、脾脏中的至少一者；和/或，第一位置信息包括目标脏器对应区域的至少一组对角顶点位置和对应区域的尺寸，第二位置信息至少包括毗邻脏器对应区域的至少一个顶点位置。Wherein, the target organ is the liver, and the adjacent organs include at least one of the kidney and the spleen; and/or, the first position information includes at least one set of diagonal vertex positions of the corresponding area of the target organ and the size of the corresponding area. The second position information includes at least one vertex position of the corresponding area adjacent to the organ.
因此，将目标脏器设置为肝脏，毗邻脏器设置为包括肾脏、脾脏中的至少一者，能够有利于定位得到肝脏对应的目标区域；将第一位置信息设置为包括目标脏器对应区域的至少一组对角顶点位置和对应区域的尺寸，第二位置信息设置为至少包括毗邻脏器对 应区域的至少一个顶点位置，能够有利于精确地定位目标脏器对应的目标区域。Therefore, setting the target organ as the liver, and setting the adjacent organ to include at least one of the kidney and the spleen can help locate the target area corresponding to the liver; setting the first position information to include the area corresponding to the target organ At least one set of diagonal vertex positions and the size of the corresponding area, and the second position information is set to include at least one vertex position adjacent to the corresponding area of the organ, which can facilitate accurate positioning of the target area corresponding to the target organ.
其中，利用每一待识别医学图像的个体特征表示和全局特征表示，确定每一待识别医学图像所属的扫描图像类别之后，方法还包括以下至少一者：将至少一个待识别医学图像按照其扫描图像类别进行排序；若待识别医学图像的扫描图像类别存在重复，则输出第一预警信息，以提示扫描人员；若至少一个待识别医学图像的扫描图像类别中不存在预设扫描图像类别，则输出第二预警信息，以提示扫描人员。Wherein, after the individual feature representation and global feature representation of each medical image to be recognized are used to determine the scanned image category to which each medical image to be recognized belongs, the method further includes at least one of the following: scanning at least one medical image to be recognized according to its scan Sort the image categories; if the scanned image categories of the medical images to be recognized are repeated, the first warning information is output to remind the scanner; if there is no preset scanned image category in the scanned image categories of at least one medical image to be recognized, then Output the second warning message to remind the scanner.
因此，在确定得到每一待识别医学图像所属的扫描图像类别之后，执行将至少一个待识别医学图像按照其扫描图像类别进行排序，能够提高医生阅片的便捷性；在待识别医学图像的扫描图像类别存在重复时，输出第一预警信息，以提示扫描人员，在至少一个待识别医学图像的扫描图像类别中不存在预设扫描图像类别时，输出第二预警信息，以提示扫描人员，能够在扫描过程中实现图像质控，以在与实际相悖时，能够及时纠错，避免病人二次挂号。Therefore, after determining the scanned image category to which each medical image to be recognized belongs, execute sorting of at least one medical image to be recognized according to its scanned image category, which can improve the convenience of doctors in reading the image; When the image categories are repeated, the first warning information is output to remind the scanner, and when the preset scanned image category does not exist in the scan image category of at least one medical image to be recognized, the second warning information is output to remind the scanner, The image quality control is implemented during the scanning process, so that when it is contrary to the actual situation, the error can be corrected in time to avoid the second registration of the patient.
其中，分别对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示之前，方法还包括：对每一目标区域的图像数据进行预处理，其中，预处理包括以下至少一种：将目标区域的图像尺寸调整至预设尺寸，将目标区域的图像强度归一化至预设范围。Among them, before the feature extraction is performed on the image data of each target region, and the individual feature representation of each medical image to be recognized is obtained, the method further includes: preprocessing the image data of each target region, wherein the preprocessing includes the following At least one: adjusting the image size of the target area to a preset size, and normalizing the image intensity of the target area to a preset range.
因此，在特征提取之前，对每一目标区域的图像数据进行预处理，且预处理包括以下至少一种：将目标区域的图像尺寸调整至预设尺寸，将目标区域的图像强度归一化至预设范围，故能够有利于提高后续图像识别的准确性。Therefore, before feature extraction, the image data of each target area is preprocessed, and the preprocessing includes at least one of the following: adjusting the image size of the target area to a preset size, and normalizing the image intensity of the target area to The preset range can help improve the accuracy of subsequent image recognition.
本申请第二方面提供了一种图像识别装置，包括：区域获取模块、特征提取模块、融合处理模块和类别确定模块，区域获取模块配置为获取至少一个扫描得到的待识别医学图像，并分别确定每个待识别医学图像中与目标脏器对应的目标区域；特征提取模块配置为分别对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示；融合处理模块配置为将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示；类别确定模块配置为利用每一待识别医学图像的个体特征表示和全局特征表示，确定每一待识别医学图像所属的扫描图像类别。The second aspect of the present application provides an image recognition device, including: a region acquisition module, a feature extraction module, a fusion processing module, and a category determination module. The region acquisition module is configured to acquire at least one scanned medical image to be identified, and respectively determine The target region corresponding to the target organ in each medical image to be recognized; the feature extraction module is configured to extract features from the image data of each target region to obtain the individual feature representation of each medical image to be recognized; configuration of the fusion processing module In order to fuse the individual feature representations of at least one medical image to be identified to obtain a global feature representation; the category determination module is configured to use the individual feature representation and the global feature representation of each medical image to be identified to determine which medical image belongs to Scanned image category.
本申请第三方面提供了一种电子设备，包括相互耦接的存储器和处理器，处理器配置为执行存储器中存储的程序指令，以实现上述第一方面中的图像识别方法。A third aspect of the present application provides an electronic device including a memory and a processor coupled to each other, and the processor is configured to execute program instructions stored in the memory to implement the image recognition method in the first aspect.
本申请第四方面提供了一种计算机可读存储介质，其上存储有程序指令，程序指令被处理器执行时实现上述第一方面中的图像识别方法。The fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, and the program instructions implement the image recognition method in the first aspect when the program instructions are executed by a processor.
上述方案，通过获取至少一个扫描得到的待识别医学图像，并分别确定每个待识别医学图像中与目标脏器对应的目标区域，从而分别对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示，能够排除其他脏器的干扰，有利于提高识别准确性，并将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示，进而每一待识别医学图像的个体特征表示和全局特征表示，不仅能够表示待识别医学图像自身的特征，还能够表示其他待识别医学图像的差异，从而在利用每一待识别医学图像的个体特征表示和全局特征表示，确定每一待识别医学图像所属的扫描图像类别时，能够提高图像识别的准确性，且通过特征表示来进行图像识别，能够免于人工参与，故能够提高图像识别的效率。In the above solution, by acquiring at least one scanned medical image to be recognized, and respectively determining the target area corresponding to the target organ in each medical image to be recognized, the feature extraction is performed on the image data of each target area to obtain each The individual feature representations of a medical image to be recognized can eliminate interference from other organs, which is beneficial to improve the accuracy of recognition, and the individual feature representations of at least one medical image to be recognized are merged to obtain a global feature representation, and then each to be recognized The individual feature representation and global feature representation of medical images can not only represent the characteristics of the medical image to be recognized, but also the differences of other medical images to be recognized, so that the individual feature representation and global feature representation of each medical image to be recognized can be used. When determining the scanned image category to which each medical image to be recognized belongs, the accuracy of image recognition can be improved, and image recognition can be performed through feature representation, which can avoid manual participation, so the efficiency of image recognition can be improved.
附图说明Description of the drawings
图1是本申请图像识别方法一实施例的流程示意图；FIG. 1 is a schematic flowchart of an embodiment of an image recognition method according to the present application;
图2是确定待识别医学图像所属的扫描图像类别过程的状态示意图；2 is a schematic diagram of the state of the process of determining the scanned image category to which the medical image to be recognized belongs;
图3是图1中步骤S11一实施例的流程示意图；FIG. 3 is a schematic flowchart of an embodiment of step S11 in FIG. 1;
图4是本申请图像识别装置一实施例的框架示意图；4 is a schematic diagram of the framework of an embodiment of the image recognition device of the present application;
图5是本申请电子设备一实施例的框架示意图；FIG. 5 is a schematic diagram of the framework of an embodiment of the electronic device of the present application;
图6是本申请计算机可读存储介质一实施例的框架示意图。Fig. 6 is a schematic diagram of a framework of an embodiment of a computer-readable storage medium according to the present application.
具体实施方式Detailed ways
下面结合说明书附图，对本申请实施例的方案进行详细说明。The following describes the solutions of the embodiments of the present application in detail with reference to the drawings in the specification.
以下描述中，为了说明而不是为了限定，提出了诸如特定***结构、接口、技术之类的具体细节，以便透彻理解本申请。In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structure, interface, technology, etc. are proposed for a thorough understanding of the present application.
本文中术语“***”和“网络”在本文中常被可互换使用。本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。此外，本文中的“多”表示两个或者多于两个。The terms "system" and "network" in this article are often used interchangeably in this article. The term "and/or" in this article is only an association relationship describing the associated objects, which means that there can be three relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, exist alone B these three situations. In addition, the character "/" in this text generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this document means two or more than two.
在本申请的实施例中，图像识别方法的执行主体可以是图像识别装置，例如，图像识别方法可以由终端设备或服务器或其它处理设备执行，其中，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字处理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等。在一些可能的实现方式中，图像识别方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。In the embodiment of the present application, the execution subject of the image recognition method may be an image recognition device. For example, the image recognition method may be executed by a terminal device or a server or other processing equipment. The terminal device may be a user equipment (User Equipment, UE). ), mobile devices, user terminals, terminals, cellular phones, cordless phones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. In some possible implementations, the image recognition method can be implemented by a processor calling computer-readable instructions stored in the memory.
请参阅图1，图1是本申请图像识别方法一实施例的流程示意图。具体而言，可以包括如下步骤：Please refer to FIG. 1. FIG. 1 is a schematic flowchart of an embodiment of an image recognition method according to the present application. Specifically, it can include the following steps:
步骤S11：获取至少一个扫描得到的待识别医学图像，并分别确定每个待识别医学图像中与目标脏器对应的目标区域。Step S11: Obtain at least one scanned medical image to be recognized, and respectively determine the target area corresponding to the target organ in each medical image to be recognized.
待识别医学图像可以包括CT图像、MR图像，在此不做限定。在一个具体的实施场景中，待识别医学图像可以是对腹部、胸部等区域进行扫描得到的，具体可以根据实际应用情况而设置，在此不做限定。例如，当肝脏、脾脏、肾脏为需要诊疗的目标脏器时，可以对腹部进行扫描，得到待识别医学图像；或者，当心脏、肺为需要诊疗的目标脏器时，可以对胸部进行扫描，得到待识别医学图像，其他情况可以以此类推，在此不再一一举例。在另一个具体的实施场景中，扫描方式可以是平扫、增强扫描等方式，在此不做限定。在又一个具体的实施场景中，待识别医学图像可以是三维图像，待识别医学图像中目标脏器对应的目标区域可以是三维区域，在此不做限定。The medical images to be recognized may include CT images and MR images, which are not limited here. In a specific implementation scenario, the medical image to be recognized may be obtained by scanning the abdomen, chest and other areas, and may be specifically set according to actual application conditions, which is not limited here. For example, when the liver, spleen, and kidney are the target organs that need diagnosis and treatment, the abdomen can be scanned to obtain medical images to be identified; or, when the heart and lungs are the target organs that need diagnosis and treatment, the chest can be scanned. Obtain the medical image to be recognized, and other situations can be deduced by analogy, so we will not give examples one by one here. In another specific implementation scenario, the scanning mode may be plain scanning, enhanced scanning, etc., which are not limited here. In another specific implementation scenario, the medical image to be recognized may be a three-dimensional image, and the target area corresponding to the target organ in the medical image to be recognized may be a three-dimensional area, which is not limited here.
目标脏器可以根据实际应用而设置，例如，当医生需要判断肝脏是否产生病变以及病变程度等时，目标脏器可以是肝脏；或者，当医生需要判断肾脏是否产生病变以及病变程度时，目标脏器可以是肾脏，其他情况可以根据实际应用而进行设置，在此不再一一举例。在一个实施场景中，可以预先训练一用于对目标脏器进行检测的脏器检测网络，从而可以直接利用脏器检测网络对每一待识别医学图像进行检测，得到对应的目标区域。The target organ can be set according to the actual application. For example, when the doctor needs to judge whether the liver has lesions and the extent of the disease, the target organ can be the liver; or when the doctor needs to judge whether the kidney has lesions and the extent of the disease, the target organ The device can be the kidney, and other conditions can be set according to the actual application, so we will not give examples one by one here. In an implementation scenario, an organ detection network for detecting target organs can be pre-trained, so that the organ detection network can be directly used to detect each medical image to be identified to obtain the corresponding target area.
步骤S12：分别对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示。Step S12: Perform feature extraction on the image data of each target area respectively to obtain the individual feature representation of each medical image to be recognized.
在一个实施场景中，为了提高后续图像识别的准确性，在对目标区域的图像数据进行特征提取之前，还可以对每一目标区域的图像数据进行预处理，具体地，预处理可以包括将目标区域的图像尺寸调整至预设尺寸(例如，32*256*256)。或者，预处理还可以包括将目标区域的图像强度归一化至预设范围(例如，0至1的范围)，在一个具体的实施场景中，可以采用灰度累积分布函数下预设比例(例如，99.9％)对应的灰度值作为归一化的钳位值，从而能够加强目标区域的图像数据的对比度，有利于提升后续图像识别的准确性。In an implementation scenario, in order to improve the accuracy of subsequent image recognition, before feature extraction is performed on the image data of the target area, the image data of each target area may also be preprocessed. Specifically, the preprocessing may include The image size of the area is adjusted to a preset size (for example, 32*256*256). Alternatively, the preprocessing may also include normalizing the image intensity of the target area to a preset range (for example, the range of 0 to 1). In a specific implementation scenario, the preset ratio ( For example, the gray value corresponding to 99.9%) is used as the normalized clamp value, so that the contrast of the image data of the target area can be enhanced, which is beneficial to improve the accuracy of subsequent image recognition.
在一个实施场景中，为了提升特征提取的便利性，还可以预先训练一识别网络，识别网络可以包括用于特征提取的特征提取子网络，从而可以利用特征提取子网络对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示。In an implementation scenario, in order to improve the convenience of feature extraction, a recognition network can also be pre-trained. The recognition network can include a feature extraction sub-network for feature extraction, so that the feature extraction sub-network can be used to analyze the image of each target area. The data is feature-extracted, and the individual feature representation of each medical image to be recognized is obtained.
在一个具体的实施场景中，特征提取子网络包括至少一组顺序连接的稠密卷积块(Dense Block)和池化层，稠密卷积块中每一层卷积下特征与下一层进行紧密拼接，并且传递后后面的每一层，从而使得特征和梯度的传递更加有效。具体地，特征提取子网络可以包括三组顺序连接的稠密卷积块和池化层，其中，除最后一组所包含的池化层为自适应池化外，其他组所包含的池化层为最大池化；此外，特征提取子网络还可以包括一组、两组、四组等其他数量组顺序连接的稠密卷积块(Dense Block)和池化层，在此不做限定。In a specific implementation scenario, the feature extraction sub-network includes at least a set of sequentially connected dense convolution blocks (Dense Block) and a pooling layer. The features of each layer of the dense convolution block are closely connected to the next layer. Splicing and transferring each layer afterwards makes the transfer of features and gradients more effective. Specifically, the feature extraction sub-network may include three sets of sequentially connected dense convolution blocks and pooling layers, where, except for the pooling layer contained in the last set of adaptive pooling, the pooling layers contained in other groups It is the maximum pooling; in addition, the feature extraction sub-network may also include a group, two groups, four groups, and other groups of dense convolution blocks (Dense Block) and a pooling layer connected in sequence, which are not limited here.
在另一个具体的实施场景中，识别网络中具体可以包括预设数量个特征提取子网络，从而可以将每一目标区域的图像数据分别输入对应一个特征提取子网络进行特征提取，得到每个待识别医学图像的个体特征表示，进而能够将每一目标区域的图像数据的特征提取操作并行处理，故能够提高特征提取的效率，能够有利于提高后续图像识别的效率，此外，预设数量可以大于或等于扫描图像类别的种类，例如，当扫描图像类别包括与时序有关的造影前平扫、动脉早期、动脉晚期、门脉期、延迟期时，预设数量可以设置为大于或等于5的整数，例如，5、6、7等等，在此不做限定；或者，当扫描图像类别包括与描参数有关的T1加权反相成像、T1加权同相成像、T2加权成像、扩散加权成像、表面扩散系数成像时，预设数量可以设置为大于或等于5的整数，例如，5、6、7等等，在此不做限定；或者，当扫描图像类别既包括与描参数有关的T1加权反相成像、T1加权同相成像、T2加权成像、扩散加权成像、表面扩散系数成像，也包括与时序有关的造影前平扫、动脉早期、动脉晚期、门脉期、延迟期时，预设数量可以设置为大于或等于10的整数，例如，10、11、12等等。具体地，动脉早期可以表示门静脉尚未增强，动脉晚期可以表示门静脉已被增强，门脉期可以表示门静脉已充分增强且肝脏血管已被前向性血流增强、肝脏软细胞组织在标记物下已达到峰值，延迟期可以表示门脉和动脉处于增强状态并弱于门脉期、且肝脏软细胞组织处于增强状态并弱于门脉期，其他扫描图像类别在此不再一一举例。In another specific implementation scenario, the recognition network may specifically include a preset number of feature extraction sub-networks, so that the image data of each target area can be input into a corresponding feature extraction sub-network for feature extraction, and each target area can be extracted. Recognize the individual feature representations of medical images, and then the feature extraction operations of the image data of each target area can be processed in parallel, so the efficiency of feature extraction can be improved, and the efficiency of subsequent image recognition can be improved. In addition, the preset number can be greater than Or equal to the category of the scanned image. For example, when the scanned image category includes timing-related pre-contrast scan, early arterial phase, late arterial phase, portal phase, and delay phase, the preset number can be set to an integer greater than or equal to 5 , For example, 5, 6, 7, etc., which are not limited here; or, when the scanned image category includes T1-weighted inverse imaging, T1-weighted in-phase imaging, T2-weighted imaging, diffusion-weighted imaging, surface diffusion related to tracing parameters In coefficient imaging, the preset number can be set to an integer greater than or equal to 5, for example, 5, 6, 7, etc., which are not limited here; or, when the scanned image category includes both T1 weighted inversion related to the tracing parameter Imaging, T1-weighted in-phase imaging, T2-weighted imaging, diffusion-weighted imaging, surface diffusion coefficient imaging, as well as timing-related pre-contrast scan, early arterial, late arterial, portal phase, and delay periods. The preset number can be set It is an integer greater than or equal to 10, for example, 10, 11, 12, and so on. Specifically, the early arterial phase can indicate that the portal vein has not been enhanced, the late arterial phase can indicate that the portal vein has been enhanced, and the portal phase can indicate that the portal vein has been fully enhanced and the liver vessels have been enhanced by forward blood flow, and the liver soft cell tissue has been under the markers. At the peak, the delay period can indicate that the portal vein and arteries are in an enhanced state and weaker than the portal stage, and the liver soft cell tissue is in an enhanced state and weaker than the portal stage. Other scan image categories will not be illustrated here.
请结合参阅图2，图2是确定待识别医学图像所属的扫描图像类别过程的状态示意图，如图2所示，以不同灰度填充的矩形框分别表示待识别医学图像1至待识别医学图像n中目标脏器对应的目标区域的图像数据提取到的个体特征表示1、个体特征表示2、个体特征表示3、……、个体特征表示n。Please refer to Figure 2 in conjunction. Figure 2 is a schematic diagram of the state of the process of determining the scanned image category to which the medical image to be recognized belongs. As shown in Figure 2, rectangular boxes filled with different gray levels represent medical image 1 to medical image to be recognized The individual feature representation 1, individual feature representation 2, individual feature representation 3, ..., individual feature representation n extracted from the image data of the target region corresponding to the target organ in n.
步骤S13：将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示。Step S13: Fusion of individual feature representations of at least one medical image to be identified to obtain a global feature representation.
在一个实施场景中，识别网络中还可以包括融合模块，从而可以利用融合模块将至少一个待识别医学图像的个体特征表示进行融合，进而得到全局特征表示。In an implementation scenario, the recognition network may also include a fusion module, so that the fusion module can be used to fuse the individual feature representations of at least one medical image to be recognized to obtain a global feature representation.
在另一个实施场景中，可以将至少一个待识别医学图像的个体特征表示进行全局池化处理，得到全局特征表示。具体地，可以将至少一个待识别医学图像的个体特征表示进行全局最大池化(Global Max Pooling，GMP)处理，得到第一全局特征表示，并将至少一个待识别医学图像的个体特征表示进行全局平均池化(Global Average Pooling，GAP)处理，得到第二全局特征表示，从而将第一特征表示和第二特征表示进行拼接处理，得到全局特征表示。请继续结合参阅图2，可以将个体特征表示1、个体特征表示2、个体特征表示3、……、个体特征表示n分别进行全局最大池化和全局平均池化，得到第一全局特征表示(图2中斜线填充矩形框)和第二全局特征表示(图2中网格线填充矩形框)，并将第一全局特征表示和第二全局特征表示进行拼接处理，得到全局特征表 示。In another implementation scenario, the individual feature representation of at least one medical image to be recognized may be subjected to global pooling processing to obtain a global feature representation. Specifically, the individual feature representation of at least one medical image to be recognized may be subjected to global maximum pooling (Global Max Pooling, GMP) processing to obtain a first global feature representation, and the individual feature representation of at least one medical image to be recognized can be global A global average pooling (GAP) process is performed to obtain a second global feature representation, so that the first feature representation and the second feature representation are spliced together to obtain a global feature representation. Please continue to refer to Figure 2 in combination, individual feature representation 1, individual feature representation 2, individual feature representation 3, ..., individual feature representation n are respectively subjected to global maximum pooling and global average pooling to obtain the first global feature representation ( The oblique line in FIG. 2 fills the rectangular frame) and the second global feature representation (the grid line fills the rectangular frame in FIG. 2), and the first global feature representation and the second global feature representation are spliced to obtain the global feature representation.
步骤S14：利用每一待识别医学图像的个体特征表示和全局特征表示，确定每一待识别医学图像所属的扫描图像类别。Step S14: Use the individual feature representation and the global feature representation of each medical image to be recognized to determine the scanned image category to which each medical image to be recognized belongs.
具体地，可以利用每一待识别医学图像的个体特征表示和全局特征表示，得到每一待识别医学图像的最终特征表示，再利用每个待识别医学图像的最终特征表示，确定每一待识别医学图像所属的扫描图像类别，从而最终特征表示不仅能够表示待识别医学图像自身的特征，还能够表示其他待识别医学图像的差异，进而在利用每个待识别医学图像的最终特征表示，确定每一待识别医学图像所属的扫描图像类别时，能够提高图像识别的准确性。为了得到每个待识别医学图像的最终特征表示，在一个具体的实施场景中，可以利用识别网络中的融合模块利用每一待识别医学图像的个体特征表示和全局特征表示，得到每一待识别医学图像的最终特征表示。在另一个具体的实施场景中，还可以将每一待识别医学图像的个体特征表示和全局特征表示进行拼接处理，得到待识别医学图像对应的最终特征表示。请结合参阅图2，如图2所示，以斜线填充矩形框表示的第一全局特征表示和以网格线填充矩形框表示的第二全局特征表示分别和以不同灰度填充矩形框表示的个体特征表示进行拼接处理，可以得到对应每一待识别医学图像的最终特征表示。Specifically, the individual feature representation and global feature representation of each medical image to be recognized can be used to obtain the final feature representation of each medical image to be recognized, and then the final feature representation of each medical image to be recognized can be used to determine each medical image to be recognized. The category of the scanned image to which the medical image belongs, so that the final feature representation can not only represent the characteristics of the medical image to be recognized, but also the difference of other medical images to be recognized, and then use the final feature representation of each medical image to be recognized to determine each When the scanned image category to which the medical image belongs can be recognized, the accuracy of image recognition can be improved. In order to obtain the final feature representation of each medical image to be recognized, in a specific implementation scenario, the fusion module in the recognition network can be used to use the individual feature representation and global feature representation of each medical image to be recognized to obtain each to be recognized The final feature representation of the medical image. In another specific implementation scenario, the individual feature representation and the global feature representation of each medical image to be recognized can also be spliced to obtain the final feature representation corresponding to the medical image to be recognized. Please refer to Figure 2 in combination. As shown in Figure 2, the first global feature represented by a rectangular box filled with diagonal lines is represented and the second global feature represented by a rectangular box filled with grid lines is represented respectively and represented by a rectangular box filled with different gray levels. The individual feature representations of is spliced, and the final feature representation corresponding to each medical image to be recognized can be obtained.
在一个实施场景中，识别网络中还可以包括分类子网络，从而可以利用分类子网络对每一待识别医学图像的最终特征表示进行预测分类，得到每一待识别医学图像所属的扫描图像类别。在一个具体的实施场景中，分类子网络中可以包括全连接层和softmax层，从而可以利用全连接层对每一待识别医学图像的最终特征表示进行特征连接，并利用softmax层进行概率归一化，得到每一待识别医学图像属于各个扫描图像类别的概率值，故可以将最大概率值对应的扫描图像类别作为待识别医学图像所属的扫描图像类别。In an implementation scenario, the recognition network may also include a classification sub-network, so that the classification sub-network can be used to predict and classify the final feature representation of each medical image to be recognized, and obtain the scanned image category to which each medical image to be recognized belongs. In a specific implementation scenario, the classification sub-network can include a fully connected layer and a softmax layer, so that the fully connected layer can be used to connect the final feature representation of each medical image to be recognized, and the softmax layer can be used for probability normalization Therefore, the probability value that each medical image to be recognized belongs to each scanned image category is obtained, so the scanned image category corresponding to the maximum probability value can be used as the scanned image category to which the medical image to be recognized belongs.
在一个具体的实施场景中，包含上述特征提取子网络、融合模块和分类子网络的识别网络可以是利用样本医学图像训练得到的。具体地，可以利用特征提取子网络对每一样本医学图像中标注的目标区域的图像数据进行特征提取，得到每个样本医学图像的个体特征表示，并利用融合模块将至少一个样本医学图像的个体特征表示进行融合，得到全局特征表示，利用每一样本医学图像的个体特征表示和全局特征表示，得到每一样本医学图像的最终特征表示，再利用分类子网络对每一样本医学图像的最终特征表示进行预测分类，得到每一样本医学图像所属的预测扫描图像类别，并利用每一样本医学图像的预测扫描图像类别和标注的真实扫描图像类别，确定识别网络的损失值(如交叉熵损失值)，最后利用损失值对识别网络的参数进行调整，以实现对识别网络的训练，具体可以采用随机梯度下降(Stochastic Gradient Descent，SGD)对参数进行调整。此外，为了提高识别网络的鲁棒性，每次训练识别网络所使用的样本医学图像数量可以不固定。具体而言，每次训练识别网络所使用的样本医学图像可以是属于同一对象的，且每次训练识别网络所使用的样本医学图像所属的扫描图像类别的种数可以不固定。例如，某一次训练识别网络所采用的样本医学图像属于T1加权反相成像、T1加权同相成像、T2加权成像，另一次训练识别网络所采用的样本医学图像属于扩散加权成像、表面扩散系数成像，具体可以根据实际应用情况进行设置，在此不再一一举例，从而能够随机化样本医学图像的数量，进而能够有利于在不同机构不同扫描协议下扫描图像类别有所缺失时，也能够准确地进行图像识别，进而能够提高识别网络的鲁棒性。此外，为了使识别结果不受统计学差异影响，还可以设置训练集和验证集，且从具有不同脏器损伤类型的对象中按照预设比例(如3：1)进行随机选择，分别作为训练集和验证集。In a specific implementation scenario, the recognition network including the feature extraction sub-network, the fusion module and the classification sub-network may be obtained by training using sample medical images. Specifically, the feature extraction sub-network can be used to perform feature extraction on the image data of the target area annotated in each sample medical image to obtain the individual feature representation of each sample medical image, and the fusion module can be used to extract the individual characteristics of at least one sample medical image. Feature representations are fused to obtain a global feature representation. The individual feature representation and global feature representation of each sample medical image are used to obtain the final feature representation of each sample medical image, and then the classification sub-network is used to determine the final feature of each sample medical image. It means to perform predictive classification to obtain the predicted scan image category to which each sample medical image belongs, and use the predicted scan image category of each sample medical image and the marked real scan image category to determine the loss value of the recognition network (such as cross-entropy loss value ), and finally use the loss value to adjust the parameters of the recognition network to realize the training of the recognition network. Specifically, Stochastic Gradient Descent (SGD) can be used to adjust the parameters. In addition, in order to improve the robustness of the recognition network, the number of sample medical images used for each training of the recognition network may not be fixed. Specifically, the sample medical images used for each training of the recognition network may belong to the same object, and the number of scanned image categories to which the sample medical images used for each training of the recognition network belongs may not be fixed. For example, the sample medical images used in a certain training recognition network belong to T1-weighted inverse imaging, T1-weighted in-phase imaging, and T2-weighted imaging, and the sample medical images used in another training recognition network belong to diffusion-weighted imaging and surface diffusion coefficient imaging. The specific settings can be set according to the actual application situation. I will not give examples one by one here, so that the number of sample medical images can be randomized, which can help to accurately scan image categories when different institutions and different scanning protocols are missing. Image recognition can improve the robustness of the recognition network. In addition, in order to prevent the recognition results from being affected by statistical differences, you can also set a training set and a validation set, and randomly select objects with different organ damage types according to a preset ratio (such as 3:1) as training Set and validation set.
在另一个具体的实施场景中，可以将上述经训练的识别网络设置于影像后处理工作站、摄片工作站、计算机辅助阅片***等，从而能够实现对待识别医学图像的自动识别， 提高识别效率。In another specific implementation scenario, the above-mentioned trained recognition network can be set in an image post-processing workstation, a filming workstation, a computer-aided image reading system, etc., so as to realize automatic recognition of medical images to be recognized and improve recognition efficiency.
在又一个具体的实施场景中，在验证阶段，可以将在一次扫描过程中属于同一对象的全部待识别医学图像在一次识别过程中，进行全部识别，从而能够对识别网络的性能进行全面验证；在应用阶段，可以将在一次扫描过程中属于同一对象的全部待识别医学图像在一次识别过程中，进行全部识别，从而能够考虑每一待识别医学图像与其他所有待识别医学图像之间的差异，进而能够有利于提高识别的准确性。In another specific implementation scenario, in the verification phase, all medical images to be recognized that belong to the same object in a scan process can be recognized in one recognition process, so that the performance of the recognition network can be fully verified; In the application stage, all medical images to be recognized that belong to the same object in one scan can be recognized in one recognition process, so that the difference between each medical image to be recognized and all other medical images to be recognized can be considered , Which in turn can help improve the accuracy of recognition.
在一个实施场景中，至少一个待识别医学图像为对同一对象扫描得到的，故为了便于医生阅片，在得到每一待识别医学图像所属的扫描图像类别之后，还可以将至少一个待识别医学图像按照其扫描图像类别进行排序，例如，可以按照T1加权反相成像、T1加权同相成像、造影前平扫、动脉早期、动脉晚期、门脉期、延迟期、T2加权成像、扩散加权成像、表面扩散系数成像的预设顺序进行排序，此外，预设顺序还可以根据医生习惯进行设置，在此不做限定，从而能够提高医生阅片的便捷性，此外，为了进一步提高阅片的便捷性，还可以将排序后的至少一个待识别医学图像在与待识别医学图像的数量对应的窗口中予以显示，例如，待识别医学图像的数量为5个，则可以在5个显示窗口中分别显示待识别医学图像。故此，能够降低医生翻阅多个待识别医学图像来回对照的时间，提升阅片效率。In an implementation scenario, at least one medical image to be recognized is obtained by scanning the same object. Therefore, in order to facilitate the doctor to read the image, after obtaining the scanned image category to which each medical image to be recognized belongs, at least one medical image to be recognized may be The images are sorted according to their scan image category, for example, T1-weighted inverse imaging, T1-weighted in-phase imaging, plain scan before angiography, early arterial, late arterial, portal phase, delay phase, T2-weighted imaging, diffusion-weighted imaging, The preset order of surface diffusion coefficient imaging is sorted. In addition, the preset order can also be set according to the doctor’s habits, which is not limited here, so as to improve the convenience of doctors in reading the film. In addition, in order to further improve the convenience of reading the film , The sorted at least one medical image to be recognized can also be displayed in a window corresponding to the number of medical images to be recognized. For example, if the number of medical images to be recognized is 5, it can be displayed in 5 display windows. Medical images to be recognized. Therefore, it is possible to reduce the time for doctors to look through multiple medical images to be identified for comparison back and forth, and to improve the efficiency of image reading.
在另一个实施场景中，至少一个待识别医学图像为对同一对象扫描得到的，故为了在扫描过程中进行质量控制，在得到每一待识别医学图像所属的扫描图像类别之后，还可以判断待识别医学图像的扫描图像类别是否存在重复，并在存在重复时，输出第一预警信息，以提示扫描人员。例如，若存在两张扫描图像类别均为“延迟期”的待识别医学图像，则可以认为扫描过程中存在扫描质量不合规的情况，故为了提示扫描人员，可以输出第一预警消息，具体地，可以输出预警原因(如，存在扫描图像类别重复的待识别医学图像等)。或者，在得到每一待识别医学图像所属的扫描图像类别之后，还可以判断至少一个待识别医学图像的扫描图像类别中不存在预设扫描图像类别，并在不存在预设扫描图像类别时，输出第二预警信息，以提示扫描人员。例如，预设扫描图像类别为“门脉期”，若至少一个待识别医学图像中不存在扫描图像类别为“门脉期”的图像，则可以认为扫描过程中存在扫描质量不合规的情况，故为了提示扫描人员，可以输出第二预警消息，具体地，可以输出预警原因(如，待识别医学图像中不存在门脉期图像等)。故此，能够在扫描过程中实现图像质控，以在与实际相悖时，能够及时纠错，避免病人二次挂号。In another implementation scenario, at least one medical image to be recognized is obtained by scanning the same object. Therefore, in order to perform quality control during the scanning process, after obtaining the scanned image category to which each medical image to be recognized belongs, it can also be determined Identify whether there is a repetition in the scanned image category of the medical image, and when there is a repetition, output first warning information to remind the scanner. For example, if there are two medical images to be recognized whose scan image categories are both "delay period", it can be considered that the scan quality is out of compliance during the scanning process. Therefore, in order to remind the scanner, the first warning message can be output. Therefore, it is possible to output the warning reason (for example, there are medical images to be recognized with repeated scan image categories, etc.). Alternatively, after obtaining the scanned image category to which each medical image to be recognized belongs, it can also be determined that the preset scanned image category does not exist in the scanned image category of at least one medical image to be recognized, and when the preset scanned image category does not exist, Output the second warning message to remind the scanner. For example, the preset scanned image category is "portal phase", and if there is no image with the scanned image category of "portal phase" in at least one medical image to be recognized, it can be considered that the scanning quality is out of compliance during the scanning process Therefore, in order to prompt the scanner, the second warning message can be output, specifically, the reason for the warning can be output (for example, there is no portal vein image in the medical image to be identified, etc.). Therefore, the image quality control can be realized during the scanning process, so that when it is contrary to the actual situation, the error can be corrected in time, and the second registration of the patient can be avoided.
上述方案，通过获取至少一个扫描得到的待识别医学图像，并分别确定每个待识别医学图像中与目标脏器对应的目标区域，从而分别对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示，能够排除其他脏器的干扰，有利于提高识别准确性，并将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示，进而每一待识别医学图像的个体特征表示和全局特征表示，不仅能够表示待识别医学图像自身的特征，还能够表示其他待识别医学图像的差异，从而在利用每一待识别医学图像的个体特征表示和全局特征表示，确定每一待识别医学图像所属的扫描图像类别时，能够提高图像识别的准确性，且通过特征表示来进行图像识别，能够免于人工参与，故能够提高图像识别的效率。In the above solution, by acquiring at least one scanned medical image to be recognized, and respectively determining the target area corresponding to the target organ in each medical image to be recognized, the feature extraction is performed on the image data of each target area to obtain each The individual feature representations of a medical image to be recognized can eliminate interference from other organs, which is beneficial to improve the accuracy of recognition, and the individual feature representations of at least one medical image to be recognized are merged to obtain a global feature representation, and then each to be recognized The individual feature representation and global feature representation of medical images can not only represent the characteristics of the medical image to be recognized, but also the differences of other medical images to be recognized, so that the individual feature representation and global feature representation of each medical image to be recognized can be used. When determining the scanned image category to which each medical image to be recognized belongs, the accuracy of image recognition can be improved, and image recognition can be performed through feature representation, which can avoid manual participation, so the efficiency of image recognition can be improved.
请参阅图3，图3是图1中步骤S11一实施例的流程示意图。具体地，图3是确定每个待识别医学图像中与目标脏器对应的目标区域一实施例的流程示意图，具体可以包括如下步骤：Please refer to FIG. 3, which is a schematic flowchart of an embodiment of step S11 in FIG. 1. Specifically, FIG. 3 is a schematic flowchart of an embodiment of determining the target region corresponding to the target organ in each medical image to be recognized, which may specifically include the following steps:
步骤S111：利用脏器检测网络分别对至少一个待识别医学图像进行检测，得到目标脏器第一位置信息和目标脏器的毗邻脏器的第二位置信息。Step S111: Use the organ detection network to detect at least one medical image to be identified, to obtain first position information of the target organ and second position information of the adjacent organs of the target organ.
在一个实施场景中，脏器检测网络的骨干网络可以采用efficient net，在其他实施场景中，脏器检测网络的骨干网络还可以采用其他网络，在此不做限定。目标脏器可以根据实际情况进行设定，例如，目标脏器可以是肝脏，目标脏器的毗邻脏器可以包括肾脏、脾脏中的至少一者。In one implementation scenario, the backbone network of the organ detection network can adopt an efficient net. In other implementation scenarios, the backbone network of the organ detection network can also adopt other networks, which is not limited here. The target organ may be set according to actual conditions. For example, the target organ may be the liver, and the adjacent organs of the target organ may include at least one of the kidney and the spleen.
在一个实施场景中，目标脏器的第一位置信息可以包括目标脏器对应区域的至少一组对角顶点位置(例如，位置坐标)和对应区域的尺寸(例如，长度、宽度等)，第二位置信息至少可以包括毗邻脏器对应区域的至少一个顶点位置(例如，位置坐标)。In an implementation scenario, the first position information of the target organ may include at least one set of diagonal vertex positions (for example, position coordinates) of the corresponding area of the target organ and the size (for example, length, width, etc.) of the corresponding area. The second position information may at least include at least one vertex position (for example, position coordinates) of the corresponding region of the adjacent organ.
在一个实施场景中，待识别医学图像可以是三维图像，为了更加准确地确定目标脏器对应的目标区域，可以将每一待识别医学图像沿冠状面进行划分，得到多个三维子图像，并将每一子图像沿垂直于冠状面的方向进行投影，得到对应的二维子图像，从而后续能够基于投影得到的多个二维子图像进行识别检测，具体地，可以利用脏器检测网络对至少一个待识别医学图像对应的二维子图像进行检测，得到第一位置信息和第二位置信息，从而能够在目标脏器大小异常或经手术干预后目标脏器的形态产生变化时，能够准确地确定其第一位置信息和目标脏器的毗邻脏器的第二位置信息。例如，目标脏器为肝脏时，当存在肝脏大小异常或经过手术干预后肝脏形态产生变化(如部分缺失)时，肝顶和肝尖的位置并不能稳定体现，故通过对冠状面上的多个二维子图像进行脏器检测，可以结合多个二维子图像上的检测结果，得到肝脏的第一位置信息和肾脏、脾脏等的第二位置信息，从而能够有效避免肝尖、肝顶位置不稳定而可能带来的检测误差。In an implementation scenario, the medical image to be recognized can be a three-dimensional image. In order to more accurately determine the target area corresponding to the target organ, each medical image to be recognized can be divided along the coronal plane to obtain multiple three-dimensional sub-images, and Project each sub-image in the direction perpendicular to the coronal plane to obtain the corresponding two-dimensional sub-image, so that subsequent identification and detection can be performed based on multiple two-dimensional sub-images obtained by the projection. Specifically, the organ detection network can be used to At least one two-dimensional sub-image corresponding to the medical image to be recognized is detected to obtain the first position information and the second position information, so that it can be accurately when the size of the target organ is abnormal or the shape of the target organ changes after surgical intervention The first location information of the target organ and the second location information of the adjacent organ of the target organ are determined. For example, when the target organ is the liver, when the liver size is abnormal or the liver morphology changes (such as partial loss) after surgical intervention, the positions of the liver apex and liver apex cannot be stably represented. Organs detection can be performed on two two-dimensional sub-images, and the detection results on multiple two-dimensional sub-images can be combined to obtain the first position information of the liver and the second position information of the kidney, spleen, etc., which can effectively avoid the apex and top of the liver. Possible detection error due to unstable position.
步骤S112：利用第一位置信息和第二位置信息，确定目标脏器对应的目标区域。Step S112: Use the first position information and the second position information to determine the target area corresponding to the target organ.
通过目标脏器的第一位置信息和其毗邻脏器的第二位置信息，能够考虑目标脏器和毗邻脏器在解剖结构上的地理相关性，故利用第一位置信息和第二位置信息，能够准确地确定目标脏器对应的目标区域。例如，以目标脏器是肝脏为例，第一位置信息可以包括肝脏对应区域的左上、左下顶点、对应区域的高度、宽度，第二位置信息可以包括脾脏、肾脏等毗邻脏器对应区域的右下顶点，故根据第一位置信息和第二位置信息在待识别医学图像上进行裁剪，可以得到肝脏对应的目标区域。其他场景可以以此类推，在此不再一一举例。Through the first location information of the target organ and the second location information of its adjacent organs, the geographic correlation between the target organ and the adjacent organs in the anatomical structure can be considered, so the first location information and the second location information are used, It can accurately determine the target area corresponding to the target organ. For example, taking the target organ as the liver as an example, the first position information may include the upper left and lower left vertices of the corresponding area of the liver, the height and width of the corresponding area, and the second position information may include the right side of the corresponding area of adjacent organs such as the spleen and kidney. The lower vertex, therefore, the target area corresponding to the liver can be obtained by cropping the medical image to be recognized according to the first position information and the second position information. Other scenes can be deduced by analogy, so I won't give examples one by one here.
区别于前述实施例，利用脏器检测网络分别对至少一个待识别医学图像进行检测，得到目标脏器的第一位置信息和目标脏器的毗邻脏器的第二位置信息，故不仅能够考虑所需识别的目标脏器，还能够考虑周边毗邻脏器，从而利用第一位置信息和第二位置信息，确定目标脏器对应的目标区域，能够确保在经手术治疗等情况下脏器形态发生改变时，也能够定位得到目标脏器对应的目标区域，故能够提高图像识别的鲁棒性。Different from the foregoing embodiment, the organ detection network is used to detect at least one medical image to be recognized, and the first position information of the target organ and the second position information of the adjacent organs of the target organ are obtained. The target organ that needs to be identified can also consider the surrounding adjacent organs, so that the first location information and the second location information can be used to determine the target area corresponding to the target organ, which can ensure that the morphology of the organ changes after surgical treatment, etc. At this time, the target area corresponding to the target organ can also be located, so the robustness of image recognition can be improved.
请参阅图4，图4是本申请图像识别装置40一实施例的框架示意图。图像识别装置40包括区域获取模块41、特征提取模块42、融合处理模块43和类别确定模块44，区域获取模块41配置为获取至少一个扫描得到的待识别医学图像，并分别确定每个待识别医学图像中与目标脏器对应的目标区域；特征提取模42配置为分别对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示；融合处理模块43配置为将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示；类别确定模块44配置为利用每一待识别医学图像的个体特征表示和全局特征表示，确定每一待识别医学图像所属的扫描图像类别。Please refer to FIG. 4, which is a schematic diagram of a framework of an embodiment of an image recognition device 40 of the present application. The image recognition device 40 includes a region acquisition module 41, a feature extraction module 42, a fusion processing module 43, and a category determination module 44. The region acquisition module 41 is configured to acquire at least one scanned medical image to be recognized, and to determine each medical image to be recognized. The target region in the image corresponding to the target organ; the feature extraction module 42 is configured to extract features from the image data of each target region to obtain the individual feature representation of each medical image to be recognized; the fusion processing module 43 is configured to at least The individual feature representations of a medical image to be recognized are fused to obtain a global feature representation; the category determination module 44 is configured to use the individual feature representation and the global feature representation of each medical image to be recognized to determine the scanned image to which each medical image to be recognized belongs category.
上述方案，通过获取至少一个扫描得到的待识别医学图像，并分别确定每个待识别医学图像中与目标脏器对应的目标区域，从而分别对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示，能够排除其他脏器的干扰，有利于提高识别准确性，并将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示，进而每一待识别医学图像的个体特征表示和全局特征表示，不仅能够表示待识别 医学图像自身的特征，还能够表示其他待识别医学图像的差异，从而在利用每一待识别医学图像的个体特征表示和全局特征表示，确定每一待识别医学图像所属的扫描图像类别时，能够提高图像识别的准确性，且通过特征表示来进行图像识别，能够免于人工参与，故能够提高图像识别的效率。In the above solution, by acquiring at least one scanned medical image to be recognized, and respectively determining the target area corresponding to the target organ in each medical image to be recognized, the feature extraction is performed on the image data of each target area to obtain each The individual feature representations of a medical image to be recognized can eliminate interference from other organs, which is beneficial to improve the accuracy of recognition, and the individual feature representations of at least one medical image to be recognized are merged to obtain a global feature representation, and then each to be recognized The individual feature representation and global feature representation of medical images can not only represent the characteristics of the medical image to be recognized, but also the differences of other medical images to be recognized, so that the individual feature representation and global feature representation of each medical image to be recognized can be used. When determining the scanned image category to which each medical image to be recognized belongs, the accuracy of image recognition can be improved, and image recognition can be performed through feature representation, which can avoid manual participation, so the efficiency of image recognition can be improved.
在一些实施例中，融合处理模块43配置为将至少一个待识别医学图像的个体特征表示进行全局池化处理，得到全局特征表示。In some embodiments, the fusion processing module 43 is configured to perform global pooling processing on the individual feature representation of at least one medical image to be recognized to obtain a global feature representation.
区别于前述实施例，通过将至少一个待识别医学图像的个体特征表示进行全局池化处理，能够快速方便地得到全局特征表示，故能够有利于提高后续图像识别的效率。Different from the foregoing embodiment, by performing global pooling processing on the individual feature representation of at least one medical image to be recognized, the global feature representation can be obtained quickly and conveniently, which can help improve the efficiency of subsequent image recognition.
在一些实施例中，融合处理模块43包括第一池化子模块，配置为将至少一个待识别医学图像的个体特征表示进行全局最大池化处理，得到第一全局特征表示，融合处理模块43包括第二池化子模块，配置为将至少一个待识别医学图像的个体特征表示进行全局平均池化处理，得到第二全局特征表示，融合处理模块43包括拼接处理子模块，配置为将第一全局特征表示和第二全局特征表示进行拼接处理，得到全局特征表示。In some embodiments, the fusion processing module 43 includes a first pooling sub-module configured to perform global maximum pooling processing on the individual feature representation of at least one medical image to be recognized to obtain the first global feature representation. The fusion processing module 43 includes The second pooling sub-module is configured to perform global average pooling processing on the individual feature representations of at least one medical image to be recognized to obtain a second global feature representation. The fusion processing module 43 includes a splicing processing sub-module configured to combine the first global The feature representation and the second global feature representation are spliced to obtain a global feature representation.
区别于前述实施例，通过将至少一个待识别医学图像的个体特征表示进行全局最大池化处理，得到第一全局特征表示，并将至少一个待识别医学图像的个体特征表示进行全局平均池化处理，得到第二全局特征表示，从而将第一全局特征表示和第二全局特征表示进行拼接处理，得到全局特征表示，故能够有利于后续准确地表示每一待识别医学图像与其他待识别医学图像之间的差异，从而能够有利于提高后续图像识别的准确性。Different from the foregoing embodiment, the first global feature representation is obtained by subjecting at least one individual feature representation of the medical image to be identified to global maximum pooling processing, and the individual feature representation of at least one medical image to be identified is subject to global average pooling processing , To obtain the second global feature representation, so that the first global feature representation and the second global feature representation are stitched together to obtain the global feature representation, so it can help to accurately represent each medical image to be recognized and other medical images to be recognized. The difference between them can help improve the accuracy of subsequent image recognition.
在一些实施例中，类别确定模块44包括特征处理子模块和类别确定子模块，特征处理子模块配置为利用每一待识别医学图像的个体特征表示和全局特征表示，得到每一待识别医学图像的最终特征表示，类别确定子模块配置为利用每个待识别医学图像的最终特征表示，确定每一待识别医学图像所属的扫描图像类别。In some embodiments, the category determination module 44 includes a feature processing sub-module and a category determination sub-module. The feature processing sub-module is configured to use the individual feature representation and the global feature representation of each medical image to be recognized to obtain each medical image to be recognized. The final feature representation of the category determination sub-module is configured to use the final feature representation of each medical image to be recognized to determine the scanned image category to which each medical image to be recognized belongs.
区别于前述实施例，利用每一待识别医学图像的个体特征表示和全局特征表示所得到的最终特征表示，不仅能够表示待识别医学图像自身的特征，还能够表示其他待识别医学图像的差异，从而在利用每个待识别医学图像的最终特征表示，确定每一待识别医学图像所属的扫描图像类别时，能够提高图像识别的准确性。Different from the foregoing embodiment, the final feature representation obtained by using the individual feature representation and the global feature representation of each medical image to be recognized can not only represent the characteristics of the medical image to be recognized, but also the differences of other medical images to be recognized. Therefore, when the final feature representation of each medical image to be recognized is used to determine the scanned image category to which each medical image to be recognized belongs, the accuracy of image recognition can be improved.
在一些实施例中，特征处理子模块配置为分别将每一待识别医学图像的个体特征表示和全局特征表示进行拼接处理，得到待识别医学图像对应的最终特征表示。In some embodiments, the feature processing sub-module is configured to respectively perform stitching processing on the individual feature representation and the global feature representation of each medical image to be recognized to obtain the final feature representation corresponding to the medical image to be recognized.
区别于前述实施例，通过分别将每一待识别医学图像的个体特征表示和全局特征表示进行拼接处理，能够快速得到待识别医学图像对应的最终特征表示，故能够有利于提高后续图像识别的效率。Different from the foregoing embodiment, by separately stitching the individual feature representation and global feature representation of each medical image to be recognized, the final feature representation corresponding to the medical image to be recognized can be quickly obtained, which can help improve the efficiency of subsequent image recognition. .
在一些实施例中，特征提取模块42配置为利用识别网络的特征提取子网络对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示，融合处理模块43配置为利用识别网络的融合模块将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示，特征处理子模块配置为采用识别网络的融合模块利用每一待识别医学图像的个体特征表示和全局特征表示，得到每一待识别医学图像的最终特征表示，类别确定子模块配置为利用识别网络的分类子网络对每一待识别医学图像的最终特征表示进行预测分类，得到每一待识别医学图像所属的扫描图像类别。In some embodiments, the feature extraction module 42 is configured to use the feature extraction sub-network of the recognition network to perform feature extraction on the image data of each target area to obtain the individual feature representation of each medical image to be recognized, and the fusion processing module 43 is configured to The fusion module of the recognition network is used to fuse the individual feature representations of at least one medical image to be recognized to obtain a global feature representation. The feature processing sub-module is configured to use the fusion module of the recognition network to use the individual feature representation and the global feature of each medical image to be recognized Feature representation, the final feature representation of each medical image to be recognized is obtained, and the category determination sub-module is configured to use the classification sub-network of the recognition network to predict and classify the final feature representation of each medical image to be recognized to obtain each medical image to be recognized The category of the scanned image to which it belongs.
区别于前述实施例，通过利用识别网络的特征提取子网络对每一目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示，并利用识别网络的融合模块将至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示，利用每一待识别医学图像的个体特征表示和全局特征表示，得到每一待识别医学图像的最终特征表示，从而利用识别网络的分类子网络对每一待识别医学图像的最终特征表示进行预测分类，得到每一待识别医学图像所属的扫描图像类别，故能够通过识别网络最终获得待 识别医学图像所属的扫描图像类别，从而能够进一步提高图像识别的效率。Different from the foregoing embodiment, the feature extraction sub-network of the recognition network is used to perform feature extraction on the image data of each target area to obtain the individual feature representation of each medical image to be recognized, and the fusion module of the recognition network is used to combine at least one The individual feature representations of the recognized medical images are fused to obtain a global feature representation, and the individual feature representations and global feature representations of each medical image to be recognized are used to obtain the final feature representation of each medical image to be recognized, thereby using the classifier of the recognition network The network predicts and classifies the final feature representation of each medical image to be recognized, and obtains the scanned image category to which each medical image to be recognized belongs. Therefore, the scanned image category to which the medical image to be recognized belongs can be finally obtained through the recognition network, which can further improve The efficiency of image recognition.
在一些实施例中，识别网络是利用样本医学图像训练得到的，每次训练识别网络所使用的样本医学图像数量不固定。In some embodiments, the recognition network is obtained by training with sample medical images, and the number of sample medical images used for each training of the recognition network is not fixed.
区别于前述实施例，每次训练识别网络采用的样本医学图像的数量并不固定，能够随机化样本医学图像的数量，从而能够有利于在不同机构不同扫描协议下扫描图像类别有所缺失时，也能够准确地进行图像识别，进而能够提高图像识别准确性。Different from the foregoing embodiment, the number of sample medical images used for each training recognition network is not fixed, and the number of sample medical images can be randomized, which can help when the types of scanned images are missing under different institutions and different scanning protocols. The image recognition can also be performed accurately, and the accuracy of the image recognition can be improved.
在一些实施例中，特征提取子网络包括至少一组顺序连接的稠密卷积块和池化层；和/或，识别网络包括预设数量个特征提取子网络，特征提取模块42配置为将每一目标区域的图像数据分别输入对应一个特征提取子网络进行特征提取，得到每个待识别医学图像的个体特征表示。In some embodiments, the feature extraction sub-network includes at least one set of dense convolutional blocks and pooling layers connected in sequence; and/or, the recognition network includes a preset number of feature extraction sub-networks, and the feature extraction module 42 is configured to The image data of a target area are respectively input into a corresponding feature extraction sub-network for feature extraction, and the individual feature representation of each medical image to be recognized is obtained.
区别于前述实施例，特征提取子网络包括至少一组顺序连接的稠密卷积块和池化层，故通过稠密卷积块的连接策略，即每一卷积层下的特征与下一层紧密拼接，并传递后后面的每一层，能够有效的缓解梯度消失问题，且加强特征传播以及特征复用，并能够极大地减少参数数量；而将识别网络设置为包括预设数量个特征提取子网络，并将每一目标区域的图像数据分别输入对应一个特征提取子网络进行特征提取，得到每个待识别医学图像的个体特征表示，能够将至少一个目标区域的图像数据的特征提取操作并行处理，故能够有利于提高图像识别的效率。Different from the foregoing embodiment, the feature extraction sub-network includes at least one set of dense convolutional blocks and pooling layers that are connected in sequence. Therefore, the dense convolutional block connection strategy is adopted, that is, the features under each convolutional layer are close to the next layer. After splicing and transferring each layer, it can effectively alleviate the problem of gradient disappearance, strengthen feature propagation and feature reuse, and can greatly reduce the number of parameters; and the recognition network is set to include a preset number of feature extractors Network, and input the image data of each target area into a corresponding feature extraction sub-network for feature extraction, and obtain the individual feature representation of each medical image to be recognized. The feature extraction operation of the image data of at least one target area can be processed in parallel , It can help improve the efficiency of image recognition.
在一些实施例中，区域获取模块41包括脏器检测子模块，配置为利用脏器检测网络分别对至少一个待识别医学图像进行检测，得到目标脏器第一位置信息和目标脏器的毗邻脏器的第二位置信息，区域获取模块41包括区域确定子模块，配置为利用第一位置信息和第二位置信息，确定目标脏器对应的目标区域。In some embodiments, the area acquisition module 41 includes an organ detection sub-module configured to detect at least one medical image to be identified using an organ detection network to obtain first position information of the target organ and adjacent organs of the target organ. For the second location information of the organ, the area acquisition module 41 includes an area determination sub-module configured to use the first location information and the second location information to determine the target area corresponding to the target organ.
区别于前述实施例，利用脏器检测网络分别对至少一个待识别医学图像进行检测，得到目标脏器的第一位置信息和目标脏器的毗邻脏器的第二位置信息，故不仅能够考虑所需识别的目标脏器，还能够考虑周边毗邻脏器，从而利用第一位置信息和第二位置信息，确定目标脏器对应的目标区域，能够确保在经手术治疗等情况下脏器形态发生改变时，也能够定位得到目标脏器对应的目标区域，故能够提高图像识别的鲁棒性。Different from the foregoing embodiment, the organ detection network is used to detect at least one medical image to be recognized, and the first position information of the target organ and the second position information of the adjacent organs of the target organ are obtained. The target organ that needs to be identified can also consider the surrounding adjacent organs, so that the first location information and the second location information can be used to determine the target area corresponding to the target organ, which can ensure that the morphology of the organ changes after surgical treatment, etc. At this time, the target area corresponding to the target organ can also be located, so the robustness of image recognition can be improved.
在一些实施例中，待识别医学图像为三维图像，区域获取模块41还包括图像划分子模块，配置为将每一待识别医学图像沿冠状面进行划分，得到多个三维子图像，区域获取模块41还包括图像投影子模块，配置为将每一子图像沿垂直于冠状面的方向进行投影，得到对应的二维子图像，脏器检测子模块配置为利用脏器检测网络对至少一个待识别医学图像对应的二维子图像进行检测，得到第一位置信息和第二位置信息。In some embodiments, the medical image to be recognized is a three-dimensional image, and the region acquisition module 41 further includes an image division sub-module configured to divide each medical image to be recognized along the coronal plane to obtain multiple three-dimensional sub-images. The region acquisition module 41 also includes an image projection sub-module, configured to project each sub-image in a direction perpendicular to the coronal plane to obtain a corresponding two-dimensional sub-image, and the organ detection sub-module is configured to use the organ detection network to identify at least one The two-dimensional sub-image corresponding to the medical image is detected to obtain the first position information and the second position information.
区别于前述实施例，在待识别医学图像为三维图像时，将每一待识别医学图像沿冠状面进行划分，得到多个三维子图像，并将每一子图像沿垂直于冠状面的方向进行投影，得到对应的二维子图像，从而利用脏器检测网络对至少一个待识别医学图像对应的二维子图像进行检测，得到第一位置信息和第二位置信息，能够进一步提高目标脏器对应的目标区域定位的准确性。Different from the foregoing embodiment, when the medical image to be recognized is a three-dimensional image, each medical image to be recognized is divided along the coronal plane to obtain multiple three-dimensional sub-images, and each sub-image is performed in a direction perpendicular to the coronal plane. Projection to obtain the corresponding two-dimensional sub-image, so that the organ detection network is used to detect the two-dimensional sub-image corresponding to at least one medical image to be recognized, and the first position information and the second position information are obtained, which can further improve the correspondence of the target organ The accuracy of the target area positioning.
在一些实施例中，目标脏器为肝脏，毗邻脏器包括肾脏、脾脏中的至少一者；和/或，第一位置信息包括目标脏器对应区域的至少一组对角顶点位置和对应区域的尺寸，第二位置信息至少包括毗邻脏器对应区域的至少一个顶点位置。In some embodiments, the target organ is the liver, and the adjacent organs include at least one of the kidney and the spleen; and/or, the first position information includes at least one set of diagonal vertex positions and corresponding areas of the corresponding area of the target organ The second position information includes at least one vertex position of the corresponding area adjacent to the organ.
区别于前述实施例，将目标脏器设置为肝脏，毗邻脏器设置为包括肾脏、脾脏中的至少一者，能够有利于定位得到肝脏对应的目标区域；将第一位置信息设置为包括目标脏器对应区域的至少一组对角顶点位置和对应区域的尺寸，第二位置信息设置为至少包括毗邻脏器对应区域的至少一个顶点位置，能够有利于精确地定位目标脏器对应的目标区域。Different from the foregoing embodiment, setting the target organ as the liver, and setting the adjacent organ to include at least one of the kidney and the spleen can help locate the target area corresponding to the liver; and set the first position information to include the target organ. At least one set of diagonal vertex positions of the organ corresponding area and the size of the corresponding area, and the second position information is set to include at least one vertex position adjacent to the organ corresponding area, which can facilitate accurate positioning of the target area corresponding to the target organ.
在一些实施例中，图像识别装置40还包括图像排序模块，配置为将至少一个待识别医学图像按照其扫描图像类别进行排序；图像识别装置40还包括第一输出模块，配置为在待识别医学图像的扫描图像类别存在重复时，输出第一预警信息，以提示扫描人员；图像识别装置40还包括第二输出模块，配置为在至少一个待识别医学图像的扫描图像类别中不存在预设扫描图像类别时，输出第二预警信息，以提示扫描人员。In some embodiments, the image recognition device 40 further includes an image sorting module configured to sort the at least one medical image to be recognized according to its scanned image category; the image recognition device 40 further includes a first output module configured to When the scanned image category of the image is repeated, the first warning information is output to remind the scanner; the image recognition device 40 further includes a second output module configured to have no preset scan in the scanned image category of the at least one medical image to be recognized In the image category, the second warning message is output to remind the scanner.
区别于前述实施例，在确定得到每一待识别医学图像所属的扫描图像类别之后，执行将至少一个待识别医学图像按照其扫描图像类别进行排序，能够提高医生阅片的便捷性；在待识别医学图像的扫描图像类别存在重复时，输出第一预警信息，以提示扫描人员，在至少一个待识别医学图像的扫描图像类别中不存在预设扫描图像类别时，输出第二预警信息，以提示扫描人员，能够在扫描过程中实现图像质控，以在与实际相悖时，能够及时纠错，避免病人二次挂号。Different from the foregoing embodiment, after the scan image category to which each medical image to be recognized belongs is determined, it is executed to sort at least one medical image to be recognized according to its scan image category, which can improve the convenience of doctor reading; When the scanned image category of the medical image is duplicated, the first warning information is output to remind the scanner, and when the preset scanned image category does not exist in the scanned image category of the at least one medical image to be recognized, the second warning information is output to remind Scanners can achieve image quality control during the scanning process, so that when it is contrary to reality, they can correct errors in time to avoid the second registration of patients.
在一些实施例中，图像识别装置40还包括预处理模块，配置为对每一目标区域的图像数据进行预处理，其中，预处理包括以下至少一种：将目标区域的图像尺寸调整至预设尺寸，将目标区域的图像强度归一化至预设范围。In some embodiments, the image recognition device 40 further includes a preprocessing module configured to preprocess the image data of each target area, wherein the preprocessing includes at least one of the following: adjusting the image size of the target area to a preset Size, normalize the image intensity of the target area to the preset range.
区别于前述实施例，在特征提取之前，对每一目标区域的图像数据进行预处理，且预处理包括以下至少一种：将目标区域的图像尺寸调整至预设尺寸，将目标区域的图像强度归一化至预设范围，故能够有利于提高后续图像识别的准确性。Different from the foregoing embodiment, before feature extraction, the image data of each target area is preprocessed, and the preprocessing includes at least one of the following: adjusting the image size of the target area to a preset size, and adjusting the image intensity of the target area It is normalized to the preset range, so it can help improve the accuracy of subsequent image recognition.
请参阅图5，图5是本申请电子设备50一实施例的框架示意图。电子设备50包括相互耦接的存储器51和处理器52，处理器52配置为执行存储器51中存储的程序指令，以实现上述任一图像识别方法实施例的步骤。在一个具体的实施场景中，电子设备50可以包括但不限于：微型计算机、服务器，此外，电子设备50还可以包括笔记本电脑、平板电脑等移动设备，在此不做限定。Please refer to FIG. 5, which is a schematic diagram of a framework of an embodiment of an electronic device 50 of the present application. The electronic device 50 includes a memory 51 and a processor 52 coupled to each other, and the processor 52 is configured to execute program instructions stored in the memory 51 to implement the steps of any of the above-mentioned image recognition method embodiments. In a specific implementation scenario, the electronic device 50 may include but is not limited to a microcomputer and a server. In addition, the electronic device 50 may also include mobile devices such as a notebook computer and a tablet computer, which are not limited herein.
具体而言，处理器52配置为控制其自身以及存储器51以实现上述任一图像识别方法实施例的步骤。处理器52还可以称为CPU(Central Processing Unit，中央处理单元)。处理器52可能是一种集成电路芯片，具有信号的处理能力。处理器52还可以是通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现场可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。另外，处理器52可以由集成电路芯片共同实现。Specifically, the processor 52 is configured to control itself and the memory 51 to implement the steps of any of the above-mentioned image recognition method embodiments. The processor 52 may also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA), or other Programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like. In addition, the processor 52 may be jointly implemented by an integrated circuit chip.
上述方案，能够提高图像识别的效率和准确性。The above solution can improve the efficiency and accuracy of image recognition.
请参阅图6，图6为本申请计算机可读存储介质60一实施例的框架示意图。计算机可读存储介质60存储有能够被处理器运行的程序指令601，程序指令601用于实现上述任一图像识别方法实施例的步骤。Please refer to FIG. 6, which is a schematic diagram of a framework of an embodiment of the computer-readable storage medium 60 of the present application. The computer-readable storage medium 60 stores program instructions 601 that can be executed by the processor, and the program instructions 601 are used to implement the steps of any of the foregoing image recognition method embodiments.
上述方案，能够提高图像识别的效率和准确性。The above solution can improve the efficiency and accuracy of image recognition.
在本申请所提供的几个实施例中，应该理解到，所揭露的方法和装置，可以通过其它的方式实现。例如，以上所描述的装置实施方式仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性、机械或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed method and device can be implemented in other ways. For example, the device implementation described above is only illustrative, for example, the division of modules or units is only a logical function division, and there may be other divisions in actual implementation, for example, units or components can be combined or integrated. To another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施方式方案的目的。The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of this embodiment.
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本申请各个实施方式方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes. .

Claims (16)

  1. 一种图像识别方法，包括：An image recognition method, including:
    获取至少一个扫描得到的待识别医学图像，并分别确定每个所述待识别医学图像中与目标脏器对应的目标区域；Acquiring at least one scanned medical image to be recognized, and respectively determining a target area corresponding to a target organ in each of the medical image to be recognized;
    分别对每一所述目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示；Perform feature extraction on the image data of each target area respectively to obtain the individual feature representation of each medical image to be recognized;
    将所述至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示；Fusing the individual feature representations of the at least one medical image to be identified to obtain a global feature representation;
    利用每一所述待识别医学图像的个体特征表示和所述全局特征表示，确定每一所述待识别医学图像所属的扫描图像类别。Using the individual feature representation and the global feature representation of each medical image to be recognized, the scanned image category to which each medical image to be recognized belongs is determined.
  2. 根据权利要求1所述的图像识别方法，其中，所述将所述至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示包括：The image recognition method according to claim 1, wherein said fusing the individual characteristic representations of the at least one medical image to be recognized to obtain a global characteristic representation comprises:
    将所述至少一个待识别医学图像的个体特征表示进行全局池化处理，得到所述全局特征表示。Perform global pooling processing on the individual feature representation of the at least one medical image to be identified to obtain the global feature representation.
  3. 根据权利要求2所述的图像识别方法，其中，所述将所述至少一个待识别医学图像的个体特征表示进行全局池化处理，得到所述全局特征表示包括：The image recognition method according to claim 2, wherein said performing global pooling processing on the individual feature representation of the at least one medical image to be recognized to obtain the global feature representation comprises:
    将所述至少一个待识别医学图像的个体特征表示进行全局最大池化处理，得到第一全局特征表示；以及，Performing global maximum pooling processing on the individual feature representation of the at least one medical image to be recognized to obtain the first global feature representation; and,
    将所述至少一个待识别医学图像的个体特征表示进行全局平均池化处理，得到第二全局特征表示；Performing global average pooling processing on the individual feature representation of the at least one medical image to be identified to obtain a second global feature representation;
    将所述第一全局特征表示和所述第二全局特征表示进行拼接处理，得到所述全局特征表示。The first global feature representation and the second global feature representation are spliced to obtain the global feature representation.
  4. 根据权利要求1所述的图像识别方法，其中，所述利用每一所述待识别医学图像的个体特征表示和所述全局特征表示，确定所述待识别医学图像所属的扫描图像类别包括：4. The image recognition method according to claim 1, wherein said using the individual feature representation and the global feature representation of each medical image to be recognized to determine the scanned image category to which the medical image to be recognized belongs comprises:
    利用每一所述待识别医学图像的个体特征表示和所述全局特征表示，得到每一所述待识别医学图像的最终特征表示；Using the individual feature representation and the global feature representation of each medical image to be recognized to obtain the final feature representation of each medical image to be recognized;
    利用每个所述待识别医学特征的最终特征表示，确定每一所述待识别医学图像所属的扫描图像类别。Using the final feature representation of each medical feature to be recognized, the scanned image category to which each medical image to be recognized belongs is determined.
  5. 根据权利要求4所述的图像识别方法，其中，所述利用每一所述待识别医学图像的个体特征表示和所述全局特征表示，得到每一所述待识别医学图像的最终特征表示包括：4. The image recognition method according to claim 4, wherein said using the individual feature representation and the global feature representation of each medical image to be recognized to obtain the final feature representation of each medical image to be recognized comprises:
    分别将每一所述待识别医学图像的个体特征表示和所述全局特征表示进行拼接处理，得到所述待识别医学图像对应的最终特征表示。The individual feature representation and the global feature representation of each medical image to be recognized are respectively spliced to obtain the final feature representation corresponding to the medical image to be recognized.
  6. 根据权利要求4所述的图像识别方法，其中，所述分别对每一所述目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示包括：4. The image recognition method according to claim 4, wherein the feature extraction of the image data of each of the target regions to obtain the individual feature representation of each medical image to be recognized comprises:
    利用识别网络的特征提取子网络对每一所述目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示；Use the feature extraction sub-network of the recognition network to perform feature extraction on the image data of each of the target regions to obtain the individual feature representation of each medical image to be recognized;
    所述将所述至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示，利用每一所述待识别医学图像的个体特征表示和所述全局特征表示，得到每一所述待识别医学图像的最终特征表示包括：Said fusing the individual feature representations of the at least one medical image to be identified to obtain a global feature representation, and using the individual feature representation of each medical image to be identified and the global feature representation to obtain each of the medical images to be identified The final feature representation of medical images includes:
    利用所述识别网络的融合模块将所述至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示，并利用每一所述待识别医学图像的个体特征表示和所述全局特征表示，得到每一所述待识别医学图像的最终特征表示；Use the fusion module of the recognition network to fuse the individual feature representations of the at least one medical image to be recognized to obtain a global feature representation, and use the individual feature representation of each medical image to be recognized and the global feature representation, Obtaining the final feature representation of each medical image to be recognized;
    所述利用每个所述待识别医学特征的最终特征表示，确定每一所述待识别医学图像所属的扫描图像类别，包括：The using the final feature representation of each medical feature to be recognized to determine the scanned image category to which each medical image to be recognized belongs includes:
    利用所述识别网络的分类子网络对每一所述待识别医学图像的最终特征表示进行预测分类，得到每一所述待识别医学图像所属的扫描图像类别。The classification sub-network of the recognition network is used to predict and classify the final feature representation of each medical image to be recognized to obtain the scanned image category to which each medical image to be recognized belongs.
  7. 根据权利要求6所述的图像识别方法，其中，所述识别网络是利用样本医学图像训练得到的，每次训练所述识别网络所使用的所述样本医学图像数量不固定。7. The image recognition method according to claim 6, wherein the recognition network is obtained by training with sample medical images, and the number of sample medical images used for each training of the recognition network is not fixed.
  8. 根据权利要求6或7所述的图像识别方法，其中，所述特征提取子网络包括至少一组顺序连接的稠密卷积块和池化层；The image recognition method according to claim 6 or 7, wherein the feature extraction sub-network includes at least one set of dense convolution blocks and pooling layers connected in sequence;
    和/或，所述识别网络包括预设数量个特征提取子网络；所述利用识别网络的特征提取子网络对每一所述目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示包括：And/or, the recognition network includes a preset number of feature extraction sub-networks; the feature extraction sub-networks of the recognition network are used to perform feature extraction on the image data of each target area to obtain the information of each medical image to be recognized Individual characteristics include:
    将每一所述目标区域的图像数据分别输入对应一个所述特征提取子网络进行特征提取，得到每个待识别医学图像的个体特征表示。The image data of each target area is input into the corresponding feature extraction sub-network to perform feature extraction, and the individual feature representation of each medical image to be recognized is obtained.
  9. 根据权利要求1至8所述的图像识别方法，其中，所述分别确定每个所述待识别医学图像中与目标脏器对应的目标区域包括：The image recognition method according to claims 1 to 8, wherein said separately determining the target area corresponding to the target organ in each medical image to be recognized comprises:
    利用脏器检测网络分别对所述至少一个待识别医学图像进行检测，得到所述目标脏器第一位置信息和所述目标脏器的毗邻脏器的第二位置信息；Use an organ detection network to detect the at least one medical image to be identified, to obtain the first position information of the target organ and the second position information of the adjacent organs of the target organ;
    利用所述第一位置信息和所述第二位置信息，确定所述目标脏器对应的目标区域。Using the first position information and the second position information, a target area corresponding to the target organ is determined.
  10. 根据权利要求9所述的图像识别方法，其中，所述待识别医学图像为三维图像，所述利用脏器检测网络分别对所述至少一个待识别医学图像进行检测，得到所述目标脏器第一位置信息和所述目标脏器的毗邻脏器的第二位置信息之前，所述方法还包括：The image recognition method according to claim 9, wherein the medical image to be recognized is a three-dimensional image, and the organ detection network is used to detect the at least one medical image to be recognized to obtain the target organ first Before the first position information and the second position information of the adjacent organ of the target organ, the method further includes:
    将每一所述待识别医学图像沿冠状面进行划分，得到多个三维子图像；Dividing each medical image to be recognized along the coronal plane to obtain a plurality of three-dimensional sub-images;
    将每一所述子图像沿垂直于冠状面的方向进行投影，得到对应的二维子图像；Project each of the sub-images in a direction perpendicular to the coronal plane to obtain a corresponding two-dimensional sub-image;
    所述利用脏器检测网络分别对所述至少一个待识别医学图像进行检测，得到所述目标脏器第一位置信息和所述目标脏器的毗邻脏器的第二位置信息包括：The use of the organ detection network to detect the at least one medical image to be identified, and obtain the first position information of the target organ and the second position information of the adjacent organs of the target organ includes:
    利用所述脏器检测网络对所述至少一个待识别医学图像对应的所述二维子图像进行检测，得到所述第一位置信息和所述第二位置信息。The organ detection network is used to detect the two-dimensional sub-image corresponding to the at least one medical image to be identified to obtain the first position information and the second position information.
  11. 根据权利要求9或10所述的图像识别方法，其中，所述目标脏器为肝脏，所述毗邻脏器包括肾脏、脾脏中的至少一者；The image recognition method according to claim 9 or 10, wherein the target organ is a liver, and the adjacent organ includes at least one of a kidney and a spleen;
    和/或，所述第一位置信息包括所述目标脏器对应区域的至少一组对角顶点位置和对应区域的尺寸，所述第二位置信息至少包括所述毗邻脏器对应区域的至少一个顶点位置。And/or, the first position information includes at least one set of diagonal vertex positions of the corresponding area of the target organ and the size of the corresponding area, and the second position information includes at least one of the corresponding areas of the adjacent organ Vertex position.
  12. 根据权利要求1至11任一项所述的图像识别方法，其中，所述利用每一所述待识别医学图像的个体特征表示和所述全局特征表示，确定每一所述待识别医学图像所属的扫描图像类别之后，所述方法还包括以下至少一者：The image recognition method according to any one of claims 1 to 11, wherein the individual feature representation and the global feature representation of each medical image to be recognized are used to determine that each medical image to be recognized belongs to After the scanned image category of, the method further includes at least one of the following:
    将所述至少一个待识别医学图像按照其扫描图像类别进行排序；Sort the at least one medical image to be recognized according to its scanned image category;
    若所述待识别医学图像的扫描图像类别存在重复，则输出第一预警信息，以提示扫描人员；If the scanned image category of the medical image to be recognized is duplicated, output first warning information to remind the scanner;
    若所述至少一个待识别医学图像的扫描图像类别中不存在预设扫描图像类别，则输出第二预警信息，以提示扫描人员。If the preset scan image category does not exist in the scan image category of the at least one medical image to be recognized, the second warning information is output to remind the scanner.
  13. 根据权利要求1至12任一项所述的图像识别方法，其中，所述分别对每一所述目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示之前，所述方法还包括：The image recognition method according to any one of claims 1 to 12, wherein, before the feature extraction is performed on the image data of each of the target regions to obtain the individual feature representation of each medical image to be recognized, the Methods also include:
    对每一所述目标区域的图像数据进行预处理，其中，所述预处理包括以下至少一种：将所述目标区域的图像尺寸调整至预设尺寸，将所述目标区域的图像强度归一化至预设 范围。Preprocessing the image data of each target area, where the preprocessing includes at least one of the following: adjusting the image size of the target area to a preset size, and normalizing the image intensity of the target area To the preset range.
  14. 一种图像识别装置，包括：An image recognition device, including:
    区域获取模块，配置为获取至少一个扫描得到的待识别医学图像，并分别确定每个所述待识别医学图像中与目标脏器对应的目标区域；An area acquisition module, configured to acquire at least one scanned medical image to be recognized, and to respectively determine a target area corresponding to a target organ in each of the to-be-recognized medical images;
    特征提取模块，配置为分别对每一所述目标区域的图像数据进行特征提取，得到每个待识别医学图像的个体特征表示；The feature extraction module is configured to perform feature extraction on the image data of each target area to obtain the individual feature representation of each medical image to be recognized;
    融合处理模块，配置为将所述至少一个待识别医学图像的个体特征表示进行融合，得到全局特征表示；A fusion processing module configured to fuse the individual feature representations of the at least one medical image to be identified to obtain a global feature representation;
    类别确定模块，配置为利用每一所述待识别医学图像的个体特征表示和所述全局特征表示，确定每一所述待识别医学图像所属的扫描图像类别。The category determination module is configured to use the individual feature representation and the global feature representation of each medical image to be recognized to determine the scanned image category to which each medical image to be recognized belongs.
  15. 一种电子设备，包括相互耦接的存储器和处理器，所述处理器配置为执行所述存储器中存储的程序指令，以实现权利要求1至13任一项所述的图像识别方法。An electronic device comprising a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement the image recognition method according to any one of claims 1 to 13.
  16. 一种计算机可读存储介质，其上存储有程序指令，所述程序指令被处理器执行时实现权利要求1至13任一项所述的图像识别方法。A computer-readable storage medium having program instructions stored thereon, and when the program instructions are executed by a processor, the image recognition method according to any one of claims 1 to 13 is realized.
PCT/CN2021/078748 2020-03-31 2021-03-02 Image recognition method and related apparatus, and device WO2021196955A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020227004540A KR20220031695A (en) 2020-03-31 2021-03-02 Image recognition method and related devices and devices
JP2021577453A JP2022539162A (en) 2020-03-31 2021-03-02 Image identification method and related device and equipment

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010246970.5 2020-03-31
CN202010246970.5A CN111507381B (en) 2020-03-31 2020-03-31 Image recognition method, related device and equipment

Publications (1)

Publication Number Publication Date
WO2021196955A1 true WO2021196955A1 (en) 2021-10-07

Family

ID=71874107

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/078748 WO2021196955A1 (en) 2020-03-31 2021-03-02 Image recognition method and related apparatus, and device

Country Status (5)

Country Link
JP (1) JP2022539162A (en)
KR (1) KR20220031695A (en)
CN (1) CN111507381B (en)
TW (1) TWI776426B (en)
WO (1) WO2021196955A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115082772A (en) * 2022-07-19 2022-09-20 小米汽车科技有限公司 Location identification method, location identification device, vehicle, storage medium and chip

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111507381B (en) * 2020-03-31 2024-04-02 上海商汤智能科技有限公司 Image recognition method, related device and equipment
CN112464701B (en) * 2020-08-26 2023-06-30 北京交通大学 Method for detecting whether person wears mask or not based on lightweight feature fusion SSD
CN112329844A (en) * 2020-11-03 2021-02-05 上海商汤智能科技有限公司 Image object classification method and related device, equipment and storage medium
CN112633086B (en) * 2020-12-09 2024-01-26 西安电子科技大学 Near-infrared pedestrian monitoring method, system, medium and equipment based on multitasking EfficientDet
CN112819811A (en) * 2021-02-24 2021-05-18 上海商汤智能科技有限公司 Image analysis method and related device, electronic equipment and storage medium
CN113065017A (en) * 2021-03-30 2021-07-02 上海商汤智能科技有限公司 Medical image display method and related device, equipment and storage medium
CN113362314B (en) * 2021-06-18 2022-10-18 北京百度网讯科技有限公司 Medical image recognition method, recognition model training method and device
CN113361637A (en) * 2021-06-30 2021-09-07 杭州东方通信软件技术有限公司 Potential safety hazard identification method and device for base station room
CN113516758A (en) * 2021-07-07 2021-10-19 上海商汤智能科技有限公司 Image display method and related device, electronic equipment and storage medium
CN114049937A (en) * 2021-11-22 2022-02-15 上海商汤智能科技有限公司 Image evaluation method and related device, electronic equipment and storage medium
CN114154512A (en) * 2021-12-09 2022-03-08 京东科技信息技术有限公司 Small sample learning processing method, device, equipment and storage medium
CN114663715B (en) * 2022-05-26 2022-08-26 浙江太美医疗科技股份有限公司 Medical image quality control and classification model training method and device and computer equipment
CN115294110B (en) * 2022-09-30 2023-01-06 杭州太美星程医药科技有限公司 Scanning period identification method and device, electronic equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9305349B2 (en) * 2013-06-28 2016-04-05 Samsung Electronics Co., Ltd. Apparatus and method for detecting lesion
CN110321920A (en) * 2019-05-08 2019-10-11 腾讯科技（深圳）有限公司 Image classification method, device, computer readable storage medium and computer equipment
CN110675940A (en) * 2019-08-01 2020-01-10 平安科技（深圳）有限公司 Pathological image labeling method and device, computer equipment and storage medium
CN110752028A (en) * 2019-10-21 2020-02-04 腾讯科技（深圳）有限公司 Image processing method, device, equipment and storage medium
CN110866897A (en) * 2019-10-30 2020-03-06 上海联影智能医疗科技有限公司 Image detection method and computer readable storage medium
CN111507381A (en) * 2020-03-31 2020-08-07 上海商汤智能科技有限公司 Image recognition method and related device and equipment

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6027546B2 (en) * 2011-12-21 2016-11-16 株式会社日立製作所 MEDICAL IMAGE DIAGNOSIS DEVICE AND PHASE DETERMINING METHOD USING MEDICAL IMAGE DIAGNOSIS DEVICE
US11238975B2 (en) * 2014-04-02 2022-02-01 University Of Louisville Research Foundation, Inc. Computer aided diagnosis system for classifying kidneys
US10332251B2 (en) * 2015-06-12 2019-06-25 Merge Healthcare Incorporated Methods and systems for automatically mapping biopsy locations to pathology results
CN106204587B (en) * 2016-05-27 2019-01-08 浙江德尚韵兴图像科技有限公司 Multiple organ dividing method based on depth convolutional neural networks and region-competitive model
WO2019000455A1 (en) * 2017-06-30 2019-01-03 上海联影医疗科技有限公司 Method and system for segmenting image
KR20200066732A (en) * 2017-11-30 2020-06-10 난토믹스, 엘엘씨 Intratumoral heterogeneity detection of molecular subtypes in pathology slide images using deep learning
EP3762937A1 (en) * 2018-03-08 2021-01-13 Koninklijke Philips N.V. Resolving and steering decision foci in machine learning-based vascular imaging
CN109492529A (en) * 2018-10-08 2019-03-19 中国矿业大学 A kind of Multi resolution feature extraction and the facial expression recognizing method of global characteristics fusion
CN110335259B (en) * 2019-06-25 2021-08-31 腾讯科技（深圳）有限公司 Medical image identification method and device and storage medium
CN110570483B (en) * 2019-08-08 2023-12-22 上海联影智能医疗科技有限公司 Scanning method, scanning device, computer equipment and storage medium
CN110503642B (en) * 2019-08-23 2021-10-29 强联智创（北京）科技有限公司 Positioning method and system based on DSA image
CN110852987B (en) * 2019-09-24 2022-04-22 西安交通大学 Vascular plaque detection method and device based on deep morphology and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9305349B2 (en) * 2013-06-28 2016-04-05 Samsung Electronics Co., Ltd. Apparatus and method for detecting lesion
CN110321920A (en) * 2019-05-08 2019-10-11 腾讯科技（深圳）有限公司 Image classification method, device, computer readable storage medium and computer equipment
CN110675940A (en) * 2019-08-01 2020-01-10 平安科技（深圳）有限公司 Pathological image labeling method and device, computer equipment and storage medium
CN110752028A (en) * 2019-10-21 2020-02-04 腾讯科技（深圳）有限公司 Image processing method, device, equipment and storage medium
CN110866897A (en) * 2019-10-30 2020-03-06 上海联影智能医疗科技有限公司 Image detection method and computer readable storage medium
CN111507381A (en) * 2020-03-31 2020-08-07 上海商汤智能科技有限公司 Image recognition method and related device and equipment

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115082772A (en) * 2022-07-19 2022-09-20 小米汽车科技有限公司 Location identification method, location identification device, vehicle, storage medium and chip

Also Published As

Publication number Publication date
CN111507381B (en) 2024-04-02
CN111507381A (en) 2020-08-07
TW202139208A (en) 2021-10-16
JP2022539162A (en) 2022-09-07
TWI776426B (en) 2022-09-01
KR20220031695A (en) 2022-03-11

Similar Documents

Publication Publication Date Title
WO2021196955A1 (en) Image recognition method and related apparatus, and device
US8369593B2 (en) Systems and methods for robust learning based annotation of medical radiographs
US20220222932A1 (en) Training method and apparatus for image region segmentation model, and image region segmentation method and apparatus
EP3612981A1 (en) Target detection in latent space
Zhang et al. Intelligent scanning: Automated standard plane selection and biometric measurement of early gestational sac in routine ultrasound examination
JP2022537866A (en) Image classification method, image classification device, image processing method, medical electronic device, image classification device, and computer program
WO2022062590A1 (en) Image recognition method and apparatus, device, storage medium and program
CN111696083B (en) Image processing method and device, electronic equipment and storage medium
TW202125415A (en) Training method, equipment and storage medium of 3d target detection and model
WO2019223123A1 (en) Lesion part identification method and apparatus, computer apparatus and readable storage medium
CN112508835B (en) GAN-based contrast agent-free medical image enhancement modeling method
WO2006020035A1 (en) Projection views and orientation of chest radiographs
WO2022095258A1 (en) Image object classification method and apparatus, device, storage medium and program
Yao et al. Advances on pancreas segmentation: a review
CN112634231A (en) Image classification method and device, terminal equipment and storage medium
CN111724371A (en) Data processing method and device and electronic equipment
Chupin et al. Fully automatic segmentation of the hippocampus and the amygdala from MRI using hybrid prior knowledge
Gheorghiță et al. Improving robustness of automatic cardiac function quantification from cine magnetic resonance imaging using synthetic image data
CN116862930B (en) Cerebral vessel segmentation method, device, equipment and storage medium suitable for multiple modes
WO2019223121A1 (en) Lesion site recognition method and apparatus, and computer apparatus and readable storage medium
Vittitoe et al. Markov random field modeling in posteroanterior chest radiograph segmentation
CN111598144B (en) Training method and device for image recognition model
CN115035133A (en) Model training method, image segmentation method and related device
CN113450306A (en) Method of providing a fracture detection tool
Huang et al. Cross-dataset multiple organ segmentation from CT imagery using FBP-derived domain adaptation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21780594

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021577453

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20227004540

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21780594

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 13.04.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21780594

Country of ref document: EP

Kind code of ref document: A1