WO2022242392A1 - 血管图像分类处理方法、装置、设备及存储介质 - Google Patents

血管图像分类处理方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2022242392A1
WO2022242392A1 PCT/CN2022/087404 CN2022087404W WO2022242392A1 WO 2022242392 A1 WO2022242392 A1 WO 2022242392A1 CN 2022087404 W CN2022087404 W CN 2022087404W WO 2022242392 A1 WO2022242392 A1 WO 2022242392A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
blood vessel
loss function
predicted
function value
Prior art date
Application number
PCT/CN2022/087404
Other languages
English (en)
French (fr)
Inventor
余双
陈文婷
马锴
郑冶枫
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to EP22803717.2A priority Critical patent/EP4273747A4/en
Publication of WO2022242392A1 publication Critical patent/WO2022242392A1/zh
Priority to US17/994,678 priority patent/US20230106222A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2431—Multiple classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/0016—Operational features thereof
    • A61B3/0025—Operational features thereof characterised by electronic signal processing, e.g. eye models
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/12—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes
    • A61B3/1241—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes specially adapted for observation of ocular blood flow, e.g. by fluorescein angiography
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30041—Eye; Retina; Ophthalmic
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present application relates to the technical field of artificial intelligence, and in particular to a blood vessel image classification processing method, device, equipment and storage medium.
  • retinal artery and vein classification lays the foundation for quantitative analysis of retinal blood vessels.
  • retinal A/V classification usually processes an input fundus image through a pre-trained end-to-end deep learning model, and directly outputs the positions of arteries and/or veins in the fundus image.
  • the above-mentioned deep learning model is usually obtained by taking fundus image samples as input, and using positions of arteries and/or veins in the fundus image samples as labeling information for training.
  • the embodiment of the present application provides a blood vessel image classification processing method, device, equipment and storage medium, which can improve the accuracy of the model in classifying blood vessels in the image, and the technical solution is as follows.
  • a method for classification and processing of blood vessel images comprising:
  • the first blood vessel image sample is a low-quality image corresponding to the second blood vessel image sample
  • the blood vessel position labeling information is used to indicate the Positions of at least two types of blood vessels marked in the first blood vessel image sample
  • the predicted enhanced image is performed on the first blood vessel image sample An image after quality enhancement; the predicted blood vessel position information is used to indicate the predicted positions of the at least two types of blood vessels in the first blood vessel image sample;
  • the first image processing model is trained based on the loss function value; the first image processing model after training is used to generate a second image processing model, and the second image processing model is used for inputting the target
  • the blood vessel image is processed to output blood vessel classification information of the target blood vessel image, the blood vessel classification information is used to indicate a target type of blood vessel in the target blood vessel image, and the target type of blood vessel is one of the at least two types of blood vessels at least one of .
  • a method for classification and processing of blood vessel images comprising:
  • the target blood vessel image into a second image processing model, and obtain blood vessel position information output by the second image processing model, where the blood vessel position information at least indicates the position of the predicted target type blood vessel in the target blood vessel image;
  • the second image processing model is generated based on the trained first image processing model;
  • the loss function value for training the first image processing model is based on the second blood vessel image sample, blood vessel position labeling information, predicted enhanced image and the predicted blood vessel position information;
  • the predicted enhanced image and the predicted blood vessel position information are output after the first image processing model processes the first blood vessel image sample;
  • the first blood vessel image sample is the A low-quality image corresponding to the second blood vessel image sample,
  • the blood vessel position annotation information is used to indicate the positions of at least two types of blood vessels marked in the first blood vessel image sample;
  • the predicted blood vessel position information is used to indicate the predicted positions of at least two types of blood vessels in the first blood vessel image sample.
  • a device for classification and processing of blood vessel images comprising:
  • a sample acquisition module configured to acquire a first blood vessel image sample, a second blood vessel image sample, and blood vessel position annotation information;
  • the first blood vessel image sample is a low-quality image corresponding to the second blood vessel image sample, and the blood vessel position annotation
  • the information is used to indicate the positions of at least two types of blood vessels marked in the first blood vessel image sample;
  • a prediction module configured to input the first blood vessel image sample into a first image processing model, obtain a predicted enhanced image output by the first image processing model, and predicted blood vessel position information; the predicted enhanced image is a reference to the first image processing model A quality-enhanced image of a blood vessel image sample; the predicted blood vessel position information is used to indicate the predicted positions of the at least two types of blood vessels in the first blood vessel image sample;
  • a loss acquisition module configured to acquire a loss function value based on the second blood vessel image sample, the blood vessel position annotation information, the predicted enhanced image, and the predicted blood vessel position information;
  • a training module configured to train the first image processing model based on the loss function value; the first image processing model after training is used to generate a second image processing model, and the second image processing model uses Processing the input target blood vessel image to output blood vessel classification information of the target blood vessel image, the blood vessel classification information is used to indicate the target type blood vessel in the target blood vessel image, the target type blood vessel is the at least At least one of two types of blood vessels.
  • a device for classification and processing of blood vessel images comprising:
  • An image acquisition module configured to acquire target blood vessel images
  • a model processing module configured to input the target blood vessel image into a second image processing model, and obtain blood vessel position information output by the second image processing model, where the blood vessel position information at least indicates the predicted position in the target blood vessel image the location of the vessel of the target type;
  • An output module configured to output a blood vessel classification result image based on the blood vessel position information, where the blood vessel classification result image is used to indicate the target type blood vessel in the target blood vessel image;
  • the second image processing model is generated based on the trained first image processing model;
  • the loss function value for training the first image processing model is based on the second blood vessel image sample, blood vessel position labeling information, predicted enhanced image and the predicted blood vessel position information;
  • the predicted enhanced image and the predicted blood vessel position information are output after the first image processing model processes the first blood vessel image sample;
  • the first blood vessel image sample is the A low-quality image corresponding to the second blood vessel image sample,
  • the blood vessel position annotation information is used to indicate the positions of at least two types of blood vessels marked in the first blood vessel image sample;
  • the predicted blood vessel position information is used to indicate the predicted positions of at least two types of blood vessels in the first blood vessel image sample.
  • a computer device in another aspect, includes a processor and a memory, at least one computer instruction is stored in the memory, and the at least one computer instruction is loaded and executed by the processor to realize the above-mentioned Vascular Image Classification Processing Method.
  • a computer-readable storage medium wherein at least one computer instruction is stored in the storage medium, and the at least one computer instruction is loaded and executed by a processor to implement the above-mentioned blood vessel image classification processing method.
  • a computer program product comprising computer instructions stored in a computer readable storage medium.
  • the processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above-mentioned blood vessel image classification processing method.
  • the first image processing model simultaneously performs vessel position prediction and image quality enhancement on the input first vessel image sample, and simultaneously uses high-quality second vessel image samples, vessel position annotation information, predicted enhanced images, and predicted
  • the blood vessel location information trains the first image processing model, that is, while training the blood vessel classification model, the influence of image quality on the blood vessel classification is also considered, so that the subsequent end-to-end generated based on the trained first image processing model
  • the vessel classification model on the end can have higher classification accuracy for low-quality vessel images, thereby improving the accuracy of classifying vessels in vessel images.
  • Fig. 1 is a system configuration diagram of a blood vessel classification system involved in various embodiments of the present application
  • Fig. 2 is a schematic flowchart of a blood vessel image classification processing method according to an exemplary embodiment
  • Fig. 3 is a schematic flowchart of a blood vessel image classification processing method according to an exemplary embodiment
  • Fig. 4 is a frame diagram of a blood vessel image classification process according to an exemplary embodiment
  • Fig. 5 is a schematic flowchart of a blood vessel image classification processing method according to an exemplary embodiment
  • Fig. 6 is a model frame diagram of the first image processing model involved in the embodiment shown in Fig. 5;
  • Fig. 7 is a schematic diagram of input and output of a model involved in the embodiment shown in Fig. 5;
  • Fig. 8 is a frame diagram showing a training and application framework for a fundus image processing model according to an exemplary embodiment
  • Fig. 9 is a structural block diagram of a device for classifying and processing blood vessel images according to an exemplary embodiment
  • Fig. 10 is a structural block diagram of a device for classifying and processing blood vessel images according to an exemplary embodiment
  • Fig. 11 is a schematic structural diagram of a computer device according to an exemplary embodiment.
  • FIG. 1 shows a system configuration diagram of a blood vessel classification system involved in various embodiments of the present application.
  • the system includes a medical image acquisition device 120 , a terminal 140 , and a server 160 ; optionally, the system may also include a database 180 .
  • the medical image collection device 120 may be a camera device or a camera device for collecting blood vessel images.
  • the blood vessel image refers to a medical image including blood vessels.
  • the blood vessel image may be a fundus image (including subretinal blood vessels), a gastroscopy image, a colonoscopy image, an internal oral cavity image, and the like.
  • the medical image acquisition device 120 may include an image output interface, such as a Universal Serial Bus (Universal Serial Bus, USB) interface, a High Definition Multimedia Interface (High Definition Multimedia Interface, HDMI) interface or an Ethernet interface, etc.; or, the above-mentioned image output interface It may also be a wireless interface, such as a wireless local area network (Wireless Local Area Network, WLAN) interface, a Bluetooth interface, and the like.
  • an image output interface such as a Universal Serial Bus (Universal Serial Bus, USB) interface, a High Definition Multimedia Interface (High Definition Multimedia Interface, HDMI) interface or an Ethernet interface, etc.
  • a wireless interface such as a wireless local area network (Wireless Local Area Network, WLAN) interface, a Bluetooth interface, and the like.
  • the operator may export the microscopic image taken by the camera in various ways, for example, importing the microscopic image to the terminal 140 through a wired or short-distance wireless method, or , and the microscopic image can also be imported to the terminal 140 or the server 160 through a local area network or the Internet.
  • the terminal 140 may be a terminal device with certain processing capabilities and interface display functions, for example, the terminal 140 may be a mobile phone, a tablet computer, an e-book reader, smart glasses, a laptop computer, a desktop computer, and the like.
  • the terminals 140 may include terminals used by developers and terminals used by medical personnel.
  • terminal 140 When terminal 140 is implemented as a terminal used by developers, developers can use terminal 140 to develop a machine learning model for identifying blood vessels in blood vessel images, and deploy the machine learning model to server 160 or terminals used by medical personnel .
  • an application program that obtains and presents the vessel classification results of the vessel image can be installed in the terminal 140. After the terminal 140 acquires the vessel image collected by the medical image acquisition device 120, it can pass the The application program obtains the processing results obtained by processing the vascular images, and presents the processing results so that doctors can perform operations such as pathological diagnosis.
  • the terminal 140 and the medical image acquisition device 120 are physically separate entity devices.
  • the terminal 140 and the medical image acquisition device 120 can also be integrated into a single physical device; for example, the terminal 140 can be an image A terminal device with a collection function.
  • the server 160 can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, and can also provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud Cloud servers for basic cloud computing services such as communications, middleware services, domain name services, security services, content delivery network (Content Delivery Network, CDN), and big data and artificial intelligence platforms.
  • cloud databases cloud computing, cloud functions, cloud storage, network services, cloud Cloud servers for basic cloud computing services such as communications, middleware services, domain name services, security services, content delivery network (Content Delivery Network, CDN), and big data and artificial intelligence platforms.
  • cloud services cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud Cloud servers for basic cloud computing services such as communications, middleware services, domain name services, security services, content delivery network (Content Delivery Network, CDN), and big data and artificial intelligence platforms.
  • CDN Content Delivery Network
  • the above-mentioned server 160 may be a server that provides background services for the application programs installed in the terminal 140.
  • the background server may be the version management of the application programs, perform background processing on the blood vessel images acquired by the application programs and return the processing results, and provide support for development.
  • the machine learning model developed by the staff performs background training and so on.
  • the foregoing database 180 may be a Redis database, or may also be other types of databases. Among them, the database 180 is used to store various types of data.
  • the terminal 140 and the server 160 are connected through a communication network.
  • the medical image acquisition device 120 is connected to the server 160 through a communication network.
  • the communication network is a wired network or a wireless network.
  • the system may further include a management device (not shown in FIG. 1 ), which is connected to the server 160 through a communication network.
  • a management device (not shown in FIG. 1 ), which is connected to the server 160 through a communication network.
  • the communication network is a wired network or a wireless network.
  • the aforementioned wireless network or wired network uses standard communication technologies and/or protocols.
  • the network is usually the Internet, but it can also be any network, including but not limited to LAN (Local Area Network, Local Area Network), MAN (Metropolitan Area Network, Metropolitan Area Network), WAN (Wide Area Network, Wide Area Network), mobile, wired or wireless Any combination of network, private network, or virtual private network.
  • technologies and/or formats including HTML (Hyper Text Mark-up Language, Hypertext Markup Language), XML (Extensible Markup Language, Extensible Markup Language), etc. are used to represent data exchanged through the network.
  • SSL Secure Socket Layer
  • TLS Transport Layer Security
  • VPN Virtual Private Network
  • IPsec Internet Protocol Security, Internet Protocol Security
  • customized and/or dedicated data communication technologies may also be used to replace or supplement the above data communication technologies.
  • Fig. 2 is a schematic flowchart of a method for classifying and processing blood vessel images according to an exemplary embodiment.
  • the method may be performed by a computer device, for example, the computer device may be a server, or the computer device may also be a terminal, or the computer device may include a server and a terminal, wherein the server may be the
  • the server 160 in the embodiment, the terminal may be the terminal 140 used by developers in the embodiment shown in FIG. 1 above.
  • the computer device can be implemented as a model training device for model training.
  • the blood vessel image classification processing method may include the following steps.
  • Step 201 acquire a first blood vessel image sample, a second blood vessel image sample, and blood vessel position annotation information; the first blood vessel image sample is a low-quality image corresponding to the second blood vessel image sample, and the blood vessel position annotation information is used to indicate that the first blood vessel image sample is a low-quality image corresponding to the second blood vessel image sample. Positions of at least two types of blood vessels marked in a blood vessel image sample.
  • the above-mentioned first blood vessel image sample is a low-quality image corresponding to the second blood vessel image sample, which may mean that the first blood vessel image sample and the second blood vessel image sample have the same image content, and the image of the first blood vessel image sample The quality is lower than the image quality of the second blood vessel image sample.
  • the difference between the first blood vessel image sample and the second blood vessel image sample may be reflected in resolution, detail restoration degree, color restoration degree, contrast, etc., and this embodiment does not limit the low quality measurement standard.
  • the first blood vessel image sample is obtained by performing a degeneration operation on the second blood vessel image sample.
  • the above-mentioned blood vessel position labeling information may be information marked by the developer based on the second blood vessel image sample in advance.
  • the above-mentioned blood vessel position labeling information may be a masked (Masked) image with at least two channels, and each channel of the mask image is used to indicate a type in the corresponding blood vessel image The location of blood vessels.
  • the above-mentioned blood vessel position labeling information may also be coordinate information, for example, the coordinate information may contain at least two sets of coordinate sets, and each set of coordinate sets contains one type of corresponding blood vessel image Position coordinates (such as pixel coordinates) of blood vessels.
  • the above-mentioned at least two types of blood vessels may include at least two of arterial blood vessels, venous blood vessels, and whole blood vessels (that is, all blood vessels including arterial blood vessels and venous blood vessels).
  • Step 202 input the first blood vessel image sample into the first image processing model, and obtain the predicted enhanced image output by the first image processing model, as well as the predicted blood vessel position information; the predicted enhanced image is the quality of the first blood vessel image sample Enhanced image: the predicted blood vessel position information is used to indicate the predicted positions of at least two types of blood vessels in the first blood vessel image sample.
  • the first image processing model is a machine learning model pre-built by the developer to be trained.
  • the first image processing model may be a deep learning model.
  • the first image processing model includes at least one input port and two output ports; wherein, during the training process, the input port is used to input the first blood vessel image sample; then, the first image processing model is used to process the first blood vessel image Two types of processing are performed on the sample, one processing is to predict the position of at least two types of blood vessels in the first blood vessel image sample (that is to say, while predicting the position of the blood vessel in the first blood vessel image sample, the predicted The blood vessels corresponding to the blood vessel positions are classified), and the predicted blood vessel position information is output through one output port, and the other processing is to enhance the image quality of the first blood vessel image sample, and output the predicted enhanced image through another output port.
  • the above-mentioned predicted blood vessel position information and the above-mentioned blood vessel position labeling information may be the same type of information, for example, both the predicted blood vessel position information and the blood vessel position labeling information are mask images, or both are coordinate information; or, the above-mentioned predicted blood vessel position
  • the information and the above-mentioned blood vessel position labeling information can also be different types of information.
  • the predicted blood vessel position information is a mask image
  • the blood vessel position labeling information is coordinate information.
  • the predicted blood vessel position information is coordinate information
  • the blood vessel position labeling information is mask image.
  • Step 203 Acquire a loss function value based on the second blood vessel image sample, the blood vessel position annotation information, the predicted enhanced image, and the predicted blood vessel position information.
  • the computer device can combine the second blood vessel image sample, the blood vessel position labeling information, the predicted enhanced image and the predicted blood vessel position information to calculate the loss function to obtain the The training loss function value.
  • Step 204 train the first image processing model based on the loss function value; the first image processing model after training is used to generate a second image processing model, and the second image processing model is used to analyze the input target blood vessel
  • the image is processed to output blood vessel classification information of the target blood vessel image, where the blood vessel classification information is used to indicate a target type of blood vessel in the target blood vessel image, and the target type of blood vessel is at least one of the at least two types of blood vessels.
  • the first image processing model simultaneously performs blood vessel position prediction and image quality enhancement on the input first blood vessel image sample, and uses the high-quality second blood vessel image sample at the same time.
  • the first image processing model is trained on the blood vessel image samples, the vessel position annotation information, the predicted enhanced image and the predicted vessel position information, that is to say, while training the vessel classification model, the influence of the image quality on the vessel classification is also considered, so that
  • the subsequent end-to-end blood vessel classification model generated based on the trained first image processing model can have higher classification accuracy for low-quality blood vessel images, thereby improving the accuracy of classifying blood vessels in the blood vessel images.
  • the second image processing model generated based on the first image processing model can be applied to various image processing including blood vessels and the blood vessels therein can be processed. Categorized scenes.
  • the second image processing model can be applied to end-to-end blood vessel classification for fundus images, gastroscopic images, colonoscopic images, or intraoral images.
  • the process of the second image processing model being used for blood vessel classification may refer to the following embodiments.
  • Fig. 3 is a schematic flowchart of a method for classifying and processing blood vessel images according to an exemplary embodiment.
  • the method may be performed by a computer device, for example, the computer device may be a server, or the computer device may also be a terminal, or the computer device may include a server and a terminal, wherein the server may be the
  • the server 160 in the embodiment, the terminal may be the terminal 140 used by medical personnel in the embodiment shown in FIG. 1 above.
  • the computer device can be implemented as a model application device for vessel classification.
  • the blood vessel image classification processing method may include the following steps.
  • Step 301 acquiring target blood vessel images.
  • the target blood vessel image is an image used for blood vessel location and blood vessel type identification.
  • the target blood vessel image is a fundus image captured by a terminal, or a gastroscope image captured by a medical instrument.
  • Step 302 Input the target blood vessel image into a second image processing model to obtain blood vessel position information output by the second image processing model, the blood vessel position information at least indicating the position of the target type blood vessel predicted in the target blood vessel image.
  • the blood vessel position information may be a mask image, or may also be coordinate information.
  • Step 303 Output a blood vessel classification result image based on the blood vessel position information, where the blood vessel classification result image is used to indicate the target type of blood vessel in the target blood vessel image.
  • the above blood vessel classification result image may be an image in which a target type of blood vessel is marked on the basis of the target blood vessel image.
  • the blood vessel classification result image may be an image in which arteries/veins are marked on the target blood vessel image (for example, arteries and veins are marked with different colors).
  • the second image processing model is generated based on the trained first image processing model; the loss function value for training the first image processing model is based on the second blood vessel image sample, blood vessel position labeling information, predicted enhanced image, and predicted The blood vessel position information is obtained; the predicted enhanced image and the predicted blood vessel position information are output after the first image processing model processes the first blood vessel image sample; the first blood vessel image sample is corresponding to the second blood vessel image sample
  • the blood vessel position annotation information is used to indicate the positions of at least two types of blood vessels marked in the first blood vessel image sample; the predicted enhanced image is an image after quality enhancement of the first blood vessel image sample;
  • the predicted blood vessel position information is used to indicate the predicted positions of at least two types of blood vessels in the first blood vessel image sample.
  • the second image processing model is generated based on the first image processing model, and during the training process of the first image processing model, the first image processing model Perform vessel position prediction and image quality enhancement on a blood vessel image sample at the same time, and simultaneously use the high-quality second blood vessel image sample, blood vessel position labeling information, predicted enhanced image, and predicted blood vessel position information to train the first image processing model, that is,
  • the influence of image quality on vascular classification is also considered, so that the subsequent end-to-end vascular classification model generated based on the first image processing model after training can have higher accuracy for low-quality vascular images.
  • the classification accuracy is improved, thereby improving the accuracy of classifying blood vessels in blood vessel images.
  • the blood vessel image classification processing solution involved in this application can be divided into two stages, namely the model training stage and the model application stage.
  • FIG. 4 is a frame diagram of a blood vessel image classification process according to an exemplary embodiment.
  • the blood vessel image classification process is as follows.
  • the computer device pre-acquires a low-quality first blood vessel image sample 401 , a high-quality second blood vessel image sample 402 , and blood vessel position annotation information 403 corresponding to the first blood vessel image sample 401 .
  • the blood vessel position annotation information 403 indicates the positions of multiple types of blood vessels in the object to be recognized indicated by the second blood vessel image sample 401 .
  • the computer device inputs the first blood vessel image sample 401 to the first image processing model 404, and the first image processing model 404 respectively performs blood vessel position prediction and image quality enhancement on the first blood vessel image sample 401, and outputs the predicted The obtained predicted blood vessel position information 405, and the predicted enhanced image 406 after quality enhancement; wherein, the predicted blood vessel position information 405 also indicates the positions of various types of blood vessels predicted in the first blood vessel image sample 401; then, through the second The blood vessel image sample 402, the blood vessel position labeling information 403, the predicted blood vessel position information 405, and the predicted enhanced image 406 are calculated to obtain a loss function value, and the first image processing model 404 is trained by the loss function value.
  • the above training process is repeated until the training of the first image processing model 404 is completed (for example, the convergence condition is reached).
  • the computer device can automatically or under the operation of the developer, generate the second image processing model 407 based on the first image processing model 404, and deploy the second image processing model 407 .
  • the computer device inputs the target blood vessel image 408 into the second image processing model 407, and the second image processing model 407 outputs blood vessel position information 409, and then, the computer device can output the The blood vessel classification result image 410 of the target type blood vessel in the blood vessel image, so that medical personnel can make corresponding decisions/judgments according to the blood vessel classification result image 410 .
  • Fig. 5 is a schematic flowchart of a method for classifying and processing blood vessel images according to an exemplary embodiment.
  • the blood vessel image classification processing method may include the following steps.
  • Step 501 acquiring a first blood vessel image sample, a second blood vessel image sample, and blood vessel position annotation information.
  • the first blood vessel image sample is a low-quality image corresponding to the second blood vessel image sample
  • the blood vessel position annotation information is used to indicate the positions of at least two types of blood vessels marked in the first blood vessel image sample.
  • the computer device may obtain the low-quality first blood vessel image sample by degrading the high-quality second blood vessel image sample.
  • the computer device can degrade the second blood vessel image sample by simulating the degradation model of factors such as uneven illumination, image blur, and artifacts to obtain the first blood vessel image sample.
  • Multiple types of blood vessel labeling for example, labeling arterial vessels and venous vessels in the second blood vessel image sample respectively to obtain vessel position labeling information indicating positions of at least two types of blood vessels.
  • Step 502 Input the first blood vessel image sample into a first image processing model, and obtain a predicted enhanced image output by the first image processing model, and predicted blood vessel position information.
  • the predicted enhanced image is an image after quality enhancement of the first blood vessel image sample; the predicted blood vessel position information is used to indicate the predicted positions of at least two types of blood vessels in the first blood vessel image sample.
  • the computer device after the computer device inputs the first blood vessel image sample into the first image processing model, it can process the first blood vessel image sample through the image segmentation branch in the first image processing model to obtain the output of the image segmentation branch. Predicting blood vessel position information; and processing the first blood vessel image sample through the image enhancement branch in the first image processing model to obtain the predicted enhanced image output by the image enhancement branch.
  • the first image processing model may include two branches, namely the image segmentation branch and the image enhancement branch; wherein, the image segmentation branch is used for blood vessel classification, that is, to predict the different types of blood vessels in the input image. position; the image enhancement branch is used to improve the image quality of the input image.
  • the first image processing model processes the first blood vessel image sample in parallel through the image segmentation branch and the image enhancement branch, and outputs predicted enhanced image and predicted blood vessel position information respectively.
  • the image segmentation branch and the image enhancement branch share an encoder
  • the image segmentation branch further includes a first decoder
  • the image enhancement branch further includes a second decoder. That is, the image segmentation branch is composed of an encoder and a first decoder, and the image enhancement branch is composed of an encoder and a second decoder.
  • the above image segmentation branch and the image enhancement branch share the encoder, which may refer to the model component of the shared encoder.
  • the image segmentation branch and the image enhancement branch have only one encoder component, and the image segmentation branch and the image enhancement branch respectively call the The encoder component performs image processing.
  • the aforementioned image segmentation branch and the image enhancement branch share an encoder, which may refer to sharing the parameters of the encoder.
  • the image segmentation branch and the image enhancement branch each have an encoder component, and the encoding in the two encoder components parameters are the same.
  • FIG. 6 shows a model frame diagram of the first image processing model involved in the embodiment of the present application.
  • the first image processing model 60 is a double-branch network, and the overall architecture includes two branches: an image segmentation branch 61 and an image enhancement branch 62; wherein, the image segmentation branch 61 includes a first decoder 61a, and the image enhancement branch 61 Branch 62 includes a second decoder 62a, and image segmentation branch 61 and image enhancement branch 62 share encoder 63 .
  • the image segmentation branch 61 is used to predict and classify the position of blood vessels in the input image, such as generating an A/V classification mask image
  • the image enhancement branch 62 obtains high-quality images based on reconstruction from input low-quality images. image.
  • the above two branches share the same encoder and each has a different decoder for simultaneous A/V classification and image enhancement. For each high-quality image, through the degradation model processing, a low-quality image can be obtained as an input image, and a real high-quality image and a true A/V classification mask are obtained, which are used to train the image enhancement branch and the image Split branches.
  • the U-Net structure can be adopted with a pre-trained ResNet18 as an encoder, which produces a three-channel probability map for the segmentation of arteries, veins, and all vessels.
  • the image enhancement branch can also be an encoder-decoder based network and share the same encoder with the image segmentation branch to encourage the encoder to extract features that are relevant for A/V classification and robust to image degradation.
  • the goal of the image enhancement branch is to reconstruct a high-quality original image from an intentionally degraded low-quality image.
  • the computer device may take the first blood vessel image After the samples are divided into blocks, they are sequentially input into the first image processing model to obtain the predicted enhanced image blocks and predicted blood vessel position information blocks corresponding to each first blood vessel image sample block respectively, and the predicted enhanced image blocks corresponding to each first blood vessel image sample block respectively Merging to obtain a predicted enhanced image, and merging the predicted blood vessel position information blocks corresponding to each first blood vessel image sample block to obtain predicted blood vessel position information.
  • Figure 7 shows a schematic diagram of input and output of a model involved in the embodiment of the present application.
  • the computer device extracts an image sample block 72 at a time, and inputs the extracted image sample block 72 into the first image processing model 73, and the first image
  • the processing model 73 outputs the prediction mask image block 74 and the prediction enhancement image block 75 corresponding to the image sample block 72; finally, the computer device will splice each prediction mask image block 74 according to the input sequence of each image sample 72, to obtain
  • the mask image 76 is predicted, and each predicted enhanced image block 75 is spliced to obtain a predicted enhanced image 77 .
  • the training process randomly extract patches for the first blood vessel image samples (such as low-quality fundus color photos) and corresponding arteriovenous labels, and then send the extracted patches to the network (the first image processing model)
  • the output result of the network is a reconstructed high-quality image and a predicted mask image of 3 channels.
  • the predicted mask image includes arterial blood vessel map, venous blood vessel map and all blood vessel maps; finally, the network corresponding to the predicted patch
  • the output results are spliced according to the original extraction order, that is, the final predicted enhanced image and arteriovenous classification results are obtained.
  • using the image blocks extracted from the low-quality first blood vessel image samples as input data for model training can increase the effective training quantity.
  • the computer device can obtain a loss function value based on the second blood vessel image sample, the blood vessel position annotation information, the predicted enhanced image, and the predicted blood vessel position information; for this process, refer to subsequent steps 503 and 504 .
  • Step 503 based on the blood vessel location labeling information and the predicted blood vessel location information, a blood vessel classification loss function value among the loss function values is obtained.
  • the computer device may calculate the difference between the blood vessel position labeling information and the predicted blood vessel position information, and obtain the blood vessel classification loss function value among the loss function values.
  • the computer device can use binary cross-entropy loss to calculate the pixel-level distance between the generated mask and the real mask, the formula of which is as follows:
  • x represents the low-quality input picture (that is, the first blood vessel image sample mentioned above)
  • L c represents the real arteriovenous classification mask image of the c class (corresponding to the above blood vessel position labeling information);
  • E and D 1 represent the representation image respectively
  • (x)) represents the probability map of a certain channel output by the image segmentation branch.
  • ⁇ c can be set to 0.4 and 0.3 respectively and 0.3.
  • the computer device can first convert it into a mask image, and then calculate the blood vessel classification loss function value L BCE according to the above formula 1.
  • Step 504 based on the second blood vessel image sample and the predicted enhanced image, obtain an image enhancement loss function value among the loss function values.
  • the computer device may calculate the difference between the second blood vessel image sample and the predicted enhanced image to obtain the image enhancement loss function value among the loss function values.
  • the obtaining the image enhancement loss function value in the loss function value based on the second blood vessel image sample and the predicted enhanced image includes:
  • the sub-loss function value includes at least one of a first sub-loss function value, a second sub-loss function value, and a third sub-loss function value kind;
  • the first sub-loss function value is used to indicate the overall difference between the second blood vessel image sample and the predicted enhanced image
  • the second sub-loss function value is used to indicate the human visual perception difference between the second blood vessel image sample and the predicted enhanced image
  • the third sub-loss function value is used to indicate the image difference of the blood vessel parts respectively corresponding to the second blood vessel image sample and the predicted enhanced image.
  • the L1 loss can be used as the reconstruction loss to calculate the overall similarity between the low-quality image and the corresponding high-quality image (corresponding to the above-mentioned first sub-loss function value):
  • D2 represents the decoder of the image enhancement branch
  • y represents the real high-quality image (ie, the second blood vessel image sample).
  • the present application can also introduce a perceptual image quality loss based on visual perception (corresponding to the above-mentioned second sub-loss function value).
  • the SSIM loss is calculated as follows:
  • ⁇ i and ⁇ i 2 denote the mean and variance of image I i respectively.
  • ⁇ 12 represents the covariance of I 1 and I 2 ;
  • the embodiments of the present application further propose a mask loss (corresponding to the value of the third sub-loss function) to enhance image reconstruction quality near the blood vessel region. Since the main task of this application is vessel classification, the quality of generated images of vessel regions is more important than background pixels.
  • the acquiring sub-loss function value based on the second blood vessel image sample and the predicted enhanced image includes:
  • the third sub-loss function value is obtained.
  • the computer device multiplies the generated blood vessel mask with the enhanced image to obtain a first mask image M 1 related to the enhanced image.
  • the second mask image M 2 related to the real image can also be obtained by calculating the element-wise multiplication of the real blood vessel mask and the real high-quality image.
  • the mask loss is constructed by optimizing the L1 loss between M1 and M2 . The calculation of the mask loss is as follows:
  • L represents the real blood vessel mask (that is, the above-mentioned blood vessel position labeling information);
  • L j represents the jth pixel of L.
  • the position corresponding to the pixel is a blood vessel, it is set to 1, otherwise it is set to 0;
  • indicates the number of pixels in L; ⁇ indicates element-by-element multiplication; M 1j and M 2j respectively indicate mask The jth pixel of the augmented image correlated with the mask and the real image correlated with the mask.
  • the sub-loss function value includes at least two values of the first sub-loss function value, the second sub-loss function value, and the third sub-loss function value, based on the The sub-loss function value obtains the value of the image enhancement loss function, including:
  • At least two values in the sub-loss function values are weighted to obtain the image enhancement loss function value.
  • the final optimization function is binary cross-entropy loss, reconstruction loss, SSIM loss and mask loss weighted combination.
  • the overall loss function can look like this:
  • ⁇ 1 , ⁇ 2 and ⁇ 3 are used to control the importance of each loss relative to the binary cross-entropy, for example, ⁇ 1 , ⁇ 2 and ⁇ 3 can be set to 0.2, 0.4 and 0.2, respectively.
  • ⁇ 1 , ⁇ 2 and ⁇ 3 can be preset in the computer device by the developer, for example, they can be preset in the loss function of the first image processing model by the developer.
  • Step 505 train the first image processing model based on the loss function value.
  • the computer device may update the parameters of the encoder based on the blood vessel classification loss function value and the image enhancement loss function value, The parameters of the first decoder are updated based on the blood vessel classification loss function value, and the parameters of the second decoder are updated based on the image enhancement loss function value.
  • the computer device can update the parameters of the above-mentioned shared encoder through L, update the parameters of the decoder (ie the first decoder) in the image segmentation branch through L BCE , and update the parameters through ⁇ 1 L rec + ⁇ 2 L ssim + ⁇ 3 L mask updates the parameters of the decoder (ie, the second decoder) in the image enhancement branch.
  • the parameter updating of the encoder is performed based on the blood vessel classification loss function value and the image enhancement loss function value, including:
  • Weighting is performed on the blood vessel classification loss function value and the image enhancement loss function value to obtain a total loss function value; based on the total loss function value, the parameters of the encoder are updated.
  • the computer device may also perform weighting processing on the blood vessel classification loss function value and the image enhancement loss function value.
  • the computer device trains the encoder shared by the image segmentation branch and the image enhancement branch, it can consider the importance of the information learned by the image enhancement branch to the blood vessel classification task, so as to calculate the blood vessel classification loss function value and the image
  • the augmented loss function values set different weights.
  • the respective weights of the above blood vessel classification loss function value and the image enhancement loss function value can be preset in the computer device by the developer, for example, can be preset in the loss function of the first image processing model by the developer.
  • the parameters of the two branches in the first image processing model can be updated respectively through different parts of the loss function value.
  • the blood vessel classification loss function value in the above loss function values (such as L BCE in Figure 6) is used to train the image segmentation branch
  • the image enhancement loss function values in the loss function values (such as L rec , L ssim , and L mask ) are used to train the image enhancement branch.
  • the image segmentation branch in the first image processing model can also learn the information of the image enhancement part, thereby improving the Accuracy of the Image Segmentation Branch for Vessel Classification on Low-Quality Images.
  • Step 506 when the training of the first image processing model is completed, generate a second image processing model based on the first image processing model.
  • the computer device After the training of the first image processing model is completed, the computer device generates the second image processing model based on the image segmentation branch.
  • the second image processing model is used to process the input target blood vessel image to output blood vessel classification information of the target blood vessel image
  • the blood vessel classification information is used to indicate the target type blood vessel in the target blood vessel image
  • the target type The blood vessel is at least one of the at least two types of blood vessels.
  • the image enhancement branch can be discarded, and only the image segmentation branch can be reserved for blood vessel classification (such as A/V classification). Compared with the complete operation of the first image processing model, it can significantly reduce Reasoning time.
  • the computer device may input the target blood vessel image into the second image processing model, and obtain blood vessel position information output by the second image processing model, where the blood vessel position information at least indicates that the target blood vessel image is Predicting the position of the target type of blood vessel; and outputting a blood vessel classification result image based on the blood vessel position information, where the blood vessel classification result image is used to indicate the target type of blood vessel in the target blood vessel image.
  • the model training loss in addition to determining the model training loss based on the image difference and visual perception difference between the predicted enhanced image and the high-quality blood vessel image sample, it is also based on the corresponding mask image between the predicted enhanced image and the high-quality blood vessel image sample The difference determines the model training loss, which helps to improve the quality of model training, and then improves the accuracy of subsequent vessel classification using the trained model.
  • FIG. 8 is a framework diagram for training and application of a fundus image processing model according to an exemplary embodiment.
  • the model training and application process for fundus image vessel classification can be as follows:
  • the developer prepares high-quality fundus image samples, processes the high-quality fundus image samples 81a through a degeneration model to obtain low-quality fundus image samples 81b, and then performs labeling operations based on the fundus image samples to obtain various types of blood vessels
  • the marked mask image 81c of the real position, and then, the high-quality fundus image sample 81a, the low-quality fundus image sample 81b and the marked mask image 81c are input into the model training device as training data.
  • the model training device inputs the low-quality fundus image sample 81b into the first image processing model 82, outputs the prediction mask image 83 through the image segmentation branch in the first image processing model 82, and passes through the first image processing model
  • the image enhancement branch in 82 outputs a prediction enhancement image 83; wherein the image segmentation branch and the image enhancement branch share an encoder;
  • Image 84 the loss function value 85 is calculated, the parameters of the image segmentation branch are updated by the blood vessel classification loss function value in the loss function value 85, and the parameters of the image enhancement branch are updated by the image enhancement loss function value in the loss function value 85 ;
  • the model training device updates the decoder in the image segmentation branch through the blood vessel classification loss function value, and through the image enhancement loss function value
  • the decoder in the image enhancement branch is updated, and the shared encoder is jointly updated by the vessel classification loss function value and the image enhancement loss function value.
  • the developer can construct the second image processing model 86 through the image segmentation branch in the first image processing model 82 and deploy it to the model application device.
  • the model application device can receive the input target fundus image 87, and input the target fundus image 87 into the second image processing model 86, and the second image processing model 86 outputs the corresponding mask image 88, and based on the mask The image 88 outputs a fundus image 89 after blood vessel classification.
  • a fundus image is a non-invasive medical image.
  • fundus images blood vessels can be observed non-invasively.
  • various systemic, cardiovascular, and brain diseases may occur.
  • biomarker studies on retinal blood vessels have found that larger venous calibers are associated with the progression of diabetic retinopathy.
  • the reduction of retinal artery caliber is related to the risk of hypertension and diabetes.
  • arterial/vein classification provides the basis for quantitative vascular analysis and facilitates retinal biomarker studies.
  • the end-to-end dual-branch network may include an enhancement branch and a segmentation branch, thereby improving the performance of A/V classification while improving image quality, Among them, the two branches can share the same encoder, and the loss of structural similarity is used to reduce the artifacts of the reconstructed image.
  • the application can also use the mask loss to emphasize the quality of the enhanced image reconstructed near the blood vessel.
  • the vascular classification model constructed by the above-mentioned dual-branch network can provide more accurate vascular classification, further assist doctors in systematically diagnosing cardiovascular and brain diseases, and in the fundus screening system, assist doctors in judging whether the distribution of blood vessels is normal or not , so as to help prevent and diagnose fundus diseases and systemic diseases, such as high blood pressure and diabetes.
  • This application conducts ablation research based on the scheme shown in the above-mentioned embodiments of this application to evaluate the effectiveness of various loss functions in the image enhancement branch (i.e., reconstruction loss, SSIM loss, and mask loss) when tested under low-quality images .
  • this application adopts three metrics of accuracy (Accuracy, Acc), sensitivity (Sensitivity, Sen) and likelihood (Specificity, Spec), based on real blood vessel pixel-level images A/V classification performance is evaluated to compare the effectiveness of different modules under the same criterion.
  • the A/V classification accuracy is increased by 2.15%, indicating that when adding the image enhancement task to the network, the encoder can Image extraction with more robust features.
  • this application also adds SSIM loss and mask loss to the image enhancement branch respectively.
  • the A/V classification accuracy increases by 1.02%.
  • the accuracy of A/V classification is increased by 0.90% when the mask loss is adopted, indicating that the mask loss helps the A/V classification task by emphasizing vessel regions.
  • the image segmentation branch has the best performance for A/V classification, and the accuracy rate for low-quality fundus images reaches 91.52%.
  • Table 2 shows the comparison of the A/V classification performance of the present application under low-quality test images and the A/V classification technology in the related art.
  • This application mainly compares the methods shown in the above embodiments with the U-Net that is widely used at present.
  • This application uses the high-quality (UNet-HighQuality, UNet-HQ) and low-quality (UNet-LowQuality, UNet-LQ) fundus images of the AV-DRIVE dataset to train two U-Net models.
  • the performance of the UNet-HQ model is evaluated on high-quality and low-quality test images, respectively, as shown in the first two rows of Table 2.
  • the A/V classification accuracy of the dual-branch network provided by this application on the AV-DRIVE dataset is 91.52%, which is better than that of the U-Net model on low-quality fundus images.
  • the training accuracy is 3.3% higher.
  • the performance of the dual-branch network provided by this application is very close to the theoretical upper limit, with a performance gap of only 0.1%.
  • the solutions shown in the above-mentioned embodiments of the present application can be implemented or executed in combination with blockchain.
  • some or all of the steps in the above-mentioned embodiments can be executed in the blockchain system; or, the data required or generated by the execution of the various steps in the above-mentioned embodiments can be stored in the blockchain system; for example , the training samples used in the above model training, and the model input data such as target blood vessel images in the model application process can be obtained by computer equipment from the block chain system; for another example, the parameters of the model obtained after the above model training (including the first The parameters of the first image processing model and the parameters of the second image processing model) can be stored in the block chain system.
  • Fig. 9 is a structural block diagram of an apparatus for classifying and processing blood vessel images according to an exemplary embodiment.
  • the device can implement all or part of the steps in the method provided by the embodiment shown in Figure 2 or Figure 5, and the blood vessel image classification processing device includes:
  • the sample acquisition module 901 is configured to acquire a first blood vessel image sample, a second blood vessel image sample, and blood vessel position annotation information;
  • the first blood vessel image sample is a low-quality image corresponding to the second blood vessel image sample, and the blood vessel position
  • the annotation information is used to indicate the positions of at least two types of blood vessels marked in the first blood vessel image sample;
  • a predicting module 902 configured to input the first blood vessel image sample into a first image processing model, obtain a predicted enhanced image output by the first image processing model, and predict blood vessel position information; the predicted enhanced image is for the described The quality-enhanced image of the first blood vessel image sample; the predicted blood vessel position information is used to indicate the predicted positions of the at least two types of blood vessels in the first blood vessel image sample;
  • a loss acquisition module 903 configured to acquire a loss function value based on the second blood vessel image sample, the blood vessel position annotation information, the predicted enhanced image, and the predicted blood vessel position information;
  • a training module 904 configured to train the first image processing model based on the loss function value
  • the first image processing model after training is used to generate a second image processing model, and the second image processing model is used to process the input target blood vessel image to output blood vessel classification information of the target blood vessel image,
  • the blood vessel classification information is used to indicate a target type blood vessel in the target blood vessel image, and the target type blood vessel is at least one of the at least two types of blood vessels.
  • the prediction module 902 includes:
  • an input submodule configured to input the first blood vessel image sample into a first image processing model
  • the segmentation sub-module is configured to process the first blood vessel image sample through the image segmentation branch in the first image processing model, and obtain the predicted blood vessel position information output by the image segmentation branch;
  • the enhancement sub-module is configured to process the first blood vessel image sample through the image enhancement branch in the first image processing model, and obtain the predicted enhanced image output by the image enhancement branch.
  • the loss acquisition module 903 includes:
  • a classification loss acquisition submodule configured to acquire a blood vessel classification loss function value among the loss function values based on the vessel position labeling information and the predicted vessel position information;
  • the enhancement loss acquisition submodule is configured to acquire an image enhancement loss function value in the loss function values based on the second blood vessel image sample and the predicted enhanced image.
  • the enhancement loss acquisition submodule is configured to:
  • the sub-loss function value includes the first sub-loss function value, the second sub-loss function value, and the third sub-loss function value at least one of
  • the first sub-loss function value is used to indicate the overall difference between the second blood vessel image sample and the predicted enhanced image
  • the second sub-loss function value is used to indicate the visual perception difference between the second blood vessel image sample and the predicted enhanced image
  • the third sub-loss function value is used to indicate the image difference of the blood vessel parts respectively corresponding to the second blood vessel image sample and the predicted enhanced image.
  • the enhancement loss acquisition submodule is configured to:
  • the third sub-loss function value is obtained.
  • the enhanced loss acquisition submodule is used to,
  • the sub-loss function values include at least two values of the first sub-loss function value, the second sub-loss function value, and the third sub-loss function value, at least two of the sub-loss function values
  • the value of the item is weighted to obtain the value of the image enhancement loss function.
  • the image segmentation branch and the image enhancement branch share an encoder
  • the image segmentation branch further includes a first decoder
  • the image enhancement branch further includes a second decoder
  • the training module 904 includes:
  • An encoder updating submodule configured to update parameters of the encoder based on the blood vessel classification loss function value and the image enhancement loss function value;
  • the first decoder update submodule is configured to update the parameters of the first decoder based on the blood vessel classification loss function value
  • the second decoder updating submodule is configured to update the parameters of the second decoder based on the image enhancement loss function value.
  • the encoder update submodule is configured to:
  • Parameter updating is performed on the encoder based on the total loss function value.
  • the device further includes:
  • a model generation module configured to generate the second image processing model based on the image segmentation branch in response to the completion of the training of the first image processing model.
  • the device further includes:
  • An image input module configured to input the target blood vessel image into the second image processing model, and obtain blood vessel position information output by the second image processing model, where the blood vessel position information at least indicates that the target blood vessel image is predicted The location of the target type of blood vessel;
  • the result output module is configured to output a blood vessel classification result image based on the blood vessel position information, and the blood vessel classification result image is used to indicate the target type blood vessel in the target blood vessel image.
  • Fig. 10 is a structural block diagram of an apparatus for classifying and processing blood vessel images according to an exemplary embodiment.
  • the blood vessel image classification processing device can implement all or part of the steps in the method provided by the embodiment shown in Figure 3 or Figure 5, and the dialogue information processing device includes:
  • An image acquisition module 1001 configured to acquire a target blood vessel image
  • a model processing module 1002 configured to input the target blood vessel image into a second image processing model, and obtain blood vessel position information output by the second image processing model, where the blood vessel position information at least indicates that the target blood vessel image is predicted The position of the target type vessel;
  • An output module 1003 configured to output a blood vessel classification result image based on the blood vessel position information, where the blood vessel classification result image is used to indicate the target type blood vessel in the target blood vessel image;
  • the second image processing model is generated based on the trained first image processing model;
  • the loss function value for training the first image processing model is based on the second blood vessel image sample, blood vessel position labeling information, predicted enhanced image and the predicted blood vessel position information;
  • the predicted enhanced image and the predicted blood vessel position information are output after the first image processing model processes the first blood vessel image sample;
  • the first blood vessel image sample is the A low-quality image corresponding to the second blood vessel image sample,
  • the blood vessel position annotation information is used to indicate the positions of at least two types of blood vessels marked in the first blood vessel image sample;
  • the predicted blood vessel position information is used to indicate the predicted positions of at least two types of blood vessels in the first blood vessel image sample.
  • Fig. 11 is a schematic structural diagram of a computer device according to an exemplary embodiment.
  • the computer device may be implemented as the computer device used for training the first image processing model in the above method embodiments, or may be implemented as the computer device used for classifying blood vessels through the second image processing model in the above method embodiments.
  • the computer device 1100 includes a central processing unit (CPU, Central Processing Unit) 1101, a system memory 1104 including a random access memory (Random Access Memory, RAM) 1102 and a read-only memory (Read-Only Memory, ROM) 1103, and A system bus 1105 that connects the system memory 1104 and the central processing unit 1101 .
  • the computer device 1100 also includes a basic input/output controller 1106 that facilitates the transfer of information between various components within the computer, and a mass storage device 1107 for storing an operating system 1113 , application programs 1114 and other program modules 1115 .
  • the mass storage device 1107 is connected to the central processing unit 1101 through a mass storage controller (not shown) connected to the system bus 1105 .
  • the mass storage device 1107 and its associated computer-readable media provide non-volatile storage for the computer device 1100 . That is to say, the mass storage device 1107 may include a computer-readable medium (not shown) such as a hard disk or a Compact Disc Read-Only Memory (CD-ROM) drive.
  • a computer-readable medium such as a hard disk or a Compact Disc Read-Only Memory (CD-ROM) drive.
  • Computer-readable media may comprise computer storage media and communication media.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
  • Computer storage media includes RAM, ROM, flash memory or other solid-state storage technologies, CD-ROM, or other optical storage, tape cartridges, magnetic tape, magnetic disk storage or other magnetic storage devices.
  • RAM random access memory
  • ROM read-only memory
  • flash memory or other solid-state storage technologies
  • CD-ROM Compact Disc
  • tape cartridges magnetic tape
  • magnetic disk storage magnetic disk storage devices
  • the computer device 1100 can be connected to the Internet or other network devices through the network interface unit 1111 connected to the system bus 1105 .
  • the memory also includes one or more programs, and the one or more programs are stored in the memory, and the central processing unit 1101 implements any one of the programs shown in FIG. 2 , FIG. 3 or FIG. 5 by executing the one or more programs. all or part of the steps of the method.
  • non-transitory computer-readable storage medium comprising instructions, such as a memory comprising a computer program (instructions), which can be executed by a processor of a computer device to perform the present application
  • instructions such as a memory comprising a computer program (instructions)
  • the non-transitory computer readable storage medium can be a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a read-only optical disc (Compact Disc Read-Only Memory, CD -ROM), tapes, floppy disks and optical data storage devices, etc.
  • a computer program product or computer program comprising computer instructions stored in a computer readable storage medium.
  • the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods shown in the foregoing embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Medical Informatics (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • Ophthalmology & Optometry (AREA)
  • Public Health (AREA)
  • Databases & Information Systems (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Surgery (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Veterinary Medicine (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Primary Health Care (AREA)
  • Signal Processing (AREA)
  • Epidemiology (AREA)
  • Human Computer Interaction (AREA)
  • Hematology (AREA)
  • Vascular Medicine (AREA)
  • Image Analysis (AREA)

Abstract

一种血管图像分类处理方法、装置、设备及存储介质，涉及人工智能技术领域。所述方法包括：将第一血管图像样本输入第一图像处理模型，获得预测增强图像以及预测血管位置信息(202)；基于第二血管图像样本、血管位置标注信息、预测增强图像以及预测血管位置信息，对第一图像处理模型进行训练(203)；上述方案在训练血管分类模型的同时，还考虑了图像质量对于血管分类的影响，使得后续基于训练完成的第一图像处理模型生成的端到端的血管分类模型能够对低质量的血管图像具有更高的分类精度，从而提高通过人工智能对血管图像中的血管进行分类的准确性。

Description

血管图像分类处理方法、装置、设备及存储介质
本申请要求于2021年05月19日提交，申请号为202110547148.7、发明名称为“血管图像分类处理方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请实施例中。
技术领域
本申请涉及人工智能技术领域，特别涉及一种血管图像分类处理方法、装置、设备及存储介质。
背景技术
在医疗领域中，对视网膜血管定量分析有助于发现各种心脑血管疾病的潜在风险，而视网膜动静脉(Artery or Vein，A/V)分类则为定量分析视网膜血管奠定了基础。
在相关技术中，视网膜A/V分类通常是通过预先训练好的端到端深度学习模型对输入的眼底图像进行处理，直接输出眼底图像中的动脉和/或静脉的位置。其中，上述深度学习模型通常以眼底图像样本为输入，并以眼底图像样本中的动脉和/或静脉的位置为标注信息进行训练得到。
然而，眼底图像采集时，很多情况下无法保证采集到的图像的质量，而低质量的眼底图像会影响深度学习模型的处理效果，从而导致对眼底图像中的血管进行分类的准确性较低。
发明内容
本申请实施例提供了一种血管图像分类处理方法、装置、设备及存储介质，可以提高模型对图像中的血管进行分类的准确性，该技术方案如下。
一方面，提供了一种血管图像分类处理方法，所述方法包括：
获取第一血管图像样本、第二血管图像样本以及血管位置标注信息；所述第一血管图像样本是所述第二血管图像样本对应的低质量图像，所述血管位置标注信息用于指示所述第一血管图像样本中被标注出的至少两种类型血管的位置；
将所述第一血管图像样本输入第一图像处理模型，获得所述第一图像处理模型输出的预测增强图像，以及预测血管位置信息；所述预测增强图像是对所述第一血管图像样本进行质量增强后的图像；所述预测血管位置信息用于指示所述第一血管图像样本中被预测出的所述至少两种类型血管的位置；
基于所述第二血管图像样本、所述血管位置标注信息、所述预测增强图像以及所述预测血管位置信息，获取损失函数值；
基于所述损失函数值对所述第一图像处理模型进行训练；训练完成后的所述第一图像处理模型用于生成第二图像处理模型，所述第二图像处理模型用于对输入的目标血管图像进行处理，以输出所述目标血管图像的血管分类信息，所述血管分类信息用于指示所述目标血管图像中的目标类型血管，所述目标类型血管是所述至少两种类型血管中的至少一种。
又一方面，提供了一种血管图像分类处理方法，所述方法包括：
获取目标血管图像；
将所述目标血管图像输入第二图像处理模型，获得所述第二图像处理模型输出的血管位置信息，所述血管位置信息至少指示所述目标血管图像中被预测出的目标类型血管的位置；
基于所述血管位置信息，输出血管分类结果图像，所述血管分类结果图像用于指示所述目标血管图像中的所述目标类型血管；
其中，所述第二图像处理模型是基于训练完成的第一图像处理模型生成的；训练所述第一图像处理模型的损失函数值是基于第二血管图像样本、血管位置标注信息、预测增强图像以及预测血管位置信息获取的；所述预测增强图像以及所述预测血管位置信息是所述第一图像处理模型对第一血管图像样本进行处理后输出的；所述第一血管图像样本是所述第二血管图像样本对应的低质量图像，所述血管位置标注信息用于指示所述第一血管图像样本中被标 注出的至少两种类型血管的位置；所述预测增强图像是对所述第一血管图像样本进行质量增强后的图像；所述预测血管位置信息用于指示所述第一血管图像样本中被预测出的至少两种类型血管的位置。
再一方面，提供了一种血管图像分类处理装置，所述装置包括：
样本获取模块，用于获取第一血管图像样本、第二血管图像样本以及血管位置标注信息；所述第一血管图像样本是所述第二血管图像样本对应的低质量图像，所述血管位置标注信息用于指示所述第一血管图像样本中被标注出的至少两种类型血管的位置；
预测模块，用于将所述第一血管图像样本输入第一图像处理模型，获得所述第一图像处理模型输出的预测增强图像，以及预测血管位置信息；所述预测增强图像是对所述第一血管图像样本进行质量增强后的图像；所述预测血管位置信息用于指示所述第一血管图像样本中被预测出的所述至少两种类型血管的位置；
损失获取模块，用于基于所述第二血管图像样本、所述血管位置标注信息、所述预测增强图像以及所述预测血管位置信息，获取损失函数值；
训练模块，用于基于所述损失函数值对所述第一图像处理模型进行训练；训练完成后的所述第一图像处理模型用于生成第二图像处理模型，所述第二图像处理模型用于对输入的目标血管图像进行处理，以输出所述目标血管图像的血管分类信息，所述血管分类信息用于指示所述目标血管图像中的目标类型血管，所述目标类型血管是所述至少两种类型血管中的至少一种。
又一方面，提供了一种血管图像分类处理装置，所述装置包括：
图像获取模块，用于获取目标血管图像；
模型处理模块，用于将所述目标血管图像输入第二图像处理模型，获得所述第二图像处理模型输出的血管位置信息，所述血管位置信息至少指示所述目标血管图像中被预测出的目标类型血管的位置；
输出模块，用于基于所述血管位置信息，输出血管分类结果图像，所述血管分类结果图像用于指示所述目标血管图像中的所述目标类型血管；
其中，所述第二图像处理模型是基于训练完成的第一图像处理模型生成的；训练所述第一图像处理模型的损失函数值是基于第二血管图像样本、血管位置标注信息、预测增强图像以及预测血管位置信息获取的；所述预测增强图像以及所述预测血管位置信息是所述第一图像处理模型对第一血管图像样本进行处理后输出的；所述第一血管图像样本是所述第二血管图像样本对应的低质量图像，所述血管位置标注信息用于指示所述第一血管图像样本中被标注出的至少两种类型血管的位置；所述预测增强图像是对所述第一血管图像样本进行质量增强后的图像；所述预测血管位置信息用于指示所述第一血管图像样本中被预测出的至少两种类型血管的位置。
再一方面，提供了一种计算机设备，所述计算机设备包含处理器和存储器，所述存储器中存储有至少一条计算机指令，所述至少一条计算机指令由所述处理器加载并执行以实现上述的血管图像分类处理方法。
又一方面，提供了一种计算机可读存储介质，所述存储介质中存储有至少一条计算机指令，所述至少一条计算机指令由处理器加载并执行以实现上述的血管图像分类处理方法。
又一方面，提供了一种计算机程序产品，该计算机程序产品包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上述血管图像分类处理方法。
在训练过程中，第一图像处理模型对输入的第一血管图像样本同时执行血管位置预测和图像质量增强，并且同时使用高质量的第二血管图像样本、血管位置标注信息、预测增强图像以及预测血管位置信息对第一图像处理模型进行训练，也就是说，在训练血管分类模型的同时，还考虑了图像质量对于血管分类的影响，使得后续基于训练完成的第一图像处理模型生成的端到端的血管分类模型能够对低质量的血管图像具有更高的分类精度，从而提高对血 管图像中的血管进行分类的准确性。
附图说明
图1是本申请各个实施例涉及的一种血管分类***的***构成图；
图2是根据一示例性实施例示出的一种血管图像分类处理方法的流程示意图；
图3是根据一示例性实施例示出的一种血管图像分类处理方法的流程示意图；
图4是根据一示例性实施例示出的一种血管图像分类处理框架图；
图5是根据一示例性实施例示出的一种血管图像分类处理方法的流程示意图；
图6是图5所示实施例涉及的第一图像处理模型的模型框架图；
图7是图5所示实施例涉及的一种模型输入输出示意图；
图8是根据一示例性实施例示出的一种用于眼底图像处理模型的训练及应用框架图；
图9是根据一示例性实施例示出的一种血管图像分类处理装置的结构方框图；
图10是根据一示例性实施例示出的一种血管图像分类处理装置的结构方框图；
图11是根据一示例性实施例示出的一种计算机设备的结构示意图。
具体实施方式
请参考图1，其示出了本申请各个实施例涉及的一种血管分类***的***构成图。该***包括医疗图像采集设备120、终端140、以及服务器160；可选的，该***还可以包括数据库180。
医疗图像采集设备120可以是用于采集血管图像的相机设备或者摄像头设备。其中，血管图像是指包含血管的医疗图像，比如，血管图像可以是眼底图像(包含视网膜下的血管)、胃镜图像、肠镜图像、口腔内部图像等等。
医疗图像采集设备120可以包含图像输出接口，比如通用串行总线(Universal Serial Bus，USB)接口、高清多媒体接口(High Definition Multimedia Interface，HDMI)接口或者以太网接口等等；或者，上述图像输出接口也可以是无线接口，比如无线局域网(Wireless Local Area Network，WLAN)接口、蓝牙接口等等。
相应的，根据上述图像输出接口的类型的不同，操作人员将照相机拍摄的显微图像导出的方式也可以有多种，比如，通过有线或者短距离无线方式将显微图像导入至终端140，或者，也可以通过局域网或者互联网将显微图像导入至终端140或者服务器160。
终端140可以是具有一定的处理能力以及界面展示功能的终端设备，比如，终端140可以是手机、平板电脑、电子书阅读器、智能眼镜、膝上型便携计算机和台式计算机等等。
终端140可以包括开发人员使用的终端，以及医疗人员使用的终端。
当终端140实现为开发人员使用的终端时，开发人员可以通过终端140开发用于对血管图像中的血管进行识别的机器学习模型，并将机器学习模型部署到服务器160或者医疗人员使用的终端中。
当终端140实现为医疗人员使用的终端时，终端140中可以安装有获取血管图像的血管分类结果并呈现的应用程序，终端140获取到医疗图像采集设备120采集到的血管图像后，可以通过上述应用程序获取对血管图像进行处理得到的处理结果，并对处理结果进行呈现，以便医生进行病理诊断等操作。
在图1所示的***中，终端140和医疗图像采集设备120是物理上分离的实体设备。可选的，在另一种可能的实现方式中，当终端140实现为医疗人员使用的终端时，终端140和医疗图像采集设备120也可以集成为单个实体设备；比如，该终端140可以是图像采集功能的终端设备。
其中，服务器160可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式***，还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络(Content Delivery Network，CDN)、以及大数据和人工智能平台等基础云计算服务的云服务器。
其中，上述服务器160可以是为终端140中安装的应用程序提供后台服务的服务器，该 后台服务器可以是应用程序的版本管理、对应用程序获取到的血管图像进行后台处理并返回处理结果、对开发人员开发的机器学习模型进行后台训练等等。
上述数据库180可以是Redis数据库，或者，也可以是其它类型数据库。其中，数据库180用于存储各类数据。
可选的，终端140与服务器160之间通过通信网络相连。可选的，医疗图像采集设备120与服务器160之间通过通信网络相连。可选的，该通信网络是有线网络或无线网络。
可选的，该***还可以包括管理设备(图1未示出)，该管理设备与服务器160之间通过通信网络相连。可选的，通信网络是有线网络或无线网络。
可选的，上述的无线网络或有线网络使用标准通信技术和/或协议。网络通常为因特网、但也可以是任何网络，包括但不限于LAN(Local Area Network，局域网)、MAN(Metropolitan Area Network，城域网)、WAN(Wide Area Network，广域网)、移动、有线或者无线网络、专用网络或者虚拟专用网络的任何组合。在一些实施例中，使用包括HTML(Hyper Text Mark-up Language，超文本标记语言)、XML(Extensible Markup Language，可扩展标记语言)等的技术和/或格式来代表通过网络交换的数据。此外还可以使用诸如SSL(Secure Socket Layer，安全套接字层)、TLS(Transport Layer Security，传输层安全)、VPN(Virtual Private Network，虚拟专用网络)、IPsec(Internet Protocol Security，网际协议安全)等常规加密技术来加密所有或者一些链路。在另一些实施例中，还可以使用定制和/或专用数据通信技术取代或者补充上述数据通信技术。
图2是根据一示例性实施例示出的一种血管图像分类处理方法的流程示意图。该方法可以由计算机设备执行，比如，该计算机设备可以是服务器，或者，该计算机设备也可以是终端，或者，该计算机设备可以包括服务器和终端，其中，该服务器可以是上述图1所示的实施例中的服务器160，该终端可以是上述图1所示的实施例中由开发人员使用的终端140。该计算机设备可以实现为进行模型训练的模型训练设备。如图2所示，该血管图像分类处理方法可以包括如下步骤。
步骤201，获取第一血管图像样本、第二血管图像样本以及血管位置标注信息；该第一血管图像样本是该第二血管图像样本对应的低质量图像，该血管位置标注信息用于指示该第一血管图像样本中被标注出的至少两种类型血管的位置。
其中，上述第一血管图像样本是该第二血管图像样本对应的低质量图像，可以是指第一血管图像样本和第二血管图像样本具有相同的图像内容，并且，第一血管图像样本的图像质量低于第二血管图像样本的图像质量。
可选的，第一血管图像样本与第二血管图像样本的差异可以体现在分辨率、细节还原度、色彩还原度、对比度等等，本实施例对低质量的衡量标准不作限定。
可选的，第一血管图像样本由第二血管图像样本进行退化操作得到。
上述血管位置标注信息可以是开发人员预先基于第二血管图像样本标注出的信息。
在一种可能的实现方式中，上述血管位置标注信息可以是一个具有至少两个通道的掩模(Masked)图像，该掩模图像的每个通道用于指示对应的血管图像中的一种类型血管的位置。
在另一种可能的实现方式中，上述血管位置标注信息也可以是坐标信息，比如，该坐标信息中可以包含至少两组坐标集，每一组坐标集中包含对应的血管图像中的一种类型血管的位置坐标(比如像素坐标)。
在本申请实施例中，上述至少两种类型血管，可以包括动脉血管、静脉血管、以及整体血管(即包含动脉血管和静脉血管的所有血管)中的至少两种。
步骤202，将该第一血管图像样本输入第一图像处理模型，获得该第一图像处理模型输出的预测增强图像，以及预测血管位置信息；该预测增强图像是对该第一血管图像样本进行质量增强后的图像；该预测血管位置信息用于指示该第一血管图像样本中被预测出的至少两种类型血管的位置。
在本申请实施例中，第一图像处理模型是开发人员预先构建好的，待训练的机器学习模 型，比如，该第一图像处理模型可以是深度学习模型。
该第一图像处理模型至少包括一个输入端口和两个输出端口；其中，在训练过程中，输入端口用于输入第一血管图像样本；然后，该第一图像处理模型用于对第一血管图像样本进行两种处理，一种处理是预测第一血管图像样本中的至少两种类型血管的位置(也就是说，在对第一血管图像样本中的血管位置进行预测的同时，还对预测出的血管位置对应的血管进行分类)，并通过一个输出端口输出预测血管位置信息，另一种处理是对第一血管图像样本的图像质量进行增强，并通过另一个输出端口输出预测增强图像。
其中，上述预测血管位置信息与上述血管位置标注信息可以是相同类型的信息，比如，预测血管位置信息与血管位置标注信息都是掩模图像，或者，都是坐标信息；或者，上述预测血管位置信息与上述血管位置标注信息也可以是不同类型的信息，比如，预测血管位置信息是掩模图像，血管位置标注信息是坐标信息，再比如，预测血管位置信息是坐标信息，血管位置标注信息是掩模图像。
步骤203，基于该第二血管图像样本、该血管位置标注信息、该预测增强图像以及该预测血管位置信息，获取损失函数值。
在本申请实施例中，计算机设备可以同时结合第二血管图像样本、该血管位置标注信息、该预测增强图像以及该预测血管位置信息，进行损失函数计算，得到用于对第一图像处理模型进行训练的损失函数值。
步骤204，基于该损失函数值对该第一图像处理模型进行训练；训练完成后的该第一图像处理模型用于生成第二图像处理模型，该第二图像处理模型用于对输入的目标血管图像进行处理，以输出该目标血管图像的血管分类信息，该血管分类信息用于指示该目标血管图像中的目标类型血管，该目标类型血管是该至少两种类型血管中的至少一种。
综上所述，本申请实施例所示的方案，在训练过程中，第一图像处理模型对输入的第一血管图像样本同时执行血管位置预测和图像质量增强，并且同时使用高质量的第二血管图像样本、血管位置标注信息、预测增强图像以及预测血管位置信息对第一图像处理模型进行训练，也就是说，在训练血管分类模型的同时，还考虑了图像质量对于血管分类的影响，使得后续基于训练完成的第一图像处理模型生成的端到端的血管分类模型能够对低质量的血管图像具有更高的分类精度，从而提高对血管图像中的血管进行分类的准确性。
基于上述图2所示的实施例训练得到第一图像处理模型后，基于该第一图像处理模型生成的第二图像处理模型，可以应用于各种对包含血管的图像处理并对其中的血管进行分类的场景。比如，第二图像处理模型可以应用于对眼底图像、胃镜图像、肠镜图像或者口腔内部图像进行端到端的血管分类。其中，第二图像处理模型用于血管分类的过程可以参考下述实施例。
图3是根据一示例性实施例示出的一种血管图像分类处理方法的流程示意图。该方法可以由计算机设备执行，比如，该计算机设备可以是服务器，或者，该计算机设备也可以是终端，或者，该计算机设备可以包括服务器和终端，其中，该服务器可以是上述图1所示的实施例中的服务器160，该终端可以是上述图1所示的实施例中由医疗人员使用的终端140。该计算机设备可以实现为进行血管分类的模型应用设备。如图3所示，该血管图像分类处理方法可以包括如下步骤。
步骤301，获取目标血管图像。
其中，该目标血管图像是用于进行血管定位以及血管类型识别的图像。比如，该目标血管图像是使用终端拍摄得到的眼底图像，或者，使用医学仪器拍摄得到的胃镜图像。
步骤302，将该目标血管图像输入第二图像处理模型，获得该第二图像处理模型输出的血管位置信息，该血管位置信息至少指示该目标血管图像中被预测出的目标类型血管的位置。
其中，该血管位置信息可以是掩模图像，或者，也可以是坐标信息。
步骤303，基于该血管位置信息，输出血管分类结果图像，该血管分类结果图像用于指示该目标血管图像中的该目标类型血管。
在一种可能的实现方式中，上述血管分类结果图像可以是在目标血管图像的基础上标注出目标类型血管的图像。比如，血管分类结果图像可以是在目标血管图像上标注出动脉/静脉血管的图像(例如，通过不同的颜色标记出动脉和静脉)。
其中，该第二图像处理模型是基于训练完成的第一图像处理模型生成的；训练该第一图像处理模型的损失函数值是基于第二血管图像样本、血管位置标注信息、预测增强图像以及预测血管位置信息获取的；该预测增强图像以及该预测血管位置信息是该第一图像处理模型对第一血管图像样本进行处理后输出的；该第一血管图像样本是该第二血管图像样本对应的低质量图像，该血管位置标注信息用于指示该第一血管图像样本中被标注出的至少两种类型血管的位置；该预测增强图像是对该第一血管图像样本进行质量增强后的图像；该预测血管位置信息用于指示该第一血管图像样本中被预测出的至少两种类型血管的位置。
综上所述，本申请实施例所示的方案，第二图像处理模型是基于第一图像处理模型生成的，而在第一图像处理模型的训练过程中，第一图像处理模型对输入的第一血管图像样本同时执行血管位置预测和图像质量增强，并且同时使用高质量的第二血管图像样本、血管位置标注信息、预测增强图像以及预测血管位置信息对第一图像处理模型进行训练，也就是说，在训练血管分类模型的同时，还考虑了图像质量对于血管分类的影响，使得后续基于训练完成的第一图像处理模型生成的端到端的血管分类模型能够对低质量的血管图像具有更高的分类精度，从而提高对血管图像中的血管进行分类的准确性。
本申请涉及到的血管图像分类处理方案可以分为两个阶段，分别为模型训练阶段和模型应用阶段。请参考图4，其是根据一示例性实施例示出的一种血管图像分类处理框架图。该血管图像分类处理过程如下。
计算机设备预先获取低质量的第一血管图像样本401、高质量的第二血管图像样本402、以及第一血管图像样本401对应的血管位置标注信息403。其中，血管位置标注信息403指示第二血管图像样本401所指示待识别对象中多种类型血管的位置。
在模型训练阶段，计算机设备将第一血管图像样本401输入到第一图像处理模型404，第一图像处理模型404分别对第一血管图像样本401进行血管位置预测和图像质量增强，并分别输出预测得到的预测血管位置信息405，以及，质量增强后的预测增强图像406；其中，预测血管位置信息405也指示第一血管图像样本401中预测出的多种类型血管的位置；然后，通过第二血管图像样本402、血管位置标注信息403、预测血管位置信息405、以及预测增强图像406，计算获得损失函数值，并通过损失函数值对第一图像处理模型404进行训练。重复上述训练过程，直至第一图像处理模型404训练完成(比如达到收敛条件)。
在第一图像处理模型404训练完成后，计算机设备可以自动的，或者在开发人员的操作下，基于第一图像处理模型404生成第二图像处理模型407，并对第二图像处理模型407进行部署。
在模型应用阶段，计算机设备将目标血管图像408输入至第二图像处理模型407，由第二图像处理模型407输出血管位置信息409，然后，计算机设备可以根据该血管位置信息409，输出能够指示目标血管图像中的该目标类型血管的血管分类结果图像410，以便医疗人员根据血管分类结果图像410做出相应的决策/判断。
图5是根据一示例性实施例示出的一种血管图像分类处理方法的流程示意图。。如图5所示，该血管图像分类处理方法可以包括如下步骤。
步骤501，获取第一血管图像样本、第二血管图像样本以及血管位置标注信息。
其中，该第一血管图像样本是该第二血管图像样本对应的低质量图像，该血管位置标注信息用于指示该第一血管图像样本中被标注出的至少两种类型血管的位置。
在一种可能的实现方式中，计算机设备可以通过对高质量的第二血管图像样本进行退化处理，得到低质量的第一血管图像样本。
例如，计算机设备可以通过模拟光照不均、图像模糊和伪影等因素的退化模型，对第二血管图像样本进行退化处理，得到第一血管图像样本，之后，开发人员对第二血管图像样本 进行多种类型的血管标注，比如，分别对第二血管图像样本中的动脉血管、静脉血管进行标注，得到指示至少两种类型血管的位置的血管位置标注信息。
步骤502，将该第一血管图像样本输入第一图像处理模型，获得该第一图像处理模型输出的预测增强图像，以及预测血管位置信息。
其中，该预测增强图像是对该第一血管图像样本进行质量增强后的图像；该预测血管位置信息用于指示该第一血管图像样本中被预测出的至少两种类型血管的位置。
其中，计算机设备将该第一血管图像样本输入第一图像处理模型后，可以通过该第一图像处理模型中的图像分割分支对该第一血管图像样本进行处理，获得该图像分割分支输出的该预测血管位置信息；并通过该第一图像处理模型中的图像增强分支对该第一血管图像样本进行处理，获得该图像增强分支输出的该预测增强图像。
在本申请实施例中，第一图像处理模型可以包含两个分支，分别为图像分割分支和图像增强分支；其中，图像分割分支用于进行血管分类，即预测输入图像中的各种类型血管的位置；图像增强分支用于提升输入图像的图像质量。第一血管图像样本被输入第一图像处理模型之后，第一图像处理模型通过图像分割分支和图像增强分支，对第一血管图像样本并行处理，并分别输出预测增强图像，以及预测血管位置信息。
在一种可能的实现方式中，该图像分割分支和该图像增强分支共享编码器，该图像分割分支还包含第一解码器，该图像增强分支还包含第二解码器。即图像分割分支由编码器以及第一解码器构成，图像增强分支由编码器以及第二解码器构成。
其中，上述图像分割分支和该图像增强分支共享编码器，可以是指共享编码器这个模型组件，比如，图像分割分支和图像增强分支只有一个编码器组件，图像分割分支和图像增强分支分别调用该编码器组件进行图像处理。
或者，上述图像分割分支和该图像增强分支共享编码器，可以是指共享编码器的参数，比如，图像分割分支和图像增强分支分别具有一个编码器组件，且这两个编码器组件中的编码器参数是相同的。
请参考图6，其示出了本申请实施例涉及的第一图像处理模型的模型框架图。如图6所示，第一图像处理模型60是一个双分支网络，整体架构包含两个分支：图像分割分支61和图像增强分支62；其中，图像分割分支61包含第一解码器61a，图像增强分支62包含第二解码器62a，并且，图像分割分支61和图像增强分支62共享编码器63。
在图6中，图像分割分支61用于对输入的图像中的血管进行位置预测和分类，比如生成A/V分类掩模图像，图像增强分支62基于输入的低质量图像中重构得到高质量图像。并且，上述两个分支共享同一个编码器，并各自具有不同的解码器，用于同时进行A/V分类和图像增强。对于每一幅高质量图像，通过退化模型处理，可以得到一幅低质量的图像作为输入图像，并得到真实高质量图像和真值A/V分类掩模，分别用来训练图像增强分支和图像分割分支。
在图像分割分支中，可以采用U-Net结构，并以预先训练好的ResNet18作为编码器，该解码器产生了一个三通道的概率图，用于动脉、静脉和所有血管的分割。
图像增强分支也可以是一个基于编码器-解码器的网络，并且与图像分割分支共享同一个编码器，以鼓励编码器提取与A/V分类相关且对图像退化具有鲁棒性的特征。图像增强分支的目标是从有意退化的低质量图像中重建出高质量的原始图像。
在本申请实施例中，在通过第一图像处理模型对第一血管图像样本进行处理，以获得第一图像处理模型输出的预测增强图像以及预测血管位置信息时，计算机设备可以将第一血管图像样本分块后依次输入第一图像处理模型，得到各个第一血管图像样本块分别对应的预测增强图像块和预测血管位置信息块，并将各个第一血管图像样本块分别对应的预测增强图像块合并，得到预测增强图像，以及，将各个第一血管图像样本块分别对应的预测血管位置信息块进行合并，得到预测血管位置信息。
以预测血管位置信息是掩模图像为例，请参考图7，其示出了本申请实施例涉及的一种 模型输入输出示意图。如图7所示，对于第一血管图像样本71，计算机设备在训练过程中，每次提取一个图像样本块72，并将提取的图像样本块72输入第一图像处理模型73，由第一图像处理模型73输出与图像样本块72对应的预测掩模图像块74和预测增强图像块75；最后，计算机设备将按照各个图像样本72的输入顺序，将各个预测掩模图像块74进行拼接，得到预测掩模图像76，并将各个预测增强图像块75进行拼接，得到预测增强图像77。
例如，在训练过程中，对于第一血管图像样本(比如低质量眼底彩照)和对应的动静脉标签进行随机提取分块(patch)，然后将提取的patch送入网络(第一图像处理模型)进行预测，网络输出结果为重构的高质量图像，以及3个通道的预测掩模图像，预测掩模图像包括动脉血管图，静脉血管图和所有血管图；最后，将预测的patch对应的网络输出结果按原始提取顺序进行拼接，即得到最后的预测增强图像以及动静脉分类结果。
在本申请实施例中，使用从低质量的第一血管图像样本中提取的图像块作为输入数据进行模型训练的，可以增加有效的训练数量。
计算机设备可以基于该第二血管图像样本、该血管位置标注信息、该预测增强图像以及该预测血管位置信息，获取损失函数值；该过程可以参考后续步骤503和步骤504。
步骤503，基于该血管位置标注信息以及该预测血管位置信息，获取该损失函数值中的血管分类损失函数值。
在本申请实施例中，计算机设备可以计算血管位置标注信息以及该预测血管位置信息之间的差异，获得损失函数值中的血管分类损失函数值。
在一种可能的实现方式中，计算机设备可以使用二进制交叉熵损失计算生成的掩模与真实掩模之间的像素级距离，其公式如下所示：
Figure PCTCN2022087404-appb-000001
其中，x表示低质量的输入图片(即上述第一血管图像样本)，L c表示第c类的真实动静脉分类掩膜图像(对应上述血管位置标注信息)；E和D 1分别表示表示图像分割分支的编码器和解码器，D 1(E|(x))表示图像分割分支输出的某个通道的概率图，对于所有血管、动脉和静脉的权重，可以分别设置μ c为0.4、0.3和0.3。
其中，当上述血管位置标注信息或者该预测血管位置信息不是掩模图像时，计算机设备可以先将其转化为掩模图像，然后再通过上述公式1计算出血管分类损失函数值L BCE。
步骤504，基于该第二血管图像样本以及该预测增强图像，获取该损失函数值中的图像增强损失函数值。
在本申请实施例中，计算机设备可以计算第二血管图像样本以及该预测增强图像之间的差异，获得损失函数值中的图像增强损失函数值。
在一种可能的实现方式中，该基于该第二血管图像样本以及该预测增强图像，获取该损失函数值中的图像增强损失函数值，包括：
基于该第二血管图像样本以及该预测增强图像，获取子损失函数值；该子损失函数值包括第一子损失函数值、第二子损失函数值、以及第三子损失函数值中的至少一种；
基于该子损失函数值获取该图像增强损失函数值；
该第一子损失函数值用于指示该第二血管图像样本与该预测增强图像之间的整体差异；
该第二子损失函数值用于指示该第二血管图像样本与该预测增强图像之间的人眼视觉感知差异；
该第三子损失函数值用于指示该第二血管图像样本与该预测增强图像各自对应的血管部位的图像差异。
需要说明的是，子损失函数值的类型越多，模型的训练效率越高，且训练得到的模型的鲁棒性越高。
在本申请实施例中，可以采用L1损失作为重建损失，计算低质量图像与相应的高质量图像之间的整体相似度(对应上述第一子损失函数值)：
L rec＝||D 2(E|(x))-y|| 1   (2)
其中，D 2代表图像增强分支的解码器，而y则代表真实的高质量图像(即第二血管图像样本)。
为了进一步保持低质量图像与高质量真实图像之间的结构相似性(Structural Similarity，SSIM)，本申请还可以引入一种基于视觉感知的感知图像质量损失(对应上述第二子损失函数值)。SSIM损失的计算如下：
L ssim＝-SSIM(D 2(E|(x))，y)   (3)
Figure PCTCN2022087404-appb-000002
其中μ i和σ i 2分别表示图像I i的均值和方差。σ 12表示I 1和I 2的协方差；C 1和C 2是用于稳定计算的常数(比如，C 1＝0.01×255 2，C 2＝0.03×255 2)。
除重建损失和SSIM损失外，本申请实施例还进一步提出一种掩模损失(对应上述第三子损失函数值)，以增强血管区域附近的图像重建质量。由于本申请的主要任务是血管分类，因此血管区域生成图像的质量比背景像素更重要。
在一种可能的实现方式中，在该子损失函数值包括该第三子损失函数值的情况下，该基于该第二血管图像样本以及该预测增强图像，获取子损失函数值，包括：
基于该第二血管图像样本以及该血管位置标注信息获取第一局部图像，该第一局部图像是该第二血管图像样本中的血管部位的图像；
基于该预测增强图像以及该预测血管位置信息获取第二局部图像，该第二局部图像是该预测增强图像中的血管部位的图像；
基于该第一局部图像和该第二局部图像，获取该第三子损失函数值。
为了获得增强图像中的血管区域，计算机设备将生成的血管掩模与增强图像进行元素相乘，得到增强图像相关的第一掩膜图像M 1。同样地，真实图像相关的第二掩膜图像M 2，也可以通过计算真实血管掩模与真实高质量图像的逐元素乘法得到。然后，针对位于真实血管区域的像素点，通过优化M 1与M 2之间的L1损失从而构建掩模损失。掩模损失的计算如下所示：
M 1＝D 1(E(x))⊙D 2(E(x))   (5)
M 2＝L⊙y   (6)
Figure PCTCN2022087404-appb-000003
其中，L表示真实的血管掩模(即上述血管位置标注信息)；L j表示L的第j个像素。在L中，若像素对应的位置为血管，则置为一，反之置为0；||L||表示L中的像素个数；⊙表示逐元素相乘；M 1j和M 2j分别表示掩模相关的增强图像和掩模相关的真实图像的第j个像素。
在一种可能的实现方式中，在该子损失函数值包括第一子损失函数值、第二子损失函数值、以及第三子损失函数值中的至少两项数值的情况下，该基于该子损失函数值获取该图像增强损失函数值，包括：
对该子损失函数值中的至少两项数值进行加权处理，获得该图像增强损失函数值。
以子损失函数值包括第一子损失函数值、第二子损失函数值、以及第三子损失函数值为例，最终的优化函数是二进制交叉熵损失、重构损失、SSIM损失和掩模损失的加权组合。总的损失函数可以如下所示：
L＝L BCE+λ 1L rec+λ 2L ssim+λ 3L mask   (8)
其中，λ 1，λ 2和λ 3用来控制每个损失相对于二进制交叉熵的重要性，比如，λ 1，λ 2和λ 3可以分别设置为0.2、0.4和0.2。
其中，上述λ 1，λ 2和λ 3可以由开发人员预先设置在计算机设备中，例如，可以由开发人员预先设置在第一图像处理模型的损失函数中。
步骤505，基于该损失函数值对该第一图像处理模型进行训练。
在本申请实施例中，在基于该损失函数值对该第一图像处理模型进行训练时，计算机设备可以基于该血管分类损失函数值以及该图像增强损失函数值，对该编码器进行参数更新，基于该血管分类损失函数值对该第一解码器进行参数更新，并基于该图像增强损失函数值对该第二解码器进行参数更新。
也就是说，计算机设备可以通过L对上述共享的编码器进行参数更新，通过L BCE对图像分割分支中的解码器(即第一解码器)进行参数更新，通过λ 1L rec+λ 2L ssim+λ 3L mask对图像增强分支中的解码器(即第二解码器)进行参数更新。
在一种可能的实现方式中，该基于该血管分类损失函数值以及该图像增强损失函数值，对该编码器进行参数更新，包括：
对该血管分类损失函数值以及该图像增强损失函数值进行加权处理，获得总损失函数值；基于该总损失函数值对该编码器进行参数更新。
在本申请实施例中，为了更精确的控制血管分类和图像增强任务对编码器的影响，计算机设备还可以对血管分类损失函数值以及该图像增强损失函数值进行加权处理。
其中，计算机设备在对上述图像分割分支和图像增强分支共享的编码器进行训练时，可以考虑图像增强分支学习到的信息对血管分类任务的重要性程度，以对血管分类损失函数值以及该图像增强损失函数值设置不同的权重。其中，上述血管分类损失函数值以及该图像增强损失函数值各自的权重，可以由开发人员预先设置在计算机设备中，例如，可以由开发人员预先设置在第一图像处理模型的损失函数中。
其中，如图6所示，计算机设备通过上述损失函数值对第一图像处理模型进行训练时，可以通过损失函数值中的不同部分，分别对第一图像处理模型中的两个分支进行参数更新，其中，上述损失函数值中的血管分类损失函数值(如图6中的L BCE)用于训练图像分割分支，而损失函数值中的图像增强损失函数值(如图6中的L rec、L ssim、以及L mask)用于训练图像增强分支，由于图像分割分支和图像增强分支共享编码器，因此，第一图像处理模型中图像分割分支也能够学习到图像增强部分的信息，从而提高了图像分割分支对低质量图像进行血管分类的准确性。
步骤506，在该第一图像处理模型训练完成的情况下，基于第一图像处理模型生成第二图像处理模型。
在一种可能的实现方式中，在该第一图像处理模型训练完成的情况下，计算机设备基于该图像分割分支生成该第二图像处理模型。
其中，该第二图像处理模型用于对输入的目标血管图像进行处理，以输出该目标血管图像的血管分类信息，该血管分类信息用于指示该目标血管图像中的目标类型血管，该目标类型血管是该至少两种类型血管中的至少一种。
如图6所示的结构，在推理应用阶段，可以舍弃图像增强分支，只保留图像分割分支进行血管分类(比如A/V分类)，相对于完整运行第一图像处理模型来说，可以显著减少推理时间。
在一种可能的实现方式中，计算机设备可以将该目标血管图像输入该第二图像处理模型，获得该第二图像处理模型输出的血管位置信息，该血管位置信息至少指示该目标血管图像中被预测出的该目标类型血管的位置；并基于该血管位置信息，输出血管分类结果图像，该血管分类结果图像用于指示该目标血管图像中的该目标类型血管。
上述实施例中，除了基于预测增强图像与高质量血管图像样本之间的图像差异和视觉感知差异确定模型训练损失外，还基于预测增强图像与高质量血管图像样本各自对应掩模图像之间的差异确定模型训练损失，有助于提高模型训练质量，进而提高后续利用训练得到的模型进行血管分类的准确性。
以上述血管图像是眼底图像，血管位置信息是掩模图像为例，请参考图8，其是根据一示例性实施例示出的一种用于眼底图像处理模型的训练及应用框架图。如图8所示，用于眼底图像血管分类的模型训练及应用过程可以如下：
首先，开发人员准备高质量的眼底图像样本，通过退化模型对高质量的眼底图像样本81a进行处理，得到低质量的眼底图像样本81b，然后，基于眼底图像样本进行标注操作，得到表示各类型血管真实位置的标注掩模图像81c，然后，将高质量的眼底图像样本81a、低质量的眼底图像样本81b以及标注掩模图像81c作为训练数据，输入至模型训练设备中。
在模型训练阶段，模型训练设备将低质量的眼底图像样本81b输入第一图像处理模型82，通过第一图像处理模型82中的图像分割分支输出预测掩模图像83，并通过第一图像处理模型82中的图像增强分支输出预测增强图像83；其中图像分割分支和图像增强分支共享编码器；模型训练设备通过高质量的眼底图像样本81a、标注掩模图像81c、预测掩模图像83以及预测增强图像84，计算得到损失函数值85，通过损失函数值85中的血管分类损失函数值对图像分割分支进行参数更新，并通过损失函数值85中的图像增强损失函数值对图像增强分支进行参数更新；其中，由于图像分割分支和图像增强分支共享编码器，因此，在进行参数更新时，模型训练设备通过血管分类损失函数值对图像分割分支中的解码器进行更新，并通过图像增强损失函数值对图像增强分支中的解码器进行更新，并通过血管分类损失函数值和图像增强损失函数值共同对共享的编码器进行更新。
在模型训练结束后，开发人员通过第一图像处理模型82中的图像分割分支，可以构建出第二图像处理模型86，并将其部署到模型应用设备中。
在血管分类阶段，模型应用设备可以接收输入的目标眼底图像87，并将目标眼底图像87输入第二图像处理模型86，由第二图像处理模型86输出对应的掩模图像88，并基于掩模图像88输出血管分类后的眼底图像89。
眼底图像是一种无介入医疗图像。通过眼底图像，可以无侵入地观察血管。当视网膜动静脉的形态学发生改变时，可能会导致各种***性、心血管和脑部疾病的发生。另外，关于视网膜血管的生物标志物研究发现，较大的静脉口径与糖尿病视网膜病变的进展有关。此外，也有报道认为视网膜动脉口径的缩小与高血压和糖尿病的发生风险有关。因此，动脉/静脉分类为定量血管分析奠定基础，并且便于视网膜生物标志物研究。
在实际的临床实践中，捕获质量变化较大的眼底图像是非常普遍的，这取决于很多外界干扰因素，比如照明条件、眼底照相机、以及技术人员的水平参差等等，例如，眼底图像成像时容易产生伪影，从而导致低质量的眼底图像。据统计显示，在采集到的眼底图像中，有12％的图像达不到眼科医生在临床实践中阅读所需要的质量，当图像质量较低时，模型的A/V分类性能将受到极大影响。针对低质量图像的动静脉分割，本申请上述实施例所提供的端到端的双分支网络，可以包含增强分支和分割分支，从而在提高图像质量的同时，还能够提高A/V分类的性能，其中，两个分支可以共享同一个编码器，利用结构相似度的损失来减少重建图像的伪影，本申请还可以通过掩模损失来强调血管附近重建的增强图像的质量。通过上述双分支网络构建的血管分类模型，可以提供更准确的血管分类，进一步辅助医生***性的诊断心血管和脑部等疾病，以及在眼底筛查***中，辅助医生判断血管分布正常与否，从而帮助预防和诊断眼底的疾病以及全身性疾病，如高血压和糖尿病等。
本申请基于本申请上述实施例所示的方案进行消融研究，以评估图像增强分支中的各种损失函数(即重建损失、SSIM损失和掩模损失)在低质量图像下进行测试时的有效性。为了评估本申请上述方案的A/V分类性能，本申请采用精度(Accuracy，Acc)、灵敏度(Sensitivity，Sen)和似然性(Specificity，Spec)3个度量指标，基于真实的血管像素级图像对A/V分类性能进行评估，以比较同一准则下不同模块的有效性。
如表1所示，通过在图像分割分支中加入具有重建损失的图像增强分支作为辅助任务，A/V分类精度提高了2.15％，说明在网络中加入图像增强任务时，编码器能够对低质量图像提取更稳健的特征。除了重建损失，本申请还将SSIM损失和掩模损失分别添加到图像增强分支中。将SSIM损失集成到图像增强分支后，A/V分类精度提高了1.02％。此外，当采用掩模损失时，A/V分类的准确率提高了0.90％，表明了该掩模损失通过强调血管区域对A/V 分类任务的帮助。最后，在图像增强分支的作用下，图像分割分支对A/V分类性能最优，对低质量眼底图像的准确率达到91.52％。
表1
Figure PCTCN2022087404-appb-000004
在AV-DRIVE和INSPIRE-AVR数据集中，表2展示了本申请在低质量测试图像下的A/V分类性能与相关技术中的A/V分类技术的对比。本申请主要将上述实施例所示的方法与目前广泛采用的U型网络(U-Net)进行比较。本申请分别使用AV-DRIVE数据集的高质量(UNet-HighQuality，UNet-HQ)和低质量(UNet-LowQuality，UNet-LQ)眼底图像训练两个U-Net模型。首先，UNet-HQ模型的性能分别在高质量和低质量的测试图像上进行评估，如表2的第前两行所示，在使用低质量测试图像进行性能评估时，与高质量测试图像评估的性能相比，性能明显下降4.66％。因此，高质量和低质量的眼底图像之间存在着巨大的域差距，这也是本申请研究低质量A/V分类的主要原因。当采用低质量图像作为训练集时，UNet-LQ模型将A/V分类性能提高1.27％。然而，UNet-LQ与理论上限(91.61％，即UNet-HQ在高质量图像上测试的Acc)之间仍存在较大的性能差距。相比之下，在相同的低质量的测试图像下，本申请提供的双分支网络对AV-DRIVE数据集的A/V分类准确率为91.52％，比U-Net模型在低质量眼底图像上训练的准确率高3.3％。此外，本申请提供的双分支网络的性能与理论上限非常接近，性能差距仅为0.1％。
对于INSPIRE-AVR数据集，由于只提供了血管中心线上的A/V标记，不具备真实血管分割，因此对分割后的血管进行A/V分类性能评估。在相同的评价标准下进行比较并且没有微调模型时，本申请提出的框架超过了U-Net，精度提高了2.53％，表明了本申请涉及的方法具有泛化能力。
表2
Figure PCTCN2022087404-appb-000005
其中，本申请上述实施例所示的方案可以结合区块链来实现或者执行。比如，上述各个实施例中的部分或者全部步骤可以在区块链***执行；或者，上述各个实施例中的各个步骤执行所需要的数据或者生成的数据，可以存储在区块链***中；例如，上述模型训练使用的训练样本，以及模型应用过程中的目标血管图像等模型输入数据，可以由计算机设备从区块链***中获取；再例如，上述模型训练后得到的模型的参数(包括第一图像处理模型的参数和第二图像处理模型的参数)，可以存储在区块链***中。
图9是根据一示例性实施例示出的一种血管图像分类处理装置的结构方框图。该装置可以实现图2或图5所示实施例提供的方法中的全部或部分步骤，该血管图像分类处理装置包括：
样本获取模块901，用于获取第一血管图像样本、第二血管图像样本以及血管位置标注信息；所述第一血管图像样本是所述第二血管图像样本对应的低质量图像，所述血管位置标 注信息用于指示所述第一血管图像样本中被标注出的至少两种类型血管的位置；
预测模块902，用于将所述第一血管图像样本输入第一图像处理模型，获得所述第一图像处理模型输出的预测增强图像，以及预测血管位置信息；所述预测增强图像是对所述第一血管图像样本进行质量增强后的图像；所述预测血管位置信息用于指示所述第一血管图像样本中被预测出的所述至少两种类型血管的位置；
损失获取模块903，用于基于所述第二血管图像样本、所述血管位置标注信息、所述预测增强图像以及所述预测血管位置信息，获取损失函数值；
训练模块904，用于基于所述损失函数值对所述第一图像处理模型进行训练；
训练完成后的所述第一图像处理模型用于生成第二图像处理模型，所述第二图像处理模型用于对输入的目标血管图像进行处理，以输出所述目标血管图像的血管分类信息，所述血管分类信息用于指示所述目标血管图像中的目标类型血管，所述目标类型血管是所述至少两种类型血管中的至少一种。
在一种可能的实现方式中，所述预测模块902，包括：
输入子模块，用于将所述第一血管图像样本输入第一图像处理模型；
分割子模块，用于通过所述第一图像处理模型中的图像分割分支对所述第一血管图像样本进行处理，获得所述图像分割分支输出的所述预测血管位置信息；
增强子模块，用于通过所述第一图像处理模型中的图像增强分支对所述第一血管图像样本进行处理，获得所述图像增强分支输出的所述预测增强图像。
在一种可能的实现方式中，所述损失获取模块903，包括：
分类损失获取子模块，用于基于所述血管位置标注信息以及所述预测血管位置信息，获取所述损失函数值中的血管分类损失函数值；
增强损失获取子模块，用于基于所述第二血管图像样本以及所述预测增强图像，获取所述损失函数值中的图像增强损失函数值。
在一种可能的实现方式中，所述增强损失获取子模块，用于，
基于所述第二血管图像样本以及所述预测增强图像，获取子损失函数值；所述子损失函数值包括第一子损失函数值、第二子损失函数值、以及第三子损失函数值中的至少一种；
基于所述子损失函数值获取所述图像增强损失函数值；
所述第一子损失函数值用于指示所述第二血管图像样本与所述预测增强图像之间的整体差异；
所述第二子损失函数值用于指示所述第二血管图像样本与所述预测增强图像之间的视觉感知差异；
所述第三子损失函数值用于指示所述第二血管图像样本与所述预测增强图像各自对应的血管部位的图像差异。
在一种可能的实现方式中，在所述子损失函数值包括所述第三子损失函数值的情况下，所述增强损失获取子模块，用于，
基于所述第二血管图像样本以及所述血管位置标注信息获取第一局部图像，所述第一局部图像是所述第二血管图像样本中的血管部位的图像；
基于所述预测增强图像以及所述预测血管位置信息获取第二局部图像，所述第二局部图像是所述预测增强图像中的血管部位的图像；
基于所述第一局部图像和所述第二局部图像，获取所述第三子损失函数值。
在一种可能的实现方式中，增强损失获取子模块，用于，
在所述子损失函数值包括第一子损失函数值、第二子损失函数值、以及第三子损失函数值中的至少两项数值的情况下，对所述子损失函数值中的至少两项数值进行加权处理，获得所述图像增强损失函数值。
在一种可能的实现方式中，所述图像分割分支和所述图像增强分支共享编码器，所述图像分割分支还包含第一解码器，所述图像增强分支还包含第二解码器；
所述训练模块904，包括：
编码器更新子模块，用于基于所述血管分类损失函数值以及所述图像增强损失函数值，对所述编码器进行参数更新；
第一解码器更新子模块，用于基于所述血管分类损失函数值对所述第一解码器进行参数更新；
第二解码器更新子模块，用于基于所述图像增强损失函数值对所述第二解码器进行参数更新。
在一种可能的实现方式中，所述编码器更新子模块，用于，
对所述血管分类损失函数值以及所述图像增强损失函数值进行加权处理，获得总损失函数值；
基于所述总损失函数值对所述编码器进行参数更新。
在一种可能的实现方式中，所述装置还包括：
模型生成模块，用于响应于所述第一图像处理模型训练完成，基于所述图像分割分支生成所述第二图像处理模型。
在一种可能的实现方式中，所述装置还包括：
图像输入模块，用于将所述目标血管图像输入所述第二图像处理模型，获得所述第二图像处理模型输出的血管位置信息，所述血管位置信息至少指示所述目标血管图像中被预测出的所述目标类型血管的位置；
结果输出模块，用于基于所述血管位置信息，输出血管分类结果图像，所述血管分类结果图像用于指示所述目标血管图像中的所述目标类型血管。
图10是根据一示例性实施例示出的一种血管图像分类处理装置的结构方框图。该血管图像分类处理装置可以实现图3或图5所示实施例提供的方法中的全部或部分步骤，该对话信息处理装置包括：
图像获取模块1001，用于获取目标血管图像；
模型处理模块1002，用于将所述目标血管图像输入第二图像处理模型，获得所述第二图像处理模型输出的血管位置信息，所述血管位置信息至少指示所述目标血管图像中被预测出的目标类型血管的位置；
输出模块1003，用于基于所述血管位置信息，输出血管分类结果图像，所述血管分类结果图像用于指示所述目标血管图像中的所述目标类型血管；
其中，所述第二图像处理模型是基于训练完成的第一图像处理模型生成的；训练所述第一图像处理模型的损失函数值是基于第二血管图像样本、血管位置标注信息、预测增强图像以及预测血管位置信息获取的；所述预测增强图像以及所述预测血管位置信息是所述第一图像处理模型对第一血管图像样本进行处理后输出的；所述第一血管图像样本是所述第二血管图像样本对应的低质量图像，所述血管位置标注信息用于指示所述第一血管图像样本中被标注出的至少两种类型血管的位置；所述预测增强图像是对所述第一血管图像样本进行质量增强后的图像；所述预测血管位置信息用于指示所述第一血管图像样本中被预测出的至少两种类型血管的位置。
图11是根据一示例性实施例示出的一种计算机设备的结构示意图。该计算机设备可以实现为上述各个方法实施例中用于训练第一图像处理模型的计算机设备，或者，可以实现为上述各个方法实施例中用于通过第二图像处理模型进行血管分类的计算机设备。所述计算机设备1100包括中央处理单元(CPU，Central Processing Unit)1101、包括随机存取存储器(Random Access Memory，RAM)1102和只读存储器(Read-Only Memory，ROM)1103的***存储器1104，以及连接***存储器1104和中央处理单元1101的***总线1105。所述计算机设备1100还包括帮助计算机内的各个器件之间传输信息的基本输入/输出控制器1106，和用于存储操作***1113、应用程序1114和其他程序模块1115的大容量存储设备 1107。
所述大容量存储设备1107通过连接到***总线1105的大容量存储控制器(未示出)连接到中央处理单元1101。所述大容量存储设备1107及其相关联的计算机可读介质为计算机设备1100提供非易失性存储。也就是说，所述大容量存储设备1107可以包括诸如硬盘或者光盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)驱动器之类的计算机可读介质(未示出)。
不失一般性，所述计算机可读介质可以包括计算机存储介质和通信介质。计算机存储介质包括以用于存储诸如计算机可读指令、数据结构、程序模块或其他数据等信息的任何方法或技术实现的易失性和非易失性、可移动和不可移动介质。计算机存储介质包括RAM、ROM、闪存或其他固态存储其技术，CD-ROM、或其他光学存储、磁带盒、磁带、磁盘存储或其他磁性存储设备。当然，本领域技术人员可知所述计算机存储介质不局限于上述几种。上述的***存储器1104和大容量存储设备1107可以统称为存储器。
计算机设备1100可以通过连接在所述***总线1105上的网络接口单元1111连接到互联网或者其它网络设备。
所述存储器还包括一个或者一个以上的程序，所述一个或者一个以上程序存储于存储器中，中央处理器1101通过执行该一个或一个以上程序来实现图2、图3或图5任一所示的方法的全部或者部分步骤。
在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例如包括计算机程序(指令)的存储器，上述程序(指令)可由计算机设备的处理器执行以完成本申请各个实施例所示的方法。例如，所述非临时性计算机可读存储介质可以是只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、只读光盘(Compact Disc Read-Only Memory，CD-ROM)、磁带、软盘和光数据存储设备等。
在示例性实施例中，还提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上述各个实施例所示的方法。
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本申请的范围仅由所附的权利要求来限制。

Claims (20)

  1. 一种血管图像分类处理方法，所述方法由计算机设备执行，所述方法包括：
    获取第一血管图像样本、第二血管图像样本以及血管位置标注信息；所述第一血管图像样本是所述第二血管图像样本对应的低质量图像，所述血管位置标注信息用于指示所述第一血管图像样本中被标注出的至少两种类型血管的位置；
    将所述第一血管图像样本输入第一图像处理模型，获得所述第一图像处理模型输出的预测增强图像，以及预测血管位置信息；所述预测增强图像是对所述第一血管图像样本进行质量增强后的图像；所述预测血管位置信息用于指示所述第一血管图像样本中被预测出的所述至少两种类型血管的位置；
    基于所述第二血管图像样本、所述血管位置标注信息、所述预测增强图像以及所述预测血管位置信息，获取损失函数值；
    基于所述损失函数值对所述第一图像处理模型进行训练；训练完成后的所述第一图像处理模型用于生成第二图像处理模型，所述第二图像处理模型用于对输入的目标血管图像进行处理，以输出所述目标血管图像的血管分类信息，所述血管分类信息用于指示所述目标血管图像中的目标类型血管，所述目标类型血管是所述至少两种类型血管中的至少一种。
  2. 根据权利要求1所述的方法，其中，所述将所述第一血管图像样本输入第一图像处理模型，获得所述第一图像处理模型输出的预测增强图像，以及预测血管位置信息，包括：
    将所述第一血管图像样本输入第一图像处理模型；
    通过所述第一图像处理模型中的图像分割分支对所述第一血管图像样本进行处理，获得所述图像分割分支输出的所述预测血管位置信息；
    通过所述第一图像处理模型中的图像增强分支对所述第一血管图像样本进行处理，获得所述图像增强分支输出的所述预测增强图像。
  3. 根据权利要求2所述的方法，其中，所述基于所述第二血管图像样本、所述血管位置标注信息、所述预测增强图像以及所述预测血管位置信息，获取损失函数值，包括：
    基于所述血管位置标注信息以及所述预测血管位置信息，获取所述损失函数值中的血管分类损失函数值；
    基于所述第二血管图像样本以及所述预测增强图像，获取所述损失函数值中的图像增强损失函数值。
  4. 根据权利要求3所述的方法，其中，所述基于所述第二血管图像样本以及所述预测增强图像，获取所述损失函数值中的图像增强损失函数值，包括：
    基于所述第二血管图像样本以及所述预测增强图像，获取子损失函数值；所述子损失函数值包括第一子损失函数值、第二子损失函数值、以及第三子损失函数值中的至少一种；
    基于所述子损失函数值获取所述图像增强损失函数值；
    所述第一子损失函数值用于指示所述第二血管图像样本与所述预测增强图像之间的整体差异；
    所述第二子损失函数值用于指示所述第二血管图像样本与所述预测增强图像之间的视觉感知差异；
    所述第三子损失函数值用于指示所述第二血管图像样本与所述预测增强图像各自对应的血管部位的图像差异。
  5. 根据权利要求4所述的方法，其中，在所述子损失函数值包括所述第三子损失函数值的情况下，所述基于所述第二血管图像样本以及所述预测增强图像，获取子损失函数值，包括：
    基于所述第二血管图像样本以及所述血管位置标注信息获取第一局部图像，所述第一局部图像是所述第二血管图像样本中的血管部位的图像；
    基于所述预测增强图像以及所述预测血管位置信息获取第二局部图像，所述第二局部图像是所述预测增强图像中的血管部位的图像；
    基于所述第一局部图像和所述第二局部图像，获取所述第三子损失函数值。
  6. 根据权利要求4所述的方法，其中，在所述子损失函数值包括第一子损失函数值、第二子损失函数值、以及第三子损失函数值中的至少两项数值的情况下，所述基于所述子损失函数值获取所述图像增强损失函数值，包括：
    对所述子损失函数值中的至少两项数值进行加权处理，获得所述图像增强损失函数值。
  7. 根据权利要求3所述的方法，其中，所述图像分割分支和所述图像增强分支共享编码器，所述图像分割分支还包含第一解码器，所述图像增强分支还包含第二解码器；
    所述基于所述损失函数值对所述第一图像处理模型进行训练，包括：
    基于所述血管分类损失函数值以及所述图像增强损失函数值，对所述编码器进行参数更新；
    基于所述血管分类损失函数值对所述第一解码器进行参数更新；
    基于所述图像增强损失函数值对所述第二解码器进行参数更新。
  8. 根据权利要求7所述的方法，其中，所述基于所述血管分类损失函数值以及所述图像增强损失函数值，对所述编码器进行参数更新，包括：
    对所述血管分类损失函数值以及所述图像增强损失函数值进行加权处理，获得总损失函数值；
    基于所述总损失函数值对所述编码器进行参数更新。
  9. 根据权利要求2所述的方法，其中，所述方法还包括：
    在所述第一图像处理模型训练完成的情况下，基于所述图像分割分支生成所述第二图像处理模型。
  10. 根据权利要求9所述的方法，其特征在于，所述方法还包括：
    将所述目标血管图像输入所述第二图像处理模型，获得所述第二图像处理模型输出的血管位置信息，所述血管位置信息至少指示所述目标血管图像中被预测出的所述目标类型血管的位置；
    基于所述血管位置信息，输出血管分类结果图像，所述血管分类结果图像用于指示所述目标血管图像中的所述目标类型血管。
  11. 一种血管图像分类处理方法，所述方法由计算机设备执行，所述方法包括：
    获取目标血管图像；
    将所述目标血管图像输入第二图像处理模型，获得所述第二图像处理模型输出的血管位置信息，所述血管位置信息至少指示所述目标血管图像中被预测出的目标类型血管的位置；
    基于所述血管位置信息，输出血管分类结果图像，所述血管分类结果图像用于指示所述目标血管图像中的所述目标类型血管；
    其中，所述第二图像处理模型是基于训练完成的第一图像处理模型生成的；训练所述第一图像处理模型的损失函数值是基于第二血管图像样本、血管位置标注信息、预测增强图像以及预测血管位置信息获取的；所述预测增强图像以及所述预测血管位置信息是所述第一图像处理模型对第一血管图像样本进行处理后输出的；所述第一血管图像样本是所述第二血管 图像样本对应的低质量图像，所述血管位置标注信息用于指示所述第一血管图像样本中被标注出的至少两种类型血管的位置；所述预测增强图像是对所述第一血管图像样本进行质量增强后的图像；所述预测血管位置信息用于指示所述第一血管图像样本中被预测出的至少两种类型血管的位置。
  12. 一种血管图像分类处理装置，所述装置包括：
    样本获取模块，用于获取第一血管图像样本、第二血管图像样本以及血管位置标注信息；所述第一血管图像样本是所述第二血管图像样本对应的低质量图像，所述血管位置标注信息用于指示所述第一血管图像样本中被标注出的至少两种类型血管的位置；
    预测模块，用于将所述第一血管图像样本输入第一图像处理模型，获得所述第一图像处理模型输出的预测增强图像，以及预测血管位置信息；所述预测增强图像是对所述第一血管图像样本进行质量增强后的图像；所述预测血管位置信息用于指示所述第一血管图像样本中被预测出的所述至少两种类型血管的位置；
    损失获取模块，用于基于所述第二血管图像样本、所述血管位置标注信息、所述预测增强图像以及所述预测血管位置信息，获取损失函数值；
    训练模块，用于基于所述损失函数值对所述第一图像处理模型进行训练；
    训练完成后的所述第一图像处理模型用于生成第二图像处理模型，所述第二图像处理模型用于对输入的目标血管图像进行处理，以输出所述目标血管图像的血管分类信息，所述血管分类信息用于指示所述目标血管图像中的目标类型血管，所述目标类型血管是所述至少两种类型血管中的至少一种。
  13. 根据权利要求12所述的装置，其中，所述预测模块，包括：
    输入子模块，用于将所述第一血管图像样本输入第一图像处理模型；
    分割子模块，用于通过所述第一图像处理模型中的图像分割分支对所述第一血管图像样本进行处理，获得所述图像分割分支输出的所述预测血管位置信息；
    增强子模块，用于通过所述第一图像处理模型中的图像增强分支对所述第一血管图像样本进行处理，获得所述图像增强分支输出的所述预测增强图像。
  14. 根据权利要求13所述的装置，其中，所述损失获取模块，包括：
    分类损失获取子模块，用于基于所述血管位置标注信息以及所述预测血管位置信息，获取所述损失函数值中的血管分类损失函数值；
    增强损失获取子模块，用于基于所述第二血管图像样本以及所述预测增强图像，获取所述损失函数值中的图像增强损失函数值。
  15. 根据权利要求14所述的装置，其中，所述增强损失获取子模块，用于：
    基于所述第二血管图像样本以及所述预测增强图像，获取子损失函数值；所述子损失函数值包括第一子损失函数值、第二子损失函数值、以及第三子损失函数值中的至少一种；
    基于所述子损失函数值获取所述图像增强损失函数值；
    所述第一子损失函数值用于指示所述第二血管图像样本与所述预测增强图像之间的整体差异；
    所述第二子损失函数值用于指示所述第二血管图像样本与所述预测增强图像之间的视觉感知差异；
    所述第三子损失函数值用于指示所述第二血管图像样本与所述预测增强图像各自对应的血管部位的图像差异。
  16. 根据权利要求15所述的装置，其中，在所述子损失函数值包括所述第三子损失函数 值的情况下，所述增强损失获取子模块，用于：
    基于所述第二血管图像样本以及所述血管位置标注信息获取第一局部图像，所述第一局部图像是所述第二血管图像样本中的血管部位的图像；
    基于所述预测增强图像以及所述预测血管位置信息获取第二局部图像，所述第二局部图像是所述预测增强图像中的血管部位的图像；
    基于所述第一局部图像和所述第二局部图像，获取所述第三子损失函数值。
  17. 一种血管图像分类处理装置，所述装置包括：
    图像获取模块，用于获取目标血管图像；
    模型处理模块，用于将所述目标血管图像输入第二图像处理模型，获得所述第二图像处理模型输出的血管位置信息，所述血管位置信息至少指示所述目标血管图像中被预测出的目标类型血管的位置；
    输出模块，用于基于所述血管位置信息，输出血管分类结果图像，所述血管分类结果图像用于指示所述目标血管图像中的所述目标类型血管；
    其中，所述第二图像处理模型是基于训练完成的第一图像处理模型生成的；训练所述第一图像处理模型的损失函数值是基于第二血管图像样本、血管位置标注信息、预测增强图像以及预测血管位置信息获取的；所述预测增强图像以及所述预测血管位置信息是所述第一图像处理模型对第一血管图像样本进行处理后输出的；所述第一血管图像样本是所述第二血管图像样本对应的低质量图像，所述血管位置标注信息用于指示所述第一血管图像样本中被标注出的至少两种类型血管的位置；所述预测增强图像是对所述第一血管图像样本进行质量增强后的图像；所述预测血管位置信息用于指示所述第一血管图像样本中被预测出的至少两种类型血管的位置。
  18. 一种计算机设备，所述计算机设备包含处理器和存储器，所述存储器中存储有至少一条计算机指令，所述至少一条计算机指令由所述处理器加载并执行以实现如权利要求1至11任一所述的血管图像分类处理方法。
  19. 一种计算机可读存储介质，所述存储介质中存储有至少一条计算机指令，所述至少一条计算机指令由处理器加载并执行以实现如权利要求1至11任一所述的血管图像分类处理方法。
  20. 一种计算机程序产品，所述计算机程序产品包括计算机指令，所述计算机指令存储在计算机可读存储介质中；计算机设备的处理器从所述计算机可读存储介质读取所述计算机指令，所述处理器执行所述计算机指令，使得所述计算机设备执行如权利要求1至11任一所述的血管图像分类处理方法。
PCT/CN2022/087404 2021-05-19 2022-04-18 血管图像分类处理方法、装置、设备及存储介质 WO2022242392A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP22803717.2A EP4273747A4 (en) 2021-05-19 2022-04-18 BLOOD VESSELS IMAGE CLASSIFICATION PROCESSING METHOD AND APPARATUS, DEVICE, AND STORAGE MEDIUM
US17/994,678 US20230106222A1 (en) 2021-05-19 2022-11-28 Vessel image classification method and apparatus, device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110547148.7A CN112966792B (zh) 2021-05-19 2021-05-19 血管图像分类处理方法、装置、设备及存储介质
CN202110547148.7 2021-05-19

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/994,678 Continuation US20230106222A1 (en) 2021-05-19 2022-11-28 Vessel image classification method and apparatus, device, and storage medium

Publications (1)

Publication Number Publication Date
WO2022242392A1 true WO2022242392A1 (zh) 2022-11-24

Family

ID=76275643

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/087404 WO2022242392A1 (zh) 2021-05-19 2022-04-18 血管图像分类处理方法、装置、设备及存储介质

Country Status (4)

Country Link
US (1) US20230106222A1 (zh)
EP (1) EP4273747A4 (zh)
CN (1) CN112966792B (zh)
WO (1) WO2022242392A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112966792B (zh) * 2021-05-19 2021-08-13 腾讯科技（深圳）有限公司 血管图像分类处理方法、装置、设备及存储介质
CN114359284B (zh) * 2022-03-18 2022-06-21 北京鹰瞳科技发展股份有限公司 用于对视网膜眼底图像进行分析的方法及其相关产品
CN116229480B (zh) * 2023-01-10 2024-05-28 北京百度网讯科技有限公司 文本识别模型训练方法、文本识别方法、装置及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110211140A (zh) * 2019-06-14 2019-09-06 重庆大学 基于3D残差U-Net和加权损失函数的腹部血管分割方法
CN111242865A (zh) * 2020-01-10 2020-06-05 南京航空航天大学 基于生成式对抗网络的眼底图像增强方法
CN111461134A (zh) * 2020-05-18 2020-07-28 南京大学 一种基于生成对抗网络的低分辨率车牌识别方法
CN111932535A (zh) * 2020-09-24 2020-11-13 北京百度网讯科技有限公司 用于处理图像的方法、装置、设备以及存储介质
CN112966792A (zh) * 2021-05-19 2021-06-15 腾讯科技（深圳）有限公司 血管图像分类处理方法、装置、设备及存储介质

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110458833B (zh) * 2019-08-15 2023-07-11 腾讯科技（深圳）有限公司 基于人工智能的医学图像处理方法、医学设备和存储介质
CN110569899B (zh) * 2019-09-03 2022-06-10 嘉陵江亭子口水利水电开发有限公司 坝面缺陷分类模型训练方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110211140A (zh) * 2019-06-14 2019-09-06 重庆大学 基于3D残差U-Net和加权损失函数的腹部血管分割方法
CN111242865A (zh) * 2020-01-10 2020-06-05 南京航空航天大学 基于生成式对抗网络的眼底图像增强方法
CN111461134A (zh) * 2020-05-18 2020-07-28 南京大学 一种基于生成对抗网络的低分辨率车牌识别方法
CN111932535A (zh) * 2020-09-24 2020-11-13 北京百度网讯科技有限公司 用于处理图像的方法、装置、设备以及存储介质
CN112966792A (zh) * 2021-05-19 2021-06-15 腾讯科技（深圳）有限公司 血管图像分类处理方法、装置、设备及存储介质

Also Published As

Publication number Publication date
EP4273747A1 (en) 2023-11-08
US20230106222A1 (en) 2023-04-06
CN112966792A (zh) 2021-06-15
CN112966792B (zh) 2021-08-13
EP4273747A4 (en) 2024-07-31

Similar Documents

Publication Publication Date Title
CN110909780B (zh) 一种图像识别模型训练和图像识别方法、装置及***
WO2022242392A1 (zh) 血管图像分类处理方法、装置、设备及存储介质
US11468564B2 (en) Systems and methods for automatic detection and quantification of pathology using dynamic feature classification
US11069062B2 (en) Automated screening of histopathology tissue samples via analysis of a normal model
CN108230296B (zh) 图像特征的识别方法和装置、存储介质、电子装置
CN110689025B (zh) 图像识别方法、装置、***及内窥镜图像识别方法、装置
CN107729929B (zh) 用于获取信息的方法及装置
WO2021093448A1 (zh) 图像处理方法、装置、服务器、医疗图像处理设备及存储介质
Kurmann et al. Expert-level automated biomarker identification in optical coherence tomography scans
WO2022007337A1 (zh) 肿瘤细胞含量评估方法、***、计算机设备及存储介质
US11967181B2 (en) Method and device for retinal image recognition, electronic equipment, and storage medium
WO2021174821A1 (zh) 眼底彩照图像分级方法、装置、计算机设备及存储介质
WO2022089257A1 (zh) 医学图像处理方法、装置、设备、存储介质及产品
Papadopoulos et al. An interpretable multiple-instance approach for the detection of referable diabetic retinopathy in fundus images
US11721023B1 (en) Distinguishing a disease state from a non-disease state in an image
CN110473176B (zh) 图像处理方法及装置、眼底图像处理方法、电子设备
CN113724185B (zh) 用于图像分类的模型处理方法、装置及存储介质
US20220130047A1 (en) Diagnostic imaging for diabetic retinopathy
US20240108276A1 (en) Systems and Methods for Identifying Progression of Hypoxic-Ischemic Brain Injury
Khalid et al. FGR-Net: interpretable fundus image gradeability classification based on deep reconstruction learning
CN113554641B (zh) 一种儿科咽部图像采集方法及装置
CN115526888A (zh) 眼图数据识别方法、装置、存储介质及电子设备
WO2022252107A1 (zh) 一种基于眼部图像的疾病检测***及方法
Sridhar et al. Artificial intelligence in medicine: diabetes as a model
CN112966620A (zh) 眼底图像处理方法、模型训练方法及设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22803717

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022803717

Country of ref document: EP

Effective date: 20230731

NENP Non-entry into the national phase

Ref country code: DE