WO2023115797A1

WO2023115797A1 - Image segmentation method and apparatus, and device and storage medium

Info

Publication number: WO2023115797A1
Application number: PCT/CN2022/093365
Authority: WO
Inventors: 王娜; 刘星龙; 陈翼男
Original assignee: 上海商汤智能科技有限公司
Priority date: 2021-12-24
Filing date: 2022-05-17
Publication date: 2023-06-29
Also published as: CN114332120A

Abstract

Disclosed in the present application are an image segmentation method and apparatus, and a device and a storage medium. The image segmentation method comprises: acquiring an image to be segmented; performing first segmentation on the image to be segmented, so as to obtain an initial segmentation result, relating to a target, of the image to be segmented; extracting, on the basis of the initial segmentation result and from the image to be segmented, a local image containing the target; and performing second segmentation on the local image, so as to obtain a final segmentation result, relating to the target, of the image to be segmented. By means of the solution, the accuracy of segmentation can be improved.

Description

图像分割方法、装置、设备及存储介质Image segmentation method, device, equipment and storage medium

本申请要求2021年12月24日提交、申请号为202111599227.9，发明名称为“图像分割方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application filed on December 24, 2021 with application number 202111599227.9 and titled "Image Segmentation Method, Device, Equipment, and Storage Medium", the entire contents of which are incorporated in this application by reference.

技术领域technical field

本申请涉及图像处理技术领域，特别是涉及一种图像分割方法、装置、设备及存储介质、计算机程序产品。The present application relates to the technical field of image processing, in particular to an image segmentation method, device, equipment, storage medium, and computer program product.

背景技术Background technique

随着社会老龄化和环境的影响，肺癌已成为我国发病率和死亡率最高的恶性肿瘤。对于早期肺癌，外科手术切除是常用的治疗方式。外科手术切除是常用的治疗方式。传统的手术规划缺少全方位、三维透视的肺部解剖结构图的指导，存在一定的视觉局限。经过科技的不断进步，现如今已经可以根据病人的CT图像，对肺部进行三维建模，以便进行手术规划。其中，进行三维建模需要十分精确的肺部区域及相邻器官之间的分割结果。With the aging of the society and the impact of the environment, lung cancer has become the malignant tumor with the highest morbidity and mortality in my country. Surgical resection is the usual treatment for early-stage lung cancer. Surgical resection is the usual treatment. Traditional surgical planning lacks the guidance of an all-round, three-dimensional perspective lung anatomical structure map, and there are certain visual limitations. With the continuous advancement of technology, it is now possible to perform three-dimensional modeling of the lungs based on the patient's CT images for surgical planning. Among them, three-dimensional modeling requires very accurate segmentation results between lung regions and adjacent organs.

但目前，一般的分割方式都是将病人的CT图像进行分割结果，用于三维建模。经发明人的不断研究，发现这种方式获取得到的分割结果并不准确。一般而言，CT图像所涵盖的器官可能较多，例如包括了头部、肩部，并且可能还包括胸腔部分的器官，导致肺部区域在CT图像并不明显，导致对CT图像进行分割得到的结果也是十分不准确。But at present, the general segmentation method is to segment the patient's CT image for 3D modeling. After continuous research by the inventor, it is found that the segmentation result obtained in this way is not accurate. Generally speaking, CT images may cover more organs, such as the head, shoulders, and possibly parts of the thoracic cavity. As a result, the lung area is not obvious in the CT image, resulting in the segmentation of the CT image to obtain The results are also very inaccurate.

发明内容Contents of the invention

本申请至少提供一种图像分割方法、装置、设备及存储介质。The present application provides at least one image segmentation method, device, equipment and storage medium.

本申请提供了一种图像分割方法，包括：获取待分割图像；对待分割图像进行第一分割，得到待分割图像关于目标的初始分割结果；基于初始分割结果，从待分割图像中提取包含目标的局部图像；对局部图像进行第二分割，得到待分割图像关于目标的最终分割结果。The present application provides an image segmentation method, comprising: acquiring an image to be segmented; performing a first segmentation on the image to be segmented to obtain an initial segmentation result of the image to be segmented with respect to a target; A partial image: performing a second segmentation on the partial image to obtain a final segmentation result of the image to be segmented with respect to the target.

因此，对待分割图像进行第一分割得到初始分割结果之后，基于该初始分割结果对待分割图像再次进行第二分割，得到待分割图像关于目标的最终分割结果，相比于只进行第一分割的方式而言，利用两次分割得到的最终分割结果更准确。另外，通过基于初始分割结果，从待分割图像中获取包含目标的局部图像，然后对局部图像进行第二分割，相对于直接对待分割图像进行第二分割而言，前者相对于后者而言数据更小，且目标在局部图像中所在比例相对更大，使得在提高分割速度的同时，能够提高分割结果的准确度。Therefore, after the first segmentation of the image to be segmented to obtain the initial segmentation result, the second segmentation is performed on the image to be segmented based on the initial segmentation result to obtain the final segmentation result of the image to be segmented about the target. Compared with the method of only performing the first segmentation In terms of segmentation, the final segmentation result obtained by using two segmentations is more accurate. In addition, by obtaining a partial image containing the target from the image to be segmented based on the initial segmentation result, and then performing a second segmentation on the partial image, compared to directly performing the second segmentation on the image to be segmented, the data of the former relative to the latter Smaller, and the proportion of the target in the local image is relatively larger, so that the accuracy of the segmentation result can be improved while increasing the segmentation speed.

其中，初始分割结果包括待分割图像中各位置属于目标的初始概率，基于初始分割结果，从待分割图像中提取包含目标的局部图像，包括：基于待分割图像中各位置属于目标的初始概率，确定目标在待分割图像中的初始位置；利用目标的初始位置对待分割图像进行裁剪，得到包含目标的局部图像。Wherein, the initial segmentation result includes the initial probability that each position in the image to be segmented belongs to the target, based on the initial segmentation result, extracting a partial image containing the target from the image to be segmented, including: based on the initial probability that each position in the image to be segmented belongs to the target, Determine the initial position of the target in the image to be segmented; use the initial position of the target to crop the image to be segmented to obtain a partial image containing the target.

因此，通过基于各位置属于目标的初始概率，即可确定目标在待分割图像中的初始位置。Therefore, based on the initial probability that each position belongs to the target, the initial position of the target in the image to be segmented can be determined.

其中，第一分割是利用第一分割网络实现的，方法还包括以下步骤，以对第一分割网络进行训练：利用第一分割网络对第一样本数据进行第一分割，得到第一分割结果；获取第一分割结果对应的第一损失，并基于第一损失调整第一分割网络的网络参数；和/或，第二分割是利用第二分割网络实现的；方法还包括以下步骤，以对第二分割网络进行训练：利用第二分割网络对第二样本数据进行第二分割，得到第二分割结果；获取第二分割结果对应的第二损失，并基于第二损失调整第二分割网络的网络参数。Wherein, the first segmentation is realized by using the first segmentation network, and the method further includes the following steps to train the first segmentation network: performing the first segmentation on the first sample data by using the first segmentation network to obtain the first segmentation result ; Obtain the first loss corresponding to the first segmentation result, and adjust the network parameters of the first segmentation network based on the first loss; and/or, the second segmentation is realized by using the second segmentation network; the method also includes the following steps to Training with the second segmentation network: using the second segmentation network to perform a second segmentation on the second sample data to obtain a second segmentation result; obtaining a second loss corresponding to the second segmentation result, and adjusting the second segmentation network based on the second loss Network parameters.

因此，通过使用分割网络执行分割的步骤，使得整个分割过程方便快捷。另外，通过分别对第一分割网络和第二分割网络分别进行训练，能够提高各网络输出结果的精度。Therefore, by using the segmentation network to perform the segmentation step, the entire segmentation process is made convenient and fast. In addition, by separately training the first segmentation network and the second segmentation network, the accuracy of the output results of each network can be improved.

其中，获取第一分割结果对应的第一损失或获取第二分割结果对应的第二损失，包括：将第一损失作为目标损失以及将第一分割结果作为目标分割结果，或将第二损失作为目标损失以及将第二分割结果作为目标分割结果；基于目标分割结果，获得第一类损失以及第二类损失；对第一类损失和第二类损失进行加权处理，得到目标损失。Wherein, obtaining the first loss corresponding to the first segmentation result or obtaining the second loss corresponding to the second segmentation result includes: taking the first loss as the target loss and the first segmentation result as the target segmentation result, or using the second loss as The target loss and the second segmentation result are used as the target segmentation result; based on the target segmentation result, the first type loss and the second type loss are obtained; the first type loss and the second type loss are weighted to obtain the target loss.

因此，通过使用两类损失，确定第一损失和第二损失，能够提高最终确定得到的损失的准确度。Therefore, by using two types of losses to determine the first loss and the second loss, the accuracy of the finally determined loss can be improved.

其中，第一类损失为指数对数损失，和/或，第二类损失为加权指数交叉熵损失。Wherein, the first type of loss is an exponential logarithmic loss, and/or, the second type of loss is a weighted exponential cross-entropy loss.

因此，通过使用上述两类损失，确定第一损失和第二损失，能够提高最终确定得到的损失的准确度。Therefore, by using the above two types of losses to determine the first loss and the second loss, the accuracy of the final determined loss can be improved.

其中，第二类损失为加权指数交叉熵损失，目标分割结果包括对应样本数据中的不同位置属于目标的目标概率；基于目标分割结果，获得第二类损失，包括：基于各位置对应的目标概率，得到各位置属于至少一种预设类别的类别概率，其中，至少一种预设类别包括位置属于目标和位置不属于目标中的至少一种；对各预设类别的类别概率进行对数处理，得到各预设类别对应的对数处理结果；对于各预设类别，利用预设类别对应的频率统计参数与对数处理结果，得到预设类别的类别损失，其中，频率统计参数基于属于各预设类别的位置数量确定的；利用各预设类别的类别损失，得到加权指数交叉熵损失。Among them, the second type of loss is weighted exponential cross-entropy loss, and the target segmentation result includes the target probability that different positions in the corresponding sample data belong to the target; based on the target segmentation result, the second type of loss is obtained, including: based on the target probability corresponding to each position , to obtain the category probability that each location belongs to at least one preset category, wherein at least one preset category includes at least one of the location belonging to the target and the location not belonging to the target; logarithmic processing is performed on the category probabilities of each preset category , to obtain the logarithmic processing results corresponding to each preset category; for each preset category, use the frequency statistical parameters corresponding to the preset category and the logarithmic processing results to obtain the category loss of the preset category, where the frequency statistical parameters are based on the The number of positions of the preset categories is determined; using the category loss of each preset category, a weighted exponential cross-entropy loss is obtained.

因此，结合各个预设类别的类别损失，确定加权指数交叉熵，相对于仅通过单一预设类别确定损失而言，前者确定得到的损失更准确。Therefore, combining the category losses of each preset category to determine the weighted exponential cross entropy is more accurate than determining the loss only through a single preset category.

其中，预设类别的频率统计参数为所有预设类别的频率之和与预设类别的频率之间的比值，预设类别的频率为属于预设类别的位置数量与位置总数量之间的比值；和/或，利用预设类别对应的频率统计参数与对数处理结果，得到预设类别的类别损失，包括：将预设类别对应的频率统计参数与对数处理结果的乘积，作为预设类别的类别损失。Among them, the frequency statistical parameter of the preset category is the ratio between the sum of the frequencies of all preset categories and the frequency of the preset category, and the frequency of the preset category is the ratio between the number of locations belonging to the preset category and the total number of locations and/or, using the frequency statistical parameters corresponding to the preset category and the logarithmic processing result to obtain the category loss of the preset category, including: taking the product of the frequency statistical parameter corresponding to the preset category and the logarithmic processing result as the preset Class-of-class loss.

因此，通过确定预设类别的频率与所有预设类别的频率之和之间的比值，确定预设类别的类别损失，能够减少因为预设类别与其他预设类别之间的比例失调问题，从而提高损失的准确度。Therefore, by determining the ratio between the frequency of the preset category and the sum of the frequencies of all preset categories, determining the category loss of the preset category can reduce the problem of imbalance between the preset category and other preset categories, thereby Improve the accuracy of the loss.

其中，在利用第一分割网络对第一样本数据进行第一分割，得到第一分割结果，以及利用第二分割网络对第二样本数据进行第二分割，得到第二分割结果之前，方法还包括：对第一样本数据和第二样本数据进行数据增强，数据增强的方式包括添加随机高斯噪声、进行随机弹性变化以及添加随机脉冲中的一种或多种。Wherein, before using the first segmentation network to perform the first segmentation on the first sample data to obtain the first segmentation result, and using the second segmentation network to perform the second segmentation on the second sample data to obtain the second segmentation result, the method further It includes: performing data enhancement on the first sample data and the second sample data, and the way of data enhancement includes one or more of adding random Gaussian noise, performing random elastic change, and adding random pulses.

因此，通过对样本数据进行数据增强，能够提高分割网络的泛化能力，进而提高分割网络输出结果的准确度。Therefore, by performing data enhancement on the sample data, the generalization ability of the segmentation network can be improved, and the accuracy of the output results of the segmentation network can be improved.

其中，待分割图像为三维医学图像，目标包括主动脉；和/或，获取待分割图像，包括：从原始医学图像中获取包含主动脉以及至少一个与主动脉相邻器官的局部医学图像；统一局部医学图像在各维度方向上的分辨率，以得到待分割图像。Wherein, the image to be segmented is a three-dimensional medical image, and the target includes the aorta; and/or, acquiring the image to be segmented includes: acquiring a partial medical image including the aorta and at least one organ adjacent to the aorta from the original medical image; The resolution of the local medical image in each dimension to obtain the image to be segmented.

因此，通过从原始医学图像中获取包括目标的局部医学图像，能够减少分割网络的数据量，从而提高分割网络的分割效率，并且增强了目标在待分割图像中的影响力，进而提高了分割网络输出结果的准确度。另外，通过统一局部医学图像在各个维度方向上的分辨率，能够提高分割网络的准确度。Therefore, by obtaining the local medical image including the target from the original medical image, the data volume of the segmentation network can be reduced, thereby improving the segmentation efficiency of the segmentation network, and enhancing the influence of the target in the image to be segmented, thereby improving the segmentation network. The accuracy of the output results. In addition, by unifying the resolution of local medical images in all dimensions, the accuracy of the segmentation network can be improved.

本申请提供了一种图像分割装置，包括：第一获取模块，用于获取待分割图像；初始分割模块，用于对待分割图像进行第一分割，得到待分割图像关于目标的初始分割结果；第二获取模块，用于基于初始分割结果，从待分割图像中提取包含目标的局部图像；最终分割模块，用于对局部图像进行第二分割，得到待分割图像关于目标的最终分割结果。The present application provides an image segmentation device, including: a first acquisition module, used to acquire an image to be segmented; an initial segmentation module, used to perform a first segmentation on the image to be segmented, and obtain an initial segmentation result of the image to be segmented with respect to a target; The second acquisition module is used to extract a partial image containing the target from the image to be segmented based on the initial segmentation result; the final segmentation module is used to perform a second segmentation on the partial image to obtain a final segmentation result of the target image to be segmented.

本申请提供了一种电子设备，包括存储器和处理器，处理器用于执行存储器中存储的程序指令，以实现上述图像分割方法。The present application provides an electronic device, including a memory and a processor, and the processor is used to execute program instructions stored in the memory, so as to realize the above image segmentation method.

本申请提供了一种计算机可读存储介质，其上存储有程序指令，程序指令被处理器执行时实现上述图像分割方法。The present application provides a computer-readable storage medium, on which program instructions are stored, and the above image segmentation method is implemented when the program instructions are executed by a processor.

本申请提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当所述计算机可读代码在电子设备的处理器中运行时，所述电子设备中的处理器执行用于实现上述方法。The present application provides a computer program product, including computer readable codes, or a non-volatile computer readable storage medium bearing computer readable codes, when the computer readable codes are run in a processor of an electronic device , the processor in the electronic device executes to implement the above method.

上述方案，对待分割图像进行第一分割得到初始分割结果之后，基于该初始分割结果对待分割图像再次进行第二分割，得到待分割图像关于目标的最终分割结果，相比于只进行第一分割的方式而言，利用两次分割得到的最终分割结果更准确。另外，通过基于初始分割结果，从待分割图像中获取包含目标的局部图像，然后对局部图像进行第二分割，相对于直接对待分割图像进行第二分割而言，前者相对于后者而言数据更小，且目标在局部图像中所在比例相对更大，使得在提高分割速度的同时，能够提高分割结果的准确度。In the above solution, after the first segmentation of the image to be segmented is performed to obtain the initial segmentation result, the image to be segmented is then secondly segmented based on the initial segmentation result to obtain the final segmentation result of the image to be segmented about the target. In terms of methods, the final segmentation result obtained by using two segmentations is more accurate. In addition, by obtaining a partial image containing the target from the image to be segmented based on the initial segmentation result, and then performing a second segmentation on the partial image, compared to directly performing the second segmentation on the image to be segmented, the former is more data-intensive than the latter Smaller, and the proportion of the target in the local image is relatively larger, so that the accuracy of the segmentation result can be improved while increasing the segmentation speed.

应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本申请。It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

附图说明Description of drawings

此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本申请的实施例，并与说明书一起用于说明本申请的技术方案。The accompanying drawings here are incorporated into the specification and constitute a part of the specification. These drawings show embodiments consistent with the application, and are used together with the description to describe the technical solution of the application.

图1是本申请图像分割方法一实施例的流程示意图；Fig. 1 is a schematic flow chart of an embodiment of the image segmentation method of the present application;

图2是本申请图像分割方法一实施例示出获取目标损失的流程示意图；Fig. 2 is a schematic flow diagram illustrating the process of obtaining target loss according to an embodiment of the image segmentation method of the present application;

图3是本申请图像分割方法一实施例的另一流程示意图；Fig. 3 is another schematic flow chart of an embodiment of the image segmentation method of the present application;

图4是本申请图像分割装置一实施例的结构示意图；4 is a schematic structural diagram of an embodiment of an image segmentation device of the present application;

图5是本申请电子设备一实施例的结构示意图；FIG. 5 is a schematic structural diagram of an embodiment of the electronic device of the present application;

图6是本申请计算机可读存储介质一实施例的结构示意图。Fig. 6 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application.

具体实施方式Detailed ways

下面结合说明书附图，对本申请实施例的方案进行详细说明。The solutions of the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

以下描述中，为了说明而不是为了限定，提出了诸如特定***结构、接口、技术之类的具体细节，以便透彻理解本申请。In the following description, for purposes of illustration rather than limitation, specific details, such as specific system architectures, interfaces, and techniques, are set forth in order to provide a thorough understanding of the present application.

本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。此外，本文中的“多”表示两个或者多于两个。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship. In addition, "many" herein means two or more than two. In addition, the term "at least one" herein means any one of a variety or any combination of at least two of the more, for example, including at least one of A, B, and C, which may mean including from A, Any one or more elements selected from the set formed by B and C.

本申请提供一些图像分割方法以及装置。该图像分割方法可以应用在医学检测上。例如，应用在肺癌诊断中，待分割图像可以是为医学设备拍摄的生物内部图像，目标可以是生物内部任意需要检测的对象，例如主动脉、肿瘤等。This application provides some image segmentation methods and devices. The image segmentation method can be applied to medical detection. For example, in the diagnosis of lung cancer, the image to be segmented may be an internal biological image taken by medical equipment, and the target may be any object that needs to be detected inside the biological body, such as aorta, tumor, etc.

本申请的方法步骤的执行主体可以为硬件执行，或者通过处理器运行计算机可执行代码的方式执行。The subject of execution of the method steps of the present application may be executed by hardware, or executed by a processor running computer executable codes.

请参阅图1，图1是本申请图像分割方法一实施例的流程示意图。具体而言，图像分割方法可以包括如下步骤：Please refer to FIG. 1 . FIG. 1 is a schematic flowchart of an embodiment of an image segmentation method of the present application. Specifically, the image segmentation method may include the following steps:

步骤S11：获取待分割图像。Step S11: Acquire the image to be segmented.

其中，待分割图像可以二维图像，也可以是三维图像。待分割图像的类别也可以有多种，例如，待分割图像可以是安防侦查图像、普通摄像图像、医学图像等等。Wherein, the image to be segmented may be a two-dimensional image or a three-dimensional image. There may also be various types of images to be segmented, for example, images to be segmented may be security surveillance images, general camera images, medical images, and so on.

获取待分割图像的方式可以是由执行图像分割方法的执行设备自身携带的摄像组件拍摄得到，还可以是其他设备通过各种通信方式传输至该执行设备。其他设备指的是不与该执行设备共享同一处理器的设备。The way to acquire the image to be segmented may be captured by the camera component carried by the execution device that executes the image segmentation method, or it may be transmitted to the execution device by other devices through various communication methods. Other devices refer to devices that do not share the same processor as the executing device.

步骤S12：对待分割图像进行第一分割，得到待分割图像关于目标的初始分割结果。Step S12: Carry out the first segmentation of the image to be segmented to obtain the initial segmentation result of the image to be segmented with respect to the target.

对待分割图像进行第一分割的方式可以是使用分割网络进行分割，还可以是基于目标的灰度值等图像属性，对待分割图像进行分割。例如，若目标本身的灰度值属性为至少一区域内相邻若干个位置的灰度值大于预设灰度值。因此，在检测到待分割图像中存在相邻若干个位置的灰度值均大于预设灰度值时，基于该检测结果，对待分割图像进行第一分割，得到待分割图像的初始分割结果。The method of first segmenting the image to be segmented may be to segment the image by using a segmentation network, or to segment the image to be segmented based on image attributes such as the gray value of the target. For example, if the grayscale attribute of the target itself is that the grayscale values of several adjacent positions in at least one area are greater than the preset grayscale value. Therefore, when it is detected that the grayscale values of several adjacent positions in the image to be segmented are greater than the preset grayscale value, based on the detection result, the image to be segmented is first segmented to obtain the initial segmentation result of the image to be segmented.

步骤S13：基于初始分割结果，从待分割图像中提取包含目标的局部图像。Step S13: Based on the initial segmentation result, extract a partial image containing the target from the image to be segmented.

其中，从待分割图像提取包含目标的局部图像的方式可以是以目标为中心，对预设尺寸内的图像进行裁剪，得到局部图像。具体地，以目标的中心点为中心，预设尺寸可根据目标的边界与目标的中心点之间的最大距离进行动态调整。例如，目标的中心点与上边界之间的距离为a，中心点与下边界之间的距离为b，中心点与左边界之间的距离为c，中心点与右边界之间的距离为d，且a>b>c>d，因此，可根据目标的中心点与上边界之间的距离确定预设尺寸的大小。示例性地，预设尺寸可以是距离a的预设数量倍，且预设数量大于或等于1。Wherein, the method of extracting the partial image including the target from the image to be segmented may be to crop the image within a preset size with the target as the center to obtain the partial image. Specifically, with the center point of the object as the center, the preset size can be dynamically adjusted according to the maximum distance between the boundary of the object and the center point of the object. For example, the distance between the center point of the target and the upper boundary is a, the distance between the center point and the lower boundary is b, the distance between the center point and the left boundary is c, and the distance between the center point and the right boundary is d, and a>b>c>d, therefore, the size of the preset size can be determined according to the distance between the center point of the target and the upper boundary. Exemplarily, the preset size may be a preset number times the distance a, and the preset number is greater than or equal to 1.

步骤S14：对局部图像进行第二分割，得到待分割图像关于目标的最终分割结果。Step S14: Perform a second segmentation on the partial image to obtain a final segmentation result of the image to be segmented with respect to the target.

对局部图像进行第二分割的方式同样可以是使用分割网络进行分割。第二分割能够结合初始分割结果，对局部图像进行分割，得到的分割结果更准确，并且能够提高分割速度。The method of performing the second segmentation on the partial image may also be to use a segmentation network to perform segmentation. The second segmentation can combine the initial segmentation result to segment the local image, the obtained segmentation result is more accurate, and the segmentation speed can be improved.

一些公开实施例中，待分割图像为三维医学图像。例如，待分割图像可以是人体三维CT图像。目标可以是主动脉。其中，获取待分割图像的方式可以是：In some disclosed embodiments, the image to be segmented is a three-dimensional medical image. For example, the image to be segmented may be a three-dimensional CT image of a human body. The target can be the aorta. Among them, the way to obtain the image to be segmented can be:

从原始医学图像中获取包含主动脉以及至少一个与主动脉相邻器官的局部医学图像。具体地，若原始医学图像包括头肩整个区域的三维医学图像与主动脉相邻的器官可以是肺。其中，从原始医学图像中获取包含主动脉以及至少一个与主动脉相邻器官的局部医学图像的方式可以是根据原始医学图像中各位置的灰度值进行确定，一般而言，CT影像中肺部很容易分辨，直接通过灰度值等图像属性，即可从原始医学图像中确定肺部区域。然后对原始医学图像进行裁剪，得到包含主动脉以及肺部区域的局部医学图像。A partial medical image including the aorta and at least one organ adjacent to the aorta is acquired from the original medical image. Specifically, if the original medical image includes a three-dimensional medical image of the entire region of the head and shoulders, the organ adjacent to the aorta may be the lung. Wherein, the manner of obtaining the local medical image including the aorta and at least one organ adjacent to the aorta from the original medical image can be determined according to the gray value of each position in the original medical image. Generally speaking, the lung in the CT image The lung area is easy to distinguish, and the lung area can be determined from the original medical image directly through image attributes such as gray value. Then the original medical image is cropped to obtain a partial medical image including the aorta and lung area.

统一局部医学图像在各个维度方向上的分辨率，以得到待分割图像。其中，统一局部医学图像在各个维度方向上的分辨率的方式可以是以其中一个维度上的分辨率为准，调整另外维度上的分辨率。Unify the resolutions of local medical images in all dimensions to obtain images to be segmented. Wherein, the method of unifying the resolutions of the local medical images in various dimensions may be based on the resolution in one dimension, and adjust the resolution in the other dimension.

通过从原始医学图像中获取包括目标的局部医学图像，能够减少分割网络的数据量，从而提高分割网络的分割效率，并且增强了目标在待分割图像中的影响力，进而提高了分割网络输出结果的准确度。另外，通过统一局部医学图像在各个维度方向上的分辨率，能够提高分割网络的准确度。By obtaining the local medical image including the target from the original medical image, the data volume of the segmentation network can be reduced, thereby improving the segmentation efficiency of the segmentation network, and enhancing the influence of the target in the image to be segmented, thereby improving the output result of the segmentation network the accuracy. In addition, by unifying the resolution of local medical images in all dimensions, the accuracy of the segmentation network can be improved.

可选地，初始分割结果中包括待分割图像中各位置属于目标的初始概率。其中，在待分割图像为二维图像的情况下，待分割图像中的各位置为待分割图像中的各个像素位置，在待分割图像为三维图像的情况下，待分割图像中的各位置为待分割图像中的各个体素位置。本公开实施例所述的其他图像中的各位置同理。Optionally, the initial segmentation result includes an initial probability that each position in the image to be segmented belongs to the target. Wherein, when the image to be segmented is a two-dimensional image, each position in the image to be segmented is each pixel position in the image to be segmented, and when the image to be segmented is a three-dimensional image, each position in the image to be segmented is Each voxel position in the image to be segmented. The same applies to the positions in other images described in the embodiments of the present disclosure.

一些公开实施例中，上述步骤S13可以包括以下步骤：In some disclosed embodiments, the above step S13 may include the following steps:

基于待分割图像中各位置属于目标的初始概率，确定目标在待分割图像中的初始位置。具体地，判断各位置属于目标的初始概率是否大于或等于预设概率，若是，则认为该位置属于目标，否则该位置不属于目标。预设概率可以根据经验进行设置，可选地，预设概率可以大于或等于0.4，例如，预设概率可以是0.5、0.7、0.8等。通过基于各位置属于目标的初始概率，即可确定目标在待分割图像中的初始位置。Based on the initial probability that each position in the image to be segmented belongs to the target, the initial position of the target in the image to be segmented is determined. Specifically, it is judged whether the initial probability of each position belonging to the target is greater than or equal to the preset probability, and if so, the position is considered to belong to the target, otherwise, the position does not belong to the target. The preset probability can be set according to experience. Optionally, the preset probability can be greater than or equal to 0.4, for example, the preset probability can be 0.5, 0.7, 0.8, and so on. Based on the initial probability that each position belongs to the target, the initial position of the target in the image to be segmented can be determined.

利用目标的初始位置对待分割图像进行裁剪，得到包含目标的局部图像。可选地，获取待分割图像中包含目标的初始位置的区域，并将该区域向外扩充，得到扩充后的区域。其中，扩充的方式可以是将在该区域外扩充预设数量个位置，得到扩充后的区域，还可以是扩充包含目标初始位置的区域大小的预设倍数，其中，该预设倍数小于1。在其他实施例中，预设倍数可根据目标的大小进行确定，预设倍数也可以是大于或等于1，关于预设倍数，此处不做具体规定。对待分割图像进行裁剪，得到裁剪后的局部图像，其中，该局部图像中包括该扩充后的区域。The image to be segmented is cropped using the initial position of the target to obtain a partial image containing the target. Optionally, the region containing the initial position of the target in the image to be segmented is obtained, and the region is expanded outward to obtain the expanded region. Wherein, the way of expansion may be to expand a preset number of positions outside the area to obtain the expanded area, or to expand a preset multiple of the size of the area including the target initial position, wherein the preset multiple is less than 1. In other embodiments, the preset multiple can be determined according to the size of the target, and the preset multiple can also be greater than or equal to 1. There is no specific regulation on the preset multiple here. The image to be segmented is cropped to obtain a cropped partial image, wherein the partial image includes the expanded region.

一些公开实施例中，上述步骤S14可以包括以下步骤：In some disclosed embodiments, the above step S14 may include the following steps:

其中，对局部图像进行第二分割的方式同样可以是使用分割网络进行分割，还可以是基于目标的灰度值等图像属性，对待分割图像进行分割。其中，基于目标的灰度值等图像属性进行分割的方式如上述，此处不再赘述。Wherein, the method of performing the second segmentation on the partial image may also be segmenting by using a segmentation network, or segmenting the image to be segmented based on image attributes such as the gray value of the target. Wherein, the manner of performing segmentation based on the image attributes such as the gray value of the target is as described above, and will not be repeated here.

然后，基于局部图像的分割结果，得到待分割图像的最终分割结果。Then, based on the segmentation result of the partial image, the final segmentation result of the image to be segmented is obtained.

其中，局部图像的分割结果中包括局部图像中各位置属于目标的概率。而局部图像是由待分割图像进行裁剪得到。因此，可以基于局部图像和待分割图像之间的位置关系，确定待分割图像中与局部图像中各位置对应的位置，以此，确定待分割图像中各个位置属于目标的概率。其中，因为局部图像只对应待分割图像中的部分位置，因此，未与局部图像对应的其他位置最终属于目标的概率仍为各位置对应的初始概率。例如，待分割图像中A位置同时处于局部图像中，A位置属于目标的初始概率为0.6，而A位置在局部图像的分割结果中属于目标的概率为0.8，则待分割图像关于目标的最终分割结果中，A位置属于目标的概率为0.8。Wherein, the segmentation result of the partial image includes the probability that each position in the partial image belongs to the target. The partial image is obtained by cropping the image to be segmented. Therefore, based on the positional relationship between the partial image and the image to be segmented, the position in the image to be segmented corresponding to each position in the partial image can be determined, so as to determine the probability that each position in the image to be segmented belongs to the target. Wherein, because the partial image only corresponds to some positions in the image to be segmented, the probability that other positions not corresponding to the partial image finally belong to the target is still the initial probability corresponding to each position. For example, position A in the image to be segmented is in the partial image at the same time, the initial probability that position A belongs to the target is 0.6, and the probability that position A belongs to the target in the segmentation result of the partial image is 0.8, then the final segmentation of the image to be segmented with respect to the target In the result, the probability that position A belongs to the target is 0.8.

通过基于初始分割结果，从待分割图像中获取包含目标的局部图像，然后对局部图像进行第二分割，相对于直接对待分割图像进行第二分割而言，前者相对于后者而言数据更小，且目标在局部图像中所在比例较大，使得在提高分割速度的同时，能够提高分割结果的准确度。By obtaining the partial image containing the target from the image to be segmented based on the initial segmentation result, and then performing the second segmentation on the partial image, the data of the former is smaller than the latter compared with the second segmentation of the image to be segmented directly , and the proportion of the target in the local image is relatively large, so that the accuracy of the segmentation result can be improved while increasing the segmentation speed.

一些公开实施例中，第一分割是利用第一分割网络实现的。图像分割方法还包括对第一分割网络的训练步骤：In some disclosed embodiments, the first segmentation is achieved using a first segmentation network. The image segmentation method also includes a training step for the first segmentation network:

利用第一分割网络对第一样本数据进行第一分割，得到第一分割结果。通过使用第一分割网络执行分割的步骤，使得整个第一分割过程方便快捷。A first segmentation is performed on the first sample data by using the first segmentation network to obtain a first segmentation result. By using the first segmentation network to perform the segmentation step, the entire first segmentation process is convenient and fast.

其中，第一分割网络包括第一数量的下采样层以及与各下采样层对应的上采样层。其中，下采样层和上采样层跳跃连接。也就是，每个跳跃连接都是将第一分割网络中下采样层与其相对应的上采样层相融合。第一样本数据在输入第一分割网络之后，先经过第一数量的下采样层再经过对应的上采样层，然后经过预设大小的卷积层以及激活层，得到待分割图像的初始分割结果。本公开实施例中，第一样本数据若是三维的图像，则卷积层的卷积核大小可以是3*3*3。Wherein, the first segmentation network includes a first number of down-sampling layers and up-sampling layers corresponding to each down-sampling layer. Among them, the down-sampling layer and the up-sampling layer are skip-connected. That is, each skip connection fuses the downsampling layer in the first segmentation network with its corresponding upsampling layer. After the first sample data is input into the first segmentation network, it first passes through the first number of downsampling layers, then passes through the corresponding upsampling layer, and then passes through the preset size convolution layer and activation layer to obtain the initial segmentation of the image to be segmented. result. In the embodiment of the present disclosure, if the first sample data is a three-dimensional image, the size of the convolution kernel of the convolution layer may be 3*3*3.

可选地，第一分割网络的第一分割结果除了包括第一样本数据中各位置属于目标的初始概率外，还可包括第一样本数据中各位置属于非目标的初始概率。若将目标称之为前景，则除目标以外的位置则称之为背景。即初始分割结果中，包括各位置属于前景的初始概率以及属于背景的初始概率。其中，各位置属于前景的初始概率和属于背景的初始概率之和为1。例如，若A位置属于前景的初始概率为0.3，则A位置属于背景的初始概率则为0.7。Optionally, the first segmentation result of the first segmentation network may include not only the initial probability that each position in the first sample data belongs to the target, but also the initial probability that each position in the first sample data belongs to a non-target. If the target is called the foreground, the position other than the target is called the background. That is, the initial segmentation results include the initial probability that each position belongs to the foreground and the initial probability that it belongs to the background. Among them, the sum of the initial probability of each location belonging to the foreground and the initial probability belonging to the background is 1. For example, if the initial probability that position A belongs to the foreground is 0.3, then the initial probability that position A belongs to the background is 0.7.

一些应用场景中，输入第一分割网络的第一样本数据为三维的单通道灰度图像，第一分割网络的输出则是两通道的三维概率分布结果。其中一个通道为第一样本数据中各体素属于目标的概率分布结果，另一通道为第一样本数据中各体素属于背景的概率分布结果，其中，这里的背景指的是非目标类别。In some application scenarios, the first sample data input to the first segmentation network is a three-dimensional single-channel grayscale image, and the output of the first segmentation network is a two-channel three-dimensional probability distribution result. One of the channels is the probability distribution result of each voxel in the first sample data belonging to the target, and the other channel is the probability distribution result of each voxel in the first sample data belonging to the background, where the background here refers to the non-target category .

获取第一分割结果对应的第一损失，并基于第一损失调整第一分割网络的网络参数。A first loss corresponding to the first segmentation result is obtained, and network parameters of the first segmentation network are adjusted based on the first loss.

其中，获取第一损失的方式可以有多种，例如，获取各位置的样本标记与第一分割结果之间的差异，根据该差异确定第一损失。本公开实施例中，具体获取第一损失的方式请参见下述，此处不过过多叙述。There may be multiple ways of obtaining the first loss, for example, obtaining the difference between the sample label at each position and the first segmentation result, and determining the first loss according to the difference. In the embodiment of the present disclosure, please refer to the following for the specific manner of obtaining the first loss, which is not described here too much.

一些公开实施例中，第二分割是利用第二分割网络实现的。其中，第二分割网络的网络结构可参见第一分割网络的网络结构，第二分割网络的网络结构同样可以包括第二数量的下采样层以及与各下采样层对应的上采样层。其中，下采样层和上采样层跳跃连接。也就是，每个跳跃连接都是将第二分割网络中下采样层与其相对应的上采样层相融合。第二样本数据在输入第一分割网络之后，先经过第二数量的下采样层再经过对应的上采样层，然后经过预设大小的卷积层以及激活层，得到第二样本数据的第二分割结果。本公开实施例中，第二样本数据若是三维的图像，则卷积层的卷积核大小可以是3*3*3。其中，第一数量和第二数量可以相同，也可以不同。In some disclosed embodiments, the second segmentation is achieved using a second segmentation network. Wherein, the network structure of the second segmentation network may refer to the network structure of the first segmentation network, and the network structure of the second segmentation network may also include a second number of downsampling layers and upsampling layers corresponding to each downsampling layer. Among them, the down-sampling layer and the up-sampling layer are skip-connected. That is, each skip connection fuses the downsampling layer in the second segmentation network with its corresponding upsampling layer. After the second sample data is input into the first segmentation network, it first passes through the second number of down-sampling layers, then passes through the corresponding up-sampling layer, and then passes through the convolution layer and activation layer of the preset size to obtain the second part of the second sample data. Split results. In the embodiment of the present disclosure, if the second sample data is a three-dimensional image, the size of the convolution kernel of the convolution layer may be 3*3*3. Wherein, the first quantity and the second quantity may be the same or different.

一些公开实施例中，第二分割网络的第二分割结果除了包括第二样本数据中各位置属于目标的初始概率外，还可包括第二样本数据中各位置属于非目标的初始概率。若将目标称之为前景，则除目标以外的位置则称之为背景。即初始分割结果中，包括各位置属于前景的初始概率以及属于背景的初始概率。其中，各位置属于前景的初始概率和属于背景的初始概率之和为1。例如，若A位置属于前景的初始概率为0.3，则A位置属于背景的初始概率则为0.7。In some disclosed embodiments, the second segmentation result of the second segmentation network may include not only the initial probability that each position in the second sample data belongs to the target, but also the initial probability that each position in the second sample data belongs to a non-target. If the target is called the foreground, the position other than the target is called the background. That is, the initial segmentation results include the initial probability that each position belongs to the foreground and the initial probability that it belongs to the background. Among them, the sum of the initial probability of each location belonging to the foreground and the initial probability belonging to the background is 1. For example, if the initial probability that position A belongs to the foreground is 0.3, then the initial probability that position A belongs to the background is 0.7.

一些应用场景中，输入第二分割网络的第二样本数据为三维的单通道灰度图像，第二分割网络的输出则是两通道的三维概率分布结果。其中一个通道为第二样本数据中各体素属于目标的概率分布结果，另一通道为第二样本数据中各体素属于背景的概率分布结果，其中，这里的背景指的是非目标类别。In some application scenarios, the second sample data input to the second segmentation network is a three-dimensional single-channel grayscale image, and the output of the second segmentation network is a two-channel three-dimensional probability distribution result. One of the channels is the probability distribution result of each voxel in the second sample data belonging to the target, and the other channel is the probability distribution result of each voxel in the second sample data belonging to the background, wherein the background here refers to a non-target category.

然后，获取第二分割结果对应的第二损失，并基于第二损失调整第二分割网络的网络参数。Then, a second loss corresponding to the second segmentation result is obtained, and network parameters of the second segmentation network are adjusted based on the second loss.

其中，获取第一损失或获取第二损失的方式可以是：Among them, the way to obtain the first loss or the second loss can be:

将第一损失作为目标损失以及将第一分割结果作为目标分割结果，或将第二损失作为目标损失以及将第二分割结果作为目标分割结果。请同时参见图2，图2是本申请图像分割方法一实施例示出获取目标损失的流程示意图。The first loss is used as the target loss and the first segmentation result is used as the target segmentation result, or the second loss is used as the target loss and the second segmentation result is used as the target segmentation result. Please refer to FIG. 2 at the same time. FIG. 2 is a schematic flow chart showing an embodiment of an image segmentation method of the present application for obtaining a target loss.

步骤S21：基于目标分割结果，获取第一类损失以及第二类损失。Step S21: Obtain the first type loss and the second type loss based on the target segmentation result.

其中，第一类损失为指数对数损失L _DSC。第二类损失为加权指数交叉熵损失L _Cross。其中，目标分割结果中包括对应样本数据中的不同位置属于目标的目标概率。 Among them, the first type of loss is the exponential logarithmic loss L _DSC . The second type of loss is the weighted exponential cross-entropy loss L _Cross . Wherein, the target segmentation result includes target probabilities that different locations in the corresponding sample data belong to the target.

其中，指数对数损失L _DSC的获取方式可以是： Among them, the acquisition method of exponential logarithmic loss L _DSC can be:

其中，x表示第一样本数据中的一个位置，在第一样本数据为三维数据时x表示一个体素，第一样本数据为二维数据时x表示一个像素，c表示对应的类别(包括目标和非目标)，其中，c为1时表示目标类别，c等于0时表示非目标类别。p _c(x)表示位置x属于c类的概率，g _c(x)表示该位置对应的真实标签，例如，若该位置对应的真实标签为1，则表示该位置属于目标，若该位置对应的真实标签为0，则表示该位置属于非目标。E[·]表示对c求平均值。ε为超参数，用于降低分母为0的概率。γ _DSC为超参数，用于控制L _DSC的范围，降低其过大或过小的概率。∑为求和符号。也就是说上述公式(2)，用于确定目标分割结果对应的DSC ₁和DSC ₀，然后再将DSC ₁和DSC ₀带入公式(1)中对c进行求平均值。 Among them, x represents a position in the first sample data, x represents a voxel when the first sample data is three-dimensional data, x represents a pixel when the first sample data is two-dimensional data, and c represents the corresponding category (including target and non-target), where c is 1 for the target category, and c equal to 0 for the non-target category. p _c (x) represents the probability that position x belongs to class c, g _c (x) represents the real label corresponding to the position, for example, if the real label corresponding to the position is 1, it means that the position belongs to the target, if the position corresponds to The ground truth label of is 0, which means that the position belongs to non-target. E[·] means to calculate the average value of c. ε is a hyperparameter used to reduce the probability of the denominator being 0. γ _DSC is a hyperparameter, which is used to control the range of L _DSC and reduce the probability that it is too large or too small. ∑ is the summation symbol. That is to say, the above formula (2) is used to determine the DSC ₁ and DSC ₀ corresponding to the target segmentation result, and then bring the DSC ₁ and DSC ₀ into the formula (1) to calculate the average value of c.

其中，获取第二类损失的方式可以是：Among them, the way to obtain the second type of loss can be:

基于各位置对应的目标概率，得到各位置属于至少一种预设类别的类别概率。其中，至少一种预设类别包括位置属于目标和位置不属于目标中的至少一种。若预设类别只有两个，则可以直接将各位置的目标概率作为其属于目标类别的类别概率，以及将1减去该目标概率的值作为其不属于目标的类别概率。Based on the target probability corresponding to each position, the class probability that each position belongs to at least one preset class is obtained. Wherein, at least one preset category includes at least one of the position belonging to the target and the position not belonging to the target. If there are only two preset categories, the target probability of each position can be directly used as the category probability of the target category, and the value of 1 minus the target probability can be used as the category probability of the non-target category.

对各预设类别的类别概率进行对数处理，得到各预设类别对应的对数处理结果。然后，对于各预设类别，利用预设类别对应的频率统计参数与对数处理结果，得到预设类别的类别损失。其中，频率统计参数基于属于各预设类别的位置数量确定的。其中，频率统计参数为所有预设类别的频率之和与预设类别的频率之间的比值。预设类别的频率为属于预设类别的位置数量与位置总数量之间的比值。其中，将预设类别对应的频率统计参数与对数处理结果的乘积，作为预设类别的类别损失。通过确定预设类别的频率与所有预设类别的频率之和之间的比值，确定预设类别的类别损失，能够减少因为预设类别与其他预设类别之间的比例失调问题，从而提高损失的准确度。Logarithmic processing is performed on the category probabilities of each preset category to obtain a logarithmic processing result corresponding to each preset category. Then, for each preset category, the category loss of the preset category is obtained by using the frequency statistics parameters corresponding to the preset category and the logarithmic processing results. Wherein, the frequency statistical parameter is determined based on the number of locations belonging to each preset category. Wherein, the frequency statistical parameter is a ratio between the sum of the frequencies of all preset categories and the frequency of the preset categories. The frequency of a preset category is the ratio between the number of locations belonging to the preset category and the total number of locations. Among them, the product of the frequency statistical parameter corresponding to the preset category and the logarithmic processing result is used as the category loss of the preset category. By determining the ratio between the frequency of the preset category and the sum of the frequencies of all preset categories, the category loss of the preset category can be determined, which can reduce the problem of proportional imbalance between the preset category and other preset categories, thereby improving the loss the accuracy.

然后利用各预设类别的类别损失，得到加权指数交叉熵损失。结合各个预设类别的类别损失，确定加权指数交叉熵，相对于仅通过单一预设类别确定损失而言，前者确定得到的损失更准确。Then using the class loss for each preset class, a weighted exponential cross-entropy loss is obtained. Combining the category losses of each preset category to determine the weighted exponential cross entropy, compared to determining the loss only through a single preset category, the loss determined by the former is more accurate.

具体地，获取加权指数交叉熵损失L _Cross的公式可以是： Specifically, the formula for obtaining the weighted exponential cross-entropy loss L _Cross can be:

其中，x表示第一样本数据中的一个位置，在第一样本数据为三维数据时x表示一个体素，第一样本数据为二维数据时x表示一个像素，c表示对应的类别(包括目标和非目标)，c取1时，得到各位置属于目标的类别损失，c取值为0时，得到各位置不属于目标的类别损失。p _c(x)表示位置x属于c类的概率，也就是上述的类别概率。γ _Cross为超参数，用于控制L _Cross的范围，降低其过大或过小的概率。其中，然后再对两种类别损失求平均，得到最终的加权指数交叉熵损失。w _c表示频率统计参数。 Among them, x represents a position in the first sample data, x represents a voxel when the first sample data is three-dimensional data, x represents a pixel when the first sample data is two-dimensional data, and c represents the corresponding category (including target and non-target), when c is 1, the category loss of each position belonging to the target is obtained, and when c is 0, the category loss of each position not belonging to the target is obtained. p _c (x) represents the probability that position x belongs to class c, which is the above-mentioned class probability. γ _Cross is a hyperparameter, which is used to control the range of L _Cross and reduce the probability of it being too large or too small. Among them, the two category losses are then averaged to obtain the final weighted exponential cross-entropy loss. w _c represents the frequency statistics parameter.

其中，

为对数处理结果。当c等于1时，得到该位置属于预设类别的类别损失，具体损失值为

通过将各位置属于预设类别的类别损失进行求平均，得到目标分割结果对应的属于目标的类别损失。同理，当c等于0时，获取目标分割结果对应的不属于目标的类别损失。然后将目标分割结果对应的属于目标的类别损失以及不属于目标的类别损失进行求平均，得到最终的加权指数交叉熵损失。 in,

Treat the result logarithmically. When c is equal to 1, the category loss of the position belonging to the preset category is obtained, and the specific loss value is

By averaging the category losses of each location belonging to the preset category, the category loss corresponding to the target segmentation result is obtained. Similarly, when c is equal to 0, the category loss corresponding to the target segmentation result that does not belong to the target is obtained. Then, the category loss corresponding to the target segmentation result and the category loss not belonging to the target are averaged to obtain the final weighted exponential cross-entropy loss.

具体地，获取频率统计参数的公式可以是：Specifically, the formula for obtaining frequency statistics parameters may be:

w _c＝((∑ _kf _k)/f _c) ^0.5 (4)； w _c ＝((∑ _k f _k )/f _c ) ^0.5 (4);

其中，f _k表示该位置属于k类别的频率，其中，k包括位置属于目标和位置不属于目标。∑为求和符号。f _c表示位置属于c类别的频率。本公开实施例中，各预设类别的频率之和为1。继上例，若c为1，属于目标的频率为0.6，所有预设类别的频率之和为1，则频率统计参数则为所有预设类别的频率之和与属于目标的频率之间的比值，该比值的具体数值为(1/0.6) ^0.5。 where f _k represents the frequency that the position belongs to k categories, where k includes the position belongs to the target and the position does not belong to the target. ∑ is the summation symbol. _fc represents the frequency with which the location belongs to category c. In the embodiment of the present disclosure, the sum of frequencies of each preset category is 1. Following the above example, if c is 1, the frequency belonging to the target is 0.6, and the sum of the frequencies of all preset categories is 1, then the frequency statistical parameter is the ratio between the sum of the frequencies of all preset categories and the frequency belonging to the target , the specific value of the ratio is (1/0.6) ^0.5 .

步骤S22：对第一类损失和第二类损失进行加权处理，得到目标损失。Step S22: Perform weighting processing on the first type loss and the second type loss to obtain the target loss.

通过使用两类损失，确定第一损失和第二损失，能够提高最终确定得到的损失的准确度。By using two types of losses, determining the first loss and the second loss, the accuracy of the loss finally determined can be improved.

其中，对第一类损失和第二类损失进行加权得到目标损失L _Exp的公式可以是： Among them, the formula for weighting the first type loss and the second type loss to obtain the target loss L _Exp can be:

L _Exp＝w _DSCL _DSC+w _CrossL _Cross (5)； L _Exp =w _DSC L _DSC +w _Cross L _Cross (5);

其中，w _DSC表示第一类损失对应的权重，w _Cross表示第二类损失对应的权重，这两个权重为超参数。可选地，第一类损失对应的权重和第二类损失对应的权重可以根据第一类损失和第二类损失之间的比值进行动态调整。示例性地，计算得到的第一类损失为0.9，第二类损失为0.05，二者之间的比值为18，则第一类损失和第二类损失的权重可以是1：10。即，这两个权重用于平衡第一类损失和第二类损失所处的量级。例如，若获取得到的第一类损失为0.9，第二类损失为0.05，很明显，若不增加权重，则最终的目标损失基本上仅取决于第一类损失，第二类损失基本没起到调整参数的作用，因此，第一类损失的权重可以设置为1，第二类损失的权重可以设置为10，从而将第一类损失和第二类损失调整为同一量级。通过分别对第一分割网络和第二分割网络分别进行训练，能够提高各网络输出结果的精度。 Among them, w _DSC represents the weight corresponding to the first type of loss, and w _Cross represents the weight corresponding to the second type of loss. These two weights are hyperparameters. Optionally, the weights corresponding to the first type of loss and the weights corresponding to the second type of loss may be dynamically adjusted according to the ratio between the first type of loss and the second type of loss. Exemplarily, the calculated first-type loss is 0.9, the second-type loss is 0.05, and the ratio between them is 18, so the weight of the first-type loss and the second-type loss may be 1:10. That is, these two weights are used to balance the magnitude of the first type loss and the second type loss. For example, if the obtained first-type loss is 0.9, and the second-type loss is 0.05, it is obvious that if the weight is not increased, the final target loss basically depends only on the first-type loss, and the second-type loss basically has no effect. To adjust the parameters, therefore, the weight of the first type of loss can be set to 1, and the weight of the second type of loss can be set to 10, so as to adjust the first type of loss and the second type of loss to the same magnitude. By separately training the first segmentation network and the second segmentation network, the accuracy of output results of each network can be improved.

一些公开实施例中，在利用第一分割网络对第一样本数据进行第一分割以及利用第二分割网络对第二样本数据进行第二分割之前，还可包括以下步骤：In some disclosed embodiments, before using the first segmentation network to perform the first segmentation on the first sample data and using the second segmentation network to perform the second segmentation on the second sample data, the following steps may also be included:

对第一样本数据和第二样本数据进行数据增强。其中，数据增强的方式可以是添加随机高斯噪声、进行随机弹性变化以及随机脉冲中的一种或多种。其中，对样本数据增加随机噪声，用于模拟不同信噪比情况。对样本数据随机进行弹性变化，用于模拟不同病人的身体结构。对样本数据进行随机脉冲，用于模拟脉冲信号。通过对样本数据进行数据增强，能够提高分割网络的泛化能力，进而提高分割网络输出结果的准确度。Data augmentation is performed on the first sample data and the second sample data. Among them, the way of data enhancement can be one or more of adding random Gaussian noise, performing random elastic change and random pulse. Among them, random noise is added to the sample data to simulate different signal-to-noise ratios. Random elastic changes are made to the sample data to simulate the body structure of different patients. Randomly pulses the sample data, used to simulate a pulsed signal. By performing data enhancement on the sample data, the generalization ability of the segmentation network can be improved, and the accuracy of the output results of the segmentation network can be improved.

为更好地理解本公开实施例提供的图像分割方法，请同时参考图3，图3是本申请图像分割方法一实施例的另一流程示意图。In order to better understand the image segmentation method provided by the embodiment of the present disclosure, please refer to FIG. 3 at the same time. FIG. 3 is another schematic flowchart of an embodiment of the image segmentation method of the present application.

如图3所示，输入第一分割网络的待分割图像为三维的z*x*y的单通道灰度图像。例如，待分割图像可以是三维的256*256*256的单通道灰度图像。当然，这里待分割图像的尺寸仅为举例，具体待分割图像的尺寸可以根据实际需求进行设定。该待分割图像经过一系列的下采样以及上采样，得到待分割图像关于主动脉的初始分割结果。然后，使用关于主动脉的初始分割结果，确定主动脉在待分割图像中的位置，进而对待分割图像进行裁剪得到包含主动脉的局部图像。然后，再将包含主动脉的局部图像输入第二分割网络，得到该局部图像关于主动脉的分割结果。进而因为局部图像和待分割图像之间的关系，得到待分割图像的最终分割结果。As shown in FIG. 3 , the image to be segmented that is input into the first segmentation network is a three-dimensional z*x*y single-channel grayscale image. For example, the image to be segmented may be a three-dimensional 256*256*256 single-channel grayscale image. Of course, the size of the image to be segmented here is only an example, and the specific size of the image to be segmented can be set according to actual requirements. The image to be segmented undergoes a series of downsampling and upsampling to obtain an initial segmentation result of the image to be segmented with respect to the aorta. Then, using the initial segmentation results about the aorta, the position of the aorta in the image to be segmented is determined, and then the image to be segmented is cropped to obtain a partial image including the aorta. Then, input the partial image including the aorta into the second segmentation network to obtain the segmentation result of the partial image with respect to the aorta. Furthermore, because of the relationship between the partial image and the image to be segmented, the final segmentation result of the image to be segmented is obtained.

其中，在得到待分割图像的最终分割结果之后，可以根据该分割结果判别待分割图像所属期像。其中，获取待分割图像的所属期像的方式可以是计算主动脉区域的平均HU值。然后根据HU值判断待分割图像的所属期像。判断扫描图像所属期像之后，选择对应模型进行其余器官勾画工作。其中，获取图像HU值的方式请参见一般的技术，此处不做过多叙述。Wherein, after the final segmentation result of the image to be segmented is obtained, the image to which the image to be segmented belongs can be determined according to the segmentation result. Wherein, the manner of acquiring the segmental image of the image to be segmented may be to calculate the average HU value of the aortic region. Then according to the HU value, it is judged which image the image to be segmented belongs to. After judging which phase the scanned image belongs to, select the corresponding model for the rest of the organ delineation work. For the method of obtaining the HU value of the image, please refer to the general technology, which will not be described here.

一些公开实施例中，在得到待分割图像的最终分割结果之后，可以结合肺部的分割结果，对肺的解剖结构进行三维几何建模，清晰直观地显示并定量肺叶、肺段内各组织及动静脉气管等解剖结构，精准定位出病灶的位置及病灶与周围血管、气管的空间毗邻关系，为外科切除手术的规划提供参考，优化手术路径。其中，肺部的分割结果也可以是根据对应的分割网络获取得到。关于肺部的分割结果，此处不做过多叙述。另，还可结合虚拟现实技术，模拟手术体现，有利于减少误差，提高手术的成功率。In some disclosed embodiments, after the final segmentation result of the image to be segmented is obtained, the anatomical structure of the lung can be combined with the segmentation result of the lung to carry out three-dimensional geometric modeling to clearly and intuitively display and quantify the lung lobes, tissues in the lung segment, and Anatomical structures such as arteries, veins, and trachea accurately locate the location of the lesion and the spatial adjacency relationship between the lesion and the surrounding blood vessels and trachea, providing reference for surgical resection planning and optimizing the surgical path. Wherein, the segmentation result of the lungs may also be obtained according to the corresponding segmentation network. Regarding the segmentation results of the lungs, we will not describe too much here. In addition, virtual reality technology can also be used to simulate surgical performance, which is beneficial to reduce errors and improve the success rate of surgery.

一些应用场景中，本公开实施例提供的图像分割方法可以应用于肺部CT图像的计算机辅助诊断***、远程诊断***等产品中。In some application scenarios, the image segmentation method provided by the embodiments of the present disclosure may be applied to products such as a computer-aided diagnosis system and a remote diagnosis system for lung CT images.

图像分割方法的执行主体可以是图像分割装置，例如，图像分割方法可以由终端设备或服务器或其它处理设备执行，其中，终端设备可以为用于医学图像分析的设备、用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字处理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备以及自动驾驶汽车，有定位及建图需求的机器人，有配准需求的医疗成像***，用于增强现实或虚拟现实的眼镜、头盔等产品等。在一些可能的实现方式中，该图像分割方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。The execution subject of the image segmentation method may be an image segmentation device. For example, the image segmentation method may be executed by a terminal device or a server or other processing equipment, wherein the terminal device may be a device for medical image analysis, a user equipment (User Equipment, UE ), mobile devices, user terminals, terminals, cellular phones, cordless phones, Personal Digital Assistant (PDA), handheld devices, computing devices, vehicle-mounted devices, wearable devices, and self-driving cars, with positioning and mapping requirements Robots, medical imaging systems with registration requirements, glasses, helmets and other products for augmented reality or virtual reality, etc. In some possible implementation manners, the image segmentation method may be implemented by a processor invoking computer-readable instructions stored in a memory.

请参阅图4，图4是本申请图像分割装置一实施例的结构示意图。图像分割装置40包括第一获取模块41、初始分割模块42、第二获取模块43以及最终分割模块44。获取模块41，用于获取待分割图像；初始分割模块42，用于对待分割图像进行第一分割，得到待分割图像关于目标的初始分割结果；第二获取模块43，用于基于初始分割结果，从待分割图像中提取包含目标的局部图像；最终分割模块44，用于对局部图像进行第二分割，得到待分割图像关于目标的最终分割结果。Please refer to FIG. 4 . FIG. 4 is a schematic structural diagram of an embodiment of an image segmentation device of the present application. The image segmentation device 40 includes a first acquisition module 41 , an initial segmentation module 42 , a second acquisition module 43 and a final segmentation module 44 . The obtaining module 41 is used to obtain the image to be segmented; the initial segmentation module 42 is used to perform the first segmentation on the image to be segmented to obtain the initial segmentation result of the image to be segmented about the target; the second obtaining module 43 is used to based on the initial segmentation result, A partial image containing the target is extracted from the image to be segmented; the final segmentation module 44 is configured to perform a second segment on the partial image to obtain a final segmentation result of the image to be segmented with respect to the target.

上述方案，对待分割图像进行第一分割得到初始分割结果之后，基于该初始分割结果对待分割图像再次进行第二分割，得到待分割图像关于目标的最终分割结果，相比于只进行第一分割的方式而言，利用两次分割得到的最终分割结果更准确。另外，通过基于初始分割结果，从待分割图像中获取包含目标的局部图像，然后对局部图像进行第二分割，相对于直接对待分割图像进行第二分割而言，前者相对于后者而言数据更小，且目标在局部图像中所在比例相对更大，使得在提高分割速度的同时，能够提高分割结果的准确度。In the above solution, after the first segmentation of the image to be segmented is performed to obtain the initial segmentation result, the image to be segmented is secondly segmented based on the initial segmentation result to obtain the final segmentation result of the image to be segmented about the target. In terms of methods, the final segmentation result obtained by using two segmentations is more accurate. In addition, by obtaining a partial image containing the target from the image to be segmented based on the initial segmentation result, and then performing a second segmentation on the partial image, compared to directly performing the second segmentation on the image to be segmented, the former is more data-intensive than the latter Smaller, and the proportion of the target in the local image is relatively larger, so that the accuracy of the segmentation result can be improved while increasing the segmentation speed.

一些公开实施例中，初始分割结果包括待分割图像中各位置属于目标的初始概率，第二获取模块43基于初始分割结果，从待分割图像中提取包含目标的局部图像，包括：基于待分割图像中各位置属于目标的初始概率，确定目标在待分割图像中的初始位置；利用目标的初始位置对待分割图像进行裁剪，得到包含目标的局部图像。In some disclosed embodiments, the initial segmentation result includes the initial probability that each position in the image to be segmented belongs to the target, and the second acquisition module 43 extracts a partial image containing the target from the image to be segmented based on the initial segmentation result, including: based on the image to be segmented The initial probability of each position in the target belongs to the target, and determine the initial position of the target in the image to be segmented; use the initial position of the target to crop the image to be segmented, and obtain a partial image containing the target.

上述方案，通过基于各位置属于目标的初始概率，即可确定目标在待分割图像中的初始位置。In the above solution, the initial position of the target in the image to be segmented can be determined based on the initial probability that each position belongs to the target.

一些公开实施例中，第一分割是利用第一分割网络实现的，图像分割装置还包括训练模块(图未示)，训练模块用于对第一分割网络进行训练：利用第一分割网络对第一样本数据进行第一分割，得到第一分割结果；获取第一分割结果对应的第一损失，并基于第一损失调整第一分割网络的网络参数；和/或，第二分割是利用第二分割网络实现的；方法还包括以下步骤，以对第二分割网络进行训练：利用第二分割网络对第二样本数据进行第二分割，得到第二分割结果；获取第二分割结果对应的第二损失，并基于第二损失调整第二分割网络的网络参数。In some disclosed embodiments, the first segmentation is realized by using the first segmentation network, and the image segmentation device further includes a training module (not shown in the figure), and the training module is used to train the first segmentation network: use the first segmentation network to train the first segmentation network Performing a first segmentation on a sample data to obtain a first segmentation result; obtaining a first loss corresponding to the first segmentation result, and adjusting network parameters of the first segmentation network based on the first loss; and/or, the second segmentation is using the first implemented by a two-segmentation network; the method also includes the following steps to train the second segmentation network: use the second segmentation network to perform a second segmentation on the second sample data to obtain a second segmentation result; obtain the second segmentation result corresponding to a second loss, and adjust the network parameters of the second segmentation network based on the second loss.

上述方案，通过使用分割网络执行分割的步骤，使得整个分割过程方便快捷。通过分别对第一分割网络和第二分割网络分别进行训练，能够提高各网络输出结果的精度。In the above solution, by using the segmentation network to perform the segmentation steps, the entire segmentation process is convenient and fast. By separately training the first segmentation network and the second segmentation network, the accuracy of output results of each network can be improved.

一些公开实施例中，训练模块获取第一分割结果对应的第一损失或获取第二分割结果对应的第二损失，包括：将第一损失作为目标损失以及将第一分割结果作为目标分割结果，或将第二损失作为目标损失以及将第二分割结果作为目标分割结果；基于目标分割结果，获得第一类损失以及第二类损失；对第一类损失和第二类损失进行加权处理，得到目标损失。In some disclosed embodiments, the training module obtains the first loss corresponding to the first segmentation result or obtains the second loss corresponding to the second segmentation result, including: using the first loss as the target loss and using the first segmentation result as the target segmentation result, Or use the second loss as the target loss and the second segmentation result as the target segmentation result; based on the target segmentation result, the first type loss and the second type loss are obtained; the first type loss and the second type loss are weighted to obtain target loss.

上述方案，通过使用两类损失，确定第一损失和第二损失，能够提高最终确定得到的损失的准确度。In the above solution, by using two types of losses to determine the first loss and the second loss, the accuracy of the final determined loss can be improved.

一些公开实施例中，第一类损失为指数对数损失，和/或，第二类损失为加权指数交叉熵损失。In some disclosed embodiments, the first type of loss is an exponential logarithmic loss, and/or the second type of loss is a weighted exponential cross-entropy loss.

上述方案，通过使用上述两类损失，确定第一损失和第二损失，能够提高最终确定得到的损失的准确度。In the above scheme, by using the above two types of losses to determine the first loss and the second loss, the accuracy of the final determined loss can be improved.

一些公开实施例中，第二类损失为加权指数交叉熵损失，目标分割结果包括对应样本数据中的不同位置属于目标的目标概率；训练模块基于目标分割结果，获得第二类损失，包括：基于各位置对应的目标概率，得到各位置属于至少一种预设类别的类别概率，其中，至少一种预设类别包括位置属于目标和位置不属于目标中的至少一种；对各预设类别的类别概率进行对数处理，得到各预设类别对应的对数处理结果；对于各预设类别，利用预设类别对应的频率统计参数与对数处理结果，得到预设类别的类别损失，其中，频率统计参数基于属于各预设类别的位置数量确定的；利用各预设类别的类别损失，得到加权指数交叉熵损失。In some disclosed embodiments, the second type of loss is a weighted exponential cross-entropy loss, and the target segmentation result includes the target probability that different positions in the corresponding sample data belong to the target; the training module obtains the second type of loss based on the target segmentation result, including: based on The target probability corresponding to each position obtains the category probability that each position belongs to at least one preset category, wherein at least one preset category includes at least one of the position belonging to the target and the position not belonging to the target; for each preset category Logarithmic processing is performed on the category probability to obtain the logarithmic processing results corresponding to each preset category; for each preset category, the category loss of the preset category is obtained by using the frequency statistical parameters corresponding to the preset category and the logarithmic processing results, where, The frequency statistics parameter is determined based on the number of positions belonging to each preset category; using the category loss of each preset category, a weighted exponential cross-entropy loss is obtained.

上述方案，结合各个预设类别的类别损失，确定加权指数交叉熵，相对于仅通过单一预设类别确定损失而言，前者确定得到的损失更准确。In the above solution, the weighted exponential cross-entropy is determined by combining the category losses of each preset category. Compared with determining the loss only through a single preset category, the loss determined by the former is more accurate.

一些公开实施例中，预设类别的频率统计参数为所有预设类别的频率之和与预设类别的频率之间的比值，预设类别的频率为属于预设类别的位置数量与位置总数量之间的比值；和/或，训练模块利用预设类别对应的频率统计参数与对数处理结果，得到预设类别的类别损失，包括：将预设类别对应的频率统计参数与对数处理结果的乘积，作为预设类别的类别损失。In some disclosed embodiments, the frequency statistical parameter of the preset category is the ratio between the sum of the frequencies of all preset categories and the frequency of the preset category, and the frequency of the preset category is the number of locations belonging to the preset category and the total number of locations and/or, the training module uses the frequency statistical parameters corresponding to the preset categories and the logarithmic processing results to obtain the category loss of the preset categories, including: combining the frequency statistical parameters corresponding to the preset categories with the logarithmic processing results The product of , as the category loss for preset categories.

上述方案，通过确定预设类别的频率与所有预设类别的频率之和之间的比值，确定预设类别的类别损失，能够减少因为预设类别与其他预设类别之间的比例失调问题，从而提高损失的准确度。The above solution, by determining the ratio between the frequency of the preset category and the sum of the frequencies of all preset categories, determines the category loss of the preset category, which can reduce the problem of proportional imbalance between the preset category and other preset categories, Thereby improving the accuracy of the loss.

一些公开实施例中，训练模块在利用第一分割网络对第一样本数据进行第一分割，得到第一分割结果，以及利用第二分割网络对第二样本数据进行第二分割，得到第二分割结果之前，训练模块还用于：对第一样本数据和第二样本数据进行数据增强，数据增强的方式包括添加随机高斯噪声、进行随机弹性变化以及添加随机脉冲中的一种或多种。In some disclosed embodiments, the training module uses the first segmentation network to perform the first segmentation on the first sample data to obtain the first segmentation result, and uses the second segmentation network to perform the second segmentation on the second sample data to obtain the second Before splitting the results, the training module is also used to: perform data enhancement on the first sample data and the second sample data, the way of data enhancement includes adding random Gaussian noise, performing random elastic changes, and adding one or more of random pulses .

上述方案，通过对样本数据进行数据增强，能够提高分割网络的泛化能力，进而提高分割网络输出结果的准确度。In the above solution, by performing data enhancement on the sample data, the generalization ability of the segmentation network can be improved, and the accuracy of the output result of the segmentation network can be improved.

一些公开实施例中，待分割图像为三维医学图像，目标包括主动脉；和/或，获取模块41获取待分割图像，包括：从原始医学图像中获取包含主动脉以及至少一个与主动脉相邻器官的局部医学图像；统一局部医学图像在各维度方向上的分辨率，以得到待分割图像。In some disclosed embodiments, the image to be segmented is a three-dimensional medical image, and the target includes the aorta; and/or, the acquisition module 41 acquires the image to be segmented, including: acquiring the aorta and at least one adjacent to the aorta from the original medical image Partial medical images of organs; unify the resolution of local medical images in each dimension to obtain images to be segmented.

上述方案，通过从原始医学图像中获取包括目标的局部医学图像，能够减少分割网络的数据量，从而提高分割网络的分割效率，并且增强了目标在待分割图像中的影响力，进而提高了分割网络输出结果的准确度。另外，通过统一局部医学图像在各个维度方向上的分辨率，能够提高分割网络的准确度。The above scheme can reduce the data volume of the segmentation network by obtaining the partial medical image including the target from the original medical image, thereby improving the segmentation efficiency of the segmentation network, and enhancing the influence of the target in the image to be segmented, thereby improving the segmentation efficiency. Accuracy of network output results. In addition, by unifying the resolution of local medical images in all dimensions, the accuracy of the segmentation network can be improved.

请参阅图5，图5是本申请电子设备一实施例的结构示意图。电子设备50包括存储器51和处理器52，处理器52用于执行存储器51中存储的程序指令，以实现上述图像分割方法实施例中的步骤。在一个具体的实施场景中，电子设备50可以包括但不限于：微型计算机、服务器，此外，电子设备50还可以包括笔记本电脑、平板电脑等移动设备，在此不做限定。Please refer to FIG. 5 . FIG. 5 is a schematic structural diagram of an embodiment of an electronic device of the present application. The electronic device 50 includes a memory 51 and a processor 52, and the processor 52 is configured to execute program instructions stored in the memory 51 to implement the steps in the above embodiment of the image segmentation method. In a specific implementation scenario, the electronic device 50 may include, but is not limited to: a microcomputer and a server. In addition, the electronic device 50 may also include mobile devices such as notebook computers and tablet computers, which are not limited here.

具体而言，处理器52用于控制其自身以及存储器51以实现上述图像分割方法实施例中的步骤。处理器52还可以称为CPU(Central Processing Unit，中央处理单元)。处理器52可能是一种集成电路芯片，具有信号的处理能力。处理器52还可以是通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现场可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。另外，处理器52可以由集成电路芯片共同实现。Specifically, the processor 52 is used to control itself and the memory 51 to implement the steps in the above embodiment of the image segmentation method. The processor 52 may also be called a CPU (Central Processing Unit, central processing unit). The processor 52 may be an integrated circuit chip with signal processing capability. The processor 52 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. In addition, the processor 52 may be jointly realized by an integrated circuit chip.

上述方案，对待分割图像进行第一分割得到初始分割结果之后，基于该初始分割结果对待分割图像再次进行第二分割，得到待分割图像关于目标的最终分割结果，相比于只进行第一分割的方式而言，利用两次分割得到的最终分割结果更准确。In the above solution, after the first segmentation of the image to be segmented is performed to obtain the initial segmentation result, the image to be segmented is then secondly segmented based on the initial segmentation result to obtain the final segmentation result of the image to be segmented about the target. In terms of methods, the final segmentation result obtained by using two segmentations is more accurate.

请参阅图6，图6为本申请计算机可读存储介质一实施例的结构示意图。计算机可读存储介质60存储有能够被处理器运行的程序指令601，程序指令601用于实现上述图像分割方法实施例中的步骤。Please refer to FIG. 6 . FIG. 6 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 60 stores program instructions 601 that can be executed by the processor, and the program instructions 601 are used to implement the steps in the above embodiments of the image segmentation method.

在一些实施例中，本公开实施例提供的装置具有的功能或包含的模块可以用于执行上文方法实施例描述的方法，其具体实现可以参照上文方法实施例的描述，为了简洁，这里不再赘述。In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the method embodiments above, and its specific implementation can refer to the description of the method embodiments above. For brevity, here No longer.

本申请提供一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当所述计算机可读代码在电子设备的处理器中运行时，所述电子设备中的处理器执行用于实现上述方法。The present application provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are run in a processor of an electronic device, The processor in the electronic device executes to implement the above method.

上文对各个实施例的描述倾向于强调各个实施例之间的不同之处，其相同或相似之处可以互相参考，为了简洁，本文不再赘述。The above descriptions of the various embodiments tend to emphasize the differences between the various embodiments, the same or similar points can be referred to each other, and for the sake of brevity, details are not repeated herein.

在本申请所提供的几个实施例中，应该理解到，所揭露的方法和装置，可以通过其它的方式实现。例如，以上所描述的装置实施方式仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性、机械或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed methods and devices may be implemented in other ways. For example, the device implementations described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本申请各个实施方式方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) execute all or part of the steps of the methods in various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .

Claims

一种图像分割方法，其特征在于，包括：An image segmentation method, characterized in that, comprising:

获取待分割图像；Obtain the image to be segmented;

对所述待分割图像进行第一分割，得到所述待分割图像关于目标的初始分割结果；performing a first segmentation on the image to be segmented to obtain an initial segmentation result of the image to be segmented with respect to the target;

基于所述初始分割结果，从所述待分割图像中提取包含所述目标的局部图像；Extracting a partial image containing the target from the image to be segmented based on the initial segmentation result;

对所述局部图像进行第二分割，得到所述待分割图像关于所述目标的最终分割结果。A second segmentation is performed on the partial image to obtain a final segmentation result of the image to be segmented with respect to the target.
根据权利要求1所述的方法，其特征在于，所述初始分割结果包括所述待分割图像中一个或多个位置属于所述目标的初始概率，所述基于所述初始分割结果，从所述待分割图像中提取包含所述目标的局部图像，包括：The method according to claim 1, wherein the initial segmentation result includes an initial probability that one or more positions in the image to be segmented belong to the target, and based on the initial segmentation result, from the Partial images containing the target are extracted from the image to be segmented, including:

基于所述待分割图像中一个或多个位置属于所述目标的初始概率，确定所述目标在所述待分割图像中的初始位置；determining an initial position of the target in the image to be segmented based on an initial probability that one or more positions in the image to be segmented belong to the target;

利用所述目标的初始位置对所述待分割图像进行裁剪，得到包含所述目标的局部图像。The image to be segmented is cropped by using the initial position of the object to obtain a partial image including the object.
根据权利要求1至2任一项所述的方法，其特征在于，所述第一分割是利用第一分割网络实现的，所述方法还包括以下步骤，以对所述第一分割网络进行训练：The method according to any one of claims 1 to 2, wherein the first segmentation is implemented using a first segmentation network, and the method further comprises the following steps to train the first segmentation network :

利用所述第一分割网络对第一样本数据进行第一分割，得到第一分割结果；performing a first segmentation on the first sample data by using the first segmentation network to obtain a first segmentation result;

获取所述第一分割结果对应的第一损失，并基于所述第一损失调整所述第一分割网络的网络参数；Obtaining a first loss corresponding to the first segmentation result, and adjusting network parameters of the first segmentation network based on the first loss;

和/或，所述第二分割是利用第二分割网络实现的；所述方法还包括以下步骤，以对所述第二分割网络进行训练：And/or, the second segmentation is implemented using a second segmentation network; the method also includes the following steps to train the second segmentation network:

利用所述第二分割网络对第二样本数据进行第二分割，得到第二分割结果；performing a second segmentation on the second sample data by using the second segmentation network to obtain a second segmentation result;

获取所述第二分割结果对应的第二损失，并基于所述第二损失调整所述第二分割网络的网络参数。Acquiring a second loss corresponding to the second segmentation result, and adjusting network parameters of the second segmentation network based on the second loss.
根据权利要求3所述的方法，其特征在于，所述获取所述第一分割结果对应的第一损失或获取所述第二分割结果对应的第二损失，包括：The method according to claim 3, wherein said obtaining the first loss corresponding to the first segmentation result or obtaining the second loss corresponding to the second segmentation result comprises:

将所述第一损失作为目标损失以及将所述第一分割结果作为目标分割结果，或将所述第二损失作为目标损失以及将所述第二分割结果作为目标分割结果；using the first loss as a target loss and the first segmentation result as a target segmentation result, or using the second loss as a target loss and using the second segmentation result as a target segmentation result;

基于所述目标分割结果，获得第一类损失以及第二类损失；Obtaining a first type loss and a second type loss based on the target segmentation result;

对所述第一类损失和第二类损失进行加权处理，得到所述目标损失。Weighting is performed on the first type loss and the second type loss to obtain the target loss.
根据权利要求4所述的方法，其特征在于，所述第一类损失为指数对数损失，和/或，所述第二类损失为加权指数交叉熵损失。The method according to claim 4, wherein the first type of loss is an exponential logarithmic loss, and/or the second type of loss is a weighted exponential cross-entropy loss.
根据权利要求4或5所述的方法，其特征在于，所述第二类损失为加权指数交叉熵损失，所述目标分割结果包括对应样本数据中的不同位置属于所述目标的目标概率；基于所述目标分割结果，获得第二类损失，包括：The method according to claim 4 or 5, wherein the second type of loss is a weighted exponential cross-entropy loss, and the target segmentation result includes the target probability that different positions in the corresponding sample data belong to the target; based on The target segmentation result obtains the second type of loss, including:

基于一个或多个所述位置对应的目标概率，得到一个或多个所述位置属于至少一种预设类别的类别概率，其中，所述至少一种预设类别包括所述位置属于所述目标和所述位置不属于所述目标中的至少一种；Based on the target probabilities corresponding to one or more of the positions, the category probability of one or more of the positions belonging to at least one preset category is obtained, wherein the at least one preset category includes that the position belongs to the target and said location does not belong to at least one of said targets;

对一个或多个所述预设类别的类别概率进行对数处理，得到一个或多个所述预设类别对应的对数处理结果；performing logarithmic processing on the category probabilities of one or more preset categories to obtain a logarithmic processing result corresponding to one or more preset categories;

对于一个或多个所述预设类别，利用所述预设类别对应的频率统计参数与所述对数处理结果，得到所述预设类别的类别损失，其中，所述频率统计参数基于属于一个或多个所述预设类别的位置数量确定的；For one or more of the preset categories, the category loss of the preset categories is obtained by using the frequency statistical parameters corresponding to the preset categories and the logarithmic processing result, wherein the frequency statistical parameters are based on belonging to one determined by the number of locations of one or more of said preset categories;

利用一个或多个所述预设类别的类别损失，得到所述加权指数交叉熵损失。The weighted exponential cross-entropy loss is obtained by using category losses of one or more preset categories.
根据权利要求6所述的方法，其特征在于，所述预设类别的频率统计参数为至少部分所述预设类别的频率之和与所述预设类别的频率之间的比值，所述预设类别的频率为属于所述预设类别的位置数量与位置总数量之间的比值；The method according to claim 6, wherein the frequency statistical parameter of the preset category is the ratio between the sum of the frequencies of at least part of the preset categories and the frequency of the preset category, and the preset Let the frequency of a category be the ratio between the number of locations belonging to said preset category and the total number of locations;

和/或，所述利用所述预设类别对应的频率统计参数与所述对数处理结果，得到所述预设类别的类别损失，包括：And/or, using the frequency statistical parameters corresponding to the preset category and the logarithmic processing result to obtain the category loss of the preset category includes:

将所述预设类别对应的频率统计参数与所述对数处理结果的乘积，作为所述预设类别的类别损失。The product of the frequency statistics parameter corresponding to the preset category and the logarithmic processing result is used as the category loss of the preset category.
根据权利要求3-7任一项所述的方法，其特征在于，在所述利用所述第一分割网络对第一样本数据进行第一分割，得到第一分割结果，以及所述利用所述第二分割网络对第二样本数据进行第二分割，得到第二分割结果之前，所述方法还包括：The method according to any one of claims 3-7, characterized in that, performing the first segmentation on the first sample data by using the first segmentation network to obtain a first segmentation result, and using the first segmentation network The second segmentation network performs a second segmentation on the second sample data, and before obtaining the second segmentation result, the method also includes:

对所述第一样本数据和所述第二样本数据进行数据增强，所述数据增强的方式包括添加随机高斯噪声、进行随机弹性变化以及添加随机脉冲中的一种或多种。Data enhancement is performed on the first sample data and the second sample data, and the data enhancement manner includes one or more of adding random Gaussian noise, performing random elastic changes, and adding random pulses.
根据权利要求1所述的方法，其特征在于，所述待分割图像为三维医学图像，所述目标包括主动脉；和/或，The method according to claim 1, wherein the image to be segmented is a three-dimensional medical image, and the target includes the aorta; and/or,

所述获取待分割图像，包括：The acquisition of the image to be segmented includes:

从原始医学图像中获取包含主动脉以及至少一个与所述主动脉相邻器官的局部医学图像；acquiring a partial medical image comprising the aorta and at least one organ adjacent to the aorta from the original medical image;

统一所述局部医学图像在一个或多个维度方向上的分辨率，以得到所述待分割图像。Unifying the resolutions of the local medical images in one or more dimensions to obtain the image to be segmented.
一种图像分割装置，其特征在于，包括：An image segmentation device, characterized in that it comprises:

第一获取模块，用于获取待分割图像；The first acquisition module is used to acquire the image to be segmented;

初始分割模块，用于对所述待分割图像进行第一分割，得到所述待分割图像关于目标的初始分割结果；The initial segmentation module is used to perform the first segmentation on the image to be segmented, and obtain the initial segmentation result of the image to be segmented about the target;

第二获取模块，用于基于所述初始分割结果，从所述待分割图像中提取包含所述目标的局部图像；A second acquisition module, configured to extract a partial image containing the target from the image to be segmented based on the initial segmentation result;

最终分割模块，用于对所述局部图像进行第二分割，得到所述待分割图像关于所述目标的最终分割结果。The final segmentation module is configured to perform a second segmentation on the partial image to obtain a final segmentation result of the image to be segmented with respect to the target.
一种电子设备，其特征在于，包括存储器和处理器，所述处理器用于执行所述存储器中存储的程序指令，以实现权利要求1至9任一项所述的方法。An electronic device, characterized by comprising a memory and a processor, the processor is configured to execute program instructions stored in the memory, so as to implement the method according to any one of claims 1 to 9.
一种计算机可读存储介质，其上存储有程序指令，其特征在于，所述程序指令被处理器执行时实现权利要求1至9任一项所述的方法。A computer-readable storage medium on which program instructions are stored, wherein the method according to any one of claims 1 to 9 is implemented when the program instructions are executed by a processor.
一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当所述计算机可读代码在电子设备的处理器中运行时，所述电子设备中的处理器用于实现权利要求1-9中的任一权利要求所述的方法。A computer program product, comprising computer readable codes, or a non-volatile computer readable storage medium bearing computer readable codes, when the computer readable codes are run in a processor of an electronic device, the electronic The processor in the device is configured to implement the method of any one of claims 1-9.