CN112102328A

CN112102328A - Image segmentation processing method and system based on deep learning and electronic equipment

Info

Publication number: CN112102328A
Application number: CN201910528368.8A
Authority: CN
Inventors: 王立新; 罗杰坚; 张晓璐
Original assignee: Sinovation Ventures Beijing Enterprise Management Co ltd
Current assignee: Sinovation Ventures Beijing Enterprise Management Co ltd
Priority date: 2019-06-18
Filing date: 2019-06-18
Publication date: 2020-12-18

Abstract

The invention provides an image segmentation processing method based on deep learning, a system and electronic equipment thereof, which can be used for carrying out sub-graph segmentation on an image to be processed, training a sub-graph fine segmentation model by using a sub-graph containing a target object, repeatedly training until the sub-graph fine segmentation model reaches a preset segmentation index, segmenting the sub-graph containing the target object, obtaining a sub-graph mask corresponding to the sub-graph, and finally carrying out sub-graph mask splicing based on sub-graph coordinates to obtain a segmentation result of the image to be processed. The image segmentation processing method based on deep learning can realize transfer learning, reduce network training time and improve segmentation precision in an iterative visual field focusing process from a macroscopic analysis mode to a microscopic analysis mode through a sub-image cutting mode. The method can be widely applied to image segmentation processing of multi-scale multi-target objects and/or high original image resolution but small target object ratio, and is particularly suitable for the segmentation task of medical images.

Description

Image segmentation processing method and system based on deep learning and electronic equipment

[ technical field ] A method for producing a semiconductor device

The present invention relates to the field of image processing, and in particular, to an image segmentation processing method and system based on deep learning, and an electronic device

[ background of the invention ]

With the continuous development of artificial intelligence, the demand for image processing is increasing, and in order to better analyze and process images, the conventional method uses a feature extractor or the like to extract corresponding image features, but due to the limitation of the conventional image analysis and processing technology, for images of multi-target and multi-scale target objects or images with high original image resolution and small target object occupation ratio, the required data processing time is long, and the requirement of high-precision image analysis and processing of users is difficult to meet.

Therefore, it is desirable to provide a novel technical solution that can effectively solve the above-mentioned image analysis processing.

[ summary of the invention ]

In order to solve the technical problems of the existing image analysis processing, the invention provides an image segmentation processing method based on deep learning and a system thereof,

An electronic device.

In order to solve the technical problems, the invention provides the following technical scheme: an image segmentation processing method based on deep learning comprises the following steps: step S1: providing an image to be processed with at least one target object, and carrying out sub-image segmentation on the image to be processed to obtain a sub-image containing the target object and sub-image coordinates thereof; step S2, training a sub-graph fine segmentation model by using the sub-graph containing the target object, repeating the training until the sub-graph fine segmentation model reaches a preset segmentation index, and segmenting the sub-graph containing the target object to obtain a sub-graph mask corresponding to the sub-graph; and step S3, combining the sub-image coordinate and the sub-image mask to carry out sub-image mask splicing so as to obtain the segmentation result of the image to be processed.

Preferably, in step S2, the variable involved in the current training is used as the initial value of the variable in the next training, and each training includes multiple iterations.

Preferably, the step S2 specifically includes the following steps: step S21, initializing variables involved in the subgraph fine segmentation model; step S22, the subgraph containing the target object is used as training data, when training starts, variables are updated, the subgraph fine segmentation model is trained by using the image of the cutting plate and the mask, and after multiple iterations until convergence or the iteration number is equal to the preset maximum iteration number, the current training is stopped and new variables are obtained; step S23, determining whether training is continued or stopped based on whether the sub-graph fine segmentation model reaches a preset segmentation index; and step S24, storing the segmentation result under the corresponding variable in the current subgraph fine segmentation model.

Preferably, in step S2, the variables related to the sub-graph fine segmentation model include a combination of a clipping coefficient, a segmentation index coefficient, training times, and a network parameter.

Preferably, in the step S23, the preset segmentation index includes a current training time equal to the maximum training time or a current training time segmentation index coefficient smaller than a previous segmentation index coefficient.

Preferably, the image and the mask of the cropped version in step S22 may be obtained by: and performing center cropping on the sub-image containing the target object and the mask corresponding to the sub-image based on the center position of the target object to obtain the image and the mask of the cropped version.

Preferably, during training, the clipping factor decreases as the number of training increases; and the central position of the current training is calculated according to the segmentation result of the previous training.

In order to solve the above technical problems, the present invention provides another technical solution as follows: an image segmentation system based on deep learning, comprising: the sub-graph cutting module is used for providing an image to be processed with at least one target object and performing sub-graph cutting on the image to be processed to obtain a sub-graph containing the target object and sub-graph coordinates thereof; the subgraph fine segmentation module is used for training a subgraph fine segmentation model by using the subgraph containing the target object, repeating the training until the subgraph fine segmentation model reaches a preset segmentation index, and segmenting the subgraph containing the target object to obtain a subgraph mask corresponding to each subgraph; and the image segmentation mask generation module is used for combining the sub-image coordinates and the sub-image masks to carry out sub-image mask splicing so as to obtain the segmentation result of the image to be processed.

In order to solve the above technical problems, the present invention provides another technical solution as follows: an electronic device includes a storage unit for storing a computer program, and a processing unit for executing the steps of the above-described image segmentation processing method based on deep learning by the computer program stored in the storage unit.

Compared with the prior art, the image segmentation processing method based on deep learning, the system and the electronic equipment thereof have the following beneficial effects:

the image segmentation processing method based on deep learning can be used for training a sub-image fine segmentation model by using a sub-image containing a target object after sub-image segmentation is carried out on an image to be processed, so that a sub-image mask corresponding to the sub-image can be obtained, and finally sub-image mask splicing is carried out based on sub-image coordinates to obtain a segmentation result of the image to be processed. The image segmentation processing method based on deep learning can realize transfer learning in an iterative visual field focusing process from a macroscopic mode to a microscopic mode through a sub-image cutting mode, so that the network training time can be effectively reduced, and the segmentation precision is improved. The image segmentation processing method based on the deep learning provided by the invention can be widely applied to image segmentation processing of multi-target objects with multi-scale and/or high original image resolution but small target object ratio, and is particularly suitable for small target segmentation tasks of medical images.

In the image segmentation processing method based on deep learning provided by the invention, based on the step S2, the variable involved in the current training is used as the initial value of the variable of the next training, and each training includes multiple iterations, so that a corresponding refined segmentation result can be obtained, the effectiveness of the training can be further improved, the network training time can be reduced, and the segmentation accuracy can be improved. And may be based on the effect of the actual training so that continuation or cessation of training may be automatically determined.

Further, in step S21, the sub-graph fine segmentation model is defined by a variable corresponding to the sub-graph fine segmentation model and the preset segmentation index is defined as the number of times of the current training is equal to the maximum training number of times; or when the segmentation index coefficient of the training is smaller than that of the last training, the sub-graph fine segmentation model has wider applicability and adjustability.

In the step S22 of obtaining the image and the mask of the cropping version, it can be ensured that the target object can be always located at the center of the image of the cropping version during the training cropping process, so that the accuracy of image segmentation processing based on the target object can be maintained during the iterative view focusing process, and the segmentation processing speed can be increased. In the training process, the cropping coefficient is reduced along with the increase of the training times, so as to shorten the side lengths of the image and the mask of the cropping plate along with the change of the cropping system, thereby realizing the further focusing of the image visual field. The central position of the current training is related to the segmentation result of the previous training, and the related variable of the current training is used as the initial value of the variable of the next training, so that the transfer learning can be realized in the training process. And multiple iterations are included in a single training, so that the segmentation refinement degree and the segmentation accuracy can be further improved based on iterative visual field focusing.

In step S1 corresponding to the image segmentation processing method based on deep learning provided by the present invention, performing sub-graph segmentation on the image to be processed includes performing rough segmentation on the image to be processed into a plurality of target objects, so as to obtain the boundary frame and the center position of the target object through the rough segmentation; and carrying out sub-graph segmentation based on the boundary frame and the central position of the corresponding target object. Based on the processing steps, the accuracy and the speed of sub-graph segmentation of the image to be processed can be improved.

The invention also provides an image segmentation system based on deep learning and an electronic device, which have the same beneficial effects as the image segmentation processing method based on deep learning, can effectively reduce the network training time of high-precision image segmentation, improve the segmentation precision, and are particularly suitable for segmentation tasks in medical images.

[ description of the drawings ]

Fig. 1A is a flowchart illustrating steps of an image segmentation processing method based on deep learning according to a first embodiment of the present invention.

Fig. 1B is a schematic flow chart of a specific step of performing sub-graph segmentation on an image to be processed in step 1 of the image segmentation processing method based on deep learning shown in fig. 1A.

Fig. 2A is a schematic diagram of the coarse division operation performed in step S01 shown in fig. 1B.

Fig. 2B is a schematic diagram of performing sub-graph segmentation after determining the bounding box and the center position of the target object based on fig. 2A.

Fig. 2C is a schematic diagram of a corresponding sub-graph cut out based on that shown in fig. 2B.

Fig. 3 is a flowchart illustrating a specific implementation step of step S2 shown in fig. 1.

Fig. 4 is a flowchart illustrating a specific implementation step of step S22 shown in fig. 3.

Fig. 5 is a flowchart illustrating a specific implementation step of step S23 shown in fig. 3.

Fig. 6 is a functional block diagram of an image segmentation processing system based on deep learning according to a second embodiment of the present invention.

Fig. 7 is a functional block diagram of the sub-graph splitting module shown in fig. 6.

Fig. 8 is a functional block diagram of an electronic device according to a third embodiment of the present invention.

The attached drawings indicate the following: s, an image; s₀、S₁And S₂A subgraph; p₁Subfigure S₁The center position of (a); p₂Subfigure S₂The center position of (a);

20. an image segmentation system based on deep learning; 21. a subgraph segmentation module; 22. a subgraph fine segmentation module; 23. an image segmentation mask generation module; 211. an image rough segmentation module; 212. a cutting module;

30. an electronic device; 31. a storage unit; 32. and a processing unit.

[ detailed description ] embodiments

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Referring to fig. 1A, a first embodiment of the invention provides an image segmentation processing method based on deep learning S10, which includes the following steps:

step S1: providing an image to be processed with at least one target object, and carrying out sub-image segmentation on the image to be processed to obtain a sub-image containing the target object and sub-image coordinates thereof;

step S2, training a sub-graph fine segmentation model by using the sub-graph containing the target object, repeating the training until the sub-graph fine segmentation model reaches a preset segmentation index, and segmenting the sub-graph containing the target object to obtain a sub-graph mask corresponding to the sub-graph; and

and step S3, combining the sub-image coordinates and the sub-image mask to carry out sub-image mask splicing so as to obtain the segmentation result of the image to be processed.

Based on the above-mentioned image segmentation processing method S10 based on deep learning, it is possible to obtain an image segmentation result with required accuracy in two application scenarios, where one application scenario is an image with multiple scales corresponding to multiple target objects, and the other application scenario is an image with high resolution of an original image and a smaller proportion of corresponding target objects. The method can realize a mode from large to small and from macro to micro based on the target object, thereby improving the image segmentation precision. It should be particularly noted that the images of multiple scales mentioned in the present invention are understood that the target object can be processed at different scales respectively. In some embodiments, the scale referred to may also be understood as multi-resolution.

Referring to fig. 1B, in step S1, the sub-image segmentation of the image to be processed specifically includes the following steps:

step S01, inputting the image to be processed into a rough segmentation model for rough segmentation to obtain a boundary frame and a central position of a corresponding target object; and

step S02: and performing sub-graph segmentation based on the bounding box and the central position of the corresponding target object.

Specifically, in step S1, the rough segmentation model may be obtained based on training, and a training set of original training images with segmentation labels is used to train the initial rough segmentation model. For convenience of processing, a memory limit value of the training image can be set, and if the memory size of the original training image exceeds the memory limit value, the original length and width of the original training image can be utilized to reduce the layout image data in an equal proportion to perform corresponding processing. The original training image with the segmentation labels and the image to be processed are the same type of image, and have similar target objects to be segmented, and the segmentation labels can correspond to different target objects. The method is particularly suitable for small target segmentation tasks in medical images, for example, the progress can be improved by 1.2% in trachea medical image segmentation compared with the situation of one-time training of a pure U-net network, and the corresponding training time can be saved by about 30%.

In the above step S01, the rough segmentation model includes but is not limited to: U-Net, SegNet, DeepLab, FCN, ENet, LinkNet, PSPNet, etc.

To better explain the image segmentation processing method S10 based on the deep learning, the following example is given:

as shown in fig. 2A, the image of the roughly divided model set in step S01 is input as an image S having a long side with a length and a width b. The image S includes N (N is 1, 2.. N-1, N, where N is a positive integer) target objects, as shown in fig. 2A, where N is 2.

In the above step S02, the bounding box and the center position of the corresponding target object may be understood as the area range of the target object and the center position within the corresponding area range. For example, taking the medical image of the articular cartilage region as an example, if the target objects in the image to be processed are patellar cartilage, femoral cartilage and tibial cartilage, the bounding boxes thereof correspond to the boundary region ranges of the patellar cartilage, femoral cartilage and tibial cartilage. Wherein the maximum length of the bounding box is set as a_max。

Combination drawing2B, the corresponding two target objects are respectively triangular and circular, and the corresponding central position is P₁、P₂The corresponding bounding boxes correspond to the edges of the triangle and circle, respectively. Set side length I of sub-diagram S₀Wherein the value range of I is (a)_maxA). For example, the corresponding side length I may take a value of 0.5 (a)_max+ a), in other embodiments, said sub-diagram S₀The length of the side(s) may also be other specific values.

Using subgraph S₀The image is sub-image cut, as shown in FIG. 2C, with each sub-image S₀Is divided into sub-graphs S₁Subfigure S₂Corresponds to a central position of P₁、P₂And setting the central position of each target object after rough segmentation.

Correspondingly, in the process of carrying out sub-image segmentation on the image S, sub-image coordinates of the corresponding target image can be obtained based on the central position of the corresponding target object.

As shown in fig. 2A-2C, the image S includes two target objects, which correspond to different scales. The scale in the invention can be understood as being based on the image scale space theory, namely different target objects in the same image can have different optimal scales.

In the step S2, the preset segmentation index refers to a segmentation standard corresponding to a sub-graph fine segmentation model in the sub-graph fine segmentation process for target objects with different sub-graphs and different scales, so as to obtain a segmentation effect required by a user. For example, as shown in fig. 2A to fig. 2C, even if different target objects of the same image are different, in the training process of sub-graph fine segmentation, the preset segmentation indexes corresponding to the sub-graph fine segmentation model may be different, and may be set based on an empirical value or a standard.

In step S2, the variables involved in the current training may be used as initial values of the variables for the next training. Variables related to the subgraph fine segmentation model comprise a combination of a cutting coefficient K, a segmentation index coefficient Dice, training times j and a network parameter M.

Specifically, in this embodiment, the cropping system K may be understood as a size relationship between a side length of an image obtained by performing center cropping on the image and a side length of the initial sub-image in each training process, and a size of the cropping coefficient K may reflect a size of the image obtained by cropping.

The segmentation index coefficient Dice may be used to evaluate image segmentation performance, and in some embodiments, the segmentation index coefficient Dice may be expressed as:

coefficient of division index

The predicted divided region area and the real divided region area are respectively expressed as a divided region area predicted based on the current clipping coefficient K and a real divided region area.

And the training times j represent the actual training times when a subgraph fine segmentation model is trained by using the subgraph containing the target object. Wherein each training may include multiple iterations.

The network parameters M may be represented as variables in a subgraph fine-segmentation model. In this embodiment, taking the U-net network as an example, the network parameter M can be understood as a variable that can embody the characteristics of the U-net network.

Referring to fig. 3, in step S2, training a sub-graph fine segmentation model using the sub-graph including the target object, repeating the training until the sub-graph fine segmentation model reaches a preset segmentation index, and segmenting the sub-graph including the target object to obtain a sub-graph mask corresponding to the sub-graph specifically includes the following steps:

step S21, initializing variables involved in the subgraph fine segmentation model;

step S22, the subgraph containing the target object is used as training data, when training is started each time, variables are updated, the subgraph fine segmentation model is trained by using the image of the cutting plate and the mask, and after repeated iteration until convergence or the current iteration number is equal to the preset maximum iteration number, current training is stopped and new variables are obtained;

step S23, determining whether training is continued or stopped based on whether the sub-graph fine segmentation model reaches a preset segmentation index; and

and step S24, saving the fine segmentation result of the corresponding variable in the current sub-graph fine segmentation model.

Specifically, in step S22 described above, the sub-image including the target object and the mask corresponding thereto are center-clipped using the image and the mask of the clipped version based on the center position of the target object to obtain the image and the mask of the clipped version.

Specifically, in the above steps, center clipping is performed on the image and the mask, the network parameter M of the current sub-graph fine segmentation model is used as an initial value of the network parameter of the current training, the image and the mask of the clipping version are used to train the sub-graph fine segmentation model, the current training is stopped and a new variable is obtained after multiple iterations until convergence or the current iteration number is equal to a preset maximum iteration number, wherein the new variable includes the network parameter M_jAnd the corresponding division index coefficient Dice.

In the above step S23, the preset segmentation index includes the number of times of the current training is equal to the maximum training number J_max(ii) a Or when the segmentation index coefficient of the training is smaller than the segmentation index coefficient of the last training.

In step S24, the corresponding variable of the current sub-graph fine segmentation model may be represented as a network parameter M_jAnd cutting system K_j。

Specifically, referring to fig. 4, the step S22 can be further subdivided;

step S221, based on the obtained initial values of the corresponding variables in the subgraph fine segmentation model; in particular, the clipping coefficient K may be initialized₀1.0; initializing a segmentation index coefficient Dice to be 0; training time J is 0, and the maximum training time J of the sub-graph fine segmentation model of the initial clipping training_maxAnd may be generally set to an integer of 3 to 8. Randomly initializing network parameters M of sub-graph fine segmentation model trained at the same time_j＝M₀。

Step S222, performing the 1 st training, where the updated training frequency 1 is 0+ 1; meterCalculating a clipping coefficient K₁＝K₀Alpha value; wherein, the alpha value can be a decimal between 0.5 and 0.95, and the selection of the specific numerical value can be related to the type of the image to be cut and the precision requirement thereof;

step S223, center cutting is carried out on the image and the mask; in particular, it can be understood that the side length of the image to be processed is a clipping factor K₁Shortening is carried out;

step S224, adding M_0＝1-1(i.e., M)_jNetwork parameter expressed as jth training) as initial value of network parameter of first iteration of current training, and training subgraph fine segmentation model by using image of cutting plate and mask to obtain iteration number (epoch) reaching E after convergence or current training_maxLater network parameter M_jA corresponding segmentation index coefficient Dice;

wherein E is_maxThe specific value range of the maximum iteration number in each training is 10-20. The maximum iteration number of each training may be determined based on the specific model to be trained and the training data thereof, and is only used as an example and not a specific limitation of the present invention.

It should be further explained that in the training process, the clipping coefficient K decreases as the training times increase, that is, in the training process of the sub-graph fine segmentation model, the side length of the sub-graph decreases as the clipping system K decreases.

And when the central position of the training is related to the segmentation result of the last training, the initial value of the network parameter of the training is consistent with the network parameter after the last training.

Specifically, referring to fig. 5, the step S23, based on the corresponding comparison of the segmentation index coefficients of the step S23, includes the following steps:

step S231, judging whether the current training time J is equal to the maximum training time J_maxIf not, go to step S232; if yes, go to step S233;

in step S232, it is determined whether or not the division index coefficient of the number of times of training is smaller than the division index coefficient of the previous training, and if so, the routine proceeds to step S233, and if not, the number of times of training is updated and the routine returns to step S22.

Step S233, the training is stopped.

In the above method, the clipping coefficient K may be adjusted according to the rise and fall of the division index coefficient and the number of training times.

Based on the above-mentioned deep learning image high-precision segmentation method using iterative view focusing in step S3, the segmented sub-graph can be effectively refined and segmented, so that the target object in the sub-graph obtained based on the rough segmentation can be finely segmented. The method provided by the invention is particularly suitable for image segmentation of a plurality of target objects with different scales in the same image to be processed, can further perform segmentation of a single target object based on a subgraph fine segmentation model after the image to be processed is roughly segmented, and can continuously cut the image and a mask thereof based on training. And transfer learning is realized in the iterative visual field focusing process, the network training time is reduced, and the segmentation precision is improved.

Continuing to combine with the description in fig. 3, in step S3, combining the sub-map coordinates and the sub-map mask, and stitching to obtain a segmentation result of the image to be processed, where the segmentation result of the image to be processed is specifically a refined segmentation mask of the image to be processed.

The image segmentation based on the deep learning provided in the embodiment can obtain the high-precision segmentation mask in two scenes, namely, an image with multiple scale target objects, and an image with high resolution and small target object ratio in the original image. The image segmentation processing method based on deep learning provided by the invention aims to realize transfer learning in an iterative visual field focusing process in a mode of dividing and treating and from macro to micro, reduce network training time and improve segmentation precision.

Referring to fig. 6, a second embodiment of the present invention provides an image segmentation system 20 based on deep learning, which includes:

the subgraph segmentation module 21 is used for providing an image to be processed with at least one target object and carrying out subgraph segmentation on the image to be processed to obtain a subgraph containing the target object and subgraph coordinates thereof;

the subgraph fine segmentation module 22 is used for training a subgraph fine segmentation model by using the subgraph containing the target object, and repeatedly training until the subgraph fine segmentation model reaches a preset segmentation index, segmenting the subgraph containing the target object to obtain a subgraph mask corresponding to each subgraph; and

and the image segmentation mask generation module 23 is used for performing sub-image mask splicing by combining the sub-image coordinates and the sub-image masks to obtain a segmentation result of the image to be processed.

In order to obtain better image segmentation effect and improve segmentation accuracy, as shown in fig. 7, the sub-graph segmentation module 21 further includes:

an image rough segmentation module 211, configured to input, based on a provided to-be-processed image with at least one target object, a rough segmentation model for rough segmentation to obtain a bounding box and a center position of the corresponding target object; and

and the segmentation module 212 is configured to perform sub-graph segmentation on the image to be processed based on the bounding box and the central position of the corresponding target object, so as to obtain a sub-graph containing the target object and sub-graph coordinates thereof.

Specifically, in this embodiment, the relevant contents of the model related to the rough image segmentation and the fine sub-image segmentation are the same as those in the first embodiment, and are not described herein again.

Referring to fig. 8, a third embodiment of the present invention provides an electronic device 30, where the electronic device 30 includes a storage unit 31 and a processing unit 32, the storage unit 31 is used for storing a computer program, and the processing unit 32 is used for executing specific steps of the image segmentation processing method based on deep learning in the first embodiment through the computer program stored in the storage unit 31.

In some specific embodiments of the present invention, the electronic device 30 may be hardware or software. When the electronic device 30 is hardware, it may be various electronic devices having a display screen and supporting video playing, including but not limited to a smart phone, a tablet computer, an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III, motion Picture Experts Group Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, motion Picture Experts Group Audio Layer 4), a laptop computer, a desktop computer, and the like. When the electronic device 30 is software, it can be installed in the electronic devices listed above. It may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.

The storage unit 31 includes a storage portion of a Read Only Memory (ROM), a Random Access Memory (RAM), a hard disk, and the like, and the processing unit 32 may perform various appropriate actions and processes according to a program stored in the Read Only Memory (ROM) or a program loaded into the Random Access Memory (RAM). In a Random Access Memory (RAM), various programs and data necessary for the operation of the electronic device 30 are also stored.

The electronic device 30 may further include an input portion (not shown) of a keyboard, a mouse, and the like; the electronic device 30 may further include an output portion (not shown) such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker and the like; and the electronic device 30 may further include a communication part (not shown) of a network interface card such as a LAN card, a modem, and the like. The communication section 35 performs communication processing via a network such as the internet.

In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the disclosed embodiments of the invention may include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section.

When executed by the processing unit 32, the computer program performs the above-described functions defined in the method for training a neural network model with an anti-counterfeiting function of the present application. It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

In the present application, a computer readable storage medium may also be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures of the present application illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will be understood that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units involved in the embodiments of the present invention may be implemented by software or hardware. The described units may also be located in the processor.

As another aspect, the fourth embodiment of the present invention also provides a computer-readable medium, which may be contained in the apparatus described in the above-described embodiments; or may be present separately and not assembled into the device. The computer-readable medium carries one or more programs, which specifically include: providing an image to be processed with at least one target object, and carrying out sub-image segmentation on the image to be processed to obtain a sub-image containing the target object and sub-image coordinates thereof; training a sub-graph fine segmentation model by using the sub-graphs containing the target object, and segmenting the sub-graphs containing the target object after the sub-graph fine segmentation model reaches a preset segmentation index to obtain sub-graph masks corresponding to each sub-graph; and combining the sub-image coordinates and the sub-image mask to carry out sub-image mask splicing so as to obtain a segmentation result of the image to be processed.

the image segmentation processing method based on deep learning can be used for training a sub-image fine segmentation model by using a sub-image containing a target object after sub-image segmentation is carried out on an image to be processed, so that a sub-image mask corresponding to the sub-image can be obtained, and finally sub-image mask splicing is carried out based on sub-image coordinates to obtain a segmentation result of the image to be processed. The image segmentation processing method based on deep learning can realize transfer learning in the process of iteratively training a subgraph fine segmentation model in a subgraph cutting mode from macro to micro, thereby effectively reducing network training time and improving segmentation precision. The image segmentation processing method based on the deep learning provided by the invention can be widely applied to the image segmentation processing of multi-target objects with multi-scale and/or high original image resolution but small target object ratio, and is particularly suitable for the segmentation task in medical images.

The image segmentation system based on the deep learning and the electronic equipment have the same beneficial effects as the image segmentation processing method based on the deep learning, can effectively reduce the network training time of high-precision image segmentation, improve the segmentation precision, and are particularly suitable for segmentation tasks in medical images.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit of the present invention are intended to be included within the scope of the present invention.

Claims

1. An image segmentation processing method based on deep learning is characterized in that: which comprises the following steps:

2. The deep learning-based image segmentation processing method as claimed in claim 1, characterized in that: in step S2, the variables involved in the current training are used as initial values of the variables for the next training, and each training includes a plurality of iterations.

3. The deep learning-based image segmentation processing method as set forth in claim 2, characterized in that: the step S2 specifically includes the following steps:

and step S24, storing the segmentation result under the corresponding variable in the current subgraph fine segmentation model.

4. The deep learning-based image segmentation processing method as set forth in claim 3, characterized in that: in step S2, the variables related to the sub-graph fine segmentation model include a combination of a clipping coefficient, a segmentation index coefficient, a training number, and a network parameter.

5. The deep learning-based image segmentation processing method as set forth in claim 3, characterized in that: in the above step S23, the preset segmentation index includes a current training time equal to the maximum training time or a current training time segmentation index coefficient smaller than a previous segmentation index coefficient.

6. The deep learning-based image segmentation processing method as set forth in claim 3, characterized in that: the image and the mask of the cropped version in step S22 may be obtained by: and performing center cropping on the sub-image containing the target object and the mask corresponding to the sub-image based on the center position of the target object to obtain the image and the mask of the cropped version.

7. The deep learning-based image segmentation processing method as set forth in claim 3, characterized in that: in the training process, the clipping coefficient is reduced along with the increase of the training times; and the central position of the current training is calculated according to the segmentation result of the previous training.

8. The deep learning-based image segmentation processing method as claimed in claim 1, characterized in that: in the step S1, the sub-image segmentation of the image to be processed specifically includes the following steps:

9. An image segmentation system based on deep learning, characterized in that: the image segmentation system based on the deep learning comprises,

the sub-graph cutting module is used for providing an image to be processed with at least one target object and performing sub-graph cutting on the image to be processed to obtain a sub-graph containing the target object and sub-graph coordinates thereof;

the subgraph fine segmentation module is used for training a subgraph fine segmentation model by using the subgraph containing the target object, repeating the training until the subgraph fine segmentation model reaches a preset segmentation index, and segmenting the subgraph containing the target object to obtain a subgraph mask corresponding to each subgraph; and

and the image segmentation mask generation module is used for carrying out sub-image mask splicing by combining the sub-image coordinates and the sub-image masks so as to obtain the segmentation result of the image to be processed.

10. An electronic device, characterized in that: the electronic equipment comprises a storage unit and a processing unit, wherein the storage unit is used for storing a computer program, and the processing unit is used for executing the steps of the image segmentation processing method based on the deep learning through the computer program stored by the storage unit.