CN113592819A - Image processing system and method - Google Patents

Image processing system and method Download PDF

Info

Publication number
CN113592819A
CN113592819A CN202110875128.2A CN202110875128A CN113592819A CN 113592819 A CN113592819 A CN 113592819A CN 202110875128 A CN202110875128 A CN 202110875128A CN 113592819 A CN113592819 A CN 113592819A
Authority
CN
China
Prior art keywords
image
human body
processed
subregion
body part
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110875128.2A
Other languages
Chinese (zh)
Other versions
CN113592819B (en
Inventor
孙世辰
韦景琪
张哲嘉
徐昕
冯建兴
张胜洪
周英
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Haohua Technology Co ltd
Original Assignee
Shanghai Haohua Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Haohua Technology Co ltd filed Critical Shanghai Haohua Technology Co ltd
Priority to CN202110875128.2A priority Critical patent/CN113592819B/en
Priority claimed from CN202110875128.2A external-priority patent/CN113592819B/en
Publication of CN113592819A publication Critical patent/CN113592819A/en
Application granted granted Critical
Publication of CN113592819B publication Critical patent/CN113592819B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Processing (AREA)

Abstract

The invention relates to the technical field of image processing, in particular to an image processing system and method, and aims to solve the problems of long transmission time and reduced image processing efficiency caused by overlarge images. To this end, the invention comprises: the image segmentation device is configured to segment the human body image; the image analysis report processing device is configured to analyze the image analysis report and determine an abnormal human body part; the image transmission device is configured to determine the transmission priority of each subregion image, transmit each subregion image to the image display terminal according to the transmission priority, and transmit the subregion image corresponding to the abnormal human body part to the image display terminal to be displayed in priority to other subregion images; the image synthesis display device is configured to synthesize all the subarea images into a new human body image and display the new human body image. By dividing the image, the subregion image of the human body part with the abnormality is preferentially sent, and the defect that the image processing efficiency is reduced due to overlong image transmission time is overcome.

Description

Image processing system and method
Technical Field
The invention relates to the technical field of image processing, and particularly provides an image processing system and method.
Background
The shot images sometimes need to be transmitted remotely in time for other users to check, and in some scenes, due to reasons such as too high resolution of the shot images, the storage space occupied by the images is often very large, the time required by transmission is often very long under the condition, the image processing efficiency is greatly influenced, and therefore how to improve the image processing efficiency is a very important problem.
For the above problems, the existing means generally includes increasing the network speed or compressing the image, and the above methods all have respective disadvantages or shortcomings, increasing the network speed needs to be supported by hardware devices, the costs in all aspects are very high, and transmitting the compressed image will result in loss of image data, reduce the quality of the image, and affect the viewing of image consultants.
Accordingly, there is a need in the art for a new image processing scheme to address the above-mentioned problems.
Disclosure of Invention
The invention aims to solve the technical problems that the transmission time is long due to overlarge images and the image processing efficiency is reduced.
In a first aspect, the present invention provides an image processing system, the system comprising:
image segmentation means configured to image-segment the human body image to obtain sub-region images of a plurality of different human body parts;
the image analysis report processing device is configured to acquire an image analysis report for describing states of different human body parts in the human body image, and perform semantic analysis on text information in the image analysis report to determine the human body parts with the abnormality;
the image transmission device is configured to determine the transmission priority of each subregion image according to the abnormal human body part and respectively transmit each subregion image to a preset image display terminal according to the transmission priority so that the subregion image corresponding to the abnormal human body part can be transmitted to the image display terminal to be displayed in priority to other subregion images;
and the image synthesis display device is configured to synthesize all the subarea images into a new human body image and display the new human body image through the image display terminal after all the subarea images are transmitted.
In one embodiment of the image processing system, the image segmentation apparatus includes a first image segmentation module and/or a second image segmentation module;
the first image segmentation module is configured to determine a position of each human body part in the human body image, and perform image segmentation on the human body image according to the position to obtain a plurality of subarea images, wherein each subarea image at least comprises image information of one human body part;
the second image segmentation module is configured to perform the following operations:
performing image semantic analysis on the human body image by adopting an image semantic segmentation model to obtain image division masks corresponding to each human body part in the human body image;
and carrying out image segmentation on the human body image according to the position of the image division mask in the human body image so as to obtain a plurality of subarea images, wherein each subarea image at least comprises image information of a human body part.
In one embodiment of the above image processing system, the text information in the image analysis report includes state description information of a human body part; the image analysis report processing apparatus includes a semantic analysis module configured to perform the following operations:
performing first semantic analysis on each state description information respectively to acquire words representing state description, words representing human body parts and words representing directions in the state description information; wherein the words representing orientation include at least words representing position and/or words representing direction;
performing second semantic analysis on each word representing the state description to screen the word representing the abnormal state and take the state description information to which the word representing the abnormal state belongs as the state description information to be processed;
searching a word representing the direction in the to-be-processed state description information, wherein the word representing the direction is nearest to the word representing the abnormal state;
and generating a keyword pair of the human body part with the abnormality according to the word which represents the orientation and is nearest to the word representing the abnormal state, and the word representing the human body part and the word representing the state description in the to-be-processed state description information.
In one aspect of the above image processing system, the image transmission apparatus includes a transmission priority determination module configured to perform the following operations:
associating the keyword pair with the subregion image by:
for each keyword pair, determining each human body part represented by a word representing the human body part in the keyword pair, and associating the keyword pair with each sub-region image corresponding to each human body part;
taking the subregion image associated with the keyword pair as a first subregion image to be processed, and taking the subregion image not associated with the keyword pair as a second subregion image to be processed;
respectively calculating the priority score of each first subarea image to be processed according to a method shown as the following formula:
Figure BDA0003190291680000031
wherein, G isiRepresenting the priority score of the ith first subregion image to be processed, CiA attention coefficient representing a preset ith first subarea image to be processed, wherein n isiIndicating the number of the associated keyword pairs of the ith first subarea image to be processed, LjRepresenting an abnormal degree coefficient corresponding to a word representing state description in a jth keyword pair associated with a preset ith first subarea image to be processed, wherein N represents the total number of the keyword pairs generated by the semantic analysis module, and alpha and beta represent preset constant coefficients;
determining the transmission priority of each first to-be-processed subregion image according to the priority score;
according to the position of each second subregion image to be processed in the human body image and a preset position sequence, sequentially determining the transmission priority of each second subregion image to be processed;
the priority degree of the transmission priority of the first to-be-processed subarea image is higher than that of the transmission priority of the second to-be-processed subarea image.
In one technical solution of the image processing system, the image semantic segmentation model is obtained by training using an image semantic segmentation algorithm.
In a second aspect, the present invention provides an image processing method, comprising:
carrying out image segmentation on the human body image to obtain a plurality of subarea images of different human body parts;
acquiring an image analysis report for describing states of different human body parts in the human body image, and performing semantic analysis on text information in the image analysis report to determine the abnormal human body parts;
determining the transmission priority of each subregion image according to the abnormal human body part, and respectively transmitting each subregion image to a preset image display terminal according to the transmission priority, so that the subregion image corresponding to the abnormal human body part can be transmitted to the image display terminal to be displayed in priority to other subregion images;
and synthesizing all the transmitted sub-region images into a new human body image and displaying the new human body image through the image display terminal.
In one technical solution of the above image processing method, "performing image segmentation on a human body image to obtain sub-region images of a plurality of different human body parts" includes:
determining the position of each human body part in the human body image, and performing image segmentation on the human body image according to the position to obtain a plurality of sub-region images, wherein each sub-region image at least comprises image information of one human body part;
and/or the specific steps of performing image segmentation on the human body image to acquire a plurality of subarea images of different human body parts comprise:
performing image semantic analysis on the human body image by adopting an image semantic segmentation model to obtain image division masks corresponding to each human body part in the human body image;
and carrying out image segmentation on the human body image according to the position of the image division mask in the human body image so as to obtain a plurality of subarea images, wherein each subarea image at least comprises image information of a human body part.
In one technical solution of the above image processing method, the text information in the image analysis report includes state description information of a human body part; the specific steps of performing semantic analysis on the text information in the image analysis report to determine the human body part with the abnormality comprise:
performing first semantic analysis on each state description information respectively to acquire words representing state description, words representing human body parts and words representing directions in the state description information; wherein the words representing orientation include at least words representing position and/or words representing direction;
performing second semantic analysis on each word representing the state description to screen the word representing the abnormal state and take the state description information to which the word representing the abnormal state belongs as the state description information to be processed;
searching a word representing the direction in the to-be-processed state description information, wherein the word representing the direction is nearest to the word representing the abnormal state;
and generating a keyword pair of the human body part with the abnormality according to the word which represents the orientation and is nearest to the word representing the abnormal state, and the word representing the human body part and the word representing the state description in the to-be-processed state description information.
In one embodiment of the above image processing method, "determining a transmission priority of each of the sub-region images according to the abnormal human body part" includes:
associating the keyword pair with the subregion image by:
for each keyword pair, determining each human body part represented by a word representing the human body part in the keyword pair, and associating the keyword pair with each sub-region image corresponding to each human body part;
taking the subregion image associated with the keyword pair as a first subregion image to be processed, and taking the subregion image not associated with the keyword pair as a second subregion image to be processed;
respectively calculating the priority score of each first subarea image to be processed according to a method shown as the following formula:
Figure BDA0003190291680000051
wherein,the G isiRepresenting the priority score of the ith first subregion image to be processed, CiA attention coefficient representing a preset ith first subarea image to be processed, wherein n isiIndicating the number of the associated keyword pairs of the ith first subarea image to be processed, LjRepresenting an abnormal degree coefficient corresponding to a word representing state description in a jth keyword pair associated with a preset ith first subarea image to be processed, wherein N represents the total number of the keyword pairs generated by the semantic analysis module, and alpha and beta represent preset constant coefficients;
determining the transmission priority of each first to-be-processed subregion image according to the priority score;
according to the position of each second subregion image to be processed in the human body image and a preset position sequence, sequentially determining the transmission priority of each second subregion image to be processed;
the priority degree of the transmission priority of the first to-be-processed subarea image is higher than that of the transmission priority of the second to-be-processed subarea image.
In one technical solution of the above image processing method, the image semantic segmentation model is obtained by training using an image semantic segmentation algorithm.
Under the condition of adopting the technical scheme, the invention can segment the human body image to obtain a plurality of subarea images of different human body parts, then analyze the report according to the images of different human body parts in the human body image, and carry out semantic analysis on text information in the image analysis report to determine the human body parts with abnormity, then determine the transmission priority of each subarea image according to the human body parts with abnormity, transmit each subarea image according to the transmission priority and display the subarea image on the image display terminal, because the subarea image of the human body part with abnormity is transmitted preferentially, the subarea image of the human body part with abnormity can be displayed on the image display terminal preferentially, image consultants can look up the subarea image of the human body part with abnormity preferentially without waiting for the transmission of all the images to be finished, namely the image processing efficiency is improved, finally, after the transmission of all the subarea images is finished, the subarea images still converge into a complete new human body image, and the normal work of image consultants cannot be influenced.
Drawings
The disclosure of the present invention will become more readily understood with reference to the accompanying drawings. As is readily understood by those skilled in the art: these drawings are for illustrative purposes only and are not intended to constitute a limitation on the scope of the present invention. Moreover, in the drawings, like numerals are used to indicate like parts, and in which:
FIG. 1 is a flow diagram illustrating the main steps of an image processing method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of an image processing method according to an embodiment of the present invention for analyzing an image analysis report to determine the existence of an abnormal human body part;
FIG. 3 is a flow chart illustrating the process of determining the sub-region image transmission priority according to an embodiment of the present invention;
FIG. 4 is a block diagram illustrating the main structure of an image processing apparatus according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of an application scenario of an image processing method according to an embodiment of the present invention;
FIG. 6 is a diagram of mask segmentation of a human image according to an image processing method of an embodiment of the present invention;
FIG. 7 is a diagram of an image processing method according to an embodiment of the present invention for segmenting an image of a human body to obtain a plurality of sub-region images;
FIG. 8 is a schematic diagram of semantic analysis of an image analysis report by an image processing method according to an embodiment of the present invention.
List of reference numerals:
41: an image segmentation means; 42: an image segmentation report processing device; 43: an image transmission device; 44: an image composition display device.
Detailed Description
Some embodiments of the invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only for explaining the technical principle of the present invention, and are not intended to limit the scope of the present invention.
In the description of the present invention, a "module" or "processor" may include hardware, software, or a combination of both. A module may comprise hardware circuitry, various suitable sensors, communication ports, memory, may comprise software components such as program code, or may be a combination of software and hardware. The processor may be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and/or signal processing functionality. The processor may be implemented in software, hardware, or a combination thereof. Non-transitory computer readable storage media include any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random-access memory, and the like. The term "a and/or B" denotes all possible combinations of a and B, such as a alone, B alone or a and B. The term "at least one A or B" or "at least one of A and B" means similar to "A and/or B" and may include only A, only B, or both A and B. The singular forms "a", "an" and "the" may include the plural forms as well.
Due to the reason that the resolution of the partial images is too high, the storage space occupied by the images may be too large, and accordingly, the time consumed when the images need to be transmitted for a long distance may be too long, and the image consultant may need to wait for a long time, which may greatly affect the image processing efficiency. In a traditional image processing scheme, schemes of improving network speed or transmitting an image after compression are generally adopted, the schemes can achieve the purpose of shortening image transmission time, but have defects, the schemes of improving network speed are simple and effective, equipment needs to be fundamentally replaced, cost is too high, and transmission of the image after compression can cause image loss details, image definition is reduced, and partial details can not be clearly presented, so that work of image consultants is influenced.
In the embodiment of the invention, the human body image can be divided to obtain a plurality of subarea images of different human body parts, then the semantic analysis is carried out on the image report to obtain the abnormal human body parts, the transmission priority of each subarea image is determined according to the abnormal human body parts, each subarea image is transmitted in sequence according to the transmission priority, the subarea images of the abnormal human body parts are transmitted preferentially, so that the abnormal human body parts can be transmitted and displayed preferentially on the picture of the image display terminal, an image consultant can check the subarea images of the abnormal human body parts more quickly, the image processing efficiency is improved, and after all the images are transmitted, all the subarea images can be combined into a complete image, the normal work of the image consultant can not be influenced.
As shown in fig. 5, in an example of an application scenario of the present invention, a right temporal lobe and a brain stem of a person are abnormal, but magnetic resonance imaging is a body image of the whole brain, and the body image needs to be remotely transmitted to other image display terminals for a magnetic resonance imaging reviewer to view to determine the reason of the abnormality. In this case, according to the method in this embodiment, for example, the image segmentation device in the image processing server shown in fig. 5 first segments the human body image to obtain a plurality of sub-region images of different human body parts, and the image analysis report processing device shown in fig. 5 performs semantic analysis on the text information in the image analysis report to determine the human body part with abnormality (i.e., "the key region and the corresponding information" shown in fig. 5), such as "brainstem: small plaque-like low density shadow, right temporal lobe: nodule-like slightly high-density shadow ", determining the transmission priority of each subregion image according to the human body parts with abnormality, namely determining the transmission priority of the subregion images of the human body parts such as the right temporal lobe and the brainstem, and for example, the human body parts such as the basal ganglia and the ventricles also need to be viewed preferentially due to some special reasons, so that the basal ganglia and the ventricles are also determined as the subregion images with preferential transmission, other human body parts without abnormality can determine the transmission sequence according to the sequence of the subregion images from left to right and from top to bottom in the whole image (namely, the 'second-priority transmission sequence' shown in fig. 5), and all the subregion images of the human body parts with abnormality are transmitted in preference to the subregion images of the human body parts without abnormality, and then all the subregion images can be added into a transmission queue according to the transmission priority, finally, each subregion image is transmitted to the display terminal according to the transmission priority, because the subregion images of human body parts including the brainstem, the right temporal lobe, the basal ganglia region, the ventricles and the like are transmitted preferentially, therefore, the transmission can be preferentially finished, namely the image can be preferentially displayed on the display terminal, the magnetic resonance imaging consultant can more quickly check the sub-area image of the abnormal human body part, the image processing efficiency can be improved, and the image importance of the sub-regions of other human body parts is lower, the magnetic resonance imaging consultant can (or can) look behind or not look at any more, after the transmission of all the images is completed, all the subarea images are still combined into a complete image, therefore, the waiting time of the magnetic resonance imaging consultant is saved, and the problem that the work of the magnetic resonance imaging consultant is influenced due to the fact that a certain part of the whole image is lost is solved.
Referring to fig. 1, fig. 1 is a flow chart illustrating the main steps of an image processing method according to an embodiment of the present invention. As shown in fig. 1, the image processing method in the embodiment of the present invention mainly includes the following steps S101 to S104.
Step S101: and carrying out image segmentation on the human body image so as to obtain a plurality of sub-region images of different human body parts.
The human body part refers to various tissues or organs of a human body, for example, a brain of the human body includes different tissues such as a brain lobe, a brain organ, gray matter and the like, so that if the human body image is the brain image, the brain image can be divided into a plurality of sub-region images of different human body parts according to the brain lobe, the brain organ, the gray matter and the like, and for example, an upper body organ of the human body includes different organs such as a heart, a liver, a lung and the like, so that if the human body image is the upper body image of the human body, the upper body image can be divided into a plurality of sub-region images of different human body parts according to the heart, the liver, the lung and the like.
In an implementation manner of this embodiment, step S101 may specifically include:
the method comprises the steps of determining the position of each human body part in a human body image, and carrying out image segmentation on the human body image according to the position to obtain a plurality of subarea images, wherein each subarea image at least comprises image information of one human body part.
In an embodiment of this embodiment, step S101 may also include steps one to two as follows:
the method comprises the following steps: and performing image semantic analysis on the human body image by adopting an image semantic segmentation model to obtain image division masks corresponding to the human body parts in the human body image.
Step two: and carrying out image segmentation on the human body image according to the position of the image segmentation mask in the human body image so as to obtain a plurality of subarea images, wherein each subarea image at least comprises image information of a human body part.
The mask refers to a specific image, graphic, or the like that is set to block an image to be processed (i.e., a human body image in the present embodiment) so as to perform a process of a certain region on the image to be processed, and the image division mask refers to a specific image, graphic, or the like that is set to divide the human body image into sub-region images of different human body parts.
In one embodiment of this embodiment, the image semantic segmentation model is obtained by training using an image semantic segmentation algorithm.
The image semantic segmentation algorithm may use deep learning, machine learning, a visual algorithm, and the like, for example, in this embodiment, a U-net (a deep neural network with an encoder-decoder structure) network may be used to construct an image semantic segmentation model, and the U-net network is trained by using a semantic segmentation algorithm based on machine learning to obtain the image semantic segmentation model. Specifically, the U-net network can be trained to obtain the image semantic segmentation model in the following way:
acquiring training data, inputting the training data into a U-net network, and predicting the predicted positions of image division masks of different human body parts in each human body image sample in the human body image sample through the U-net network, wherein the training data can comprise the human body image sample and human body image sample labels, and the human body image sample labels comprise the actual positions of the image division masks of different human body parts in the human body image sample;
according to the deviation between the actual position and the predicted position, carrying out back propagation to adjust the network parameters of the U-net network, and carrying out training again until the deviation between the predicted position predicted by the U-net network and the actual position is smaller than a preset difference threshold value or the training times reach a preset times threshold value, so as to obtain the trained U-net network;
and taking the trained U-net network as an image semantic segmentation model.
It should be noted that the U-net network is only an example, and this embodiment may also use a network with other structures to train to obtain the image semantic segmentation model, for example, the 3D _ U-net network may be used to train to obtain the image semantic segmentation model, and a training process of the 3D _ U-net network is similar to the training process of the U-net network, and is not described herein again for brevity of description.
For the image semantic segmentation model obtained by training the U-net network, the input human body image can be a two-dimensional human body image, and for the image semantic segmentation model obtained by training the 3D _ U-net network, the input human body image can be a three-dimensional human body image.
As shown in FIG. 6, 3 × 256 × 256_ image in FIG. 6 represents an input three-channel two-dimensional image, and Down conv1-4 (convolution layer) and max pool (maximum pooling layer) in the figure are Down sampling modules, and Up conv1-3
(convolution layer) and up sample are upsampling modules, and k × 256 × 256_ mask represents the output image partition mask. After a two-dimensional or three-dimensional human body image which is subjected to preprocessing (the preprocessing can be affine transformation, resampling, image intensity adjustment, normalization and the like) is obtained by the image semantic segmentation model (namely the trained U-net network or the trained 3D _ U-net network), the shallow layer and deep layer characteristics of the image are extracted by performing down-sampling and convolution operation on an encoder, then the characteristics are up-sampled and decoded by the decoder, and then the segmentation result of the image is obtained by combining the characteristics of the encoder and the decoder, so that the image division mask of the sub-region is obtained.
As shown in fig. 7, the image in the upper half of fig. 7 is a magnetic resonance imaging image of a brain, and the image in the lower half is a CT imaging image of a chest, the two human images are preprocessed, and then the human images are input into an image semantic segmentation model, which can divide a brain MRI scene schematic diagram into a plurality of sub-regions, such as a left frontal lobe, a right frontal lobe, a left temporal lobe, a right temporal lobe, a cerebellum, a brainstem, and the like, to obtain image division masks of the sub-regions, divide a chest bone structure into a spine, a rib, and the like, to obtain the image division masks of the sub-regions, and divide the rib into a plurality of sub-region masks according to connectivity, position coordinates, and the like of the mask images, so as to obtain the image division masks of the complete sub-regions. After the image division mask of the sub-region is obtained, the human body image can be subjected to image segmentation according to the position of the image division mask in the human body image so as to obtain a plurality of sub-region images.
Through the above step S101, the human body image may be divided into a plurality of sub-region images, so that the plurality of sub-region images may be transmitted, respectively.
Step S102: and acquiring an image analysis report for describing states of different human body parts in the human body image, and performing semantic analysis on text information in the image analysis report to determine the abnormal human body parts.
The image analysis report refers to an analysis result of analyzing normal or abnormal expression of a human body part in a human body image, and the analysis result may include text information, such as a piece of text and the like.
In one embodiment of this embodiment, the text information in the image analysis report may include status description information of the human body part; the specific steps of "performing semantic analysis on the text information in the image analysis report to determine the human body part with the abnormality" may include steps S201 to S204 as shown in fig. 2:
step S201: performing first semantic analysis on each state description information respectively to acquire words representing state description, words representing human body parts and words representing directions in the state description information; wherein the words indicating an orientation include at least words indicating a position and/or words indicating a direction.
The first semantic analysis may be an analysis of the whole sentence, such as separating one or more words representing state descriptions, words representing parts of the human body, words representing orientations, words representing positions, words representing directions, etc. from a certain sentence.
The words indicating state description refer to words for a certain body part to be normal or abnormal, such as "normal, abnormal, lump, bleeding" and the like, the words indicating a body part refer to words for a specific body part, such as "head, chest, shoulder" and the like, the words indicating orientation refer to words for describing a normal or abnormal position, the words indicating a position refer to words for describing a specific position to be normal or abnormal, such as "sulcus, cisterna" and the like, the words indicating orientation refer to words for describing an orientation of a normal or abnormal position, such as "left, right, inside, outside" and the like.
Step S202: and performing second semantic analysis on each word representing the state description to screen the words representing the abnormal state and take the state description information to which the words representing the abnormal state belong as the to-be-processed state description information.
The second semantic analysis may be a semantic analysis of words representing state descriptions, e.g. analyzing words of a certain description state from words of that description state representing normal or abnormal states.
The term indicating an abnormal state refers to a term for describing that the state of a certain body part of a person is abnormal, such as "abnormality, lump, bleeding, lesion" and the like.
Step S203: and searching the words which represent the orientations and are nearest to the words representing the abnormal state in the to-be-processed state description information.
The nearest neighbor means a nearest neighbor to a position in the state description information indicating an abnormal state, for example, "no abnormality occurs inside the left anterior rib 1, 2, 3, 4, and the cortical bone running inside the left anterior rib 5, 6 is concavely twisted", and in the state description information, a word indicating an orientation nearest to "the cortical bone running is concavely twisted" is "inside the left anterior rib 5, 6" instead of "inside the left anterior rib 1, 2, 3, 4".
When words representing the direction and nearest neighbor words representing abnormal states in the to-be-processed state description information are searched, whether the search result is reasonable can be judged according to punctuation marks in the image analysis report, for example, the bilateral pleural regions are thickened, and obvious effusion exists in the bilateral pleural regions. The cortex of the marginal cortex inside the 5 th and 6 th anterior ribs on the left side is twisted inwards. In the phrase "there is an obvious fluid accumulation in both thoracic cavities" in the search, the word indicating the orientation of the nearest neighbor is "left side 5 th and 6 th", but in the above sentence, "there is an obvious fluid accumulation in both thoracic cavities" and "left side 5 th and 6 th". "(i.e., period), so that the" 5 th, 6 th "can not be used as the word indicating the orientation of the nearest neighbor to the" obvious fluid accumulation in the two thoracic cavities ".
In addition, the image consultant can also preset a method for judging whether the search result is reasonable through the man-machine interaction device so as to accurately search the words which represent the direction and are nearest to the words which represent the abnormal state in the description information of the state to be processed. For example, in the image analysis report, the word indicating the orientation is usually located in front of the word indicating the human body part, that is, "the word indicating the orientation + the word indicating the human body part", and then the word indicating the orientation determined to be nearest to the word indicating the abnormal state according to the positional relationship between the word indicating the orientation and the word indicating the human body part in the image analysis report, for example, "the right temporal lobe sees a nodule-like slightly high-density shadow", and the searched word indicating the orientation is "down", but the position of "down" in the above sentence is located behind the "temporal lobe", and "right" is located in front of the "temporal lobe" in the above sentence, so that "right" is used as the word nearest to the word indicating the abnormal state (that is, "the nodule-like slightly high-density shadow").
It should be noted that the present embodiment only provides two specific embodiments for determining whether the search result is reasonable. However, it is understood by those skilled in the art that the protection scope of the present invention is not limited to these two specific embodiments, and those skilled in the art can modify or replace the above method for determining whether the search result is reasonable according to actual requirements without departing from the technical principle of the present invention, as long as the search result (the word representing the orientation nearest to the word representing the abnormal state) can be determined whether the search result is reasonable, and these modified or replaced technical solutions still fall within the protection scope of the present invention.
Step S204: and generating a keyword pair of the human body part with the abnormality according to the word which represents the direction and is nearest to the word representing the abnormal state, and the word representing the human body part and the word representing the state description in the to-be-processed state description information.
In this embodiment, a BERT-BiLSTM-CRF (named entity recognition) model may be used to perform a first semantic analysis, as shown in fig. 8, the state description information is input into the BERT-BiLSTM-CRF model (i.e., "input text" shown in fig. 8), the BERT layer in the BERT-BiLSTM-CRF model performs semantic analysis on the input text, the input text is processed into word vectors containing a plurality of words, and then the word vectors are input into the BiLSTM-CRF layer in the BERT-BiLSTM-CRF model, the word vectors are classified into a plurality of different types of words respectively through the BiLSTM layer in the BiLSTM-CRF layer (i.e., the Bidirectional-LSTM layer shown in fig. 8) and the CRF layer in the BiLSTM-BiLSTM model, and finally output labels are obtained, that is a final classification result is obtained, for example, a word is classified into a word representing an orientation, a certain word is classified into a word indicating a human body part, and the like.
The output category labels labeled as different words, such as a word representing a state expression, a word representing a human body part, a word representing an orientation, and the like.
After different parts of speech are obtained, performing second semantic analysis on the words representing the state description, screening out the words representing the abnormal state, and then taking the state description information to which the words representing the abnormal state belong as the state description information to be processed, wherein it needs to be noted that the words representing the state can be subjected to the second semantic analysis by using a keyword dictionary or by using a BERT layer in the BERT-BilSTM-CRF model, and in order to simplify the description, too much introduction is not performed here.
Words that indicate orientation may include words that indicate position and words that indicate direction, where words that indicate position may in some cases be numbers, such as 1 st, 2 nd, etc.
For example, the input state description information is:
bilateral pleura is thickened in a limited way, and no obvious hydrops is found in the pleural cavities of both sides. The cortex of the marginal cortex inside the 5 th and 6 th anterior ribs on the left side is twisted inwards.
The brainstem, bilateral basal ganglia, lateral ventricles and the center of the hemioval have a plurality of small-spot flaky low-density shadows, and the right temporal lobe has a nodule-like slightly high-density shadow with a diameter of about 9 mm. The third ventricle and the lateral ventricle are enlarged, the sulcus and the cisterna cerebri are widened, and the midline structure has no displacement.
The output result may be:
the words indicating orientation: left, (5 th, 6 th), right, third;
words representing human body parts: medial anterior costal, brainstem, (bilateral) basilar region, lateral ventricle, hemioval center, temporal lobe, ventricle, sulcus, lateral ventricle, cisterna;
words representing state descriptions: the cortex lycii radicis is in inward concave type distortion, multiple small spot sheet-shaped low-density shadow, nodular shape slightly high-density shadow (diameter 9 mm), enlargement and widening.
After the to-be-processed state description information is screened out, a word representing the orientation, such as the above-mentioned "left side, 5 th, 6 th", and the like, which is nearest to the word representing the abnormal state in the to-be-processed state description information may be searched.
And then generating a keyword pair of the human body part with the abnormality according to the word which represents the direction and is nearest to the word representing the abnormal state, and the word representing the human body part and the word representing the state description in the to-be-processed state description information.
For example, the keyword pair of the human body part with the abnormality may be generated according to the output result:
(left, 5 th and 6 th) the inner side of the anterior rib, i.e., the cortical bone, is twisted in an inward concave manner.
Brainstem, (bilateral) basal ganglia, lateral ventricle, central hemioval-multiple small spot lamellar low density ghost.
(Right) temporal lobe-nodular-like slightly high density shadow [ diameter 9mm ].
Third ventricle, lateral ventricle-enlargement.
The sulcus and cisterna cerebri-widen.
Through the steps S201 to S204, semantic analysis may be performed on the text information in the image analysis report to obtain a keyword pair of the human body part with the abnormality.
Step S103: determining the transmission priority of each subarea image according to the abnormal human body part, and respectively transmitting each subarea image to a preset image display terminal according to the transmission priority, so that the subarea image corresponding to the abnormal human body part can be transmitted to the image display terminal to be displayed in priority to other subarea images.
In one embodiment of this embodiment, the specific steps of "determining the transmission priority of each subregion image according to the human body part having the abnormality" include steps S301 to S305 as shown in fig. 3:
step S301: associating the keyword pair with the subregion image by:
for each keyword pair, each human body part represented by a word representing the human body part in the keyword pair is determined, and the keyword pairs are associated with respective sub-region images corresponding to each human body part.
Step S302: and taking the subregion image associated with the keyword pair as a first subregion image to be processed, and taking the subregion image not associated with the keyword pair as a second subregion image to be processed.
Step S303: calculating the priority score of each first subregion image to be processed according to a method shown in the following formula (1):
Figure BDA0003190291680000141
wherein G isiRepresenting the priority score, C, of the ith first subregion image to be processediA focus coefficient, n, representing a preset ith first subregion image to be processediIndicating the number of the associated keyword pairs of the ith first subarea image to be processed, LjAnd the abnormal degree coefficient corresponding to the word representing the state description in the jth keyword pair associated with the preset ith first subarea image to be processed is represented, N represents the total number of the keyword pairs generated by the semantic analysis module, and alpha and beta represent preset constant coefficients.
The attention coefficient in the formula (1) refers to an attention coefficient which is matched based on preset attention coefficients corresponding to different human body parts and according to a human body part corresponding to the first to-be-processed subregion image. In this embodiment, the preset attention coefficients corresponding to different human body parts may be pre-input by an image viewer of the human body image through a human-computer interaction device, so that the image viewer sets different attention coefficients for different human body parts according to respective actual attention requirements, and the first to-be-processed sub-region image of the human body part with a higher attention coefficient can be preferentially transmitted.
The abnormality degree coefficient in the formula (1) refers to an abnormality degree coefficient corresponding to each of words described based on preset different expression states, and is matched according to the words described in the expression states in the keyword pair associated with the first subregion image to be processed. In this embodiment, the preset abnormality degree coefficients corresponding to the words described in different representation states may be input in advance by an image viewer of the human body image through the human-computer interaction device, for example, the image viewer may first determine the respective abnormality degrees corresponding to the words described in different representation states based on a medical theory, and then set different abnormality degree coefficients for the words described in different representation states according to the respective abnormality degrees, where the abnormality degree coefficient and the abnormality degree have a positive correlation, so that the first to-be-processed subregion image of the human body part with a higher abnormality degree coefficient can be preferentially transmitted.
Step S304: and determining the transmission priority of each first to-be-processed subarea image according to the priority scores.
Step S305: and sequentially determining the transmission priority of each second to-be-processed subregion image according to the position of each second to-be-processed subregion image in the human body image and a preset position sequence.
The priority degree of the transmission priority of the first to-be-processed subarea image is higher than that of the transmission priority of the second to-be-processed subarea image.
For example, the above-mentioned "sulci, brain pool — broadening" keyword pair is associated with the subregion image corresponding to the sulci and the brain pool, then the subregion image corresponding to the sulci and the brain pool is used as the first to-be-processed subregion image, and the subregion image not associated with the keyword pair is used as the second to-be-processed image.
In the above example, the subregion image associated with "sulcus, brain pool-broadening" is a subregion image in which there is an abnormality, and therefore, it is considered that transmission is performed preferentially as the first subregion image to be processed.
Then, a priority score of each first sub-region image to be processed is calculated, for example, the priority score of each sub-region image may be determined according to the method of the above formula (1), where the attention coefficient may be set according to human tissues, organs, and the like included in the sub-region image, for example, a human image of a brain CT, and one attention coefficient may be set for each of the left and right frontal lobes, the left and right temporal lobes, the cerebellum, and the brainstem, and the setting may be set according to a possibility of a lesion occurring in a certain tissue or organ, or according to a specific requirement of an image viewer, for example, the image viewer needs to view the left frontal lobe first, and then the left frontal lobe may be set as a higher attention coefficient.
For the degree of abnormality coefficient, it may be determined according to the state descriptions contained in the keyword pair, for example, words of state descriptions with higher degree of abnormality are set to higher coefficients, such as "canceration, hemorrhage, lumen peduncle", and the like, and words of state descriptions with lower degree of abnormality are set to lower coefficients, such as "suspicious, probable, broadened", and the like.
After the priority scores of all the first to-be-processed subarea images are determined, the transmission priority of the second to-be-processed subarea images can be determined according to the position of each second to-be-processed subarea image in the human body image and a preset position sequence, for example, the transmission priority can be determined according to the sequence of the second to-be-processed subarea images in the human body image from left to right and from top to bottom, and it should be noted that the priority degrees of the transmission priority degrees of all the first to-be-processed subarea images are higher than the priority degrees of the transmission priority degrees of the second to-be-processed subarea images.
Through the steps S301 to S305, the transmission priority of each sub-area image can be determined, and the sub-area images transmitted preferentially can be transmitted in sequence according to the transmission priority, so that the sub-area images transmitted preferentially can be transmitted and finished earlier, and can be displayed on the picture of the image display terminal in advance, and the waiting time of image consultants is saved.
In one embodiment of this embodiment, the specific step of "determining the transmission priority of each subregion image according to the human body part having the abnormality" includes the following steps S401 to S403:
step S401: for each keyword pair, each human body part represented by a word representing the human body part in the keyword pair is determined, and the keyword pairs are associated with respective sub-region images corresponding to each human body part.
Step S402: and taking the subregion image associated with the keyword pair as a first subregion image to be processed, and taking the subregion image not associated with the keyword pair as a second subregion image to be processed.
Step S403: and sequentially determining the transmission priority of each first to-be-processed subarea image according to the position of each first to-be-processed subarea image in the human body image and a preset position sequence. And sequentially determining the transmission priority of each second to-be-processed subregion image according to the position of each second to-be-processed subregion image in the human body image and a preset position sequence. The priority degree of the transmission priority of the first to-be-processed subarea image is higher than that of the transmission priority of the second to-be-processed subarea image.
Step S104: and synthesizing all the transmitted sub-region images into a new human body image and displaying the new human body image through an image display terminal.
After all the images are transmitted, all the subarea images can be combined into a complete image, so that an image looking up person can still look up a complete human body image after looking up the subarea images with abnormality preferentially.
Based on the steps S101 to S104, the human body image can be segmented to obtain a plurality of sub-region images of different human body parts, then the image analysis report is based on the states of different human body parts in the human body image, and the text information in the image analysis report is subjected to semantic analysis to determine the human body parts with abnormalities, the transmission priority of each sub-region image is determined according to the human body parts with abnormalities, each sub-region image is transmitted according to the transmission priority and displayed on the image display terminal, since the sub-region images of the human body parts with abnormalities are transmitted preferentially, the sub-region images of the human body parts with abnormalities can be displayed on the image display terminal preferentially, image consultants can look up the sub-region images of the human body parts with abnormalities preferentially without waiting for the transmission of all images to be completed, i.e. the image processing work efficiency is improved, finally, after the transmission of all the subarea images is finished, the subarea images still converge into a complete new human body image, and the normal work of image consultants cannot be influenced.
It should be noted that, although the foregoing embodiments describe each step in a specific sequence, those skilled in the art will understand that, in order to achieve the effect of the present invention, different steps do not necessarily need to be executed in such a sequence, and they may be executed simultaneously (in parallel) or in other sequences, and these changes are all within the protection scope of the present invention.
Furthermore, the invention also provides an image processing system.
Referring to fig. 4, fig. 4 is a main configuration block diagram of an image processing system according to an embodiment of the present invention. As shown in fig. 4, the image processing system in the embodiment of the present invention mainly includes an image dividing means 41, an image analysis report processing means 42, an image transmission means 43, and an image composition display means 44. In some embodiments, one or more of the image segmentation means 41, the image analysis report processing means 42, the image transmission means 43, and the image composition display means 44 may be incorporated together into one module. In some embodiments the image segmentation means 41 may be configured to image segment the image of the body to obtain images of sub-regions of a plurality of different body parts. The image analysis report processing device 42 may be configured to obtain an image analysis report describing the status of different human body parts in the human body image, and perform semantic analysis on text information in the image analysis report to determine the human body part with the abnormality. The image transmission device 43 may be configured to determine a transmission priority of each subregion image according to the abnormal human body part, and respectively transmit each subregion image to a preset image display terminal according to the transmission priority, so that the subregion image corresponding to the abnormal human body part can be transmitted to the image display terminal to be displayed in preference to other subregion images. The image composition display device 44 may be configured to, after the transmission of all the subarea images is completed, synthesize all the subarea images into a new human body image and display the new human body image through the image display terminal. In one embodiment, the description of the specific implementation function may refer to the description of step S101 to step S104.
In one embodiment, the image segmentation apparatus comprises a first image segmentation module and/or a second image segmentation module;
the first image segmentation module is configured to determine the position of each human body part in the human body image, and perform image segmentation on the human body image according to the position to obtain a plurality of subarea images, wherein each subarea image at least comprises image information of one human body part;
the second image segmentation module is configured to perform the following operations:
performing image semantic analysis on the human body image by adopting an image semantic segmentation model to obtain image division masks corresponding to human body parts in the human body image;
and carrying out image segmentation on the human body image according to the position of the image segmentation mask in the human body image so as to obtain a plurality of subarea images, wherein each subarea image at least comprises image information of a human body part.
In one embodiment, the textual information in the image analysis report includes state descriptive information of the human body part; the image analysis report processing apparatus includes a semantic analysis module configured to perform the following operations:
performing first semantic analysis on each state description information respectively to acquire words representing state description, words representing human body parts and words representing directions in the state description information; wherein the words indicating orientation at least include words indicating position and/or words indicating direction;
performing second semantic analysis on each word representing the state description to screen the words representing the abnormal state and take the state description information to which the words representing the abnormal state belong as the state description information to be processed;
searching the words which represent the direction and are nearest to the words which represent the abnormal state in the description information of the state to be processed;
and generating a keyword pair of the human body part with the abnormality according to the word which represents the direction and is nearest to the word representing the abnormal state, and the word representing the human body part and the word representing the state description in the to-be-processed state description information. In one embodiment, the description of the specific implementation function may refer to the descriptions of step S201 to step S204.
In one embodiment, an image transmission apparatus includes a transmission priority determination module configured to perform the following operations:
associating the keyword pair with the subregion image by:
for each keyword pair, determining each human body part represented by the words representing the human body part in the keyword pair, and associating the keyword pair with the sub-region images respectively corresponding to each human body part;
taking the subregion image associated with the keyword pair as a first subregion image to be processed, and taking the subregion image not associated with the keyword pair as a second subregion image to be processed;
respectively calculating the priority score of each first subarea image to be processed according to a method shown as the following formula:
Figure BDA0003190291680000181
wherein G isiRepresenting the priority score, C, of the ith first subregion image to be processediA focus coefficient, n, representing a preset ith first subregion image to be processediIndicating the number of the associated keyword pairs of the ith first subarea image to be processed, LjThe abnormal degree coefficient corresponding to a word representing state description in a jth keyword pair associated with a preset ith first subarea image to be processed is represented, N represents the total number of the keyword pairs generated by the semantic analysis module, and alpha and beta represent preset constant coefficients;
determining the transmission priority of each first to-be-processed subregion image according to the priority score;
determining the transmission priority of each second to-be-processed subregion image in sequence according to the position of each second to-be-processed subregion image in the human body image and a preset position sequence;
the priority degree of the transmission priority of the first to-be-processed subarea image is higher than that of the transmission priority of the second to-be-processed subarea image. In one embodiment, the description of the specific implementation function may refer to the descriptions of step S301 to step S305.
In one embodiment, the image semantic segmentation model is obtained by training through an image semantic segmentation algorithm.
For the above-mentioned image processing system to be used for executing the embodiment of the image processing method shown in fig. 1, the technical principles, the solved technical problems and the generated technical effects of the two are similar, and it can be clearly understood by those skilled in the art that for convenience and brevity of description, the specific working process and related descriptions of the image processing system may refer to the content described in the embodiment of the image processing method, and no further description is given here.
It will be understood by those skilled in the art that all or part of the flow of the method according to the above-described embodiment may be implemented by a computer program, which may be stored in a computer-readable storage medium and used to implement the steps of the above-described embodiments of the method when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying said computer program code, media, usb disk, removable hard disk, magnetic diskette, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunication signals, software distribution media, etc. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
Further, it should be understood that, since the configuration of each device is only for explaining the functional units of the device of the present invention, the corresponding physical devices of these devices may be the processor itself, or a part of software, a part of hardware, or a part of a combination of software and hardware in the processor. Thus, the number of individual devices in the figures is merely illustrative.
Those skilled in the art will appreciate that the various devices may be adaptively split or combined. Such splitting or combining of specific devices does not cause the technical solutions to deviate from the principle of the present invention, and therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.

Claims (10)

1. An image processing system, characterized in that the method comprises:
image segmentation means configured to image-segment the human body image to obtain sub-region images of a plurality of different human body parts;
the image analysis report processing device is configured to acquire an image analysis report for describing states of different human body parts in the human body image, and perform semantic analysis on text information in the image analysis report to determine the human body parts with the abnormality;
the image transmission device is configured to determine the transmission priority of each subregion image according to the abnormal human body part and respectively transmit each subregion image to a preset image display terminal according to the transmission priority so that the subregion image corresponding to the abnormal human body part can be transmitted to the image display terminal to be displayed in priority to other subregion images;
and the image synthesis display device is configured to synthesize all the subarea images into a new human body image and display the new human body image through the image display terminal after all the subarea images are transmitted.
2. The image processing system according to claim 1, wherein the image segmentation means comprises a first image segmentation module and/or a second image segmentation module;
the first image segmentation module is configured to determine a position of each human body part in the human body image, and perform image segmentation on the human body image according to the position to obtain a plurality of subarea images, wherein each subarea image at least comprises image information of one human body part;
the second image segmentation module is configured to perform the following operations:
performing image semantic analysis on the human body image by adopting an image semantic segmentation model to obtain image division masks corresponding to each human body part in the human body image;
and carrying out image segmentation on the human body image according to the position of the image division mask in the human body image so as to obtain a plurality of subarea images, wherein each subarea image at least comprises image information of a human body part.
3. The image processing system of claim 1, wherein the textual information in the image analysis report includes state descriptive information of a human body part; the image analysis report processing apparatus includes a semantic analysis module configured to perform the following operations:
performing first semantic analysis on each state description information respectively to acquire words representing state description, words representing human body parts and words representing directions in the state description information; wherein the words representing orientation include at least words representing position and/or words representing direction;
performing second semantic analysis on each word representing the state description to screen the word representing the abnormal state and take the state description information to which the word representing the abnormal state belongs as the state description information to be processed;
searching a word representing the direction in the to-be-processed state description information, wherein the word representing the direction is nearest to the word representing the abnormal state;
and generating a keyword pair of the human body part with the abnormality according to the word which represents the orientation and is nearest to the word representing the abnormal state, and the word representing the human body part and the word representing the state description in the to-be-processed state description information.
4. The image processing system according to claim 3, wherein the image transmission apparatus includes a transmission priority determination module configured to perform operations of:
associating the keyword pair with the subregion image by:
for each keyword pair, determining each human body part represented by a word representing the human body part in the keyword pair, and associating the keyword pair with each sub-region image corresponding to each human body part;
taking the subregion image associated with the keyword pair as a first subregion image to be processed, and taking the subregion image not associated with the keyword pair as a second subregion image to be processed;
respectively calculating the priority score of each first subarea image to be processed according to a method shown as the following formula:
Figure FDA0003190291670000021
wherein, G isiRepresenting the priority score of the ith first subregion image to be processed, CiA attention coefficient representing a preset ith first subarea image to be processed, wherein n isiIndicating the number of the associated keyword pairs of the ith first subarea image to be processed, LjRepresenting an abnormal degree coefficient corresponding to a word representing state description in a jth keyword pair associated with a preset ith first subarea image to be processed, wherein N represents the total number of the keyword pairs generated by the semantic analysis module, and alpha and beta represent preset constant coefficients;
determining the transmission priority of each first to-be-processed subregion image according to the priority score;
according to the position of each second subregion image to be processed in the human body image and a preset position sequence, sequentially determining the transmission priority of each second subregion image to be processed;
the priority degree of the transmission priority of the first to-be-processed subarea image is higher than that of the transmission priority of the second to-be-processed subarea image.
5. The image processing system of claim 2, wherein the image semantic segmentation model is trained using an image semantic segmentation algorithm.
6. An image processing method, characterized in that the method comprises:
carrying out image segmentation on the human body image to obtain a plurality of subarea images of different human body parts;
acquiring an image analysis report for describing states of different human body parts in the human body image, and performing semantic analysis on text information in the image analysis report to determine the abnormal human body parts;
determining the transmission priority of each subregion image according to the abnormal human body part, and respectively transmitting each subregion image to a preset image display terminal according to the transmission priority, so that the subregion image corresponding to the abnormal human body part can be transmitted to the image display terminal to be displayed in priority to other subregion images;
and synthesizing all the transmitted sub-region images into a new human body image and displaying the new human body image through the image display terminal.
7. The image processing method according to claim 6, wherein the specific step of performing image segmentation on the human body image to obtain the sub-region images of a plurality of different human body parts comprises:
determining the position of each human body part in the human body image, and performing image segmentation on the human body image according to the position to obtain a plurality of sub-region images, wherein each sub-region image at least comprises image information of one human body part;
and/or the specific steps of performing image segmentation on the human body image to acquire a plurality of subarea images of different human body parts comprise:
performing image semantic analysis on the human body image by adopting an image semantic segmentation model to obtain image division masks corresponding to each human body part in the human body image;
and carrying out image segmentation on the human body image according to the position of the image division mask in the human body image so as to obtain a plurality of subarea images, wherein each subarea image at least comprises image information of a human body part.
8. The image processing method of claim 6, wherein the text information in the image analysis report includes state description information of a human body part; the specific steps of performing semantic analysis on the text information in the image analysis report to determine the human body part with the abnormality comprise:
performing first semantic analysis on each state description information respectively to acquire words representing state description, words representing human body parts and words representing directions in the state description information; wherein the words representing orientation comprise at least text words representing position and/or words representing direction;
performing second semantic analysis on each word representing the state description to screen the word representing the abnormal state and take the state description information to which the word representing the abnormal state belongs as the state description information to be processed;
searching a word representing the direction in the to-be-processed state description information, wherein the word representing the direction is nearest to the word representing the abnormal state;
and generating a keyword pair of the human body part with the abnormality according to the word which represents the orientation and is nearest to the word representing the abnormal state, and the word representing the human body part and the word representing the state description in the to-be-processed state description information.
9. The image processing method according to claim 8, wherein the step of determining the transmission priority of each of the sub-region images according to the abnormal human body part comprises:
associating the keyword pair with the subregion image by:
for each keyword pair, determining each human body part represented by a word representing the human body part in the keyword pair, and associating the keyword pair with each sub-region image corresponding to each human body part;
taking the subregion image associated with the keyword pair as a first subregion image to be processed, and taking the subregion image not associated with the keyword pair as a second subregion image to be processed;
respectively calculating the priority score of each first subarea image to be processed according to a method shown as the following formula:
Figure FDA0003190291670000041
wherein, G isiRepresenting the priority score of the ith first subregion image to be processed, CiA attention coefficient representing a preset ith first subarea image to be processed, wherein n isiIndicating the number of the associated keyword pairs of the ith first subarea image to be processed, LjRepresenting an abnormal degree coefficient corresponding to a word representing state description in a jth keyword pair associated with a preset ith first subarea image to be processed, wherein N represents the total number of the keyword pairs generated by the semantic analysis module, and alpha and beta represent preset constant coefficients;
determining the transmission priority of each first to-be-processed subregion image according to the priority score;
according to the position of each second subregion image to be processed in the human body image and a preset position sequence, sequentially determining the transmission priority of each second subregion image to be processed;
the priority degree of the transmission priority of the first to-be-processed subarea image is higher than that of the transmission priority of the second to-be-processed subarea image.
10. The image processing system of claim 7, wherein the image semantic segmentation model is trained using an image semantic segmentation algorithm.
CN202110875128.2A 2021-07-30 Image processing system and method Active CN113592819B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110875128.2A CN113592819B (en) 2021-07-30 Image processing system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110875128.2A CN113592819B (en) 2021-07-30 Image processing system and method

Publications (2)

Publication Number Publication Date
CN113592819A true CN113592819A (en) 2021-11-02
CN113592819B CN113592819B (en) 2024-07-26

Family

ID=

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116152687A (en) * 2023-04-21 2023-05-23 深圳市慧明捷科技有限公司 Unmanned aerial vehicle data acquisition module

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102385756A (en) * 2010-08-31 2012-03-21 佳能株式会社 Image processing apparatus and control method thereof
CN108847276A (en) * 2018-03-20 2018-11-20 青岛海信医疗设备股份有限公司 The processing method and server of medical image based on the network architecture
CN109271992A (en) * 2018-09-26 2019-01-25 上海联影智能医疗科技有限公司 A kind of medical image processing method, system, device and computer readable storage medium
CN110363138A (en) * 2019-07-12 2019-10-22 腾讯科技（深圳）有限公司 Model training method, image processing method, device, terminal and storage medium
CN110648338A (en) * 2019-09-25 2020-01-03 上海微创医疗器械（集团）有限公司 Image segmentation method, readable storage medium, and image processing apparatus
CN111916184A (en) * 2020-07-31 2020-11-10 平安好医投资管理有限公司 Medical examination image downloading method and device and computer equipment
CN112241966A (en) * 2020-09-21 2021-01-19 西北大学 Method and system for establishing and segmenting multitask and multi-classification chest organ segmentation model
CN112330640A (en) * 2020-11-09 2021-02-05 平安科技（深圳）有限公司 Segmentation method, device and equipment for nodule region in medical image
CN112382360A (en) * 2020-12-03 2021-02-19 卫宁健康科技集团股份有限公司 Automatic generation system of diagnosis report, storage medium and electronic equipment

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102385756A (en) * 2010-08-31 2012-03-21 佳能株式会社 Image processing apparatus and control method thereof
CN108847276A (en) * 2018-03-20 2018-11-20 青岛海信医疗设备股份有限公司 The processing method and server of medical image based on the network architecture
CN109271992A (en) * 2018-09-26 2019-01-25 上海联影智能医疗科技有限公司 A kind of medical image processing method, system, device and computer readable storage medium
CN110363138A (en) * 2019-07-12 2019-10-22 腾讯科技（深圳）有限公司 Model training method, image processing method, device, terminal and storage medium
CN110648338A (en) * 2019-09-25 2020-01-03 上海微创医疗器械（集团）有限公司 Image segmentation method, readable storage medium, and image processing apparatus
CN111916184A (en) * 2020-07-31 2020-11-10 平安好医投资管理有限公司 Medical examination image downloading method and device and computer equipment
CN112241966A (en) * 2020-09-21 2021-01-19 西北大学 Method and system for establishing and segmenting multitask and multi-classification chest organ segmentation model
CN112330640A (en) * 2020-11-09 2021-02-05 平安科技（深圳）有限公司 Segmentation method, device and equipment for nodule region in medical image
CN112382360A (en) * 2020-12-03 2021-02-19 卫宁健康科技集团股份有限公司 Automatic generation system of diagnosis report, storage medium and electronic equipment

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116152687A (en) * 2023-04-21 2023-05-23 深圳市慧明捷科技有限公司 Unmanned aerial vehicle data acquisition module

Similar Documents

Publication Publication Date Title
WO2020233272A1 (en) Medical image processing method and apparatus, electronic medical device, and storage medium
WO2021036616A1 (en) Medical image processing method, medical image recognition method and device
Zhou et al. A multi-modality fusion network based on attention mechanism for brain tumor segmentation
CN111524106B (en) Skull fracture detection and model training method, device, equipment and storage medium
CN110070540B (en) Image generation method and device, computer equipment and storage medium
CN112712906B (en) Video image processing method, device, electronic equipment and storage medium
US20220335600A1 (en) Method, device, and storage medium for lesion segmentation and recist diameter prediction via click-driven attention and dual-path connection
CN110991254B (en) Ultrasonic image video classification prediction method and system
US20230143452A1 (en) Method and apparatus for generating image, electronic device and storage medium
WO2022242392A1 (en) Blood vessel image classification processing method and apparatus, and device and storage medium
WO2021139351A1 (en) Image segmentation method, apparatus, medium, and electronic device
KR20210098381A (en) Device and method for visualizing image of lesion
CN112819819A (en) Pneumoconiosis grading method, device, medium and equipment based on deep learning
CN110363776B (en) Image processing method and electronic device
CN114004854A (en) System and method for processing and displaying slice image under microscope in real time
CN113592819A (en) Image processing system and method
Xu et al. Correlation via synthesis: end-to-end nodule image generation and radiogenomic map learning based on generative adversarial network
CN113592819B (en) Image processing system and method
CN111667469A (en) Lung disease classification method, device and equipment
WO2021197176A1 (en) Systems and methods for tumor characterization
WO2022252107A1 (en) Disease examination system and method based on eye image
CN114764788A (en) Intracranial arterial stenosis detection method and system
CN114341996A (en) Disease analysis method based on VRDS 4D and related product
Xu et al. Correlation via synthesis: End-to-end image generation and radiogenomic learning based on generative adversarial network
Pérez-García et al. RadEdit: stress-testing biomedical vision models via diffusion image editing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant