CN109771052A

CN109771052A - 3-D image method for building up and system based on the imaging of more mesh and multi-polarization state imaging

Info

Publication number: CN109771052A
Application number: CN201811625313.0A
Authority: CN
Inventors: 舒远; 李梓彤; 陈维宇; 刘楚明; 阮思纯; 徐炜文; 王星泽
Original assignee: Hop Edge Technology (shenzhen) Co Ltd
Current assignee: Hop Edge Technology (shenzhen) Co Ltd
Priority date: 2018-12-28
Filing date: 2018-12-28
Publication date: 2019-05-21
Anticipated expiration: 2038-12-28
Also published as: CN109771052B

Abstract

This application provides a kind of 3-D image method for building up being imaged based on the imaging of more mesh and multi-polarization state and systems, the described method includes: broad-spectrum light source generates broadband wide spectrum light, spectrum imaging device multi-angle acquires the wide spectrum light by the operative region reflects and is formed multiple narrow-band spectrum images, the preliminary three-dimensional reconstruction image of the operative region is constructed using multi-view stereo vision algorithm, multi-polarization state imaging device acquires the polarization light by the operative region reflects and is formed multiple narrow-band polarization images, obtain the body fluid covering image of the human organ in the operative region, processing system covers image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ, obtain the three-dimensional reconstruction image of the operative region, and it exports to display equipment.

Description

3-D image method for building up and system based on the imaging of more mesh and multi-polarization state imaging

Technical field

This application involves electronic fields more particularly to a kind of 3-D image based on the imaging of more mesh and multi-polarization state imaging to build Cube method and system.

Background technique

As attention degree of the public to health is higher and higher and the continuous promotion of medical level, surgical operation is also more and more normal See.In surgical operation, the success rate and effect of operation and the accuracy of operation are closely related, and traditional operation is all by doctor Raw experience is visually judged that accuracy is not high to operation tool and lesion locations, and the requirement to doctor is also very It is high.

But the precision of surgical operation how is improved, it is a still unsolved problem so far.

Summary of the invention

This application provides 3-D image method for building up and system based on the imaging of more mesh and multi-polarization state imaging, Neng Gouyou Improve the precision of surgical operation in effect ground.

In a first aspect, a kind of 3-D image method for building up based on the imaging of more mesh and multi-polarization state imaging is provided, it is described Method the following steps are included:

Broad-spectrum light source generates broadband wide spectrum light, wherein the wide spectrum light is described wide for irradiating operative region Composing light includes available light and polarization light, and the operative region is the body region that patient is performed the operation；

Spectrum imaging device multi-angle acquires the wide spectrum light by the operative region reflects and is formed multiple narrow waves Section spectrum picture, the preliminary three-dimensional reconstruction image of the operative region is constructed using multi-view stereo vision algorithm, wherein different Narrow-band spectrum image has different imaging narrow-bands, and the quantity of the imaging narrow-band is 100 or more；

Multi-polarization state imaging device acquires the polarization light by the operative region reflects and is formed multiple narrow-bands Polarization image obtains the body fluid covering image of the human organ in the operative region；

Processing system covers image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ, obtains institute The three-dimensional reconstruction image of operative region is stated, and is exported to display equipment.

Optionally, the method also includes: the processing system is according to multiple narrow-band polarization images, combining classification model To the body fluid components type and organ type progress instant analysis in the operative region and analysis is exported as a result, described instant Analysis includes organ floristic analysing, kinds of protein analysis, protein content analysis, crystallization floristic analysing and crystalline content point Analysis, wherein the disaggregated model is the body fluid classification mould being trained in advance using sample data to convolutional neural networks Type and organ classes' model；

Determine whether protein content or crystalline content in the analysis result are greater than first threshold or less than the Two threshold values, wherein the first threshold is greater than the second threshold；

Protein content or crystalline content in the analysis result are greater than the first threshold or are less than described In the case where second threshold, prompt information is issued.

Optionally, after the three-dimensional reconstruction image for obtaining the operative region, the method also includes:

The rendering that the three-dimensional reconstruction image carries out target area is strengthened, the three-dimensional reconstruction image after output reinforcing, In, the target area includes the specific region in the region being blocked in operative region and operative region.

Optionally, include: by the rendering reinforcing that the three-dimensional reconstruction image carries out target area

In the case where target area is the region being blocked in operative region, the position of occlusion removal target area, wash with watercolours Dye the target area not being blocked；Alternatively,

In the case where target area is the specific region in operative region, the brightness of the specific region is promoted, is rendered Target area after luminance raising out.

Optionally, the target area is the region that processing system is determined according to the location information that operation tool is sent, In, it is provided with gyroscope in the operation tool, accelerometer, one or more in range sensor, the position letter Breath is one or more collected positional number according to the gyroscope, in the accelerometer and the range sensor According to what is be calculated；Alternatively,

The target area is that user requests determining region to processing system based on the received.

Second aspect is provided a kind of be imaged based on more mesh and establishes system with the 3-D image of multi-polarization state imaging, described System includes:

Broad-spectrum light source, broad-spectrum light source generate broadband wide spectrum light, wherein the wide spectrum light is for irradiating field of operation Domain, the wide spectrum light include available light and polarization light, and the operative region is the body region that patient is performed the operation；

Spectrum imaging device acquires the wide spectrum light for multi-angle and is reflected and formed multiple by the operative region Narrow-band spectrum image constructs the preliminary three-dimensional reconstruction image of the operative region using multi-view stereo vision algorithm, wherein no Same narrow-band spectrum image has different imaging narrow-bands, and the quantity of the imaging narrow-band is 100 or more；

Multi-polarization state imaging device is reflected and formed multiple narrow by the operative region for acquiring the wide spectrum light Wave band polarization image obtains the body fluid covering image of the human organ in the operative region；

Processing system, processing system is according to the preliminary three-dimensional reconstruction image and the body fluid coverage diagram of the human organ Picture obtains the three-dimensional reconstruction image of the operative region, and exports to display equipment.

Optionally, the processing system is also used to according to multiple narrow-band polarization images, and combining classification model is to the hand Body fluid components type and organ type in art region carry out instant analysis and export analysis as a result, the instant analysis includes Organ floristic analysing, kinds of protein analysis, protein content analysis, crystallization floristic analysing and crystalline content analysis, wherein The disaggregated model is the body fluid disaggregated model and organ being trained in advance using sample data to convolutional neural networks Disaggregated model；

The processing system is also used to determine whether the analysis result is greater than first threshold or is less than second threshold, In, the first threshold is greater than the second threshold；

The processing system is also used to be greater than the first threshold in the analysis result or is less than the second threshold In the case where, issue prompt information.

Optionally, after the processing system is also used to obtain the three-dimensional reconstruction image of the operative region, by the three-dimensional The rendering that reconstruction image carries out target area is strengthened, the three-dimensional reconstruction image after output reinforcing, wherein the target area includes Specific region in the region being blocked in operative region and operative region.

Optionally, the processing system is specifically used for the case where target area is the region being blocked in operative region Under, the position of occlusion removal target area renders the target area not being blocked；Alternatively,

The processing system is specifically used in the case where target area is the specific region in operative region, described in promotion The brightness of specific region, the target area after rendering luminance raising.

3-D image method for building up and system provided by the present application based on the imaging of more mesh and multi-polarization state imaging, by wide It composes light source and generates broadband wide spectrum light, it is anti-by the operative region that spectrum imaging device multi-angle acquires the wide spectrum light The multiple narrow-band spectrum images penetrated and formed construct the preliminary three-dimensional weight of the operative region using multi-view stereo vision algorithm Image is built, it is inclined that multi-polarization state imaging device acquires multiple narrow-bands that the polarization light is reflected and formed by the operative region Shake image, the body fluid covering image of the human organ in the operative region is obtained, so that processing system is according to described primary three The body fluid for tieing up reconstruction image and the human organ covers image, obtains the three-dimensional reconstruction image of the operative region, and defeated Out to display equipment.By obtaining richer effective letter according to multiple narrow-band spectrum images and multiple narrow-band polarization images Breath, to reconstruct the higher 3-D image of accuracy, also, can also be by virtually enhancing technology to the three-dimensional figure after reconstruction As carrying out rendering reinforcing according to doctor's demand, so that guidance or auxiliary doctor correctly perform the operation, the precision of operation is improved.

Detailed description of the invention

Fig. 1 is a kind of stream of 3-D image method for building up based on the imaging of more mesh and multi-polarization state imaging provided by the present application Journey schematic diagram；

Fig. 2 is light in a kind of 3-D image method for building up based on the imaging of more mesh and multi-polarization state imaging provided by the present application It composes light source and arranges schematic diagram；

Fig. 3 is a kind of schematic diagram that prompt information is carried out by way of augmented reality provided by the present application；

Fig. 4 a is the operative region reconstruction image before a kind of augmented reality provided by the present application；

Fig. 4 b is the operative region reconstruction image after a kind of augmented reality provided by the present application；

Fig. 5 a is the operative region reconstruction image before another augmented reality provided by the present application；

Fig. 5 b is the operative region reconstruction image after another augmented reality provided by the present application；

Fig. 6 is the knot that a kind of 3-D image based on the imaging of more mesh and multi-polarization state imaging provided by the present application establishes system Structure schematic diagram.

Specific embodiment

Refering to fig. 1, Fig. 1 is a kind of three-dimensional figure based on the imaging of more mesh and multi-polarization state imaging provided by the embodiments of the present application As the flow diagram of method for building up.The 3-D image foundation side based on the imaging of more mesh and multi-polarization state imaging of the present embodiment Method includes the following steps:

S101: broad-spectrum light source generates broadband wide spectrum light, wherein and the wide spectrum light is used to irradiate operative region, The wide spectrum light includes available light and polarization light, and the operative region is the body region that patient is performed the operation.

In the application specific embodiment, broad-spectrum light source is for generating broadband wide spectrum light.Broad-spectrum light source produces The range of raw wavelength can extend to ultraviolet from infrared, visible light be extended to from infrared, alternatively, extending to from visible light ultraviolet Etc..In practical applications, the quantity of broad-spectrum light source can be one or more.In a specific embodiment, Duo Geguang The range of the wave band for the wide spectrum light that spectrum light source generates all be it is identical, for example, the wide spectrum light that the first broad-spectrum light source generates Wave-length coverage be 350 nanometers to 850 nanometers, the second broad-spectrum light source generate wide spectrum light wave-length coverage be also 350 nanometers extremely 850 nanometers etc..In another specific embodiment, the range of the wave band for the wide spectrum light that multiple broad-spectrum light sources generate all is phase With, for example, the wave-length coverage for the wide spectrum light that the first broad-spectrum light source generates is 350 nanometers to 550 nanometers, the second broad-spectrum light source The wave-length coverage of the wide spectrum light of generation is also 550 nanometers to 850 nanometers etc..

In the above example, it is all illustrated so that the wide spectrum light that broad-spectrum light source generates is continuous broadband as an example, In practical application, the wide spectrum light that broad-spectrum light source generates is also possible to discontinuous broadband, for example, the first broad-spectrum light source generates The wave-length coverage of wide spectrum light be 350 nanometers to 450 nanometers, the wave-length coverage of wide spectrum light that the first broad-spectrum light source generates is It 650 nanometers to 850 nanometers etc., is not especially limited herein.

In the application specific embodiment, polarization light be wide spectrum light is obtained by the polarizer, that is, It says, the polarizer is placed below a part of broad-spectrum light source for obtaining polarization light, is not put below another part broad-spectrum light source The polarizer is set for obtaining available light, it is to be understood that polarization light is for multi-polarization state imaging device acquisition polarization figure Picture, available light acquire spectrum picture for spectrum imaging device.

In the application specific embodiment, the arrangement mode of broad-spectrum light source can be horizontally arranged, be also possible to hang down In line column can also be that inclination angle arranges, can also be the combination of various arrangement mode, for example, Fig. 2 is that the embodiment of the present application provides Broad-spectrum light source arrangement mode, wherein the polarizer is placed with below A group light source not to be had for obtaining polarised light, B group light source lower section The polarizer, A1, B1 light source are vertical irradiation array of source, and A2, B2, A3, B3 are inclination angle light source, are separately positioned on A1, B1 light source Two sides, and with horizontal direction be in θ angle, it is to be understood that due to collected biological tissue under the light beam of different angle Characteristic information is different, therefore light source arrangement mode shown in Fig. 2 is conducive to sufficiently excavate biological information, it should be appreciated that Fig. 2 institute The broad-spectrum light source arrangement mode shown is only used for for example, specific restriction can not be constituted, and the broad-spectrum light source in the application can be with Possess more perpendicular light source arrays and inclination angle array of source.

In the application specific embodiment, the operative region is the body region that patient is performed the operation, without Entire operating table, operative region may include: the operation tool that patient operative site, the operative site performed the operation are placed and Operation label.Wherein, operative site can position to need to be performed an operation with patient, and the position that needs to be performed an operation Neighbouring position.Operation tool can be scalpel, surgical scissors, hook for surgery operation and dilator etc..Operation label can be It is convenient for the operation marker of positioning, is not especially limited herein.

S102: spectrum imaging device multi-angle acquires the wide spectrum light and is reflected and formed multiple by the operative region Narrow-band spectrum image constructs the preliminary three-dimensional reconstruction image of the operative region using multi-view stereo vision algorithm, wherein no Same narrow-band spectrum image has different imaging narrow-bands, and the quantity of the imaging narrow-band is 100 or more.

In the application specific embodiment, spectrum imaging device can form multiple narrow-band spectrum images.Wherein, Spectrum imaging device can be more mesh spectrum imaging devices that are multiple and being multi-angle arrangement, is also possible to high light spectrum image-forming and sets Standby and depth camera combination, spectrum imaging device can also be high light spectrum image-forming equipment or hyperspectral imaging apparatus etc. Deng the application is not especially limited.Wherein, high light spectrum image-forming equipment is with hundreds of to thousands of continuous and subdivision spectral band pair Operative region is imaged simultaneously, and hyperspectral imaging apparatus is same to operative region with thousands of to tens of thousands of continuous and subdivision spectral bands When be imaged.Certainly, spectrum imaging device can also be imaged simultaneously operative region with more continuous and subdivision spectral band, Such as tens of thousands of extremely hundreds of thousands of continuous and subdivision spectral bands do not limit operative region imaging etc. simultaneously specifically herein It is fixed.

In the application specific embodiment, spectrum imaging device can use the SPECTRAL DIVERSITY of body surface ingredient, Target is identified and is classified, that is to say, that different narrow-band spectrum images has different imaging narrow-bands, will be different The narrow-band spectrum image that angle shot obtains is integrated, and the broader visual field can be obtained, thus can be obtained richer Information, recycle multi-view stereo vision algorithm, three-dimensional reconstruction may be implemented.For example, the skin of human body can be preferably It reflects a and narrow-band is imaged, the muscle of human body can preferably reflect b imaging narrow-band, and the blood vessel of human body can preferably reflect c Narrow-band is imaged, human skeleton can preferably reflect d imaging narrow-band.Therefore, narrow-band can be imaged according to a to obtain instead The A narrow-band spectrum image of human skin situation is answered, the B that narrow-band acquisition reaction human muscle's situation can be imaged according to b is narrow The C narrow-band spectrum image that narrow-band obtains reaction human vas situation, Ke Yigen can be imaged according to c in band spectrum image The D narrow-band spectrum image of reaction skeleton situation is obtained according to d imaging narrow-band.It is appreciated that when spectral band subdivision Degree is higher, then the visual field of spectrum imaging device is more wide, and the information that spectrum generating device obtains is also naturally abundanter, acquisition Human body information is also more acurrate, and then realizes more accurate three reconstruction models.

Preferably, binocular stereo vision matching (Sum of Absolute can be used in spectrum imaging device Differences, SAD) algorithm carry out operative region three-dimensional reconstruction, that is to say, that obtained using binocular high light spectrum image-forming equipment Different angle narrow-band spectrum image, the difference of the corresponding numerical value of pixel in each image is taken absolute value summation, with The similarity degree of the narrow-band spectrum image of this assessment different angle, to obtain disparity map.After obtaining parallax information, according to The depth information and three-dimensional information of original image can be obtained in projection model.Certainly, spectrum imaging device can also use block With (Block Matching, BM) algorithm, half global Block- matching (Semi-Global Block Matching, SGM) algorithm etc. Other multi-view stereo vision algorithms carry out the preliminary three-dimensional reconstruction of operative region, and the application is not especially limited.

S103: multi-polarization state imaging device acquires the polarization light and is reflected and formed multiple narrow by the operative region Wave band polarization image obtains the body fluid covering image of the human organ in the operative region.

In the application specific embodiment, multi-polarization state imaging device can form multiple narrow-band polarization images. Wherein, multi-polarization state imaging device can be one or more, multi-polarization state imaging device can be orthogonal multi-polarization state at As equipment or interference multi-polarization state imaging device etc..Wherein, orthogonal multi-polarization state imaging device is with hundreds of to thousands of companies Operative region is imaged in continuous and subdivision spectral band simultaneously, interfere multi-polarization state imaging device with it is thousands of to tens of thousands of it is a continuously and Operative region is imaged in the spectral band of subdivision simultaneously.Certainly, multi-polarization state imaging device can also be with more continuous and thin The polarised light wave band opponent's art region divided is imaged simultaneously, such as tens of thousands of to hundreds of thousands of polarised light wave bands pair that are continuous and segmenting Operative region imaging etc. simultaneously, is not especially limited herein.

In the application specific embodiment, the corresponding imaging narrow-band of narrow-band polarization image and narrow-band spectrum figure As corresponding imaging narrow-band can be it is identical, such as: the blood vessel of human body can preferably reflect c imaging narrow-band, therefore The C narrow-band spectrum image and C narrow-band polarization image that narrow-band obtains reaction human vas situation can be imaged according to c. The corresponding imaging narrow-band of narrow-band polarization image imaging narrow-band corresponding with narrow-band spectrum image can also be not identical , such as: the blood vessel of human body can preferably reflect c1 imaging narrow-band, preferable to reflect c2 polarization imaging narrow-band, therefore The C1 narrow-band spectrum image that narrow-band obtains reaction human vas situation can be imaged according to C1, according to the narrow wave of C2 polarization imaging Section obtains the C2 narrow-band polarization image of reaction human vas situation.It should be understood that the above citing is merely to illustrate, can not constitute It is specific to limit.

In the application specific embodiment, multi-polarization state imaging device can be according in multiple narrow-band polarization images The angle of polarization, degree of polarization and the ellipticity angle of light beam obtain the polarization characteristic parameter of operative region, provide the organ table of operative region The signs such as surface roughness, texture trend, surface orientation, water content, compare spectrum imaging device, and multi-polarization state imaging device can be with To organ contours and surface orientation identification provide more accurate information, can protrude the organ in operative region in picture with back The contrast of scape, and then improve the accuracy rate next identified with organ classes' model, body fluid disaggregated model.Therefore, the application The three-dimensional reconstruction that operative region is carried out in conjunction with spectrum imaging device and multi-polarization state imaging device, can provide completeer for doctor The information that whole, more reliable and human eye can not extract.Operation region entirety is carried out that is, spectrum imaging device can be Feature extraction realizes that the preliminary three-dimensional of operative region is rebuild using multi-view stereo vision algorithm, and multi-polarization state imaging device can be with It is the organ surface roughness that can not be shown to spectrum imaging device, grain of meat trend, the class of liquids of organ surface covering Equal profiles and surface carry out feature extraction, obtain the human organ clearly described in operative region surface roughness or The image of body fluid coverage condition.

S104: processing system covers image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ, The three-dimensional reconstruction image of the operative region is obtained, and is exported to display equipment.

In presently filed embodiment, processing system is inclined according to the multiple narrow-band spectrum image and multiple narrow-bands The 3-D image of operative region described in picture construction of shaking.That is, processing system combines narrow-band spectrum image and narrow wave The information that section polarization image extracts respectively, establishes an operative region steric information figure detailed, abundant.Such as: processing system System can be according to the 3-D image A of the A narrow-band spectrum image reconstruction skin of different angle, according to the B narrow-band of different angle Spectrum picture rebuilds the 3-D image B of muscle, according to the 3-D image C of the C narrow-band spectrum image reconstruction blood vessel of different angle, According to the 3-D image D of the D narrow-band spectrum image reconstruction bone of different angle, is polarized and schemed according to the E narrow-band of different angle 3-D image E as rebuilding capilary rebuilds each organ surface Covering Liguid according to the F narrow-band polarization image of different angle 3-D image F, by 3-D image A, 3-D image B, 3-D image C, 3-D image D, 3-D image E and 3-D image F After being merged, so that it may which acquisition can reflect skin, muscle, blood vessel, bone, capilary, organ surface degree of roughness simultaneously Detailed, true operative region 3-D image.

In the application specific embodiment, the method also includes: the processing system is inclined according to multiple narrow-bands Shake image, combining classification model in the operative region body fluid components type and organ type carry out instant analysis and defeated Analysis is as a result, the instant analysis includes kinds of protein analysis, protein content analysis, crystallization floristic analysing and crystallization out Content analysis, wherein the disaggregated model is the body fluid being trained in advance using sample data to convolutional neural networks Disaggregated model and organ classes' model；Determine whether protein content or crystalline content in the analysis result are greater than first Threshold value is less than second threshold, wherein the first threshold is greater than the second threshold；Albumen in the analysis result In the case that matter content or crystalline content are greater than the first threshold or are less than the second threshold, prompt information is issued. Wherein, body fluid disaggregated model and organ classes' model can be in advance using the body fluid picture sample for being largely labeled with prediction result And organ sample, convolutional neural networks are trained with the trained disaggregated model of acquisition, the body fluid disaggregated model can With the narrow-band polarization image comprising the aqueous measure feature of organ surface obtained according to multi-polarization state imaging system, body fluid point is obtained Class is as a result, organ classes' model can be according to the narrow comprising organ surface contour feature of multi-polarization state imaging system acquisition Wave band polarization image obtains organ classification results, and exports in the reality enhancing or mixed reality equipment that doctor wears, in hand More information are provided for doctor during art, further to judge the state of an illness and to perform the operation.Wherein, prompt information can be One of sound, light or text.For example, as shown in figure 3, can be existed by the method for augmented reality or mixed reality Increasing text on 3-D image, content of hemoglobin is abnormal in blood samples of patients during surgical to remind.Also, more polarizations State imaging device can also carry out the judgement of organ classes in conjunction with neural network model, can also will sentence according to polarization optical information Determine result and text is increased on 3-D image to remind doctor by the method for augmented reality or mixed reality, the application does not make It repeats.

In the application specific embodiment, processing system is according to the multiple narrow-band spectrum image and multiple narrow waves The 3-D image that section polarization image constructs the operative region specifically may is that processing system by the operative site, the hand Art tool and operation label are positioned and divide to obtain locating segmentation as a result, to true according to locating segmentation result Relative position between the fixed operative site, the operation tool and the operation label, then, further according to the Surgery Relative position reconstruction of three-dimensional images between position, the operation tool and the operation label.It is appreciated that passing through multiple narrow waves Section spectrum picture and multiple narrow-band polarization images can obtain information more abundant, therefore, it is possible to more accurately to institute Operative site, the operation tool and operation label is stated to be positioned and divided.

In the application specific embodiment, processing system is by the operative site, the operation tool and the hand Art label positioned and divided may is that obtaining locating segmentation result specifically obtain multiple narrow-band spectrum images and After multiple narrow-band polarization images, processing system can be extracted multiple with feature extraction algorithm (for example, convolutional neural networks) The textural characteristics of narrow-band spectrum image and multiple narrow-band polarization images.Then, in conjunction with image recognition algorithm (for example, volume Product neural network) and movement tracing algorithm operation consent is done label, operation tool with perform the operation relevant range be accurately positioned Or segmentation, to obtain locating segmentation result.

In the application specific embodiment, after the three-dimensional reconstruction image for obtaining the operative region, the method is also It include: to strengthen the rendering that the three-dimensional reconstruction image carries out target area, the three-dimensional reconstruction image after output reinforcing, wherein The target area includes the specific region in the region being blocked in operative region and operative region.By the three-dimensional reconstruction It includes: to move in the case where target area is the region being blocked in operative region that the rendering that image carries out target area, which is strengthened, Except the position for blocking the target area, the target area not being blocked is rendered；Alternatively, being in operative region in target area Specific region in the case where, promote the brightness of the specific region, the target area after rendering luminance raising.

Preferably, the target area not being blocked rendered can be to be obtained by using the trained supplementary model that blocks , the supplementary model that blocks is to use the image of the shelter in a large amount of binocular images and binocular image as sample, It is obtained after being trained to the convolutional neural networks (SymmNet) of symmetrical structure, the supplementary model that blocks has learnt largely Feature is blocked in binocular image, the occlusion issue of more mesh spectrum imaging device images can be predicted by bilateral symmetry Network model.More operation informations can be brought for the operation of doctor using the result after supplementary model rendering is strengthened is blocked, For example, certain operative region is as shown in fig. 4 a, in the case that target area is the region being blocked in operative region, doctor is used Reality enhancing (Augmented Reality, AR) glasses can see the three-dimensional reconstruction figure after rendering as shown in Figure 4 b is strengthened Picture, and performed the operation according to the three-dimensional reconstruction image after reinforcing.It is understood that strong in no rendering for carrying out target area Before change, doctor can only rule of thumb perform the operation, and after the rendering for carrying out target area is strengthened, AR is can be used in doctor The progress that glasses are performed the operation according to the target area image not being blocked rendered, auxiliary or guidance.It should be understood that the example above is only For illustrating, the rendering of occlusion area can also use other algorithms to obtain, such as use cross inspection (Left- Right-cross-checking, LRC) algorithm infers the position blocked by the parallax result precalculated, the application is not Make specific limit.

Preferably, the target area after rendering luminance raising can be to be obtained using trained brightness supplementary model , the brightness supplementary model is to be trained using the operative region samples pictures before and after luminance raising to convolutional neural networks Obtained model, the result after being strengthened using the rendering of brightness supplementary model can bring more operation informations for the operation of doctor, For example, as shown in Figure 5 a, when target area be operative region in specific region in the case where, for example, certain lesion regions or The region that these doctors of person shadow region can not see by visual observation, doctor can see using after AR glasses such as Fig. 5 b institute Three-dimensional reconstruction image after the rendering reinforcing shown, and performed the operation according to the three-dimensional reconstruction image after reinforcing.It is understood that Before no rendering for carrying out target area is strengthened, doctor can only rule of thumb perform the operation, and having carried out, target area is bright After degree is promoted, AR glasses are can be used according to the target area image after the luminance raising rendered, auxiliary or guidance in doctor The progress of operation.

In the application specific embodiment, the target area is the position that processing system is sent according to operation tool The region that information determines, wherein be provided with gyroscope in the operation tool, accelerometer, one in range sensor or Person is multiple, the location information be one according to the gyroscope, in the accelerometer and the range sensor or What multiple collected position datas were calculated.Such as: when surgical knife is placed on a certain lesion region, the AR of doctor Glasses can show the three-dimensional reconstruction image after the lesion region highlights automatically.It should be understood that the above citing is merely to illustrate, not Specific limit can be constituted.

In the application specific embodiment, the target area can also be processing system, and user is asked based on the received Ask determining region.Such as: doctor needs to check in the course of surgery a certain target area being blocked, and doctor can lead at this time It crosses and selects what needs were checked by using other terminal device frames being connected with processing system using AR equipment or feldsher The target area being blocked, processing system determine target area according to the target area information received, mesh are rendered not In the AR glasses for the target area image input doctor being blocked.It should be understood that the above citing is merely to illustrate, tool can not be constituted Body limits.

In the application specific embodiment, the three-dimensional reconstruction image after output reinforcing, which can be, is output to doctor's wearing Head-mounted display apparatus display screen, such as AR glasses, mixed reality (Mediated Reality, MR) glasses etc.；It can also To be to be output to terminal device display screen, such as computer display screen, panel display screen etc.；It can also be and be output to manipulator In art system, operation is carried out for mechanical arm, operation guidance or auxiliary are provided.

In above scheme, broad-spectrum light source generates broadband wide spectrum light, and the acquisition of spectrum imaging device multi-angle is described wide Light is composed by multiple narrow-band spectrum images that the operative region reflects and is formed, constructs institute using multi-view stereo vision algorithm The preliminary three-dimensional reconstruction image of operative region is stated, it is anti-by the operative region that multi-polarization state imaging device acquires the polarization light The multiple narrow-band polarization images penetrated and formed obtain the body fluid covering image of the human organ in the operative region, so that Processing system covers image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ, obtains the field of operation The three-dimensional reconstruction image in domain, and export to display equipment.It, can be according to multiple narrow-band spectrum images and more by the above method A narrow-band polarization image obtains richer effective information, to reconstruct the higher 3-D image of accuracy, also, may be used also To carry out rendering reinforcing according to doctor's demand to the 3-D image after reconstruction by virtually enhancing technology, thus guidance or auxiliary doctor Life is correctly performed the operation, and the precision of operation is improved.

It is that a kind of 3-D image based on the imaging of more mesh and multi-polarization state imaging provided by the present application is established referring to Fig. 6, Fig. 6 The structural schematic diagram of system.In the embodiment of the present application, the 3-D image based on the imaging of more mesh and multi-polarization state imaging establishes system System includes: broad-spectrum light source 110, spectrum imaging device 120, multi-polarization state imaging device 130 and processing system 140.

Broad-spectrum light source 110 is for generating broadband wide spectrum light, wherein the wide spectrum light is for irradiating field of operation Domain, the wide spectrum light include available light and polarization light, and the operative region is the body region that patient is performed the operation；

Spectrum imaging device 120 acquires the wide spectrum light for multi-angle and is reflected and formed more by the operative region A narrow-band spectrum image constructs the preliminary three-dimensional reconstruction image of the operative region using multi-view stereo vision algorithm, wherein Different narrow-band spectrum images has different imaging narrow-bands, and the quantity of the imaging narrow-band is 100 or more；

Multi-polarization state imaging device 130 is reflected and formed multiple by the operative region for acquiring the wide spectrum light Narrow-band polarization image obtains the body fluid covering image of the human organ in the operative region；

Processing system 140 is used for the body fluid coverage diagram according to the preliminary three-dimensional reconstruction image and the human organ Picture obtains the three-dimensional reconstruction image of the operative region, and exports to display equipment.

In the application specific embodiment, broad-spectrum light source 110 is for generating broadband wide spectrum light.Broad-spectrum light source The range of the wavelength of generation can extend to ultraviolet from infrared, visible light be extended to from infrared, alternatively, extending to purple from visible light It is outer etc..In practical applications, the quantity of broad-spectrum light source 110 can be one or more.In a specific embodiment, more The range of the wave band for the wide spectrum light that a broad-spectrum light source 110 generates all be it is identical, for example, the wide spectrum that the first broad-spectrum light source generates The wave-length coverage of light is 350 nanometers to 850 nanometers, and the wave-length coverage for the wide spectrum light that the second broad-spectrum light source generates also is 350 Nanometer is to 850 nanometers etc..In another specific embodiment, the range of the wave band for the wide spectrum light that multiple broad-spectrum light sources generate It is all identical, for example, the wave-length coverage for the wide spectrum light that the first broad-spectrum light source generates is 350 nanometers to 550 nanometers, second is wide The wave-length coverage for composing the wide spectrum light that light source generates also is 550 nanometers to 850 nanometers etc..

In the above example, it is all illustrated so that the wide spectrum light that broad-spectrum light source 110 generates is continuous broadband as an example, In practical applications, the wide spectrum light that broad-spectrum light source 110 generates is also possible to discontinuous broadband, for example, the first broad-spectrum light The wave-length coverage for the wide spectrum light that source generates is 350 nanometers to 450 nanometers, the wavelength for the wide spectrum light that the first broad-spectrum light source generates Range is 650 nanometers to 850 nanometers etc., is not especially limited herein.

In the application specific embodiment, polarization light be wide spectrum light is obtained by the polarizer, that is, It says, the polarizer is placed below a part of broad-spectrum light source 110 for obtaining polarization light, under another part broad-spectrum light source 110 The polarizer is not placed for obtaining available light in side, it is to be understood that polarization light is acquired for multi-polarization state imaging device Polarization image, available light acquire spectrum picture for spectrum imaging device.

In the application specific embodiment, the arrangement mode of broad-spectrum light source 110 can be horizontally arranged, be also possible to Vertical arrangement can also be that inclination angle arranges, can also be the combination of various arrangement mode, for example, Fig. 2 is that the embodiment of the present application mentions The arrangement mode of the broad-spectrum light source of confession, wherein be placed with the polarizer below A group light source for obtaining polarised light, do not have below B group light source There is the polarizer, A1, B1 light source are vertical irradiation array of source, and A2, B2, A3, B3 are inclination angle light source, are separately positioned on A1, B1 light The two sides in source, and be in θ angle with horizontal direction, it is to be understood that due to biology group collected under the light beam of different angle Characteristic information difference is knitted, therefore light source arrangement mode shown in Fig. 2 is conducive to sufficiently excavate biological information, it should be appreciated that Fig. 2 Shown in broad-spectrum light source arrangement mode be only used for for example, specific restriction can not be constituted, the broad-spectrum light source in the application can To possess more perpendicular light source arrays and inclination angle array of source.

In the application specific embodiment, spectrum imaging device 120 can form multiple narrow-band spectrum images.Its In, spectrum imaging device 120 can be more mesh spectrum imaging devices that are multiple and being multi-angle arrangement, be also possible to EO-1 hyperion The combination of imaging device and depth camera, spectrum imaging device 120 can also be high light spectrum image-forming equipment or ultraphotic spectrum at As equipment etc., the application is not especially limited.Wherein, high light spectrum image-forming equipment is with hundreds of to thousands of continuous and subdivision light Operative region is imaged in spectrum wave band simultaneously, and hyperspectral imaging apparatus is with thousands of to tens of thousands of continuous and subdivision spectral band opponents Art region is imaged simultaneously.Certainly, spectrum imaging device can also be with more continuous and subdivision spectral band to operative region It is imaged simultaneously, such as tens of thousands of to hundreds of thousands of continuous and subdivision spectral bands are imaged operative region etc. simultaneously, herein It is not especially limited.

In the application specific embodiment, spectrum imaging device can use the SPECTRAL DIVERSITY of body surface ingredient, Target is identified and is classified, that is to say, that different narrow-band spectrum images has different imaging narrow-bands, will be different The narrow-band spectrum image that angle shot obtains is integrated, and the broader visual field can be obtained, thus can be obtained richer Information, recycle multi-view stereo vision algorithm, three-dimensional reconstruction may be implemented.For example, the skin of human body can be preferably It reflects a and narrow-band is imaged, the muscle of human body can preferably reflect b imaging narrow-band, and the blood vessel of human body can preferably reflect c Narrow-band is imaged, human skeleton can preferably reflect d imaging narrow-band.Therefore, narrow-band can be imaged according to a to obtain instead The A narrow-band spectrum image of human skin situation is answered, the B that narrow-band acquisition reaction human muscle's situation can be imaged according to b is narrow The C narrow-band spectrum image that narrow-band obtains reaction human vas situation, Ke Yigen can be imaged according to c in band spectrum image The D narrow-band spectrum image of reaction skeleton situation is obtained according to d imaging narrow-band.It is appreciated that when spectral band subdivision Degree is higher, then the visual field of spectrum imaging device 120 is more wide, and the information that spectrum generating device 120 obtains is also naturally abundanter, The human body information of acquisition is also more acurrate, and then realizes more accurate three reconstruction models.

In the application specific embodiment, multi-polarization state imaging device 130 can form multiple narrow-band polarization figures Picture.Wherein, multi-polarization state imaging device 130 can be one or more, and multi-polarization state imaging device 130 can be orthogonal more Polarization state imaging device or interference multi-polarization state imaging device etc..Wherein, orthogonal multi-polarization state imaging device with it is hundreds of extremely Operative region is imaged in thousands of continuous and subdivision spectral band simultaneously, interferes multi-polarization state imaging device with thousands of to tens of thousands of Operative region is imaged in a continuous and subdivision spectral band simultaneously.Certainly, multi-polarization state imaging device can also be with more Continuous and subdivision polarised light wave band opponent's art region is imaged simultaneously, such as tens of thousands of to hundreds of thousands of continuous and subdivision polarizations Optical band is not especially limited operative region imaging etc. simultaneously herein.

In the application specific embodiment, multi-polarization state imaging device 130 can be polarized according to multiple narrow-bands schemes The angle of polarization of light beam, degree of polarization and ellipticity angle obtain the polarization characteristic parameter of operative region as in, such as provide operative region Organ surface roughness, texture trend, the signs such as surface orientation, water content, compare spectrum imaging device 120, multi-polarization state Imaging device 130 can provide more accurate information to organ contours and surface orientation identification, the organ in prominent operative region In picture with the contrast of background, and then improve next with organ classes' model, body fluid disaggregated model identify accuracy rate. Therefore, the application combination spectrum imaging device 120 and multi-polarization state imaging device 130 carry out the three-dimensional reconstruction of operative region, can To provide the information that more complete, more reliable and human eye can not extract for doctor.That is, spectrum imaging device 120 can be with It is that feature extraction is carried out to operation region entirety, realizes that the preliminary three-dimensional of operative region is rebuild using multi-view stereo vision algorithm, Multi-polarization state imaging device 130 can be the organ surface roughness that spectrum imaging device can not be shown, grain of meat trend, It is clearly described in the profiles such as the class of liquids of organ surface covering and surface orientation progress feature extraction acquisition operative region The surface roughness of human organ or the image of body fluid coverage condition.

In presently filed embodiment, processing system 140 is according to the multiple narrow-band spectrum image and multiple narrow waves Section polarization image constructs the 3-D image of the operative region.That is, processing system combine narrow-band spectrum image and The information that narrow-band polarization image extracts respectively establishes an operative region steric information figure detailed, abundant.Such as: place Reason system can be narrow according to the B of different angle according to the 3-D image A of the A narrow-band spectrum image reconstruction skin of different angle The 3-D image B of band spectrum image reconstruction muscle, according to the three-dimensional figure of the C narrow-band spectrum image reconstruction blood vessel of different angle It is inclined according to the E narrow-band of different angle according to the 3-D image D of the D narrow-band spectrum image reconstruction bone of different angle as C The 3-D image E of vibration image reconstruction capilary rebuilds each organ surface according to the F narrow-band polarization image of different angle and covers The 3-D image F of liquid, by 3-D image A, 3-D image B, 3-D image C, 3-D image D, 3-D image E and three-dimensional figure After being merged as F, so that it may which acquisition can reflect the coarse journey of skin, muscle, blood vessel, bone, capilary, organ surface simultaneously Detailed, the true operative region 3-D image of degree.

In the application specific embodiment, the processing system 140 is also used to according to multiple narrow-band polarization images, Combining classification model is to the body fluid components type and organ type progress instant analysis in the operative region and exports analysis As a result, the instant analysis includes organ floristic analysing, kinds of protein analysis, protein content analysis, crystallization floristic analysing And crystalline content analysis, wherein the disaggregated model is to be trained in advance using sample data to convolutional neural networks The body fluid disaggregated model and organ classes' model arrived；The processing system is also used to determine whether the analysis result is greater than first Threshold value is less than second threshold, wherein the first threshold is greater than the second threshold；The processing system is also used in institute Analysis result is stated to be greater than the first threshold or less than in the case where the second threshold, issue prompt information.Wherein, body fluid Disaggregated model and organ classes' model can be in advance using the body fluid picture sample and organ for being largely labeled with prediction result Sample is trained the trained disaggregated model of acquisition to convolutional neural networks, and the body fluid disaggregated model can be according to more The narrow-band polarization image comprising the aqueous measure feature of organ surface that polarization state imaging system obtains obtains body fluid classification results, The narrow-band comprising organ surface contour feature that organ classes' model can be obtained according to multi-polarization state imaging system is inclined Shake image, obtains organ classification results, and exports in the reality enhancing or mixed reality equipment that doctor wears, in surgical procedure In for doctor provide more information, further to judge the state of an illness and to perform the operation.Wherein, prompt information can be sound, One of light or text.For example, as shown in figure 3, text can be increased on 3-D image by the method for augmented reality Content of hemoglobin is abnormal in blood samples of patients during surgical to remind for word.Also, multi-polarization state imaging device can be with According to polarization optical information, the judgement of organ classes is carried out in conjunction with neural network model, can also pass through augmented reality or mixing The method of reality increases text on 3-D image to remind doctor, and the application does not repeat.

In the application specific embodiment, processing system 140 is according to the multiple narrow-band spectrum image and multiple The 3-D image that narrow-band polarization image constructs the operative region specifically may is that processing system by the operative site, institute It states operation tool and operation label is positioned and divides to obtain locating segmentation as a result, to according to locating segmentation knot Fruit determines the relative position between the operative site, the operation tool and the operation label, then, further according to the hand Relative position reconstruction of three-dimensional images between art position, the operation tool and the operation label.It is appreciated that by multiple Narrow-band spectrum image and multiple narrow-band polarization images can obtain information more abundant, therefore, it is possible to more accurately The operative site, the operation tool and the operation label are positioned and divided.

In the application specific embodiment, processing system 140 is by the operative site, the operation tool and described Operation label, which is positioned and divided to obtain locating segmentation result specifically and may is that, is obtaining multiple narrow-band spectrum images After multiple narrow-band polarization images, processing system can be extracted more with feature extraction algorithm (for example, convolutional neural networks) The textural characteristics of a narrow-band spectrum image and multiple narrow-band polarization images.Then, in conjunction with image recognition algorithm (for example, Convolutional neural networks) and movement tracing algorithm label, operation tool and operation relevant range that operation consent is done carry out it is accurately fixed Position or segmentation, to obtain locating segmentation result.

In the application specific embodiment, optionally, the processing system is also used to obtain the operative region After three-dimensional reconstruction image, the rendering that the three-dimensional reconstruction image carries out target area is strengthened, the three-dimensional reconstruction after output reinforcing Image, wherein the target area includes the specific region in the region being blocked in operative region and operative region.It is described Processing system is specifically used in the case where target area is the region being blocked in operative region, occlusion removal target area Position renders the target area not being blocked；Alternatively, the processing system is specifically used in target area being in operative region Specific region in the case where, promote the brightness of the specific region, the target area after rendering luminance raising.

Preferably, the target area not being blocked rendered can be to be obtained by using the trained supplementary model that blocks , the supplementary model that blocks is to use the image of the shelter in a large amount of binocular images and binocular image as sample, It is obtained after being trained to the convolutional neural networks (SymmNet) of symmetrical structure, the supplementary model that blocks has learnt largely Feature is blocked in binocular image, the occlusion issue of more mesh spectrum imaging device images can be predicted by bilateral symmetry Network model.More operation informations can be brought for the operation of doctor using the result after supplementary model rendering is strengthened is blocked, For example, certain operative region is as shown in fig. 4 a, in the case that target area is the region being blocked in operative region, doctor is used Reality enhancing (AR) glasses can see the three-dimensional reconstruction image after rendering as shown in Figure 4 b is strengthened, and according to three after reinforcing Dimension reconstruction image is performed the operation.It is understood that doctor can only basis before no rendering for carrying out target area is strengthened Experience is performed the operation, and after the rendering for carrying out target area is strengthened, AR glasses are can be used according to rendering not in doctor The progress of the target area image being blocked, auxiliary or guidance operation.It should be understood that the example above is merely to illustrate, occlusion area Rendering other algorithms can also be used to obtain, such as use cross inspection (Left-right-cross- Checking, LRC) algorithm infers the position blocked by the parallax result precalculated, the application is not especially limited.

In the application specific embodiment, the target area is that processing system 140 is sent according to operation tool The region that location information determines, wherein it is provided with gyroscope in the operation tool, accelerometer, one in range sensor A or multiple, the location information is one according to the gyroscope, in the accelerometer and the range sensor Or multiple collected position datas are calculated.Such as: when surgical knife is placed on a certain lesion region, doctor AR glasses can show the three-dimensional reconstruction image after the lesion region highlights automatically.It should be understood that the above citing is merely to illustrate, Specific restriction can not be constituted.

In the application specific embodiment, the target area can also be that processing system 140 is used based on the received The determining region of family request.Such as: doctor needs to check in the course of surgery a certain target area being blocked, and doctor can at this time It is looked into selecting needs by using other terminal device frames being connected with processing system by using AR equipment or feldsher The target area being blocked seen, processing system determine target area according to the target area information received, mesh are rendered Be not blocked target area image input doctor AR glasses in.It should be understood that the above citing is merely to illustrate, it can not structure It is limited at specific.

In the application specific embodiment, the three-dimensional reconstruction image after the output reinforcing of processing system 140 can be defeated The head-mounted display apparatus display screen (such as AR glasses, MR glasses etc.) for arriving doctor's wearing out, is also possible to be output to terminal Device display screen (such as computer display screen, panel display screen etc.), can also be and be output in mechanical operation system, be machine Tool arm carries out operation and provides operation guidance or auxiliary.

In above system, broad-spectrum light source generates broadband wide spectrum light, and the acquisition of spectrum imaging device multi-angle is described wide Light is composed by multiple narrow-band spectrum images that the operative region reflects and is formed, constructs institute using multi-view stereo vision algorithm The preliminary three-dimensional reconstruction image of operative region is stated, it is anti-by the operative region that multi-polarization state imaging device acquires the polarization light The multiple narrow-band polarization images penetrated and formed obtain the body fluid covering image of the human organ in the operative region, so that Processing system covers image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ, obtains the field of operation The three-dimensional reconstruction image in domain, and export to display equipment.It, can be according to multiple narrow-band spectrum images and more by above system A narrow-band polarization image obtains richer effective information, to reconstruct the higher 3-D image of accuracy, also, passes through Virtual enhancing technology carries out rendering reinforcing according to doctor's demand to the 3-D image after reconstruction, so that guidance or auxiliary doctor are correct It performs the operation, improves the precision of operation.

In several embodiments provided herein, it should be understood that disclosed system, terminal and method, it can be with It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit It divides, only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components It can be combined or can be integrated into another system, or some features can be ignored or not executed.In addition, shown or beg for Opinion mutual coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING of device or unit Or communication connection, it is also possible to electricity, mechanical or other form connections.

The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple In network unit.Some or all of unit therein can be selected to realize the embodiment of the present invention according to the actual needs Purpose.

It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit It is that each unit physically exists alone, is also possible to two or more units and is integrated in one unit.It is above-mentioned integrated Unit both can take the form of hardware realization, can also realize in the form of software functional units.

If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product When, it can store in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words It embodies, which is stored in a storage medium, including some instructions are used so that a computer Equipment (can be personal computer, server or the network equipment etc.) executes the complete of each embodiment the method for the present invention Portion or part steps.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic or disk etc. are various can store journey The medium of sequence code.The above description is merely a specific embodiment, but protection scope of the present invention is not limited to This, anyone skilled in the art in the technical scope disclosed by the present invention, can readily occur in various equivalent Modifications or substitutions, these modifications or substitutions should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention It should be subject to the protection scope in claims.

Claims

1. a kind of 3-D image method for building up based on the imaging of more mesh and multi-polarization state imaging, which is characterized in that the method packet It includes:

Broad-spectrum light source generates broadband wide spectrum light, wherein the wide spectrum light is for irradiating operative region, the broad-spectrum light Line includes available light and polarization light, and the operative region is the body region that patient is performed the operation；

Spectrum imaging device multi-angle acquires the wide spectrum light by the operative region reflects and is formed multiple narrow-band light Spectrogram picture constructs the preliminary three-dimensional reconstruction image of the operative region using multi-view stereo vision algorithm, wherein different narrow waves Section spectrum picture has different imaging narrow-bands, and the quantity of the imaging narrow-band is 100 or more；

Multi-polarization state imaging device acquires the polarization light and is polarized by multiple narrow-bands that the operative region reflects and is formed Image obtains the body fluid covering image of the human organ in the operative region；

Processing system covers image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ, obtains the hand The three-dimensional reconstruction image in art region, and export to display equipment.

2. the method according to claim 1, wherein the method also includes:

The processing system is according to multiple narrow-band polarization images, and combining classification model is to the body fluid components in the operative region Type and organ type carry out instant analysis and export analysis as a result, the instant analysis includes organ floristic analysing, albumen Matter floristic analysing, protein content analysis, crystallization floristic analysing and crystalline content analysis, wherein the disaggregated model is pre- The body fluid disaggregated model and organ classes' model that first convolutional neural networks are trained using sample data；

Determine whether protein content or crystalline content in the analysis result are greater than first threshold or less than the second threshold Value, wherein the first threshold is greater than the second threshold；

Protein content or crystalline content in the analysis result are greater than the first threshold or less than described second In the case where threshold value, prompt information is issued.

3. the method according to claim 1, wherein after obtaining the three-dimensional reconstruction image of the operative region, institute State method further include:

The rendering that the three-dimensional reconstruction image carries out target area is strengthened, the three-dimensional reconstruction image after output reinforcing, wherein institute Stating target area includes the specific region in the region being blocked in operative region and operative region.

4. according to the method described in claim 3, it is characterized in that, the three-dimensional reconstruction image to be carried out to the rendering of target area Reinforcing includes:

In the case where target area is the region being blocked in operative region, the position of target area described in occlusion removal, wash with watercolours Dye the target area not being blocked；Alternatively,

In the case where target area is the specific region in operative region, the brightness of the specific region is promoted, is rendered bright Target area after degree promotion.

5. according to the method described in claim 4, it is characterized in that,

The target area is the region that processing system is determined according to the location information that operation tool is sent, wherein the operation It is provided with gyroscope in tool, accelerometer, one or more in range sensor, the location information is according to Gyroscope, one or more collected position data in the accelerometer and the range sensor are calculated 's；Alternatively,

6. a kind of 3-D image based on the imaging of more mesh and multi-polarization state imaging establishes system characterized by comprising

Broad-spectrum light source, broad-spectrum light source generate broadband wide spectrum light, wherein and the wide spectrum light is used to irradiate operative region, The wide spectrum light includes available light and polarization light, and the operative region is the body region that patient is performed the operation；

Spectrum imaging device acquires the wide spectrum light by the operative region reflects and is formed multiple narrow waves for multi-angle Section spectrum picture, the preliminary three-dimensional reconstruction image of the operative region is constructed using multi-view stereo vision algorithm, wherein different Narrow-band spectrum image has different imaging narrow-bands, and the quantity of the imaging narrow-band is 100 or more；

Multi-polarization state imaging device, for acquiring the wide spectrum light by the operative region reflects and is formed multiple narrow-bands Polarization image obtains the body fluid covering image of the human organ in the operative region；

Processing system, processing system cover image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ, The three-dimensional reconstruction image of the operative region is obtained, and is exported to display equipment.

7. system according to claim 6, which is characterized in that

The processing system is also used to according to multiple narrow-band polarization images, and combining classification model is to the body in the operative region Liquid ingredient type and organ type carry out instant analysis and export analysis as a result, the instant analysis includes organ type point Analysis, kinds of protein analysis, protein content analysis, crystallization floristic analysing and crystalline content analysis, wherein the classification mould Type is the body fluid disaggregated model and organ classes' model being trained in advance using sample data to convolutional neural networks；

The processing system is also used to determine whether protein content or crystalline content in the analysis result are greater than first Threshold value is less than second threshold, wherein the first threshold is greater than the second threshold；

The processing system is also used to be greater than first threshold in the protein content analyzed in result or crystalline content In the case where being worth or being less than the second threshold, prompt information is issued.

8. system according to claim 6, which is characterized in that processing system is also used to obtain the three-dimensional of the operative region After reconstruction image, the rendering that the three-dimensional reconstruction image carries out target area is strengthened, the three-dimensional reconstruction image after output reinforcing, Wherein, the target area includes the specific region in the region being blocked in operative region and operative region.

9. system according to claim 8, which is characterized in that

The processing system is specifically used in the case where target area is the region being blocked in operative region, occlusion removal mesh The position for marking region, renders the target area not being blocked；Alternatively,

The processing system is specifically used for being promoted described specific in the case where target area is the specific region in operative region The brightness in region, the target area after rendering luminance raising.

10. system according to claim 9, which is characterized in that