CN109771052A - 3-D image method for building up and system based on the imaging of more mesh and multi-polarization state imaging - Google Patents
3-D image method for building up and system based on the imaging of more mesh and multi-polarization state imaging Download PDFInfo
- Publication number
- CN109771052A CN109771052A CN201811625313.0A CN201811625313A CN109771052A CN 109771052 A CN109771052 A CN 109771052A CN 201811625313 A CN201811625313 A CN 201811625313A CN 109771052 A CN109771052 A CN 109771052A
- Authority
- CN
- China
- Prior art keywords
- image
- region
- operative region
- narrow
- target area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Apparatus For Radiation Diagnosis (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
This application provides a kind of 3-D image method for building up being imaged based on the imaging of more mesh and multi-polarization state and systems, the described method includes: broad-spectrum light source generates broadband wide spectrum light, spectrum imaging device multi-angle acquires the wide spectrum light by the operative region reflects and is formed multiple narrow-band spectrum images, the preliminary three-dimensional reconstruction image of the operative region is constructed using multi-view stereo vision algorithm, multi-polarization state imaging device acquires the polarization light by the operative region reflects and is formed multiple narrow-band polarization images, obtain the body fluid covering image of the human organ in the operative region, processing system covers image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ, obtain the three-dimensional reconstruction image of the operative region, and it exports to display equipment.
Description
Technical field
This application involves electronic fields more particularly to a kind of 3-D image based on the imaging of more mesh and multi-polarization state imaging to build
Cube method and system.
Background technique
As attention degree of the public to health is higher and higher and the continuous promotion of medical level, surgical operation is also more and more normal
See.In surgical operation, the success rate and effect of operation and the accuracy of operation are closely related, and traditional operation is all by doctor
Raw experience is visually judged that accuracy is not high to operation tool and lesion locations, and the requirement to doctor is also very
It is high.
But the precision of surgical operation how is improved, it is a still unsolved problem so far.
Summary of the invention
This application provides 3-D image method for building up and system based on the imaging of more mesh and multi-polarization state imaging, Neng Gouyou
Improve the precision of surgical operation in effect ground.
In a first aspect, a kind of 3-D image method for building up based on the imaging of more mesh and multi-polarization state imaging is provided, it is described
Method the following steps are included:
Broad-spectrum light source generates broadband wide spectrum light, wherein the wide spectrum light is described wide for irradiating operative region
Composing light includes available light and polarization light, and the operative region is the body region that patient is performed the operation;
Spectrum imaging device multi-angle acquires the wide spectrum light by the operative region reflects and is formed multiple narrow waves
Section spectrum picture, the preliminary three-dimensional reconstruction image of the operative region is constructed using multi-view stereo vision algorithm, wherein different
Narrow-band spectrum image has different imaging narrow-bands, and the quantity of the imaging narrow-band is 100 or more;
Multi-polarization state imaging device acquires the polarization light by the operative region reflects and is formed multiple narrow-bands
Polarization image obtains the body fluid covering image of the human organ in the operative region;
Processing system covers image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ, obtains institute
The three-dimensional reconstruction image of operative region is stated, and is exported to display equipment.
Optionally, the method also includes: the processing system is according to multiple narrow-band polarization images, combining classification model
To the body fluid components type and organ type progress instant analysis in the operative region and analysis is exported as a result, described instant
Analysis includes organ floristic analysing, kinds of protein analysis, protein content analysis, crystallization floristic analysing and crystalline content point
Analysis, wherein the disaggregated model is the body fluid classification mould being trained in advance using sample data to convolutional neural networks
Type and organ classes' model;
Determine whether protein content or crystalline content in the analysis result are greater than first threshold or less than the
Two threshold values, wherein the first threshold is greater than the second threshold;
Protein content or crystalline content in the analysis result are greater than the first threshold or are less than described
In the case where second threshold, prompt information is issued.
Optionally, after the three-dimensional reconstruction image for obtaining the operative region, the method also includes:
The rendering that the three-dimensional reconstruction image carries out target area is strengthened, the three-dimensional reconstruction image after output reinforcing,
In, the target area includes the specific region in the region being blocked in operative region and operative region.
Optionally, include: by the rendering reinforcing that the three-dimensional reconstruction image carries out target area
In the case where target area is the region being blocked in operative region, the position of occlusion removal target area, wash with watercolours
Dye the target area not being blocked;Alternatively,
In the case where target area is the specific region in operative region, the brightness of the specific region is promoted, is rendered
Target area after luminance raising out.
Optionally, the target area is the region that processing system is determined according to the location information that operation tool is sent,
In, it is provided with gyroscope in the operation tool, accelerometer, one or more in range sensor, the position letter
Breath is one or more collected positional number according to the gyroscope, in the accelerometer and the range sensor
According to what is be calculated;Alternatively,
The target area is that user requests determining region to processing system based on the received.
Second aspect is provided a kind of be imaged based on more mesh and establishes system with the 3-D image of multi-polarization state imaging, described
System includes:
Broad-spectrum light source, broad-spectrum light source generate broadband wide spectrum light, wherein the wide spectrum light is for irradiating field of operation
Domain, the wide spectrum light include available light and polarization light, and the operative region is the body region that patient is performed the operation;
Spectrum imaging device acquires the wide spectrum light for multi-angle and is reflected and formed multiple by the operative region
Narrow-band spectrum image constructs the preliminary three-dimensional reconstruction image of the operative region using multi-view stereo vision algorithm, wherein no
Same narrow-band spectrum image has different imaging narrow-bands, and the quantity of the imaging narrow-band is 100 or more;
Multi-polarization state imaging device is reflected and formed multiple narrow by the operative region for acquiring the wide spectrum light
Wave band polarization image obtains the body fluid covering image of the human organ in the operative region;
Processing system, processing system is according to the preliminary three-dimensional reconstruction image and the body fluid coverage diagram of the human organ
Picture obtains the three-dimensional reconstruction image of the operative region, and exports to display equipment.
Optionally, the processing system is also used to according to multiple narrow-band polarization images, and combining classification model is to the hand
Body fluid components type and organ type in art region carry out instant analysis and export analysis as a result, the instant analysis includes
Organ floristic analysing, kinds of protein analysis, protein content analysis, crystallization floristic analysing and crystalline content analysis, wherein
The disaggregated model is the body fluid disaggregated model and organ being trained in advance using sample data to convolutional neural networks
Disaggregated model;
The processing system is also used to determine whether the analysis result is greater than first threshold or is less than second threshold,
In, the first threshold is greater than the second threshold;
The processing system is also used to be greater than the first threshold in the analysis result or is less than the second threshold
In the case where, issue prompt information.
Optionally, after the processing system is also used to obtain the three-dimensional reconstruction image of the operative region, by the three-dimensional
The rendering that reconstruction image carries out target area is strengthened, the three-dimensional reconstruction image after output reinforcing, wherein the target area includes
Specific region in the region being blocked in operative region and operative region.
Optionally, the processing system is specifically used for the case where target area is the region being blocked in operative region
Under, the position of occlusion removal target area renders the target area not being blocked;Alternatively,
The processing system is specifically used in the case where target area is the specific region in operative region, described in promotion
The brightness of specific region, the target area after rendering luminance raising.
Optionally, the target area is the region that processing system is determined according to the location information that operation tool is sent,
In, it is provided with gyroscope in the operation tool, accelerometer, one or more in range sensor, the position letter
Breath is one or more collected positional number according to the gyroscope, in the accelerometer and the range sensor
According to what is be calculated;Alternatively,
The target area is that user requests determining region to processing system based on the received.
3-D image method for building up and system provided by the present application based on the imaging of more mesh and multi-polarization state imaging, by wide
It composes light source and generates broadband wide spectrum light, it is anti-by the operative region that spectrum imaging device multi-angle acquires the wide spectrum light
The multiple narrow-band spectrum images penetrated and formed construct the preliminary three-dimensional weight of the operative region using multi-view stereo vision algorithm
Image is built, it is inclined that multi-polarization state imaging device acquires multiple narrow-bands that the polarization light is reflected and formed by the operative region
Shake image, the body fluid covering image of the human organ in the operative region is obtained, so that processing system is according to described primary three
The body fluid for tieing up reconstruction image and the human organ covers image, obtains the three-dimensional reconstruction image of the operative region, and defeated
Out to display equipment.By obtaining richer effective letter according to multiple narrow-band spectrum images and multiple narrow-band polarization images
Breath, to reconstruct the higher 3-D image of accuracy, also, can also be by virtually enhancing technology to the three-dimensional figure after reconstruction
As carrying out rendering reinforcing according to doctor's demand, so that guidance or auxiliary doctor correctly perform the operation, the precision of operation is improved.
Detailed description of the invention
Fig. 1 is a kind of stream of 3-D image method for building up based on the imaging of more mesh and multi-polarization state imaging provided by the present application
Journey schematic diagram;
Fig. 2 is light in a kind of 3-D image method for building up based on the imaging of more mesh and multi-polarization state imaging provided by the present application
It composes light source and arranges schematic diagram;
Fig. 3 is a kind of schematic diagram that prompt information is carried out by way of augmented reality provided by the present application;
Fig. 4 a is the operative region reconstruction image before a kind of augmented reality provided by the present application;
Fig. 4 b is the operative region reconstruction image after a kind of augmented reality provided by the present application;
Fig. 5 a is the operative region reconstruction image before another augmented reality provided by the present application;
Fig. 5 b is the operative region reconstruction image after another augmented reality provided by the present application;
Fig. 6 is the knot that a kind of 3-D image based on the imaging of more mesh and multi-polarization state imaging provided by the present application establishes system
Structure schematic diagram.
Specific embodiment
Refering to fig. 1, Fig. 1 is a kind of three-dimensional figure based on the imaging of more mesh and multi-polarization state imaging provided by the embodiments of the present application
As the flow diagram of method for building up.The 3-D image foundation side based on the imaging of more mesh and multi-polarization state imaging of the present embodiment
Method includes the following steps:
S101: broad-spectrum light source generates broadband wide spectrum light, wherein and the wide spectrum light is used to irradiate operative region,
The wide spectrum light includes available light and polarization light, and the operative region is the body region that patient is performed the operation.
In the application specific embodiment, broad-spectrum light source is for generating broadband wide spectrum light.Broad-spectrum light source produces
The range of raw wavelength can extend to ultraviolet from infrared, visible light be extended to from infrared, alternatively, extending to from visible light ultraviolet
Etc..In practical applications, the quantity of broad-spectrum light source can be one or more.In a specific embodiment, Duo Geguang
The range of the wave band for the wide spectrum light that spectrum light source generates all be it is identical, for example, the wide spectrum light that the first broad-spectrum light source generates
Wave-length coverage be 350 nanometers to 850 nanometers, the second broad-spectrum light source generate wide spectrum light wave-length coverage be also 350 nanometers extremely
850 nanometers etc..In another specific embodiment, the range of the wave band for the wide spectrum light that multiple broad-spectrum light sources generate all is phase
With, for example, the wave-length coverage for the wide spectrum light that the first broad-spectrum light source generates is 350 nanometers to 550 nanometers, the second broad-spectrum light source
The wave-length coverage of the wide spectrum light of generation is also 550 nanometers to 850 nanometers etc..
In the above example, it is all illustrated so that the wide spectrum light that broad-spectrum light source generates is continuous broadband as an example,
In practical application, the wide spectrum light that broad-spectrum light source generates is also possible to discontinuous broadband, for example, the first broad-spectrum light source generates
The wave-length coverage of wide spectrum light be 350 nanometers to 450 nanometers, the wave-length coverage of wide spectrum light that the first broad-spectrum light source generates is
It 650 nanometers to 850 nanometers etc., is not especially limited herein.
In the application specific embodiment, polarization light be wide spectrum light is obtained by the polarizer, that is,
It says, the polarizer is placed below a part of broad-spectrum light source for obtaining polarization light, is not put below another part broad-spectrum light source
The polarizer is set for obtaining available light, it is to be understood that polarization light is for multi-polarization state imaging device acquisition polarization figure
Picture, available light acquire spectrum picture for spectrum imaging device.
In the application specific embodiment, the arrangement mode of broad-spectrum light source can be horizontally arranged, be also possible to hang down
In line column can also be that inclination angle arranges, can also be the combination of various arrangement mode, for example, Fig. 2 is that the embodiment of the present application provides
Broad-spectrum light source arrangement mode, wherein the polarizer is placed with below A group light source not to be had for obtaining polarised light, B group light source lower section
The polarizer, A1, B1 light source are vertical irradiation array of source, and A2, B2, A3, B3 are inclination angle light source, are separately positioned on A1, B1 light source
Two sides, and with horizontal direction be in θ angle, it is to be understood that due to collected biological tissue under the light beam of different angle
Characteristic information is different, therefore light source arrangement mode shown in Fig. 2 is conducive to sufficiently excavate biological information, it should be appreciated that Fig. 2 institute
The broad-spectrum light source arrangement mode shown is only used for for example, specific restriction can not be constituted, and the broad-spectrum light source in the application can be with
Possess more perpendicular light source arrays and inclination angle array of source.
In the application specific embodiment, the operative region is the body region that patient is performed the operation, without
Entire operating table, operative region may include: the operation tool that patient operative site, the operative site performed the operation are placed and
Operation label.Wherein, operative site can position to need to be performed an operation with patient, and the position that needs to be performed an operation
Neighbouring position.Operation tool can be scalpel, surgical scissors, hook for surgery operation and dilator etc..Operation label can be
It is convenient for the operation marker of positioning, is not especially limited herein.
S102: spectrum imaging device multi-angle acquires the wide spectrum light and is reflected and formed multiple by the operative region
Narrow-band spectrum image constructs the preliminary three-dimensional reconstruction image of the operative region using multi-view stereo vision algorithm, wherein no
Same narrow-band spectrum image has different imaging narrow-bands, and the quantity of the imaging narrow-band is 100 or more.
In the application specific embodiment, spectrum imaging device can form multiple narrow-band spectrum images.Wherein,
Spectrum imaging device can be more mesh spectrum imaging devices that are multiple and being multi-angle arrangement, is also possible to high light spectrum image-forming and sets
Standby and depth camera combination, spectrum imaging device can also be high light spectrum image-forming equipment or hyperspectral imaging apparatus etc.
Deng the application is not especially limited.Wherein, high light spectrum image-forming equipment is with hundreds of to thousands of continuous and subdivision spectral band pair
Operative region is imaged simultaneously, and hyperspectral imaging apparatus is same to operative region with thousands of to tens of thousands of continuous and subdivision spectral bands
When be imaged.Certainly, spectrum imaging device can also be imaged simultaneously operative region with more continuous and subdivision spectral band,
Such as tens of thousands of extremely hundreds of thousands of continuous and subdivision spectral bands do not limit operative region imaging etc. simultaneously specifically herein
It is fixed.
In the application specific embodiment, spectrum imaging device can use the SPECTRAL DIVERSITY of body surface ingredient,
Target is identified and is classified, that is to say, that different narrow-band spectrum images has different imaging narrow-bands, will be different
The narrow-band spectrum image that angle shot obtains is integrated, and the broader visual field can be obtained, thus can be obtained richer
Information, recycle multi-view stereo vision algorithm, three-dimensional reconstruction may be implemented.For example, the skin of human body can be preferably
It reflects a and narrow-band is imaged, the muscle of human body can preferably reflect b imaging narrow-band, and the blood vessel of human body can preferably reflect c
Narrow-band is imaged, human skeleton can preferably reflect d imaging narrow-band.Therefore, narrow-band can be imaged according to a to obtain instead
The A narrow-band spectrum image of human skin situation is answered, the B that narrow-band acquisition reaction human muscle's situation can be imaged according to b is narrow
The C narrow-band spectrum image that narrow-band obtains reaction human vas situation, Ke Yigen can be imaged according to c in band spectrum image
The D narrow-band spectrum image of reaction skeleton situation is obtained according to d imaging narrow-band.It is appreciated that when spectral band subdivision
Degree is higher, then the visual field of spectrum imaging device is more wide, and the information that spectrum generating device obtains is also naturally abundanter, acquisition
Human body information is also more acurrate, and then realizes more accurate three reconstruction models.
Preferably, binocular stereo vision matching (Sum of Absolute can be used in spectrum imaging device
Differences, SAD) algorithm carry out operative region three-dimensional reconstruction, that is to say, that obtained using binocular high light spectrum image-forming equipment
Different angle narrow-band spectrum image, the difference of the corresponding numerical value of pixel in each image is taken absolute value summation, with
The similarity degree of the narrow-band spectrum image of this assessment different angle, to obtain disparity map.After obtaining parallax information, according to
The depth information and three-dimensional information of original image can be obtained in projection model.Certainly, spectrum imaging device can also use block
With (Block Matching, BM) algorithm, half global Block- matching (Semi-Global Block Matching, SGM) algorithm etc.
Other multi-view stereo vision algorithms carry out the preliminary three-dimensional reconstruction of operative region, and the application is not especially limited.
S103: multi-polarization state imaging device acquires the polarization light and is reflected and formed multiple narrow by the operative region
Wave band polarization image obtains the body fluid covering image of the human organ in the operative region.
In the application specific embodiment, multi-polarization state imaging device can form multiple narrow-band polarization images.
Wherein, multi-polarization state imaging device can be one or more, multi-polarization state imaging device can be orthogonal multi-polarization state at
As equipment or interference multi-polarization state imaging device etc..Wherein, orthogonal multi-polarization state imaging device is with hundreds of to thousands of companies
Operative region is imaged in continuous and subdivision spectral band simultaneously, interfere multi-polarization state imaging device with it is thousands of to tens of thousands of it is a continuously and
Operative region is imaged in the spectral band of subdivision simultaneously.Certainly, multi-polarization state imaging device can also be with more continuous and thin
The polarised light wave band opponent's art region divided is imaged simultaneously, such as tens of thousands of to hundreds of thousands of polarised light wave bands pair that are continuous and segmenting
Operative region imaging etc. simultaneously, is not especially limited herein.
In the application specific embodiment, the corresponding imaging narrow-band of narrow-band polarization image and narrow-band spectrum figure
As corresponding imaging narrow-band can be it is identical, such as: the blood vessel of human body can preferably reflect c imaging narrow-band, therefore
The C narrow-band spectrum image and C narrow-band polarization image that narrow-band obtains reaction human vas situation can be imaged according to c.
The corresponding imaging narrow-band of narrow-band polarization image imaging narrow-band corresponding with narrow-band spectrum image can also be not identical
, such as: the blood vessel of human body can preferably reflect c1 imaging narrow-band, preferable to reflect c2 polarization imaging narrow-band, therefore
The C1 narrow-band spectrum image that narrow-band obtains reaction human vas situation can be imaged according to C1, according to the narrow wave of C2 polarization imaging
Section obtains the C2 narrow-band polarization image of reaction human vas situation.It should be understood that the above citing is merely to illustrate, can not constitute
It is specific to limit.
In the application specific embodiment, multi-polarization state imaging device can be according in multiple narrow-band polarization images
The angle of polarization, degree of polarization and the ellipticity angle of light beam obtain the polarization characteristic parameter of operative region, provide the organ table of operative region
The signs such as surface roughness, texture trend, surface orientation, water content, compare spectrum imaging device, and multi-polarization state imaging device can be with
To organ contours and surface orientation identification provide more accurate information, can protrude the organ in operative region in picture with back
The contrast of scape, and then improve the accuracy rate next identified with organ classes' model, body fluid disaggregated model.Therefore, the application
The three-dimensional reconstruction that operative region is carried out in conjunction with spectrum imaging device and multi-polarization state imaging device, can provide completeer for doctor
The information that whole, more reliable and human eye can not extract.Operation region entirety is carried out that is, spectrum imaging device can be
Feature extraction realizes that the preliminary three-dimensional of operative region is rebuild using multi-view stereo vision algorithm, and multi-polarization state imaging device can be with
It is the organ surface roughness that can not be shown to spectrum imaging device, grain of meat trend, the class of liquids of organ surface covering
Equal profiles and surface carry out feature extraction, obtain the human organ clearly described in operative region surface roughness or
The image of body fluid coverage condition.
S104: processing system covers image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ,
The three-dimensional reconstruction image of the operative region is obtained, and is exported to display equipment.
In presently filed embodiment, processing system is inclined according to the multiple narrow-band spectrum image and multiple narrow-bands
The 3-D image of operative region described in picture construction of shaking.That is, processing system combines narrow-band spectrum image and narrow wave
The information that section polarization image extracts respectively, establishes an operative region steric information figure detailed, abundant.Such as: processing system
System can be according to the 3-D image A of the A narrow-band spectrum image reconstruction skin of different angle, according to the B narrow-band of different angle
Spectrum picture rebuilds the 3-D image B of muscle, according to the 3-D image C of the C narrow-band spectrum image reconstruction blood vessel of different angle,
According to the 3-D image D of the D narrow-band spectrum image reconstruction bone of different angle, is polarized and schemed according to the E narrow-band of different angle
3-D image E as rebuilding capilary rebuilds each organ surface Covering Liguid according to the F narrow-band polarization image of different angle
3-D image F, by 3-D image A, 3-D image B, 3-D image C, 3-D image D, 3-D image E and 3-D image F
After being merged, so that it may which acquisition can reflect skin, muscle, blood vessel, bone, capilary, organ surface degree of roughness simultaneously
Detailed, true operative region 3-D image.
In the application specific embodiment, the method also includes: the processing system is inclined according to multiple narrow-bands
Shake image, combining classification model in the operative region body fluid components type and organ type carry out instant analysis and defeated
Analysis is as a result, the instant analysis includes kinds of protein analysis, protein content analysis, crystallization floristic analysing and crystallization out
Content analysis, wherein the disaggregated model is the body fluid being trained in advance using sample data to convolutional neural networks
Disaggregated model and organ classes' model;Determine whether protein content or crystalline content in the analysis result are greater than first
Threshold value is less than second threshold, wherein the first threshold is greater than the second threshold;Albumen in the analysis result
In the case that matter content or crystalline content are greater than the first threshold or are less than the second threshold, prompt information is issued.
Wherein, body fluid disaggregated model and organ classes' model can be in advance using the body fluid picture sample for being largely labeled with prediction result
And organ sample, convolutional neural networks are trained with the trained disaggregated model of acquisition, the body fluid disaggregated model can
With the narrow-band polarization image comprising the aqueous measure feature of organ surface obtained according to multi-polarization state imaging system, body fluid point is obtained
Class is as a result, organ classes' model can be according to the narrow comprising organ surface contour feature of multi-polarization state imaging system acquisition
Wave band polarization image obtains organ classification results, and exports in the reality enhancing or mixed reality equipment that doctor wears, in hand
More information are provided for doctor during art, further to judge the state of an illness and to perform the operation.Wherein, prompt information can be
One of sound, light or text.For example, as shown in figure 3, can be existed by the method for augmented reality or mixed reality
Increasing text on 3-D image, content of hemoglobin is abnormal in blood samples of patients during surgical to remind.Also, more polarizations
State imaging device can also carry out the judgement of organ classes in conjunction with neural network model, can also will sentence according to polarization optical information
Determine result and text is increased on 3-D image to remind doctor by the method for augmented reality or mixed reality, the application does not make
It repeats.
In the application specific embodiment, processing system is according to the multiple narrow-band spectrum image and multiple narrow waves
The 3-D image that section polarization image constructs the operative region specifically may is that processing system by the operative site, the hand
Art tool and operation label are positioned and divide to obtain locating segmentation as a result, to true according to locating segmentation result
Relative position between the fixed operative site, the operation tool and the operation label, then, further according to the Surgery
Relative position reconstruction of three-dimensional images between position, the operation tool and the operation label.It is appreciated that passing through multiple narrow waves
Section spectrum picture and multiple narrow-band polarization images can obtain information more abundant, therefore, it is possible to more accurately to institute
Operative site, the operation tool and operation label is stated to be positioned and divided.
In the application specific embodiment, processing system is by the operative site, the operation tool and the hand
Art label positioned and divided may is that obtaining locating segmentation result specifically obtain multiple narrow-band spectrum images and
After multiple narrow-band polarization images, processing system can be extracted multiple with feature extraction algorithm (for example, convolutional neural networks)
The textural characteristics of narrow-band spectrum image and multiple narrow-band polarization images.Then, in conjunction with image recognition algorithm (for example, volume
Product neural network) and movement tracing algorithm operation consent is done label, operation tool with perform the operation relevant range be accurately positioned
Or segmentation, to obtain locating segmentation result.
In the application specific embodiment, after the three-dimensional reconstruction image for obtaining the operative region, the method is also
It include: to strengthen the rendering that the three-dimensional reconstruction image carries out target area, the three-dimensional reconstruction image after output reinforcing, wherein
The target area includes the specific region in the region being blocked in operative region and operative region.By the three-dimensional reconstruction
It includes: to move in the case where target area is the region being blocked in operative region that the rendering that image carries out target area, which is strengthened,
Except the position for blocking the target area, the target area not being blocked is rendered;Alternatively, being in operative region in target area
Specific region in the case where, promote the brightness of the specific region, the target area after rendering luminance raising.
Preferably, the target area not being blocked rendered can be to be obtained by using the trained supplementary model that blocks
, the supplementary model that blocks is to use the image of the shelter in a large amount of binocular images and binocular image as sample,
It is obtained after being trained to the convolutional neural networks (SymmNet) of symmetrical structure, the supplementary model that blocks has learnt largely
Feature is blocked in binocular image, the occlusion issue of more mesh spectrum imaging device images can be predicted by bilateral symmetry
Network model.More operation informations can be brought for the operation of doctor using the result after supplementary model rendering is strengthened is blocked,
For example, certain operative region is as shown in fig. 4 a, in the case that target area is the region being blocked in operative region, doctor is used
Reality enhancing (Augmented Reality, AR) glasses can see the three-dimensional reconstruction figure after rendering as shown in Figure 4 b is strengthened
Picture, and performed the operation according to the three-dimensional reconstruction image after reinforcing.It is understood that strong in no rendering for carrying out target area
Before change, doctor can only rule of thumb perform the operation, and after the rendering for carrying out target area is strengthened, AR is can be used in doctor
The progress that glasses are performed the operation according to the target area image not being blocked rendered, auxiliary or guidance.It should be understood that the example above is only
For illustrating, the rendering of occlusion area can also use other algorithms to obtain, such as use cross inspection (Left-
Right-cross-checking, LRC) algorithm infers the position blocked by the parallax result precalculated, the application is not
Make specific limit.
Preferably, the target area after rendering luminance raising can be to be obtained using trained brightness supplementary model
, the brightness supplementary model is to be trained using the operative region samples pictures before and after luminance raising to convolutional neural networks
Obtained model, the result after being strengthened using the rendering of brightness supplementary model can bring more operation informations for the operation of doctor,
For example, as shown in Figure 5 a, when target area be operative region in specific region in the case where, for example, certain lesion regions or
The region that these doctors of person shadow region can not see by visual observation, doctor can see using after AR glasses such as Fig. 5 b institute
Three-dimensional reconstruction image after the rendering reinforcing shown, and performed the operation according to the three-dimensional reconstruction image after reinforcing.It is understood that
Before no rendering for carrying out target area is strengthened, doctor can only rule of thumb perform the operation, and having carried out, target area is bright
After degree is promoted, AR glasses are can be used according to the target area image after the luminance raising rendered, auxiliary or guidance in doctor
The progress of operation.
In the application specific embodiment, the target area is the position that processing system is sent according to operation tool
The region that information determines, wherein be provided with gyroscope in the operation tool, accelerometer, one in range sensor or
Person is multiple, the location information be one according to the gyroscope, in the accelerometer and the range sensor or
What multiple collected position datas were calculated.Such as: when surgical knife is placed on a certain lesion region, the AR of doctor
Glasses can show the three-dimensional reconstruction image after the lesion region highlights automatically.It should be understood that the above citing is merely to illustrate, not
Specific limit can be constituted.
In the application specific embodiment, the target area can also be processing system, and user is asked based on the received
Ask determining region.Such as: doctor needs to check in the course of surgery a certain target area being blocked, and doctor can lead at this time
It crosses and selects what needs were checked by using other terminal device frames being connected with processing system using AR equipment or feldsher
The target area being blocked, processing system determine target area according to the target area information received, mesh are rendered not
In the AR glasses for the target area image input doctor being blocked.It should be understood that the above citing is merely to illustrate, tool can not be constituted
Body limits.
In the application specific embodiment, the three-dimensional reconstruction image after output reinforcing, which can be, is output to doctor's wearing
Head-mounted display apparatus display screen, such as AR glasses, mixed reality (Mediated Reality, MR) glasses etc.;It can also
To be to be output to terminal device display screen, such as computer display screen, panel display screen etc.;It can also be and be output to manipulator
In art system, operation is carried out for mechanical arm, operation guidance or auxiliary are provided.
In above scheme, broad-spectrum light source generates broadband wide spectrum light, and the acquisition of spectrum imaging device multi-angle is described wide
Light is composed by multiple narrow-band spectrum images that the operative region reflects and is formed, constructs institute using multi-view stereo vision algorithm
The preliminary three-dimensional reconstruction image of operative region is stated, it is anti-by the operative region that multi-polarization state imaging device acquires the polarization light
The multiple narrow-band polarization images penetrated and formed obtain the body fluid covering image of the human organ in the operative region, so that
Processing system covers image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ, obtains the field of operation
The three-dimensional reconstruction image in domain, and export to display equipment.It, can be according to multiple narrow-band spectrum images and more by the above method
A narrow-band polarization image obtains richer effective information, to reconstruct the higher 3-D image of accuracy, also, may be used also
To carry out rendering reinforcing according to doctor's demand to the 3-D image after reconstruction by virtually enhancing technology, thus guidance or auxiliary doctor
Life is correctly performed the operation, and the precision of operation is improved.
It is that a kind of 3-D image based on the imaging of more mesh and multi-polarization state imaging provided by the present application is established referring to Fig. 6, Fig. 6
The structural schematic diagram of system.In the embodiment of the present application, the 3-D image based on the imaging of more mesh and multi-polarization state imaging establishes system
System includes: broad-spectrum light source 110, spectrum imaging device 120, multi-polarization state imaging device 130 and processing system 140.
Broad-spectrum light source 110 is for generating broadband wide spectrum light, wherein the wide spectrum light is for irradiating field of operation
Domain, the wide spectrum light include available light and polarization light, and the operative region is the body region that patient is performed the operation;
Spectrum imaging device 120 acquires the wide spectrum light for multi-angle and is reflected and formed more by the operative region
A narrow-band spectrum image constructs the preliminary three-dimensional reconstruction image of the operative region using multi-view stereo vision algorithm, wherein
Different narrow-band spectrum images has different imaging narrow-bands, and the quantity of the imaging narrow-band is 100 or more;
Multi-polarization state imaging device 130 is reflected and formed multiple by the operative region for acquiring the wide spectrum light
Narrow-band polarization image obtains the body fluid covering image of the human organ in the operative region;
Processing system 140 is used for the body fluid coverage diagram according to the preliminary three-dimensional reconstruction image and the human organ
Picture obtains the three-dimensional reconstruction image of the operative region, and exports to display equipment.
In the application specific embodiment, broad-spectrum light source 110 is for generating broadband wide spectrum light.Broad-spectrum light source
The range of the wavelength of generation can extend to ultraviolet from infrared, visible light be extended to from infrared, alternatively, extending to purple from visible light
It is outer etc..In practical applications, the quantity of broad-spectrum light source 110 can be one or more.In a specific embodiment, more
The range of the wave band for the wide spectrum light that a broad-spectrum light source 110 generates all be it is identical, for example, the wide spectrum that the first broad-spectrum light source generates
The wave-length coverage of light is 350 nanometers to 850 nanometers, and the wave-length coverage for the wide spectrum light that the second broad-spectrum light source generates also is 350
Nanometer is to 850 nanometers etc..In another specific embodiment, the range of the wave band for the wide spectrum light that multiple broad-spectrum light sources generate
It is all identical, for example, the wave-length coverage for the wide spectrum light that the first broad-spectrum light source generates is 350 nanometers to 550 nanometers, second is wide
The wave-length coverage for composing the wide spectrum light that light source generates also is 550 nanometers to 850 nanometers etc..
In the above example, it is all illustrated so that the wide spectrum light that broad-spectrum light source 110 generates is continuous broadband as an example,
In practical applications, the wide spectrum light that broad-spectrum light source 110 generates is also possible to discontinuous broadband, for example, the first broad-spectrum light
The wave-length coverage for the wide spectrum light that source generates is 350 nanometers to 450 nanometers, the wavelength for the wide spectrum light that the first broad-spectrum light source generates
Range is 650 nanometers to 850 nanometers etc., is not especially limited herein.
In the application specific embodiment, polarization light be wide spectrum light is obtained by the polarizer, that is,
It says, the polarizer is placed below a part of broad-spectrum light source 110 for obtaining polarization light, under another part broad-spectrum light source 110
The polarizer is not placed for obtaining available light in side, it is to be understood that polarization light is acquired for multi-polarization state imaging device
Polarization image, available light acquire spectrum picture for spectrum imaging device.
In the application specific embodiment, the arrangement mode of broad-spectrum light source 110 can be horizontally arranged, be also possible to
Vertical arrangement can also be that inclination angle arranges, can also be the combination of various arrangement mode, for example, Fig. 2 is that the embodiment of the present application mentions
The arrangement mode of the broad-spectrum light source of confession, wherein be placed with the polarizer below A group light source for obtaining polarised light, do not have below B group light source
There is the polarizer, A1, B1 light source are vertical irradiation array of source, and A2, B2, A3, B3 are inclination angle light source, are separately positioned on A1, B1 light
The two sides in source, and be in θ angle with horizontal direction, it is to be understood that due to biology group collected under the light beam of different angle
Characteristic information difference is knitted, therefore light source arrangement mode shown in Fig. 2 is conducive to sufficiently excavate biological information, it should be appreciated that Fig. 2
Shown in broad-spectrum light source arrangement mode be only used for for example, specific restriction can not be constituted, the broad-spectrum light source in the application can
To possess more perpendicular light source arrays and inclination angle array of source.
In the application specific embodiment, the operative region is the body region that patient is performed the operation, without
Entire operating table, operative region may include: the operation tool that patient operative site, the operative site performed the operation are placed and
Operation label.Wherein, operative site can position to need to be performed an operation with patient, and the position that needs to be performed an operation
Neighbouring position.Operation tool can be scalpel, surgical scissors, hook for surgery operation and dilator etc..Operation label can be
It is convenient for the operation marker of positioning, is not especially limited herein.
In the application specific embodiment, spectrum imaging device 120 can form multiple narrow-band spectrum images.Its
In, spectrum imaging device 120 can be more mesh spectrum imaging devices that are multiple and being multi-angle arrangement, be also possible to EO-1 hyperion
The combination of imaging device and depth camera, spectrum imaging device 120 can also be high light spectrum image-forming equipment or ultraphotic spectrum at
As equipment etc., the application is not especially limited.Wherein, high light spectrum image-forming equipment is with hundreds of to thousands of continuous and subdivision light
Operative region is imaged in spectrum wave band simultaneously, and hyperspectral imaging apparatus is with thousands of to tens of thousands of continuous and subdivision spectral band opponents
Art region is imaged simultaneously.Certainly, spectrum imaging device can also be with more continuous and subdivision spectral band to operative region
It is imaged simultaneously, such as tens of thousands of to hundreds of thousands of continuous and subdivision spectral bands are imaged operative region etc. simultaneously, herein
It is not especially limited.
In the application specific embodiment, spectrum imaging device can use the SPECTRAL DIVERSITY of body surface ingredient,
Target is identified and is classified, that is to say, that different narrow-band spectrum images has different imaging narrow-bands, will be different
The narrow-band spectrum image that angle shot obtains is integrated, and the broader visual field can be obtained, thus can be obtained richer
Information, recycle multi-view stereo vision algorithm, three-dimensional reconstruction may be implemented.For example, the skin of human body can be preferably
It reflects a and narrow-band is imaged, the muscle of human body can preferably reflect b imaging narrow-band, and the blood vessel of human body can preferably reflect c
Narrow-band is imaged, human skeleton can preferably reflect d imaging narrow-band.Therefore, narrow-band can be imaged according to a to obtain instead
The A narrow-band spectrum image of human skin situation is answered, the B that narrow-band acquisition reaction human muscle's situation can be imaged according to b is narrow
The C narrow-band spectrum image that narrow-band obtains reaction human vas situation, Ke Yigen can be imaged according to c in band spectrum image
The D narrow-band spectrum image of reaction skeleton situation is obtained according to d imaging narrow-band.It is appreciated that when spectral band subdivision
Degree is higher, then the visual field of spectrum imaging device 120 is more wide, and the information that spectrum generating device 120 obtains is also naturally abundanter,
The human body information of acquisition is also more acurrate, and then realizes more accurate three reconstruction models.
Preferably, binocular stereo vision matching (Sum of Absolute can be used in spectrum imaging device
Differences, SAD) algorithm carry out operative region three-dimensional reconstruction, that is to say, that obtained using binocular high light spectrum image-forming equipment
Different angle narrow-band spectrum image, the difference of the corresponding numerical value of pixel in each image is taken absolute value summation, with
The similarity degree of the narrow-band spectrum image of this assessment different angle, to obtain disparity map.After obtaining parallax information, according to
The depth information and three-dimensional information of original image can be obtained in projection model.Certainly, spectrum imaging device can also use block
With (Block Matching, BM) algorithm, half global Block- matching (Semi-Global Block Matching, SGM) algorithm etc.
Other multi-view stereo vision algorithms carry out the preliminary three-dimensional reconstruction of operative region, and the application is not especially limited.
In the application specific embodiment, multi-polarization state imaging device 130 can form multiple narrow-band polarization figures
Picture.Wherein, multi-polarization state imaging device 130 can be one or more, and multi-polarization state imaging device 130 can be orthogonal more
Polarization state imaging device or interference multi-polarization state imaging device etc..Wherein, orthogonal multi-polarization state imaging device with it is hundreds of extremely
Operative region is imaged in thousands of continuous and subdivision spectral band simultaneously, interferes multi-polarization state imaging device with thousands of to tens of thousands of
Operative region is imaged in a continuous and subdivision spectral band simultaneously.Certainly, multi-polarization state imaging device can also be with more
Continuous and subdivision polarised light wave band opponent's art region is imaged simultaneously, such as tens of thousands of to hundreds of thousands of continuous and subdivision polarizations
Optical band is not especially limited operative region imaging etc. simultaneously herein.
In the application specific embodiment, the corresponding imaging narrow-band of narrow-band polarization image and narrow-band spectrum figure
As corresponding imaging narrow-band can be it is identical, such as: the blood vessel of human body can preferably reflect c imaging narrow-band, therefore
The C narrow-band spectrum image and C narrow-band polarization image that narrow-band obtains reaction human vas situation can be imaged according to c.
The corresponding imaging narrow-band of narrow-band polarization image imaging narrow-band corresponding with narrow-band spectrum image can also be not identical
, such as: the blood vessel of human body can preferably reflect c1 imaging narrow-band, preferable to reflect c2 polarization imaging narrow-band, therefore
The C1 narrow-band spectrum image that narrow-band obtains reaction human vas situation can be imaged according to C1, according to the narrow wave of C2 polarization imaging
Section obtains the C2 narrow-band polarization image of reaction human vas situation.It should be understood that the above citing is merely to illustrate, can not constitute
It is specific to limit.
In the application specific embodiment, multi-polarization state imaging device 130 can be polarized according to multiple narrow-bands schemes
The angle of polarization of light beam, degree of polarization and ellipticity angle obtain the polarization characteristic parameter of operative region as in, such as provide operative region
Organ surface roughness, texture trend, the signs such as surface orientation, water content, compare spectrum imaging device 120, multi-polarization state
Imaging device 130 can provide more accurate information to organ contours and surface orientation identification, the organ in prominent operative region
In picture with the contrast of background, and then improve next with organ classes' model, body fluid disaggregated model identify accuracy rate.
Therefore, the application combination spectrum imaging device 120 and multi-polarization state imaging device 130 carry out the three-dimensional reconstruction of operative region, can
To provide the information that more complete, more reliable and human eye can not extract for doctor.That is, spectrum imaging device 120 can be with
It is that feature extraction is carried out to operation region entirety, realizes that the preliminary three-dimensional of operative region is rebuild using multi-view stereo vision algorithm,
Multi-polarization state imaging device 130 can be the organ surface roughness that spectrum imaging device can not be shown, grain of meat trend,
It is clearly described in the profiles such as the class of liquids of organ surface covering and surface orientation progress feature extraction acquisition operative region
The surface roughness of human organ or the image of body fluid coverage condition.
In presently filed embodiment, processing system 140 is according to the multiple narrow-band spectrum image and multiple narrow waves
Section polarization image constructs the 3-D image of the operative region.That is, processing system combine narrow-band spectrum image and
The information that narrow-band polarization image extracts respectively establishes an operative region steric information figure detailed, abundant.Such as: place
Reason system can be narrow according to the B of different angle according to the 3-D image A of the A narrow-band spectrum image reconstruction skin of different angle
The 3-D image B of band spectrum image reconstruction muscle, according to the three-dimensional figure of the C narrow-band spectrum image reconstruction blood vessel of different angle
It is inclined according to the E narrow-band of different angle according to the 3-D image D of the D narrow-band spectrum image reconstruction bone of different angle as C
The 3-D image E of vibration image reconstruction capilary rebuilds each organ surface according to the F narrow-band polarization image of different angle and covers
The 3-D image F of liquid, by 3-D image A, 3-D image B, 3-D image C, 3-D image D, 3-D image E and three-dimensional figure
After being merged as F, so that it may which acquisition can reflect the coarse journey of skin, muscle, blood vessel, bone, capilary, organ surface simultaneously
Detailed, the true operative region 3-D image of degree.
In the application specific embodiment, the processing system 140 is also used to according to multiple narrow-band polarization images,
Combining classification model is to the body fluid components type and organ type progress instant analysis in the operative region and exports analysis
As a result, the instant analysis includes organ floristic analysing, kinds of protein analysis, protein content analysis, crystallization floristic analysing
And crystalline content analysis, wherein the disaggregated model is to be trained in advance using sample data to convolutional neural networks
The body fluid disaggregated model and organ classes' model arrived;The processing system is also used to determine whether the analysis result is greater than first
Threshold value is less than second threshold, wherein the first threshold is greater than the second threshold;The processing system is also used in institute
Analysis result is stated to be greater than the first threshold or less than in the case where the second threshold, issue prompt information.Wherein, body fluid
Disaggregated model and organ classes' model can be in advance using the body fluid picture sample and organ for being largely labeled with prediction result
Sample is trained the trained disaggregated model of acquisition to convolutional neural networks, and the body fluid disaggregated model can be according to more
The narrow-band polarization image comprising the aqueous measure feature of organ surface that polarization state imaging system obtains obtains body fluid classification results,
The narrow-band comprising organ surface contour feature that organ classes' model can be obtained according to multi-polarization state imaging system is inclined
Shake image, obtains organ classification results, and exports in the reality enhancing or mixed reality equipment that doctor wears, in surgical procedure
In for doctor provide more information, further to judge the state of an illness and to perform the operation.Wherein, prompt information can be sound,
One of light or text.For example, as shown in figure 3, text can be increased on 3-D image by the method for augmented reality
Content of hemoglobin is abnormal in blood samples of patients during surgical to remind for word.Also, multi-polarization state imaging device can be with
According to polarization optical information, the judgement of organ classes is carried out in conjunction with neural network model, can also pass through augmented reality or mixing
The method of reality increases text on 3-D image to remind doctor, and the application does not repeat.
In the application specific embodiment, processing system 140 is according to the multiple narrow-band spectrum image and multiple
The 3-D image that narrow-band polarization image constructs the operative region specifically may is that processing system by the operative site, institute
It states operation tool and operation label is positioned and divides to obtain locating segmentation as a result, to according to locating segmentation knot
Fruit determines the relative position between the operative site, the operation tool and the operation label, then, further according to the hand
Relative position reconstruction of three-dimensional images between art position, the operation tool and the operation label.It is appreciated that by multiple
Narrow-band spectrum image and multiple narrow-band polarization images can obtain information more abundant, therefore, it is possible to more accurately
The operative site, the operation tool and the operation label are positioned and divided.
In the application specific embodiment, processing system 140 is by the operative site, the operation tool and described
Operation label, which is positioned and divided to obtain locating segmentation result specifically and may is that, is obtaining multiple narrow-band spectrum images
After multiple narrow-band polarization images, processing system can be extracted more with feature extraction algorithm (for example, convolutional neural networks)
The textural characteristics of a narrow-band spectrum image and multiple narrow-band polarization images.Then, in conjunction with image recognition algorithm (for example,
Convolutional neural networks) and movement tracing algorithm label, operation tool and operation relevant range that operation consent is done carry out it is accurately fixed
Position or segmentation, to obtain locating segmentation result.
In the application specific embodiment, optionally, the processing system is also used to obtain the operative region
After three-dimensional reconstruction image, the rendering that the three-dimensional reconstruction image carries out target area is strengthened, the three-dimensional reconstruction after output reinforcing
Image, wherein the target area includes the specific region in the region being blocked in operative region and operative region.It is described
Processing system is specifically used in the case where target area is the region being blocked in operative region, occlusion removal target area
Position renders the target area not being blocked;Alternatively, the processing system is specifically used in target area being in operative region
Specific region in the case where, promote the brightness of the specific region, the target area after rendering luminance raising.
Preferably, the target area not being blocked rendered can be to be obtained by using the trained supplementary model that blocks
, the supplementary model that blocks is to use the image of the shelter in a large amount of binocular images and binocular image as sample,
It is obtained after being trained to the convolutional neural networks (SymmNet) of symmetrical structure, the supplementary model that blocks has learnt largely
Feature is blocked in binocular image, the occlusion issue of more mesh spectrum imaging device images can be predicted by bilateral symmetry
Network model.More operation informations can be brought for the operation of doctor using the result after supplementary model rendering is strengthened is blocked,
For example, certain operative region is as shown in fig. 4 a, in the case that target area is the region being blocked in operative region, doctor is used
Reality enhancing (AR) glasses can see the three-dimensional reconstruction image after rendering as shown in Figure 4 b is strengthened, and according to three after reinforcing
Dimension reconstruction image is performed the operation.It is understood that doctor can only basis before no rendering for carrying out target area is strengthened
Experience is performed the operation, and after the rendering for carrying out target area is strengthened, AR glasses are can be used according to rendering not in doctor
The progress of the target area image being blocked, auxiliary or guidance operation.It should be understood that the example above is merely to illustrate, occlusion area
Rendering other algorithms can also be used to obtain, such as use cross inspection (Left-right-cross-
Checking, LRC) algorithm infers the position blocked by the parallax result precalculated, the application is not especially limited.
Preferably, the target area after rendering luminance raising can be to be obtained using trained brightness supplementary model
, the brightness supplementary model is to be trained using the operative region samples pictures before and after luminance raising to convolutional neural networks
Obtained model, the result after being strengthened using the rendering of brightness supplementary model can bring more operation informations for the operation of doctor,
For example, as shown in Figure 5 a, when target area be operative region in specific region in the case where, for example, certain lesion regions or
The region that these doctors of person shadow region can not see by visual observation, doctor can see using after AR glasses such as Fig. 5 b institute
Three-dimensional reconstruction image after the rendering reinforcing shown, and performed the operation according to the three-dimensional reconstruction image after reinforcing.It is understood that
Before no rendering for carrying out target area is strengthened, doctor can only rule of thumb perform the operation, and having carried out, target area is bright
After degree is promoted, AR glasses are can be used according to the target area image after the luminance raising rendered, auxiliary or guidance in doctor
The progress of operation.
In the application specific embodiment, the target area is that processing system 140 is sent according to operation tool
The region that location information determines, wherein it is provided with gyroscope in the operation tool, accelerometer, one in range sensor
A or multiple, the location information is one according to the gyroscope, in the accelerometer and the range sensor
Or multiple collected position datas are calculated.Such as: when surgical knife is placed on a certain lesion region, doctor
AR glasses can show the three-dimensional reconstruction image after the lesion region highlights automatically.It should be understood that the above citing is merely to illustrate,
Specific restriction can not be constituted.
In the application specific embodiment, the target area can also be that processing system 140 is used based on the received
The determining region of family request.Such as: doctor needs to check in the course of surgery a certain target area being blocked, and doctor can at this time
It is looked into selecting needs by using other terminal device frames being connected with processing system by using AR equipment or feldsher
The target area being blocked seen, processing system determine target area according to the target area information received, mesh are rendered
Be not blocked target area image input doctor AR glasses in.It should be understood that the above citing is merely to illustrate, it can not structure
It is limited at specific.
In the application specific embodiment, the three-dimensional reconstruction image after the output reinforcing of processing system 140 can be defeated
The head-mounted display apparatus display screen (such as AR glasses, MR glasses etc.) for arriving doctor's wearing out, is also possible to be output to terminal
Device display screen (such as computer display screen, panel display screen etc.), can also be and be output in mechanical operation system, be machine
Tool arm carries out operation and provides operation guidance or auxiliary.
In above system, broad-spectrum light source generates broadband wide spectrum light, and the acquisition of spectrum imaging device multi-angle is described wide
Light is composed by multiple narrow-band spectrum images that the operative region reflects and is formed, constructs institute using multi-view stereo vision algorithm
The preliminary three-dimensional reconstruction image of operative region is stated, it is anti-by the operative region that multi-polarization state imaging device acquires the polarization light
The multiple narrow-band polarization images penetrated and formed obtain the body fluid covering image of the human organ in the operative region, so that
Processing system covers image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ, obtains the field of operation
The three-dimensional reconstruction image in domain, and export to display equipment.It, can be according to multiple narrow-band spectrum images and more by above system
A narrow-band polarization image obtains richer effective information, to reconstruct the higher 3-D image of accuracy, also, passes through
Virtual enhancing technology carries out rendering reinforcing according to doctor's demand to the 3-D image after reconstruction, so that guidance or auxiliary doctor are correct
It performs the operation, improves the precision of operation.
In several embodiments provided herein, it should be understood that disclosed system, terminal and method, it can be with
It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit
It divides, only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components
It can be combined or can be integrated into another system, or some features can be ignored or not executed.In addition, shown or beg for
Opinion mutual coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING of device or unit
Or communication connection, it is also possible to electricity, mechanical or other form connections.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.Some or all of unit therein can be selected to realize the embodiment of the present invention according to the actual needs
Purpose.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, is also possible to two or more units and is integrated in one unit.It is above-mentioned integrated
Unit both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially
The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words
It embodies, which is stored in a storage medium, including some instructions are used so that a computer
Equipment (can be personal computer, server or the network equipment etc.) executes the complete of each embodiment the method for the present invention
Portion or part steps.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only
Memory), random access memory (RAM, Random Access Memory), magnetic or disk etc. are various can store journey
The medium of sequence code.The above description is merely a specific embodiment, but protection scope of the present invention is not limited to
This, anyone skilled in the art in the technical scope disclosed by the present invention, can readily occur in various equivalent
Modifications or substitutions, these modifications or substitutions should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention
It should be subject to the protection scope in claims.
Claims (10)
1. a kind of 3-D image method for building up based on the imaging of more mesh and multi-polarization state imaging, which is characterized in that the method packet
It includes:
Broad-spectrum light source generates broadband wide spectrum light, wherein the wide spectrum light is for irradiating operative region, the broad-spectrum light
Line includes available light and polarization light, and the operative region is the body region that patient is performed the operation;
Spectrum imaging device multi-angle acquires the wide spectrum light by the operative region reflects and is formed multiple narrow-band light
Spectrogram picture constructs the preliminary three-dimensional reconstruction image of the operative region using multi-view stereo vision algorithm, wherein different narrow waves
Section spectrum picture has different imaging narrow-bands, and the quantity of the imaging narrow-band is 100 or more;
Multi-polarization state imaging device acquires the polarization light and is polarized by multiple narrow-bands that the operative region reflects and is formed
Image obtains the body fluid covering image of the human organ in the operative region;
Processing system covers image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ, obtains the hand
The three-dimensional reconstruction image in art region, and export to display equipment.
2. the method according to claim 1, wherein the method also includes:
The processing system is according to multiple narrow-band polarization images, and combining classification model is to the body fluid components in the operative region
Type and organ type carry out instant analysis and export analysis as a result, the instant analysis includes organ floristic analysing, albumen
Matter floristic analysing, protein content analysis, crystallization floristic analysing and crystalline content analysis, wherein the disaggregated model is pre-
The body fluid disaggregated model and organ classes' model that first convolutional neural networks are trained using sample data;
Determine whether protein content or crystalline content in the analysis result are greater than first threshold or less than the second threshold
Value, wherein the first threshold is greater than the second threshold;
Protein content or crystalline content in the analysis result are greater than the first threshold or less than described second
In the case where threshold value, prompt information is issued.
3. the method according to claim 1, wherein after obtaining the three-dimensional reconstruction image of the operative region, institute
State method further include:
The rendering that the three-dimensional reconstruction image carries out target area is strengthened, the three-dimensional reconstruction image after output reinforcing, wherein institute
Stating target area includes the specific region in the region being blocked in operative region and operative region.
4. according to the method described in claim 3, it is characterized in that, the three-dimensional reconstruction image to be carried out to the rendering of target area
Reinforcing includes:
In the case where target area is the region being blocked in operative region, the position of target area described in occlusion removal, wash with watercolours
Dye the target area not being blocked;Alternatively,
In the case where target area is the specific region in operative region, the brightness of the specific region is promoted, is rendered bright
Target area after degree promotion.
5. according to the method described in claim 4, it is characterized in that,
The target area is the region that processing system is determined according to the location information that operation tool is sent, wherein the operation
It is provided with gyroscope in tool, accelerometer, one or more in range sensor, the location information is according to
Gyroscope, one or more collected position data in the accelerometer and the range sensor are calculated
's;Alternatively,
The target area is that user requests determining region to processing system based on the received.
6. a kind of 3-D image based on the imaging of more mesh and multi-polarization state imaging establishes system characterized by comprising
Broad-spectrum light source, broad-spectrum light source generate broadband wide spectrum light, wherein and the wide spectrum light is used to irradiate operative region,
The wide spectrum light includes available light and polarization light, and the operative region is the body region that patient is performed the operation;
Spectrum imaging device acquires the wide spectrum light by the operative region reflects and is formed multiple narrow waves for multi-angle
Section spectrum picture, the preliminary three-dimensional reconstruction image of the operative region is constructed using multi-view stereo vision algorithm, wherein different
Narrow-band spectrum image has different imaging narrow-bands, and the quantity of the imaging narrow-band is 100 or more;
Multi-polarization state imaging device, for acquiring the wide spectrum light by the operative region reflects and is formed multiple narrow-bands
Polarization image obtains the body fluid covering image of the human organ in the operative region;
Processing system, processing system cover image according to the body fluid of the preliminary three-dimensional reconstruction image and the human organ,
The three-dimensional reconstruction image of the operative region is obtained, and is exported to display equipment.
7. system according to claim 6, which is characterized in that
The processing system is also used to according to multiple narrow-band polarization images, and combining classification model is to the body in the operative region
Liquid ingredient type and organ type carry out instant analysis and export analysis as a result, the instant analysis includes organ type point
Analysis, kinds of protein analysis, protein content analysis, crystallization floristic analysing and crystalline content analysis, wherein the classification mould
Type is the body fluid disaggregated model and organ classes' model being trained in advance using sample data to convolutional neural networks;
The processing system is also used to determine whether protein content or crystalline content in the analysis result are greater than first
Threshold value is less than second threshold, wherein the first threshold is greater than the second threshold;
The processing system is also used to be greater than first threshold in the protein content analyzed in result or crystalline content
In the case where being worth or being less than the second threshold, prompt information is issued.
8. system according to claim 6, which is characterized in that processing system is also used to obtain the three-dimensional of the operative region
After reconstruction image, the rendering that the three-dimensional reconstruction image carries out target area is strengthened, the three-dimensional reconstruction image after output reinforcing,
Wherein, the target area includes the specific region in the region being blocked in operative region and operative region.
9. system according to claim 8, which is characterized in that
The processing system is specifically used in the case where target area is the region being blocked in operative region, occlusion removal mesh
The position for marking region, renders the target area not being blocked;Alternatively,
The processing system is specifically used for being promoted described specific in the case where target area is the specific region in operative region
The brightness in region, the target area after rendering luminance raising.
10. system according to claim 9, which is characterized in that
The target area is the region that processing system is determined according to the location information that operation tool is sent, wherein the operation
It is provided with gyroscope in tool, accelerometer, one or more in range sensor, the location information is according to
Gyroscope, one or more collected position data in the accelerometer and the range sensor are calculated
's;Alternatively,
The target area is that user requests determining region to processing system based on the received.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811625313.0A CN109771052B (en) | 2018-12-28 | 2018-12-28 | Three-dimensional image establishing method and system based on multi-view imaging and multi-polarization state imaging |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811625313.0A CN109771052B (en) | 2018-12-28 | 2018-12-28 | Three-dimensional image establishing method and system based on multi-view imaging and multi-polarization state imaging |
Publications (2)
Publication Number | Publication Date |
---|---|
CN109771052A true CN109771052A (en) | 2019-05-21 |
CN109771052B CN109771052B (en) | 2021-07-27 |
Family
ID=66498721
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201811625313.0A Active CN109771052B (en) | 2018-12-28 | 2018-12-28 | Three-dimensional image establishing method and system based on multi-view imaging and multi-polarization state imaging |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109771052B (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110197156A (en) * | 2019-05-30 | 2019-09-03 | 清华大学 | Manpower movement and the shape similarity metric method and device of single image based on deep learning |
CN113724372A (en) * | 2021-08-23 | 2021-11-30 | 水利部交通运输部国家能源局南京水利科学研究院 | Imaging device and fish body three-dimensional modeling method based on orthogonal polarization double-view imaging |
CN113763566A (en) * | 2020-06-05 | 2021-12-07 | 光宝电子(广州)有限公司 | Image generation system and image generation method |
Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050194537A1 (en) * | 2004-03-03 | 2005-09-08 | Fauci Mark A. | Integrated multi-spectral imaging systems and methods of tissue analyses using same |
US20070237374A1 (en) * | 2006-03-02 | 2007-10-11 | Kollias Nikiforos | Method for demonstrating pre-emergent pimples |
CN101313847A (en) * | 2008-07-01 | 2008-12-03 | 北京师范大学 | Apparatus and method for nondestructive optical constant imaging of dermatosis tissue of human body |
CN201806692U (en) * | 2009-12-31 | 2011-04-27 | 中国人民解放军空军总医院 | Multispectral dermoscopy image automatic analytical instrument for diagnosing malignant melanocyte tumour |
CN103018179A (en) * | 2012-12-11 | 2013-04-03 | 江苏大学 | Device and method for near-infrared polarization detection of crop water stress |
CN103743649A (en) * | 2013-12-31 | 2014-04-23 | 北京大学 | Detection device of ground feature density and rocks based on multi-angle polarization imaging |
CN103815875A (en) * | 2013-10-28 | 2014-05-28 | 重庆西南医院 | Near-infrared spectrum imaging system and near-infrared spectrum imaging method for diagnosis of depth and area of burn skin necrosis |
CN105744883A (en) * | 2013-11-20 | 2016-07-06 | 乔治华盛顿大学 | Systems and methods for hyperspectral analysis of cardiac tissue |
CN106236060A (en) * | 2015-06-04 | 2016-12-21 | 松下知识产权经营株式会社 | Bioinformation detecting device |
CN106901698A (en) * | 2017-04-14 | 2017-06-30 | 中国人民解放军第三军医大学第附属医院 | Two waveband optical spectrum imagers and its spectrometric method based on adjustable polarization |
US20170199120A1 (en) * | 2014-06-24 | 2017-07-13 | Nikon Corporation | Image analysis apparatus, imaging system, surgery support system, image analysis method, and storage medium |
US20180242901A1 (en) * | 2008-05-13 | 2018-08-30 | Spectral Image, Inc. | Systems and methods for hyperspectral medical imaging using real-time projection of spectral information |
-
2018
- 2018-12-28 CN CN201811625313.0A patent/CN109771052B/en active Active
Patent Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050194537A1 (en) * | 2004-03-03 | 2005-09-08 | Fauci Mark A. | Integrated multi-spectral imaging systems and methods of tissue analyses using same |
US20070237374A1 (en) * | 2006-03-02 | 2007-10-11 | Kollias Nikiforos | Method for demonstrating pre-emergent pimples |
US20180242901A1 (en) * | 2008-05-13 | 2018-08-30 | Spectral Image, Inc. | Systems and methods for hyperspectral medical imaging using real-time projection of spectral information |
CN101313847A (en) * | 2008-07-01 | 2008-12-03 | 北京师范大学 | Apparatus and method for nondestructive optical constant imaging of dermatosis tissue of human body |
CN201806692U (en) * | 2009-12-31 | 2011-04-27 | 中国人民解放军空军总医院 | Multispectral dermoscopy image automatic analytical instrument for diagnosing malignant melanocyte tumour |
CN103018179A (en) * | 2012-12-11 | 2013-04-03 | 江苏大学 | Device and method for near-infrared polarization detection of crop water stress |
CN103815875A (en) * | 2013-10-28 | 2014-05-28 | 重庆西南医院 | Near-infrared spectrum imaging system and near-infrared spectrum imaging method for diagnosis of depth and area of burn skin necrosis |
CN105744883A (en) * | 2013-11-20 | 2016-07-06 | 乔治华盛顿大学 | Systems and methods for hyperspectral analysis of cardiac tissue |
CN103743649A (en) * | 2013-12-31 | 2014-04-23 | 北京大学 | Detection device of ground feature density and rocks based on multi-angle polarization imaging |
US20170199120A1 (en) * | 2014-06-24 | 2017-07-13 | Nikon Corporation | Image analysis apparatus, imaging system, surgery support system, image analysis method, and storage medium |
CN106236060A (en) * | 2015-06-04 | 2016-12-21 | 松下知识产权经营株式会社 | Bioinformation detecting device |
CN106901698A (en) * | 2017-04-14 | 2017-06-30 | 中国人民解放军第三军医大学第附属医院 | Two waveband optical spectrum imagers and its spectrometric method based on adjustable polarization |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110197156A (en) * | 2019-05-30 | 2019-09-03 | 清华大学 | Manpower movement and the shape similarity metric method and device of single image based on deep learning |
CN110197156B (en) * | 2019-05-30 | 2021-08-17 | 清华大学 | Single-image human hand action and shape reconstruction method and device based on deep learning |
CN113763566A (en) * | 2020-06-05 | 2021-12-07 | 光宝电子(广州)有限公司 | Image generation system and image generation method |
CN113724372A (en) * | 2021-08-23 | 2021-11-30 | 水利部交通运输部国家能源局南京水利科学研究院 | Imaging device and fish body three-dimensional modeling method based on orthogonal polarization double-view imaging |
CN113724372B (en) * | 2021-08-23 | 2022-06-28 | 水利部交通运输部国家能源局南京水利科学研究院 | Imaging device and fish body three-dimensional modeling method based on orthogonal polarization double-view imaging |
Also Published As
Publication number | Publication date |
---|---|
CN109771052B (en) | 2021-07-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11350072B1 (en) | Augmented reality guidance for bone removal and osteotomies in spinal surgery including deformity correction | |
US11145121B2 (en) | Surface and image integration for model evaluation and landmark determination | |
Hacihaliloglu et al. | Local phase tensor features for 3-D ultrasound to statistical shape+ pose spine model registration | |
US20130170726A1 (en) | Registration of scanned objects obtained from different orientations | |
JP5295562B2 (en) | Flexible 3D rotational angiography-computed tomography fusion method | |
CN109771052A (en) | 3-D image method for building up and system based on the imaging of more mesh and multi-polarization state imaging | |
CA2985061A1 (en) | A spinal navigation method, a spinal navigation system and a computer program product | |
US11676706B2 (en) | Medical image processing apparatus and medical image processing method which are for medical navigation device | |
EP3143599A1 (en) | Indication-dependent display of a medical image | |
US10667786B2 (en) | Ultrasound imaging apparatus and method for segmenting anatomical objects | |
KR102433473B1 (en) | Method, apparatus and computer program for providing augmented reality based medical information of patient | |
Hacihaliloglu et al. | Statistical shape model to 3D ultrasound registration for spine interventions using enhanced local phase features | |
CN109191475A (en) | Terminal plate of vertebral body dividing method, device and computer readable storage medium | |
Bucioli et al. | Holographic real time 3D heart visualization from coronary tomography for multi-place medical diagnostics | |
EP3459442B1 (en) | Method and system for supporting a medical brain mapping procedure | |
CN111445575A (en) | Image reconstruction method and device of Wirisi ring, electronic device and storage medium | |
Oriola et al. | Virtual anthropology: Forensic applications to cranial skeletal remains from the Spanish Civil War | |
US9639977B2 (en) | Processing model data representing a biological body section having a defect portion | |
CN115607286B (en) | Knee joint replacement surgery navigation method, system and equipment based on binocular calibration | |
US20230360334A1 (en) | Positioning medical views in augmented reality | |
KR20190069751A (en) | Point based registration apparatus and method using multiple candidate points | |
US11869216B2 (en) | Registration of an anatomical body part by detecting a finger pose | |
CN109528313B (en) | Spectrum three-dimensional positioning and identifying system | |
EP3436992A1 (en) | Biomechanical model generation for human or animal torsi | |
WO2024141152A1 (en) | Conjunction of 2d and 3d visualisations in augmented reality |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |