Embodiment
Below in conjunction with accompanying drawing and embodiment, the embodiment of the utility model is described in further detail.Following examples are used to explain the utility model, but are not used for limiting the scope of the utility model.
Fig. 1 is described thermal imagery thermometric of the utility model embodiment and localization of fault inspection tour system structural drawing.As shown in Figure 1, this thermal imagery thermometric comprises with the localization of fault inspection tour system: the Video service applicator platform and the client platform that are connected through Ethernet.
Said Video service applicator platform; Mainly comprise many interconnected binary channels network video servers of group and router; The binary channels network video server comprises thermal infrared camera and visible image capturing head; Be used to gather with pre-service and make an inspection tour on-the-spot visible images and thermal infrared images, and pretreated visible images and thermal infrared images are sent to said client platform.
Said client platform mainly comprises mutual connection supervisory control comuter and monitoring server, and said supervisory control comuter is connected with Ethernet with router through hub with monitoring server, and then connects said Video service applicator platform.Said client platform is used to adopt method for registering images and thermal imagery temp measuring method that the said visible images and the thermal infrared images that receive are handled, and the visible images after will handling and thermal infrared images show; Said client platform also is used for to said Video service applicator platform transmitting control commands.
Said client platform comprises video preview monitoring temperature module; Said video preview monitoring temperature module is used to adopt method for registering images and thermal imagery temp measuring method that said visible images and thermal infrared images are handled (concrete processing procedure such as following step S100~S400 and P100~P500), and be used for said visible images and the thermal infrared images after the synchronous display process respectively accordingly.Said video preview monitoring temperature module also is used for dynamically showing the temperature maximal value and the minimum value of said visible images and thermal infrared images subregion or full figure.Said video preview monitoring temperature module also is used for the multi-channel video of said visible images and thermal infrared images is carried out preview.
Fig. 3 is the process flow diagram of the said method for registering images of the utility model embodiment.As shown in Figure 3, said method for registering images comprises step:
S100: judge whether the thermal infrared images of object (being the photographic subjects of said thermal infrared images and visible images) and the right quantity of coupling of visible images surpass 15 pairs, if, execution in step S300, otherwise, execution in step S200.
It is following to mate right deterministic process:
Read in thermal infrared images and visible images, be designated as img1, img2 respectively.Thermal infrared images and 255 is got difference, be converted into its negative-appearing image, visible images is converted into gray level image;
Img1, img2 are carried out based on the redundant figure image intensifying of gray scale;
Detect the unique point of img1 and img2 respectively with SURF (Speeded Up Robust Features, fast robust property feature detection) operator, and the 64 dimension descriptors of confirming these points;
Unique point is slightly mated: being similarity measurement with the Euclidean distance matees feature description of two width of cloth images.Certain key point in the heat-obtaining infrared image; And find out European nearest preceding two key points in itself and the visible images, in these two key points,, nearest distance closely is less than certain proportion threshold value in proper order if removing; Then accept this a pair of match point, this paper threshold value gets 0.7.Obtain the right set P of coupling of img1, img2, i.e. the feature point set I1 of img1 and the feature point set I2 of corresponding img2 with it.
The smart coupling of unique point: through RANSAC (Random Sample Consensus) algorithm, i.e. random sampling consistency algorithm, remove mistake mate right.Mainly be divided into following three steps:
1. { (x
1, y
1)~(x
1', y
1'), (x
2, y
2)~(x
2', y
2'), (x
3, y
3)~(x
3', y
3') for 3 couplings randomly drawing among the P are right, according to affine model, promptly formula (1) is asked affine coefficients.According to 3 couplings derivation is drawn ask affine coefficients formula shown in formula (2).The affine model parameter that utilization is obtained is carried out affined transformation respectively to each unique point I1i of point set I1, obtains new point set I
2', ask for the error e of each point
i=‖ I
2i'-I
2i‖ belongs to error the interior point of I1 less than the point of a certain threshold value.
2. repeat N step 1., the interior some set that more at every turn obtains obtains wherein maximum interior some set.According to experiment experience, N generally gets 200.
3. repeating step 1. 2., the unanimity of in calculating, counting out is confirmed last interior some set C, wherein the point among the img1 is (X
i, Y
i), the point of img2 be (X '
i, Y '
i).
S200: obtain many group thermal infrared images and the visible images of object of reference apart from the camera different distance place of inspection tour system, at least 3 pairs of couplings of on the thermal infrared images of every group of said object of reference and visible images, looking for are right.Generally speaking, with 0.8 meter be single order, obtain between object of reference and the camera distance from 0.8 meter to 20 meters many groups thermal infrared images and the visible images in the variation range.It is the most clear that the focal length that will guarantee the thermal infrared camera lens when obtaining the thermal infrared images of object of reference transfers to.
S300: the coupling according among said step S100 or the S200 is right, uses least square method, tries to achieve affine parameter.Coupling in using step S200 to the time, need be according to the distance of object apart from camera, it is right that thermal infrared images and 3 couple on the visible images who chooses the said object of reference of corresponding group mated.The selection standard of corresponding group is that the distance of object and camera is near the distance of object of reference and camera.
S400: thermal infrared images and visible images to said object carry out based on the redundant image enhancement processing of gray scale; According to said affine parameter the thermal infrared images of said object is carried out affined transformation; Obtain the 3rd image through linear interpolation processing, the visible images and the 3rd image of said object carried out image co-registration.
Said step S200 specifically comprises step:
S201: corner angle are arranged with one; Emissivity is high, and the object that color and background are clearly demarcated is an object of reference, at the different distance place; Through the thermal infrared camera and the visible image capturing head of said thermal imagery thermometric and localization of fault inspection tour system, obtain the many groups thermal infrared images and the visible images of said object of reference;
S202: it is right on every group of thermal infrared images and visible images, to obtain three pairs of couplings, and said three pairs of match points can guarantee that the fused images after this group thermal infrared images and the visible images affined transformation is normal.
Fig. 4 is the process flow diagram of the said thermal imagery temp measuring method of the utility model embodiment.As shown in Figure 4, said thermal imagery temp measuring method comprises step:
P100: the thermal infrared images to object carries out the calorific value reduction, obtains the calorific value of said each pixel of thermal infrared images.Its calorific value reduction formula is following:
V=Gray
1*(V
max-V
min)/255+V
min (1)
Wherein, V
MaxRepresent calorific value maximal value in each pixel, V
MinRepresent calorific value minimum value in each pixel, Gray
18 gray-scale values of remarked pixel, the calorific value of V remarked pixel.
P200: according to making an inspection tour on-the-spot environment temperature and calorific value compensated curve, the calorific value that calculates said environment temperature correspondence is poor.
P300:, obtain the calorific value after calorific value compensates with the calorific value and the addition of said calorific value difference of said each pixel.
P400:, calculate the corresponding temperature value of said each pixel according to the calorific value after the said compensation and calorific value-temperature curve.
P500: after the influence that the distance between the emissivity of object and said object and the said inspection tour system is produced compensates, obtain the corresponding true temperature value of said each pixel.
Said calorific value-temperature curve and calorific value compensated curve obtain through following calibration experiment:
Calibration tool: the HL1 of Nan Qixing company type black matrix, one of climatic chamber, one of notebook.
Demarcate object: thermal imagery thermometric and localization of fault inspection tour system (hereinafter to be referred as the thermal imagery system)
Experiment is prepared: the temperature of climatic chamber is made as the environment temperature of 25 degree as experiment, with 1 meter of blackbody chamber distance.The thermal imagery system is put into said climatic chamber, and the thermal infrared camera of thermal imagery system is aimed at the germanite glass of climatic chamber, and make the two center and blackbody chamber center on same horizontal line.
(1) calorific value-temperature curve is asked for experimental procedure:
Record data:
Demarcate temperature T
1(30 degree are to 90 degree).
Environment temperature T
2(25 degree)
Actual calorific value computation process:
Demarcate temperature T
1With 5 degree is that single order changes, and obtains a plurality of thermal infrared imagess, demarcates temperature T according to each rank
1Calorific value maximal value V in the corresponding thermal infrared images
MaxWith minimum value V
Min,, obtain and demarcate temperature spot gray-scale value Gray through following formula (2)
1Corresponding black matrix calorific value V.
V=Gray
1*(V
max-V
min)/255+V
min) (2)
With so a series of demarcation temperature T
1Pass through least square fitting with its corresponding calorific value V, fit to secondary line linearity curve (3), wherein y is T
1, x is calorific value V.
y=a*x
2+b*x+c (3)
During actual thermometric according to above-mentioned curve, the corresponding relation of calorific value and blackbody temperature just.
(2) the calorific value compensated curve is asked for step
Experiment is prepared the same, just will demarcate temperature here and fix, and the conversion environment temperature is asked for environment temperature T
1' and calorific value change the relation between the V '.
Record data:
Demarcate temperature T
1(50 degree)
Environment temperature T
2(0-50 degree)
Calorific value compensated curve computation process:
Environment temperature is initially set to 0 degree, the corresponding calorific value V of record blackbody chamber temperature at this moment
0
Regulate the climatic chamber temperature, make its environment temperature T
2With 5 degree is that single order changes, and changes to 50 degree from 0 degree, and notes corresponding calorific value V, and computing heating value changes V '=V-V
0
So a series of calorific values are changed V ' and its corresponding environment temperature T
2Through least square fitting, fit to secondary line linearity curve (4), wherein y is V ', x is T
2
y=a*x
2+b*x+c (4)
The corresponding true temperature value T of each pixel described in the said step P500
rTemperature compensation computing formula (5) as follows:
Wherein, ε representes the object under test emissivity; τ representes atmospheric transmissivity;
The blackbody equivalent temperature that expression observes; A representes the object under test reflectivity;
Expression radiation background temperature; ε
aExpression atmosphere emissivity;
The expression atmospheric temperature; N representes the coefficient relevant with radiation wavelength, and when radiation wavelength was 8~14 microns, n got 4; ε
a=1-τ, a=1-ε.
Said thermal imagery thermometric algorithm can be realized:
(1), two-way thermometric: optionally on visible images a bit can correspondence mark corresponding point on the thermal infrared images, and the temperature that demonstrates this point; On thermal infrared images, choose wantonly a bit, can corresponding mark the corresponding point on the visible images, and the temperature that demonstrates this point.
(2), regional thermometric: an optional zone on thermal infrared images mark this regional temperature maximal value and minimum value, and correspondence is presented on the visible images.
(3), image co-registration: the fused images of infrared image and visible images is shown, see the Overlay of the two, more intuitively the position at failure judgement place.
(4), the three-phase temperature difference measures automatically: the inconsistent meeting of three-phase directly causes system's phase-deficient operation, causes transformer and consumer phase-deficient operation, thereby causes voltage to raise or dead electricity, influences equipment operation even burns.On visible images, discern three-phase breaker automatically, record its three-phase temperature difference then, trouble saving takes place, and this is the function that embeds to power equipment specially.
Said client platform also comprises the inspection management module, is used for to said Video service applicator platform transmitting control commands, carries out corresponding policer operation with the The Cloud Terrace and the camera lens of the said Video service applicator platform of remote control.
Said client platform also comprises alarm module, is used for the design temperature alarm threshold value, and when the temperature of making an inspection tour on-the-spot device surpasses said temperature alarming threshold value, reports to the police.This alarm module can exceed then and report to the police thermal infrared images zone, full figure design temperature alarm threshold value.
Said client platform also comprises the playing back videos module, is used in real time, regularly video recording, and is used for the device context that exceeds the temperature alarming threshold value is recorded a video, and be used to carry out function such as playing back videos montage.
Said client platform also comprises, is used to carry out the log management module of log record and management, is used to carry out the user management module of subscriber information management, is used to carry out the device management module of equipment information management.
Said Video service applicator platform comprises with TMS320DM355 being the processing unit of primary processor.Fig. 2 is the described processing unit module structural drawing of the utility model embodiment.As shown in Figure 2, primary processor TMS320DM355 digital video SOC(system on a chip) (being called for short DM355) is a DSP (Digital SignalProcessing, the digital signal processing) chip that aims at the high integration of low-power consumption portable digital media Application Design.The ARM926EJ-S nuclear that it uses is one 32 processor core, and the clock rate of 216MHz or 270MHz can be provided, and it has adopted the two-stage pipelining simultaneously.ARM926EJ-S nuclear guarantees the running of operating system emphatically, the work of management peripheral interface.DM355 is also integrated MPEG/JPEG coprocessor can be realized the MPEG-4 encoding and decoding of digital video, has improved the real-time of data processing greatly.Chip also has peripheral hardware in the abundant sheet; Like ASP (Audio Serial Port; The audio frequency serial ports), timer, I2C (Inter-Integrated Circui) EBI, UART (UniversalAsynchronous Receiver/Transmitter; Universal asynchronous reception/dispensing device) interface, MMC (Multimedia Card, multimedia card)/SD (Secure Digital Memory Card, safe digital card) card etc.This processing unit has also extended out visible light video acquisition module, thermal infrared images acquisition module, humiture module and Yun Jing control module as required except basic functions module (memory module power module and mixed-media network modules mixed-media).
Said visible light video acquisition module adopts CCD (Charge-coupled Device; Charge coupled cell) imageing sensor; Be used to gather the visible images at the scene of tour; Said visible images is carried out signal amplify and analog to digital conversion, send to said client platform through said mixed-media network modules mixed-media then.
Said thermal infrared images acquisition module adopts FPGA (Field-Programmable GateArray; Field programmable gate array); Be used to gather the thermal infrared images at the scene of tour; Said thermal infrared images is carried out format conversion, normalization processing and JPEG (JointPhotographic Experts Group, joint image expert group) compression successively, send to said client platform through said mixed-media network modules mixed-media then.
Said cloud mirror control module; Adopt DM355 external interrupt interface and UART485 (Universal Asynchronous Receiver/Transmitter485; Universal asynchronous reception/dispensing device 485) bus is used for receiving through said mixed-media network modules mixed-media the control command of said client platform, and according to said control command; The The Cloud Terrace of controlling said Video service applicator platform changes the monitoring angle, and the camera lens of controlling said Video service applicator platform changes focal length.
What the humiture module adopted is the Temperature Humidity Sensor of SHT1x/SHT7x series.This sensor comprises two calibrated miniature temperature and humidity sensors, adopts full calibration output, and need not to demarcate is interchangeable use, and two-wire system digital interface and request formula are measured.It with GPIO (General PurposeInput Output, general input and output) mouthful of connection of DM355, carries out data transmit-receive through 14 built-in bit A/D converter SOD serial output datas.
The course of work of said thermal imagery thermometric of the utility model embodiment and localization of fault inspection tour system is following:
After system powered on, client platform was logined the video server website through supervisory control comuter input username and password; If network is unobstructed and video server is in proper working order, then the display video server state is not work, not then be off-line state; The click website connects; To client platform, supervisory control comuter shows each the road video that inserts through video preview monitoring temperature module through Network Transmission for thermal infrared images that video server will be gathered in real time and visible images.
In the inspection management module of client platform, click and stop to wait various command up and down; Instruction is arrived the Video service applicator platform through Network Transmission; Resolve to corresponding cradle head control instruction and camera lens control command, and through cloud mirror control module, control The Cloud Terrace and camera lens are accomplished corresponding operation.
The thermometric process is divided into unidirectional thermometric and two kinds of patterns of two-way thermometric.During unidirectional thermometric, the humiture module of Video service applicator platform is gathered the humiture parameter, can be through the thermometric algorithm to thermal infrared images zone or full figure thermometric, and will measured temperature value and the temperature alarming threshold ratio, exceeding then reports to the police and begin records a video; The humiture module of Video service applicator platform is gathered the humiture parameter equally, through image registration algorithm and thermometric algorithm, visible images and thermal infrared images real time fusion identification hot stall parts is also reported to the police, is recorded a video.
Each hardware components of this inspection tour system adopts modular design, and each module can be upgraded or replace at any time, has good versatility.The Video service applicator platform of this inspection tour system, has guaranteed the efficient operation of system, and has reduced power consumption through using high performance TMS320DM355 Digital Media System as primary processor based on the exploitation of embedded system platform.
The Video service applicator platform of this inspection tour system adopts FPGA as the collection of thermal infrared images data and the processor of front-end processing module, has high speed and high flexibility, for the further optimization and the expansion of system provides the foundation.
The video preview monitoring temperature module of this inspection tour system has adopted redundant and the thermal infrared of SURF algorithm and the image registration of visible images based on gray scale, and its speed is faster, and robustness is higher, and registration accuracy is also higher, thereby has effectively improved the precision of localization of fault.Through the temp measuring method after black matrix demarcation and the temperature compensation, make temperature measurement accuracy higher in addition.
Above embodiment only is used to explain the utility model; And be not the restriction to the utility model; The those of ordinary skill in relevant technologies field under the situation of spirit that does not break away from the utility model and scope, can also be made various variations and modification; Therefore all technical schemes that are equal to also belong to the category of the utility model, and the scope of patent protection of the utility model should be defined by the claims.