CN101936761B - Visual measuring method of stockpile in large-scale stock ground - Google Patents

Abstract

The invention relates to a non-contact measuring method, in particular to visual measuring method of a stockpile in a large-scale stock ground. The method comprises the following steps of: (1) constructing a visual measuring system of stock ground; (2) mounting a binocular vision subsystem; (3) establishing a geometric model of a CCD (Charge Coupled Device) camera; (4) calibrating the parameters of the CCD camera; (5) collecting images; (6) processing the images; (7) searching and matching angular points; (8) confirming the three-dimensional coordinate values of an object point; and (9) interpolating a curve and establishing a height map in a world coordinate system; and (10) measuring the volume. The visual measuring method of the invention automatically eliminates the image distortion caused by a wide-angle lens, can automatically and accurately match the angular points of a left view and a right view, automatically generates the three-dimensional space coordinates of the object point by utilizing an image data conversion relation, optimizes the three-dimensional angular point distribution and constructs the height map of the world coordinate system so that the volume a geometrical irregular stockpile can be simply, intuitively and accurately calculate.

Description

Large-scale stock ground stockpile vision measuring method

Technical field

What the present invention relates to is a kind of non-contact measurement method, specifically is a kind of large-scale stock ground stockpile vision measuring method.

Background technology

Up to now; The heap of the large-scale stock yards that enterprise had such as iron and steel, chemical industry, power plant is got material operation and input and output material planning; Basically still relying on the manually-operated is main management and construction pattern, has therefore influenced the lifting of these whole enterprise robotizations and informationization technology level dramatically.

Because the raw material of being stacked in this type of stock ground is bulk materials; Like iron ore sand, coke, lime etc., in process of production, its volume is in a kind of dynamic change state all the time; Generally speaking; Enterprise measures these storage of raw materials amounts and dynamic management all relies on the method for manual measurement to carry out, and inefficiency also will be shut down at regular intervals and carried out the continuity that manual measurement influences enterprise production.Therefore, realize that the automatic management in stock ground certainly will at first realize the digitizing technique in stock ground, and the key point of stock ground digitizing technique is how to realize measuring automatically and dynamic observing of stock ground volume.Begun to adopt laser ranging technique to realize measuring automatically and dynamic observing in the prior art to the stock ground volume; Because there is the device complicacy in the application of laser technology in this field, working environment required condition restriction such as harshness, expensive, makes the popularization of this technology that significant limitation arranged.Measuring technique based on machine vision is current a kind of emerging technology, but because its technology is not perfect, therefore the beyond example still of the application in the digitization system of stock ground does not have ready-made mature technology to use for reference in addition.

Retrieval through to the prior art document is found; Chinese invention patent numbers 200510026200.5 discloses the method that the computer vision system measured in a kind of stock ground is measured automatically; This invention is to selected raw material stockpile design of CCD video camera layout, and the picture signal that ccd video camera collected is transported to the Flame Image Process front end via vision cable and image pick-up card; Image processing software carries out the processing of background flatten and edge extracting in real time to picture signal in the Flame Image Process front end, accomplishes the digital information of image and expresses; The information that the digitizing of Flame Image Process front end is expressed is connected with host computer through the IEEE1394 protocol interface, realizes the information interaction of Flame Image Process front end and host computer; Host computer is reprocessed the digitizing expressing information of image, asks for the 3-D view of public angle point reconstruct object, the space geometry yardstick measurement result of the output of computing meanwhile stockpile.Can affirm; This invention technical method is effectively in the computer automatic analysis process that realizes the stock ground, and still, this technology too relies on " ccd video camera layout; satisfy as high-order vision collecting point, gather the stockpile panoramic picture " to carry out the collection to the stock ground image information; Simultaneously, this invention does not still possess influence factors such as pattern distortion is eliminated ability automatically.

Summary of the invention

The object of the present invention is to provide a kind of large-scale stock ground stockpile vision measuring method; This measuring method is based on the stock ground volume measuring method of machine vision; Utilize stock ground existing stacker-reclaimer walking mechanism; In the stacker-reclaimer operation process, accomplish stock ground IMAQ and processing automatically, finally realize the accurate Calculation of material stack volume.

A kind of large-scale stock ground stockpile vision measuring method may further comprise the steps:

Step 1, structure stock ground vision measurement system; Form by the first binocular vision subsystem, the second binocular vision subsystem and server three parts, the image information processing module of server receive handle behind the signal of the first binocular vision subsystem, the second binocular vision subsystem and computing after output finally to identification, understanding and the measurement result of image information;

Step 2, installation binocular vision subsystem; The first binocular vision subsystem is installed on stacker near on stockpile one side frame; The second binocular vision subsystem is installed on reclaimer near on stockpile one side frame, and the installation of two ccd video cameras in the first and second binocular vision subsystems should be followed: the photocentre of two video cameras is in same surface level and two optical axises and two photocentre lines and forms an isosceles triangle;

Step 3, set up the ccd video camera geometric model; Comprise: the conversion to the computer picture coordinate system of mathematical model, normalization imaginary plane image coordinate system distortional point coordinate is corrected in pattern distortion in the conversion of world coordinate system and camera coordinate system, desirable perspective projection transformation, the normalization imaginary plane image coordinate system, obtains to be tied to from world coordinates the mapping relations equation of computer picture coordinate system at last;

Step 4, ccd video camera parameter calibration; Comprise inner parameter and external parameter, adopt the substep scaling method, the aspect ratio of first uncalibrated image and principal point coordinate; Utilize the radial parallel constraint again; Find the solution most external parameter, introduce distortion model, linear solution inner parameter and translational movement at last again;

Step 5, IMAQ, two binocular vision subsystems are respectively along with the walking online acquisition stockpile and the stock ground image of stacker-reclaimer; Each binocular vision subsystem can be gathered two width of cloth images through two ccd video cameras wherein simultaneously in each sampling instant; In front end signal processor, preserve image sequence; And, accept simultaneously to require the front end signal processor on the stacker-reclaimer to transmit digital picture to upper server by the remote control program steering order with after this two width of cloth image process pre-service; Server can receive four width of cloth images simultaneously in each sampling instant; Comprise each two width of cloth image from first, second binocular vision subsystem; Per two width of cloth images divide left and right two width of cloth images again; Between whole sampling period, all images that collect form two groups of left and right sides views of first, second binocular vision subsystem respectively to sequence;

Step 6, Flame Image Process; The removal of images distortion; By computer picture coordinate system pixel coordinate (u; (u is v) at normalization imaginary plane image coordinate system distortional point coordinate (x v) to ask for corresponding point through normalization imaginary plane image coordinate system distortional point coordinate to the transfer equation calculating of computer picture coordinate system _d, y _d), again with (x _d, y _d) value transport in the substitution normalization imaginary plane image coordinate system pattern distortion and correct mathematical model and carry out inversion operation, therefore try to achieve the normalization imaginary plane image coordinate system ideal point coordinate (x that corrects after the distortion _u, y _u), again by ideal point coordinate (x _u, y _u) alternative (x _d, y _d) value substitution normalization imaginary plane image coordinate system in pattern distortion correct mathematical model and obtain corresponding point (u, v) new coordinate figure, promptly new pixel position; All pixel coordinates in the computer picture coordinate system (u, v) pass through the aforementioned calculation process one by one after, just can obtain the ideal alignment of pixel on picture, the recovery of promptly true scene image in other words, obtains the ideal image that a width of cloth reflects true scenery;

The searching of step 7, angle point and coupling are sought several angle points as the marginal point of representing tool characteristic in sampled images, adopt the Corner Detection Algorithm based on gradation of image, mainly reach the purpose that detects angle point through computing curvature and gradient; To in left and right view, seeking with it the point of coupling the most under the guidance of the angle point of seeking out polar curve constraint outside; Actual mechanical process need be obtained abundant three-dimensional angle point from the digital picture of left and right view;

Confirming of step 8, object point D coordinates value, the angular coordinate (u that in left and right view, matees each other _l, v _l), (u _r, v _r) can convert corresponding world coordinate (X into through the mapping relations equation that is tied to the computer picture coordinate system from world coordinates _Wl, Y _Wl, Z _Wl) and (X _Wr, Y _Wr, Z _Wr), adopt the method for asking for two straight line common vertical line intermediate values to obtain the best fit approximation of object point volume coordinate at last;

Step 9, in world coordinate system, carry out surface interpolation and set up height map; To limited the discrete pixel that image shows on full black background, must between adjacent discrete point, adopt Bei Saier surface interpolation algorithm to carry out interpolation to form one near actual continuous height map; This height map has comprised about the full detail in stock ground and has stored for use in follow-up depth analysis and data of database as the main result of image processing system;

Step 10, cubing; At first, definition " vacant lot ", the i.e. floor level of windrow not; From first to last seek vacant lot; The position judgment that occurs continuously according to vacant lot should be divided into several stockpiles with the stock ground, and gets rid of pseudo-stockpile and noise that very little fluctuating causes, obtains the position of the initial sum termination of each stockpile; Secondly, intercepting goes out a material stack height figure from the height map of stock ground; The true altitude that makes the gray-scale value representative of some pixels in the height map is h _i, it highly is h that the volume of this place's stockpile can be regarded as by limited _i, floorage is δ _iLittle rectangular parallelepiped add up and form (pixel of i for participating in calculating, i=1,2, K, n, the pixel sum of n) for participating in calculating, therefore, all how much irregular material stack volume V all can calculate according to following formula: $V=\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{\mathrm{\δ}}_i{h}_i.$

Each binocular vision subsystem in the vision measurement system of described structure stock ground comprises: two ccd video cameras, image pick-up card, front end signal processor, two photoelectric commutators and optical fiber; Left and right ccd video camera walks abreast the left and right view that is collected in sampling instant and inputs to image pick-up card; Image pick-up card converts original image data image signal to and transports to front end signal processor; Front end signal processor is removed noise and is strengthened pre-service digital picture; Through first photoelectric commutator electrical signal conversion is become light signal through pretreated data image signal, light signal is through Optical Fiber Transmission to the second photoelectric commutator; Second photoelectric commutator converts image light signals to electric signal again, and last, electric signal inputs to the image information processing module of server through the input interface of server; The left and right view simulating signal that first, second binocular vision subsystem will collect respectively separately walks abreast and transports to the image pick-up card in this subsystem; Image pick-up card converts image analoging signal data image signal to and transports to front end signal processor; Front end signal processor is removed noise and is strengthened pre-service digital picture; Through first photoelectric commutator electrical signal conversion is become light signal through pretreated data image signal; Light signal is through Optical Fiber Transmission to the second photoelectric commutator, and second photoelectric commutator converts image light signals to electric signal again, and is last; Electric signal inputs to the image information processing module of server through first input interface of server, server with data image signal handle with computing after output finally to identification, understanding and the measurement result of image information.

Described ccd video camera parameter calibration, specific as follows:

The external parameter that ccd video camera need be demarcated: comprise R and t, promptly

$\left[\begin{array}{cc}R& t\end{array}\right]=\left[\begin{array}{cccc}{r}_1& r_2& r_3& t_x\\ r_4& r_5& r_6& t_y\\ r_7& r_8& r_9& t_z\end{array}\right]$

Totally 12, but, must satisfy 6 quadrature constraints because of R is the unit orthogonal matrix, so in fact have only 6 external parameters to need to demarcate;

Inner parameter: comprise f, k ₁, s _xAnd c _x, c _y, totally 5;

At first confirm the aspect ratio of image, vertically take an annulus, calculate then its in the horizontal direction with vertical direction on pixel diameter ratio be aspect ratio s _x

Secondly take same scenery with the different focal of camera, calculate its convergent-divergent center then and obtain principal point coordinate (c _x, c _y);

Find the solution R and t through setting transition parameter at last _x, t _y, the orthogonality according to R calculates effective focal length f, coefficient of radial distortion k then ₁With translational movement t _z

Saidly in world coordinate system, carry out surface interpolation and set up height map, specific as follows:

Image is shown abundant discrete pixel on full black background; As 1400, between adjacent discrete point, adopt Bei Saier surface interpolation algorithm to carry out interpolation to form one near actual continuous height map, this height map is as the main result of image processing system; Comprised about the full detail in stock ground and stored for use in follow-up depth analysis and data of database; Height map is a plane gray level image, and the X-direction of horizontal direction and world coordinate system is consistent, and vertical direction is consistent with Z-direction; The height of the pixel grey scale of certain point (0～255) and this point (Y component) is directly proportional; Corresponding to 0 (complete black), 16 meters of peaks are as 255 (complete white) with 0 meter of minimum point, and middle height is corresponding to corresponding grey;

Each pixel is represented an area, and corresponding entire image is just being represented a specific length * wide stock yard, as: pixel is represented 0.5 * 0.5m ², the image of a 1440 * 96pixel is just represented the complete stock ground of 720m * 48m, wherein any point corresponding corresponding position height;

Digital picture is deposited the analysis for next step with the BMP form, and the information interaction that is used for data base management system (DBMS).

Said cubing, specific as follows:

1. definition: highly be lower than 0.5 meter be defined as " vacant lot ", i.e. windrow not; At first from first to last seek vacant lot, the position judgment that occurs continuously according to vacant lot should be divided into several stockpiles with the stock ground, gets rid of pseudo-stockpile and the noise that very little fluctuating causes simultaneously, obtains the position of the initial sum termination of each stockpile;

2. intercepting goes out to comprise the rectangle figure that stockpile is calculated in requirement from the height map of stock ground, and the true altitude that makes the gray-scale value representative of some pixels in the height map is h _i, floorage is δ _i, the pixel of i for participating in calculating, i=1,2, K, n, the pixel sum of n for participating in calculating, therefore, all how much irregular material stack volume V all can be according to formula $V=\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{\mathrm{\δ}}_i{h}_i$ Calculate, work as δ _i=const=0.25m ²The time, can also directly be expressed as $V=0.25\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{h}_i,$

When being " vacant lot " below the definition 0.5m, carry out the volume computing with the rectangular bottom surface zone that comprises stockpile, formula can also be expressed as: $V=d_x{d}_y[\underset{l}{\overset{m}{\mathrm{\Σ}}}\underset{k}{\overset{n}{\mathrm{\Σ}}}{h}_{\mathrm{kl}}-0.5m\×n],$ In the formula, h _KlBe k row, pixel respective heights that l is capable, k=1,2, K, n, n are the total columns of pixel, l=1,2, K, m, m are the total line number of pixel, dx, dy be respectively pixel capable to row to the pairing actual range in interval, unit is m.

Vision measuring method of the present invention adopts the method for stock ground grid height map to set up the material stack volume computing formula $V=\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{\mathrm{\δ}}_i{h}_i,$ And under the condition of definition " vacant lot ", computing further succinctly is expressed as $V=d_x{d}_y[\underset{l}{\overset{m}{\mathrm{\Σ}}}\underset{k}{\overset{n}{\mathrm{\Σ}}}{h}_{\mathrm{Kl}}-0.5m\×n]$ Realize the accurate Calculation of stockpile and stock ground volume; Automatically eliminated the pattern distortion that wide-angle lens causes; Can carry out the accurate coupling of angle point to left and right view automatically; Utilize the view data transformation relation to generate the three dimensional space coordinate of object point automatically; Optimize three-dimensional angle point and distribute, make up the world coordinate system height map, make the irregular stockpile of geometry also can realize the accurate Calculation of its volume simple and direct, intuitively.

Description of drawings

Fig. 1 is video camera geometric model figure;

Fig. 2 is outer utmost point geometric graph;

Fig. 3 is the horizontal installation diagram of left and right CCD camera in the binocular vision system;

Fig. 4 is the working state figure of left and right ccd video camera on heap, reclaimer.

Among the figure: 11 first left ccd video cameras, 12 first right ccd video cameras, 21 second left ccd video cameras, 22 second right ccd video cameras.

Embodiment

Below in conjunction with accompanying drawing embodiments of the invention are elaborated: present embodiment provided detailed embodiment and concrete operating process, but protection scope of the present invention is not limited to following embodiment being to implement under the prerequisite with technical scheme of the present invention.

Video camera geometric model as shown in Figure 1, as to adopt among the embodiment:

1. computer picture coordinate system o _UvUv: initial point o _UvBe positioned at the upper left corner on ccd image plane, u and v remarked pixel respectively are positioned at the columns and the line number of array, and unit is pixel (pixel).

2. physical image coordinate system oxy: initial point (being principal point o) is defined in the intersection point on camera optical axis and physical image plane, and x, y axle are parallel with u, v axle respectively.p _u(x _u, y _u) the P ideal image coordinate of ordering under the expression pin-hole model, p _d(x _d, y _d) be to depart from p by what the lens radial distortion caused _u(x _u, y _u) the real image coordinate.

3. camera coordinate system o _cx _cy _cz _c: initial point o _cBe defined in the photocentre of video camera, x _c, y _cAxle is parallel to x, y axle, z respectively _cAxle and optical axis coincidence.o _cO is the effective focal length f of video camera, (x _c, y _c, z _c) three-dimensional coordinate of expression object point P under camera coordinate system.

4. world coordinate system O _WX _WY _WZ _W: it is fixed to come according to concrete environment, (X _W, Y _W, Z _W) three-dimensional coordinate of expression object point P under world coordinate system.

As shown in Figure 2, described outer utmost point geometrical constraint promptly is directed against binocular vision system from two same points of angular observation, seeks the relevant geometry problem of the outer polar curve constraint between two imaging points, is called as how much of the outer utmost points.Among the figure, left and right video camera intersects to be put, and P is the same three-dimensional point of left video camera and right cameras view, O _l, O _rBe respectively the photocentre of left and right video camera; The imaging point of P point on the empty imaging plane of the normalization of left and right video camera is respectively p _l[x _ly _l1] ^TAnd p _r[x _ry _r1] ^Te _l, e _rBe respectively the outer utmost point center on the left and right image planes, for left and right two outer polar curves of common observation station P respectively through a some P _l, e _lAnd P _r, e _r

Present embodiment comprises following concrete steps:

Step 1, structure stock ground vision measurement system

Form the stock ground vision measurement system by the first binocular vision subsystem, the second binocular vision subsystem and server three parts.Each binocular vision subsystem comprises: two ccd video cameras, image pick-up card, front end signal processor, two photoelectric commutators and optical fiber.Left and right ccd video camera walks abreast the left and right view that is collected in sampling instant and inputs to image pick-up card; Image pick-up card converts original image data image signal to and transports to front end signal processor; Front end signal processor is removed pre-service such as noise and reinforcement to digital picture; Through first photoelectric commutator electrical signal conversion is become light signal through pretreated data image signal, light signal is through Optical Fiber Transmission to the second photoelectric commutator; Second photoelectric commutator converts image light signals to electric signal again, and last, electric signal inputs to the image information processing module of server through the input interface of server.

The left and right view simulating signal that first, second binocular vision subsystem will collect respectively separately walks abreast and transports to the image pick-up card in this subsystem; Image pick-up card converts image analoging signal data image signal to and transports to front end signal processor; Front end signal processor is removed pre-service such as noise and reinforcement to digital picture; Through first photoelectric commutator electrical signal conversion is become light signal through pretreated data image signal; Light signal is through Optical Fiber Transmission to the second photoelectric commutator, and second photoelectric commutator converts image light signals to electric signal again, and is last; Electric signal inputs to the image information processing module of server through first input interface of server, server with data image signal handle with computing after output finally to identification, understanding and the measurement result of image information.

Step 2, installation binocular vision subsystem

As shown in Figure 3; The first binocular vision subsystem is installed on stacker near on stockpile one side frame; The second binocular vision subsystem is installed on reclaimer near on stockpile one side frame, and the installation of two ccd video cameras in the first and second binocular vision subsystems should be followed: the photocentre of two video cameras is in same surface level principle and two optical axises and two photocentre lines and forms an isosceles triangle principle.

As shown in Figure 4, the left and right ccd video camera 11,12 in the first binocular vision subsystem is installed on the stacker, and geometric relationship between the two confirms that method is following:

1. the level interval between the first left ccd video camera 11 and the first right ccd video camera 12; Be distance between both the camera coordinate system initial point: greatly about about 20m, therefore can confirm that two camera lens level intervals are 20m according to the maximum extensible width of stacker platform.

2. camera lens vertical direction angle: according to stockpile is that 16m angle of repose high, that be about 4m, stockpile from the fore-and-aft distance of material pin as the upright position of head is that 26 °, the highest available height of stationary platform of stacker are about 6m; Camera lens can only be installed on the platform of 6m height; The coboundary line of vertical field of view is parallel with the stock ground slope; Greatly about about 2.926m, the following depression angle that therefore can budget goes out camera lens is 19 ° apart from the camera lens vertical point for the lower limb line of vertical field of view and the intersection point of stock ground bottom line.

3. camera lens horizontal direction angle: according to the universal law of stereoscopic vision; The principle that two video cameras in the binocular vision device and measured object point should be formed an isosceles triangle; According to above-mentioned level interval and camera lens vertical direction angle, camera lens horizontal direction angle is 32 ° on the stacker.

Left and right ccd video camera 21,22 in the second binocular vision subsystem is installed on the reclaimer, and geometric relationship between the two confirms that method is following:

1. the level interval between the second left ccd video camera 21 and the second right ccd video camera 22; Be distance between both the camera coordinate system initial point: according to the maximum extensible width of reclaimer platform greatly about about 10m; Receive its limit, can only confirm that two camera lens level intervals are 10m.

2. the same on camera lens vertical direction angle and the stacker.

3. camera lens horizontal direction angle: according to the universal law of stereoscopic vision; The principle that two video cameras in the binocular vision device and measured object point should be formed an isosceles triangle; According to above-mentioned level interval and camera lens vertical direction angle, get that camera lens horizontal direction angle is 17.35 ° on the reclaimer.

Step 3, set up the ccd video camera geometric model

All geometric model comprises:

1) conversion of world coordinate system and camera coordinate system, describe with rotation matrix R and translation vector t:

$\left[\begin{array}{c}{x}_c\\ y_c\\ z_c\end{array}\right]=R\left[\begin{array}{c}{X}_W\\ Y_W\\ Z_W\end{array}\right]+t;$ $R=\left[\begin{array}{ccc}{r}_1& r_2& r_3\\ r_4& r_5& r_6\\ r_7& r_8& r_9\end{array}\right],t=\left[\begin{array}{c}{t}_x\\ t_y\\ t_z\end{array}\right]$ (formula 1)

[X _WY _WZ _W] ^TBe the world coordinate system point coordinate; [x _cy _cz _c] ^TBe the camera coordinate system point coordinate.

2) desirable perspective projection transformation, promptly point coordinate is tied to the conversion of normalization imaginary plane image coordinate system from camera coordinates:

x _u=fx _c/ z _c, y _u=fy _c/ z _c(formula 2)

In the formula, f is the imaging focal length of video camera, (x _u, y _u) be normalization imaginary plane image coordinate system ideal point coordinate.

3) mathematical model is corrected in pattern distortion in the normalization imaginary plane image coordinate system, and the distortion equation when only considering the single order radial distortion is:

x _d=(1+k ₁r ²) x _u, y _d=(1+k ₁r ²) y _u(formula 3)

In the formula, $r^2=x_u^2+y_u^2;$ k ₁Be the single order coefficient of radial distortion.

4) normalization imaginary plane image coordinate system distortional point coordinate is to the conversion of computer picture coordinate system:

U=x _d/ dx+c _x, v=y _d/ dy+c _y, s _x=dy/dx (formula 4)

In the formula, (u v) is a computer picture coordinate system pixel coordinate, (c _x, c _y) for the pixel coordinate of principal point o is the principal point coordinate, (dx dy) is respectively the distance between unit picture element on x on the plane of delineation, the y direction, s _xBe aspect ratio.

Get by formula 1～4

$\left[\begin{array}{c}\mathrm{<figure-callout\; id="1"\; label="p"\; filenames="H2009100541302E00012.png"\; state="\{\{state\}\}">p</figure-callout>}\\ 1\end{array}\right]=\frac{1}{s}K\left[\begin{array}{cc}R& t\end{array}\right]\left[\begin{array}{c}P\\ 1\end{array}\right]$ (formula 5)

Formula 5 is called the mapping relations equation that world coordinates is tied to the computer picture coordinate system, wherein, and p=[u v] ^T, P=[X _WY _WZ _W] ^T, s is that spatial point is mapped to ccd video camera coordinate system (x in the world coordinate system _c, y _c, z _c) middle z _cComponent on the axle, its numerical value equal the element value in resulting the 3rd column vector of above-mentioned mapping relations equation the right result of calculation; $K=\left[\begin{array}{ccc}{f}_x& 0& c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right]$ Be the equal proportion scaled matrix of three-dimensional point coordinate from the empty imaging plane of normalization to the physics imaging plane, the ratio of its convergent-divergent (logic focal length) is relevant with real focal length f, and unit is " pixel/m ", promptly

$f_x=\frac{f(1+k_1{r}_d^2)}{\mathrm{dx}};$ $f_y=\frac{f(1+k_1{r}_d^2)}{\mathrm{dy}}$ (formula 6)

This expression though have only a physics focal length, shows to have 2 different logical focal lengths on the image.

Said normalization imaginary plane image coordinate system; Be meant: along optical axis with the camera optics imaging plane form the symmetry the plane be called imaginary plane; The coordinate system of being set up above that is called the imaginary plane image coordinate system; The image pixel characteristic quantity is carried out the normalization imaginary plane coordinate system that the expression on the imaginary plane coordinate system after the normalization is called image express, the imaginary plane coordinate system plane of living in of using normalization to express is called the normalization imaginary plane again.

Step 4, ccd video camera parameter calibration

The parameter that ccd video camera need be demarcated comprises inner parameter and external parameter.

External parameter: comprise R and t, promptly

$\left[\begin{array}{cc}R& t\end{array}\right]=\left[\begin{array}{cccc}{r}_1& r_2& r_3& t_x\\ r_4& r_5& r_6& t_y\\ r_7& r_8& r_9& t_z\end{array}\right]$

Totally 12, but, must satisfy 6 quadrature constraints because of R is the unit orthogonal matrix, so in fact have only 6 external parameters to need to demarcate.

Inner parameter: comprise f, k ₁, s _xAnd c _x, c _y, totally 5.

Calibration process adopts substep scaling method, the aspect ratio s of first uncalibrated image _xWith principal point coordinate (c _x, c _y), utilize the radial parallel constraint again, find the solution most external parameter, introduce distortion model at last again, linear solution inner parameter f, k ₁With translational movement t _z, concrete steps are following:

At first, aspect ratio confirms

Because focal length of camera is zoomed image on x, y direction simultaneously, therefore, vertically takes an annulus, calculate then its in the horizontal direction with vertical direction on pixel diameter ratio be aspect ratio s _x

Secondly, the demarcation of principal point coordinate

When the effective focal length of video camera changes, corresponding image generation convergent-divergent, and optical axis is constant, so the intersection point of the optical axis and the plane of delineation (principal point) is constant.Therefore, can use the different focal of camera to take same scenery, calculating its convergent-divergent center then is winner's point coordinate.

Suppose that focus of camera is by f ₁Become f ₂, get by formula 4:

$\frac{x_2-c_x}{x_1-c_x}=\frac{y_2-c_y}{y_1-c_y}\&DoubleRightArrow;\left[\begin{array}{cc}{y}_1-y_2& x_2-x_1\end{array}\right]{\&times;\left[\begin{array}{cc}{c}_x& c_y\end{array}\right]}^T=x_2{y}_1-x_1{y}_2$ (formula 7)

In the formula, (x ₁, y ₁) be at effective focal length f ₁The pixel coordinate of following certain unique point, (x ₂, y ₂) be that same unique point is at effective focal length f ₂Under pixel coordinate.Select a plurality of unique points, utilize least square method linearity to solve (c _x, c _y).

Be again, rotation matrix R and translational component t _x, t _yFind the solution, detailed process is following:

1. ask for transition parameter earlier

Because what we used is coplanar point, so Z _W=0, have by the radial parallel constraint

$\frac{x_d}{y_d}=\frac{x_c}{y_c}=\frac{r_1{X}_W+r_2{Y}_W+t_x}{r_4{X}_W+r_5{Y}_W+t_y};$ $x_d=\frac{\mathrm{dy}(u-c_x)}{s_x},$ y _d=dy (v-c _y) (formula 8)

Consider lens distortion to the camera effective focal length with to the influence of demarcating plan range, so same divided by t to following formula the right molecule denominator _y, and arrangement can obtain the equation of following matrix form:

$\left[\begin{array}{ccccc}{y}_d{X}_W& y_d{Y}_W& y_d& -x_d{X}_W& -x_d{Y}_W\end{array}\right]\&times;{\left[\begin{array}{ccccc}\frac{r_1}{t_y}& \frac{r_2}{t_y}& \frac{t_x}{t_y}& \frac{r_4}{t_y}& \frac{r_5}{t_y}\end{array}\right]}^T=x_d$ (formula 9)

In the formula, 5 elements in the column vector are unknown, for each calibration point; If known its world coordinates and corresponding image coordinate just can be listed an equation as above, get a plurality of calibration points (greater than 5); Ask this overdetermined equation with least square method, promptly get transition parameter:

l ₁=r ₁/ t _y, l ₂=r ₂/ t _y, l ₃=t _x/ t _y, l ₄=r ₄/ t _y, l ₅=r ₅/ t _y(formula 10)

2. find the solution R and t according to transition parameter _x, t _y

After obtaining above-mentioned transition parameter, just can obtain t according to the orthogonality of R _ySize, work as parameter l ₁, l ₂, l ₃, l ₄, l ₅Be not 0 o'clock in twos simultaneously,

$t_y^2=[A-{(A^2-4B)}^{1/2}]/2B;A={\mathrm{<figure-callout\; id="1"\; label="l"\; filenames="H2009100541302E00012.png"\; state="\{\{state\}\}">l</figure-callout>}}_1^2+l_2^2+l_4^2+l_5^2,B={(l_1{l}_5-l_2{l}_4)}^2$ (formula 11)

Otherwise t _y ²Be all the other two parametric t _xWith t _zThe inverse of quadratic sum.

Judge t _ySymbol, in utilization radial parallel when constraint, comprise two parallel vectors in the same way with two kinds of reverse situation, so t _ySymbol have two kinds maybe.Can adopt following method to judge:

Establish t earlier _yFor just, obtain r by transition parameter ₁, r ₂, r ₄, r ₅, t _x, thus calibration point is projected on the plane of delineation again, calculate the image coordinate x of corresponding point _c, y _cIf, x _dWith x _cJack per line, and y _dWith y _cJack per line, t so _yFor just, otherwise t _yFor negative, all the other external parameters of obtaining above then all make corresponding changes.

According to the orthogonality of R, calculate all the other parameters of R

$r_3=\sqrt{1-{\mathrm{<figure-callout\; id="1"\; label="r"\; filenames="H2009100541302E00012.png"\; state="\{\{state\}\}">r</figure-callout>}}_1^2-r_2^2},$ $r_6=\sqrt{1-r_4^2-r_5^2},$ r ₇＝r ₂r ₆-r ₃r ₅、

r ₈=r ₃r ₄-r ₁r ₆, r ₉=r ₁r ₅-r ₂r ₄(formula 12)

Wherein, if r ₁r ₄+ r ₂r ₅Symbol for just, then at r ₆Before add negative sign; If the f of following calculating is negative sign, then r ₃, r ₆, r ₇, r ₈Symbol and top opposite.

Effective focal length f, coefficient of radial distortion k ₁With translational movement t _zFind the solution, by formula 3:

$x_u=f\frac{x_c}{z_c}=\frac{{\mathrm{<figure-callout\; id="1"\; label="x\; d"\; filenames="H2009100541302E00012.png"\; state="\{\{state\}\}">x</figure-callout>}}_d}{1+k_1{r}_d^2}=f\frac{r_1{X}_W+r_2{Y}_W+t_x}{r_7{X}_W+r_8{Y}_W+t_z},$

$y_u=f\frac{y_c}{z_c}=\frac{{\mathrm{<figure-callout\; id="1"\; label="y\; d"\; filenames="H2009100541302E00012.png"\; state="\{\{state\}\}">y</figure-callout>}}_d}{1+k_1{r}_d^2}=f\frac{r_4{X}_W+r_5{Y}_W+t_y}{r_7{X}_W+r_8{Y}_W+t_z}$ (formula 13)

Can get after the arrangement and contain unknown number f, t as follows _z, k system of linear equations:

$\left[\begin{array}{ccc}E& {\mathrm{Er}}_d^2& -x_d\end{array}\right]\&times;{\left[\begin{array}{ccc}f& k& t_z\end{array}\right]}^T=x_{d}G,$

$\left[\begin{array}{ccc}F& {\mathrm{Fr}}_d^2& -y_d\end{array}\right]\&times;{\left[\begin{array}{ccc}f& k& t_z\end{array}\right]}^T=y_{d}G$ (formula 14)

In the formula, E=r ₁X _W+ r ₂Y _W+ t _x, F=r ₄X _W+ r ₅Y _W+ t _y, G=r ₇X _W+ r ₈Y _W, k=fk ₁, unite with least square method and to separate this system of linear equations, get final product f, t _z, the separating of k.Further judge the symbol of f, if f is for negative, then with the f opposite sign.Further solve k ₁=k/f, all the other parameter symbols are by top said making corresponding changes.

So far, demarcate, adopt linear method to solve whole inside and outside parameter of video camera, therefore avoided the loaded down with trivial details and unstable of nonlinear optimization through substep.

Step 5, IMAQ

The binocular vision subsystem adopts 120 ° of wide-angle optical lens, and two binocular vision subsystems are respectively along with the walking online acquisition stockpile and the stock ground image of stacker-reclaimer.Each binocular vision subsystem can be gathered two width of cloth images through two ccd video cameras wherein simultaneously in each sampling instant; Preserve image sequence in the signal processor of speed on stacker-reclaimer according to 1 frame/rice; And, accept simultaneously to require the front end signal processor on the stacker-reclaimer to transmit digital picture to upper server by the remote control program steering order with after this two width of cloth image process pre-service.Server can receive four width of cloth images simultaneously in each sampling instant; Comprise each two width of cloth image from first, second binocular vision subsystem; Per two width of cloth images divide left and right two width of cloth images again; Between whole sampling period, all images that collect form two groups of left and right sides views of first, second binocular vision subsystem respectively to sequence.

Communications optical cable can reach for 5 frame/seconds to the transmission speed of not having compression full width (being 768*576) color digital image at present; The gait of march of stacker-reclaimer is 1 meter per second; Therefore, default whenever can satisfy the requirement of communications optical cable to the image network transfer rate at a distance from 1 meter distance collection 1 width of cloth image fully.

Image scene is preserved hereof with the form of BMP, supplies subsequent treatment and database subsystem to use.Be stored in respectively under two files according to left camera lens, right camera lens,, represent the left and right image folder of stacker respectively like df, dr etc.; Filename is address date, and unit is a rice, like 201.bmp, 202.bmp etc., representes the image that 201,202 meters collect respectively.Image resolution ratio is consistent with resolution of video camera, is 768*576 to the video camera of pal mode.

Step 6, Flame Image Process

The removal of images distortion, (u, (u is v) at normalization imaginary plane image coordinate system distortional point coordinate (x v) to ask for corresponding point through formula 4 calculating by computer picture coordinate system pixel coordinate _d, y _d), again with (x _d, y _d) value transport to substitution formula 3 and carry out inversion operation, therefore try to achieve the normalization imaginary plane image coordinate system ideal point coordinate (x that corrects after the distortion _u, y _u), again by ideal point coordinate (x _u, y _u) alternative (x _d, y _d) value substitution formula 4 obtain corresponding point (u, v) new coordinate figure, promptly new pixel position.

All pixel coordinates in the computer picture coordinate system (u, v) pass through the aforementioned calculation process one by one after, just can obtain the ideal alignment of pixel on picture, the recovery of promptly true scene image in other words, obtains the ideal image that a width of cloth reflects true scenery.

The searching of step 7, angle point and coupling

In sampled images, seek several angle points as the marginal point of representing tool characteristic, adopt Corner Detection Algorithm, mainly reach the purpose that detects angle point through computing curvature and gradient based on gradation of image.Specifically be embodied as: confirm a local window in the image,, confirm the grey scale change value of image through window moving in image.Three kinds of possibilities are arranged in the process that moves: if window is in the place of uniform gray level, then result of calculation is a less value; If window has been crossed over the edge, be a less value in the result who moves along edge line so, and will be a bigger grey scale change in place perpendicular to the edge; If window be in angle point or independently on, on each direction of this point, all be the bigger place of grey scale change so, angle point can be detected rapidly.

To in left and right view, seeking with it the point of coupling the most under the guidance of the angle point of seeking out polar curve constraint outside.Adopt " outer polar curve " constraint whether the subpoint of obtaining spatial point on two width of cloth images is mated and judge, be i.e. the subpoint P of spatial point P on image _l, P _rMust be in the space any 1 P and the O of the centre of perspectivity, the left and right sides _l, O _rConstitute the intersection O of a plane and two imaging planes _lP, O _rP is last.

For the CCD that intersection is put, O _l, O _rLine intersects at e with left and right view image planes respectively _l, e _r, promptly left.Outer utmost point center on the right image planes; Left and right two outer polar curves are respectively through a P _l, e _lAnd P _r, e _rCan know that by the stereoscopic vision theory any a pair of corresponding point must be outside they are corresponding on the polar curve on the left and right view image planes.

Make left and right camera coordinate system be respectively (x _l, y _l, z _l) and (x _r, y _r, z _r), the transformation relation of left and right camera coordinate system is:

$\left[\begin{array}{c}{x}_l\\ y_l\\ z_l\end{array}\right]=\left[\begin{array}{ccc}{r}_{\mathrm{rl}1}& r_{\mathrm{rl}2}& r_{\mathrm{rl}3}\\ r_{\mathrm{rl}4}& r_{\mathrm{rl}5}& r_{\mathrm{rl}6}\\ r_{\mathrm{rl}7}& r_{\mathrm{rl}8}& r_{\mathrm{rl}9}\end{array}\right]\left[\begin{array}{c}{x}_r\\ y_r\\ z_r\end{array}\right]+\left[\begin{array}{c}{t}_{\mathrm{rlx}}\\ t_{\mathrm{rly}}\\ t_{\mathrm{rlz}}\end{array}\right];\left[\begin{array}{c}{x}_r\\ y_r\\ z_r\end{array}\right]=\left[\begin{array}{ccc}{r}_{\mathrm{lr}1}& r_{\mathrm{lr}2}& r_{\mathrm{lr}3}\\ r_{\mathrm{lr}4}& r_{\mathrm{lr}5}& r_{\mathrm{lr}6}\\ r_{\mathrm{lr}7}& r_{\mathrm{lr}8}& r_{\mathrm{lr}9}\end{array}\right]\left[\begin{array}{c}{x}_l\\ y_l\\ z_l\end{array}\right]+\left[\begin{array}{c}{t}_{\mathrm{lrx}}\\ t_{\mathrm{lry}}\\ t_{\mathrm{lrz}}\end{array}\right]$ (formula 15)

After converting left and right camera coordinate system into the world coordinate system expression formula separately, simultaneous equations can be asked for all parameter values in the formula 15.

The left side video camera O of the centre of perspectivity _lSubpoint in right image planes is utmost point center outside the right side

The in like manner right video camera O of the centre of perspectivity _rSubpoint in left image planes is utmost point center outside the left side

Any 1 P of left side image planes _l ^*Outer polar curve P in right image planes _r ^*e _rOn, any 1 P of in like manner right image planes _r ^*Outer polar curve P in left image planes _l ^*e _lOn, therefore outer polar curves all on the left and right image planes can find the solution outer polar curve equation according to left and right cameras coordinate corresponding relation all through left and right outer utmost point center.If outer polar curve slope is k on the left image planes _l, and cross e _lThe point; Outer polar curve slope on right image planes is k _r, and cross e _rPoint is derived k _lTo k _rTransformation relation and k _rTo k _lTransformation relation:

$k_r=-\frac{t_{\mathrm{lrz}}(r_{\mathrm{lr}4}+r_{\mathrm{lr}5}{k}_l)-t_{\mathrm{lry}}(r_{\mathrm{lr}7}+r_{\mathrm{lr}8}{k}_l)}{t_{\mathrm{lrx}}(r_{\mathrm{lr}7}+r_{\mathrm{lr}8}{k}_l)-t_{\mathrm{lrz}}({\mathrm{<figure-callout\; id="1"\; label="r\; lr"\; filenames="H2009100541302E00012.png"\; state="\{\{state\}\}">r</figure-callout>}}_{\mathrm{lr}}+r_{\mathrm{lr}2}{k}_l)};$

$k_l=-\frac{t_{\mathrm{rlz}}(r_{\mathrm{rl}4}+r_{\mathrm{rl}5}{k}_r)-t_{\mathrm{rly}}(r_{\mathrm{rl}7}+r_{\mathrm{rl}8}{k}_r)}{t_{\mathrm{rlx}}(r_{\mathrm{rl}7}+r_{\mathrm{rl}8}{k}_r)-t_{\mathrm{rlz}}({\mathrm{<figure-callout\; id="1"\; label="r\; rl"\; filenames="H2009100541302E00012.png"\; state="\{\{state\}\}">r</figure-callout>}}_{\mathrm{rl}}+r_{\mathrm{rl}2}{k}_r)}$ (formula 16)

For the curve after finding the solution, outer utmost point characteristic capable of using is sought corresponding point along curve.Suppose that the left view curve image is a reference picture, then right view is an image to be matched, at first obtains along any point P of left view label curve _l ^*With utmost point center e outside the left side _lThe slope k of line _l, ask P again _l ^*The slope k of polar curve outside right view _r, by slope k _rWith polar curve center e outside the right side _rObtain outer polar curve.Utilize the fuzzy similarity method to realize the accurate coupling of angle point in left and right two views again, Fig. 8 is illustrated in the effect of seeking angle point in left and right two views and realizing coupling, and red point is a match point on the correct line among the figure, and yellow point is wrong global registration point.

Actual mechanical process need be obtained abundant three-dimensional angle point from the digital picture of left and right view, as: in 700 meters long stock grounds, build together and found 1400 three-dimensional angle points (each point of every meter stacker and reclaimer).The coordinate of these three-dimensional angle points is based on unified stock ground world coordinate system, obtains the spatial coordinate point through aforesaid method, and proofreaies and correct according to the material machine translational coordination that the stroke position encoder provides, and standard is to unified stock ground world coordinate system.The X of stock ground world coordinate system _WThe axle perpendicular to ground, Y _WAxle is parallel to ground, Z _WThe axle perpendicular to track, the lower left corner of coordinate origin planimetric map in the stock ground.

Confirming of the 8th step, object point D coordinates value

With the angular coordinate (u that matees each other in the left and right view _l, v _l), (u _r, v _r) convert corresponding world coordinate (X into _Wl, Y _Wl, Z _Wl) and (X _Wr, Y _Wr, Z _Wr).

Because the slight error of Effects of Noise and calibrating parameters when video camera imaging model and nonideal pinhole imaging system and CCD imaging; The capital makes being connected straight line and may not intersecting at the space a bit of object point conversion coordinate and the centre of perspectivity in the left and right view; And these two straight lines can not be parallel, so these two straight lines certainly will be different surface beelines.In other words, generally speaking, (X _Wl, Y _Wl, Z _Wl) and (X _Wr, Y _Wr, Z _Wr) can not overlap fully, therefore need be employed in the best fit approximation that the method for asking for intermediate value on the common vertical line of above-mentioned two straight lines obtains the object point volume coordinate at last.

On how much, the error of the mid point approximate spatial point of use different surface beeline common vertical line is less.In concrete the calculating, at first calculate the bee-line of two different surface beelines, i.e. the length of common vertical line.If the length of common vertical line is shorter, we just suppose that the mid point of common vertical line is a point to be asked (when the length of common vertical line was 0, two different surface beelines were intersecting straight lines, and intersection point is point to be asked); If common vertical line is oversize, can conclude that so the match point in the left and right view that is obtained is too big at the coupling time error, this point should be given up.

The 9th goes on foot, in world coordinate system, carries out surface interpolation and set up height map

To 1400 discrete pixels that image shows on full black background, must between adjacent discrete point, adopt Bei Saier surface interpolation algorithm to carry out interpolation to form one near actual continuous height map.Height map has comprised about the full detail in stock ground and has stored for use in follow-up depth analysis and data of database as the main result of image processing system.Height map is a plane gray level image; The X-direction of horizontal direction and world coordinate system is consistent; Vertical direction is consistent with Z-direction, and the height of the pixel grey scale of certain point (0～255) and this point (Y component) is directly proportional, with 0 meter of minimum point corresponding to 0 (complete black); 16 meters of peaks are as 255 (complete white), and middle height is corresponding to corresponding grey.

Each pixel is represented 0.5 * 0.5m ², this is the image of a 1440 * 96pixel, can express the complete stock ground of 720m * 48m, wherein any point corresponding corresponding position height.

Digital picture is deposited the analysis for next step with the BMP form, and the information interaction that is used for data base management system (DBMS).

The tenth step, cubing

1. definition: highly be lower than 0.5 meter be defined as " vacant lot ", i.e. windrow not.At first from first to last seek vacant lot, the position judgment that occurs continuously according to vacant lot should be divided into several stockpiles with the stock ground, gets rid of pseudo-stockpile and the noise that very little fluctuating causes simultaneously, obtains the position of the initial sum termination of each stockpile.

2. intercepting goes out a material stack height figure from the height map of stock ground.

Theoretically, this is the computing of a cube integration, and is rather complicated, but utilizes data in the height map to calculate the volume of stockpile, just seems very simple and direct, directly perceived.

The true altitude that makes the gray-scale value representative of some pixels in the height map is h _i, floorage is δ _i, the pixel of i for participating in calculating, i=1,2, K, n, the pixel sum of n for participating in calculating, therefore, all how much irregular material stack volume V all can calculate according to following formula:

$V=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{\mathrm{\&delta;}}_i{h}_i$ (formula 17)

Work as δ _i=const=0.25m ²The time, formula 17 can directly be expressed as

$V=0.25\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{h}_i$ (formula 18)

When being " vacant lot " below the definition 0.5m, carry out the volume computing with the rectangular bottom surface zone that comprises stockpile, formula can also be expressed as:

$V=d_x{d}_y[\underset{l}{\overset{m}{\mathrm{\&Sigma;}}}\underset{k}{\overset{n}{\mathrm{\&Sigma;}}}{h}_{\mathrm{kl}}-0.5m\&times;n]$ (formula 19)

In the formula, h _KlBe k row, pixel respective heights that l is capable, k=1,2, K, n, n are the total columns of pixel, l=1,2, K, m, m are the total line number of pixel, dx, dy be respectively pixel capable to row to the pairing actual range in interval, unit is m.Adopt formula 19 to carry out volume calculation; As long as the rectangle of an intercepting figure has comprised the stockpile that requires calculating in height map; Can go to take into account the regularity of distribution of this boundary line, stockpile bottom surface; Therefore can simplify many computation process, save CPU operation time, very convenient, simple and direct to computer programming.

Claims (6)

1. one kind large-scale stock ground stockpile vision measuring method is characterized in that, may further comprise the steps:

Step 1, structure stock ground vision measurement system; Form by the first binocular vision subsystem, the second binocular vision subsystem and server three parts, the image information processing module of server receive handle behind the signal of the first binocular vision subsystem, the second binocular vision subsystem and computing after output finally to identification, understanding and the measurement result of image information;

Step 2, installation binocular vision subsystem; The first binocular vision subsystem is installed on stacker near on stockpile one side frame; The second binocular vision subsystem is installed on reclaimer near on stockpile one side frame, in the first and second binocular vision subsystems separately the installation of two ccd video cameras should follow: the photocentre of two ccd video cameras is in same surface level and two optical axises and two photocentre lines and forms an isosceles triangle;

Step 3, set up the ccd video camera geometric model; Comprise: the conversion to the computer picture coordinate system of mathematical model, normalization imaginary plane image coordinate system distortional point coordinate is corrected in pattern distortion in the conversion of world coordinate system and camera coordinate system, desirable perspective projection transformation, the normalization imaginary plane image coordinate system, obtains to be tied to from world coordinates the mapping relations equation of computer picture coordinate system at last;

Step 4, ccd video camera parameter calibration; Comprise inner parameter and external parameter, adopt the substep scaling method, the aspect ratio of first uncalibrated image and principal point coordinate; Utilize the radial parallel constraint again; Find the solution most external parameter, introduce distortion model, linear solution inner parameter and translational movement at last again;

Step 5, IMAQ, two binocular vision subsystems are respectively along with the walking online acquisition stockpile and the stock ground image of stacker-reclaimer; Each binocular vision subsystem can be gathered two width of cloth images through two ccd video cameras wherein simultaneously in each sampling instant; In front end signal processor, preserve image sequence; And, accept simultaneously to require the front end signal processor on the stacker-reclaimer to transmit digital picture to upper server by the remote control program steering order with after this two width of cloth image process pre-service; Server can receive four width of cloth images simultaneously in each sampling instant; Comprise each two width of cloth image from first, second binocular vision subsystem; Per two width of cloth images are left and right two width of cloth images; Between whole sampling period, all images that collect form two groups of left and right sides views of first, second binocular vision subsystem respectively to sequence;

Step 6, Flame Image Process; The removal of images distortion; By computer picture coordinate system pixel coordinate (u; (u is v) at normalization imaginary plane image coordinate system distortional point coordinate (x v) to ask for corresponding point through normalization imaginary plane image coordinate system distortional point coordinate to the transfer equation calculating of computer picture coordinate system _d, y _d), again with (x _d, y _d) value transport in the substitution normalization imaginary plane image coordinate system pattern distortion and correct mathematical model and carry out inversion operation, therefore try to achieve the normalization imaginary plane image coordinate system ideal point coordinate (x that corrects after the distortion _u, y _u), again by ideal point coordinate (x _u, y _u) alternative (x _d, y _d) value substitution normalization imaginary plane image coordinate system in pattern distortion correct mathematical model and obtain corresponding point (u, v) new coordinate figure, promptly new pixel position; All pixel coordinates in the computer picture coordinate system (u, v) pass through the aforementioned calculation process one by one after, just can obtain the ideal alignment of pixel on picture, the recovery of promptly true scene image in other words, obtains the ideal image that a width of cloth reflects true scenery;

The searching of step 7, angle point and coupling are sought several angle points as the marginal point of representing tool characteristic in sampled images, adopt the Corner Detection Algorithm based on gradation of image, mainly reach the purpose that detects angle point through computing curvature and gradient; To in left and right view, seeking with it the point of coupling the most under the guidance of the angle point of seeking out polar curve constraint outside; Actual mechanical process need be obtained abundant three-dimensional angle point from the digital picture of left and right view;

Confirming of step 8, object point D coordinates value, the angular coordinate (u that in left and right view, matees each other _l, v _l), (u _r, v _r) can convert corresponding world coordinate (X into through the mapping relations equation that is tied to the computer picture coordinate system from world coordinates _Wl, Y _Wl, Z _Wl) and (X _Wr, Y _Wr, Z _Wr), adopt and ask for spatial point (X _Wl, Y _Wl, Z _Wl), (X _Wr, Y _Wr, Z _Wr) respectively with its separately the method for the common vertical line intermediate value of two straight lines being connected of the video camera centre of perspectivity obtain the best fit approximation of object point volume coordinate at last;

Step 9, in world coordinate system, carry out surface interpolation and set up height map; To limited the discrete pixel that image shows on full black background, must between adjacent discrete point, adopt Bei Saier surface interpolation algorithm to carry out interpolation to form one near actual continuous height map; This height map has comprised about the full detail in stock ground and has stored for use in follow-up depth analysis and data of database as the main result of image processing system;

Step 10, cubing; At first, definition " vacant lot ", the i.e. floor level of windrow not; From first to last seek vacant lot; The position judgment that occurs continuously according to vacant lot should be divided into several stockpiles with the stock ground, and gets rid of pseudo-stockpile and noise that very little fluctuating causes, obtains the position of the initial sum termination of each stockpile; Secondly, intercepting goes out a material stack height figure from the height map of stock ground; The true altitude that makes the gray-scale value representative of some pixels in the height map is h _i, it highly is h that the volume of this place's stockpile can be regarded as by limited _i, floorage is δ _iLittle rectangular parallelepiped add up and form, the pixel of i for participating in calculating, i=1,2 ..., n, the pixel sum of n for participating in calculating, therefore, all how much irregular material stack volume V all can calculate according to following formula:

2. large-scale stock ground according to claim 1 stockpile vision measuring method; It is characterized in that; Each binocular vision subsystem in the vision measurement system of described structure stock ground comprises: two ccd video cameras, image pick-up card, front end signal processor, two photoelectric commutators and optical fiber; Left and right ccd video camera walks abreast the left and right view that is collected in sampling instant and inputs to image pick-up card; Image pick-up card converts original image data image signal to and transports to front end signal processor; Front end signal processor is removed noise and is strengthened pre-service digital picture; Through first photoelectric commutator electrical signal conversion is become light signal through pretreated data image signal, light signal is through Optical Fiber Transmission to the second photoelectric commutator; Second photoelectric commutator converts image light signals to electric signal again, and last, electric signal inputs to the image information processing module of server through the input interface of server; The left and right view simulating signal that first, second binocular vision subsystem will collect respectively separately walks abreast and transports to the image pick-up card in this subsystem; Image pick-up card converts image analoging signal data image signal to and transports to front end signal processor; Front end signal processor is removed noise and is strengthened pre-service digital picture; Through first photoelectric commutator electrical signal conversion is become light signal through pretreated data image signal; Light signal is through Optical Fiber Transmission to the second photoelectric commutator, and second photoelectric commutator converts image light signals to electric signal again, and is last; Electric signal inputs to the image information processing module of server through first input interface of server, server with data image signal handle with computing after output finally to identification, understanding and the measurement result of image information.

3. large-scale stock ground according to claim 1 stockpile vision measuring method is characterized in that, described ccd video camera parameter calibration is specific as follows:

The external parameter that ccd video camera need be demarcated: comprise R and t, promptly

$\left[\begin{array}{cc}R& t\end{array}\right]=\left[\begin{array}{cccc}{r}_1& r_2& r_3& t_x\\ r_4& r_5& r_6& t_y\\ r_7& r_8& r_9& t_z\end{array}\right]$

Totally 12, but, must satisfy 6 quadrature constraints because of R is the unit orthogonal matrix, so in fact have only 6 external parameters to need to demarcate;

Inner parameter: comprise f, k ₁, s _xAnd c _x, c _y, totally 5;

At first confirm the aspect ratio of image, vertically take an annulus, calculate then its in the horizontal direction with vertical direction on pixel diameter ratio be aspect ratio s _x

Secondly take same scenery with the different focal of camera, calculate its convergent-divergent center then and obtain principal point coordinate (c _x, c _y);

Find the solution R and t through setting transition parameter at last _x, t _y, R is a rotation matrix, $R=\left[\begin{array}{ccc}{r}_1& r_2& r_3\\ r_4& r_5& r_6\\ r_7& r_8& r_9\end{array}\right],$ t _x, t _yBe translation vector $t=\left[\begin{array}{c}{t}_x\\ t_y\\ t_z\end{array}\right]$ Element, the orthogonality according to R calculates effective focal length f, coefficient of radial distortion k then ₁With translational movement t _z

4. according to the described large-scale stock ground of any claim stockpile vision measuring method in the claim 1～3, it is characterized in that, saidly in world coordinate system, carry out surface interpolation and set up height map, specific as follows:

Image is shown abundant discrete pixel on full black background; Between adjacent discrete point, adopt Bei Saier surface interpolation algorithm to carry out interpolation to form one near actual continuous height map; This height map has comprised about the full detail in stock ground and has stored for use in follow-up depth analysis and data of database as the main result of image processing system, and height map is a plane gray level image; The X-direction of horizontal direction and world coordinate system is consistent; Vertical direction is consistent with Z-direction, and the height of the pixel grey scale of certain point and this point is that the Y component is directly proportional, and 0 meter of minimum point is black entirely corresponding to 0; 16 meters of peaks are complete white as 255, and middle height is corresponding to corresponding grey;

Each pixel is represented an area, and corresponding entire image is just being represented a specific length * wide stock yard, wherein any point corresponding corresponding position height;

Digital picture is deposited the analysis for next step with the BMP form, and the information interaction that is used for data base management system (DBMS).

5. large-scale stock ground according to claim 4 stockpile vision measuring method is characterized in that said each pixel is represented 0.5 * 0.5m ², the image of a 1440 * 96pixel is just represented the complete stock ground of 720m * 48m.

6. according to the described large-scale stock ground of any claim stockpile vision measuring method in the claim 1～3, it is characterized in that, said cubing, specific as follows:

1. definition: highly be lower than 0.5 meter be defined as " vacant lot ", i.e. windrow not; At first from first to last seek vacant lot, the position judgment that occurs continuously according to vacant lot should be divided into several stockpiles with the stock ground, gets rid of pseudo-stockpile and the noise that very little fluctuating causes simultaneously, obtains the position of the initial sum termination of each stockpile;

2. intercepting goes out to comprise the rectangle figure that stockpile is calculated in requirement from the height map of stock ground, and the true altitude that makes the gray-scale value representative of some pixels in the height map is h _i, floorage is δ _i, the pixel of i for participating in calculating, i=1,2 ..., n, the pixel sum of n for participating in calculating, therefore, all how much irregular material stack volume V all can be according to formula

Calculate, work as δ _i=const=0.25m ²The time, can also directly be expressed as $V=0.25\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{h}_i,$

When being " vacant lot " below the definition 0.5m, carry out the volume computing with the rectangular bottom surface zone that comprises stockpile, formula can also be expressed as:

In the formula, h _KlBe k row, pixel respective heights that l is capable, k=1,2 ..., n, n are the total columns of pixel, l=1,2 ..., m, m are the total line number of pixel, dx, dy be respectively pixel capable to row to the pairing actual range in interval, unit is m.

Images