CN102060234B - Tire crane traveling track video correction device and method - Google Patents
Tire crane traveling track video correction device and method Download PDFInfo
- Publication number
- CN102060234B CN102060234B CN 201010519880 CN201010519880A CN102060234B CN 102060234 B CN102060234 B CN 102060234B CN 201010519880 CN201010519880 CN 201010519880 CN 201010519880 A CN201010519880 A CN 201010519880A CN 102060234 B CN102060234 B CN 102060234B
- Authority
- CN
- China
- Prior art keywords
- dsp
- camera
- crane
- video processing
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000012937 correction Methods 0.000 title claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 55
- 230000000007 visual effect Effects 0.000 claims abstract description 29
- 230000008569 process Effects 0.000 claims description 16
- 230000008676 import Effects 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 238000011426 transformation method Methods 0.000 claims description 5
- 239000004744 fabric Substances 0.000 claims description 4
- 238000013461 design Methods 0.000 abstract description 13
- 239000000758 substrate Substances 0.000 description 17
- 238000004891 communication Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 239000011159 matrix material Substances 0.000 description 8
- 238000010276 construction Methods 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000012467 final product Substances 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000010339 medical test Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000013456 study Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Landscapes
- Closed-Circuit Television Systems (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the technical field of realizing real-time processing of a video image by using an embedded system and applying a result to control, in particular to a tire crane traveling track video correction device and a tire crane traveling track video correction method. The device comprises a power supply and a computer, wherein two digital signal processor (DSP) dual-channel video processing subsystems are connected with a singlechip and the computer respectively; one DSP dual-channel video processing subsystem is connected with a front camera and a rear camera on one side of a tire crane, and the other DSP dual-channel video processing subsystem is connected with a front camera and a rear camera on the other side of the tire crane; and the singlechip is connected with a programmable logic controller (PLC) control system. By using the embedded system and a modularized design, the system has a compact structure, high extensibility, low cost, high real-time property and high robustness, so that automatic correction of a tire crane traveling track based on visual feedback can be realized. By popularizing the device and the method, throughput efficiency of cargoes at a dock/harbor can be improved, working strength of a tire crane driver is relieved, potential safety hazard is reduced and working efficiency is improved.
Description
Technical field
The present invention relates to utilize the real-time processing of Implementation of Embedded System video image and the technical field that the result is used to control, track video deviation correcting device and method are advanced by especially a kind of tyre crane garage.
Background technology
The development of international trade makes the cargo handling capacity of port and pier increase day by day, and at present a lot of port and piers all are equipped with the tire crane to carry out the lifting of goods and move.Tyre crane cart requires to control its deviation angle in the process of moving about 5 ° (positive and negative), and this safe driving technical requirement to tire crane driver is higher, promptly will avoid collision constantly.Accurately the equipment of angle measurement has electronic compass and GPS etc. at present; Because the crane working environment has heavy-duty motor etc., exist high-intensity magnetic field to disturb, the declination error that electronic compass is measured is bigger; And GPS precision commonly used is not enough, therefore all can not satisfy this specific application environment of port and pier.Present tyre crane wheel paths correction is main to rely on the driver to pay close attention to the trajectory of having finished on the ground constantly; Driving technology through superb is controlled, and working strength is big and dull, is easy to cause driver fatigue; Have bigger potential safety hazard, and work efficiency is not high.
Along with the development of computer vision, lot of domestic and international scholar has carried out correlative studys such as Flame Image Process, target detection, has emerged a large amount of achievements, and image processing techniques is in medical test, diagnosis, and the industrial monitoring field has also obtained using widely.The system that much controls based on visual feedback has appearred at present; But to this application-specific background of tyre crane wheel paths correction; Do not occur solution as yet, still can't solve the tire crane at present and carry out this difficult problem of track automatic deviation correction in the process based on visual feedback.
Summary of the invention
Existingly not occur deficiency as yet in order overcoming, to the invention provides a kind of tyre crane garage and advance track video deviation correcting device and method based on the solution of visual feedback to this application-specific background of tyre crane wheel paths correction.
The technical solution adopted for the present invention to solve the technical problems is: a kind of tyre crane garage advances track video deviation correcting device and comprises power supply and computing machine; Two DSP binary channels Video processing subsystems connect single-chip microcomputer and computing machine respectively; A DSP binary channels Video processing subsystem connects former and later two cameras of tire crane one side; Another piece DSP binary channels Video processing subsystem connects former and later two cameras of tire crane opposite side, and single-chip microcomputer is connected with the PLC control system.
According to another embodiment of the invention, comprise that further said DSP binary channels Video processing subsystem adopts the modular construction of core board+substrate.
According to another embodiment of the invention, the modular construction that further comprises said core board+substrate comprises core board, substrate, watchdog module, state indicating module and power module; Said core board comprises TMS320DM642 chip, Nor Flash storage chip and two SDRAM chips; Said substrate mainly comprises power supply chip, serial ports expansion chip, two video decoding chips and a video coding chip, and said state indicating module is made up of 2 LED.
According to another embodiment of the invention, comprise that further said DSP binary channels Video processing subsystem has two serial ports and a JTAG mouth, communicates with single-chip microcomputer and computing machine respectively through two serial ports, through a JTAG mouth and compunication.
According to another embodiment of the invention, comprise that further said single-chip microcomputer has four serial ports and a JTAG mouth, four serial ports are set to RS485, RS485, RS232 and RS232 respectively.
Track video method for correcting error advances in a kind of tyre crane garage, comprises the steps:
(1) image of gathering is carried out DSP binary channels Video processing: a DSP binary channels Video processing subsystem connects former and later two cameras in tire crane left side; The front and back two-path video is gathered and handled to VP1 and VP2 mouth through DSP simultaneously; Obtain the angle that crane under former and later two visual angles departs from set travel track; Another piece DSP binary channels Video processing subsystem connects former and later two cameras on tire crane right side; Two-path video before and after VP1 through DSP and VP2 mouth are gathered simultaneously and handled obtains the angle that crane under former and later two visual angles departs from set travel track, and the result of two-path video before and after all can alternately showing through the VP0 mouth; Through interrupt priority level being set and gathering time-delay and wait for, can realize two-way alternately gather with show synchronously;
(2) camera calibration scheme: the image that DSP binary channels Video processing subsystem is gathered imports computing machine through JTAG; Utilize the projective transformation method to calculate the calibrating parameters of camera, and utilize the calibrating parameters of the camera that calculates that the parameter in the DSP program is carried out initialization; Generate executable file after the DSP program compilation, through JTAG or serial ports with program burn writing in DSP binary channels Video processing subsystem;
(3) the track automatic deviation correction in the process advances in realization tyre crane garage: single-chip microcomputer merges four detected drift angles of camera; Drift angle after will merging is then issued the PLC control system through serial ports the tire crane is controlled, and realizes that steadily tyre crane garage advances the track automatic deviation correction in the process.
According to another embodiment of the invention, comprise that further the camera calibration scheme may further comprise the steps in the said step (2):
(1) camera is installed, and the true origin of definite earth coordinates, regulate the corresponding angle of pitch, its visual field can be observed apart from the zone of its floor projections the place ahead 2～7m;
(2) in the camera area of visual field, placing at least 4 and demarcate thing, is reference with the earth coordinates initial point, and the ground coordinate position of thing demarcated in record;
(3) in the demarcation scene image that DSP gathers, select a wherein width of cloth, it imported in the computing machine, and in image, selects the demarcation thing to check its location of pixels through mouse, and with its ground coordinate paired record that is mapped;
(4), calculate the calibrating parameters of camera through projective transformation according to the ground coordinate position of the demarcation thing that obtains and the coordinate position in the image;
(5) utilize calibrating parameters that the straight line that detects in the image is carried out coordinate transform, obtain its position corresponding under earth axes, and then estimate the straight line that detects in the present image and the drift angle between the set track.
The invention has the beneficial effects as follows, adopt embedded system and modular design, make system architecture compactness, favorable expandability, cost is low and have good real time performance and robustness, thereby can realize advancing the automatic deviation correction of track based on the tyre crane garage of visual feedback.Should invent through promoting, can improve the throughput efficiency of port and pier goods, alleviate tire crane driver's working strength, reduce potential safety hazard, increase work efficiency.
Description of drawings
Below in conjunction with accompanying drawing and embodiment the present invention is further specified.
Fig. 1 is the theory diagram of tire crane video deviation-rectifying system of the present invention;
Fig. 2 is the theory diagram of DSP binary channels Video processing subsystem;
Fig. 3 is the perspective transform synoptic diagram under the Z=0 situation;
Fig. 4 is the software algorithm process flow diagram that track video deviation-rectifying system advances in tyre crane garage.
Embodiment
Like Fig. 1 is the theory diagram of tire crane video deviation-rectifying system of the present invention; A kind of tyre crane garage advances track video deviation correcting device and comprises power supply and computing machine; Two DSP binary channels Video processing subsystems connect single-chip microcomputer and computing machine respectively; A DSP binary channels Video processing subsystem connects former and later two cameras of tire crane one side, and another piece DSP binary channels Video processing subsystem connects former and later two cameras of tire crane opposite side, and single-chip microcomputer is connected with the PLC control system.
Said DSP binary channels Video processing subsystem adopts the modular construction of core board+substrate.
The modular construction of said core board+substrate comprises core board, substrate, watchdog module, state indicating module and power module; Said core board comprises TMS320DM642 chip, Nor Flash storage chip and two SDRAM chips; Said substrate mainly comprises power supply chip, serial ports expansion chip, two video decoding chips and a video coding chip, and said state indicating module is made up of 2 LED.
Said DSP binary channels Video processing subsystem has two serial ports and a JTAG mouth, communicates with single-chip microcomputer and computing machine respectively through two serial ports, through a JTAG mouth and compunication.
Said single-chip microcomputer has four serial ports and a JTAG mouth, and four serial ports are set to RS485, RS485, RS232 and RS232 respectively.
Track video method for correcting error advances in a kind of tyre crane garage, comprises the steps:
(1) image of gathering is carried out DSP binary channels Video processing: a DSP binary channels Video processing subsystem connects former and later two cameras in tire crane left side; The front and back two-path video is gathered and handled to VP1 and VP2 mouth through DSP simultaneously; Obtain the angle that crane under former and later two visual angles departs from set travel track; Another piece DSP binary channels Video processing subsystem connects former and later two cameras on tire crane right side; Two-path video before and after VP1 through DSP and VP2 mouth are gathered simultaneously and handled obtains the angle that crane under former and later two visual angles departs from set travel track, and the result of two-path video before and after all can alternately showing through the VP0 mouth; Through interrupt priority level being set and gathering time-delay and wait for, can realize two-way alternately gather with show synchronously;
(2) camera calibration scheme: the image that DSP binary channels Video processing subsystem is gathered imports computing machine through JTAG; Utilize the projective transformation method to calculate the calibrating parameters of camera, and utilize the calibrating parameters of the camera that calculates that the parameter in the DSP program is carried out initialization; Generate executable file after the DSP program compilation, through JTAG or serial ports with program burn writing in DSP binary channels Video processing subsystem;
(3) the track automatic deviation correction in the process advances in realization tyre crane garage: single-chip microcomputer merges four detected drift angles of camera; Drift angle after will merging is then issued the PLC control system through serial ports the tire crane is controlled, and realizes that steadily tyre crane garage advances the track automatic deviation correction in the process.
The camera calibration scheme may further comprise the steps in the said step (2):
(1) camera is installed, and the true origin of definite earth coordinates, regulate the corresponding angle of pitch, its visual field can be observed apart from the zone of its floor projections the place ahead 2～7m;
(2) in the camera area of visual field, placing at least 4 and demarcate thing, is reference with the earth coordinates initial point, and the ground coordinate position of thing demarcated in record;
(3) in the demarcation scene image that DSP gathers, select a wherein width of cloth, it imported in the computing machine, and in image, selects the demarcation thing to check its location of pixels through mouse, and with its ground coordinate paired record that is mapped;
(4), calculate the calibrating parameters of camera through projective transformation according to the ground coordinate position of the demarcation thing that obtains and the coordinate position in the image;
(5) utilize calibrating parameters that the straight line that detects in the image is carried out coordinate transform, obtain its position corresponding under earth axes, and then estimate the straight line that detects in the present image and the drift angle between the set track.
The objective of the invention is to carry out this problem of track automatic deviation correction to the tire crane; Propose and realize that a kind of tyre crane garage based on visual feedback advances track video correction scheme; Adopt the Embedded Real-Time treatment technology, handle four road videos that obtain under the different visual angles simultaneously, obtain the drift angle of departing from set track in the crane traveling process fast, accurately, steadily; And send it to PLC, realize that finally tyre crane garage advances the automatic deviation correction of track.
For realizing above-mentioned purpose, technical solution of the present invention is that design one cover tire crane carries out track video deviation-rectifying system, and system design specifically comprises following aspect:
(1) DSP binary channels Video processing subsystem: adopt the modular construction of core board+substrate to design DSP Video processing subsystem; This system have minimize, advantages such as low-cost, good versatility and extendability; Can handle the video of two-way camera collection simultaneously, and can be through the result of VP0 passage Alternation Display two-path video;
(2) two DSP Video processing subsystems add chip microcontroller four tunnel Video processing and information fusion: two cameras of the each side installation of tire crane; Link to each other with a DSP Video processing subsystem respectively; Two DSP process result are sent into single-chip microcomputer through serial ports, simplify the communication between the DSP with this.Single-chip microcomputer merges the track drift angle that four cameras that are installed under the different visual angles detect; Strengthen the robustness that the drift angle is detected; Drift angle after will merging is then issued PLC through serial ports crane is controlled, and realizes the track automatic deviation correction in the crane traveling process;
(3) flexible design of many expansion interfaces: every DSP processing system for video has two serial ports (one of them configurable RS485 of being) and can communicate with single-chip microcomputer and computing machine; Also have in addition a JTAG mouth can with compunication, make things convenient for system debug and function to expand; Single-chip microcomputer has four serial ports, and wherein two are configured to RS485, and two are configured to RS232;
(4) camera calibration scheme: to the certain applications background, the image that DSP is gathered imports on the computing machine through JTAG, utilizes the projective transformation method to calculate the calibrating parameters of camera, and through JTAG or serial ports with program burn writing in DSP; In addition, the camera of same model and fixed focal length can be selected for use,, the work of camera parameter calibration can be simplified through the fixedly angle of pitch and antenna height.
Said DSP binary channels Video processing subsystem is the real-time processing that utilizes the Implementation of Embedded System video, and its design comprises following aspect:
A, DSP core board: core board only comprises TMS320DM642 chip, Nor Flash storage chip and two main chip such as SDRAM chip, does not comprise expansion interface, and Fig. 2 has provided the schematic diagram of DSP binary channels Video processing subsystem.IO that dsp chip is all and peripheral functionality are drawn through two row's stitch, through core board is inserted on the substrate, can make full use of the function of DSP, and this modular design can be accelerated development progress, reduces cost of development, improves the extendability of system;
B, expansion substrate: mainly comprise power supply chip, serial ports expansion chip, two video decoding chips (TVP5150) and a video coding chip (SAA7121H) on the substrate; In order to guarantee the operate as normal of video module; Vision signal is prevented the surge protection, adopt three to be the anti-surge module of SJ3-1/2 model video lightning protection that technological company limited produces; Core board is inserted on the substrate,, can realizes the difference in functionality demand neatly, minimize the ground cutting system through changing the substrate design;
C, watchdog module and state indicating module: watchdog module is used for whether operate as normal of surveillance, if work undesired then restart system.The state indicating module mainly is made up of 2 LED, and whether one of them is used to indicate power supply normal, and whether another one is used for indication mechanism in proper working order.Simultaneously, in the state indicating module, reserve a few road IO, be used for system extension;
D, power module: the power supply chip on the substrate is that DSP core board and substrate are supplied power simultaneously, wherein adopts the switching power source chip TPS5450 of TI, input voltage range 5.5V～36V; Conversion efficiency is up to more than 90%, and output voltage is 5V, output current 5A; Power supply for DSP core board and substrate; Dsp processor IO and peripheral SDRAM, chips such as Nor flash adopt the power supply of 3.3V power supply, and the DSP kernel adopts the power supply of 1.4V power supply.
Be serious problems such as prevent that system short-circuit from burning, the power supply input should be connected in series one and can recover fuse, comes type selecting according to system's rated disspation parameters such as (electric currents).Because of the connecting system power supply is the 24V direct current,, also can select the rectifier bridge mode for use in anti-reverse system of burning of commutation diode of the anodal serial connection of circuit; No matter but how all the cathode and anode directions that direct supply inserts operate as normal, selecting commutation diode here is to consider system power dissipation, and required electric current is bigger during operate as normal; The about 1A of theoretical peak; Because long-term forward conduction, the big heat radiation of PN junction area is also better, and the ability of the high temperature that conforms is better; And the commutation diode price is not high, so select the commutation diode of 5A to be serially connected in the cathode power supply line.Because work under bad environment exists heavy-duty motor and hi-line, circuit self inductance is easy to cause surge, and the power supply protection needs especially to consider.Consider the characteristics of surge current, peak point current big (energy is bigger), rising and fall time are slower; And pulse train peak hour weak point, pulse width, PV is high; Take all factors into consideration, the power supply chip input end adopts the method for voltage dependent resistor (VDR) and the protection of TVS pipe jointing, and voltage dependent resistor (VDR) has safeguard function preferably to surge; The first order that is positioned at input protection is comparatively suitable, and short characteristics paired pulses crowd can play and clamp down on mild effect preferably the TVS response time, was positioned at the second level of protection.
Said two DSP processing system for video add chip microcontroller four tunnel Video processing and information fusion, and its design comprises following aspect:
A, binary channels real-time video are handled: a DSP Video processing subsystem connects former and later two cameras in tire crane left side; The front and back two-path video is gathered and handled to VP1 and VP2 mouth through DSP simultaneously; Obtain the angle
that crane under former and later two visual angles departs from set travel track; Another piece DSP Video processing subsystem connects former and later two cameras on tire crane right side; The front and back two-path video is gathered and handled to VP1 and VP2 mouth through DSP simultaneously; Obtain the angle
that crane under former and later two visual angles departs from set travel track; Shown in the theory diagram of Fig. 1; And all can replace the result that shows the front and back two-path video through the VP0 mouth; Through interrupt priority level being set and gathering time-delay and wait for, can realize two-way alternately gather with show synchronously;
B, single-chip machine information merge: for realize between two DSP processing system for video synchronously; And the robustness of enhanced system; The modularization of simplifying the communication between the DSP and realizing DSP Video processing subsystem; Drift angle
and
of the tire crane left and right sides travel track that two DSP Video processing subsystems are obtained under the visual angle, front and back delivers in the single-chip microcomputer through serial ports and merges according to the weighted mean criterion; As shown in Figure 1, obtain the drift angle of higher, more accurate drift angle of confidence level as ultimate demand;
C, based on the automatic control of visual feedback: single-chip microcomputer with each fusion drift angle that obtains constantly according to certain data format; Adopt the Modbus protocol mode to send to PLC through serial ports; The automatic control that realization is advanced direction to tyre crane garage, the automatic deviation correction of realization travel track.
The design of expansion interface comprises:
A, every DSP Video processing subsystem expansion two serial ports (one of them is configurable to be RS485) and a JTAG mouth.Because TMS320DM642 self does not have serial ports, so adopt the serial port chip expansion scheme.The serial ports expansion chip is selected TL16C752B, expansion two-way serial ports, and level transferring chip adopts MAX3232 and SN75176 (or SN75LBC184) to carry out the conversion of TTL-RS232, TTL-RS485 level respectively, can realize RS232 or RS485 communication simultaneously; The JTAG mouth can with compunication, convenient debugging and camera calibration are simplified software and the algorithm of DSP, help the modularization of DSP Video processing subsystem;
B, single-chip microcomputer have four serial ports, and wherein two are configured to RS485, and two are configured to RS232, and a JTAG mouth is arranged, and are convenient to debugging, when environment electromagnetics disturbs by force, when communication distance is far away, can adopt RS485 to transmit;
C, state are indicated and resetted: DSP core board and single-chip microcomputer all expand system reset switch, and expand a plurality of pilot lamp interfaces, are used to indicate power supply or duty;
D, communication protocol design: utilize serial ports can make things convenient for and communicate with single-chip microcomputer and computing machine; When the design of communications agreement; Should make the DSP video processing module interface (RS232 port) can to carry out communication one to one direct-connected with PLC (model is HE-XE102) port; The present invention adopts industrial control field to use Modbus agreement comparatively widely, mainly comprise between DSP and the single-chip microcomputer communicate by letter and single-chip microcomputer and PLC between communicate by letter.Communications parameter comprises: normally whether angle is represented, data communication parity checking, dsp system work indication, danger warning indication.
Said camera calibration scheme may further comprise the steps:
A, camera installed set up; And the true origin of definite earth coordinates; The projection on the ground of the desirable camera of true origin center; Through regulating the angle of pitch of camera, its visual field can be observed apart from the zone of its floor projections the place ahead 2-7m, guarantee in image, to detect the travel track of crane;
B, in the camera area of visual field, placing several little, brightly painted demarcation things (being no less than 4) by certain intervals, is reference with the earth coordinates initial point, and the ground coordinate of thing demarcated in record;
C, in the demarcation scene image that DSP gathers, select a wherein width of cloth; It is imported in the notebook computer; Open the software that utilizes the VC++6.0 establishment; Import and demarcate scene image, in image, select to demarcate thing and check its location of pixels, correspondence is demarcated the actual ground coordinate input of thing through mouse;
D, according to obtaining coordinate position in the ground coordinate position of demarcating thing and the image, calculate the calibrating parameters of camera through projective transformation.To the specific application environment of straight-line detection on the ground level, traditional projective transformation method is simplified.The formula of projective transformation is following:
（1）
Or representing matrix launches form
Here
is the world coordinates of a spatial point;
is the coordinate of this spot projection at the plane of delineation, is unit with the pixel.
is known as video camera matrix, perhaps intrinsic parameter matrix.
is reference point (common center at image), and
is to be the focal length of unit with the pixel.If so because some factor rises sampling or falls sampling the piece image that comes from video camera; All these parameters (
,
and
) all will be by the same yardstick of convergent-divergent (take advantage of or remove).The intrinsic parameter matrix does not rely on the view of scene, in case calculate, can be repeated to use (as long as fixed focal length).Rotation-translation matrix
is known as outer parameter matrix; It is used for describing the motion of camera with respect to a fixed scene; Perhaps opposite, object around camera rigid motion.Just
arrives certain coordinate system with the coordinate transform of point
, and this coordinate system is changeless with respect to video camera.The present invention adopts a kind of perspective transform method of simplification, and its principle is as shown in Figure 3.Formula (2) can be reduced to
（3）
Wherein,
is matrix of coefficients; Purpose is that feasible
following data are 1; Constitute homogeneous equation,
is rotation matrix.
and
is respectively the horizontal ordinate of space calibration point corresponding point in image and the vector that ordinate constitutes, and
and
is respectively the vector of space calibration point in the formation of coordinate in length and breadth of ground level.Here suppose that vertically the direction on plane is the Z axle.
Utilize formula (3) can calculate the calibrating parameters of camera, and can bring whether observe the camera calibration parameter that estimates accurate through the projection contravariant.
5) utilize calibrating parameters that the straight line that detects in the image is carried out coordinate transform, obtain its position corresponding under earth axes, and then estimate the straight line that detects in the present image and the drift angle between the set travel track.
Embodiment
It is as shown in Figure 2 that the theory diagram of track video deviation-rectifying system advances in tyre crane garage; This system is made up of four cameras, two DSP binary channels Video processing subsystems, single-chip microcomputer and connector, need be equipped with notebook computer and DSP emulator during initial Installation and Debugging.Wherein camera is noted parameters such as waterproof, temperature range, visual angle and resolution when selecting; DSP adopts a chip TMS320DM642 that is applicable to Flame Image Process of TI exploitation; Video decoding chip adopts TVP5150, and coding chip adopts SAA7121H, and single-chip microcomputer adopts the MSP430F5438 chip; The serial ports expansion chip of DSP and single-chip microcomputer adopts TL16C752B; Expansion two-way serial ports, level transferring chip adopts MAX3232 and SN75176 (or SN75LBC184) to carry out TTL-RS232, TTL-RS485 level conversion respectively, can realize RS232 or RS485 communication simultaneously.
System software partly relates to dsp program, and communicating by letter between DSP and single-chip microcomputer and the computing machine communicated by letter the camera calibration scheme between single-chip microcomputer and the PLC.The real-time video Processing Algorithm adopts dsp program to realize that camera calibration software is that instrument is developed with Visual C++6.0, and the image that DSP capable of using gathers carries out camera calibration, and calculates calibrating parameters.
Below in conjunction with Fig. 4 the practical implementation process of this invention in real work is described, its step is following:
A, system host connected the power supply of 24VDC; The video line that connects four cameras from four middle BNC of main frame rear panel; And the camera power lead is connected on three air plugs in the right, two BNC of the rightmost side are respectively the VP1 mouth of two DSP, are used to show the result of gathering video;
B, the JTAG mouth is connected DSP emulator and notebook computer, power-on switch then, the image that DSP is gathered is saved in the notebook computer through the JTAG mouth;
C, in notebook computer, open camera calibration software, import the image of gathering by DSP, carry out camera calibration, and utilize the camera calibration parameter that calculates that the parameter in the DSP program is carried out initialization;
Form executable file after d, the DSP program compilation, and pass through the programming of JTAG mouth to DSP;
E, emulator is taken off from the DSP main frame, re-power and reset, DSP gets final product operate as normal, realizes twin-channel real-time processing;
The image of f, collection will pass through cutting, fall sampling, Sobel rim detection and image segmentation etc., utilize the Hough conversion to detect the straight line corresponding parameters then;
G, utilize the camera calibration parameter with the straight line parameter transformation that detects in the image under earth coordinates, estimate the angle that the straight line that detects in the image departs from set track, and reject the drift angle that some disturb straight lines and correspondence thereof through cluster and filtering;
H, each camera is obtained the drift angle that image detects issue single-chip microcomputer through serial ports; Reject wrong angle through judging; Carry out weighted mean then, and the drift angle after will merging sends to PLC through serial ports, realize tyre crane garage is advanced the automatic deviation correction of track.
We test the detection that foregoing invention is applied to bicycle lane border white line on the highway, promptly utilize the trolley that sets up camera to advance along white line, provide the drift angle that the dolly center line departs from set track in the traveling process.In addition; On the tire crane of Shenzhen port and pier, also carried out the lots of field test; Test findings shows; Track video correction scheme is advanced by the tyre crane garage based on visual feedback that we propose can solve travel track automatic deviation correction problem well, has very high reliability, practicality and generalization, and market outlook are wide.
Claims (2)
1. track video method for correcting error advances in a tyre crane garage; Comprise that tyre crane garage advances track video deviation correcting device; This device comprises power supply and computing machine, and two DSP binary channels Video processing subsystems connect single-chip microcomputer and computing machine respectively, and single-chip microcomputer is connected with the PLC control system; It is characterized in that, comprise the steps:
(1) image of gathering is carried out DSP binary channels Video processing: a DSP binary channels Video processing subsystem connects former and later two cameras in tire crane left side; The front and back two-path video is gathered and handled to VP1 and VP2 mouth through DSP simultaneously; Obtain the angle that crane under former and later two visual angles departs from set travel track; Another piece DSP binary channels Video processing subsystem connects former and later two cameras on tire crane right side; Two-path video before and after VP1 through DSP and VP2 mouth are gathered simultaneously and handled obtains the angle that crane under former and later two visual angles departs from set travel track, and the result of two-path video before and after all can alternately showing through the VP0 mouth; Through interrupt priority level being set and gathering time-delay and wait for, can realize two-way alternately gather with show synchronously;
(2) camera calibration scheme: the image that DSP binary channels Video processing subsystem is gathered imports computing machine through the JTAG mouth; Utilize the projective transformation method to calculate the calibrating parameters of camera, and utilize the calibrating parameters of the camera that calculates that the parameter in the DSP program is carried out initialization; Generate executable file after the DSP program compilation, through JTAG mouth or serial ports with program burn writing in DSP binary channels Video processing subsystem;
(3) the track automatic deviation correction in the process advances in realization tyre crane garage: single-chip microcomputer merges four detected drift angles of camera; Drift angle after will merging is then issued the PLC control system through serial ports the tire crane is controlled, and realizes that steadily tyre crane garage advances the track automatic deviation correction in the process.
2. track video method for correcting error advances in tyre crane according to claim 1 garage, it is characterized in that, the camera calibration scheme may further comprise the steps in the said step (2):
(1) camera is installed, and the true origin of definite earth coordinates, regulate the corresponding angle of pitch, its visual field can be observed apart from the zone of its floor projections the place ahead 2～7m;
(2) in the camera area of visual field, placing at least 4 and demarcate thing, is reference with the earth coordinates initial point, and the ground coordinate position of thing demarcated in record;
(3) in the demarcation scene image that DSP gathers, select a wherein width of cloth, it imported in the computing machine, and in image, selects the demarcation thing to check its location of pixels through mouse, and with its ground coordinate paired record that is mapped;
(4), calculate the calibrating parameters of camera through projective transformation according to the ground coordinate position of the demarcation thing that obtains and the coordinate position in the image;
(5) utilize calibrating parameters that the straight line that detects in the image is carried out coordinate transform, obtain its position corresponding under earth axes, and then estimate the straight line that detects in the present image and the drift angle between the set track.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010519880 CN102060234B (en) | 2010-10-26 | 2010-10-26 | Tire crane traveling track video correction device and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010519880 CN102060234B (en) | 2010-10-26 | 2010-10-26 | Tire crane traveling track video correction device and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102060234A CN102060234A (en) | 2011-05-18 |
| CN102060234B true CN102060234B (en) | 2012-12-26 |
Family
ID=43995770
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010519880 Active CN102060234B (en) | 2010-10-26 | 2010-10-26 | Tire crane traveling track video correction device and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102060234B (en) |
Families Citing this family (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103601077A (en) * | 2013-09-14 | 2014-02-26 | 韦文新 | Automatic framing system for van-type tailgate |
| CN106774339A (en) * | 2017-01-18 | 2017-05-31 | 上海振华重工电气有限公司 | Tyre crane based on image procossing crosses the street system and method automatically |
| CN106892356B (en) * | 2017-02-09 | 2018-02-27 | 江苏大学 | A kind of tyre crane running gear automatic correction method based on machine vision |
| CN107344696B (en) * | 2017-08-25 | 2018-11-02 | 上海沪东集装箱码头有限公司 | Tire crane anti-collision based on real time video image identification hits early warning system and its method |
| CN107943020B (en) * | 2017-10-17 | 2021-07-23 | 上海辛格林纳新时达电机有限公司 | Automatic deviation rectifying method for tire crane cart |
| CN109993788B (en) * | 2017-12-29 | 2021-10-26 | 西门子（中国）有限公司 | Deviation rectifying method, device and system for tyre crane |
| CN110155157A (en) * | 2018-03-09 | 2019-08-23 | 驭达工业装备（苏州）有限公司 | A kind of non-Mecanum wheel high-precision omnidirectional vehicle |
| CN110870301B (en) * | 2018-08-30 | 2021-04-02 | 深圳市大疆创新科技有限公司 | Image display method and device and image processing equipment |
| CN109816588B (en) * | 2018-12-29 | 2023-03-21 | 百度在线网络技术（北京）有限公司 | Method, device and equipment for recording driving trajectory |
| CN110054089B (en) * | 2019-04-29 | 2020-06-09 | 北京航天自动控制研究所 | Automatic vision deviation rectifying system and method for tire crane |
| CN110287884B (en) * | 2019-06-26 | 2021-06-22 | 长安大学 | Voltage line detection method in auxiliary driving |
| CN110963409B (en) * | 2019-11-29 | 2021-02-09 | 北京航天自动控制研究所 | Method for measuring automatic deviation rectification deviation of tire crane machine vision |
| CN112666586A (en) * | 2021-02-01 | 2021-04-16 | 诸葛驾到(深圳)科技有限公司 | Vehicle-mounted high-precision positioning terminal |
| CN114253290B (en) * | 2021-12-15 | 2024-03-19 | 成都飞机工业（集团）有限责任公司 | Method and system for automatic tracking and accurate positioning of airplane component transport vehicle |
| CN117094988B (en) * | 2023-10-16 | 2024-02-02 | 成都电科星拓科技有限公司 | ResNet50 network model-based piezoresistor surface microscopic defect detection method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1394190A (en) * | 2000-10-27 | 2003-01-29 | 三菱重工业株式会社 | Container position measuring method and device for cargo crane and container landing/stacking method |
| CN101256399A (en) * | 2007-12-27 | 2008-09-03 | 清华大学 | Double DSP built-in control system for visual feedback during soldering process |
| CN101393010A (en) * | 2008-08-11 | 2009-03-25 | 张育民 | Embedded vision detecting system |
| CN201229497Y (en) * | 2008-07-08 | 2009-04-29 | 高旭 | Container tyre crane automatic driving system based on image processing |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0820489A (en) * | 1994-07-06 | 1996-01-23 | Mitsubishi Heavy Ind Ltd | Position deflection detection device for traveling body |
-
2010
- 2010-10-26 CN CN 201010519880 patent/CN102060234B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1394190A (en) * | 2000-10-27 | 2003-01-29 | 三菱重工业株式会社 | Container position measuring method and device for cargo crane and container landing/stacking method |
| CN101256399A (en) * | 2007-12-27 | 2008-09-03 | 清华大学 | Double DSP built-in control system for visual feedback during soldering process |
| CN201229497Y (en) * | 2008-07-08 | 2009-04-29 | 高旭 | Container tyre crane automatic driving system based on image processing |
| CN101393010A (en) * | 2008-08-11 | 2009-03-25 | 张育民 | Embedded vision detecting system |
Non-Patent Citations (1)
| Title |
|---|
| JP特開平8-20489A 1996.01.23 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102060234A (en) | 2011-05-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102060234B (en) | Tire crane traveling track video correction device and method | |
| CN201569917U (en) | Intelligent tracking drift dolly | |
| CN203241826U (en) | Mobile robot positioning system based on hybrid navigation ribbon | |
| CN105487536A (en) | Low-cost autonomous obstacle avoidance method for mobile robot | |
| CN100578413C (en) | Solar light three-dimensional tracing sensor | |
| CN107962589B (en) | Insulating protection system of electrified water washing robot of substation equipment based on multisensor fuses | |
| CN201789447U (en) | Position corrector for solar cells | |
| CN107067762A (en) | Intelligent traffic light | |
| CN102837311A (en) | Five-axis medium-low speed full-automatic dispensing robot servo controller | |
| CN103324207B (en) | A kind of cluster solar energy tracking control system | |
| CN205022902U (en) | Agricultural unmanned aerial vehicle keeps away barrier device based on IR evading obstacle sensors | |
| CN203037261U (en) | Image stabilizing system for small-sized gyroscope | |
| CN205139705U (en) | Wireless small -size intelligent vehicle based on monocular vision | |
| CN103472838A (en) | Fast sprint controller of four-wheel micro-mouse based on double processors | |
| CN110888446A (en) | Agricultural machinery autopilot system | |
| CN105955316A (en) | Double-shaft solar video sun tracking system and method | |
| CN206781742U (en) | A kind of intelligent vehicle that can be tracked with avoidance | |
| CN108563230A (en) | A kind of path planning mobile robot based on laser radar | |
| CN210270157U (en) | Ultrasonic radar fusion sensing system for intelligent electric sweeper | |
| CN204346449U (en) | Navigator is independently detected in a kind of omnidirectional ranging indoor | |
| CN203133615U (en) | Two-degree of freedom point light source tracking system | |
| CN202342879U (en) | AVR SCM (Single-Chip Microcomputer)-based direction control device for model car | |
| CN106406151A (en) | Control circuit and algorithm for intelligent intertillage fertilization and pesticide machine | |
| CN113759787A (en) | Unmanned robot for closed park and working method | |
| CN203216514U (en) | Photoelectric rotary platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C53 | Correction of patent for invention or patent application | ||
| CB03 | Change of inventor or designer information |
Inventor after: Zhou Dongxiang Inventor after: Liu Yunhui Inventor after: Yang Yanguang Inventor after: Liu Shun Inventor after: Fan Caizhi Inventor after: Liang Hua Inventor after: Cai Xuanping Inventor before: Liu Yunhui Inventor before: Yang Yanguang Inventor before: Fan Caizhi Inventor before: Zhou Dongxiang Inventor before: Liang Hua Inventor before: Cai Xuanping |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: INVENTOR; FROM: LIU YUNHUI YANG YANGUANG FAN CAIZHI ZHOU DONGXIANG LIANG HUA CAI XUANPING TO: ZHOU DONGXIANG LIU YUNHUI YANG YANGUANG LIU SHUN FAN CAIZHI LIANG HUA CAI XUANPING |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C53 | Correction of patent for invention or patent application | ||
| CB03 | Change of inventor or designer information |
Inventor after: Zhou Dongxiang Inventor after: Yang Yanguang Inventor after: Liu Shun Inventor after: Fan Caizhi Inventor after: Liang Hua Inventor after: Cai Xuanping Inventor before: Zhou Dongxiang Inventor before: Liu Yunhui Inventor before: Yang Yanguang Inventor before: Liu Shun Inventor before: Fan Caizhi Inventor before: Liang Hua Inventor before: Cai Xuanping |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: INVENTOR; FROM: ZHOU DONGXIANG LIU YUNHUI YANG YANGUANG LIU SHUN FAN CAIZHI LIANG HUA CAI XUANPING TO: ZHOU DONGXIANG YANG YANGUANG LIU SHUN FAN CAIZHI LIANG HUA CAI XUANPING |
|
| CB03 | Change of inventor or designer information |
Inventor after: Zhou Dongxiang Inventor after: Liu Yunhui Inventor after: Yang Yanguang Inventor after: Liu Shun Inventor after: Fan Caizhi Inventor after: Liang Hua Inventor after: Cai Xuanping Inventor before: Zhou Dongxiang Inventor before: Yang Yanguang Inventor before: Liu Shun Inventor before: Fan Caizhi Inventor before: Liang Hua Inventor before: Cai Xuanping |
|
| CB03 | Change of inventor or designer information | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20170306 Address after: Nanshan District Guangdong streets Shenzhen Yuexing Guangdong province 518000 city two Road No. 10 Chinese University Hong Kong Shenzhen Research Institute of building 3 room 326 Patentee after: Future robot (Shenzhen) Co., Ltd. Address before: 213000 Jiangsu Province, the Clock Tower District, ERON Road, No. 213 High-tech Venture Center, room 1008 Patentee before: Changzhou Hyper Medium and Sensing Technology Institute Co., Ltd. |