Specific embodiment
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to
When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment
Described in embodiment do not represent all embodiments consistented with the present invention.On the contrary, they be only with it is such as appended
The example of device and method being described in detail in claims, some aspects of the invention are consistent.
It is only to be not intended to limit the invention merely for for the purpose of describing particular embodiments in terminology used in the present invention.
It is also intended in the present invention and the "an" of singular used in the attached claims, " described " and "the" including majority
Form, unless the context clearly indicates other meaning.It is also understood that term "and/or" used herein refers to and wraps
It may be combined containing one or more associated any or all of project listed.
It will be appreciated that though various information, but this may be described using term first, second, third, etc. in the present invention
A little information should not necessarily be limited by these terms.These terms are only used to for same type of information being distinguished from each other out.For example, not departing from
In the case where the scope of the invention, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as
One information.Depending on context, word as used in this " if " can be construed to " ... when " or " when ...
When " or " in response to determination ".
Barcode scanning payment is a kind of means of payment very universal at present, can complete to prop up using mobile terminals such as mobile phones
It pays, without using cash, without small change, consumer need to only show payment code to businessman, scan payment code by barcode scanning equipment
It can complete to pay, greatly simplifie process of exchange.Since existing barcode scanning device location is fixed, scanning range
Also limited, thus special occasion some for parking lot, charge station etc., it there is also some problems using barcode scanning payment, for example,
In charge station or Parking, the position of barcode scanning equipment and driver are each different, the barcode scanning range of barcode scanning equipment and away from
It is not visible from for a user, user do not know the two dimensional code on this mobile phone prevent wherein could barcode scanning success.In addition,
Due to the distance of charge equipment and driver may farther out, the operating distance that need barcode scanning equipment to support also farther out, usual charge station
Barcode scanning equipment barcode scanning distance need to meet 0.2m~1m, it is and also larger to the need of barcode scanning range, thus swept apart from excessive will lead to
Code visual angle is less than normal, usually 30 °~40 ° or so, causes the contradiction that barcode scanning range and barcode scanning distance can not be taken into account.In addition,
The vehicle of parking lot or charge station has cart also to have trolley, since the height of trolley and cart differs larger, such as trolley vehicle
For window height in 1m or so, the vehicle window height of truck has been more than 2.5m, and single barcode scanning equipment cannot be considered in terms of, and often to use more
Barcode scanning equipment, and it is higher suitable for the professional barcode scanning equipment price of high speed/parking scene at present, cause the cost of whole system big
It is big to improve.
The barcode scanning model of barcode scanning equipment and separate unit barcode scanning equipment can not be accurately aimed in order to solve user during barcode scanning
Smaller the problem of not being able to satisfy user demand is enclosed, present description provides a kind of scan code system, the scan code system can be used for respectively
Kind barcode scanning scene, it should be pointed out that the scan code system of this specification is not limited to for scanning the items such as one-dimension code, two dimensional code
Code, it can also be used to scanning be similar to bar code it is this include the specific informations such as payment information image.Scan code system includes two
Video acquisition device, a position for the determining target object comprising target barcode information, one for according to object
Barcode scanning again after the position of body is rotated to suitable barcode scanning position, in this way, user is without oneself being accurately directed at scan code system,
Scan code system can voluntarily adjust the video acquisition device for barcode scanning to suitable barcode scanning position according to the position of target object
Barcode scanning again afterwards.Specifically, the scan code system is as shown in Figure 1a, including processing unit 11, the first view being connected with processing unit
Frequency acquisition device 12 and the second video acquisition device 13, in certain embodiments, the first video acquisition device and second view
Frequency acquisition device can be connected by MIPI interface, DVP interface, LVDS interface or USB interface with processing unit.First video is adopted
Acquisition means 12 are used to acquire the video image of target object, and the target object is used for displaying target bar code information, can be use
The mobile terminals such as mobile phone, the PAD at family are also possible to the picture comprising bar code information.The acquisition of first video acquisition device 12
To after the video image of target object, it is sent to processing unit 11, processing unit 11 is determined according to the video image of target object
Then target object controls the second video acquisition device 13 according to the location information of target object in the location information of three-dimensional space
Turning to target position, wherein target position is the position that the second video acquisition device can be clearly captured target barcode,
It target position can be with sets itself.In one embodiment, target barcode can be located to the bat of the second video acquisition device 13
Take the photograph regional center when the second video acquisition device described in position be set as target position, it is clear to collect in this way
Complete target barcode.After second video acquisition device turns to target position, the video image of target barcode is acquired, is sent to
Processing unit 11 after processing unit receives the video image of target barcode, identifies the target barcode in image and exports barcode scanning knot
Fruit.
Since user is when showing two dimensional code, due to the position difference of user and the not visible property of barcode scanning range,
Cause two dimensional code that may not be placed on suitable barcode scanning range, for example, user in charge station since vehicle parking position is different,
When user being caused to show payment code to scan code system with mobile phone, it may be not aligned with two dimensional code, lead to not barcode scanning success.This
Specification goes video image of the acquisition comprising the target object of target barcode information to determine mesh by the first video acquisition device
Object is marked in the location information of three-dimensional space.In order to scan target object as large-scale as possible, the first video acquisition
The camera that device can select visual angle as big as possible, such as, wide-angle camera can be selected in certain embodiments, it is certain
The horizontal view angle of wide-angle camera up to 90 °~150 °, and can be adjusted by different disposing ways horizontal direction and
The coverage of vertical direction, thus the angular field of view that can be scanned is larger.Target detection or motion detection can be passed through
Video image of the mode to detect the acquisition of the first video acquisition device in target object, such as target object can be set
It is mobile phone, processing unit can extract feature from the video image that the first video acquisition device acquires, pass through the feature of extraction
When determining in video image comprising mobile phone, just thinks that the object is target object, then determine the location information of target object.
The second video acquisition device in this specification be it is rotary, the second video acquisition device major function is to adopt
Collect the image of target barcode, so that processing unit identifies target barcode.It is filled in processing unit according to the first video acquisition
Set acquisition target object video image determine the position of target object after, processing unit can control the second video acquisition device and turn
Suitable position is moved, so that the second video acquisition device can collect the image of clear target barcode.In some embodiments
In, as shown in Figure 1 b, barcode scanning device further includes a rotary holder 14, and the second image collecting device 13 is to be mounted on cloud
A focal length camera on platform, wherein holder is connected with processing unit, can be Pan/tilt in the control of processing unit
Rotation, and drive the focal length camera rotation being mounted on holder.And focal length camera is imaged greatly since its focal length is long, is suitble to
Farther away object, thus the image suitable for acquisition target barcode are shot, and since the imaging of the target barcode of shooting is big,
High resolution also facilitates processing unit and identifies to it, and certainly, in certain embodiments, the camera can also be adopted
Lesser bar code amplification at a distance will be shot, be obtained larger by the zoom function of zoom camera with zoom camera
Bar code image, improve barcode scanning success rate.The selection of camera can not be limited according to verse demand flexible choice, this specification
System.
After the video image for collecting target object by the first video acquisition device 12, processing unit 11 can be according to mesh
The video image for marking object carries out detection and localization to target object, to determine the three-dimensional space position of target object.In certain realities
It applies in example, the three-dimensional spatial position information of target object can be determined using method as shown in Figure 2, specifically includes following step
It is rapid:
S201,2D position coordinates of the target object in video image are determined;
S202, the target object is determined according to the intrinsic parameter and the 2D position coordinates of first video acquisition device
In the location information of three-dimensional space.
Before carrying out barcode scanning using scan code system, can to the first video acquisition device and the second video acquisition device and
The two relative positional relationship is demarcated, wherein calibration is to determine the object of a three-dimensional space in world coordinate system
Coordinate and the 2D position coordinates of its video image after being imaged mapping relations.In general, carrying out calibration master to camera
If determining the internal reference of camera, outer ginseng, wherein internal reference is parameter relevant to camera self-characteristic, such as the coke of camera
Away from, principle point location and distortion factor etc., outer ginseng generally comprises spin matrix and translation vector.Outer ginseng is taken the photograph for two commonly used in description
As the relative positional relationship between head.Wherein, the distortion factor number in internal reference is the radially and tangentially offset of camera lens.
In certain embodiments, demarcating to the parameter of the first video acquisition device and the second video acquisition device can be with
Using Zhang Zhengyou plane reference method.Multiple the video figures acquired by the first video acquisition device and the second video acquisition device
Picture, by Zhang Zhengyou plane reference method can be calculated the first video acquisition device and the second video acquisition device internal reference,
Outer ginseng.
Since the first video acquisition device can be wide-angle camera, the pattern distortion of wide-angle camera shooting is larger, because
And after receiving the video image of target object of the first video acquisition device acquisition, first demarcated in advance can be used
Distortion factor in the calibrating parameters of video acquisition device handles the video image of target object, wherein to object
It includes carrying out distortion correction to the video image of target object that the video image of body, which carries out processing, due to the first video acquisition device
It can be monocular cam, be also possible to binocular camera, if it is binocular camera, then the video image of target object carries out
Processing includes the video image progress binocular polar curve alignment correction for further including target object.The video image of target object is carried out
After processing, 2D position coordinates of the target object in video image can be determined by algorithm of target detection, by calculating target
The depth information of object obtains target object in three dimensions in conjunction with calibrating parameters and the image 2D coordinate of target object
Physical location coordinate information.
In certain embodiments, determine target object in video image according to the video image of treated target object
2D position coordinates can using be based on sorting algorithms or the base such as Haar feature or HOG feature combination SVM or AdaBoost
It is determined in the Fast R-CNN algorithm of deep learning.It is to compare based on sorting algorithms such as Haar feature combination SVM or AdaBoost
A kind of method of traditional target detection, the principle of this method assume that target object detection when have a child window to
Constantly displacement sliding, the every position of child window will calculate the feature in the region in the video image window of detection,
Then this feature can be screened with SVM classifier, once this feature has passed through the screening of classifier, then determines the region
For target object.But this method effect is poor, and efficiency is lower.The algorithm of target detection more commonly used at present is based on deep
The Fast R-CNN algorithm of study is spent, Fast-RCNN is the optimization of R-CNN.R-CNN(Region with CNN feature)
It is that convolutional neural networks are applied to target detection problems.CNN has good feature extraction and classification performance, uses
RegionProposal method realizes that target detection problems mainly include following three step: (1) candidate region selects.(2) CNN feature
It extracts.(3) classification is returned with boundary.RCNN algorithm is Chong Die with a large amount of candidate frames in image, causes to extract in characteristic manipulation
Bulk redundancy.And Fast RCNN then this problem of very good solution, speed when training are improved.
Since camera imaging is the 3D coordinate of target object to the projection process of 2D coordinate, can lose in this process
Depth information, thus after obtaining the 2D position coordinates of target object, if to determine that the three-dimensional space position of target object is sat
Mark also needs to know depth distance.In addition, the first video acquisition device can be monocular cam, it is also possible to binocular camera shooting
Depth distance is not present if the first video acquisition device is monocular cam in head, sits in the position 2D for determining target object
After mark, can the physical size according to known to size and target object that target object is imaged in video image determine object
Body is at a distance from the first video acquisition device, it is of course also possible to determine target object and the first video acquisition according to motion parallax
The distance of device.After determining distance, according to the 2D position coordinates of target object, target object and the first video acquisition device
The calibrating parameters of distance and the first video acquisition device determine the target object in the position coordinates of three-dimensional space.
For binocular camera, now widely used is parallel stereopsis (it is considered that not having between binocular camera
Have a rotation, only translate) calculate depth.In parallel stereopsis, the imaging formula of two cameras is as follows:
According to camera imaging formula, and method for correcting polar line is utilized, it is double that biocular systems can be corrected to ideal parallelism
Mesh system (only horizontal direction parallax, internal reference are identical), obtains depth calculation formula are as follows:
Wherein Z is depth information, and f is equivalent focal length, and d is parallax, txThe direction x translates between two camera optical centers.
Polar curve correcting algorithm can use existing algorithm, such as Bouguet algorithm.Binocular parallax can use existing binocular camera
It is calculated with image algorithm, such as the matching algorithm based on dense depth map and the matching process based on sparse features point pair.It is right
In the motion parallax method of monocular cam, it is believed that the motion parallax that monocular cam movement generates is similar to binocular camera
Binocular parallax, can use the principle of binocular camera to calculate depth.
If the first video acquisition device is binocular camera, after the 2D position coordinates for determining target object, then can adopt
The depth distance of target object and the first video acquisition device is calculated with Binocular Vision Principle.It can be by the mesh detected
It marks object and carries out characteristic point detection and matching primitives depth distance.Then according to 2D position coordinates, the target object of target object
Determine target object three with the calibrating parameters of the depth distance of the first video acquisition device and the first video acquisition device
The position coordinates of dimension space.
Wherein, the first video acquisition device is if it is binocular camera, then either what two monocular cams were constituted
Binocular camera, is also possible to binocular mould group camera, and the two-path video image of binocular camera acquisition can first be spliced into one
It is exported again after the video image of road to processing unit.
In certain embodiments, target object is being determined after the location information of three-dimensional space, processing unit can be according to pre-
The internal reference for the second video acquisition device first demarcated, the outer ginseng of the second video acquisition device and target object are in three-dimensional space
Positional information calculation, which goes out, horizontally rotates angle and vertical for what the second video acquisition device turned to target position from current location
Then rotational angle controls the second video acquisition device target position according to the rotational angle being calculated.Such second video
Acquisition device can collect the video image of clearly target barcode, then export the image to processing unit, so as to
Processing unit identifies the target two dimensional code in image.
The principle for calculating the second video acquisition device horizontal/vertical rotation angle is as follows:
It is considered that the camera of the second video acquisition device is rotated around optical center, imaging relations in this way before and after rotation
There is spin matrix, without translation vector.The imaging formula that rotation front and back can be listed is as follows:
Due to the world coordinate system of target object be it is identical:
WhereinIt is the image coordinate before known rotation,It is the postrotational image coordinate of setting, K is to have demarcated
Internal reference, R is the spin matrix to be calculated, and can be converted into horizontal rotation angle α and vertical rotation angle β and solve.
When target barcode is directed at scan code system again for user, the position of scan code system can be accurately found,
And when light is than darker scene, image collecting device light filling can be given.In one embodiment, as shown in Figure 3a, institute
Scan code system is stated in addition to including that processing unit 31, the first video acquisition device 32 being connected with processing unit and the second video are adopted
It further include a LED light 34 except acquisition means 33, LED light 34 is connected with processing unit 31, for filling for the first video acquisition
It sets and the second video acquisition device light filling, so as to than darker, the first video acquisition device and the second video acquisition
Device, which can collect, compares clearly image.In addition, LED can be also used for the position where prompt user's scan code system, with
Just user can by the target barcode shown be aligned scan code system certainly in certain embodiments can also be according to target object
Distance control the brightness of LED light, the brightness apart from remote target object LED is higher, apart from close target object LED light
Brightness can reduce.
In one embodiment, as shown in Figure 3b, scan code system further includes communication interface, the communication interface and payment system
35 communication interface is connected, for barcode scanning result to be sent to payment system 35.The communication interface can be USB interface or serial
Interface, for processing unit 31 after the target barcode image for receiving the acquisition of the second video acquisition device, identification target barcode simultaneously will
As a result it exports to payment system 35, so that payment system 35 is completed to pay according to barcode scanning result.
In order to allow user to be known that whether barcode scanning is successful, in one embodiment, as shown in Figure 3 C, the scan code system
It further include loudspeaker 36, loudspeaker 36 is connected with processing unit 31, prompts barcode scanning payment result for voice broadcast.For example it is sweeping
When code is paid successfully or barcode scanning fails, processing unit 31 can send an instruction to loudspeaker 36, so that loudspeaker broadcasts barcode scanning
The voice prompting of success or barcode scanning failure, such user and staff can know barcode scanning result.
It is specific with one in conjunction with Fig. 4 a and Fig. 4 b below in order to which the scan code system of this specification offer is explained further
Embodiment is illustrated.
It, can not be accurate right during barcode scanning in order to solve user when parking lot and charge station's occasion are paid using barcode scanning
The barcode scanning range of quasi- barcode scanning equipment and separate unit barcode scanning equipment smaller the problem of not being able to satisfy user demand, this specification provide
A kind of scan code system, Fig. 4 a are a schematic diagram of the scan code system application scenarios, and scan code system includes for detecting hand
The positioning shooting head C1 that seat in the plane the is set and barcode scanning camera C2 for scanning the two-dimensional code.Wherein, the internal logic of scan code system
Structural schematic diagram is as shown in Figure 4 b, including master chip 401, and the positioning shooting head 402 being connected with master chip is connected with master chip
Barcode scanning camera 403, wherein 403 are installed on rotary holder 404, holder 404 is also connected with main core 401, by leading
Chip 401 controls its rotation, and scan code system further includes the LED light 405 being connected with master chip 401, for giving positioning camera
402 and 403 light filling of barcode scanning camera, and prompt to user the position of scan code system.Scan code system further includes and master chip 401
Connected loudspeaker 406, for broadcasting payment result.In addition, scan code system further includes the communication module being connected with master chip 401
407, memory RAM408, ROM409 and the electricity 4010 for its power supply.The master chip 401 of scan code system passes through USB interface 4011
Or serial ports 4012 is connected with the master chip 4013 of payment equipment, in addition, payment equipment further includes being connected with master chip 4013
Network interface 4014, communication module 4015, power supply 4016, memory RAM4017 and ROM4018, for user show payment result
Screen 4019 with payment equipment master chip 4013 be connected.Wherein, positioning shooting head 402 is the wide-angle of a binocular mould group
Camera, resolution ratio are 1920*1080@15fps, horizontal/vertical visual angle about 90deg, vertical angle of view about 120deg, positioning shooting
First 402 scanning visual angle is bigger, thus can be used for detecting the position of the mobile phone of displaying target two dimensional code, positioning shooting head
Positioning time is less than 0.5s.Barcode scanning camera 403 is mainly used for barcode scanning, thus selects the larger more visible focal length camera shooting of imaging
Head, barcode scanning camera 403 support 1280*960@60fps, and horizontal view angle about 30deg, vertical angle of view is about 40deg, barcode scanning away from
From about 200~1000mm, the barcode scanning of longer-distance target two dimensional code can be supported.Wherein, barcode scanning camera 403 is mounted on
On holder 404, holder can carry out Pan/Tilt rotation, the direction Pan/Tilt slewing area horizontal direction 45deg, vertical direction
40deg.Barcode scanning camera 403 uses binocular mould group camera, after binocular mould group camera collects two-path video image, merges
And it at camera all the way and exports.Positioning shooting head 402 and barcode scanning camera 403 can be single with processing by following any interface
Member 401 connects: USB interface, MIPI interface, DVP interface.
The scan code system of this specification can first pass through the position of mobile phone of the detection of positioning shooting head 402 comprising target two dimensional code
It sets, then rotates further according to the position guidance barcode scanning camera of mobile phone to suitable position, so as to success barcode scanning, such barcode scanning system
System can voluntarily be automatically aligned to target two dimensional code, go to be aligned without user.Before carrying out barcode scanning using scan code system, need first
Positioning shooting head and barcode scanning camera are demarcated, to determine the position of mobile phone according to calibrating parameters obtained by calibrating
And the angle that barcode scanning camera needs to rotate.Barcode scanning camera 403 and positioning can be taken the photograph using Zhang Zhengyou plane reference method
As first 402 are demarcated, the interior of the internal reference of positioning shooting head 402, outer ginseng, distortion parameter and barcode scanning camera 403 is obtained
Ginseng, outer ginseng, distortion parameter.Due to including also LED light in scan code system, thus user can find according to the prompt of LED light and sweep
Payment code is simultaneously directed at scan code system by the position of code system.When user shows payment code with mobile phone, due to positioning shooting
First 402 scanning visual angle is larger, thus can collect the video image of mobile phone and send processing unit 401, since positioning is taken the photograph
Picture head is wide-angle camera, and the video image distortion of acquisition is more serious, thus processing unit 401 is in the video for receiving mobile phone
After image, distortion correction can be carried out to video image according to the distortion parameter for the positioning shooting head demarcated in advance.And positioning shooting
Head is binocular mould group camera, thus also needs to carry out polar curve to it to correct it, is calculated convenient for subsequent depth distance.
Processing unit 401 can use after the video image of the mobile phone acquired to positioning shooting head 401 is handled
2D position coordinates of the mobile phone in video image are determined based on the Fast R-CNN algorithm of deep learning, in the 2D that mobile phone has been determined
After position coordinates, the depth distance of mobile phone Yu positioning shooting head can be calculated using deep vision principle, by detecting
Target object carry out characteristic point detection and matching primitives depth distance, then further according to the 2D position coordinates of mobile phone, mobile phone with
The calibrating parameters of the depth distance of positioning shooting head and positioning shooting head determine mobile phone in the position coordinates of three-dimensional space.?
Mobile phone is determined behind the position of three-dimensional space, processing unit can be true according to the outer ginseng of barcode scanning camera 403 and the position of mobile phone
Determine the angle B that barcode scanning camera turns to the angle A rotated in the horizontal direction when target position and needs to rotate in vertical direction.
Wherein, target position is target two dimensional code the location of barcode scanning camera at barcode scanning camera shooting area center, is calculated
Out after rotational angle A and B, the holder 404 where processing unit 401 can control barcode scanning camera 403 rotates above-mentioned angle to mesh
Cursor position after barcode scanning camera 404 turns to target position, can acquire the video image of target two dimensional code and be sent to processing list
Member 401, processing unit 401 can identify target two dimensional code after receiving the video image of target two dimensional code, then by target two dimension
The corresponding payment information of code is sent to payment equipment, completes payment to pay equipment.Wherein, the LED light 405 of scan code system can
403 light filling of positioning shooting head 402 or barcode scanning camera is thought, so that the video image of acquisition is more clear.After the completion of payment,
Loudspeaker meeting voice broadcast payment result in scan code system, so as to the result for notifying user's barcode scanning to pay.
In addition, this specification additionally provides a kind of barcode scanning method, the barcode scanning method is applied to sweeping in above-described embodiment
Code system, as shown in figure 5, the barcode scanning method the following steps are included:
S501, the video image for receiving the target object that the first video acquisition device acquires, the target object include mesh
Mark bar code;
S502 and the location information that the target object is determined according to the video image of the target object;
S503, second image collecting device is controlled according to the positional information turn to target position;
S504, the video figure for receiving the target barcode that second video acquisition device is acquired in the target position
Picture;
S505, the identification target barcode simultaneously export barcode scanning result.
Scan code system includes two video acquisition devices, and the first image collecting device is used for the target comprising target barcode
Object positioning, the second image collecting device are used for barcode scanning.According to the view of first collected target object of video acquisition device
Frequency image determines the location information of target object, controls the second image collecting device further according to location information and turns to and can understand
The target position for taking target barcode, then acquire target barcode image and identification, export barcode scanning result.By this
Method, scan code system can automatically determine the position of target barcode, and the second image collecting device is then turned to suitable position
Barcode scanning is postponed, removes alignment scan code system without user oneself, and the second image collecting device can be according to the position of target barcode
Rotation, increases barcode scanning range.
Due to this specification provide barcode scanning method the part that contributes to existing technology or the technical solution it is complete
Portion or part can be embodied in the form of software products, which is stored in a storage medium, packet
Some instructions are included with so that a terminal device executes all or part of the steps of each embodiment method of the application.And it is aforementioned
Storage medium include: USB flash disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory
The various media that can store program code such as (Random Access Memory, RAM), magnetic or disk.
The foregoing is merely the preferred embodiments of the application, not to limit the application, all essences in the application
Within mind and principle, any modification, equivalent substitution, improvement and etc. done be should be included within the scope of the application protection.