CN103345752B - A kind of robot cooperates with mobile phone the method for tracking pedestrians - Google Patents

Abstract

The present invention proposes a kind of mobile robot and to cooperate with mobile phone the method for tracking pedestrians, adopts mobile robot and the co-operating mode of mobile phone to follow the tracks of pedestrian.The present invention is directed to the universal of current intelligent machine, the mobile phone carried with by pedestrian is cooperated with mobile robot, completes tracing task, and the present invention follows the tracks of specific people by the movable information of people.Mobile phone calculates speed and the direction of people's walking according to the measurement data of integrated inertial sensor, and this information is sent to robot.And robot measures walking speed and the direction of neighbouring pedestrian by self laser range finder or other devices, find out the people of the motion feature meeting target pedestrian, the present invention is by illumination effect, and applied range, accurate rate is high.

Description

A kind of robot cooperates with mobile phone the method for tracking pedestrians

Technical field

The invention belongs to Information Technology Equipment field, especially relate to a kind of mobile robot and to cooperate with mobile phone the method for tracking pedestrians.

Background technology

What the present invention studied is the problem that Information Mobile Service robot follows specific pedestrian.Current, the pedestrian target based on computer vision is followed the tracks of and has been developed to obtain comparative maturity, have also been obtained effective application in a lot of occasion.But the method for tracking target of view-based access control model sensor also also exists a lot of inherent shortcoming, the loss of depth information in as comparatively large by illumination effect in algorithm, image imaging process, algorithm complex are high, make such as blocking, result of use in the situation such as night is unsatisfactory.

Due to popularizing of smart mobile phone, the mobile phone that the present invention is carried with by pedestrian cooperates with mobile robot, completes tracing task.Tracking based on video follows the tracks of specific people by the characteristics of image of people; And the present invention follows the tracks of specific people by the motion of people and spatial information, these information are provided jointly by sensors such as the inertial sensor (accelerometer, electronic compass and gyroscope etc.) on mobile phone and the laser range finders in robot.

Have some literature research calculate people walking situation by mobile phone: document [2] [3] [7] [8] have studied the problem walked by mobile phone meter, document [1] mainly compares the several methods being estimated pedestrian's walking direction by mobile phone, and document [2] [3] [4] [5] [6] have studied the problem how estimating pedestrian's step-length.By these methods, speed and the direction of pedestrian's walking can be determined.

In addition, in existing document, also detect the position of people and these people existed around often through the laser range finder that robot loads, as [9] [10] [11].By continuous print range determination, movement velocity and the direction of these people just can be judged.

List of references:

[1]Steinhoff，U.andSchiele，B.Deadreckoningfromthepocket-Anexperimentalstudy.InproceedingsofIEEEInternationalConferenceonPervasiveComputingandCommunications(PerCom)，Mannheim，Germany，March29-April22010，162-170.

[2]Bylemans，I.，Weyn，M.andKlepal，M.MobilePhone-BasedDisplacementEstimationforOpportunisticLocalisationSystems.InProceedingsofThirdInternationalConferenceonMobileUbiquitousComputing，Systems，ServicesandTechnologies，2009(UBICOMM′09)，Sliema，Malta，11-16October2009，113-118.

[3]Shin，S.H.，Lee，M.S.，Park，C.G.andHong，H.S.Pedestriandeadreckoningsystemwithphonelocationawarenessalgorithm.InProceedingsofIEEE/IONPositionLocationandNavigationSymposium(PLANS)，IndianWells/PalmSprings，CaliforniaUSA，3-6May2010，97-101.

[4]Alzantot，M.andYoussef，M.UPTIME：Ubiquitouspedestriantrackingusingmobilephones.InProceedingsofWirelessCommunicationsandNetworkingConference(WCNC)，Paris，France，1-4April2012，3204-3209.

[5]HATANO，Y.″Useofthepedometerforpromotingdailywalkingexercise.″Int.CouncilHealthPhys.Educ.Recreat，1993，29：4-8.

[6]HarveyWeinberg，UsingtheADXL202inPedometerandPersonalNavigationApplications，AnalogDevicesAN-602applicationNote，2002.

[7]Mladenov，M.，andMock，M.：‘AstepcounterserviceforJava-enableddevicesusingabuilt-inaccelerometer’.InProceedingsofthe1stInternationalWorkshoponContext-AwareMiddlewareandServices：affiliatedwiththe4thInternationalConferenceonCommunicationSystemSoftwareandMiddleware(COMSWARE2009)，Dublin，Ireland，15-19June2009，1-5.

[8]NajmeZehraNaqvi，AshwaniKumar，AanchalChauhanandKritkaSahni.StepCountingUsingSmartphone-BasedAccelerometer.InternationalJournalonComputerScienceandEngineering(IJCSE)，May2012，4(5)：675-681.

[9]Luo，R.C.，C.ChihChia，etal.(2011).″MultisensorFusionandIntegration：Theories，Applications，anditsPerspectives.″SensorsJournal，IEEE11(12)：3122-3138.

[10]Kristou，M.，A.Ohya，etal.(2011).TargetpersonidentificationandfollowingbasedonomnidirectionalcameraandLRFdatafusion.RO-MAN，2011IEEE.

[11]Fod，A.，A.Howard，etal.Alaser-basedpeopletracker.InproceedingsofIEEEInternationalConferenceonRoboticsandAutomation(ICRA′02)，2002.

Summary of the invention

In order to make up above-mentioned mobile robot tracking pedestrian poor reliability, the defects such as calculated amount is large, the invention provides a kind of mobile robot and to cooperate with mobile phone the method for tracking pedestrians.

The technical scheme realizing above-mentioned beneficial effect is, a kind of mobile robot cooperates with mobile phone the method for tracking pedestrians, and the mode adopting mobile robot and mobile phone to cooperatively interact is followed the tracks of pedestrian, and described tracing process is as follows:

(1) integrated on target person mobile phone inertial sensor is measured target person motion conditions, calculates its movement velocity.

(2) set up coordinate system, mobile robot sets up certainly be positioned at conplane two-dimensional coordinate system as true origin and pedestrian;

(3) candidate is detected, mobile robot detects pedestrian position around, using the pedestrian that detects as a candidate, and testing result is showed in a coordinate system, obtain the coordinate position of a candidate, assuming that the number of candidates detected is N people, be then that the probability of target pedestrian is set as 1/N by candidate;

(4) upgrade coordinate system, mobile robot re-establishes coordinate system after traveling time t, is still initial point with self-position, sets up the parallel and new coordinate system that unit is identical of coordinate system change in coordinate axis direction middle with step (1);

(5) detect secondary candidate, mobile robot detects pedestrian position around again, using the pedestrian that detects as secondary candidate, and identifies out respectively by it in new coordinate system, obtains the coordinate position of secondary candidate;

(6) candidate's coordinate is upgraded, the candidate's coordinate renew detected in step 2 is the coordinate under new coordinate system according to self translational speed and moving direction by mobile robot, is put in same coordinate system by the coordinate position of a candidate and secondary candidate;

(7) all pedestrian's speed and velocity reversal is calculated, mobile robot list once candidate and secondary candidate may for the combinations with a group traveling together, being calculated velocity magnitude and the direction of movement by the coordinate difference of this pedestrian when detecting for twice, obtaining translational speed and moving direction data V ', the A ' of each combination;

(8) calculating candidate that is current and a upper moment in each combination is all the probability of the combination of target, target person mobile phone calculates movement velocity V and the direction A of time t internal object people, and result of calculation is sent to mobile robot, mobile robot compares, by formula the translational speed of each combination in step (7) and direction according to mobile phone result of calculation

$\mathrm{Pro}({\mathrm{Subject}}_{\mathrm{last}\_j},{\mathrm{Subject}}_{\mathrm{now}\_k})=(\mathrm{\α}(1-{\left(\frac{V^{\′}-V}{V\max }\right)}^2)+(1-\mathrm{\α})(1-{\left(\frac{A^{\′}-A}{360}\right)}^2))*P_{{\mathrm{subject}}_{\mathrm{last}\_j}}$

Calculating current time and the candidate in a upper moment in each combination is all the probability of target, wherein Subject _{last_j}a upper moment jth candidate, Subject _{now_k}be a current time kth candidate, α is weight, Psubject _{last_j}subject _{last_j}be just in time the probability of target pedestrian, Vmax is the maximal rate of pedestrian's walking:

(9) calculate the probability that each candidate detected in combining above is just in time target person, computing formula is as follows:

$P_{{\mathrm{subject}}_{\mathrm{now}\_k}={\mathrm{\Σ}}_{i=1}^M\mathrm{Pro}({\mathrm{Subject}}_{\mathrm{last}\_i},{\mathrm{Subject}}_{\mathrm{now}\_k})}$

M is wherein the number of the candidate that the probability that screened in a upper moment is higher.

(10) screen as the higher several candidates of probability using M maximum for correct probability in step (9) candidate, simultaneously the highest to a probability candidate follows the tracks of.

The computing formula of described weight α is as follows:

$\mathrm{\α}=\frac{E\left({(A_{\mathrm{robot}}-A_{\mathrm{truth}})}^2{(A_{\mathrm{phone}}-A_{\mathrm{truth}})}^2\right)}{E\left({(A_{\mathrm{robot}}-A_{\mathrm{truth}})}^2{(A_{\mathrm{phone}}-A_{\mathrm{truth}})}^2\right)+E\left({(V_{\mathrm{robot}}-V_{\mathrm{truth}})}^2{(V_{\mathrm{phone}}-V_{\mathrm{truth}})}^2\right)}$

Wherein, E represents and asks expectation value, A _robot, V _robotrepresent the candidate target pedestrian movement direction that robot measures and speed, A _phone, V _phonerepresent mobile phone for measuring target pedestrian direction of motion out and speed, A _truth, V _truthrepresent real motion direction and the real motion speed of candidate target pedestrian or target pedestrian.。

The present invention is directed to the universal of current intelligent machine, the mobile phone carried with by pedestrian is cooperated with mobile robot, completes tracing task, and the present invention follows the tracks of specific people by the movable information of people.Mobile phone calculates speed and the direction of people's walking, and this information is sent to robot.And robot measures walking speed and the direction of neighbouring pedestrian by self laser range finder or other devices, find out the people of the motion feature meeting target pedestrian, the present invention is by illumination effect, and applied range, accurate rate is high.

Accompanying drawing explanation

Fig. 1 is the schematic flow sheet of one embodiment of the present invention;

Embodiment

Below in conjunction with accompanying drawing, the present invention will be further described.

Mobile robot cooperates with mobile phone the method for tracking pedestrians, and adopt mobile robot and the co-operating mode of mobile phone to follow the tracks of pedestrian, it is characterized in that, described tracing process is as follows:

(1) integrated on target person mobile phone inertial sensor is measured target person motion conditions, calculates its movement velocity.

(2) set up coordinate system, mobile robot sets up certainly be positioned at conplane two-dimensional coordinate system as true origin and pedestrian;

(3) candidate is detected, mobile robot detects pedestrian position around, using the pedestrian that detects as a candidate, and testing result is showed in a coordinate system, obtain the coordinate position of a candidate, assuming that the number of candidates detected is N people, be then that the probability of target pedestrian is set as 1/N by candidate;

(4) upgrade coordinate system, mobile robot re-establishes coordinate system after traveling time t, is still initial point with self-position, sets up the parallel and new coordinate system that unit is identical of coordinate system change in coordinate axis direction middle with step (1);

(5) detect secondary candidate, mobile robot detects pedestrian position around again, using the pedestrian that detects as secondary candidate, and identifies out respectively by it in new coordinate system, obtains the coordinate position of secondary candidate;

(6) candidate's coordinate is upgraded, the candidate's coordinate renew detected in step 2 is the coordinate under new coordinate system according to self translational speed and moving direction by mobile robot, is put in same coordinate system by the coordinate position of a candidate and secondary candidate;

(7) all pedestrian's speed and velocity reversal is calculated, mobile robot list once candidate and secondary candidate may for the combinations with a group traveling together, velocity magnitude and the direction of movement is calculated by the coordinate difference of this pedestrian when detecting for twice, obtain the translational speed of each combination and moving direction data V ', A ' (V ' with " meter per second " for unit, direction A ' for unit, is the number of degrees turned clockwise from earth magnetism direct north with " angle ");

(8) calculating candidate that is current and a upper moment in each combination is all the probability of the combination of target, target person mobile phone calculates movement velocity V and the direction A of time t internal object people, and result of calculation is sent to mobile robot, mobile robot compares, by formula the translational speed of each combination in step (7) and direction according to mobile phone result of calculation

$\mathrm{Pro}({\mathrm{Subject}}_{\mathrm{last}\_j},{\mathrm{Subject}}_{\mathrm{now}\_k})=(\mathrm{\α}(1-{\left(\frac{V^{\′}-V}{V\max }\right)}^2)+(1-\mathrm{\α})(1-{\left(\frac{A^{\′}-A}{360}\right)}^2))*P_{{\mathrm{subject}}_{\mathrm{last}\_j}}$

Calculating current time and the candidate in a upper moment in each combination is all the probability of target, wherein Subject _{last_j}a upper moment jth candidate, Subject _now-kbe a current time kth candidate, α is weight, Psubject _{last_j}subject _{last_j}be just in time the probability of target pedestrian, Vmax is the maximal rate of pedestrian's walking:

(9) calculate the probability that each candidate detected in combining above is just in time target person, computing formula is as follows:

$P_{{\mathrm{subject}}_{\mathrm{now}\_k}={\mathrm{\Σ}}_{i=1}^M\mathrm{Pro}({\mathrm{Subject}}_{\mathrm{last}\_i},{\mathrm{Subject}}_{\mathrm{now}\_k})}$

M is wherein the number of the candidate that the probability that screened in a upper moment is higher.

(10) screen as the higher several candidates of probability using M maximum for correct probability in step (9) candidate, simultaneously the highest to a probability candidate follows the tracks of.

The computing formula of described weight α is as follows:

$\mathrm{\α}=\frac{E\left({(A_{\mathrm{robot}}-A_{\mathrm{truth}})}^2{(A_{\mathrm{phone}}-A_{\mathrm{truth}})}^2\right)}{E\left({(A_{\mathrm{robot}}-A_{\mathrm{truth}})}^2{(A_{\mathrm{phone}}-A_{\mathrm{truth}})}^2\right)+E\left({(V_{\mathrm{robot}}-V_{\mathrm{truth}})}^2{(V_{\mathrm{phone}}-V_{\mathrm{truth}})}^2\right)}$

Wherein, E represents and asks expectation value, A _robot, V _robotrepresent the candidate target pedestrian movement direction that robot measures and speed, A _phone, V _phonerepresent mobile phone for measuring target pedestrian direction of motion out and speed, A _truth, V _truthrepresent real motion direction and the real motion speed of candidate target pedestrian or target pedestrian, in the present invention, the accuracy situation that expectation value E measures pedestrian according to mobile phone and robot specifically calculates, and depends on the error of equipment and the quality of measuring method.

Concrete illustrating is carried out to the present invention below.In the present embodiment, suppose that initial pedestrian has four, mobile robot is with certainly as initial point, and selectively magnetic field north is as Y-axis positive dirction, sets up coordinate system using the vertical direction of Y-axis positive dirction as X-axis; To set up after coordinate system respectively using these four pedestrians as a candidate, in coordinate system, record its coordinate (x1, y1), (x2, y2), (x3, y3), (x4, y4), the probability being simultaneously all target pedestrian by four pedestrians is set as 1/4.After elapsed time t, suppose that mobile robot self displacement is an X-axis positive dirction S unit, mobile robot still using this position selectively magnetic field north as Y-axis positive dirction, set up new coordinate system using the vertical direction of Y-axis positive dirction as X-axis, and new coordinate system is consistent with old coordinate system unit; After new establishment of coordinate system, detect the position coordinates (xt1, yt1) of pedestrian under current coordinate system, (xt2, yt2), (xt3, yt3), (xt4, yt4), and it can be used as second time candidate.The coordinate renew of the first candidate recorded is the coordinate under new coordinate system by mobile robot, update mode is the coordinate displacement that the coordinate figure of a candidate deducts mobile robot, i.e. (x1, y1), (x2, y2), (x3, y3), (x4, y4) (x1-s, y1), (x2-S, y2), (x3-s is updated to, y3), (x4-S, y4).After secondary candidate and candidate coordinate position in the same coordinate system being detected, list once candidate's combination that may overlap with secondary candidate.Then its translational speed V ' in time t and moving direction A ' is calculated respectively according to the changes in coordinates of these combinations.Suppose that in four pedestrians, pedestrian 1 is pedestrian's reference of this embodiment, the movement velocity V of time t one skilled in the art 1 and direction of motion A is sent to mobile robot by the mobile phone then entrained by pedestrian 1, mobile robot is by V, A and V ', A ' compares, calculating the candidate in former and later two moment in each combination is probability of target, adds up the probability that each candidate that current time detects is just in time target person after having calculated again.M (in the present embodiment M≤4) the individual candidate of mobile robot to the maximum probability calculated retains, and the candidate of maximum probability in this M candidate is followed the trail of as target pedestrian simultaneously.

The above; for better case study on implementation of the present invention; not the present invention is imposed any restrictions, every above embodiment is done according to the technology of the present invention essence any simple modification, change and equivalent structure change, all still belong in the protection domain of technical solution of the present invention.

Claims (2)

1. mobile robot cooperates with mobile phone a method for tracking pedestrians, and adopt mobile robot and the co-operating mode of mobile phone to follow the tracks of pedestrian, it is characterized in that, described tracing process is as follows:

(1) integrated on target person mobile phone inertial sensor is measured target person motion conditions, calculates its movement velocity;

(2) set up coordinate system, mobile robot sets up certainly be positioned at conplane two-dimensional coordinate system as true origin and pedestrian;

(3) candidate is detected, mobile robot detects pedestrian position around, using the pedestrian that detects as a candidate, and testing result is showed in a coordinate system, obtain the coordinate position of a candidate, assuming that the number of candidates detected is N people, be then that the probability of target pedestrian is set as 1/N by candidate;

(4) upgrade coordinate system, mobile robot re-establishes coordinate system after traveling time t, is still initial point with self-position, sets up the parallel and new coordinate system that unit is identical of coordinate system change in coordinate axis direction middle with step (2);

(5) detect secondary candidate, mobile robot detects pedestrian position around again, using the pedestrian that detects as secondary candidate, and identifies out respectively by it in new coordinate system, obtains the coordinate position of secondary candidate;

(6) candidate's coordinate is upgraded, the candidate's coordinate renew detected in step (3) is the coordinate under new coordinate system according to self translational speed and moving direction by mobile robot, is put in same coordinate system by the coordinate position of a candidate and secondary candidate;

(7) all pedestrian's speed and velocity reversal is calculated, mobile robot list once candidate and secondary candidate may for the combinations with a group traveling together, being calculated velocity magnitude and the direction of movement by the coordinate difference of this pedestrian when detecting for twice, obtaining translational speed and moving direction data V ', the A ' of each combination;

(8) calculating candidate that is current and a upper moment in each combination is all the probability of the combination of target, target person mobile phone calculates movement velocity V and the direction A of time t internal object people, and result of calculation is sent to mobile robot, mobile robot compares, by formula the translational speed of each combination in step (7) and direction according to mobile phone result of calculation

$\mathrm{Pr}o\left({\mathrm{Subject}}_{last\_j},{\mathrm{Subject}}_{now\_k}\right)=\left(\mathrm{\α}\left(1-{\left(\frac{V^{,}-V}{V\max }\right)}^2\right)+\left(1-\mathrm{\α}\right)\left(1-{\left(\frac{A^{\′}-A}{360}\right)}^2\right)\right)*P_{{\mathrm{subject}}_{last\_j}}$

Calculating current time and the candidate in a upper moment in each combination is all the probability of target, wherein Subject _{last_j}a upper moment jth candidate, Subject _{now_k}be a current time kth candidate, α is weight, Psubject _{last_j}subject _{last_j}be just in time the probability of target pedestrian, Vmax is the maximal rate of pedestrian's walking:

(9) calculate the probability that each candidate detected in combining above is just in time target person, computing formula is as follows:

$P_{{\mathrm{subject}}_{now\_k}={\Σ}_{i=1}^M\mathrm{Pr}o\left({\mathrm{Subject}}_{last\_i},{\mathrm{Subject}}_{now\_k}\right)}$

M is wherein the number of the candidate that the probability that screened in a upper moment is higher;

(10) screen as the higher several candidates of probability using M maximum for correct probability in step (9) candidate, simultaneously the highest to a probability candidate follows the tracks of.

2. mobile robot according to claim 1 cooperates with mobile phone the method for tracking pedestrians, and it is characterized in that, the computing formula of described weight α is as follows:

$\mathrm{\α}=\frac{E\left({\left(A_{robot}-A_{truth}\right)}^2{\left(A_{phone}-A_{truth}\right)}^2\right)}{E\left({\left(A_{robot}-A_{truth}\right)}^2{\left(A_{phone}-A_{truth}\right)}^2\right)+E\left({\left(V_{robot}-V_{truth}\right)}^2{\left(V_{phone}-V_{truth}\right)}^2\right)}$

Wherein, E represents and asks expectation value, A _robot, V _robotrepresent the candidate target pedestrian movement direction that robot measures and speed, A _phone, V _phonerepresent mobile phone for measuring target pedestrian direction of motion out and speed, A _truth, V _truthrepresent real motion direction and the real motion speed of candidate target pedestrian or target pedestrian.