CN100450179C - Household safe and security equipment for solitary old person based on omnibearing computer vision - Google Patents

Abstract

The present invention relates to a household safe and security device for solitary old persons based on the omni-bearing computer vision, which comprises a microprocessor and an omni-bearing visual sensor for monitoring the health and security condition of the old persons, wherein the omni-bearing visual sensor is connected with the microprocessor to collect the video information of the spatial position of the main place where the old persons move and the time information corresponding to the moving process, and the mechanical visual method is used to establish the outdoor and indoor motion model for the solitary old persons. Because each solitary old person has different inhabiting environments and living habits, the model needs to learn the daily-life regular motion in a self-adapted way. Then, the present invention discoveries and forecasts the abnormal condition of the old persons in the process of living by capturing the living regularity on the variation of time and space. Moreover, the established old person indoor and outdoor motion model not only can identify the abnormal condition happening in the visual monitoring range, but also can forecast the appearing abnormality happening out of the visual monitoring range (including the outdoor space). The present invention can make the solitary old persons obtain the timely and proper rescue or service at the moment of truth.

Description

Household safe and security equipment for solitary old person based on omnidirectional computer vision

(1) technical field

The invention belongs to the application of technology aspect aging society old man monitoring such as omnidirectional computer vision transducer, image understanding, information processing, be applicable to the family that need obtain weak populations such as old solitary people instant and that suitably serve.

(2) background technology

An investigation according to the United Nations shows that expect the year two thousand thirty, the population of Chinese over-65s will account for 12.7% of population; Its hollow nest old man family will account for 90% of old man family sum.The old solitary people Increase of population needs us that various monitoring services are provided.GE company points out that in the whole world research that 2003 do the problem that the long distance monitoring person of old solitary people is concerned about most is that falling down of old man is unusual.Another research is pointed out: the probability that the old man of over-65s falls down in a year is 30%, and the probability that the old man more than 75 years old falls down is 42%.Therefore, how to detect falling down of old man and caused domestic and international scientific research person's interest unusually gradually.

Abroad, there have been many researchers to carry out the research of this respect.Some researchers are installed in acceleration transducer on one's body the old man, fall down unusually by unusual detection of monitoring acceleration.Also have some researchers use a computer vision technique monitoring old man's behavioral activity detect fall down unusual.But unusual for falling down outside the monitoring place, these methods are just powerless.

At home, before the present invention makes the family of weak populations such as solitary old age being taken place when unusual mainly is to report to the police and the request service to the side of rescuing with active form by the alarm button in phone or the family, or adopts the method for visiting to the doorstep to confirm whether the old man is safe and comfortable.A storm may arise from a clear sky, Man's fate is as uncertain as the weather, no one can expect own can be sick in some time one day, can not expect oneself may occur phone and all ask to report to the police rescue all difficult the time.Recent years, people found intelligently that by information technology old man's life is unusual, and the patent No. is that the activity that adopts infrared sensor to survey the old man in the CN200410017289.4 household safe and security equipment for solitary old person judges whether the old man occurs unusually; The patent No. is that CN200410066707.9 is based on adopting the amount of water used in everyday, electricity, coal gas and hot gas among the elder person to judge whether the old man occurs unusually in the old solitary people safety protection device of life supply line.

On the other hand, the omnibearing vision sensor ODVS (OmniDirectional VisionSensors) that gets up of developed recently provides a kind of new solution for the panoramic picture that obtains scene in real time.The characteristics of ODVS are looking away (360 degree), can become piece image to the Information Compression in the hemisphere visual field, and the amount of information of piece image is bigger; When obtaining a scene image, the riding position of ODVS in scene is free more; ODVS is without run-home during protected environment; Algorithm is simpler during moving object in the detection and tracking monitoring scope; Can obtain the realtime graphic of scene.This ODVS video camera mainly is made up of a ccd video camera and a reflective mirror that faces camera.Reflective mirror reflects the image in one week of horizontal direction to the ccd video camera imaging, like this, just can obtain the environmental information of horizontal direction 360o in piece image.This omnidirectional vision camera has very outstanding advantage, under the real-time processing requirements to panorama, is a kind of quick, approach of visual information collection reliably especially.

Because omni-directional visual is a kind of typical machine vision, is that the people can not possess.The principle of the principle of camera acquisition image and eye-observation object is different, and the image difference that makes omnidirectional images and human eye see is also very big, even according to cylinder unwrapping, its deformation still exists.Therefore how by comprehensive optical image technology, computer image processing technology and the network communications technology for intelligent old solitary people family security field provide a kind of guard fast, reliably the field on a large scale in visual information gather approach, and the real-time omnidirectional images that obtains according to the ODVS video camera, falling down unusually in the monitoring place not only can be detected by intelligentized treatment technology, but also falling down unusually outside the monitoring place can be predicted.

Therefore how provide instant and suitably service by comprehensive optical image technology, computer image processing technology and the network communications technology to the family of weak populations such as solitary old age, to the quality of life that improves the elderly, solve the endowment of the existing family of China and go up existing social concern positive effect is arranged.

(3) summary of the invention

For overcome in the prior art old solitary people safety protection device crucial moment can not be in time, suitably service is provided and rescues help for old solitary people, the invention provides a kind of household safe and security equipment for solitary old person based on omnidirectional computer vision, adopt this device not only can automatically detect falling down unusually of old man in the monitoring place, can also predict that simultaneously falling down of old man outside the monitoring place is unusual, and can obtain the old solitary people safety protection device of rescuing or serving instantaneity, that be fit in crucial moment.

The technical solution adopted for the present invention to solve the technical problems is:

A kind of household safe and security equipment for solitary old person based on omnidirectional computer vision, described household safe and security equipment for solitary old person comprises microprocessor, be used to guard the omnibearing vision sensor of the safe and comfortable situation of old man, described omnibearing vision sensor is connected with microprocessor, described omnibearing vision sensor comprises the evagination catadioptric minute surface in order to object in the reflection monitoring field, in order to the dark circles cone that prevents that anaclasis and light are saturated, transparent cylinder, camera, described evagination catadioptric minute surface is positioned at the top of transparent cylinder, evagination catadioptric minute surface down, the dark circles cone is fixed on the center of catadioptric minute surface male part, camera faces toward the evagination mirror surface up, and described camera is positioned at the virtual focus position of evagination mirror surface;

Described microprocessor also comprises:

The view data read module is used to read the video image information of coming from the omnibearing vision sensor biography;

The image data file memory module, the video image information that is used for reading is kept at memory cell by file mode;

The omnibearing vision sensor demarcating module is used for the parameter of omnibearing vision sensor is demarcated, and sets up the material picture in space and the corresponding relation of the video image that is obtained;

Image launches processing module, and the circular video image that is used for reading expands into the panorama block diagram;

The motion obj ect detection module, present frame live video image and a relatively stable reference image of being used for being obtained carry out the difference computing, and the computing formula of image subtraction is represented suc as formula (28):

f _d(X，t ₀，t _i)＝f(X，t _i)-f(X，t ₀) (28)

In the following formula, f _d(X, t ₀, t _i) be to photograph the result who carries out image subtraction between image and reference image in real time; F (X, t _i) be to photograph image in real time; F (X, t ₀) be the reference image;

And with in the present image with the image subtraction computing formula of adjacent K frame shown in (29):

f _d(X，t _i-k，t _i)＝f(X，t _i)-f(X，t _i-k) (29)

In the following formula, f _d(X, t _I-k, t _i) be to photograph the result who carries out image subtraction between image and adjacent K two field picture in real time; F (X, t _I-k) image when being adjacent K frame;

As f _d(X, t ₀, t _i) 〉=threshold value, f _d(X, t _I-k, t _iWhen) 〉=threshold value is set up, be judged to be the motion object;

As f _d(X, t ₀, t _i) 〉=threshold value, f _d(X, t _I-k, t _i)＜threshold value is judged stationary objects, and upgrades replacement reference image with formula (30):

$f(X,t_0)\⇐f(X,t_{i-k})---\left(30\right)$

As f _d(X, t ₀, t _i)＜threshold value is judged to be stationary objects;

The connected region computing module, be used for present image is carried out mark, pixel grey scale is that no old man's activity in this sub-district is represented in 0 sub-district, pixel grey scale is that 1 this sub-district of expression has old man's activity, whether the pixel of calculating in the present image equates with the pixel of some points adjacent around the current pixel, equate to be judged as gray scale and have connectedness, all are had connective pixel as a connected region; And then calculate its area and center of gravity according to the connected region of being tried to achieve; Old man's center of gravity obtains by the X that calculates resulting connected region area Si and this connected region, the accumulation calculated for pixel values of Y direction, and computing formula is calculated by formula (42),

$X_{\mathrm{cg}\left(i\right)}=\frac{\underset{x,y\∈S_i}{\mathrm{\Σ}}x}{S_i};Y_{\mathrm{cg}\left(i\right)}=\frac{\underset{x,y\∈S_i}{\mathrm{\Σ}}y}{S_i}---\left(42\right);$

The acquisition module of old man's activities of daily living data is used for movable time started, perdurabgility and the location in space in the monitoring visual range according to the old man, obtains the rule of old man's activities of daily living;

Old man's activities of daily living model module, be used for having grasped after old man's the rule of daily life by the method for self study, can summarize old man's activities of daily living model by the summary of this rule, adopt two-dimentional Gaussian distribution model to describe the room and time distributed model of the activity of certain old man's daily life:

σ in the formula (32) _xThe mathematics variance of brightness value on the x axle of expression connected region, σ _yThe mathematics variance of brightness value on the y axle of expression connected region, μ _xThe mathematic expectaion mean value of brightness value on the x axle of expression connected region, μ _yThe mathematic expectaion mean value of brightness value on the y axle of expression connected region;

The renewal of model uses the digital low-pass filtering method of formula (33)～(36) to carry out,

μ _xnew＝(1-k ₁)μ _xold+k ₁(x-μ _xold)，(33)

μ _ynew＝(1-k ₂)μ _yold+k ₂(y-μ _yold)，(34)

${\mathrm{\&sigma;}}_{\mathrm{<figure-callout\; id="2"\; label="xnew"\; filenames="C20061005172900331.png,C20061005172900361.png"\; state="\{\{state\}\}">xnew</figure-callout>}}^2=(1-k_3){\mathrm{\&sigma;}}_{\mathrm{xold}}^2+k_3{(x-{\mathrm{\&mu;}}_{\mathrm{xold}})}^2,---\left(35\right)$

${\mathrm{\&sigma;}}_{\mathrm{<figure-callout\; id="2"\; label="ynew"\; filenames="C20061005172900331.png,C20061005172900361.png"\; state="\{\{state\}\}">ynew</figure-callout>}}^2=(1-k_4){\mathrm{\&sigma;}}_{\mathrm{<figure-callout\; id="2"\; label="yold"\; filenames="C20061005172900331.png,C20061005172900361.png"\; state="\{\{state\}\}">yold</figure-callout>}}^2+k_4{(y-{\mathrm{\&mu;}}_{\mathrm{yold}})}^2,---\left(36\right)$

Wherein, ki is a learning rate, and span is [0,1], and learning phase can be selected big learning rate (0.8), accelerates pace of learning; The monitoring stage can be selected little learning rate (0.01);

Fall down unusual detection module in the vision monitoring scope, be used for entering the threshold value that is still in certain position again for a long time behind certain monitoring field and surpasses old man's activities of daily living model module defined and detecting judgement by detecting the old man, the time that detection old man's position of centre of gravity x is judged to inactivity point surpasses threshold value, system-computed following formula so:

PS＝max{p(x|i，μi，∑ _i)，i＝1，...，n} (37)

P in the formula (x|i, μ i, ∑ _i) be the probability that old man's position of centre of gravity x belongs to the gauss hybrid models of the inactivity zone that is numbered i, PS is the maximum probability that position x belongs to certain inactivity zone;

When PS value during less than threshold value, judge that position x does not belong to any known inactivity zones, promptly the old man be still in take place on some positions unusual;

Further, calculate the probability P T that the old man is in the attitude that couches,, use formula (38) to calculate at last and fall down the probability of unusual generation by human body attitude recognition methods based on hidden Markov model from detected image identification old man's attitude:

P＝K1×(1-PS)+K2×PT (38)

Wherein K1, K2 are weights, and PS is that the old man is still in certain locational probability for a long time, and PT is the probability that the old man is in the attitude of lying, and when P surpasses threshold value, judge that the old man falls down unusually;

Non-vision is guarded the unusual prediction module of falling down in the scope, be used for crossing and predict when old man's situation does not also appear in the certain hour threshold value by vision monitoring wide-ultra, by detect the old man moment t1 leave the monitoring field of vision and institute's elapsed time predict, at first to from old man's activities of daily living model, retrieve the predicted value μ i of this movable duration of corresponding movable duration model, then the real-time following expression formula of calculating of system:

PE＝t-t1-μi (39)

PC＝p(t-t1|i，μi，σi) (40)

Wherein, t is current system time, and t-tl is the duration of leaving monitoring vision place, and PE is the actual duration in monitoring vision place and the difference of prediction duration value left; PC is the probability that belongs to the Gauss model of movable duration the actual activity time, the promptly actual probability that leaves the monitoring vision place duration;

Current time t, the probability that the old man falls down in non-monitoring vision place is:

P＝K1×PE+K2×(1-PC) (41)

Wherein K1, K2 are weights, and when P surpassed threshold value, the judgement old man may be taken place unusually; The abnormality alarming module is used for notifying the rescue personnel by communication module after judging that the old man may be taken place unusually.

Further, described microprocessor also comprises the background maintenance module, and described background maintenance module comprises:

The background luminance computing unit is used to calculate average background brightness Yb computing formula as the formula (25):

${\stackrel{\&OverBar;}{Y}}_b=\frac{\underset{x=0}{\overset{W-1}{\mathrm{\&Sigma;}}}\underset{y=0}{\overset{H-1}{\mathrm{\&Sigma;}}}{Y}_n(x,y)(1-M_n(x,y))}{\underset{x=0}{\overset{W-1}{\mathrm{\&Sigma;}}}\underset{y=0}{\overset{H-1}{\mathrm{\&Sigma;}}}(1-M_n(x,y))}---\left(25\right)$

In the formula (25), Yn (x y) is the brightness of each pixel of present frame, Mn (x y) is the mask table of present frame, and described mask table is to write down each pixel with one with the measure-alike array M of frame of video whether motion change is arranged, referring to formula (27):

Yb0 is the background luminance of former frame when being judged to be the motion object, and Yb1 is when being judged to be the motion object

The background luminance of first frame, being changed to of two frame mean flow rates:

ΔY＝Yb1-Yb0 (26)

If Δ Y, then thinks the incident of turning on light that taken place greater than higher limit; If Δ Y, then thinks the incident of turning off the light that taken place less than certain lower limit; Between higher limit and lower limit, think then that light changes naturally as Δ Y;

The background adaptive unit is used for carrying out adaptive learning according to following formula (22) when light changes naturally:

X _mix，bn+1(i)＝(1-λ)X _mix，bn(i)+λX _mix，cn(i) (22)

In the formula: X _{Mix, cn}(i) be present frame RGB vector, X _{Mix, bn}(i) be present frame background RGB vector, X _{Mix, bn+1}(i) be next frame background forecast RGB vector, λ is the speed of context update; Changeless background (initial background) is used in λ=0; Present frame is used as a setting in λ=1; 0＜λ＜1, background is mixed by the background and the present frame of previous moment;

When light is caused that by switch lamp background pixel is reset according to present frame, referring to formula (23):

X _mix，bn+1(i)＝X _mix，cn(i) (23)。

Further again, described microprocessor also comprises: noise is rejected module, is used for the average displacement of each pixel value with all values in its local neighborhood, as shown in Equation (16):

h[i，j]＝(1/M)∑f[k，1] (32)

In the following formula (32), M is the pixel sum in the neighborhood.

Further, described image launches processing module, is used for according to a point (x on the circular omnidirectional images ^*, y ^*) and rectangle column panorama sketch on a point (x ^*, y ^*) corresponding relation, set up (x ^*, y ^*) and (x ^*, y ^*) mapping matrix, shown in the formula (21):

P ^**(x ^**，y ^**)←M×P ^*(x ^*，y ^*) (21)

In the following formula, M is a mapping matrix, P ^*(x ^*, y ^*) be the picture element matrix on the circular omnidirectional images, P ^*(x ^*, y ^*) be the picture element matrix on the rectangle column panorama sketch.

Described microprocessor also comprises: network transmission module, and the live video image that is used for being obtained is gone out by netcast in the mode of video flowing, so that the user can grasp field condition in real time by diverse network; Real-time playing module, the live video image that is used for being obtained is played to display device by this module.

Described omnibearing vision sensor is installed in the main activities place that can guard the daily activity of people until old.

The present invention at first is the manufacturing technology scheme of the opticator of ODVS camera head, and the ODVS camera head is mainly constituted by vertically downward catadioptric mirror with towards last camera.It is concrete that to constitute be to be fixed on bottom by the cylinder of transparent resin or glass by the image unit that collector lens and CCD constitute, the top of cylinder is fixed with the catadioptric mirror of a downward deep camber, the dark circles cone that between catadioptric mirror and collector lens, has a diameter to diminish gradually, this coniform body is fixed on the middle part of catadioptric mirror, and the pyramidal purpose of dark circles is to cause light in cylinder inside light reflex saturated and that produce by the cylinder body wall in order to prevent superfluous light from injecting.Fig. 2 is the schematic diagram of the optical system of expression omnibearing imaging device of the present invention.

Catadioptric omnidirectional imaging system can be carried out imaging analysis with the pin-hole imaging model, but obtaining the perspective panorama picture must be to the contrary projection of the real scene image of gathering, thereby amount of calculation is big, particularly is used in to the movable of people or by the activity that the people produces to guard, and must satisfy the requirement of real-time.

At first select for use CCD (CMOS) device and imaging len to constitute camera in the design, preresearch estimates system overall dimension on the basis that the camera inner parameter is demarcated is determined the mirror surface shape parameter according to the visual field of short transverse then.

As shown in Figure 1, the projection centre C of camera is the horizontal scene h of distance place above horizontal scene, and the summit of speculum is above projection centre, apart from projection centre zo place.Be that the origin of coordinates is set up coordinate system with the camera projection centre among the present invention, the face shape of speculum is with z (X) function representation.The pixel q of distance images central point ρ has accepted from horizontal scene O point (apart from Z axle d), at the light of mirror M point reflection in as the plane.Horizontal scene is undistorted to require the coordinate of the horizontal coordinate of scene object point and corresponding picture point linear;

By formula (8), (9), (10) and initial condition, separate the digital solution that the differential equation can obtain reflecting mirror surface shape.The main digital reflex mirror of system's overall dimension is from the distance H o and the aperture of a mirror D of camera.Select suitable camera according to application requirements during the refractive and reflective panorama system design, calibrate Rmin, the focal distance f of lens is determined the distance H o of speculum from camera, calculates aperture of a mirror Do by (1) formula.

Determining of system parameters:

d(ρ)＝αρ (1)

ρ is and the distance of the face shape central point of speculum in the formula (1), and α is the magnification ratio of imaging system.

If the normal that speculum is ordered at M and the angle of Z axle are γ, the angle of incident ray and Z axle is Φ, and the angle of reflection ray and Z axle is θ.Then

$\mathrm{tg}\left(x\right)=\frac{d\left(x\right)-x}{z\left(x\right)-h}---\left(2\right)$

$\mathrm{tg\&gamma;}=\frac{\mathrm{dz}\left(x\right)}{\mathrm{dx}}---\left(3\right)$

$\mathrm{tg}\left(2\mathrm{\&gamma;}\right)=\frac{2\frac{\mathrm{dz}\left(x\right)}{\mathrm{dx}}}{1-\frac{d^{2}z\left(x\right)}{{\mathrm{dx}}^2}}---\left(4\right)$

$\mathrm{tg\&theta;}=\frac{\mathrm{\&rho;}}{f}=\frac{x}{z\left(x\right)}---\left(5\right)$

By reflection law

2γ＝φ-θ

$\stackrel{\&CenterDot;}{\&CenterDot;\&CenterDot;}\mathrm{tg}\left(2\mathrm{\&gamma;}\right)=\mathrm{tg}(\mathrm{\&phi;}-\mathrm{\&theta;})=\frac{\mathrm{tg\&phi;}-\mathrm{tg\&theta;}}{1+\mathrm{tg\&phi;tg\&theta;}}---\left(6\right)$

Obtain the differential equation (7) by formula (2), (4), (5) and (6)

$\frac{d^{2}z\left(x\right)}{{\mathrm{<figure-callout\; id="2"\; label="dx"\; filenames="C20061005172900331.png,C20061005172900361.png"\; state="\{\{state\}\}">dx</figure-callout>}}^2}+2k\frac{\mathrm{dz}\left(x\right)}{\mathrm{dx}}-1=0---\left(7\right)$

In the formula; $k=\frac{z\left(x\right)[z\left(x\right)-h]+x[d\left(x\right)-x]}{z\left(x\right)[d\left(x\right)-x]+x[z\left(x\right)-h]}---\left(8\right)$

Obtain the differential equation (9) by formula (7)

$\frac{\mathrm{dz}\left(x\right)}{\mathrm{dx}}+k-\sqrt{{\mathrm{<figure-callout\; id="2"\; label="k"\; filenames="C20061005172900331.png,C20061005172900361.png"\; state="\{\{state\}\}">k</figure-callout>}}^2+1}=0---\left(9\right)$

Obtain formula (10) by formula (1), (5)

$d\left(x\right)=\frac{\mathrm{afx}}{z\left(x\right)}---\left(10\right)$

Determine system parameters af according to the visual field of using desired short transverse.Obtain formula (11) by formula (1), (2) and (5), done some simplification here, with z (x) ≈ z ₀, main consideration is smaller with respect to the change in location of minute surface and camera for the height change of minute surface;

$\mathrm{tg\&phi;}=\frac{(\mathrm{af}-z_0)\frac{\mathrm{\&rho;}}{f}}{z_0-h}---\left(11\right)$

With the inconocenter point largest circumference place in the center of circle as the plane $\mathrm{\&rho;}=R\min \&RightArrow;{\mathrm{\&omega;}}_{\max }=\frac{R_{\min }}{f}$

Corresponding visual field is φ max.Then can obtain formula (12);

$\frac{\mathrm{\&rho;}}{f}=\frac{(z_0-h)\mathrm{tg}{\mathrm{\&phi;}}_{\max }}{{\mathrm{\&omega;}}_{\max }}+z_0---\left(12\right)$

The imaging simulation adopts the direction opposite with actual light to carry out.If light source is in the camera projection centre, equally spaced selected pixels point in the picture plane by the light of these pixels, intersects with horizontal plane after mirror reflects, if intersection point is equally spaced, illustrates that then speculum has the distortionless character of horizontal scene.The imaging simulation can be estimated the imaging character of speculum on the one hand, can calculate aperture of a mirror and thickness exactly on the other hand.

Further specify the present invention and in implementation process, relate to Several Key Problems such as demarcation and target identification:

(1) how to demarcate the pixel distance in the imaging plane of omnibearing vision sensor and the corresponding relation of actual three dimensions distance, and on this basis, moving image is classified.

(2) how to carry out target following, tracking is equivalent to the corresponding matching problem of creating features relevant such as position-based, speed, shape, texture, color in continuous images interframe, attribute information with personage in the activity among the present invention combines, and a kind of effective, robustness method for tracking target high, that real-time is good is provided.This tracking be actually based on model, based on the zone, based on active contour and based on color characteristic etc. tracking a kind of comprehensive.

The demarcation of omni-directional visual camera field of view distance relates to the theory of imaging geometry, and the three-dimensional scenic of objective world is projected the two-dimentional image plane of video camera, need set up the model of video camera and describe.These image transformations relate to the conversion between the different coordinates.In the imaging system of video camera, what relate to has following 4 coordinate systems; (1) real-world coordinates is XYZ; (2) with the video camera be the coordinate system x^y^z^ that formulate at the center; (3) photo coordinate system, formed photo coordinate system x in video camera ^*y ^*o ^*(4) computer picture coordinate system, the coordinate system MN that the computer-internal digital picture is used is a unit with the pixel.

According to the different transformational relation of above several coordinate systems, just can obtain needed omnidirectional vision camera imaging model, converse the corresponding relation of two dimensional image to three-dimensional scenic.The approximate perspective imaging analytical method that adopts catadioptric omnibearing imaging system among the present invention is with the formed corresponding relation that is converted to three-dimensional scenic as the plane coordinates two dimensional image in the video camera, Fig. 3 is general perspective imaging model, d is people's height, ρ is the image height of human body, t is the distance of human body, and F is the image distance (equivalent focal length) of human body.Can obtain formula (13)

$d=\frac{t}{F}\mathrm{\&rho;}---\left(13\right)$

When the design of the catadioptric omnibearing imaging system that above-mentioned horizontal scene does not have, require the coordinate of the horizontal coordinate of scene object point and corresponding picture point linear, represent suc as formula (1); Comparison expression (13), (1), horizontal as can be seen scene does not have the be imaged as perspective imaging of the catadioptric omnibearing imaging system of distortion to horizontal scene.Therefore with regard to horizontal scene imaging, the catadioptric omnibearing imaging system that horizontal scene can not had distortion is considered as having an X-rayed camera, and α is the magnification ratio of imaging system.If the projection centre of this virtual perspective camera is C point (seeing accompanying drawing 3), its equivalent focal length is F.Comparison expression (13), (1) formula can obtain formula (14);

$\mathrm{\&alpha;}=\frac{t}{F};t=h---\left(14\right)$

Obtain formula (15) by formula (12), (14)

$F=\frac{\mathrm{fh}{\mathrm{\&omega;}}_{\max }}{(z_0-h)\mathrm{tg}{\mathrm{\&phi;}}_{\max }+{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="C20061005172900341.png,C20061005172900361.png"\; state="\{\{state\}\}">z</figure-callout>}}_0{\mathrm{\&omega;}}_{\max 0}}---\left(15\right)$

Carry out the system imaging simulation according to above-mentioned omnidirectional vision camera imaging model, by the camera projection centre send through in the pixel planes equidistantly after the reflection of the light family of pixel, intersection point on the horizontal plane of distance projection centre 3m is equally spaced basically, as shown in Figure 4.Therefore according in the above-mentioned design principle this patent relation between the coordinate of the coordinate of level road and corresponding comprehensive picture point being reduced to linear relationship, that is to say that design by mirror surface be XYZ to the conversion of photo coordinate system with real-world coordinates can be the linear dependence of ratio with magnification ratio α.Be conversion below from photo coordinate system to the used coordinate system of computer-internal digital picture, the image coordinate unit that uses in the computer is the number of discrete pixel in the memory, so also need round the imaging plane that conversion just can be mapped to computer to reality as the coordinate on plane, its conversion expression formula is for to be provided by formula (16);

$M=O_m-\frac{x^{*}}{S_x};$ $N=O_n-\frac{y^{*}}{S_y};---\left(16\right)$

In the formula: O _m, O _nBe respectively the line number and the columns at the some pixel place that the initial point of image plane shone upon on the computer picture plane; S _x, S _yBe respectively scale factor in the x and y direction.S _x, S _yDetermine it is by between camera and mirror surface, placing scaling board, video camera being demarcated obtained S apart from the Z place _x, S _yNumerical value, unit is (pixel); O _m, O _nDetermine it is that unit is (pixel) according to selected camera resolution pixel.

Further, 360 ° of comprehensive principles of making a video recording are described, a some A on the space (x1, y1, z1) through catadioptric 1 direct reflection to the lens 4 to a subpoint P1 (x should be arranged ^*1, y ^*1), the light of scioptics 4 becomes directional light and projects CCD image unit 5, microprocessor 6 reads in this ring-type image by video interface, adopts software that this ring-type image is launched to obtain omnibearing image and be presented on the display unit 7 or by video server to be distributed on the webpage.

Further, on method of deploying, adopted a kind of algorithm of approximate expansion fast in this patent, can drop to minimum, kept Useful Information simultaneously as much as possible with time loss with to the requirement of various parameters.Launching rule has three,

(1) X ^*Axle is an original position, launches by counterclockwise mode;

(2) X among the left figure ^*Axle and the intersection point O of internal diameter r correspond to the initial point O (0,0) in the lower left corner among the right figure;

(3) width of the right figure after the expansion equals the girth of the circle shown in the dotted line among the left figure.Wherein broken circle is the concentric circles of external diameter in the left figure, and its radius r 1=(r+R)/2.

If the center of circle O of circular diagram ^*Coordinate (x ^*0, y ^*0), the histogram lower left corner origin O of expansion ^*(0,0), any 1 P in the histogram ^*=(x ^*, y ^*) pairing coordinate in circular diagram is (x ^*, y ^*) below we need ask is (x ^*, y ^*) and (x ^*, y ^*) corresponding relation.Can obtain following formula according to geometrical relationship:

β＝tan ^-1(y ^*/x ^*) (17)

r1＝(r+R)/2 (18)

Make the radius r 1=(r+R)/2 of broken circle, purpose is in order to allow the figure after launching seem that deformation is even.

x ^*＝y ^*/(tan(2x ^**/(R+r))) (19)

y ^*＝(y ^**+r)cosβ (20)

Can obtain a point (x on the circular omnidirectional images from formula (19), (20) ^*, y ^*) and the rectangle panorama sketch on a point (x ^*, y ^*) corresponding relation.This method has come down to do the process of an image interpolation.After the expansion, the image of dotted line top is that transverse compression is crossed, and the image of dotted line below is that cross directional stretch is crossed, dotted line originally on one's body point then remain unchanged.

The calculating needs equally can be according to a point (x on the circular omnidirectional images in real time in order to satisfy ^*, y ^*) and the rectangle panorama sketch on a point (x ^*, y ^*) corresponding relation, set up (x ^*, y ^*) and (x ^*, y ^*) mapping matrix.Because this one-to-one relationship can be being transformed into indeformable panoramic picture by the mapping matrix method.Can set up formula (21) relation by the M mapping matrix.

P ^**(x ^**，y ^**)←M×P ^*(x ^*，y ^*) (21)

According to formula (21), for each the pixel P on the imaging plane ^*(x ^*, y ^*) a some P arranged on omnidirectional images ^*(x ^*, y ^*) correspondence, set up the M mapping matrix after, the task that realtime graphic is handled can obtain simplifying.

A kind of household safe and security equipment for solitary old person based on omnidirectional computer vision, Fig. 5 is the overall process framework of monitor system, the old solitary people indoor and outdoor motility model of being set up has characteristics such as personalized self study, self adaptation, this model can not only detect unusual in the video monitoring scope simultaneously, and can predict the contingent unusual of (comprising the open air) beyond the video monitoring scope.The device operation begins to need a learning phase, obtains rule each old solitary people, that can reflect the daily routines of its individual character by self-learning method; Learning phase enters the monitoring stage after finishing, because old man's age growth and changes of seasons require system can constantly be updated in the resultant old man's activities of daily living of learning phase model, makes it to have adaptive function.

For the old man is effectively guarded, among the present invention omnibearing vision sensor is installed in the main place of old man's daily routines, parlor for example, system just can monitor most of daily routines of old man like this.Because each old man's living conditions, the daily life habits and customs are not quite similar, after the rule of starting stage, obtain old man's activities of daily living model and can carry out abnormality and unusual judgement with that by automatic study old man daily life, as Fig. 6, omnibearing vision sensor shown in Figure 7 is installed in the parlor situation of the activity centre that can reflect the old man, at least will learn the information of the space-time aspect of following 3 old man's daily lifes: 1) place of daily life behavior often appears in the study old man at the parlor back warp, the position of for example being accustomed to the chair of seat, be referred to as inactivity zones 2) study the parlor all gateways, for example lead to the position of the door in bedroom, be referred to as entry zones; 3) study certain period of old man is left the regularity of duration that the monitoring place enters the activity in other places by certain entry zone, for example the regularity of old man's duration of sleeping in the bedroom.

Device has been grasped by the method for self study after old man's the rule of daily life, can summarize old man's activities of daily living model by the summary of this rule, the video information of coming out according to this model and actual detected compares just can be judged and abnormal conditions such as fall down.Such as for abnormal conditions such as monitoring falling down in the place, can go out great abnormal conditions such as the old man falls down by probability calculation at all known inactivity zones, be judged as if probability is lower than threshold value and fall down or other are great unusual, at this time can confirm by video image; Great unusual for falling down outside the video monitoring place etc., by calculating sometime, the probability of three Gaussian Profile such as a certain orientation (such as the door that is the some discrepancy that communicates with the parlor) and a certain duration, the parameter that to calculate resulting probability and Gaussian distribution model then compares, be judged as if the probability that is calculated is lower than threshold value fall down etc. unusual, especially when seriously surpass predicted value sometime inertia perdurabgility in some spaces of old man, illustrate it very likely is to take action after the old man has fallen down, must provide and nurse timely and succour.

As shown in Figure 7, family for certain old man, the monitoring vision scope in by the parlor and enter bedroom, kitchen, toilet and go out the door form, be numbered 1 the door be the gateway in room, be numbered the bedroom that 2 door connects the parlor, being numbered 3 door is the door in turnover kitchen, and being numbered 4 door is the door that passes in and out the toilet.Omnibearing vision sensor is installed in parlor middle part, and can gather the old man and in the parlor, carry out be connected with the parlor information in space of activities of daily living and discrepancy, and the temporal information of record activity, set up corresponding room and time model.

In the study and the understanding stage of image, device can be followed the tracks of the monitoring old man by computer vision, calculate the movement locus of old man's human body center, going out the now and the time of staying in certain monitoring space, going out the now and the time of staying can obtain by the system for computer time, and the information of old man in certain locus will obtain by the video information that omnibearing vision sensor obtained, and this modeling processing procedure can be described with Fig. 8.

Above-mentioned modeling processing procedure can be described with mathematical way, the room and time probability data of old man's activities of daily living approaches normal distribution, therefore uses Gaussian distribution model to describe the room and time distributed model of the activity of certain old man's daily life in this patent.Because the rule of old solitary people may slowly change with changes of seasons with advancing age, for adaptively changing old man's daily life time and spatial model, uses the low-pass filtering method to upgrade the parameter of Gaussian distribution model in the present invention.

Beneficial effect of the present invention mainly shows: adopt omnibearing vision sensor, not only can automatically detect falling down unusually of old man in the monitoring place, can also predict that simultaneously falling down of old man outside the monitoring place is unusual, and crucial moment can obtain instantaneity, be fit to rescue or serve.

(4) description of drawings

Fig. 1 is the omni-directional visual optical schematic diagram;

Fig. 2 is that a kind of household safe and security equipment for solitary old person based on omnidirectional computer vision is at the hardware configuration schematic diagram of using aspect old man's monitoring;

Fig. 3 is the perspective projection imaging model schematic diagram of omnibearing vision device and general perspective imaging model equivalence;

Fig. 4 is the omnibearing vision device undeformed simulation schematic diagram of epigraph in the horizontal direction;

Fig. 5 is a kind of image processing flow chart of the household safe and security equipment for solitary old person based on omnidirectional computer vision;

Fig. 6 is that a kind of monitor system of the household safe and security equipment for solitary old person based on omnidirectional computer vision is handled overall structure figure;

Fig. 7 sets up schematic diagram for old solitary people indoor and outdoor motility model;

Fig. 8 be at the learning phase of model from following the tracks of the movement locus of old man's human body center, the process of the movable beginning of record, the room and time information of the finish time is handled schematic diagram;

Among Fig. 9, (a) be omnibearing vision sensor software and hardware device, (b) 360 ° of comprehensive original images, (c) 360 ° of comprehensive column expanded views for gathering;

Figure 10 is the associated diagram based on each module in the household safe and security equipment for solitary old person of omnidirectional computer vision.

(5) embodiment

Below in conjunction with accompanying drawing the present invention is further described.

Embodiment 1

With reference to Fig. 1～10, a kind of household safe and security equipment for solitary old person based on omnidirectional computer vision, comprise microprocessor 6, be used to guard the omnibearing vision sensor 13 of the safe and comfortable situation of old man, described omnibearing vision sensor 13 is connected with microprocessor 6, described omnibearing vision sensor 13 comprises the evagination catadioptric minute surface 1 in order to object in the reflection monitoring field, in order to the dark circles cone 2 that prevents that anaclasis and light are saturated, transparent cylinder 3, camera 5, described evagination catadioptric minute surface 1 is positioned at the top of transparent cylinder 3, evagination catadioptric minute surface 1 down, dark circles cone 2 is fixed on the center of catadioptric minute surface male part, camera 5 facing to evagination mirror surface 1 up, described camera 5 is positioned at the virtual focus position of evagination mirror surface, and camera 5 also comprises camera lens 4;

Microprocessor 6 comprises: view data read module 16 is used to read the video image information of coming from the omnibearing vision sensor biography, and image is carried out preliminary treatment; Image data file memory module 18, the video image information that is used for reading is kept at memory cell by file mode; Omnibearing vision sensor demarcating module 17 is used for the parameter of omnibearing vision sensor is demarcated, and sets up the material picture in space and the corresponding relation of the video image that is obtained; Image launches processing module 19, and the circular video image that is used for reading expands into the panorama block diagram;

Motion obj ect detection module 23, present frame live video image and a relatively stable reference image of being used for being obtained carry out the difference computing, and the computing formula of image subtraction is represented suc as formula (28):

f _d(X，t ₀，t _i)＝f(X，t _i)-f(X，t ₀) (28)

In the following formula, f _d(X, t ₀, t _i) be to photograph the result who carries out image subtraction between image and reference image in real time; F (X, t _i) be to photograph image in real time; F (X, t ₀) be the reference image;

And with in the present image with the image subtraction computing formula of adjacent K frame shown in (29):

f _d(X，t _i-k，t _i)＝f(X，t _i)-f(X，t _i-k) (29)

In the following formula, f _d(X, t _I-k, t _i) be to photograph the result who carries out image subtraction between image and adjacent K two field picture in real time; F (X, t _I-k) image when being adjacent K frame;

As f _d(X, t ₀, t _i) 〉=threshold value, f _d(X, t _I-k, t _iWhen) 〉=threshold value is set up, be judged to be the motion object;

As f _d(X, t ₀, t _i) 〉=threshold value, f _d(X, t _I-k, t _i)＜threshold value is judged stationary objects, and upgrades replacement reference image with formula (30):

$f(X,t_0)\&DoubleLeftArrow;f(X,t_{i-k})---\left(30\right)$

As f _d(X, t ₀, t _i)＜threshold value is judged to be stationary objects;

The connected region computing module, be used for present image is carried out mark, pixel grey scale is that no old man's activity in this sub-district is represented in 0 sub-district, pixel grey scale is that 1 this sub-district of expression has old man's activity, whether the pixel of calculating in the present image equates with the pixel of some points adjacent around the current pixel, equate to be judged as gray scale and have connectedness, all are had connective pixel as a connected region; And then calculate its area and center of gravity according to the connected region of being tried to achieve; Old man's center of gravity obtains by the X that calculates resulting connected region area Si and this connected region, the accumulation calculated for pixel values of Y direction, and computing formula is calculated by formula (42),

$X_{\mathrm{cg}\left(i\right)}=\frac{\underset{x,y\&Element;S_i}{\mathrm{\&Sigma;}}x}{S_i};Y_{\mathrm{cg}\left(i\right)}=\frac{\underset{x,y\&Element;S_i}{\mathrm{\&Sigma;}}y}{S_i}---\left(42\right);$

The acquisition module 25 of old man's activities of daily living data is used for movable time started, perdurabgility and the location in space in the monitoring visual range according to the old man, obtains the rule of old man's activities of daily living;

Old man's activities of daily living model module 26, be used for having grasped after old man's the rule of daily life by the method for self study, can summarize old man's activities of daily living model by the summary of this rule, adopt two-dimentional Gaussian distribution model to describe the room and time distributed model of the activity of certain old man's daily life:

σ in the formula (32) _xThe mathematics variance of brightness value on the x axle of expression connected region, σ _yThe mathematics variance of brightness value on the y axle of expression connected region, μ _xThe mathematic expectaion mean value of brightness value on the x axle of expression connected region, μ _yThe mathematic expectaion mean value of brightness value on the y axle of expression connected region;

The renewal of model uses the digital low-pass filtering method of formula (33)～(36) to carry out,

μ _xnew＝(1-k ₁)μ _xold+k ₁(x-μ _xold)，(33)

μ _ynew＝(1-k ₂)μ _yold+k ₂(y-μ _yold)，(34)

${\mathrm{\&sigma;}}_{\mathrm{xnew}}^2=(1-k_3){\mathrm{\&sigma;}}_{\mathrm{xold}}^2+k_3{(x-{\mathrm{\&mu;}}_{\mathrm{xold}})}^2,---\left(35\right)$

${\mathrm{\&sigma;}}_{\mathrm{ynew}}^2=(1-k_4){\mathrm{\&sigma;}}_{\mathrm{yold}}^2+k_4{(y-{\mathrm{\&mu;}}_{\mathrm{yold}})}^2,---\left(36\right)$

Wherein, ki is a learning rate, and span is [0,1], and learning phase can be selected big learning rate (0.8), accelerates pace of learning; The monitoring stage can be selected little learning rate (0.01);

Fall down unusual detection module 27 in the vision monitoring scope, be used for entering the threshold value that is still in certain position again for a long time behind certain monitoring field and surpasses old man's activities of daily living model module defined and detecting judgement by detecting the old man, the time that detection old man's position of centre of gravity x is judged to inactivity point surpasses threshold value, system-computed following formula so:

PS＝max{p(x|i，μi，∑ _i)，i＝1，...，n}(37)

P in the formula (x|i, μ i, ∑ _i) be the probability that old man's position of centre of gravity x belongs to the gauss hybrid models of the inactivity zone that is numbered i, PS is the maximum probability that position x belongs to certain inactivity zone;

When PS value during less than threshold value, judge that position x does not belong to any known inactivity zones, promptly the old man be still in take place on some positions unusual;

Further, calculate the probability P T that the old man is in the attitude that couches,, use formula (38) to calculate at last and fall down the probability of unusual generation by human body attitude recognition methods based on hidden Markov model from detected image identification old man's attitude:

P＝K1×(1-PS)+K2×PT (38)

Wherein K1, K2 are weights, and PS is that the old man is still in certain locational probability for a long time, and PT is the probability that the old man is in the attitude of lying, and when P surpasses threshold value, judge that the old man falls down unusually;

Fall down unusual prediction module 29 in the non-vision monitoring scope, be used for crossing and predict when old man's situation does not also appear in the certain hour threshold value by vision monitoring wide-ultra, by detect the old man moment t1 leave the monitoring field of vision and institute's elapsed time predict, at first to from old man's activities of daily living model, retrieve the predicted value μ i of this movable duration of corresponding movable duration model, then the real-time following expression formula of calculating of system:

PE＝t-t1-μi (39)

PC＝p(t-t1|i，μi，σi)(40)

Wherein, t is current system time, and t-t1 is the duration of leaving monitoring vision place, and PE is the actual duration in monitoring vision place and the difference of prediction duration value left; PC is the probability that belongs to the Gauss model of movable duration the actual activity time, the promptly actual probability that leaves the monitoring vision place duration;

Current time t, the probability that the old man falls down in non-monitoring vision place is:

P＝K1×PE+K2×(1-PC) (41)

Wherein K1, K2 are weights, and when P surpassed threshold value, the judgement old man may be taken place unusually; The abnormality alarming module is used for notifying the rescue personnel by communication module after judging that the old man may be taken place unusually.

After obtaining comprehensive video information, next carry out background earlier and eliminate and ask the study of the obtaining of activity connected region, old man's activities of daily living data, old man's daily life rule and set up model, unusual evaluation works such as judgement, handling process roughly can be explained by Fig. 5.

Described motion obj ect detection module 23 is mainly extracted by background elimination and target and is realized, it is the problem that brightness changes that background is eliminated the problem that at first will solve, as operating the sudden change of the intensity of illumination that is caused for indoor monitoring meeting owing to turning on light, turning off the light, therefore the background model that adopts in background is eliminated will adapt to these above-mentioned variations.

For video monitoring, because the comprehensive scene visual field is bigger, human body shared ratio in entire image is less, so personage's motion can be similar to and regards rigid motion as; In addition, the scene of video monitoring is fixed, and can think to have the relatively background of fixed range, and the Fast Segmentation Algorithm that therefore can adopt background to cut algorithm is come the motion personage in the real-time detection and tracking video monitoring; Background is eliminated and to be based on background and to cut algorithm and detect the key of motion object, its directly influence detect integrality and accuracy of motion object.Adopted the background adaptive method among the present invention, its core concept is the current mixed number (Xmix that uses 1 group of vector RGB to change to each background pixel, bi) represent the permission value (i is a frame number) of legal background pixel, and adopt IIR filtering that it is carried out following renewal.The background adaptive method is realized in background refresh process module 24;

(1) change (not being that switch lamp causes) naturally when light, and no abnormal object is when existing, 1 group of vector (being respectively RGB) carries out adaptive learning:

X _mix，bn+1(i)＝(1-λ)X _mix，bn(i)+λX _mix，cn(i) (22)

In the formula: X _{Mix, cn}(i) be present frame RGB vector, X _{Mix, bn}(i) be present frame background RGB vector, X _{Mix, bn+1}(i) be next frame background forecast RGB vector, λ is the speed of context update: λ=0, uses changeless background (initial background); Present frame is used as a setting in λ=1; 0＜λ＜1, background is mixed by the background and the present frame of previous moment.

When having sudden change, light (causes) that (2) 1 group of vector is pressed present frame and reset by switch lamp:

X _mix，bn+1(i)＝X _mix，cn(i) (23)

(3) when object entered the monitoring scope, background remained unchanged.For avoiding that the partial pixel study of motion object is background pixel, adopt:

X _mix，bn+1(i)＝X _mix，bn(i) (24)

X in the following formula _{Mix, bn+1}(i) (i=1,2,3) represent R respectively, G, and B3 component, for simplicity, above-mentioned formula has omitted coordinate (x, y) part of each pixel.

Can be used to judge what whether detected motion object caused because of switch lamp for the variation of indoor monitoring background luminance, background luminance uses average background brightness Yb to measure, and computing formula is provided by formula (25),

${\stackrel{\&OverBar;}{Y}}_b=\frac{\underset{x=0}{\overset{W-1}{\mathrm{\&Sigma;}}}\underset{y=0}{\overset{H-1}{\mathrm{\&Sigma;}}}{Y}_n(x,y)(1-M_n(x,y))}{\underset{x=0}{\overset{W-1}{\mathrm{\&Sigma;}}}\underset{y=0}{\overset{H-1}{\mathrm{\&Sigma;}}}(1-M_n(x,y))}---\left(25\right)$

In the formula (25), (x y) is the brightness of each pixel of present frame to Yn, and (x y) is the mask table of present frame to Mn.The background luminance of former frame when representing to find moving object is arranged with Yb0, the background luminance of first frame when Yb1 represents to detect moving object, being changed to of two frame mean flow rates:

ΔY＝Yb1-Yb0 (26)

If Δ Y is greater than certain value then think the incident of turning on light that taken place, if Δ Y is less than certain negative value then think the incident of turning off the light that taken place.Present frame is reset with formula (23) according to above-mentioned judged result.

Described mask table is to write down each pixel with one with the measure-alike array M of frame of video whether motion change is arranged, and this array is called mask mapping table (Mask Map):

Array M is the bianry image of motion object, is partitioned into the motion object thereby not only can be used to the mask frame of video, also can be used for tracking, analysis and the classification of motion object.

Described background cuts algorithm and is also referred to as difference method, is a kind of image processing method that is usually used in detected image variation and moving object. detects those pixel portion that have light source point to exist according to the correspondence relation of three dimensions and image pixel; A more stable reference image at first will be arranged; And this reference image is stored in the memory of computer; And by above-mentioned Adaptive background subtraction method the reference image is dynamically updated; Carry out image subtraction by photographing in real time between image and this reference image; The regional luminance that the result who subtracts each other changes strengthens; The computing formula of image subtraction represents suc as formula (28)

f _d(X，t ₀，t _i)＝f(X，t _i)-f(X，t ₀) (28)

F in the formula _d(X, t ₀, t _i) be to photograph the result who carries out image subtraction between image and reference image in real time; F (X, t _i) be to photograph image in real time, be equivalent to the X in the formula (22) _{Mix, cn}(i); F (X, t ₀) be the reference image, be equivalent to the X in the formula (22) _{Mix, bn}(i).Background cuts algorithm and realizes in moving region detection module 23.

Because the omnibearing vision sensor in the video monitoring is all fixed, and the stationary objects in the background may be moved sometimes, cut algorithm based on background and to detect the resulting motion pixel of motion object and may comprise object and move the hole that stays.Because the hole can not moved in frame of video subsequently, therefore available adjacent K frame difference method is eliminated the hole, adopts adjacent K frame difference method to judge whether certain pixel is the hole that background object stays among the present invention.Need to carry out the calculating of formula (29) for this reason,

f _d(X，t _i-k，t _i)＝f(X，t _i)-f(X，t _i-k) (29)

Moving in the unit that generally can consider to divide in the time of stationary objects worked as f _d(X, t ₀, t _i) 〉=threshold value and f _d(X, t _I-k, t _iWhen) 〉=threshold value is all set up, be considered to the motion object; If f _d(X, t ₀, t _i) 〉=threshold value and f _d(X, t _I-k, t _i)＜threshold value thinks among the present invention that the stationary objects in the background is moved the hole that the back is produced, and upgrades replacement reference image in order to eliminate the hole with formula (30),

$f(X,t_0)\&DoubleLeftArrow;f(X,t_{i-k})---\left(30\right)$

Include noise in the actual image signal, and generally all show as high-frequency signal, therefore in identifying, will reject the image border point that produces by noise.

Described rejecting is by image border point that noise produced, use the method for neighbours territory traversal in the present invention, the value that the average gray value of pixel removes each pixel of alternate image in the neighborhood that it is determined with the filtering mask, be of the average displacement of each pixel value with all values in its local neighborhood, shown in formula (31):

h[i，j]＝(1/M)∑f[k，1] (31)

In the formula, M is the pixel sum in the neighborhood, is taken as 4 among the present invention.

Connectedness between pixel is to determine a key concept in zone.In two dimensional image, the individual adjacent pixels of m (m＜=8) is arranged around the hypothetical target pixel, if this pixel grey scale equate with the gray scale of some some A in this m pixel, claim this pixel so and put A to have connectedness.Connectedness commonly used has 4 connected sums 8 to be communicated with.4 are communicated with four points in upper and lower, left and right of generally choosing object pixel.8 are communicated with and then choose object pixel all neighbor in two-dimensional space.All are had connective pixel then constituted a connected region as a zone.

Described connected region is calculated and is mainly solved in image processing process, a width of cloth bianry image, and its background and target have gray value 0 and 1 respectively.We are that this sub-district attonity object is represented in 0 sub-district with pixel, if there is action object 1 this sub-district of expression.So can adopt connection composition scale notation to carry out the merging of defect area.The connection labeling algorithm can find all the connection compositions in the image, and the institute in the same connection composition is distributed same mark a little.Be the connected region algorithm below,

1) from left to right, scan image from top to bottom;

2) if pixel is 1, then:

If upper point and left side point have a mark, then duplicate this mark.

If have identical mark, duplicate this mark at 2.

If 2 have different marks, then duplicate a little mark and with in two marks input table of equal value as mark of equal value.

Otherwise give the new mark of this picture element distribution and this mark is imported table of equal value.

3) go on foot if need to consider more point then get back to the 2nd.

4) find minimum mark each of equal value concentrating of equivalence table.

5) scan image replaces each mark with the minimum mark in the table of equal value.

Obtaining of described old man's activities of daily living data; because above connected region can be thought the variation space field that produced behind the physical activity; the center of gravity of human body can be thought in the center of connected region, therefore can obtain the time of old man's daily routines and the data in space by these information.Among the present invention omnibearing vision sensor is installed in the main place of old man's daily routines, parlor for example, system just can monitor most of daily routines of old man like this.

As Fig. 6, shown in Figure 10, the learning phase that old man's daily life rule is arranged before system's input, learning phase finishes the back system input monitoring stage, the study of described old man's daily life rule is to obtain the activity data that obtains the old man in 25 in the data in module daily life time and space, set up old man's activities of daily living model in the moment that study finishes according to the above-mentioned a large amount of activity datas that obtain then, described activities of daily living model carries out in module 26; Because each old man's living conditions, daily life habits and customs are not quite similar, after the rule of starting stage, obtain old man's activities of daily living model and can carry out abnormality and unusual judgement with that by automatic study old man daily life, be installed in the parlor situation of the activity centre that can reflect the old man as Fig. 7, omnibearing vision sensor shown in Figure 9, at least will learn the information of the space-time aspect of following 3 old man's daily lifes: 1) place of daily life behavior often appears in the study old man at the parlor back warp, for example the position of the chair of custom seat is referred to as inactivity zones; 2) all gateways in study parlor for example lead to the position of the door in bedroom, are referred to as entry zones; 3) study certain period of old man is left the regularity of duration that the monitoring place enters the activity in other places by certain entry zone, for example the regularity of old man's duration of sleeping in the bedroom.

Described old man's activities of daily living model, device has been grasped by the method for self study after old man's the rule of daily life, can summarize old man's activities of daily living model by the summary of this rule, the video information of coming out according to this model and actual detected compares just can be judged and abnormal conditions such as fall down.Such as for abnormal conditions such as monitoring falling down in the place, can go out great abnormal conditions such as the old man falls down by probability calculation at all known inactivity zones, be judged as if probability is lower than threshold value and fall down or other are great unusual, at this time can confirm by video image; Great unusual for falling down outside the video monitoring place etc., by calculating sometime, the probability of three Gaussian Profile such as a certain orientation (such as the door that is the some discrepancy that communicates with the parlor) and a certain duration, the parameter that to calculate resulting probability and Gaussian distribution model then compares, be judged as if the probability that is calculated is lower than threshold value fall down etc. unusual, especially when seriously surpass predicted value sometime inertia perdurabgility in some spaces of old man, illustrate it very likely is to take action after the old man has fallen down, must provide and nurse timely and succour.

As shown in Figure 7, family for certain old man, the monitoring vision scope in by the parlor and enter bedroom, kitchen, toilet and go out the door form, be numbered 1 the door be the gateway in room, be numbered the bedroom that 2 door connects the parlor, being numbered 3 door is the door in turnover kitchen, and being numbered 4 door is the door that passes in and out the toilet.Omnibearing vision sensor is installed in parlor middle part, and can gather the old man and in the parlor, carry out be connected with the parlor information in space of activities of daily living and discrepancy, and the temporal information of record activity, set up corresponding room and time model.

Fig. 5 is the overall process framework of monitor system, the old solitary people indoor and outdoor motility model of being set up has characteristics such as personalized self study, self adaptation, this model can not only detect unusual in the video monitoring scope simultaneously, and can predict the contingent unusual of (comprising the open air) beyond the video monitoring scope.The device operation begins to need a learning phase, obtains rule each old solitary people, that can reflect the daily routines of its individual character by self-learning method; Learning phase enters the monitoring stage after finishing, because old man's age growth and changes of seasons require system can constantly be updated in the resultant old man's activities of daily living of learning phase model, makes it to have adaptive function.

In the study and the understanding stage of image, device can be followed the tracks of the monitoring old man by computer vision, calculate the movement locus of old man's human body center, going out the now and the time of staying in certain monitoring space, going out the now and the time of staying can obtain by the system for computer time, and the information of old man in certain locus will obtain by the video information (connected region of aforementioned calculation gained) that omnibearing vision sensor obtained, and this processing procedure can be described with Fig. 5.

Described modeling processing procedure can be described with mathematical way, the room and time probability data of old man's activities of daily living approaches normal distribution, therefore uses Gaussian distribution model to describe the room and time distributed model of the activity of certain old man's daily life in this patent.Because the rule of old solitary people may slowly change with changes of seasons with advancing age, for adaptively changing old man's daily life time and spatial model, uses the low-pass filtering method to upgrade the parameter of Gaussian distribution model in the present invention.

For spatial model, this paper uses two-dimentional Gaussian distribution model:

The renewal of model uses the digital low-pass filtering method of formula (33)～(36) to carry out,

μ _xnew＝(1-k ₁)μ _xold+k ₁(x-μ _xold)，(33)

μ _ynew＝(1-k ₂)μ _yold+k ₂(y-μ _yold)，(34)

${\mathrm{\&sigma;}}_{\mathrm{xnew}}^2=(1-k_3){\mathrm{\&sigma;}}_{\mathrm{xold}}^2+k_3{(x-{\mathrm{\&mu;}}_{\mathrm{xold}})}^2,---\left(35\right)$

${\mathrm{\&sigma;}}_{\mathrm{ynew}}^2=(1-k_4){\mathrm{\&sigma;}}_{\mathrm{yold}}^2+k_4{(y-{\mathrm{\&mu;}}_{\mathrm{yold}})}^2,---\left(36\right)$

Wherein, ki is a learning rate, and span is [0,1].Learning phase can be selected big learning rate (0.8), accelerates pace of learning; The monitoring stage can be selected little learning rate (0.01), can keep the stability of model, can catch the minor variations of model again, has adaptive characteristics.

Fall down unusual detection in the described vision monitoring scope, in Figure 10, carry out in life event processing module 27 and the unusual judge module 29, after entering certain monitoring field, the old man is still in certain position by detecting again for a long time in the present invention, be the time that old man's position of centre of gravity x is judged to inactivity point to surpass threshold value, system-computed following formula so:

PS＝max{p(x|i，μi，∑ _i)，i＝1，...，n} (37)

P in the formula (x|i, μ i, ∑ _i) be the probability that old man's position of centre of gravity x belongs to the gauss hybrid models of the inactivity zone that is numbered i, PS is the maximum probability that position x belongs to certain inactivity zone.

When PS value during less than threshold value, just think that position x does not belong to any known inactivity zones, promptly the old man be still in take place on some positions unusual.In order to judge that further the old man falls down the possibility of unusual generation, further, calculate the probability P T that the old man is in the attitude that couches from detected image identification old man's attitude, can be by human body attitude recognition methods based on hidden Markov model.Use formula (38) to calculate at last and fall down the probability of unusual generation:

P＝K1×(1-PS)+K2×PT (38)

Wherein K1, K2 are weights, and PS is that the old man is still in certain locational probability for a long time, and PT is the probability that the old man is in the attitude of lying.When P surpasses threshold value, system is judged as the generation old man and falls down unusually, system obtains old man's home address and tutorial contact method from user basic information 30, and the abnormal class combination judged of coupling system is by alarm module 31 person that sends to the long distance monitoring, require the long distance monitoring person to check that by video server the image at old man's life scene further confirms old man's Ankang, so as to take instantaneity, rescuing of being fit to.

Described non-vision is guarded the unusual prediction of falling down in the scope, and the prediction judgement is being handled in unusual judging treatmenting module 29; Be to cross by vision monitoring wide-ultra to predict when old man's situation does not also appear in certain hour, with a relatively popular words expression, promptly detect this old man that but do not detect who old man's figure occurs and occur, in the present invention by detect the old man moment t1 by the entry zone that is numbered i leave the monitoring field of vision and institute's elapsed time predict, at first to from old man's activities of daily living model, retrieve the predicted value μ i of this movable duration of corresponding movable duration model, then the real-time following expression formula of calculating of system:

PE＝t-t1-μi (39)

PC＝p(t-t1|i，μi，σi) (40)

Wherein t is current system time, and t-t1 is the duration of leaving monitoring vision place, so PE is the actual duration in monitoring vision place and the difference of prediction duration value left; PC is the probability that belongs to the Gauss model of movable duration the actual activity time, the promptly actual probability that leaves the monitoring vision place duration.

Current time t, the probability that the old man falls down in non-monitoring vision place is:

P＝K1×PE+K2×(1-PC) (41)

Wherein K1, K2 are weights.When P surpassed threshold value, system just sends the old man from trend long distance monitoring person, and abnormal information may take place, and the person in time confirms the safe and comfortable information of old man to require the long distance monitoring; Along with the increase of P value, the possibility of abnormal condition takes place also along with increase in the old man, and the person confirms old man's Ankang by various means at this moment to require the long distance monitoring, so that take instantaneity, suitable rescuing.From formula (39)～(41) as can be seen, abnormal condition takes place beyond monitoring field of vision institute the old man is that to leave the probability distribution value in the elapsed time in monitoring vision place at ordinary times with the old man in certain time be basis for estimation, checks whether leave the time that monitoring vision place experiences specifically exceeds the prediction threshold value.

It is unusual to find and predict that by the variation on time and space of omni-directional visual picture catching old man rule of life the abnormality of old man's life reaches, in invention, set up old man's indoor and outdoor motility model unusual in can not only identification video monitoring scope, and can predict the contingent unusual of (comprising the open air) beyond the video monitoring scope, for the old solitary people long distance monitoring provides a kind of new method, the quality of the life that improves old solitary people there is positive effect, has bigger implementary value and economic results in society.

Claims (6)

1, a kind of household safe and security equipment for solitary old person based on omnidirectional computer vision, it is characterized in that: described household safe and security equipment for solitary old person comprises microprocessor, be used to guard the omnibearing vision sensor of the safe and comfortable situation of old man, described omnibearing vision sensor is connected with described microprocessor, described omnibearing vision sensor comprises the evagination catadioptric minute surface in order to object in the reflection monitoring field, in order to the dark circles cone that prevents that anaclasis and light are saturated, transparent cylinder, camera, described evagination catadioptric minute surface is positioned at the top of described transparent cylinder, described evagination catadioptric minute surface down, described dark circles cone is fixed on the center of described evagination catadioptric minute surface male part, described camera faces toward described evagination mirror surface up, and described camera is positioned at the virtual focus position of described evagination mirror surface;

Described microprocessor also comprises:

The view data read module is used to read the video image information of coming from described omnibearing vision sensor biography;

The image data file memory module, the video image information that is used for reading is kept at memory cell by file mode;

The omnibearing vision sensor demarcating module is used for the parameter of described omnibearing vision sensor is demarcated, and sets up the material picture in space and the corresponding relation of the video image that is obtained;

Image launches processing module, and the circular video image that is used for reading expands into the panorama block diagram;

The motion obj ect detection module, present frame live video image and a relatively stable reference image of being used for being obtained carry out the difference computing, and the computing formula of image subtraction is represented suc as formula (28):

f _d(X，t ₀，t _i)＝f(X，t _i)-f(X，t ₀) (28)

In the following formula, f _d(X, t ₀, t _i) be to photograph the result who carries out image subtraction between image and reference image in real time; F (X, t _i) be to photograph image in real time; F (X, t ₀) be the reference image;

And with in the present image with the image subtraction computing formula of adjacent K frame shown in (29):

f _d(X，t _i-k，t _i)＝f(X，t ₁)-f(X，t _i-k) (29)

In the following formula, f _d(X, t _I-k, t _i) be to photograph the result who carries out image subtraction between image and adjacent K two field picture in real time; F (X, t _I-k) image when being adjacent K frame;

As f _d(X, t ₀, t _i) 〉=threshold value, f _d(X, t _I-k, t _iWhen) 〉=threshold value is set up, be judged to be the motion object;

As f _d(X, t ₀, t _i) 〉=threshold value, f _d(X, t _1-k, t _i)＜threshold value is judged stationary objects, and upgrades replacement reference image with formula (30):

$f(X,t_0)\&DoubleLeftArrow;f(X,t_{i-k})---\left(30\right)$

As f _d(X, t ₀, t _i)＜threshold value is judged to be stationary objects;

The connected region computing module, be used for present image is carried out mark, pixel grey scale is that no old man's activity in this sub-district is represented in 0 sub-district, pixel grey scale is that 1 this sub-district of expression has old man's activity, whether the pixel of calculating in the present image equates with the pixel of some points adjacent around the current pixel, equate to be judged as gray scale and have connectedness, all are had connective pixel as a connected region; And then calculate its area and center of gravity according to the connected region of being tried to achieve; Old man's center of gravity obtains by the X that calculates resulting connected region area Si and this connected region, the accumulation calculated for pixel values of Y direction, and computing formula is calculated by formula (42),

$X_{\mathrm{cg}\left(i\right)}=\frac{\underset{x,y\&Element;S_i}{\mathrm{\&Sigma;}}x}{S_i};$ $Y_{\mathrm{cg}\left(i\right)}=\frac{\underset{x,y\&Element;S_i}{\mathrm{\&Sigma;}}y}{S_i}---\left(42\right);$

In the formula (42),

Represent the accumulation pixel value of connected region area Si in X-direction,

Expression connected region area Si is at the accumulation pixel value of Y direction, and Si is the connected region area;

The acquisition module of old man's activities of daily living data is used for movable time started, perdurabgility and the location in space in the monitoring visual range according to the old man, obtains the rule of old man's activities of daily living;

Old man's activities of daily living model module, be used for having grasped after old man's the rule of daily life by the method for self study, can summarize old man's activities of daily living model by the summary of this rule, adopt two-dimentional Gaussian distribution model to describe the room and time distributed model of the activity of certain old man's daily life:

σ in the formula (32) _xThe mathematics variance of brightness value on the x axle of expression connected region, σ _yThe mathematics variance of brightness value on the y axle of expression connected region, μ _xThe mathematic expectaion mean value of brightness value on the x axle of expression connected region, μ _yThe mathematic expectaion mean value of brightness value on the y axle of expression connected region;

The renewal of model uses the digital low-pass filtering method of formula (33)～(36) to carry out,

μ _xnew＝(1-k ₁)μ _xold+k ₁(x-μ _xold)， (33)

μ _ynew＝(1-k ₂)μ _yold+k ₂(y-μ _yold)， (34)

${\mathrm{\&sigma;}}_{\mathrm{xnew}}^2=(1-k_3){\mathrm{\&sigma;}}_{\mathrm{xold}}^2+k_3{(x-{\mathrm{\&mu;}}_{\mathrm{xold}})}^2,---\left(35\right)$

${\mathrm{\&sigma;}}_{\mathrm{ynew}}^2=(1-k_4){\mathrm{\&sigma;}}_{\mathrm{yold}}^2+k_4{(y-{\mathrm{\&mu;}}_{\mathrm{yold}})}^2,---\left(36\right)$

Wherein, ki is a learning rate, and span is [0,1], and learning phase can be selected big learning rate, accelerates pace of learning; The monitoring stage can be selected little learning rate;

Fall down unusual detection module in the vision monitoring scope, be used for entering the threshold value that is still in certain position again for a long time behind certain monitoring field and surpasses old man's activities of daily living model module defined and detecting judgement, detect old man's position of centre of gravity x by detecting the old man _pThe time that is judged to non-moving point surpasses threshold value, system-computed following formula so:

PS＝max{p(x _p|i，μi，∑ _i)，i＝1，...，n} (37)

P in the formula (x|i, μ i, ∑ _i) be old man's position of centre of gravity x _pThe probability that belongs to the gauss hybrid models of the non-zone of action that is numbered i, PS are position x _pThe maximum probability that belongs to certain non-zone of action;

When PS value during, judge position x less than threshold value _pDo not belong to any known non-zone of action, promptly the old man is still in and takes place on some positions unusually;

Further, calculate the probability P T that the old man is in the attitude that couches,, use formula (38) to calculate at last and fall down the probability of unusual generation by human body attitude recognition methods based on hidden Markov model from detected image identification old man's attitude:

P＝K1×(1-PS)+K2×PT (38)

Wherein K1, K2 are weights, and PS is that the old man is still in certain locational probability for a long time, and PT is the probability that the old man is in the attitude of lying, and when P surpasses threshold value, judge that the old man falls down unusually;

Non-vision is guarded the unusual prediction module of falling down in the scope, be used for crossing and predict when old man's situation does not also appear in the certain hour threshold value by vision monitoring wide-ultra, by detect the old man moment t1 leave the monitoring field of vision and institute's elapsed time predict, at first to from old man's activities of daily living model, retrieve the predicted value μ i of this movable duration of corresponding movable duration model, then the real-time following expression formula of calculating of system:

PE＝t-t1-μi (39)

PC＝p(t-t1|i，μi，σi) (40)

Wherein, t is current system time, and t-t1 is the duration of leaving monitoring vision place, and PE is the actual duration in monitoring vision place and the difference of prediction duration value left; PC is the probability that belongs to the Gauss model of movable duration the actual activity time, the promptly actual probability that leaves the monitoring vision place duration;

Current time t, the probability that the old man falls down in non-monitoring vision place is:

P＝K1×PE+K2×(1-PC) (41)

Wherein K1, K2 are weights, and when P surpassed threshold value, the judgement old man may be taken place unusually;

The abnormality alarming module is used for notifying the rescue personnel by communication module after judging that the old man may be taken place unusually.

2, the household safe and security equipment for solitary old person based on omnidirectional computer vision as claimed in claim 1, it is characterized in that: described microprocessor also comprises the background maintenance module, described background maintenance module comprises:

The background luminance computing unit is used to calculate average background brightness Yb computing formula as the formula (25):

${\stackrel{\&OverBar;}{Y}}_b=\frac{\underset{x=0}{\overset{W-1}{\mathrm{\&Sigma;}}}\underset{y=0}{\overset{H-1}{\mathrm{\&Sigma;}}}{Y}_n(x,y)(1-M_n(x,y))}{\underset{x=0}{\overset{W-1}{\mathrm{\&Sigma;}}}\underset{y=0}{\overset{H-1}{\mathrm{\&Sigma;}}}(1-M_n(x,y))}---\left(25\right)$

In the formula (25), Yn (x y) is the brightness of each pixel of present frame, Mn (x y) is the mask table of present frame, and described mask table is to write down each pixel with one with the measure-alike array M of frame of video whether motion change is arranged, referring to formula (27):

Yb0 is the background luminance of former frame when being judged to be the motion object, and Yb1 is when being judged to be the motion object

The background luminance of first frame, being changed to of two frame mean flow rates:

ΔY＝Yb1-Yb0 (26)

If Δ Y, then thinks the incident of turning on light that taken place greater than higher limit; If Δ Y, then thinks the incident of turning off the light that taken place less than certain lower limit; Between higher limit and lower limit, think then that light changes naturally as Δ Y;

The background adaptive unit is used for carrying out adaptive learning according to following formula (22) when light changes naturally:

X _mix，bn+1(i)＝(1-λ)X _mix，bn(i)+λX _mix，cn(i) (22)

In the formula: X _{Mix, cn}(i) be each color vector in the present frame RGB color space, X _{Mix, bn}(i) be each color vector in the present frame background color space, X _{Mix, bn+1}(i) be each color vector amount in the next frame background forecast color space, λ is the speed of context update; Changeless background is used in λ=0; Present frame is used as a setting in λ=1; 0＜λ＜1, background is mixed by the background and the present frame of previous moment;

When light is caused that by switch lamp background pixel is reset according to present frame, referring to formula (23):

X _mix，bn+1(i)＝X _mix，cn(i) (23)。

3, the household safe and security equipment for solitary old person based on omnidirectional computer vision as claimed in claim 2, it is characterized in that: described microprocessor also comprises:

Noise is rejected module, is used for the average displacement of each pixel value with all values in its local neighborhood, as shown in Equation (16):

h[i，j]＝(1/M)∑f[k，1] (32)

In the following formula (32), M is the pixel sum in the neighborhood; H[i, j] certain pixel in the presentation video, f[k, 1] described certain pixel adjacent pixels point in expression and the image.

4, the household safe and security equipment for solitary old person based on omnidirectional computer vision as claimed in claim 3 is characterized in that: described image launches processing module, is used for according to a point (x on the circular omnidirectional images ^*, y ^*) and rectangle column panorama sketch on a point (x ^*, y ^*) corresponding relation, set up (x ^*, y ^*) and (x ^*, y ^*) mapping matrix, shown in the formula (21):

P ^**(x ^**，y ^**)←M×P ^*(x ^*，y ^*) (21)

In the following formula, M is a mapping matrix, P ^*(x ^*, y ^*) be the picture element matrix on the circular omnidirectional images, P ^*(x ^*, y ^*) be the picture element matrix on the rectangle column panorama sketch.

5, the household safe and security equipment for solitary old person based on omnidirectional computer vision as claimed in claim 4, it is characterized in that: described microprocessor also comprises:

Network transmission module, the live video image that is used for being obtained is gone out by Network Transmission in the mode of video flowing, so that the user can grasp field condition in real time by diverse network;

Real-time playing module, the live video image that is used for being obtained is played to display device by this module.

6, the household safe and security equipment for solitary old person based on omnidirectional computer vision as claimed in claim 5 is characterized in that: described omnibearing vision sensor is installed in the zone line in the main activities place that can guard the daily activity of people until old.

Images