CN102880873B - Personnel behavior identification implementation system and method based on image segmentation and semantic extraction - Google Patents
Personnel behavior identification implementation system and method based on image segmentation and semantic extraction Download PDFInfo
- Publication number
- CN102880873B CN102880873B CN201210317234.XA CN201210317234A CN102880873B CN 102880873 B CN102880873 B CN 102880873B CN 201210317234 A CN201210317234 A CN 201210317234A CN 102880873 B CN102880873 B CN 102880873B
- Authority
- CN
- China
- Prior art keywords
- image
- subelement
- human behavior
- semantic
- segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Image Analysis (AREA)
Abstract
The invention relates to a personnel behavior identification and detection implementation system and method based on image segmentation and semantic characteristic extraction. The system comprises an image acquisition unit, a personnel behavior detection host computer, a user inquiry unit and an output interface unit. The method comprises the following steps: the personnel behavior detection host computer identifies personnel behaviors in the image data acquired by the image acquisition unit through image segmentation and image semantic characteristic extraction so as to generate personnel behavior presentation information. In the method, the personnel behavior detection host computer maps low-level features of an image into high-level semantics through a support vector machine so as to establish a mapping relation between the image and the image description, so that the content in the picture can be comprehended through digital image processing and analysis, the behaviors of personnel in the scene can be intelligently detected, and the identification accuracy of personnel behaviors in the image can be greatly improved. The system provided by the invention has a simple structure, and the method provided by the invention is simple and convenient to implement, low in application cost and wider in application range.
Description
Technical field
The present invention relates to technical field of image information processing, particularly image information intelligent identification technology field, specifically refer to a kind of system and method realizing human behavior identification based on Iamge Segmentation and extraction of semantics.
Background technology
Along with the fast development of computer technology and multimedia technology, the quantity of image also greatly increases, from the image library of magnanimity, how to detect that required image becomes the hot issue studied in current multimedia technology fast and accurately.Traditional text based image retrieval technologies needs keeper to mark image by hand, not only consumes a large amount of manpowers, and the subjectivity of artificial mark image is very large, and for different keepers, the possibility of result of mark is different.
CBIR method (Content-Based Image Retrieval, CBIR) is arisen at the historic moment, and particularly the image, semantic feature of image content-based becomes new study hotspot.Image, semantic is divided into three levels according to complexity: ground floor is Feature Semantics layer.Relevant semantic description is extracted by bottom visual signature such as color, texture and shape etc. and the combination thereof of image; The second layer is Object Semanteme layer.Find out the objectives object in image and relation each other thereof by identification and reasoning, then provide semantic meaning representation; Third layer is abstract semantics layer.The object comprised by image, the implication of scene and target carry out high-rise reasoning, obtain the semantic description of being correlated with.The semanteme of this level relates generally to the semantic and emotional semantic of the Scene Semantics of image, behavior.CBIR technology relies on the Low Level Vision feature (color, texture, shape etc.) of image to carry out retrieving, but the understanding of people to image is a process utilizing the priori reasoning image, semantic of oneself, this results in " semantic gap " between the bottom visual signature of image and image, semantic, image, semantic and actual the expression between implication of image is caused to there is distance thus, recognition accuracy is low, and then causes CBIR method efficiency lower.
Summary of the invention
The object of the invention is to overcome above-mentioned shortcoming of the prior art, there is provided a kind of the high-level semantic of image and bottom visual signature are combined, by support vector machine (SVM), the low-level image feature of image is mapped as high-level semantic, mapping relations are set up between image and iamge description, thus enable computing machine by Digital Image Processing and analyze the content understood in picture, when not needing human intervention, realize the Intelligent Measurement to human behavior in scene, reduce " semantic gap ", thus utilize video monitoring image treatment technology from a large amount of video datas, extract user to need a small amount of video information, significantly improve the accuracy of human behavior identification in image, and implementation is easy, application cost is cheap, range of application also realizes the system and method for human behavior recognition detection comparatively widely based on Iamge Segmentation and semantic feature extraction.
In order to realize above-mentioned object, the system realizing human behavior recognition detection based on Iamge Segmentation and semantic feature extraction of the present invention has following formation:
This system comprises: image acquisition units, human behavior detect host computer, user's query unit and output interface unit.Wherein, image acquisition units is in order to acquisition of image data; Human behavior detects host computer and is connected to described image acquisition units, in order to identify for the human behavior in described view data, and produces corresponding human behavior representation manners by Iamge Segmentation and image, semantic feature extraction; User's query unit is connected to described human behavior and detects host computer, the inquiry carrying out for described human behavior representation manners in order to provide user; Output interface unit is in order to externally connection device or network provide described human behavior representation manners.
Should realize in the system of human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described human behavior detects host computer and comprises: image segmentation unit and semantic feature extraction unit.Wherein image segmentation unit is connected to described image acquisition units, in order to described view data is divided into sub-image data; Semantic feature extraction unit is connected to described image segmentation unit, user's query unit and output interface unit, in order to identify the human behavior in described sub-image data and to describe, and the human behavior representation manners described in generation.
Should realize in the system of human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described image segmentation unit comprises: Frame sampling subelement, smoothing processing subelement, wavelet transformation subelement and Threshold segmentation subelement.Wherein, the image acquisition units described in Frame sampling subelement connects, carries out Frame sampling to gathered view data; Frame sampling subelement described in smoothing processing subelement connects, to the smoothing process of Frame image that sampling obtains; Smoothing processing subelement described in wavelet transformation subelement connects, carries out wavelet transform process to the image through smoothing processing; Wavelet transformation subelement described in Threshold segmentation subelement connects and described semantic feature extraction unit, carry out Threshold segmentation to the image through wavelet transform process, the sub-image data described in generation.
Should realize in the system of human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described image segmentation unit also comprises: bottom Visual Feature Retrieval Process subelement, this bottom Visual Feature Retrieval Process subelement is connected between described Threshold segmentation subelement and described semantic feature extraction unit, in order to extract the bottom visual signature of described subimage, described bottom visual signature comprises color characteristic, textural characteristics, shape facility and locus feature.
Should realize in the system of human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described semantic feature extraction unit comprises: standard picture semantic knowledge-base, high-level semantics features map subelement and behavior description subelement.Wherein standard picture semantic knowledge-base is in order to store behavior template image and corresponding semantic information; High-level semantics features maps subelement and is connected to described image segmentation unit, in order to map the high-level semantics features producing support vector machine according to described sub-image data; Behavior description subelement connects described high-level semantics features respectively and maps subelement and described standard picture semantic knowledge-base, mate with the behaviour template image in described standard picture semantic knowledge-base according to described high-level semantics features, obtain corresponding semantic information, the human behavior representation manners described in generation.
Should realize in the system of human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described standard picture semantic knowledge-base comprises: behaviour template storehouse and semantic knowledge-base; Wherein, behaviour template storehouse is in order to store behavior Template Information; Semantic knowledge-base is in order to store the semantic knowledge information corresponding with described behaviour template information; Described behavior description subelement comprises: Activity recognition functional unit and behavior extraction of semantics functional unit, wherein Activity recognition functional unit, connect described high-level semantics features respectively and map subelement and described behaviour template storehouse, in order to according to the behaviour template information matches in described high-level semantics features and described behaviour template storehouse, produce the image, semantic feature of classifying based on tree-like vector machine; Behavior extraction of semantics functional unit then connects described Activity recognition functional unit and described semantic knowledge-base respectively, in order to according to the described image, semantic feature of classifying based on tree-like vector machine, obtain corresponding semantic knowledge information from described semantic knowledge-base, produce sing on web Ontology Language structurized iamge description high-level semantics data as described human behavior representation manners.
Should realize in the system of human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described output interface unit is USB interface.
The present invention also provides a kind of method utilizing described system to realize carrying out based on Iamge Segmentation and semantic feature extraction human behavior recognition detection, and the method comprises the following steps:
(1) the image acquisition units acquisition of image data described in;
(2) human behavior described in is detected host computer and is identified the human behavior in described view data by Iamge Segmentation and image, semantic feature extraction, and produces corresponding human behavior representation manners;
(3) user is inquired about for described human behavior representation manners by described user's query unit;
(4) by described output interface unit, externally connection device or network provide described human behavior representation manners to system.
Should carry out in the method for human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described human behavior detects host computer and comprises image segmentation unit and semantic feature extraction unit; Described image segmentation unit is connected to described image acquisition units; Described semantic feature extraction unit is connected to described image segmentation unit, user's query unit and output interface unit; Described step (2) specifically comprises the following steps:
(21) described view data is divided into sub-image data by the image segmentation unit described in;
(22) the semantic feature extraction unit described in identifies the human behavior in described sub-image data and describes, the human behavior representation manners described in generation.
Should carry out in the method for human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described image segmentation unit comprises: Frame sampling subelement, smoothing processing subelement, wavelet transformation subelement and Threshold segmentation subelement, and described step (21) specifically comprises the following steps:
(21-1) the Frame sampling subelement described in carries out Frame sampling to gathered view data;
(21-2) the smoothing process of Frame image that the smoothing processing subelement described in obtains sampling;
(21-3) the wavelet transformation subelement described in carries out wavelet transform process to the image through smoothing processing;
(21-4) the Threshold segmentation subelement described in carries out Threshold segmentation to the image through wavelet transform process, the sub-image data described in generation.
Should carry out in the method for human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described image segmentation unit also comprises the bottom Visual Feature Retrieval Process subelement be connected between described Threshold segmentation subelement and described semantic feature extraction unit, and described step (21) is further comprising the steps of after described step (21-4):
(21-5) the bottom Visual Feature Retrieval Process subelement described in extracts the bottom visual signature of described subimage, and described bottom visual signature comprises color characteristic, textural characteristics, shape facility and locus feature.
Should carry out in the method for human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described semantic feature extraction unit comprises: standard picture semantic knowledge-base, high-level semantics features map subelement and behavior description subelement, and described step (22) specifically comprises the following steps:
(22-1) high-level semantics features described in maps subelement and maps the high-level semantics features producing support vector machine according to described sub-image data;
(22-2) the behavior description subelement described in mates with the behaviour template image in described standard picture semantic knowledge-base according to described high-level semantics features, obtains corresponding semantic information, the human behavior representation manners described in generation.
Should carry out in the method for human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described standard picture semantic knowledge-base comprises behaviour template storehouse and semantic knowledge-base; Described behavior description subelement comprises: Activity recognition functional unit and behavior extraction of semantics functional unit; Described step (22-2) specifically comprises the following steps:
(22-2a) the Activity recognition functional unit described in, by the behaviour template information matches in described high-level semantics features and described behaviour template storehouse, produces the image, semantic feature of classifying based on tree-like vector machine;
(22-2b) the behavior extraction of semantics functional unit described in obtains the semantic knowledge information corresponding with described image, semantic feature of classifying based on tree-like vector machine from described semantic knowledge-base, produce sing on web Ontology Language structurized iamge description high-level semantics data, and using these iamge description high-level semantics data as described human behavior representation manners.
Have employed the system and method realizing human behavior recognition detection based on Iamge Segmentation and semantic feature extraction of this invention, its system comprises: image acquisition units, human behavior detect host computer, user's query unit and output interface unit.In the method, human behavior detection host computer is identified the human behavior in the view data of image acquisition units collection by Iamge Segmentation and image, semantic feature extraction, and produce corresponding human behavior representation manners, be supplied to user and undertaken inquiring about or being exported by output interface unit external device or network by user's query unit.Human behavior detects host computer, by support vector machine (SVM), the low-level image feature of image is mapped as high-level semantic, mapping relations are set up between image and iamge description, enable computing machine by Digital Image Processing and analyze the content understood in picture, thus when not needing human intervention, realize the Intelligent Measurement to human behavior in scene, reduce " semantic gap ", significantly improve the accuracy of human behavior identification in image, and the system and method realizing human behavior recognition detection based on Iamge Segmentation and semantic feature extraction of the present invention, its system architecture is simple, method implementation is easy, application cost is cheap, range of application is also comparatively extensive.
Accompanying drawing explanation
Fig. 1 is the block diagram realizing the system of human behavior recognition detection based on Iamge Segmentation and semantic feature extraction of the present invention.
Fig. 2 is the Iamge Segmentation schematic flow sheet realizing the method for human behavior recognition detection based on Iamge Segmentation and semantic feature extraction of the present invention.
Fig. 3 is the human behavior Intelligent Measurement schematic diagram in the method realizing human behavior recognition detection based on Iamge Segmentation and semantic feature extraction of the present invention.
Fig. 4 is the image, semantic feature decision figure based on tree-like svm classifier realized based on Iamge Segmentation and semantic feature extraction in the method for human behavior recognition detection of the present invention.
Fig. 5 utilizes the system and method realizing human behavior recognition detection based on Iamge Segmentation and semantic feature extraction of the present invention to realize the example schematic of human behavior Intelligent Measurement.
Embodiment
In order to more clearly understand technology contents of the present invention, describe in detail especially exemplified by following examples.
Referring to shown in Fig. 1, is the block diagram realizing the system of human behavior recognition detection based on Iamge Segmentation and semantic feature extraction of the present invention.
In one embodiment, this system comprises: image acquisition units 1, human behavior detect host computer 2, user's query unit 5 and output interface unit 6.Wherein, human behavior detects host computer 2 and is connected to described image acquisition units 1, and user's query unit 5 and output interface unit 6 are connected to described human behavior and detect host computer 2.Described output interface unit 6 can be USB interface.
The method utilizing the system of this embodiment to realize carrying out based on Iamge Segmentation and semantic feature extraction human behavior recognition detection comprises the following steps:
(1) the image acquisition units acquisition of image data described in;
(2) human behavior described in is detected host computer and is identified the human behavior in described view data by Iamge Segmentation and image, semantic feature extraction, and produces corresponding human behavior representation manners;
(3) user is inquired about for described human behavior representation manners by described user's query unit;
(4) by described output interface unit, externally connection device or network provide described human behavior representation manners to system.
In one more preferably embodiment, described human behavior detects host computer 2 and comprises: image segmentation unit 3 and semantic feature extraction unit 4.Wherein image segmentation unit 3 is connected to described image acquisition units 1; Semantic feature extraction unit 4 is connected to described image segmentation unit 3, user's query unit 5 and output interface unit 6.
Utilize this more preferably embodiment system realize carry out in the method for human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described step (2) specifically comprises the following steps:
(21) described view data is divided into sub-image data by the image segmentation unit described in;
(22) the semantic feature extraction unit described in identifies the human behavior in described sub-image data and describes, the human behavior representation manners described in generation.
In a kind of further preferred embodiment, described image segmentation unit comprises: Frame sampling subelement, smoothing processing subelement, wavelet transformation subelement and Threshold segmentation subelement.Wherein, the image acquisition units described in the connection of Frame sampling subelement; Frame sampling subelement described in smoothing processing subelement connects; Smoothing processing subelement described in wavelet transformation subelement connects; Wavelet transformation subelement described in Threshold segmentation subelement connects and described semantic feature extraction unit.
Utilize the system of this further preferred embodiment to realize carrying out in the method for human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described step (21) specifically comprises the following steps:
(21-1) the Frame sampling subelement described in carries out Frame sampling to gathered view data;
(21-2) the smoothing process of Frame image that the smoothing processing subelement described in obtains sampling;
(21-3) the wavelet transformation subelement described in carries out wavelet transform process to the image through smoothing processing;
(21-4) the Threshold segmentation subelement described in carries out Threshold segmentation to the image through wavelet transform process, the sub-image data described in generation.
Further preferred embodiment in, described image segmentation unit also comprises: bottom Visual Feature Retrieval Process subelement, and this bottom Visual Feature Retrieval Process subelement is connected between described Threshold segmentation subelement and described semantic feature extraction unit.
Utilize this further system preferred embodiment realize carrying out in the method for human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described step (21) is further comprising the steps of after described step (21-4):
(21-5) the bottom Visual Feature Retrieval Process subelement described in extracts the bottom visual signature of described subimage, and described bottom visual signature comprises color characteristic, textural characteristics, shape facility and locus feature.
In another kind of further preferred embodiment, described semantic feature extraction unit comprises: standard picture semantic knowledge-base, high-level semantics features map subelement and behavior description subelement.Wherein standard picture semantic knowledge-base is in order to store behavior template image and corresponding semantic information; High-level semantics features maps subelement and is connected to described image segmentation unit; Behavior description subelement connects described high-level semantics features respectively and maps subelement and described standard picture semantic knowledge-base.
Utilize the system of this another kind of further preferred embodiment to realize carrying out in the method for human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described step (22) specifically comprises the following steps:
(22-1) high-level semantics features described in maps subelement and maps the high-level semantics features producing support vector machine according to described sub-image data;
(22-2) the behavior description subelement described in mates with the behaviour template image in described standard picture semantic knowledge-base according to described high-level semantics features, obtains corresponding semantic information, the human behavior representation manners described in generation.
In the preferred embodiment of one, described standard picture semantic knowledge-base comprises: behaviour template storehouse and semantic knowledge-base; Wherein, behaviour template storehouse is in order to store behavior Template Information; Semantic knowledge-base is in order to store the semantic knowledge information corresponding with described behaviour template information; Described behavior description subelement comprises: Activity recognition functional unit and behavior extraction of semantics functional unit, and wherein Activity recognition functional unit connects described high-level semantics features mapping subelement and described behaviour template storehouse respectively; Behavior extraction of semantics functional unit then connects described Activity recognition functional unit and described semantic knowledge-base respectively.
Utilize the system of this preferred embodiment to realize carrying out in the method for human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, described step (22-2) specifically comprises the following steps:
(22-2a) the Activity recognition functional unit described in, by the behaviour template information matches in described high-level semantics features and described behaviour template storehouse, produces the image, semantic feature of classifying based on tree-like vector machine;
(22-2b) the behavior extraction of semantics functional unit described in obtains the semantic knowledge information corresponding with described image, semantic feature of classifying based on tree-like vector machine from described semantic knowledge-base, produce sing on web Ontology Language structurized iamge description high-level semantics data, and using these iamge description high-level semantics data as described human behavior representation manners.
In an application of the invention, as shown in Figure 1, system of the present invention comprises: be made up of human behavior Intelligent Measurement upper computer software (solidifying in 2), image acquisition units 1, image segmentation unit 3, image, semantic feature extraction unit 4, user's query unit 5 and USB device 6.Wherein image acquisition units 1 is connected with image segmentation unit 3, image, semantic feature extraction unit 4 by pci bus interface, and USB device 6 connects.
In practical application, image acquisition units can select Microsoft HD-300 high definition pickup camera, has CMOS photo-sensitive cell, the widescreen display of USB interface and 16:9; AD sampling adopts the AD9849 of Analog Device company to realize; Human behavior Intelligent Measurement host computer can select common PC.
As shown in Figure 2, image segmentation unit 3 is made up of following four subroutines:
A, to gather image carry out Frame A D sample subroutine;
B, to the smoothing process subroutine of image;
C, wavelet transformation subroutine;
D, Threshold segmentation process subroutine.
Wherein, the object of picture smooth treatment is to reduce picture noise, thus improves picture quality.
Image partition method can be divided into threshold segmentation method, edge detection method, method for extracting region and dividing method four class in conjunction with Specific Theory Tools usually.To apply in Iamge Segmentation based on the newest research results such as fractal theory and wavelet transformation theory in the present invention.Wavelet transformation has good local characteristics, when filter function yardstick is larger, the ability doing noise is strong, when wavelet transform dimension is less, the ability extracting image detail is strong, the contradiction that so just can solve restraint speckle well and extract between image edge details.
The specific works process of Iamge Segmentation is:
1), after human behavior Intelligent Measurement upper computer software is opened, HD-300 high definition pickup camera enters viewfinder mode automatically;
2) image collected is carried out Frame AD conversion by pci data bus transfer to AD9849;
3) image is after the disposal of gentle filter such as medium filtering, improves picture quality;
4) eventually pass threshold determination, Iamge Segmentation can be become several subimages.
Also to extract the bottom visual signature (comprising color characteristic, textural characteristics, shape facility, locus feature etc.) about each subimage, as the foundation that dimensioned plan differentiates as semantic feature coupling simultaneously.
For discrete case, definition discrete wavelet function:
W
j,k=2
-j/2w (2
-jt-k), wherein j, k ∈ Z,
Then the wavelet transform of signal f (t) is just defined as:
D[W
f(j,k)]＝2
-j/2∑f(t)W(2
-jt-k)Δt
For Threshold segmentation process, threshold estimation method can be defined:
Through above-mentioned four sub-routine processes, obtain the subimage after dividing processing.Also to extract the bottom visual signature (comprising color characteristic, textural characteristics, shape facility, locus feature etc.) about each subimage, as the foundation that dimensioned plan differentiates as semantic feature coupling simultaneously.
As shown in Figure 3, for realizing comprising the disposal route that human behavior carries out Intelligent Measurement according to image, semantic feature: original image storehouse, behaviour template storehouse, semantic knowledge-base, extracts low-level image feature, Activity recognition, and behavior of extracting is semantic.
Image becomes a series of subimage through dividing processing, extracts the bottom visual signature (comprising color characteristic, textural characteristics, shape facility, locus feature etc.) about each subimage.Visual signature is mapped to high-level diagram based on support vector machine (SVM) as semantic feature, differentiates by carrying out mating with behaviour template storehouse, obtaining the description of relevant personnel's behavior through semantic knowledge-base.
As shown in Figure 4, carry out mating differentiate for realizing image, semantic feature and human behavior semantic knowledge-base, method of the present invention chooses high-level diagram based on support vector machine (SVM) as semantic feature.The present invention, on tree-like SVM basis, utilizes cluster the larger classification of distance to be first separated, proposes a kind of Image application processing method of tree-like support vector machine.From root node, comprised semantic category can be assigned to its child nodes by each non-leaf nodes, and last leaf node only comprises a semantic category.
In tree-like SVM, each non-leaf nodes is divided into two groups semantic category, utilizes the Euclidean distance in support vector machine kernel space as measuring.The expression formula of Euclidean distance:
Wherein X1, X2 represent two proper vectors, x
1i, x
2icorresponding i-th component.
Its image, semantic differentiates process:
Suppose to obtain 3 semantic categories { stand, race, fights } state;
Calculate the distance between them respectively, the distance obtained between " standing " state and other two classes state is maximum, and so semantic category state of " standing " is assigned to left subtree g1, and other two kinds of semantic categories " race ", " fighting " state are assigned to right subtree g2;
The rest may be inferred, from right subtree g2, just can draw " race ", " fighting " semantic category state respectively;
In sum, the image, semantic feature decision figure based on tree-like svm classifier as shown in Figure 4 can be obtained.
After the process of image, semantic feature decision, video image characteristic is carried out computing and the operations such as cluster, coupling, Xie Yi, classification, differentiation, produce sing on web Ontology Language (OWL) structurized iamge description high-level semantics data.Between image and iamge description, set up mapping relations, thus enable computing machine by image procossing and analyze the content understood in video pictures, realize the automatic detection to human behavior in scene.
As shown in Figure 5, be one and concrete realize case.First gather an image from pickup camera, carry out image segmentation routine process after AD sampling, obtain several sub-image areas; Secondly, low-level image feature extraction carried out for obtained subimage and maps the image, semantic feature produced based on tree-like svm classifier, carrying out semantic feature with human behavior knowledge base and mate and differentiate, judge that the people in subimage processes the state of " standing "; Finally, we just can produce sing on web Ontology Language (OWL) structurized iamge description high-level semantics data, are the data of description information shown in figure.
Have employed the system and method realizing human behavior recognition detection based on Iamge Segmentation and semantic feature extraction of this invention, its system comprises: image acquisition units, human behavior detect host computer, user's query unit and output interface unit.In the method, human behavior detection host computer is identified the human behavior in the view data of image acquisition units collection by Iamge Segmentation and image, semantic feature extraction, and produce corresponding human behavior representation manners, be supplied to user and undertaken inquiring about or being exported by output interface unit external device or network by user's query unit.Human behavior detects host computer, by support vector machine (SVM), the low-level image feature of image is mapped as high-level semantic, mapping relations are set up between image and iamge description, enable computing machine by Digital Image Processing and analyze the content understood in picture, thus when not needing human intervention, realize the Intelligent Measurement to human behavior in scene, reduce " semantic gap ", significantly improve the accuracy of human behavior identification in image, and the system and method realizing human behavior recognition detection based on Iamge Segmentation and semantic feature extraction of the present invention, its system architecture is simple, method implementation is easy, application cost is cheap, range of application is also comparatively extensive.
In this description, the present invention is described with reference to its specific embodiment.But, still can make various amendment and conversion obviously and not deviate from the spirit and scope of the present invention.Therefore, instructions and accompanying drawing are regarded in an illustrative, rather than a restrictive.
Claims (7)
1. realize a system for human behavior recognition detection based on Iamge Segmentation and semantic feature extraction, it is characterized in that, described system comprises:
Image acquisition units, in order to acquisition of image data;
Human behavior detects host computer, is connected to described image acquisition units, in order to identify for the human behavior in described view data, and produces corresponding human behavior representation manners by Iamge Segmentation and image, semantic feature extraction;
User's query unit, is connected to described human behavior and detects host computer, the inquiry carrying out for described human behavior representation manners in order to provide user;
Output interface unit, in order to externally connection device or network provide described human behavior representation manners;
Wherein, described human behavior detects host computer and comprises:
Image segmentation unit, is connected to described image acquisition units, in order to described view data is divided into sub-image data;
Semantic feature extraction unit, is connected to described image segmentation unit, user's query unit and output interface unit, in order to identify the human behavior in described sub-image data and to describe, and the human behavior representation manners described in generation;
Wherein, described semantic feature extraction unit comprises:
Standard picture semantic knowledge-base, in order to store behavior template image and corresponding semantic information;
High-level semantics features maps subelement, is connected to described image segmentation unit, in order to map the high-level semantics features producing support vector machine according to described sub-image data;
Behavior description subelement, connect described high-level semantics features respectively and map subelement and described standard picture semantic knowledge-base, mate with the behaviour template image in described standard picture semantic knowledge-base according to described high-level semantics features, obtain corresponding semantic information, the human behavior representation manners described in generation;
Wherein, described standard picture semantic knowledge-base comprises:
Behaviour template storehouse, in order to store behavior Template Information;
Semantic knowledge-base, in order to store the semantic knowledge information corresponding with described behaviour template information;
In addition, described behavior description subelement comprises:
Activity recognition functional unit, connect described high-level semantics features respectively and map subelement and described behaviour template storehouse, in order to according to the behaviour template information matches in described high-level semantics features and described behaviour template storehouse, produce the image, semantic feature of classifying based on tree-like vector machine;
Behavior extraction of semantics functional unit, connect described Activity recognition functional unit and described semantic knowledge-base respectively, in order to according to the described image, semantic feature of classifying based on tree-like vector machine, obtain corresponding semantic knowledge information from described semantic knowledge-base, produce sing on web Ontology Language structurized iamge description high-level semantics data as described human behavior representation manners.
2. the system realizing human behavior recognition detection based on Iamge Segmentation and semantic feature extraction according to claim 1, is characterized in that, described image segmentation unit comprises:
Frame sampling subelement, the image acquisition units described in connection, carries out Frame sampling to gathered view data;
Smoothing processing subelement, the Frame sampling subelement described in connection, to the smoothing process of Frame image that sampling obtains;
Wavelet transformation subelement, the smoothing processing subelement described in connection, carries out wavelet transform process to the image through smoothing processing;
Threshold segmentation subelement, the wavelet transformation subelement described in connection and described semantic feature extraction unit, carry out Threshold segmentation to the image through wavelet transform process, the sub-image data described in generation.
3. the system realizing human behavior recognition detection based on Iamge Segmentation and semantic feature extraction according to claim 2, is characterized in that, described image segmentation unit also comprises:
Bottom Visual Feature Retrieval Process subelement, be connected between described Threshold segmentation subelement and described semantic feature extraction unit, in order to extract the bottom visual signature of described subimage, described bottom visual signature comprises color characteristic, textural characteristics, shape facility and locus feature.
4. the system realizing human behavior recognition detection based on Iamge Segmentation and semantic feature extraction according to claim 1, is characterized in that, described output interface unit is USB interface.
5. utilize the system described in claim 1 to realize carrying out based on Iamge Segmentation and semantic feature extraction a method for human behavior recognition detection, it is characterized in that, described method comprises the following steps:
(1) the image acquisition units acquisition of image data described in;
(2) human behavior described in is detected host computer and is identified the human behavior in described view data by Iamge Segmentation and image, semantic feature extraction, and produces corresponding human behavior representation manners;
Described step (2) specifically comprises the following steps:
(21) described view data is divided into sub-image data by the image segmentation unit described in;
(22) the semantic feature extraction unit described in identifies the human behavior in described sub-image data and describes, the human behavior representation manners described in generation;
Described step (22) specifically comprises the following steps:
(22-1) high-level semantics features described in maps subelement and maps the high-level semantics features producing support vector machine according to described sub-image data;
(22-2) the behavior description subelement described in mates with the behaviour template image in described standard picture semantic knowledge-base according to described high-level semantics features, obtains corresponding semantic information, the human behavior representation manners described in generation;
Described step (22-2) specifically comprises the following steps:
(22-2a) the Activity recognition functional unit described in, by the behaviour template information matches in described high-level semantics features and described behaviour template storehouse, produces the image, semantic feature of classifying based on tree-like vector machine;
(22-2b) the behavior extraction of semantics functional unit described in obtains the semantic knowledge information corresponding with described image, semantic feature of classifying based on tree-like vector machine from described semantic knowledge-base, produce sing on web Ontology Language structurized iamge description high-level semantics data, and using these iamge description high-level semantics data as described human behavior representation manners;
(3) user is inquired about for described human behavior representation manners by described user's query unit;
(4) by described output interface unit, externally connection device or network provide described human behavior representation manners to system.
6. method of carrying out human behavior recognition detection based on Iamge Segmentation and semantic feature extraction according to claim 5, it is characterized in that, described image segmentation unit comprises: Frame sampling subelement, smoothing processing subelement, wavelet transformation subelement and Threshold segmentation subelement, and described step (21) specifically comprises the following steps:
(21-1) the Frame sampling subelement described in carries out Frame sampling to gathered view data;
(21-2) the smoothing process of Frame image that the smoothing processing subelement described in obtains sampling;
(21-3) the wavelet transformation subelement described in carries out wavelet transform process to the image through smoothing processing;
(21-4) the Threshold segmentation subelement described in carries out Threshold segmentation to the image through wavelet transform process, the sub-image data described in generation.
7. method of carrying out human behavior recognition detection based on Iamge Segmentation and semantic feature extraction according to claim 6, it is characterized in that, described image segmentation unit also comprises the bottom Visual Feature Retrieval Process subelement be connected between described Threshold segmentation subelement and described semantic feature extraction unit, and described step (21) is further comprising the steps of after described step (21-4):
(21-5) the bottom Visual Feature Retrieval Process subelement described in extracts the bottom visual signature of described subimage, and described bottom visual signature comprises color characteristic, textural characteristics, shape facility and locus feature.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210317234.XA CN102880873B (en) | 2012-08-31 | 2012-08-31 | Personnel behavior identification implementation system and method based on image segmentation and semantic extraction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210317234.XA CN102880873B (en) | 2012-08-31 | 2012-08-31 | Personnel behavior identification implementation system and method based on image segmentation and semantic extraction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102880873A CN102880873A (en) | 2013-01-16 |
| CN102880873B true CN102880873B (en) | 2015-06-03 |
Family
ID=47482190
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210317234.XA Active CN102880873B (en) | 2012-08-31 | 2012-08-31 | Personnel behavior identification implementation system and method based on image segmentation and semantic extraction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102880873B (en) |
Families Citing this family (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104217205B (en) | 2013-05-29 | 2018-05-18 | 华为技术有限公司 | A kind of method and system for identifying User Activity type |
| CN104036505A (en) * | 2014-06-06 | 2014-09-10 | 中山大学深圳研究院 | Heart failure detection method based on combined semantic technology and medical image segmentation |
| CN104735468B (en) * | 2015-04-03 | 2018-08-31 | 北京威扬科技有限公司 | A kind of method and system that image is synthesized to new video based on semantic analysis |
| CN105184260B (en) * | 2015-09-10 | 2019-03-08 | 北京大学 | A kind of image characteristic extracting method and pedestrian detection method and device |
| CN106506528A (en) * | 2016-12-05 | 2017-03-15 | 深圳万智联合科技有限公司 | A kind of Network Safety Analysis system under big data environment |
| CN106776972A (en) * | 2016-12-05 | 2017-05-31 | 深圳万智联合科技有限公司 | A kind of virtual resources integration platform in system for cloud computing |
| CN106779830A (en) * | 2016-12-05 | 2017-05-31 | 深圳万发创新进出口贸易有限公司 | A kind of public community electronic-commerce service platform |
| CN108182443B (en) * | 2016-12-08 | 2020-08-07 | 广东精点数据科技股份有限公司 | Automatic image labeling method and device based on decision tree |
| CN108573229B (en) * | 2017-12-29 | 2021-09-10 | 西安电子科技大学 | Video behavior identification method based on intelligent high-level semantics |
| CN108090472B (en) * | 2018-01-12 | 2021-05-04 | 浙江大学 | Pedestrian re-identification method and system based on multi-channel consistency characteristics |
| CN109271999B (en) * | 2018-09-06 | 2020-12-22 | 北京京东尚科信息技术有限公司 | Image processing method and device and computer readable storage medium |
| CN109816663B (en) * | 2018-10-15 | 2021-04-20 | 华为技术有限公司 | Image processing method, device and equipment |
| CN111435537B (en) * | 2019-01-13 | 2024-01-23 | 北京魔门塔科技有限公司 | Model training method and device and pose optimization method and device based on mosaic |
| CN110008881B (en) * | 2019-03-28 | 2021-03-12 | 北京农业信息技术研究中心 | Method and device for identifying cow behaviors of multiple moving targets |
| CN111507355B (en) * | 2020-04-17 | 2023-08-22 | 北京百度网讯科技有限公司 | Character recognition method, device, equipment and storage medium |
| CN112967293B (en) * | 2021-03-04 | 2024-07-12 | 江苏中科重德智能科技有限公司 | Image semantic segmentation method, device and storage medium |
| CN114299067A (en) * | 2022-03-04 | 2022-04-08 | 西安华创马科智能控制***有限公司 | Underground coal wall caving early warning method and device |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101778260A (en) * | 2009-12-29 | 2010-07-14 | 公安部第三研究所 | Method and system for monitoring and managing videos on basis of structured description |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7853072B2 (en) * | 2006-07-20 | 2010-12-14 | Sarnoff Corporation | System and method for detecting still objects in images |
-
2012
- 2012-08-31 CN CN201210317234.XA patent/CN102880873B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101778260A (en) * | 2009-12-29 | 2010-07-14 | 公安部第三研究所 | Method and system for monitoring and managing videos on basis of structured description |
Non-Patent Citations (1)
| Title |
|---|
| 基于高级语义特征的图像检索技术研究;周璐;《中国优秀硕士学位论文全文数据库 信息科技辑》;20090915;摘要，第5-7页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102880873A (en) | 2013-01-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102880873B (en) | Personnel behavior identification implementation system and method based on image segmentation and semantic extraction | |
| Zaiane et al. | Mining multimedia data. | |
| Feng et al. | Attention-driven salient edge (s) and region (s) extraction with application to CBIR | |
| CN105404886B (en) | Characteristic model generation method and characteristic model generating means | |
| WO2016095487A1 (en) | Human-computer interaction-based method for parsing high-level semantics of image | |
| CN106126585B (en) | The unmanned plane image search method combined based on quality grading with perceived hash characteristics | |
| CN104182765A (en) | Internet image driven automatic selection method of optimal view of three-dimensional model | |
| CN101944183B (en) | Method for identifying object by utilizing SIFT tree | |
| CN101334780A (en) | Figure image searching method, system and recording media for storing image metadata | |
| Despotovic et al. | Prediction and analysis of heating energy demand for detached houses by computer vision | |
| CN108875456A (en) | Object detection method, object detecting device and computer readable storage medium | |
| CN108536780A (en) | A kind of cross-module state object material search method based on touch texture feature | |
| CN111461000A (en) | Intelligent office garbage classification method based on CNN and wavelet analysis | |
| Siraj et al. | Flower image classification modeling using neural network | |
| Jiang et al. | An ontology-based approach to retrieve digitized art images | |
| Mojsilovic et al. | Semantic metric for image library exploration | |
| Zhao et al. | Hybrid generative/discriminative scene classification strategy based on latent Dirichlet allocation for high spatial resolution remote sensing imagery | |
| Bai et al. | An incremental structured part model for object recognition | |
| Jiang et al. | Visual ontology construction for digitized art image retrieval | |
| CN113920498B (en) | Point cloud 3D object detection method based on multilayer feature pyramid | |
| CN102436487B (en) | Optical flow method based on video retrieval system | |
| Gudewar et al. | Ontology to improve CBIR system | |
| Kumaraguru et al. | An image feature extraction and image representation strategies for the analysis of image processing | |
| Mohd-Ekhsan et al. | Classification of flower images based on colour and texture features using neural network | |
| CN104484676B (en) | A kind of interactive mode ancient wall disease identification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |