CN111881754A - Behavior detection method, system, equipment and computer equipment - Google Patents
Behavior detection method, system, equipment and computer equipment Download PDFInfo
- Publication number
- CN111881754A CN111881754A CN202010597281.9A CN202010597281A CN111881754A CN 111881754 A CN111881754 A CN 111881754A CN 202010597281 A CN202010597281 A CN 202010597281A CN 111881754 A CN111881754 A CN 111881754A
- Authority
- CN
- China
- Prior art keywords
- human body
- target human
- frame
- image
- behavior
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 83
- 238000000034 method Methods 0.000 claims abstract description 45
- 230000006399 behavior Effects 0.000 claims description 104
- 210000003423 ankle Anatomy 0.000 claims description 20
- 210000003127 knee Anatomy 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 17
- 238000013135 deep learning Methods 0.000 claims description 5
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000012549 training Methods 0.000 claims description 4
- 230000003542 behavioural effect Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 6
- 238000012544 monitoring process Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000004888 barrier function Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
Abstract
The application relates to a behavior detection method, a system, equipment and computer equipment, wherein the behavior detection method comprises the following steps: in continuous multi-frame images, key point information of a target human body in each frame of image is obtained according to a key point detection model, the geometric position relation of the key points is obtained according to the key point information, the corresponding sub-behaviors of the target human body in each frame of image are determined according to the geometric position relation, and in the continuous multi-frame images, the behavior detection result of the target human body is determined under the condition that the sub-behaviors are the same. Through the method and the device, the problem that the specific behavior of the target human body cannot be detected only by performing behavior detection through quantity comparison in the related technology is solved, the behavior of the target human body is detected, the behavior of the target human body in a public place is normalized, and the safety of the public place is improved.
Description
Technical Field
The present application relates to the field of intelligent monitoring technologies, and in particular, to a method, a system, a device, and a computer device for behavior detection.
Background
With the development of science and technology, public transportation systems including public transportation vehicles, subways and other transportation vehicles are gradually built in all cities. The subway is the first choice for citizens to go out due to the characteristics of no weather influence, no traffic jam, on-time departure and the like, however, the passenger ticket evasion behavior usually occurs in the operation process of the subway, and huge economic loss is brought to subway operation.
Under the general condition, staff can rely on the manual work to monitor passenger's unusual action in the subway through video monitoring, and then judge whether the passenger has the action of escaping a ticket. However, due to the long subway line, it is time-consuming and labor-consuming to perform the monitoring of the fare evasion behavior in a manual polling manner.
In the related technology, the number of passengers who swipe the card is obtained, then image information of the passengers in the passage is collected, the number statistics is carried out according to the image information, and the number of the passengers who swipe the card is compared with the number of the passengers in the passage to judge whether the passenger has the ticket evasion behavior. However, the fare evasion detection is performed by quantity comparison, only whether fare evasion behaviors exist can be judged, the specific behaviors of passengers cannot be detected, and the detection precision is low.
In other scenes, such as shopping malls and pedestrian crossings, it is also necessary to detect illegal behaviors of pedestrians, and determine whether there is a behavior violating regulations, such as crossing obstacles, for pedestrians, so as to maintain public security and traffic order according to the detection result.
At present, no effective solution is provided for the problem that the behavior detection is only carried out through quantity comparison and the specific behavior of the passenger cannot be detected in the related technology.
Disclosure of Invention
The embodiment of the application provides a behavior detection method, a behavior detection system, behavior detection equipment, computer equipment and a computer readable storage medium, and aims to at least solve the problems that in the related art, the number comparison is only used for detecting the fare evasion, and the specific passenger cannot be judged to have the fare evasion, so that the detection precision is low.
In a first aspect, an embodiment of the present application provides a method for behavior detection, where the method includes:
acquiring key point information of a target human body in each frame of image through a key point detection model in continuous multi-frame images;
acquiring the relative position relation of the key points according to the key point information, and determining the corresponding sub-behaviors of the target human body in each frame of image according to the relative position relation;
and determining the behavior detection result of the target human body under the condition that the sub-behaviors of the continuous multi-frame images are the same.
In some embodiments, the obtaining, according to the key point information, a relative position relationship of the key points, and determining, according to the relative position relationship, a corresponding sub-behavior of the target human body in each frame of image includes:
acquiring the key point information in a first frame image and a second frame image, wherein the key points comprise upper body key points and lower body key points of the target human body;
according to the keypoint information, when the upper body keypoints in the second frame image are all higher than the upper body keypoints in the first frame image, and when the lower body keypoints in the second frame image form a triangle, it is determined that the child behavior of the target human body in the second frame image is a squat.
In some embodiments, the obtaining, according to the key point information, a relative position relationship of the key points, and determining, according to the relative position relationship, a corresponding sub-behavior of the target human body in each frame of image further includes:
acquiring the key point information in a third frame image and a fourth frame image, wherein the key points comprise the knee and the ankle of the target human body;
and according to the key point information, in the case that the knee and the ankle in the fourth frame image are respectively lower than those in the third frame image, judging that the behavior of the target human body in the fourth frame image is crossing.
In some embodiments, the obtaining of the key point information of the target human body in each frame of image includes:
and acquiring two-dimensional image information of the target human body according to a pedestrian target human body detection algorithm through a monocular camera, and acquiring the key point information according to the two-dimensional image information.
In some embodiments, after the determining the behavior detection result of the target human body, the method further includes:
and under the condition that the detection result is squatting or crossing, marking the target human body and sending out a warning signal.
In some of these embodiments, further comprising:
obtaining a plurality of sample images in a target human body scene, performing behavior annotation on the sample images, and training the behavior-annotated sample images through a deep learning framework to obtain the key point detection model.
In a second aspect, an embodiment of the present application provides a system for behavior detection, where the system includes an image capture device and a processor;
the camera device acquires continuous multi-frame images of a target human body, and the processor acquires key point information of the target human body in each frame of image through a key point detection model in the continuous multi-frame images;
the processor acquires the relative position relation of the key points according to the key point information, and determines the corresponding sub-behaviors of the target human body in each frame of image according to the relative position relation;
and under the condition that the sub-behaviors of the continuous multi-frame images are the same, the processor determines the behavior detection result of the target human body.
In a third aspect, an embodiment of the present application provides a device for behavior detection, where the device includes: the device comprises an acquisition module, a calculation module and a determination module:
the acquisition module is used for acquiring key point information of a target human body in each frame of image through a key point detection model in continuous multi-frame images;
the computing module is used for acquiring the relative position relation of the key points according to the key point information and determining the corresponding sub-behaviors of the target human body in each frame of image according to the relative position relation;
the determining module is configured to determine a behavior detection result of the target human body when the child behaviors of the consecutive multi-frame images are all the same behavior.
In a fourth aspect, the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements any one of the above methods when executing the computer program.
In a fifth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement any of the above methods.
Compared with the related art, the behavior detection method provided by the embodiment of the application obtains the key point information of the target human body in each frame of image according to the key point detection model in the continuous multi-frame images, obtains the geometric position relation of the key point according to the key point information, determines the corresponding sub-behavior of the target human body in each frame of image according to the geometric position relation, and determines the behavior detection result of the target human body under the condition that the sub-behaviors are the same in the continuous multi-frame images.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a schematic diagram of an application environment of a method of behavior detection according to an embodiment of the present application;
FIG. 2 is a flow diagram of a method of behavior detection according to an embodiment of the present application;
FIG. 3 is a flow chart of a method of squat behavior determination according to an embodiment of the present application;
FIG. 4 is a flow diagram of a method of determination of crossing behavior according to an embodiment of the present application;
FIG. 5 is a block diagram of a system for behavior detection according to an embodiment of the present application;
FIG. 6 is a block diagram of an apparatus for behavior detection according to an embodiment of the present application;
fig. 7 is an internal structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments provided in the present application without any inventive step are within the scope of protection of the present application.
It is obvious that the drawings in the following description are only examples or embodiments of the present application, and that it is also possible for a person skilled in the art to apply the present application to other similar contexts on the basis of these drawings without inventive effort. Moreover, it should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of ordinary skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments without conflict.
Unless defined otherwise, technical or scientific terms referred to herein shall have the ordinary meaning as understood by those of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar words throughout this application are not to be construed as limiting in number, and may refer to the singular or the plural. The present application is directed to the use of the terms "including," "comprising," "having," and any variations thereof, which are intended to cover non-exclusive inclusions; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or elements, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Reference to "connected," "coupled," and the like in this application is not intended to be limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as referred to herein means two or more. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. Reference herein to the terms "first," "second," "third," and the like, are merely to distinguish similar objects and do not denote a particular ordering for the objects.
The method for behavior detection provided by the present application may be applied to an application environment as shown in fig. 1, where fig. 1 is a schematic view of an application environment of the method for behavior detection according to the embodiment of the present application, as shown in fig. 1. The image pickup device 102 and the processor 104 perform data transmission through a network, the image pickup device 102 acquires multi-frame images of pedestrians, the processor 104 acquires, in consecutive multi-frame images, information of a key point of a pedestrian in each frame image through a key point detection model, the processor 104 acquires a geometric position relationship of the key point according to the information of the key point, and determines a corresponding sub-behavior of the pedestrian in each frame image according to the geometric position relationship, and in consecutive multi-frame images, the processor 104 determines a behavior detection result of the pedestrian when the sub-behaviors are the same.
The present embodiment provides a method for behavior detection, and fig. 2 is a flowchart of a method for behavior detection according to an embodiment of the present application, and as shown in fig. 2, the method includes the following steps:
step S201, in continuous multi-frame images, key point information of a target human body in each frame of image is obtained through a key point detection model.
The target human body in the embodiment can be a passenger in a subway station, a pedestrian crossing a road, or a customer in a shopping mall.
The continuous multi-frame images in this embodiment may be multi-frame photos obtained by a camera of the capturing device, or a plurality of continuous frame video images in a video shot by the camera device. The key point detection model in this embodiment is configured to detect key points of a human body, and output key point information, where the key points of the human body include a head, a shoulder, an elbow, a hand, a crotch, a knee, an ankle, and the like, and the key point information may be plane position coordinates of the key points in the drawing and may further include depth information of the key points in the image.
Step S202, obtaining the relative position relation of the key points according to the key point information, and determining the corresponding sub-behaviors of the target human body in each frame of image according to the relative position relation.
The key point information includes a plane position coordinate of the key point in the image, and a relative position relationship of the key point in the image is obtained through coordinate operations, for example, distance calculation, angle calculation, and the like are performed on the plane position coordinate, where the relative position relationship includes a position change of the same key point in different frame images, and also includes a position relationship between different key points in the same image frame, and the position relationship may be a shift of the key point to different directions, or an angle relationship or a distance relationship between multiple key points.
The different relative position relationships correspond to different sub-behaviors, and the sub-behaviors in the embodiment include standing, squatting, crossing and the like.
Step S203, determining the behavior detection result of the target human body when the child behaviors of the continuous multi-frame images are all the same behavior.
In this embodiment, each frame of image corresponds to one sub-behavior, and the behavior detection result of the target human body can be determined when the sub-behaviors of consecutive multi-frame images are determined to be the same behavior. For example, in the continuous multi-frame images, each frame image is judged to be "squat", the "squat" is the behavior detection result of the target human body, and if in the continuous multi-frame images, the child behaviors in some images are judged to be standing and the child behaviors in some images are judged to be squat, the target human body has no corresponding behavior detection result.
Through the steps S201 to S203, the behavior of the target human body in the image is determined according to the relative position relationship between the key point information of the target human body and the key point, and the behavior of the target human body can be detected and identified, so that the problem that the specific behavior of the target human body cannot be detected only by performing behavior detection through quantity comparison in the related art is solved, the behavior of the target human body in a public place is normalized by detecting the behavior of the target human body, and the safety of the public place is improved.
In some embodiments, fig. 3 is a flow chart of a method for squat behavior determination according to embodiments of the present application, as shown in fig. 3, the method including the steps of:
step S301, obtaining the key point information in the first frame image and the second frame image, where the key points include the upper body key point and the lower body key point of the target human body.
The first frame image and the second frame image in this embodiment are two continuous frame images, the upper body key points include the head, shoulder, and elbow of the target human body, and the lower body key points include the crotch, knee, and ankle of the target human body.
Step S302 is to determine that the child behavior of the target human body in the second frame image is squat when the upper body keypoints in the second frame image are higher than the upper body keypoints in the first frame image and the lower body keypoints in the second frame image form a triangle, based on the keypoint information.
The key point information in this embodiment may be represented by a coordinate kj(xj，yj) Is represented by, wherein kjDenotes the jth keypoint, xj，yjIndicating the coordinate position of the keypoint in the image. The judgment formula of the head key points is shown as formula 1:
yi+1,h>yi,hequation 1
The coordinate system in this embodiment uses the upper left corner of the image as the origin of coordinates, coordinates (x)i+1,h,yi+1,h) Representing the location of the head keypoints in the image, y in equation 1i+1,hRepresenting head keypoints, y, in frame i +1i,hThe head keypoint in the ith frame image is represented, where the (i + 1) th frame is the second frame image in this embodiment, the ith frame is the first frame image in this embodiment, and formula 1 represents that the head keypoint in the second frame image is higher than the head keypoint in the first frame image.
The judgment formula of the shoulder key points is shown in formula 2:
yi+1,s>yi,sequation 2
Coordinate (x)i+1,s,yi+1,s) Indicates the location of the shoulder keypoints in the image, in equation 2, yi+1,sRepresenting shoulder keypoints, y, in frame i +1i,sRepresenting the shoulder key points in the ith frame image, formula 2 represents that the shoulder key points in the second frame image are higher than those in the first frame image.
The judgment formula of the elbow key point is shown as formula 3:
yi+1,e>yi,eequation 3
Coordinate (x)i+1,e,yi+1,e) Representing the location of the elbow keypoint in the image, y in equation 3i+1,eRepresenting the elbow keypoint, y, in the i +1 th framei,eAn elbow key point in the ith frame image is represented, and equation 3 represents that the elbow key point in the second frame image is higher than the elbow key point in the first frame image.
The relative position relationship of the upper body key point in the first frame image and the second frame image can be determined through formulas 1 to 3.
The formula for judging the key points of the lower half body is shown in formula 4:
in equation 4, (x)i+1,hp,yi+1,hp) Indicating the position of the crotch in the image, (x)i+1,n,yi+1,n) Representing the position of the knee in the image, (x)i+1,a,yi+1,a) Indicating the position of the ankle in the image. Equation 4 calculates the distance between the crotch, the knee, and the ankle, and refers to the distance between the crotch and the ankle as the first distance, the distance between the crotch and the knee as the second distance, and the distance between the knee and the ankle as the third distance, and equation 4 indicates that the first distance is smaller than the sum of the second distance and the third distance, and at this time, the crotch, the knee, and the ankle form a triangle.
In the case where the key points of the target human body simultaneously satisfy formulas 1 to 4, it may be determined that the child behavior of the target human body in the (i + 1) th frame is squat.
Through the steps S301 and S302, the keypoints are calculated according to the keypoint information of each frame in the image, and when the relative position relationship of the keypoints satisfies the formulas 1 to 4, it is determined that the sub-behavior of the target human body is squatting.
In some embodiments, fig. 4 is a flow chart of a method of decision of crossing behavior according to an embodiment of the present application, as shown in fig. 4, the method comprising the steps of:
step S401, obtaining information of key points in the third frame image and the fourth frame image, where the key points include the knee and the ankle of the target human body.
The third frame image and the fourth frame image in the present embodiment are two consecutive frame images.
Step S402, according to the key point information, in a case that the knee and the ankle in the fourth frame image are lower than the knee and the ankle in the third frame image, respectively, it is determined that the behavior of the target human body in the fourth frame image is a step-over.
In this embodiment, the formula for determining the knee key points is shown in formula 5:
yi+1,n<yi,nequation 5
The coordinate system in this embodiment takes the upper left corner of the image as the origin of coordinates, and in equation 5, yi+1,nRepresenting knee keypoints, y, in frame i +1i,nThe knee key point in the ith frame image is represented, the (i + 1) th frame in the embodiment is the fourth frame image, the ith frame is the third frame image, and formula 5 represents that the knee key point in the fourth frame image is lower than the knee key point in the third frame image.
The formula for judging ankle key points is shown in formula 6:
yi+1,a<yi,aequation 6
In equation 6, yi+1,aDenotes ankle keypoint, y, in frame i +1i,aRepresenting the ankle key point in the ith frame image, equation 6 represents that the ankle key point in the fourth frame image is lower than the ankle key point in the third frame image.
In the case where the knee key point and the ankle key point of the target human body satisfy both of equations 5 and 6, it is determined that the sub-behavior of the target human body in the fourth frame image is a cross.
Through the steps S401 and S402, the keypoints are calculated according to the keypoint information of each frame in the image, and when the relative position relationship of the keypoints satisfies the formulas 5 and 6, the sub-behavior of the target human body is determined to be the crossing.
In some embodiments, obtaining the key point information of the target human body in each frame of image further includes: and acquiring two-dimensional image information of the target human body according to a pedestrian target human body detection algorithm through a monocular camera, and acquiring key point information according to the two-dimensional image information. The pedestrian target human body detection algorithm in the embodiment can be trained through a deep learning framework so as to improve the accuracy of identifying the target human body pedestrians. The two-dimensional image information in the embodiment includes position information of the target human body in the image and does not include depth information, so that space consumed by image storage is reduced. The embodiment acquires the key point information of the human body on the basis of identifying the two-dimensional image information of the target human body, can improve the identification accuracy, acquires the image of the target human body through the monocular camera, is low in cost, and is convenient to calibrate and identify.
In some embodiments, after determining the behavior detection result of the target human body, the method further includes: and under the condition that the detection result is squatting or crossing, marking the target human body and sending out a warning signal. In a subway scene, passengers generally squat or cross through a gate to realize ticket evasion, the behavior can influence the riding order, and pedestrians can cross a fence or other barriers in the daily road passing process to threaten the safety of the passengers. Therefore, the squat behavior and the crossing behavior may be illegal behaviors in a specific scene, and when the squat or crossing is detected, the camera or the monitoring platform sends out a warning signal to remind a worker to confirm the illegal behavior of the marked target human body, and when the illegal behavior of the target human body is confirmed, the target human body is punished. The warning signal in this embodiment may be a sound signal or a light signal. The warning signal is passed through to this embodiment, reminds the staff to carry out the violation and confirms to the target human body of mark, is favorable to looking over in time to the violation, reduces the potential safety hazard, especially in the subway scene, reduces the phenomenon of fleing for a fee, avoids economic loss.
In some embodiments, training the keypoint detection model comprises: obtaining a plurality of sample images in a target human body scene, carrying out behavior annotation on the sample images, and training the behavior-annotated sample images through a deep learning framework to obtain the key point detection model. The target human body scene in this embodiment may be a subway, a mall, or a road, and after the scene is selected, the behavior labeling is performed on the sample image in the scene, and the sample image may be labeled as standing, squatting, crossing, or other preset behaviors. In the embodiment, the sample image is trained through deep learning to obtain the key point detection model, so that the accuracy of the model in identifying the key points of the target human body in the image can be improved.
It should be noted that the steps illustrated in the above-described flow diagrams or in the flow diagrams of the figures may be performed in a computer system, such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flow diagrams, in some cases, the steps illustrated or described may be performed in an order different than here.
The present embodiment provides a system for behavior detection, and fig. 5 is a block diagram of a structure of the system for behavior detection according to an embodiment of the present application, and as shown in fig. 5, the system includes an image capturing device 51 and a processor 52, the image capturing device 51 obtains continuous multi-frame images of a target human body, and in the continuous multi-frame images, the processor 52 obtains key point information of the target human body in each frame image through a key point detection model. The processor 52 obtains the relative position relationship of the key points according to the key point information, and determines the corresponding sub-behavior of the target human body in each frame of image according to the relative position relationship. In the case that the sub-behaviors of the consecutive multi-frame images are all the same behavior, the processor 52 determines the behavior detection result of the target human body. In this embodiment, the processor 52 obtains the relative position relationship of the key points according to the key point information of the target human body, determines the behavior of the target human body in the image according to the relative position relationship, can detect and identify the behavior of the target human body, solves the problem that the specific behavior of the target human body cannot be detected only by comparing the quantity in the related art, and detects the behavior of the target human body to standardize the behavior of the target human body in a public place, thereby improving the safety of the public place.
The present embodiment further provides a behavior detection device, which is used to implement the foregoing embodiments and preferred embodiments, and the description of the behavior detection device is omitted here for brevity. As used hereinafter, the terms "module," "unit," "subunit," and the like may implement a combination of software and/or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementations in hardware or a combination of software and hardware are also possible and contemplated.
Fig. 6 is a block diagram of a device for behavior detection according to an embodiment of the present application, and as shown in fig. 6, the apparatus includes an obtaining module 61, a calculating module 62, and a determining module 63:
the obtaining module 61 is configured to obtain, in consecutive multiple frames of images, the key point information of the target human body in each frame of image through the key point detection model.
And the calculating module 62 is configured to obtain a relative position relationship of the key points according to the key point information, and determine a corresponding sub-behavior of the target human body in each frame of image according to the relative position relationship.
And the determining module 63 is configured to determine a behavior detection result of the target human body when the child behaviors of the consecutive multi-frame images are the same.
In this embodiment, the calculating module 62 obtains the relative position relationship of the key points according to the image in the obtaining module 61, and determines the behavior of the target human body in the image according to the relative position relationship, so that the behavior of the target human body can be detected and identified, the problem that the specific behavior of the target human body cannot be detected only by performing behavior detection through quantity comparison in the related art is solved, and the behavior of the target human body in a public place is normalized by detecting the behavior of the target human body, so that the safety of the public place is improved.
The above modules may be functional modules or program modules, and may be implemented by software or hardware. For a module implemented by hardware, the modules may be located in the same processor; or the modules can be respectively positioned in different processors in any combination.
In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of behavior detection. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
In one embodiment, fig. 7 is a schematic diagram of an internal structure of an electronic device according to an embodiment of the present application, and as shown in fig. 7, there is provided an electronic device, which may be a server, and an internal structure diagram of which may be as shown in fig. 7. The electronic device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the electronic device is used for storing data. The network interface of the electronic device is used for connecting and communicating with an external terminal through a network. The computer program is executed by a processor to implement a method of behavior detection.
Those skilled in the art will appreciate that the architecture shown in fig. 7 is a block diagram of only a portion of the architecture associated with the subject application, and does not constitute a limitation on the electronic devices to which the subject application may be applied, and that a particular electronic device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the steps in the method for behavior detection provided by the above embodiments are implemented.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when being executed by a processor, implements the steps in the method of behavior detection provided by the various embodiments described above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method of behavior detection, the method comprising:
acquiring key point information of a target human body in each frame of image through a key point detection model in continuous multi-frame images;
acquiring the relative position relation of the key points according to the key point information, and determining the corresponding sub-behaviors of the target human body in each frame of image according to the relative position relation;
and determining the behavior detection result of the target human body under the condition that the sub-behaviors of the continuous multi-frame images are the same.
2. The method according to claim 1, wherein the obtaining the relative position relationship of the key points according to the key point information, and the determining the corresponding sub-behavior of the target human body in each frame of image according to the relative position relationship comprises:
acquiring the key point information in a first frame image and a second frame image, wherein the key points comprise upper body key points and lower body key points of the target human body;
according to the keypoint information, when the upper body keypoints in the second frame image are all higher than the upper body keypoints in the first frame image, and when the lower body keypoints in the second frame image form a triangle, it is determined that the child behavior of the target human body in the second frame image is a squat.
3. The method according to claim 1, wherein the obtaining the relative position relationship of the key points according to the key point information, and determining the corresponding sub-behavior of the target human body in each frame of image according to the relative position relationship further comprises:
acquiring the key point information in a third frame image and a fourth frame image, wherein the key points comprise the knee and the ankle of the target human body;
and according to the key point information, in the case that the knee and the ankle in the fourth frame image are respectively lower than those in the third frame image, judging that the behavior of the target human body in the fourth frame image is crossing.
4. The method according to claim 1, wherein the obtaining key point information of the target human body in each frame of image comprises:
and acquiring two-dimensional image information of the target human body according to a pedestrian target human body detection algorithm through a monocular camera, and acquiring the key point information according to the two-dimensional image information.
5. The method of claim 1, wherein after the determining the behavior detection result of the target human body, the method further comprises:
and under the condition that the detection result is squatting or crossing, marking the target human body and sending out a warning signal.
6. The method of claim 1, further comprising:
obtaining a plurality of sample images in a target human body scene, performing behavior annotation on the sample images, and training the behavior-annotated sample images through a deep learning framework to obtain the key point detection model.
7. A system for behavioral detection, the system comprising a camera and a processor;
the camera device acquires continuous multi-frame images of a target human body, and the processor acquires key point information of the target human body in each frame of image through a key point detection model in the continuous multi-frame images;
the processor acquires the relative position relation of the key points according to the key point information, and determines the corresponding sub-behaviors of the target human body in each frame of image according to the relative position relation;
and under the condition that the sub-behaviors of the continuous multi-frame images are the same, the processor determines the behavior detection result of the target human body.
8. An apparatus for behavior detection, the apparatus comprising: the device comprises an acquisition module, a calculation module and a determination module:
the acquisition module is used for acquiring key point information of a target human body in each frame of image through a key point detection model in continuous multi-frame images;
the computing module is used for acquiring the relative position relation of the key points according to the key point information and determining the corresponding sub-behaviors of the target human body in each frame of image according to the relative position relation;
the determining module is configured to determine a behavior detection result of the target human body when the child behaviors of the consecutive multi-frame images are all the same behavior.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 6 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010597281.9A CN111881754A (en) | 2020-06-28 | 2020-06-28 | Behavior detection method, system, equipment and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010597281.9A CN111881754A (en) | 2020-06-28 | 2020-06-28 | Behavior detection method, system, equipment and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111881754A true CN111881754A (en) | 2020-11-03 |
Family
ID=73157150
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010597281.9A Pending CN111881754A (en) | 2020-06-28 | 2020-06-28 | Behavior detection method, system, equipment and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111881754A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112348035A (en) * | 2020-11-11 | 2021-02-09 | 东软睿驰汽车技术（沈阳）有限公司 | Vehicle key point detection method and device and electronic equipment |
| CN112580543A (en) * | 2020-12-24 | 2021-03-30 | 四川云从天府人工智能科技有限公司 | Behavior recognition method, system and device |
| CN113627334A (en) * | 2021-08-10 | 2021-11-09 | 浙江大华技术股份有限公司 | Object behavior identification method and device |
| CN114518801A (en) * | 2022-02-18 | 2022-05-20 | 美的集团(上海)有限公司 | Device control method, computer program product, control device, and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108229355A (en) * | 2017-12-22 | 2018-06-29 | 北京市商汤科技开发有限公司 | Activity recognition method and apparatus, electronic equipment, computer storage media, program |
| CN109919122A (en) * | 2019-03-18 | 2019-06-21 | 中国石油大学(华东) | A kind of timing behavioral value method based on 3D human body key point |
| CN110110613A (en) * | 2019-04-19 | 2019-08-09 | 北京航空航天大学 | A kind of rail traffic exception personnel's detection method based on action recognition |
| WO2019232894A1 (en) * | 2018-06-05 | 2019-12-12 | 中国石油大学(华东) | Complex scene-based human body key point detection system and method |
| US20200042776A1 (en) * | 2018-08-03 | 2020-02-06 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and apparatus for recognizing body movement |
-
2020
- 2020-06-28 CN CN202010597281.9A patent/CN111881754A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108229355A (en) * | 2017-12-22 | 2018-06-29 | 北京市商汤科技开发有限公司 | Activity recognition method and apparatus, electronic equipment, computer storage media, program |
| WO2019232894A1 (en) * | 2018-06-05 | 2019-12-12 | 中国石油大学(华东) | Complex scene-based human body key point detection system and method |
| US20200042776A1 (en) * | 2018-08-03 | 2020-02-06 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and apparatus for recognizing body movement |
| CN109919122A (en) * | 2019-03-18 | 2019-06-21 | 中国石油大学(华东) | A kind of timing behavioral value method based on 3D human body key point |
| CN110110613A (en) * | 2019-04-19 | 2019-08-09 | 北京航空航天大学 | A kind of rail traffic exception personnel's detection method based on action recognition |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112348035A (en) * | 2020-11-11 | 2021-02-09 | 东软睿驰汽车技术（沈阳）有限公司 | Vehicle key point detection method and device and electronic equipment |
| CN112580543A (en) * | 2020-12-24 | 2021-03-30 | 四川云从天府人工智能科技有限公司 | Behavior recognition method, system and device |
| CN112580543B (en) * | 2020-12-24 | 2024-04-16 | 四川云从天府人工智能科技有限公司 | Behavior recognition method, system and device |
| CN113627334A (en) * | 2021-08-10 | 2021-11-09 | 浙江大华技术股份有限公司 | Object behavior identification method and device |
| CN114518801A (en) * | 2022-02-18 | 2022-05-20 | 美的集团(上海)有限公司 | Device control method, computer program product, control device, and storage medium |
| CN114518801B (en) * | 2022-02-18 | 2023-10-27 | 美的集团(上海)有限公司 | Device control method, control device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111881754A (en) | Behavior detection method, system, equipment and computer equipment | |
| Fang et al. | Computer vision aided inspection on falling prevention measures for steeplejacks in an aerial environment | |
| CN109670441B (en) | Method, system, terminal and computer readable storage medium for realizing wearing recognition of safety helmet | |
| Chen et al. | A proactive workers' safety risk evaluation framework based on position and posture data fusion | |
| CN109740424A (en) | Traffic violations recognition methods and Related product | |
| CN112085952B (en) | Method and device for monitoring vehicle data, computer equipment and storage medium | |
| CN110459064B (en) | Vehicle illegal behavior detection method and device and computer equipment | |
| US9177214B1 (en) | Method and apparatus for an adaptive threshold based object detection | |
| CN112071084A (en) | Method and system for judging illegal parking by utilizing deep learning | |
| CN109785637B (en) | Analysis and evaluation method and device for vehicle violation | |
| CN110516518A (en) | A kind of illegal manned detection method of non-motor vehicle, device and electronic equipment | |
| CN113470374B (en) | Vehicle overspeed monitoring method and device, computer equipment and storage medium | |
| Oh et al. | Development of conflict techniques for left-turn and cross-traffic at protected left-turn signalized intersections | |
| CN112016528B (en) | Behavior recognition method and device, electronic equipment and readable storage medium | |
| US20170243067A1 (en) | Side window detection through use of spatial probability maps | |
| JP2023505122A (en) | Annotating, collecting and processing crowdsourced on-demand AI data | |
| CN112562406B (en) | Method and device for identifying off-line driving | |
| CN110838230B (en) | Mobile video monitoring method, monitoring center and system | |
| CN111401282A (en) | Target detection method, target detection device, computer equipment and storage medium | |
| CN112712703A (en) | Vehicle video processing method and device, computer equipment and storage medium | |
| Lu et al. | Automated visual surveying of vehicle heights to help measure the risk of overheight collisions using deep learning and view geometry | |
| CN113469115A (en) | Method and apparatus for outputting information | |
| CN112307989A (en) | Method and device for identifying road surface object, computer equipment and storage medium | |
| Zhang | Multi-object trajectory extraction based on YOLOv3-DeepSort for pedestrian-vehicle interaction behavior analysis at non-signalized intersections | |
| Huang et al. | Skeleton-based automatic assessment and prediction of intrusion risk in construction hazardous areas |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |