CN114202804A - Behavior action recognition method and device, processing equipment and storage medium - Google Patents
Behavior action recognition method and device, processing equipment and storage medium Download PDFInfo
- Publication number
- CN114202804A CN114202804A CN202210135133.4A CN202210135133A CN114202804A CN 114202804 A CN114202804 A CN 114202804A CN 202210135133 A CN202210135133 A CN 202210135133A CN 114202804 A CN114202804 A CN 114202804A
- Authority
- CN
- China
- Prior art keywords
- target
- class object
- class
- preset
- objects
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a behavior action recognition method, a behavior action recognition device, processing equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: detecting a two-dimensional image target of a target scene, and determining a first class object and a target second class object matched with the first class object in the target scene; detecting skeleton key points of a first class of objects in the two-dimensional image to obtain the skeleton key points of the first class of objects; and detecting whether abnormal behaviors aiming at the target second class object exist in the first class object or not according to the position relation between preset skeleton key points in the skeleton key points of the first class object and the target second class object. And determining a target second class object matched with the first class object in position, wherein the target second class object is an object capable of contacting with the first class object, and the determined target second class object is more reasonable, and then detecting whether the first class object has abnormal behaviors aiming at the target second class object or not, so that the detected abnormal behavior is more accurate.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a behavior action recognition method, a behavior action recognition device, a behavior action recognition processing device and a storage medium.
Background
With the continuous development of emerging technologies, the artificial intelligence technology has made great progress, and has been widely applied in multiple industries and fields, and it has also become a research hotspot to identify behaviors and actions in the acquired images.
In the related technology, a camera can shoot a three-dimensional scene to obtain two-dimensional image information; and identifying the two-dimensional image information by adopting a target detection algorithm, so that the behavior action between two types of objects in the two-dimensional image can be identified.
However, in the related art, since the motion is recognized based on the two-dimensional image information, the recognized behavior and motion are likely to be inaccurate.
Disclosure of Invention
The present invention is directed to provide a behavior action recognition method, device, processing device and storage medium, so as to solve the problem in the related art that the behavior action is not accurate when the action is recognized based on two-dimensional image information.
In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solutions:
in a first aspect, an embodiment of the present invention provides a behavior action recognition method, where the method includes:
carrying out target detection on a two-dimensional image of a target scene, and determining a first class of object in the target scene and a target second class of object matched with the first class of object in position;
detecting skeleton key points of the first class of objects in the two-dimensional image to obtain the skeleton key points of the first class of objects;
and detecting whether abnormal behaviors aiming at the target second class object exist in the first class object or not according to the position relation between preset bone key points in the bone key points of the first class object and the target second class object.
Optionally, the performing target detection on the two-dimensional image of the target scene, determining a first class of object in the target scene, and determining a target second class of object whose position matches the first class of object includes:
performing target detection on the two-dimensional image to obtain a plurality of target objects in the target scene, wherein the plurality of target objects comprise: the first class of objects and at least one second class of objects;
and determining a second class object matched with the region range from the at least one second class object as the target second class object according to the region range where the first class object is located in the target scene.
Optionally, the determining, according to the area range where the first class object is located in the target scene, a second class object matched with the area range from the at least one second class object as the target second class object includes:
dividing the two-dimensional image of the target scene into a plurality of grids;
taking the grid where the first type of object is located as the area range where the first type of object is located;
determining a target grid matched with the grid where the first class object is located from the grid where the at least one second class object is located by adopting a preset mapping relation;
and taking the second class object in the target grid as the target second class object.
Optionally, the determining, according to the area range where the first class object is located in the target scene, a second class object matched with the area range from the at least one second class object as the target second class object includes:
judging whether the area range is a preset behavior detection range of the first class of objects;
and if the area range is the preset behavior detection range, determining a second class object matched with the area range from the at least one second class object as the target second class object according to the area range.
Optionally, before detecting whether there is an abnormal behavior for the target second class object in the first class object according to a position relationship between preset bone key points in the bone key points of the first class object and the target second class object, the method further includes:
calculating an action change parameter of the first class of objects according to preset skeleton key points of the first class of objects detected within a preset time range;
judging whether the first type of object has a preset type of action according to the action change parameter;
the detecting whether the first class object has abnormal behavior aiming at the target second class object according to the position relationship between preset bone key points in the bone key points of the first class object and the target second class object comprises:
if the first class object has the action of the preset type, detecting whether the first class object has abnormal behaviors aiming at the target second class object or not according to the position relation between the preset skeleton key point and the target second class object.
Optionally, the calculating, according to the bone key point of the first class of object detected within the preset time range, the motion parameter of the first class of object includes:
and calculating the action change parameters of the preset part in the first class of objects according to the bone key points of the preset part of the first class of objects detected in the preset time range.
Optionally, if the first class object has the action of the preset type, detecting whether the first class object has an abnormal behavior for the target second class object according to a position relationship between the preset skeletal key point and the target second class object, including:
determining a target frame of the target second-class object, wherein the target frame is used for framing a target part of the target second-class object;
if the first class object has the action of the preset type and the preset skeleton key point is in a target frame of the target second class object, determining that the first class object has abnormal behavior aiming at the target second class object;
and if the first class object has the action of the preset type and the preset skeleton key point is not in the target frame of the target second class object, determining that the first class object does not have abnormal behavior aiming at the target second class object.
In a second aspect, an embodiment of the present invention further provides a behavior action recognition apparatus, where the apparatus includes:
the determining module is used for carrying out target detection on the two-dimensional image of a target scene, determining a first class of object in the target scene and a target second class of object matched with the first class of object in position;
the acquisition module is used for detecting the bone key points of the first class of objects in the two-dimensional image to obtain the bone key points of the first class of objects;
the detection module is used for detecting whether the first class object has abnormal behaviors aiming at the target second class object according to the position relation between preset bone key points in the bone key points of the first class object and the target second class object.
Optionally, the determining module is further configured to perform target detection on the two-dimensional image to obtain a plurality of target objects in the target scene, where the plurality of target objects include: the first class of objects and at least one second class of objects; and determining a second class object matched with the region range from the at least one second class object as the target second class object according to the region range where the first class object is located in the target scene.
Optionally, the determining module is further configured to divide the two-dimensional image of the target scene into a plurality of grids; taking the grid where the first type of object is located as the area range where the first type of object is located; determining a target grid matched with the grid where the first class object is located from the grid where the at least one second class object is located by adopting a preset mapping relation; and taking the second class object in the target grid as the target second class object.
Optionally, the determining module is further configured to determine whether the area range is a preset behavior detection range of the first class of object; and if the area range is the preset behavior detection range, determining a second class object matched with the area range from the at least one second class object as the target second class object according to the area range.
Optionally, the apparatus further comprises:
the calculation module is used for calculating the motion change parameters of the first class of objects according to preset skeleton key points of the first class of objects detected within a preset time range;
the judging module is used for judging whether the first type of object has the action of the preset type or not according to the action change parameter;
the detection module is further configured to detect whether the first class object has an abnormal behavior for the target second class object according to a position relationship between the preset skeletal key point and the target second class object if the first class object has the action of the preset type.
Optionally, the calculating module is further configured to calculate an action change parameter of the preset portion in the first class of object according to the bone key point of the preset portion of the first class of object detected within the preset duration range.
Optionally, the detection module is further configured to determine a target frame of the target second class object, where the target frame is used to frame a target portion of the target second class object; if the first class object has the action of the preset type and the preset skeleton key point is in a target frame of the target second class object, determining that the first class object has abnormal behavior aiming at the target second class object; and if the first class object has the action of the preset type and the preset skeleton key point is not in the target frame of the target second class object, determining that the first class object does not have abnormal behavior aiming at the target second class object.
In a third aspect, an embodiment of the present invention further provides a server, including: a memory storing a computer program executable by the processor, and a processor implementing the behavior action recognition method according to any one of the first aspect.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the storage medium, and when the computer program is read and executed, the method for recognizing a behavioral action according to any one of the above first aspects is implemented.
The invention has the beneficial effects that: in summary, an embodiment of the present invention provides a behavior action recognition method, including: carrying out target detection on the two-dimensional image of the target scene, and determining a first class of object in the target scene and a target second class of object matched with the first class of object in position; detecting skeleton key points of a first class of objects in the two-dimensional image to obtain the skeleton key points of the first class of objects; and detecting whether abnormal behaviors aiming at the target second class object exist in the first class object or not according to the position relation between preset skeleton key points in the skeleton key points of the first class object and the target second class object. The method comprises the steps of determining a target second class object matched with the first class object in position, wherein the target second class object is an object capable of being in contact with the first class object, so that the determined target second class object is more reasonable, and then detecting whether abnormal behaviors aiming at the target second class object exist in the first class object or not according to the position relation between a preset skeleton key point of the first class object and the target second class object, so that the detected abnormal behavior is more accurate.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a schematic flow chart of a behavior action recognition method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a behavior action recognition method according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of a behavior action recognition method according to an embodiment of the present invention;
fig. 4 is a schematic flow chart of a behavior action recognition method according to an embodiment of the present invention;
fig. 5 is a schematic flow chart of a behavior action recognition method according to an embodiment of the present invention;
fig. 6 is a schematic flow chart of a behavior action recognition method according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a behavior recognition device according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a processing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the description of the present application, it should be noted that if the terms "upper", "lower", etc. are used for indicating the orientation or positional relationship based on the orientation or positional relationship shown in the drawings or the orientation or positional relationship which is usually arranged when the product of the application is used, the description is only for convenience of describing the application and simplifying the description, but the indication or suggestion that the referred device or element must have a specific orientation, be constructed in a specific orientation and operation, and thus, cannot be understood as the limitation of the application.
Furthermore, the terms "first," "second," and the like in the description and in the claims, as well as in the drawings, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the features of the embodiments of the present application may be combined with each other without conflict.
With the continuous development of emerging technologies, the artificial intelligence technology has made great progress, and has been widely applied in multiple industries and fields, and it has also become a research hotspot to identify behaviors and actions in the acquired images. In the related technology, a camera can shoot a three-dimensional scene to obtain two-dimensional image information; and identifying the two-dimensional image information by adopting a target detection algorithm, so that the behavior action between two types of objects in the two-dimensional image can be identified.
However, in the related art, the two types of objects do not make actual contact, but the two types of objects make contact on the screen displayed by the two-dimensional image, and the motion recognition is directly performed based on the two-dimensional image information, so that the problem that the recognized behavior and motion are inaccurate easily occurs.
In view of the above technical problems in the related art, an embodiment of the present application provides a behavior and action recognition method, which not only determines a first class object in a target scene, but also determines a target second class object matched with the first class object in position, where the target second class object is an object that can be in contact with the first class object, so that the determined target second class object is more reasonable, and then detects whether there is an abnormal behavior for the target second class object in the first class object according to a position relationship between a preset skeletal key point of the first class object and the target second class object, so that the detected behavior and action are more accurate.
In this embodiment of the present application, the execution subject may be a processing device, and the processing device may be a terminal or a server, and when the processing device is a terminal, the processing device may be any one of the following: desktop computers, notebook computers, tablet computers, smart phones, and the like.
The following explains a behavior and action recognition method provided in an embodiment of the present application, with a processing device as an execution subject.
Fig. 1 is a schematic flow chart of a behavior action recognition method according to an embodiment of the present invention, as shown in fig. 1, the method may include:
s101, carrying out target detection on the two-dimensional image of the target scene, and determining a first class object in the target scene and a target second class object matched with the first class object in position.
The two-dimensional image of the target scene may include at least one image in a sequential order.
In some embodiments, a processing device may acquire a two-dimensional image of a target scene; performing target detection on the two-dimensional image of the target scene according to preset first class object characteristics and second class object characteristics, determining a first class object and a second class object in the target scene, and determining the position of the first class object where the first class object is located; and then determining a target second-class object matched with the first-class object in position from the second-class objects.
It should be noted that the two-dimensional image of the target scene may be an image received by the processing device and acquired in real time by the image acquisition device, may also be an image pre-stored in the processing device, and may also be an image acquired in another manner, which is not specifically limited in this embodiment of the application.
S102, detecting skeleton key points of the first class of objects in the two-dimensional image to obtain the skeleton key points of the first class of objects.
The processing device can extract the bone key points of the first type of object in the two-dimensional image by adopting a preset algorithm to obtain the bone key points of the first type of object.
For example, the preset algorithm may be an alphapos (a real-time multi-person posture estimation system) algorithm, and certainly, the preset algorithm may also be other algorithms capable of detecting bone key points, which is not specifically limited in the embodiment of the present application.
In this application embodiment, both the first-class object and the target second-class object may be a person or a robot, may also be an object, and may also be other objects that need to be monitored, which is not specifically limited in this application embodiment.
S103, detecting whether abnormal behaviors aiming at the target second class object exist in the first class object or not according to the position relation between preset skeleton key points in the skeleton key points of the first class object and the target second class object.
In some embodiments, the processing device detects abnormal behavior between the first class object and the target second class object according to a position relationship between a preset skeleton key point of the first class object and the target second class object, so as to obtain an abnormal behavior detection result. The abnormal behavior detection result may indicate that the first class object has an abnormal behavior for the target second class object, or that the first class object does not have an abnormal behavior for the target second class object.
In addition, the processing device can display the abnormal behavior detection result, can also send the abnormal behavior detection result to other devices, and can also give an alarm and the like according to the abnormal behavior detection result. For example, when the abnormal behavior detection result indicates that the first class object has abnormal behavior aiming at the target second class object, an alarm is given.
In summary, an embodiment of the present invention provides a behavior action recognition method, including: carrying out target detection on the two-dimensional image of the target scene, and determining a first class of object in the target scene and a target second class of object matched with the first class of object in position; detecting skeleton key points of a first class of objects in the two-dimensional image to obtain the skeleton key points of the first class of objects; and detecting whether abnormal behaviors aiming at the target second class object exist in the first class object or not according to the position relation between preset skeleton key points in the skeleton key points of the first class object and the target second class object. The method comprises the steps of determining a target second class object matched with the first class object in position, wherein the target second class object is an object capable of being in contact with the first class object, so that the determined target second class object is more reasonable, and then detecting whether abnormal behaviors aiming at the target second class object exist in the first class object or not according to the position relation between a preset skeleton key point of the first class object and the target second class object, so that the detected abnormal behavior is more accurate.
Optionally, fig. 2 is a schematic flow chart of a behavior recognition method according to an embodiment of the present invention, and as shown in fig. 2, the process of performing target detection on the two-dimensional image of the target scene in S101, and determining the first class object in the target scene and the target second class object whose position is matched with the first class object may include:
s201, carrying out target detection on the two-dimensional image to obtain a plurality of target objects in a target scene.
Wherein the plurality of target objects may include: a first class of objects and at least one second class of objects.
In some embodiments, the processing device performs model training using a large number of artificially labeled datasets via a target detection algorithm to obtain a trained model, and performs target detection on the two-dimensional image using the trained model to obtain a plurality of target objects in a target scene.
It should be noted that, the model is used to identify a plurality of target objects, so that the classification of the first class of objects and the at least one second class of objects is more accurate, and a basis is provided for identifying abnormal behaviors between the subsequent first class of objects and the target second class of objects.
S202, according to the area range where the first class object is located in the target scene, determining a second class object matched with the area range from at least one second class object as a target second class object.
In a possible implementation manner, the processing device determines an area range in which a first type of object is located in a target scene; and determining a target area range matched with the area range where the first type of object is located from the area range where the at least one second type of object is located by adopting a preset mapping relation, and taking the second type of object in the target area range as a target second type of object.
In the embodiment of the present application, the second class object that matches the area range where the first class object is located is a target second class object that can be in contact with the first class object.
Optionally, fig. 3 is a schematic flowchart of a method for identifying a behavior action according to an embodiment of the present invention, and as shown in fig. 3, the process of determining, in S202, a second class object matched with the area range from at least one second class object according to the area range where the first class object is located in the target scene as the target second class object includes:
s301, dividing the two-dimensional image of the target scene into a plurality of grids.
The processing device may perform spatial division on the two-dimensional image of the target scene by using a gridding operation to obtain a plurality of grids.
S302, taking the grid where the first type of object is located as the area range where the first type of object is located.
It should be noted that, according to the position of the first type object, the grid where the first type object is located is determined, and then the area range where the first type object is located is determined.
S303, determining a target grid matched with the grid where the first type object is located from the grid where the at least one second type object is located by adopting a preset mapping relation.
The preset mapping relation can be used for representing the corresponding relation between the grids where the first type of objects are located and the grids where the second type of objects are located.
In some embodiments, the processing device may determine, from the grids in which the at least one second-type object is located, at least one grid in which the second-type object adjacent to the grid in which the first-type object is located, where the at least one grid is the target grid, by using a preset mapping relationship.
Of course, a preset mapping relationship may also be adopted, and at least one grid where the second type object is located, which is closest to the grid where the first type object is located, is determined from the grid where the at least one second type object is located, and the at least one grid is the target grid; this is not particularly limited by the embodiments of the present application.
S304, taking the second-class object in the target grid as a target second-class object.
In summary, in order to avoid the loss of information mapped to the two-dimensional information of the camera in the real world, the two-dimensional image of the target scene is divided into a plurality of grids, a preset mapping relation is adopted, the target grid matched with the grid where the first type of object is located is further determined, the target second type of object is further determined, the information lost by the two-dimensional information of the camera is supplemented, the problem of false recognition of non-abnormal behaviors in the two-dimensional image information can be greatly avoided, and the accuracy of abnormal behavior recognition is effectively improved. And only the target second-class object is monitored, the second-class object which does not have abnormal action behaviors is screened, and unnecessary calculation and false identification are reduced.
Optionally, fig. 4 is a flowchart illustrating a method for identifying a behavior action according to an embodiment of the present invention, and as shown in fig. 4, the step of determining, in S202, a second class object matched with the area range from at least one second class object as a target second class object according to the area range where the first class object is located in the target scene may include:
s401, judging whether the area range is a preset behavior detection range of the first class of objects.
The processing device may store a preset behavior detection range for a target scene, where the preset behavior detection range may be an area where the first type of object may have abnormal behavior, and the processing device may determine whether the area where the first type of object is located belongs to the preset behavior detection range of the first type of object.
S402, if the area range is the preset behavior detection range, determining a second class object matched with the area range from at least one second class object as a target second class object according to the area range.
In the embodiment of the application, when the area range where the first class object is located is the preset behavior detection range, the second class object matched with the area range is determined to be the target second class object from at least one second class object according to the area range. In this way, unnecessary calculations are reduced, and the probability of misidentification of an abnormal behavior action is reduced.
Optionally, fig. 5 is a schematic flow chart of the behavior action recognition method according to the embodiment of the present invention, as shown in fig. 5, before the process of detecting whether the first class object has an abnormal behavior for the target second class object according to the position relationship between the preset bone key points in the bone key points of the first class object and the target second class object in S103, the method may further include:
s501, calculating motion change parameters of the first class of objects according to preset skeleton key points of the first class of objects detected within a preset time range.
Optionally, the motion change parameter of the preset portion in the first type of object is calculated according to the bone key point of the preset portion of the first type of object detected within the preset duration range.
And S502, judging whether the first-class object has the action of the preset type or not according to the action change parameter.
The preset bone key points of the first class of objects may be: the preset skeleton key points of the target part of the first-class object can be set according to actual requirements, and the method is not particularly limited in the embodiment of the application. For example, when the first type object is a person, the target portion may be a wrist, an elbow, a shoulder, or the like, or may be a leg.
In some embodiments, the processing device may detect skeletal keypoints of the wrist, elbow, shoulder, etc. of the first type of subject, and calculate an angle of the elbow position formed by the wrist, elbow, shoulder, etc. in the two-dimensional image at the current time. Wherein, the action change parameters of the first class of objects may include: the angle of the elbow position.
It should be noted that the two-dimensional image may include a plurality of images in a sequential order, and for each two-dimensional image, the processing device may calculate the angle of the elbow position, and determine the angle change curve of the elbow angle in the current time through a time window.
In some embodiments, the processing device may determine whether an angle change curve of the elbow angle and a curve of the elbow angle change of the previously trained abnormal behavior (a preset type of motion) belong to the same class; if the first type of object belongs to the same type, determining that the first type of object has an action of a preset type; and if the first type object does not belong to the same type, determining that the first type object does not have the action of the preset type.
In summary, based on the motion change parameter within the preset duration range, it is determined whether the first type of object has the motion of the preset type, and the semantic information of the time dimension is added, so that the motion recognition of the first type of object is more accurate.
In the step S103, a process of detecting whether the first class object has an abnormal behavior for the target second class object according to a position relationship between preset bone key points in the bone key points of the first class object and the target second class object may include:
s503, if the first-class object has the action of the preset type, detecting whether the first-class object has abnormal behavior aiming at the target second-class object or not according to the position relation between the preset skeleton key point and the target second-class object.
In the embodiment of the application, if the first-class object has the action of the preset type, it is indicated that the first-class object is suspected to have the abnormal action, and under the condition that the first-class object is suspected to have the abnormal action, whether the first-class object has the abnormal action aiming at the target second-class object is detected according to the position relation between the preset skeleton key point and the target second-class object, so that the frequent detection of the position relation between the preset skeleton key point and the target second-class object is avoided, the action of the invalid action habit of the first-class object is filtered, and the calculation amount is reduced.
Optionally, fig. 6 is a flowchart illustrating a method for identifying a behavior action according to an embodiment of the present invention, as shown in fig. 6, if the first class object has a preset type of action, a process of detecting whether the first class object has an abnormal behavior for the target second class object according to a position relationship between a preset skeletal key point and the target second class object in S503 may include:
s601, determining a target frame of the target second-class object, wherein the target frame is used for framing a target part of the target second-class object.
In some embodiments, the processing device may identify the target portion of the target second type object according to a feature of the target portion, and select the target portion of the target second type object with the target frame.
S602, if the first-class object has the action of the preset type and the preset skeleton key point is in the target frame of the target second-class object, determining that the first-class object has abnormal behavior aiming at the target second-class object.
In the embodiment of the application, if the first-class object has a preset type of motion and the preset skeleton key point is in the target frame of the target second-class object, it is indicated that the first-class object and the target second-class object have substantial contact and the motion of the first-class object is abnormal, that is, the first-class object has abnormal behavior for the target second-class object.
S603, if the first-class object has the action of the preset type and the preset skeleton key point is not in the target frame of the target second-class object, determining that the first-class object does not have abnormal behavior aiming at the target second-class object.
In the embodiment of the application, if the first-class object has a preset type of motion and the preset skeleton key point is not in the target frame of the target second-class object, it is indicated that the first-class object and the target second-class object do not make substantial contact, and although the motion of the first-class object is abnormal, the first-class object does not have abnormal behavior for the target second-class object.
In addition, if the first-class object does not have the action of the preset type, the first-class object does not have the abnormal behavior aiming at the target second-class object.
In practical applications, the target scene may be a teaching scene in a classroom, the first type object may be a classroom, and the second type object may be a student. Of course, this is merely an example, and the target scene, the first class object, and the second class object may all be applied according to actual requirements, which is not specifically limited in this embodiment of the application.
In summary, an embodiment of the present invention provides a behavior action recognition method, including: carrying out target detection on the two-dimensional image of the target scene, and determining a first class of object in the target scene and a target second class of object matched with the first class of object in position; detecting skeleton key points of a first class of objects in the two-dimensional image to obtain the skeleton key points of the first class of objects; and detecting whether abnormal behaviors aiming at the target second class object exist in the first class object or not according to the position relation between preset skeleton key points in the skeleton key points of the first class object and the target second class object. The method comprises the steps of determining a target second class object matched with the first class object in position, wherein the target second class object is an object capable of being in contact with the first class object, so that the determined target second class object is more reasonable, and then detecting whether abnormal behaviors aiming at the target second class object exist in the first class object or not according to the position relation between a preset skeleton key point of the first class object and the target second class object, so that the detected abnormal behavior is more accurate.
Moreover, a grid division mode is adopted, so that the information loss caused by mapping three-dimensional information in the real world to two-dimensional information of a camera is avoided, and the abnormal behavior detection of the first class object aiming at the target second class object is more accurate; by adopting a time sequence classification algorithm, certain time semantics is increased, and the method is not limited to gesture recognition of the first class of objects at a certain static moment; by adopting the modes of target detection and service screening, the accuracy of behavior and action recognition in a target scene is improved to a certain extent.
The following describes a behavior and action recognition apparatus, a processing device, a storage medium, and the like for executing the behavior and action recognition method provided in the present application, and specific implementation processes and technical effects thereof refer to relevant contents of the behavior and action recognition method, and are not described in detail below.
Fig. 7 is a schematic structural diagram of a behavior action recognition device according to an embodiment of the present invention, and as shown in fig. 7, the behavior action recognition device may include:
a determining module 701, configured to perform target detection on a two-dimensional image of a target scene, and determine a first class object in the target scene and a target second class object whose position matches the first class object;
an obtaining module 702, configured to perform skeleton key point detection on the first class of object in the two-dimensional image, to obtain a skeleton key point of the first class of object;
a detecting module 703, configured to detect whether there is an abnormal behavior for the target second class object in the first class object according to a position relationship between a preset skeleton key point in the skeleton key points of the first class object and the target second class object.
Optionally, the determining module 701 is further configured to perform target detection on the two-dimensional image to obtain a plurality of target objects in the target scene, where the plurality of target objects include: the first class of objects and at least one second class of objects; and determining a second class object matched with the region range from the at least one second class object as the target second class object according to the region range where the first class object is located in the target scene.
Optionally, the determining module 701 is further configured to divide the two-dimensional image of the target scene into a plurality of grids; taking the grid where the first type of object is located as the area range where the first type of object is located; determining a target grid matched with the grid where the first class object is located from the grid where the at least one second class object is located by adopting a preset mapping relation; and taking the second class object in the target grid as the target second class object.
Optionally, the determining module 701 is further configured to determine whether the area range is a preset behavior detection range of the first class of object; and if the area range is the preset behavior detection range, determining a second class object matched with the area range from the at least one second class object as the target second class object according to the area range.
Optionally, the apparatus further comprises:
the calculation module is used for calculating the motion change parameters of the first class of objects according to preset skeleton key points of the first class of objects detected within a preset time range;
the judging module is used for judging whether the first type of object has the action of the preset type or not according to the action change parameter;
the detecting module 703 is further configured to detect whether the first class object has an abnormal behavior for the target second class object according to a position relationship between the preset skeletal key point and the target second class object if the first class object has the action of the preset type.
Optionally, the calculating module is further configured to calculate an action change parameter of the preset portion in the first class of object according to the bone key point of the preset portion of the first class of object detected within the preset duration range.
Optionally, the detecting module 703 is further configured to determine a target frame of the target second class object, where the target frame is used to frame a target portion of the target second class object; if the first class object has the action of the preset type and the preset skeleton key point is in a target frame of the target second class object, determining that the first class object has abnormal behavior aiming at the target second class object; and if the first class object has the action of the preset type and the preset skeleton key point is not in the target frame of the target second class object, determining that the first class object does not have abnormal behavior aiming at the target second class object.
The above-mentioned apparatus is used for executing the method provided by the foregoing embodiment, and the implementation principle and technical effect are similar, which are not described herein again.
These above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Fig. 8 is a schematic structural diagram of a processing apparatus according to an embodiment of the present invention, and as shown in fig. 8, the processing apparatus may include: a processor 801 and a memory 802.
The memory 802 is used for storing programs, and the processor 801 calls the programs stored in the memory 802 to execute the above method embodiments. The specific implementation and technical effects are similar, and are not described herein again.
Optionally, the invention also provides a program product, for example a computer-readable storage medium, comprising a program which, when being executed by a processor, is adapted to carry out the above-mentioned method embodiments.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for behavior action recognition, the method comprising:
carrying out target detection on a two-dimensional image of a target scene, and determining a first class of object in the target scene and a target second class of object matched with the first class of object in position;
detecting skeleton key points of the first class of objects in the two-dimensional image to obtain the skeleton key points of the first class of objects;
and detecting whether abnormal behaviors aiming at the target second class object exist in the first class object or not according to the position relation between preset bone key points in the bone key points of the first class object and the target second class object.
2. The method of claim 1, wherein the performing target detection on the two-dimensional image of the target scene, determining a first type of object in the target scene, and determining a target second type of object that is matched with the first type of object in position comprises:
performing target detection on the two-dimensional image to obtain a plurality of target objects in the target scene, wherein the plurality of target objects comprise: the first class of objects and at least one second class of objects;
and determining a second class object matched with the region range from the at least one second class object as the target second class object according to the region range where the first class object is located in the target scene.
3. The method according to claim 2, wherein the determining, from the at least one second-class object, a second-class object that matches the region range as the target second-class object according to the region range where the first-class object is located in the target scene comprises:
dividing the two-dimensional image of the target scene into a plurality of grids;
taking the grid where the first type of object is located as the area range where the first type of object is located;
determining a target grid matched with the grid where the first class object is located from the grid where the at least one second class object is located by adopting a preset mapping relation;
and taking the second class object in the target grid as the target second class object.
4. The method according to claim 2, wherein the determining, from the at least one second-class object, a second-class object that matches the region range as the target second-class object according to the region range where the first-class object is located in the target scene comprises:
judging whether the area range is a preset behavior detection range of the first class of objects;
and if the area range is the preset behavior detection range, determining a second class object matched with the area range from the at least one second class object as the target second class object according to the area range.
5. The method according to claim 1, wherein before detecting whether there is abnormal behavior for the target second class object in the first class object according to a positional relationship between preset bone key points in the bone key points of the first class object and the target second class object, the method further comprises:
calculating an action change parameter of the first class of objects according to preset skeleton key points of the first class of objects detected within a preset time range;
judging whether the first type of object has a preset type of action according to the action change parameter;
the detecting whether the first class object has abnormal behavior aiming at the target second class object according to the position relationship between preset bone key points in the bone key points of the first class object and the target second class object comprises:
if the first class object has the action of the preset type, detecting whether the first class object has abnormal behaviors aiming at the target second class object or not according to the position relation between the preset skeleton key point and the target second class object.
6. The method according to claim 5, wherein the calculating the motion parameters of the first type of object according to the bone key points of the first type of object detected within a preset time range comprises:
and calculating the action change parameters of the preset part in the first class of objects according to the bone key points of the preset part of the first class of objects detected in the preset time range.
7. The method according to claim 5, wherein the detecting whether the first class object has abnormal behavior for the target second class object according to a position relationship between the preset skeletal key point and the target second class object if the first class object has the preset type of action comprises:
determining a target frame of the target second-class object, wherein the target frame is used for framing a target part of the target second-class object;
if the first class object has the action of the preset type and the preset skeleton key point is in a target frame of the target second class object, determining that the first class object has abnormal behavior aiming at the target second class object;
and if the first class object has the action of the preset type and the preset skeleton key point is not in the target frame of the target second class object, determining that the first class object does not have abnormal behavior aiming at the target second class object.
8. A behavioral action recognition apparatus, characterized in that the apparatus comprises:
the determining module is used for carrying out target detection on the two-dimensional image of a target scene, determining a first class of object in the target scene and a target second class of object matched with the first class of object in position;
the acquisition module is used for detecting the bone key points of the first class of objects in the two-dimensional image to obtain the bone key points of the first class of objects;
the detection module is used for detecting whether the first class object has abnormal behaviors aiming at the target second class object according to the position relation between preset bone key points in the bone key points of the first class object and the target second class object.
9. A server, comprising: a memory storing a computer program executable by the processor, and a processor implementing the behavioral action recognition method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that the storage medium has stored thereon a computer program which, when read and executed, implements the behavioral action recognition method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210135133.4A CN114202804A (en) | 2022-02-15 | 2022-02-15 | Behavior action recognition method and device, processing equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210135133.4A CN114202804A (en) | 2022-02-15 | 2022-02-15 | Behavior action recognition method and device, processing equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114202804A true CN114202804A (en) | 2022-03-18 |
Family
ID=80658918
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210135133.4A Pending CN114202804A (en) | 2022-02-15 | 2022-02-15 | Behavior action recognition method and device, processing equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114202804A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117315791A (en) * | 2023-11-28 | 2023-12-29 | 杭州华橙软件技术有限公司 | Bone action recognition method, device and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110070003A (en) * | 2019-04-01 | 2019-07-30 | 浙江大华技术股份有限公司 | The method and relevant apparatus that unusual checking and light stream autocorrelation determine |
| CN111259839A (en) * | 2020-01-20 | 2020-06-09 | 芯梯众和科技服务有限公司 | Target object behavior monitoring method, device, equipment, system and storage medium |
| CN112052774A (en) * | 2020-08-31 | 2020-12-08 | 杭州飞步科技有限公司 | Behavior detection method and device |
| CN113111747A (en) * | 2021-03-31 | 2021-07-13 | 新疆爱华盈通信息技术有限公司 | Abnormal limb behavior detection method, device, terminal and medium |
-
2022
- 2022-02-15 CN CN202210135133.4A patent/CN114202804A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110070003A (en) * | 2019-04-01 | 2019-07-30 | 浙江大华技术股份有限公司 | The method and relevant apparatus that unusual checking and light stream autocorrelation determine |
| CN111259839A (en) * | 2020-01-20 | 2020-06-09 | 芯梯众和科技服务有限公司 | Target object behavior monitoring method, device, equipment, system and storage medium |
| CN112052774A (en) * | 2020-08-31 | 2020-12-08 | 杭州飞步科技有限公司 | Behavior detection method and device |
| CN113111747A (en) * | 2021-03-31 | 2021-07-13 | 新疆爱华盈通信息技术有限公司 | Abnormal limb behavior detection method, device, terminal and medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117315791A (en) * | 2023-11-28 | 2023-12-29 | 杭州华橙软件技术有限公司 | Bone action recognition method, device and storage medium |
| CN117315791B (en) * | 2023-11-28 | 2024-02-20 | 杭州华橙软件技术有限公司 | Bone action recognition method, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107358149B (en) | Human body posture detection method and device | |
| CN110909611B (en) | Method and device for detecting attention area, readable storage medium and terminal equipment | |
| CN111209774B (en) | Target behavior recognition and display method, device, equipment and readable medium | |
| CN111401318B (en) | Action recognition method and device | |
| CN111612822B (en) | Object tracking method, device, computer equipment and storage medium | |
| JP2022542199A (en) | KEYPOINT DETECTION METHOD, APPARATUS, ELECTRONICS AND STORAGE MEDIA | |
| WO2017154655A1 (en) | Crowd type identification system, crowd type identification method, and storage medium for storing crowd type identification program | |
| CN111507285A (en) | Face attribute recognition method and device, computer equipment and storage medium | |
| TW202201275A (en) | Device and method for scoring hand work motion and storage medium | |
| CN114202804A (en) | Behavior action recognition method and device, processing equipment and storage medium | |
| CN108875901B (en) | Neural network training method and universal object detection method, device and system | |
| CN113780145A (en) | Sperm morphology detection method, sperm morphology detection device, computer equipment and storage medium | |
| CN116453226A (en) | Human body posture recognition method and device based on artificial intelligence and related equipment | |
| CN116433722A (en) | Target tracking method, electronic device, storage medium, and program product | |
| JP6405603B2 (en) | Information processing apparatus, information processing system, and program | |
| CN114332990A (en) | Emotion recognition method, device, equipment and medium | |
| US9536145B2 (en) | System and method for selecting features for identifying human activities in a human-computer interacting environment | |
| JP2022185872A (en) | Image processing device, image processing method and imaging apparatus | |
| CN108696722B (en) | Target monitoring method, system and device and storage medium | |
| CN115035129A (en) | Goods identification method and device, electronic equipment and storage medium | |
| JP6954070B2 (en) | Discrimination program, discrimination method and discrimination device | |
| CN115512131B (en) | Image detection method and training method of image detection model | |
| KR102599020B1 (en) | Method, program, and apparatus for monitoring behaviors based on artificial intelligence | |
| CN117475160A (en) | Target object following method, system and related device | |
| CN111460977B (en) | Cross-view personnel re-identification method, device, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220318 |