CN114969623A - Data processing method and system for lepidoptera insect motion capture - Google Patents
Data processing method and system for lepidoptera insect motion capture Download PDFInfo
- Publication number
- CN114969623A CN114969623A CN202210900490.5A CN202210900490A CN114969623A CN 114969623 A CN114969623 A CN 114969623A CN 202210900490 A CN202210900490 A CN 202210900490A CN 114969623 A CN114969623 A CN 114969623A
- Authority
- CN
- China
- Prior art keywords
- optical marker
- motion capture
- wing
- lepidoptera
- insect
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000033001 locomotion Effects 0.000 title claims abstract description 176
- 241000255777 Lepidoptera Species 0.000 title claims abstract description 103
- 238000003672 processing method Methods 0.000 title claims abstract description 35
- 230000003287 optical effect Effects 0.000 claims abstract description 139
- 239000003550 marker Substances 0.000 claims abstract description 126
- 238000012217 deletion Methods 0.000 claims abstract description 47
- 230000037430 deletion Effects 0.000 claims abstract description 47
- 238000000034 method Methods 0.000 claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 34
- 238000005457 optimization Methods 0.000 claims abstract description 16
- 238000012360 testing method Methods 0.000 claims abstract description 13
- 238000004458 analytical method Methods 0.000 claims abstract description 11
- 238000002474 experimental method Methods 0.000 claims abstract description 11
- 238000009499 grossing Methods 0.000 claims abstract description 11
- 230000005021 gait Effects 0.000 claims abstract description 10
- 210000000038 chest Anatomy 0.000 claims description 32
- 241000238631 Hexapoda Species 0.000 claims description 30
- 238000004364 calculation method Methods 0.000 claims description 9
- 210000000481 breast Anatomy 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 3
- 230000006870 function Effects 0.000 description 6
- 230000036544 posture Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 244000241796 Christia obcordata Species 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 241000382353 Pupa Species 0.000 description 1
- 238000010009 beating Methods 0.000 description 1
- 230000003592 biomimetic effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 238000013481 data capture Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000010561 standard procedure Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000008685 targeting Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Computational Mathematics (AREA)
- Pure & Applied Mathematics (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Algebra (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention provides a data processing method and a data processing system for lepidoptera insect motion capture, wherein the method comprises the following steps: testing the maximum sensing distance between the camera and the optical Marker, setting a motion capture space according to the value of the maximum sensing distance, uniformly setting a plurality of cameras in the motion capture space, and combining a gait analysis system to form a motion capture system; testing and analyzing the optical markers with different sizes, selecting the optical Marker with the correct model, and then performing an experiment to obtain the movement data of the lepidoptera insects; and sequentially performing deletion marker completion, noise track smoothing and key parameter optimization processing on the movement data of the lepidoptera insects to obtain a lepidoptera insect movement data set meeting the requirements. The method provided by the invention can accurately acquire the flight motion data of the butterfly and improve the accuracy of the data.
Description
Technical Field
The invention relates to the technical field of computer motion capture data processing, in particular to a data processing method and system for lepidoptera insect motion capture.
Background
As is well known, butterflies have an unusual and beautiful flying attitude and are therefore used in the fields of movies, games, VR/AR, virtual worlds, and biomimetic robots. Heretofore, researchers have used various experimental methods to capture the movement of butterflies. For example, researchers have captured the aerodynamic forces generated by an insect flapping its wings through a wind tunnel or captured the butterfly's flight path through the use of multiple cameras. However, the above-described methods are either too expensive (e.g., by way of wind tunnel experiments) or not sufficiently adaptable to applications in the field of graphics (e.g., the flight trajectory of a butterfly alone is generally not sufficient to create realistic butterfly animations). In fact, because butterflies have unique flying styles and very small sizes, it is a great challenge to capture the detailed postures of various parts of the body (such as wings and chest) during the flying process of butterflies.
Meanwhile, motion capture has become a standard method for obtaining precise movements of humans and animals in computer graphics and animation applications. Thus, there are many published large-scale human motion datasets, which have greatly facilitated the explosive growth of motion capture-based research in recent years. However, due to the above mentioned challenges of capturing detailed movements of butterflies, there is currently no published butterfly motion data set (including movements of wings and chest) available for scientific research or engineering applications, especially in the field of computer graphics and computer animation applications, and due to the lack of available butterfly motion data sets, which not only limits the application of butterfly simulation results to various entertainment and VR applications, but also hinders the scientific community's progress in research related to butterfly motion.
In view of the above, there is a need to provide a data processing method and system for lepidopteran insect motion capture to solve the above-mentioned technical problems.
Disclosure of Invention
In view of the above, the main objective of the present invention is to provide a data processing method and system for lepidopteran insect motion capture, which is used to solve the above technical problems.
The embodiment of the invention provides a data processing method for lepidoptera insect motion capture, wherein the lepidoptera insect at least comprises butterflies, and the method comprises the following steps:
step one, setting experimental equipment:
testing the maximum sensing distance between a camera and an optical Marker, setting a motion capture space according to the value of the maximum sensing distance, uniformly setting a plurality of cameras in the motion capture space, combining the plurality of cameras with a gait analysis system to form a motion capture system, wherein the motion capture system is used for capturing and obtaining the motion data of lepidoptera insects;
selecting and placing an optical Marker;
testing and analyzing optical markers with different sizes, selecting an optical Marker with a correct model, symmetrically arranging the optical markers with the correct model on the front and back surfaces of the front wing of the lepidoptera insect, arranging the optical markers with the correct model on the chest of the lepidoptera insect, and performing a motion capture experiment of the lepidoptera insect to obtain motion data of the lepidoptera insect; the optical Marker arranged on the front wing is used for acquiring flapping motion data of wings of the lepidoptera insect, and the optical Marker arranged on the chest is used for acquiring body motion track information of the lepidoptera insect;
step three, processing the motion data:
after obtaining the movement data of the lepidoptera insects, sequentially performing deletion mark completion, noise track smoothing and key parameter optimization processing on the movement data of the lepidoptera insects to finally obtain a lepidoptera insect movement data set meeting the requirements; wherein in the step of deletion Marker completion, the deletion markers include chest optical Marker deletion, two-wing optical Marker deletion and one-side wing optical Marker deletion.
The invention provides a data processing method for lepidoptera insect motion capture, which is characterized in that a motion capture system is formed by combining a plurality of cameras and a gait analysis system and is used for accurately capturing motion data of lepidoptera insects; after obtaining the movement data of the lepidoptera insects, sequentially performing deletion mark completion, noise track smoothing and key parameter optimization processing on the movement data of the lepidoptera insects, and thus obtaining a lepidoptera insect movement data set meeting the requirements finally. The data processing method for lepidoptera insect motion capture can accurately acquire the flight motion data of butterflies, and improves the accuracy of the data.
The data processing method for lepidoptera insect motion capture includes that in the first step, the video cameras are ualisys cameras, the maximum sensing distance is 1.5m, the motion capture space is a cylindrical motion capture space, the radius of the cylindrical motion capture space is 1.0m, the height of the cylindrical motion capture space is 1.0m, 7 video cameras are arranged in the motion capture space, the full resolution of the video cameras is 600 ten thousand pixels, the shooting speed is 450 frames/second, and the capture frequency is 500 fps.
The data processing method for lepidoptera insect motion capture is characterized in that the lepidoptera insect is a butterfly, and in the third step, for the situation that a single-side wing optical Marker is absent, the corresponding deficiency mark completion method comprises the following steps:
using auxiliary point coordinates
Constructing a reference vertical plane of the body posture of the butterfly, and determining a normal vector of the reference vertical plane according to the reference vertical plane
Wherein the turning angle of the butterfly body on the reference vertical plane is zero;
according to the normal vector of the reference vertical plane
Angle of rotation with respect to the roll-over angle
Calculating to obtain the normal vector of the actual attitude plane
；
Obtaining the normal vector of the actual attitude plane according to calculation
Symmetrically filling the optical Marker on the wing on one side of the butterfly on the wing on the missing side to complete completion;
wherein,
is shown as
The coordinates of the auxiliary points in the frame image,
is shown as
The normal vector of the reference vertical plane in the frame image,
is shown as
And (4) a normal vector of an actual attitude plane in the frame image.
The data processing method for lepidoptera insect motion capture comprises the steps of assisting point coordinates
The corresponding calculation formula is expressed as:
wherein,
is shown as
The coordinates of the optical Marker of the chest of the butterfly in the frame image,
represents a unit vector in the positive direction of the Y-axis in the three-dimensional coordinate system,
is a scalar quantity.
The data processing method for lepidopteran insect motion capture is described in
Coordinates of left wing missing optical Marker in frame image
The calculation formula of (2) is as follows:
wherein,
is shown as
Coordinates of the optical Marker with missing left wings in the frame image,
is shown as
The coordinates of the optical Marker of the right wing in the frame image,
is the distance between the coordinates of the optical Marker of the right wing and the actual pose plane.
The data processing method for lepidoptera insect motion capture is characterized in that the normal vector of the actual attitude plane
Is expressed as:
wherein,
is shown as
The flip angle between the reference vertical plane and the actual pose plane in the frame image.
The data processing method for lepidopteran insect motion capture is characterized in that in the step three, the order of the third step is
Under the condition that the chest optical Marker in the frame image is lost, the corresponding missing mark completion method comprises the following steps:
to the first
The optical Marker of the breast in the adjacent frame image of the frame image is subjected to cubic spline interpolation directly at the third
Filling a missing chest optical Marker in the frame image;
in the third step, for
Two-wing optical Marker missing in frame imageIn the case of (2), the corresponding deletion marker complementing method comprises the steps of:
to the first
The optical Marker of the wing in the adjacent frame image of the frame image is subjected to cubic spline interpolation to directly obtain the optical Marker of the wing in the adjacent frame image
And filling the missing two-wing optical Marker in the frame image.
The data processing method for lepidopteran insect motion capture is characterized in that in the step three, in the step of key parameter optimization processing, the key parameter comprises a step size
Polynomial order
And sliding window size
；
The automatic optimization method of the key parameters comprises the following steps:
selecting part of completely captured butterfly motion data as standard data;
defining an error between the standard data and the acquired lepidopteran insect movement data
；
According to a defined error
Constructing to obtain an objective function, and calculating to obtain a fitness value according to the objective function
Wherein the fitness value is used for measuring the quality of different combinationsAmount of the compound (A).
The data processing method for lepidopteran insect motion capture is characterized in that errors exist
The corresponding expression is:
wherein,
indicating belonging to either the left or right wing,
is shown as
Coordinates of an optical Marker captured on the left wing or the right wing in the frame image,
is shown as
Coordinates of the optical Marker after completion and noise reduction processing on the left wing or the right wing in the frame image,
representing the left side wing of the butterfly,
representing the right wing of the butterfly;
the expression of the objective function is:
wherein,
the value of the fitness value is represented,
representing the total number of standard data frames used.
The present invention also proposes a data processing system for motion capture of lepidopteran insects, wherein said lepidopteran insects comprise at least butterflies, said system comprising:
a first processing module to:
testing the maximum sensing distance between a camera and an optical Marker, setting a motion capture space according to the value of the maximum sensing distance, uniformly setting a plurality of cameras in the motion capture space, combining the plurality of cameras with a gait analysis system to form a motion capture system, wherein the motion capture system is used for capturing and obtaining the motion data of lepidoptera insects;
a second processing module to:
testing and analyzing optical markers with different sizes, selecting an optical Marker with a correct model, symmetrically arranging the optical markers with the correct model on the front and back surfaces of the front wing of the lepidoptera insect, arranging the optical markers with the correct model on the chest of the lepidoptera insect, and performing a motion capture experiment of the lepidoptera insect to obtain motion data of the lepidoptera insect; the optical Marker arranged on the front wing is used for acquiring flapping motion data of wings of the lepidoptera insect, and the optical Marker arranged on the chest is used for acquiring body motion track information of the lepidoptera insect;
a data processing module to:
after obtaining the movement data of the lepidoptera insects, sequentially performing deletion mark completion, noise track smoothing and key parameter optimization processing on the movement data of the lepidoptera insects to finally obtain a lepidoptera insect movement data set meeting the requirements; wherein in the step of deletion Marker completion, the deletion markers include chest optical Marker deletion, two-wing optical Marker deletion and one-side wing optical Marker deletion.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a flow chart of a data processing method for lepidopteran insect motion capture according to the present invention;
FIG. 2 is a schematic view of a virtual space displayed in a gait analysis system;
FIG. 3 is a schematic diagram of an optical Marker placed on the back side of a butterfly;
FIG. 4 is a schematic view of an optical Marker positioned on the ventral side of a butterfly;
FIG. 5 is a schematic diagram of the automatic filling of the missing left wing optical Marker;
FIG. 6 is a block diagram of a data processing system for lepidopteran insect motion capture in accordance with the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
These and other aspects of embodiments of the invention will be apparent with reference to the following description and attached drawings. In the description and drawings, particular embodiments of the invention have been disclosed in detail as being indicative of some of the ways in which the principles of the embodiments of the invention may be practiced, but it is understood that the scope of the embodiments of the invention is not limited correspondingly. On the contrary, the embodiments of the invention include all changes, modifications and equivalents coming within the spirit and terms of the claims appended hereto.
Referring to fig. 1, the present invention provides a data processing method for capturing the motion of lepidopteran insects, wherein the lepidopteran insects at least comprise butterflies, the method comprising the steps of:
and S101, setting experimental equipment.
Step S101 specifically includes:
the maximum sensing distance between the camera and the optical Marker is tested, a motion capture space is set according to the value of the maximum sensing distance, a plurality of cameras are uniformly arranged in the motion capture space, the plurality of cameras are combined with a gait analysis system (QTM software) to form a motion capture system, and the motion capture system is used for capturing and obtaining the motion data of lepidoptera insects. It should be noted that the optical Marker refers to an optical mark.
Specifically, the video camera is a ualisys camera, and the maximum sensing distance is 1.5 m. In this embodiment, the motion capture volume is a cylindrical motion capture volume. Wherein the designed cylindrical motion capture volume has a radius of 1.0m and a height of 1.0 m. As shown in fig. 2, 7 cameras are uniformly arranged in the motion capture space for capturing motion data of the butterfly. In addition, a curtain is placed on the floor to reduce the interference of background light, and a gauze net is used to prevent the butterfly from escaping. In this embodiment, the full resolution of the camera is 600 ten thousand pixels, the shooting speed is 450 frames/second, and the capture frequency is 500 fps. Wherein the capture frequency is set at 500 fps, two orders of magnitude higher than the butterfly's beating frequency (about 11 Hz).
And S102, selecting and placing an optical Marker.
Step S102 specifically includes:
the method comprises the steps of carrying out test analysis on optical markers with different sizes, selecting the optical Marker with the correct model, symmetrically arranging the optical markers with the correct model on the front and back surfaces of the front wing of the lepidoptera insect, arranging the optical markers with the correct model on the chest of the lepidoptera insect, and then carrying out a motion capture experiment on the lepidoptera insect to obtain motion data of the lepidoptera insect. The optical Marker arranged on the front wing is used for acquiring flapping motion data of wings of the lepidoptera insect, and the optical Marker arranged on the chest is used for acquiring body motion track information of the lepidoptera insect.
Specifically, the surface of the optical Marker is coated with a special material, so that infrared light can be effectively reflected and sensed by the camera. In order to reduce the influence on the butterfly flight as much as possible, the parameters corresponding to the optical Marker used in this embodiment are: the shape is hemispherical, the diameter is 2.5mm, the mass is 0.004g, and the ratio of the optical Marker sphere to the weight of the butterfly is 1.1%. In the invention, the butterfly used in the experiment is hatched from a butterfly pupa, and the variety of the butterfly is a Paenilia leucocephala.
Further, for the specific setup of the optical Marker, as shown in fig. 3 and 4, a total of 5 markers are placed on the butterfly, 4 of which are in the front wing (placed on the dorsal and ventral sides, respectively) and 1 in the chest. The optical markers are symmetrically adhered to two sides of the wing, when one side of the optical Marker disappears from the visual field of the camera in the shooting process, the other side of the optical Marker can replace the optical Marker, and therefore the problem of incomplete capture is effectively avoided. It can be understood that the optical Marker arranged on the front wing can reflect the flapping motion of the butterfly wing, and the optical Marker arranged on the chest is mainly used for providing the body motion track information of the butterfly.
And S103, processing the motion data.
After obtaining the movement data of the lepidoptera insects, sequentially performing deletion mark completion, noise track smoothing and key parameter optimization processing on the movement data of the lepidoptera insects to finally obtain a lepidoptera insect movement data set meeting the requirements.
Wherein in the step of deletion Marker completion, the deletion markers include chest optical Marker deletion, two-wing optical Marker deletion and one-side wing optical Marker deletion. Filling up the deletion marker:
in the butterfly motion capture experiment, three possible Marker deletion conditions exist, (i) chest optical Marker deletion; (ii) optical Marker missing of a single-side wing; (iii) two-wing optical Marker is missing. Therefore, the missing completion scheme will be described in detail below.
(1) For the condition of single-side wing optical Marker deletion, the corresponding deletion mark completion method comprises the following steps:
s1031, utilizing auxiliary point coordinate
Constructing a reference vertical plane of the butterfly body posture, and determining a normal vector of the reference vertical plane according to the reference vertical plane
Wherein the turning angle of the butterfly body on the reference vertical plane is zero.
For the first
And (4) frame images, defining a reference vertical plane for the butterfly, and indicating that the turning angle of the butterfly body is zero. As shown in part (a) of figure 5,
is shown as
The coordinates of the optical Marker of the chest of the butterfly in the frame image,
is shown as
The coordinates of the optical Marker of the breast on the frame image,
the step size is indicated. Furthermore, according to the coordinates
And the coordinates
The body direction of the butterfly can be determined. .
Auxiliary point coordinates
The corresponding calculation formula is expressed as:
wherein,
represents a unit vector in the positive direction of the Y-axis in the three-dimensional coordinate system,
is a scalar quantity. In the present embodiment, scalar quantity
Set to 10. Using auxiliary point coordinates
A reference vertical plane with a zero roll angle in the butterfly body posture can be constructed, and a corresponding normal vector is determined and obtained according to the reference vertical plane
。
S1032, according to the normal vector of the reference vertical plane
Angle of rotation with respect to the roll-over angle
Calculating to obtain the normal vector of the actual attitude plane
。
As shown in part (b) of figure 5,
the coordinates of the optical Marker representing the left wing of the butterfly,
the coordinates of the optical Marker representing the right wing of the butterfly,
to represent
And
the middle point between the two points is,
is shown as
And (4) a normal vector of an actual attitude plane in the frame image. Angle between reference vertical plane and actual attitude plane
Represents the posture of the butterfly from a roll angle of 0
The angle rotation becomes the actual posture. I.e. the roll angle
Can be calculated by calculating the normal vector of the reference vertical plane
And the normal vector of the actual attitude plane
The included angle between the two is obtained.
Normal vector of actual attitude plane
Is expressed as:
wherein,
is shown as
A flip angle between the reference vertical plane and the actual pose plane in the frame image.
S1033, obtaining the normal vector of the plane of the actual attitude according to calculation
And symmetrically filling the optical Marker on the wing on one side of the butterfly on the wing on the missing side to complete the completion.
If the optical Marker on one side of the butterfly is lost, the reference vertical plane normal vector can be used based on the assumption that the butterfly's two sides are symmetrical
And the normal vector of the actual attitude plane
And filling the wing of the missing optical Marker with the optical Marker on the other side wing. In order not to lose generality, in this embodiment, the optical Marker of the butterfly left wing is assumed to be missing. First, the
The optical Marker (see fig. 5, part (c)) with the missing left wing in the frame image can be calculated by the following formula:
in the first place
Coordinates of left wing missing optical Marker in frame image
The calculation formula of (2) is as follows:
wherein,
is shown as
Coordinates of the optical Marker with missing left wings in the frame image,
is shown as
The coordinates of the optical Marker of the right wing in the frame image,
is the distance between the coordinates of the optical Marker of the right wing and the actual pose plane.
As a supplement, to
Under the condition that the chest optical Marker in the frame image is lost, the corresponding missing mark completion method comprises the following steps:
to the first
The optical Marker of the breast in the adjacent frame image of the frame image is subjected to cubic spline interpolation directly at the third
Filling a missing chest optical Marker in the frame image;
for the first
Under the condition that two wings of optical markers in a frame image are lost, the corresponding method for complementing the lost mark comprises the following steps:
to the first
The optical Marker of the wing in the adjacent frame image of the frame image is subjected to cubic spline interpolation to directly perform the third-order spline interpolation
And filling the missing two-wing optical Marker in the frame image.
Smoothing the noise track:
and denoising the captured butterfly motion data by adopting a Savitzky-Golay (S-G) filter algorithm. The core idea of the S-G filter is in the sliding window size
Performing polynomial order on multiple continuous frames
So as to obtain smooth results and not influence the original movement trend. The trueness of the smoothing result depends to a large extent on its key parameters
And the size of the sliding window
Selection of (2).
Optimizing key parameters:
in the step of the key parameter optimization process, the key parameter includes a step size
Polynomial order
And sliding window size
；
The automatic optimization method of the key parameters comprises the following steps:
selecting part of completely captured butterfly motion data as standard data;
defining an error between the standard data and the acquired lepidopteran insect movement data
；
According to a defined error
Constructing to obtain an objective function, and calculating to obtain a fitness value according to the objective function
Wherein the fitness value is used to measure the quality of different combinations.
Wherein, errors
The corresponding expression is:
wherein,
indicating belonging to either the left or right wing,
is shown as
Left or right wings in frame imageThe coordinates of the optical Marker captured thereon,
is shown as
Coordinates of the optical Marker after completion and noise reduction processing on the left wing or the right wing in the frame image,
representing the left side wing of the butterfly,
representing the right wing of the butterfly;
the expression of the objective function is:
wherein,
the value of the fitness value is represented,
representing the total number of standard data frames used.
In genetic algorithms, fitness values
For measuring the quality of different combinations. Specifically, 20 individuals were used per generation, and the propagation parameters used were a mutation probability of 2% and a crossover probability of 92%. Each optimization is designed to be 100 iterations. To increase the fitness value
For the purpose of targeting, iteration obtains the optimal values of the three parameters.
Furthermore, compared with two methods (cubic spline interpolation method and extreme gradient lifting method) in the prior art, the data processing method for lepidoptera insect motion capture can more accurately fill the motion data of butterflies, and is very suitable for butterfly motion data capture mainly because the motion processing method fully utilizes the space characteristic of butterfly flight.
In addition, the present invention utilizes errors
The processing method of butterfly motion data is compared quantitatively with the two methods in the prior art (cubic spline interpolation method and extreme gradient lifting method). The average errors generated by the three methods after processing the motion data are respectively 18.04 mm (cubic spline interpolation), 37.95 mm (extreme gradient lifting method) and 4.11 mm (method of the invention). From a comparison of the above average errors it can be seen that: compared with the classic cubic spline interpolation method and the extreme gradient lifting method, the movement processing method designed aiming at butterfly movement processing has obvious advantages.
Furthermore, by the data processing method for lepidoptera insect motion capture, a butterfly motion data set can be obtained, and the following characteristics can be found by carrying out quantitative analysis on the butterfly motion data:
the frequency of the butterfly flapping wing is kept between 8 and 12 Hz;
the amplitude change of wing flapping is obvious, and the average amplitude is 154;
the fluctuation frequency of the butterfly body is similar to the flapping frequency of the butterfly wings;
the average amplitude of the bulk wave is 2.82 mm;
the average flying speed of the butterfly is 1.20 m/s.
The invention provides a data processing method for lepidoptera insect motion capture, which is characterized in that a motion capture system is formed by combining a plurality of cameras and a gait analysis system and is used for accurately capturing motion data of lepidoptera insects; after obtaining the movement data of the lepidoptera insects, sequentially performing deletion mark completion, noise track smoothing and key parameter optimization processing on the movement data of the lepidoptera insects, and thus obtaining a lepidoptera insect movement data set meeting the requirements finally. The data processing method for lepidoptera insect motion capture can accurately acquire the flight motion data of butterflies, and improves the accuracy of the data.
Referring to fig. 6, the present invention further provides a data processing system for capturing the movement of lepidopteran insects, wherein the lepidopteran insects at least comprise butterflies, the system comprising:
a first processing module to:
testing the maximum sensing distance between a camera and an optical Marker, setting a motion capture space according to the value of the maximum sensing distance, uniformly setting a plurality of cameras in the motion capture space, combining the plurality of cameras with a gait analysis system to form a motion capture system, wherein the motion capture system is used for capturing and obtaining the motion data of lepidoptera insects;
a second processing module to:
testing and analyzing optical markers with different sizes, selecting an optical Marker with a correct model, symmetrically arranging the optical markers with the correct model on the front and back surfaces of the front wing of the lepidoptera insect, arranging the optical markers with the correct model on the chest of the lepidoptera insect, and performing a motion capture experiment of the lepidoptera insect to obtain motion data of the lepidoptera insect; the optical Marker arranged on the front wing is used for acquiring flapping motion data of wings of the lepidoptera insect, and the optical Marker arranged on the chest is used for acquiring body motion track information of the lepidoptera insect;
a data processing module to:
after obtaining the movement data of the lepidoptera insects, sequentially performing deletion mark completion, noise track smoothing and key parameter optimization processing on the movement data of the lepidoptera insects to finally obtain a lepidoptera insect movement data set meeting the requirements; in the step of completing the deletion markers, the deletion markers comprise chest optical Marker deletion, two-wing optical Marker deletion and single-wing optical Marker deletion.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent should be subject to the appended claims.
Claims (10)
1. A data processing method for lepidopteran insect motion capture, said lepidopteran insect comprising at least a butterfly, said method comprising the steps of:
step one, setting experimental equipment:
testing the maximum sensing distance between a camera and an optical Marker, setting a motion capture space according to the value of the maximum sensing distance, uniformly setting a plurality of cameras in the motion capture space, combining the plurality of cameras with a gait analysis system to form a motion capture system, wherein the motion capture system is used for capturing and obtaining the motion data of lepidoptera insects;
selecting and placing an optical Marker;
testing and analyzing optical markers with different sizes, selecting an optical Marker with a correct model, symmetrically arranging the optical markers with the correct model on the front and back surfaces of the front wing of the lepidoptera insect, arranging the optical markers with the correct model on the chest of the lepidoptera insect, and performing a motion capture experiment of the lepidoptera insect to obtain motion data of the lepidoptera insect; the optical Marker arranged on the front wing is used for acquiring flapping motion data of wings of the lepidoptera insect, and the optical Marker arranged on the chest is used for acquiring body motion track information of the lepidoptera insect;
step three, processing the motion data:
after obtaining the movement data of the lepidoptera insects, sequentially performing deletion mark completion, noise track smoothing and key parameter optimization processing on the movement data of the lepidoptera insects to finally obtain a lepidoptera insect movement data set meeting the requirements; wherein in the step of deletion Marker completion, the deletion markers include chest optical Marker deletion, two-wing optical Marker deletion and one-side wing optical Marker deletion.
2. The data processing method for lepidopteran insect motion capture according to claim 1, wherein in said first step, said video camera is a ualisys camera, said maximum sensing distance is 1.5m, said motion capture space is a cylindrical motion capture space, the radius of said cylindrical motion capture space is 1.0m, the height of said cylindrical motion capture space is 1.0m, 7 video cameras are arranged in said motion capture space, the full resolution of said video cameras is 600 ten thousand pixels, the shooting speed is 450 frames/second, and the capture frequency is 500 fps.
3. The data processing method for lepidoptera insect motion capture of claim 2, wherein said lepidoptera insect is a butterfly, and in said third step, the corresponding deletion Marker complementing method comprises the following steps for the case of a single-sided wing optical Marker deletion:
using auxiliary point coordinates
Constructing a reference vertical plane of the butterfly body posture, and determining a normal vector of the reference vertical plane according to the reference vertical plane
Wherein the turning angle of the butterfly body on the reference vertical plane is zero;
according to the normal vector of the reference vertical plane
Angle of rotation with respect to the roll-over angle
Calculating to obtain the normal vector of the actual attitude plane
；
Obtaining the normal vector of the actual attitude plane according to calculation
Symmetrically filling the optical Marker on the wing on one side of the butterfly on the wing on the missing side to complete completion;
wherein,
is shown as
The coordinates of the auxiliary points in the frame image,
is shown as
The normal vector of the reference vertical plane in the frame image,
is shown as
And (4) a normal vector of an actual attitude plane in the frame image.
4. The data processing method for lepidopteran insect motion capture of claim 3, wherein the auxiliary point coordinates
The corresponding calculation formula is expressed as:
wherein,
is shown as
The coordinates of the optical Marker of the chest of the butterfly in the frame image,
representing a unit vector in the positive direction of the Y-axis in a three-dimensional coordinate system,
is a scalar quantity.
5. The data processing method for lepidopteran insect motion capture of claim 4, wherein the first step is performed
Coordinates of left wing missing optical Marker in frame image
The calculation formula of (2) is as follows:
wherein,
is shown as
Coordinates of the optical Marker with missing left wings in the frame image,
denotes the first
The coordinates of the optical Marker of the right wing in the frame image,
is the distance between the coordinates of the optical Marker of the right wing and the actual pose plane.
6. The data processing method for lepidopteran insect motion capture of claim 5, wherein an actual attitude plane normal vector
Is expressed as:
wherein,
is shown as
The flip angle between the reference vertical plane and the actual pose plane in the frame image.
7. The data processing method for lepidopteran insect motion capture of claim 6, wherein in said third step, for step three
Under the condition that the chest optical Marker in the frame image is lost, the corresponding missing mark completion method comprises the following steps:
to the first
The optical Marker of the breast in the adjacent frame image of the frame image is subjected to cubic spline interpolation directly at the third
Filling a missing chest optical Marker in the frame image;
in the third step, for
Under the condition that two wings of optical markers in a frame image are lost, the corresponding method for complementing the lost mark comprises the following steps:
to the first
The optical Marker of the wing in the adjacent frame image of the frame image is subjected to cubic spline interpolation to directly obtain the optical Marker of the wing in the adjacent frame image
And filling the missing two-wing optical Marker in the frame image.
8. The data processing method for lepidopteran insect motion capture of claim 7, wherein in said step three, in the step of critical parameter optimization processing, the critical parameter comprises a step size
Polynomial order
And sliding window size
；
The automatic optimization method of the key parameters comprises the following steps:
selecting part of completely captured butterfly motion data as standard data;
defining an error between the standard data and the acquired lepidopteran insect movement data
；
According to a defined error
Constructing to obtain an objective function, and calculating to obtain a fitness value according to the objective function
Wherein the fitness value is used to measure the quality of different combinations.
9. The data processing method for lepidopteran insect motion capture of claim 8, wherein the error is a function of a distance between the insect and the target location
The corresponding expression is:
wherein,
indicating belonging to either the left or right wing,
is shown as
Coordinates of an optical Marker captured on the left wing or the right wing in the frame image,
is shown as
Coordinates of the optical Marker after completion and noise reduction processing on the left wing or the right wing in the frame image,
representing the left side wing of the butterfly,
representing the right wing of the butterfly;
the expression of the objective function is:
wherein,
the value of the fitness value is represented,
indicating the total number of standard data frames used.
10. A data processing system for motion capture of lepidopteran insects, said lepidopteran insects comprising at least butterflies, said system comprising:
a first processing module to:
testing the maximum sensing distance between a camera and an optical Marker, setting a motion capture space according to the value of the maximum sensing distance, uniformly setting a plurality of cameras in the motion capture space, combining the plurality of cameras with a gait analysis system to form a motion capture system, wherein the motion capture system is used for capturing and obtaining the motion data of lepidoptera insects;
a second processing module to:
testing and analyzing optical markers with different sizes, selecting an optical Marker with a correct model, symmetrically arranging the optical markers with the correct model on the front and back surfaces of the front wing of the lepidoptera insect, arranging the optical markers with the correct model on the chest of the lepidoptera insect, and performing a motion capture experiment of the lepidoptera insect to obtain motion data of the lepidoptera insect; the optical Marker arranged on the front wing is used for acquiring flapping motion data of wings of the lepidoptera insect, and the optical Marker arranged on the chest is used for acquiring body motion track information of the lepidoptera insect;
a data processing module to:
after obtaining the movement data of the lepidoptera insects, sequentially performing deletion mark completion, noise track smoothing and key parameter optimization processing on the movement data of the lepidoptera insects to finally obtain a lepidoptera insect movement data set meeting the requirements; wherein in the step of deletion Marker completion, the deletion markers include chest optical Marker deletion, two-wing optical Marker deletion and one-side wing optical Marker deletion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210900490.5A CN114969623B (en) | 2022-07-28 | 2022-07-28 | Data processing method and system for lepidoptera insect motion capture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210900490.5A CN114969623B (en) | 2022-07-28 | 2022-07-28 | Data processing method and system for lepidoptera insect motion capture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114969623A true CN114969623A (en) | 2022-08-30 |
| CN114969623B CN114969623B (en) | 2022-10-25 |
Family
ID=82968612
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210900490.5A Active CN114969623B (en) | 2022-07-28 | 2022-07-28 | Data processing method and system for lepidoptera insect motion capture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114969623B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115362990A (en) * | 2022-09-22 | 2022-11-22 | 中国农业科学院植物保护研究所 | Insect flapping frequency measuring device and method |
| CN116461714A (en) * | 2023-05-17 | 2023-07-21 | 北京科技大学 | Butterfly-simulated ornithopter flight parameter determination method based on dynamic capturing system |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070183631A1 (en) * | 2006-02-06 | 2007-08-09 | Beijing University Of Aeronautics And Astronautics | Methods and apparatus for measuring the flapping deformation of insect wings |
| US20080152192A1 (en) * | 2005-07-07 | 2008-06-26 | Ingenious Targeting Laboratory, Inc. | System For 3D Monitoring And Analysis Of Motion Behavior Of Targets |
| CN104766358A (en) * | 2015-04-08 | 2015-07-08 | 浙江大学 | Optimization method of flying insect moving model parameters based on statistic estimation |
| CN106600627A (en) * | 2016-12-07 | 2017-04-26 | 成都通甲优博科技有限责任公司 | Rigid body motion capturing method and system based on mark point |
| CN106846462A (en) * | 2016-12-30 | 2017-06-13 | 北京农业信息技术研究中心 | Insect identifying device and method based on three-dimensional simulation |
| CN106940734A (en) * | 2017-04-24 | 2017-07-11 | 南京信息工程大学 | A kind of Migrating Insects monitor recognition methods and device in the air |
| US20180089876A1 (en) * | 2016-09-29 | 2018-03-29 | Sony Interactive Entertainment America, LLC | System to identify and use markers for motion capture |
| CN108830861A (en) * | 2018-05-28 | 2018-11-16 | 上海大学 | A kind of hybrid optical motion capture method and system |
| CN110260845A (en) * | 2019-06-10 | 2019-09-20 | 北京航空航天大学 | A kind of insect motion acquisition device based on three high-speed cameras |
| CN110753930A (en) * | 2018-12-29 | 2020-02-04 | 深圳市瑞立视多媒体科技有限公司 | Fitting processing method of three-dimensional trajectory data and optical motion capturing method |
| CN112581574A (en) * | 2020-12-25 | 2021-03-30 | 江苏环影动漫文化有限公司 | Human body motion capture system and method |
| US20210166390A1 (en) * | 2019-11-28 | 2021-06-03 | South China Normal University | Method, apparatus and storage medium for analyzing insect feeding behavior |
| CN113160367A (en) * | 2021-05-11 | 2021-07-23 | 华东交通大学 | Lepidoptera insect model simulation and control method, system and readable storage medium |
| CN113592898A (en) * | 2021-05-13 | 2021-11-02 | 黑龙江省科学院智能制造研究所 | Method for reconstructing missing mark in motion capture |
-
2022
- 2022-07-28 CN CN202210900490.5A patent/CN114969623B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080152192A1 (en) * | 2005-07-07 | 2008-06-26 | Ingenious Targeting Laboratory, Inc. | System For 3D Monitoring And Analysis Of Motion Behavior Of Targets |
| US20070183631A1 (en) * | 2006-02-06 | 2007-08-09 | Beijing University Of Aeronautics And Astronautics | Methods and apparatus for measuring the flapping deformation of insect wings |
| CN104766358A (en) * | 2015-04-08 | 2015-07-08 | 浙江大学 | Optimization method of flying insect moving model parameters based on statistic estimation |
| US20180089876A1 (en) * | 2016-09-29 | 2018-03-29 | Sony Interactive Entertainment America, LLC | System to identify and use markers for motion capture |
| CN106600627A (en) * | 2016-12-07 | 2017-04-26 | 成都通甲优博科技有限责任公司 | Rigid body motion capturing method and system based on mark point |
| CN106846462A (en) * | 2016-12-30 | 2017-06-13 | 北京农业信息技术研究中心 | Insect identifying device and method based on three-dimensional simulation |
| CN106940734A (en) * | 2017-04-24 | 2017-07-11 | 南京信息工程大学 | A kind of Migrating Insects monitor recognition methods and device in the air |
| CN108830861A (en) * | 2018-05-28 | 2018-11-16 | 上海大学 | A kind of hybrid optical motion capture method and system |
| CN110753930A (en) * | 2018-12-29 | 2020-02-04 | 深圳市瑞立视多媒体科技有限公司 | Fitting processing method of three-dimensional trajectory data and optical motion capturing method |
| CN110260845A (en) * | 2019-06-10 | 2019-09-20 | 北京航空航天大学 | A kind of insect motion acquisition device based on three high-speed cameras |
| US20210166390A1 (en) * | 2019-11-28 | 2021-06-03 | South China Normal University | Method, apparatus and storage medium for analyzing insect feeding behavior |
| CN112581574A (en) * | 2020-12-25 | 2021-03-30 | 江苏环影动漫文化有限公司 | Human body motion capture system and method |
| CN113160367A (en) * | 2021-05-11 | 2021-07-23 | 华东交通大学 | Lepidoptera insect model simulation and control method, system and readable storage medium |
| CN113592898A (en) * | 2021-05-13 | 2021-11-02 | 黑龙江省科学院智能制造研究所 | Method for reconstructing missing mark in motion capture |
Non-Patent Citations (5)
| Title |
|---|
| QIANG CHEN 等: "A Practical Model for Realistic Butterfly Flight Simulation", 《ACM TRANSACTIONS ON GRAPHICS》 * |
| 何真等: "昆虫飞行特性的视觉测量方法", 《南京理工大学学报(自然科学版)》 * |
| 吴升等: "一种有效的光学运动捕捉数据处理方法", 《应用基础与工程科学学报》 * |
| 王颖等: "昆虫飞行中翅膀序列图像特征匹配方法", 《光学技术》 * |
| 高倩等: "蜻蜓爬升过程飞行特征实验研究", 《北京航空航天大学学报》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115362990A (en) * | 2022-09-22 | 2022-11-22 | 中国农业科学院植物保护研究所 | Insect flapping frequency measuring device and method |
| CN115362990B (en) * | 2022-09-22 | 2024-02-13 | 中国农业科学院植物保护研究所 | Insect wing vibration frequency measuring device and method |
| CN116461714A (en) * | 2023-05-17 | 2023-07-21 | 北京科技大学 | Butterfly-simulated ornithopter flight parameter determination method based on dynamic capturing system |
| CN116461714B (en) * | 2023-05-17 | 2024-01-16 | 北京科技大学 | Butterfly-simulated ornithopter flight parameter determination method based on dynamic capturing system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114969623B (en) | 2022-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114969623B (en) | Data processing method and system for lepidoptera insect motion capture | |
| CN108830826B (en) | System and method for detecting pulmonary nodules | |
| JP5388932B2 (en) | Information processing apparatus and control method thereof | |
| CN106027852A (en) | Video image stabilization method for micro/nano-satellite | |
| CN103733226A (en) | Fast articulated motion tracking | |
| CN107315470A (en) | Graphic processing method, processor and virtual reality system | |
| CN109461208A (en) | Three-dimensional map processing method, device, medium and calculating equipment | |
| CN108447337A (en) | Simulated flight implementation method based on virtual reality | |
| JP4184028B2 (en) | How to create a fluid-structure coupled numerical model | |
| CN113435282A (en) | Unmanned aerial vehicle image ear recognition method based on deep learning | |
| CN107945217A (en) | A kind of image characteristic point suitable for automatic assembling is to rapid screening method and system | |
| CN112529963A (en) | Stair detection method and device and mobile robot | |
| CN110428464A (en) | Multi-class out-of-order workpiece robot based on deep learning grabs position and orientation estimation method | |
| CN103679797A (en) | Human limb modeling method adopting deformable models and virtual human model control platform | |
| CN112401369A (en) | Body parameter measuring method, system, equipment, chip and medium based on human body reconstruction | |
| CN110517348A (en) | Target object three-dimensional point cloud method for reconstructing based on display foreground segmentation | |
| CN115346262A (en) | Method, device and equipment for determining expression driving parameters and storage medium | |
| CN109272572A (en) | A kind of modeling method and device based on double Kinect cameras | |
| CN111862278A (en) | Animation obtaining method and device, electronic equipment and storage medium | |
| CN112183316A (en) | Method for measuring human body posture of athlete | |
| CN116996771A (en) | Automatic shooting control device and control method based on multiple sensors and deep reinforcement learning | |
| Luo et al. | Calibration-free monocular vision-based robot manipulations with occlusion awareness | |
| CN113011344B (en) | Pull-up quantity calculation method based on machine vision | |
| CN107292957A (en) | A kind of crop canopies three-dimensional rebuilding method and device | |
| CN117352126A (en) | Muscle stress visualization method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |