CN110598013B - Digital music painting interactive fusion method - Google Patents
Digital music painting interactive fusion method Download PDFInfo
- Publication number
- CN110598013B CN110598013B CN201910996562.9A CN201910996562A CN110598013B CN 110598013 B CN110598013 B CN 110598013B CN 201910996562 A CN201910996562 A CN 201910996562A CN 110598013 B CN110598013 B CN 110598013B
- Authority
- CN
- China
- Prior art keywords
- music
- image
- display
- information
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000010422 painting Methods 0.000 title claims abstract description 13
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 7
- 230000002452 interceptive effect Effects 0.000 title description 4
- 238000000034 method Methods 0.000 claims abstract description 18
- 230000004927 fusion Effects 0.000 claims abstract description 7
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000010606 normalization Methods 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 claims description 5
- 238000005070 sampling Methods 0.000 claims description 5
- 230000015572 biosynthetic process Effects 0.000 claims description 2
- 238000003786 synthesis reaction Methods 0.000 claims description 2
- 238000013507 mapping Methods 0.000 claims 4
- 238000006243 chemical reaction Methods 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 5
- 230000003993 interaction Effects 0.000 abstract 1
- 230000000007 visual effect Effects 0.000 description 3
- 230000033764 rhythmic process Effects 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 230000001747 exhibiting effect Effects 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/44—Browsing; Visualisation therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/20—Drawing from basic elements, e.g. lines or circles
- G06T11/203—Drawing of straight lines or curves
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B19/00—Driving, starting, stopping record carriers not specifically of filamentary or web form, or of supports therefor; Control thereof; Control of operating function ; Driving both disc and head
- G11B19/02—Control of operating function, e.g. switching from recording to reproducing
- G11B19/022—Control panels
- G11B19/025—'Virtual' control panels, e.g. Graphical User Interface [GUI]
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B3/00—Recording by mechanical cutting, deforming or pressing, e.g. of grooves or pits; Reproducing by mechanical sensing; Record carriers therefor
- G11B3/02—Arrangements of heads
- G11B3/10—Arranging, supporting, or driving of heads or of transducers relatively to record carriers
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention provides an image music fusion method and an interaction device based on music and image digital information. The method comprises the steps of obtaining music and image files for fusion; the time sequence digital information of the music and the space digital information in the image are mapped and fused skillfully; creatively combines the style of the traditional record player, simulates the effect that the painting brush draws on the record along with music, and has very good entertainment and ornamental value in the whole process. The works drawn by the system not only contain image information, but also contain music information, so that the works are easy to understand and accept by the masses, and the practicability is high.
Description
Technical Field
The invention relates to a digital multimedia entertainment interactive fusion method, in particular to an interactive entertainment method for realizing music and picture fusion.
Background
Painting and music are important artistic forms that express human ideas and emotions: music is auditory art, temporal art, dynamic art; painting is visual art, space art and static art; if the two can be mutually fused, brand new experience is brought to the user. The current related research and products are often not practical enough, lack of visual logic association and require improvement of user experience. For example, "a method and a system for generating a music video" (application (patent) No. 201310142642) describe a method for generating a music video, in which a system automatically generates a corresponding image and a corresponding video according to the music input by a user, and the matching degree of the image and the video generated by the system is not high and the relevance is not strong because the matching algorithm has a great gap compared with the understanding of people on the music image.
Disclosure of Invention
Based on the above, it is necessary to provide a more visual and interesting music drawing fusion method aiming at the problems of poor user experience and less product selection of music and drawing fusion products.
In addition, it is also necessary to provide a device capable of exhibiting the fusion effect of music and painting.
A digital music painting fusion method comprises the following steps: a music acquisition module for acquiring a music file for synthesis; the music preprocessing module acquires digital information such as pitch, tone intensity, music duration and the like of music changing along with time sequence according to the digital music file; the image acquisition module acquires an image file for processing; and the image preprocessing module is used for carrying out size normalization processing on the image, and then carrying out gray level processing and histogram equalization processing on the image.
The calculation module calculates the position of the brush holder terminal corresponding to each sampling time point on the plane of the display area according to the set rotation speed of the display area and the music duration, and calculates the inclination angle of the brush holder and the offset of the vertical central axis and the vibration amplitude of the brush holder according to the gray value of the position information mapped to the corresponding image sampling point.
And the drawing control module is used for playing music according to the data calculated by the calculation module, controlling the display area to rotate, gradually moving the drawing rod from the center of the display area to the periphery of the display area, changing the inclination angle and the vibration amplitude according to the calculated data, and displaying corresponding lines according to the calculated line width and color by the display module. Along with the music set by the user, the device simulates that a virtual brush (the brush only simulates, lines are displayed by a display module) rotates along with the record and the music rhythm "draws" the image set by the user on the record player, so that the dynamic fusion effect of the music and the drawing is presented.
Drawings
FIG. 1 is a flow chart of a music drawing fusion process in one embodiment.
Fig. 2 is a flowchart illustrating step S120 of fig. 1 according to an embodiment.
FIG. 3 is a flowchart illustrating the step S140 of FIG. 1 according to an embodiment.
Fig. 4 is a flowchart illustrating a specific step S210 in fig. 1 according to an embodiment.
Fig. 5 is a schematic structural view of the present invention.
FIG. 6 is a diagram showing a comparison of a painting output effect and an original input image in one embodiment.
Detailed Description
As shown in fig. 5, the control core processing unit 1 of the system mainly comprises a power module, a processor chip, a communication chip, a storage chip and a multimedia chip, and a user transmits music, picture files and user configuration parameters to be processed to the system through a mobile phone via a wireless transmission module or a physical interface.
The user configuration parameters mainly include the rotation angular velocity ω of the display area 3 in fig. 5 and the related calculation parameters L 1 ，L 2 ，L 3 ，L 4 ，L 5 ，L 6 ，L 7 ，L 8 ，L 9 ，L 10 ，L 11 ，L 12 。
The system respectively preprocesses the image and the music file after receiving the image and the music file.
The system preprocesses the music file, acquires the duration time T of the music file, and the tone intensity E (T) corresponding to the content of the music file at each time point T, and performs normalization processing;
（1）
（2）
in particular, in this embodiment, normalization is performed according to formulas (1) and (2), and in other embodiments, other common normalization methods may be used for performing the normalization.
The system pre-processes the image file, first normalizes the image to a proper size, in this embodiment to 800 x 800 pixel size; converting the normalized image into a gray format; aiming at images with poor contrast, carrying out histogram equalization treatment on the images to enhance the contrast; and acquiring image gray value information, and establishing a plane coordinate system by taking the center of the image as an origin, wherein the coordinates of a pixel are (x, y), and the gray value corresponding to the pixel is P (x, y).
The system calculation module calculates sequentially according to the duration time T and the rotation angular velocity omega of the music file, and calculates the total number of painting turns N and the maximum width A of the line through mathematical operation;
（3）
（4）
in this embodiment, we calculate the total number of turns N of the drawing corresponding to the drawing image in fig. 5 by using formula (3), and calculate the corresponding maximum width a of the line according to formula (4), wherein L 1 Different values are set according to actual needs for the scaling factor, and in particular, the value is 1 in this embodiment.
An arbitrary time point t within the music file time range;
(5)
(6)
the system calculates the corresponding coordinate position (x (t), y (t)) of the brush holder control unit 4 in fig. 5 in the image according to formula (5), formula (6); obtaining an image gray value P (x (t), y (t)) corresponding to the position according to the coordinate information (x (t), y (t)) by combining a graphic information system;
(7)
(8)
and then obtaining the corresponding line width D (t) and the penholder included angle theta (t) through mathematical calculation.
In particular, in the present embodiment, the line width D (t) of the drawing image of the position at time t shown in fig. 5 is calculated according to the formula (7), and the angle θ (t) between the pen holder 2 and the perpendicular bisector shown in fig. 5 is calculated according to the formula (8), wherein L 2 ，L 3 For the proportionality coefficient, different values are set according to actual needs, in particular L in the embodiment 2 Take the value of 1, L 3 The value is 1.
According to the pitch H (t) of music, the intensity E (t) is calculated mathematically to obtain the real-time amplitude gamma (t) of the painting holder;
(9)
in particular, in the present embodiment, the real-time amplitude y (t) of the pen shaft 2 in fig. 5 is calculated according to the formula (9), where L 4 ，L 5 ，L 6 For the proportionality coefficient, different values are set according to actual needs, in particular L in the embodiment 4 The value is 0.28, L 5 Takes the value of 0.72 and L 6 The value is 1.
According to the pitch H (t) of music, the tone intensity E (t) is calculated mathematically to obtain the line color C (t) of the current position;
(10)
(11)
(12)
in particular, in this embodiment, according to formula (10), formula (11), formula (12) calculates three components of red R (t), green G (t), and blue B (t) corresponding to color C (t) of the line, wherein L 7 ，L 8 ，L 9 ，L 10 ，L 11 ，L 12 For the proportionality coefficient, different values are set according to actual needs, in particular L in the embodiment 7 Take the value of 2, L 8 Take the value of 4, L 9 Takes the value of 1, L 10 Take the value of 4, L 11 Take the value of 2, L 12 The value of (2) is 55.
The system controls the device to operate according to the calculation result: (1) the system plays the music set by the user; (2) Similar to a disc player, the link lever 5 in fig. 5 controls the pen-holder control unit 4 to gradually move from the center position to the outer ring of the display area until the music ends; (3) At a time t of music playing, the brush control unit 4 in fig. 5 adjusts the dip angle of the brush holder 2 to be θ (t), and the larger the dip angle is, the wider the line is, so that the effect that the virtual brush draws the line on the display area is formed; (4) The drawing pen control unit 4 in fig. 5 controls the drawing pen holder 2 to swing by an amplitude gamma (t) in a plane where the pen holder and the perpendicular bisector are located; (5) Fig. 5 shows a module 3, where the color of the display line is C (t), the corresponding position is (x (t), y (t)), and the line width is D (t).
With the playing of music, we can see that the drawing pen holder 2 in fig. 5 "draws" an image set by the user on the display module 3 along with the rhythm of the music, and the comparison schematic result is shown in fig. 6, where the left side is the original image set by the user, and the right side is the displayed result.
After the music playing is finished, the display module displays the complete drawing figure, and the system controls the drawing rod to automatically restore to the initial position or the user resets manually.
Claims (10)
1. A digital music painting fusion method, comprising the steps of:
acquiring music and picture files for fusion;
preprocessing music and picture files to obtain corresponding digital information;
mapping and calculating according to the music and image digital information to obtain the width and color of the drawing lines and the moving and vibrating data of the drawing bars;
the preprocessing of the music and picture files comprises normalization of the pictures, gray conversion processing, obtaining of duration, pitch and tone intensity data information of the music files, and normalization processing;
aligning the edges of the pictures with the display edges of the display module one by one, and carrying out position mapping;
according to the duration of the music file, the gray value of the corresponding position of the image is determined, and the width of the drawing line of the corresponding position is determined;
determining the inclination angle of the drawing rod according to the width of the drawing line at the corresponding position;
the system plays music according to the calculated data, and controls the display module to display drawing lines according to the calculated data and controls the drawing bars to move according to the calculated movement data.
2. The method of claim 1, wherein,
the method comprises the steps that music and picture files used for fusion are obtained, and the music and picture files required to be fused by a user are obtained through a physical data interface or a wireless signal; and calculating the total number of painting turns and the maximum width of the line according to the duration of the music file and the rotation angular velocity of the display area.
3. The method of claim 1, wherein,
the preprocessing step of the picture file further comprises the step of carrying out histogram equalization processing on the picture, and obtaining black-white, uniform-size and vivid-contrast image files through processing.
4. The method of claim 1, wherein,
determining real-time amplitude of the drawing pen holder according to the pitch and the intensity of the music file; and determining the positions of the painting penholder corresponding to all the moments according to the rotation angular velocity of the display area.
5. The method of claim 1, wherein,
the information mapping mode is that the center of the picture is taken as the origin of a coordinate system, the center of the display module is taken as the origin of a corresponding coordinate system, the edges of the picture are aligned with the display edges of the display module one by one, and each position in the picture corresponds to the position in the display module one by one.
6. The method of claim 1, wherein,
the calculation is carried out according to the digital information of the picture, the color and coordinate information of the image are used, and the width and color information corresponding to the current coordinate of the display drawing line is obtained through mathematical calculation.
7. The method of claim 1, wherein,
the digital information of the music is used for calculation, the pitch and tone intensity information of the music are used, and the inclination angle and vibration amplitude information of the drawing rod and the perpendicular bisector are obtained through mathematical calculation; the inclination angle of the drawing rod is related to the line width at the corresponding position, and the larger the inclination angle is, the wider the line is.
8. The method of claim 1, wherein,
the display module displays the drawing lines, and displays the thickness and color data of the display lines at the current time point obtained by setting the display speed of the drawing lines and calculating the picture information along with the playing of music; the total number of turns of the drawing line is determined by the rotational angular velocity, and the duration of the music.
9. The method of claim 1, wherein,
the control drawing rod moves, and the inclination angle, the vibration amplitude and the coordinate position data of the drawing rod at the current time point obtained by calculating the music information are controlled; the inclination angle of the drawing rod is related to the line width at the corresponding position, and the larger the inclination angle is, the wider the line is.
10. The method of claim 1, wherein,
after music is played and a complete drawing figure is displayed, two resetting methods exist, one is that the system automatically controls the drawing rod to restore to the initial position, and the display module clears the line displayed in the front; the other is that the drawing rod is kept at the drawing end position, the display lines are kept unchanged, the drawing rod needs to be manually shifted to the initial position, and meanwhile, the display module clears the lines displayed in the front;
the method comprises the following steps:
acquiring a music file for synthesis, and acquiring music digital information comprising the pitch, the intensity and the duration of music, which are changed with time according to the music file;
acquiring an image file for processing, and sequentially carrying out size normalization processing, gray level processing and histogram equalization processing on the image file to acquire image digital information;
calculating the position of a brush holder terminal corresponding to each sampling time point on the plane of the display area according to the preset display area rotation speed and the music duration, mapping the position to the gray value of the corresponding image sampling point, and calculating the inclination angle of the brush holder and the offset of the vertical central shaft and the vibration amplitude of the brush holder;
playing music files, synchronously controlling the rotation of a display area, synchronously controlling the drawing pen rod to gradually move to the periphery of the display area within the music duration time from the center of the display area, keeping the drawing pen rod terminal at the corresponding position of the plane of the display area all the time at each sampling time point, inclining the drawing pen rod according to the calculated inclination angle, vibrating according to the calculated vibration amplitude, and simultaneously controlling the display area to display corresponding lines according to the calculated line width and color.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910996562.9A CN110598013B (en) | 2019-10-19 | 2019-10-19 | Digital music painting interactive fusion method |
| CN202310560279.8A CN116501901A (en) | 2019-10-19 | 2019-10-19 | Digital music image display method, system and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910996562.9A CN110598013B (en) | 2019-10-19 | 2019-10-19 | Digital music painting interactive fusion method |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310560279.8A Division CN116501901A (en) | 2019-10-19 | 2019-10-19 | Digital music image display method, system and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110598013A CN110598013A (en) | 2019-12-20 |
| CN110598013B true CN110598013B (en) | 2023-08-01 |
Family
ID=68851133
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310560279.8A Pending CN116501901A (en) | 2019-10-19 | 2019-10-19 | Digital music image display method, system and device |
| CN201910996562.9A Active CN110598013B (en) | 2019-10-19 | 2019-10-19 | Digital music painting interactive fusion method |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310560279.8A Pending CN116501901A (en) | 2019-10-19 | 2019-10-19 | Digital music image display method, system and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN116501901A (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112035686A (en) * | 2020-09-02 | 2020-12-04 | 北京字节跳动网络技术有限公司 | Play control method and device, electronic equipment and computer readable storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102289778A (en) * | 2011-05-10 | 2011-12-21 | 南京大学 | Method for converting image into music |
| CN104978329A (en) * | 2014-04-04 | 2015-10-14 | 庄婷婷 | Method for generating musical melody in line drawing form |
| CN207752778U (en) * | 2017-10-20 | 2018-08-21 | 北京星际元会展有限公司 | The interactive device of color paint converting music melody |
-
2019
- 2019-10-19 CN CN202310560279.8A patent/CN116501901A/en active Pending
- 2019-10-19 CN CN201910996562.9A patent/CN110598013B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN116501901A (en) | 2023-07-28 |
| CN110598013A (en) | 2019-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9996979B2 (en) | Augmented reality technology-based handheld viewing device and method thereof | |
| US10489956B2 (en) | Robust attribute transfer for character animation | |
| WO2019041351A1 (en) | Real-time aliasing rendering method for 3d vr video and virtual three-dimensional scene | |
| US20180197345A1 (en) | Augmented reality technology-based handheld viewing device and method thereof | |
| WO2016114930A2 (en) | Systems and methods for augmented reality art creation | |
| CN107636534A (en) | General sphere catching method | |
| CN105872353A (en) | System and method for implementing playback of panoramic video on mobile device | |
| US20190080510A1 (en) | Creating a synthetic model with organic veracity | |
| US10484599B2 (en) | Simulating depth of field | |
| WO2023077742A1 (en) | Video processing method and apparatus, and neural network training method and apparatus | |
| WO2022166896A1 (en) | Video generation method and apparatus, and device and readable storage medium | |
| CN106792151A (en) | A kind of virtual reality panoramic video player method | |
| CN106683195A (en) | AR scene rendering method based on indoor location | |
| WO2024012459A1 (en) | Method and system for terminal-cloud combined virtual concert rendering for vr terminal | |
| WO2023138548A1 (en) | Image processing method and apparatus, and device and storage medium | |
| CN115272570A (en) | Virtual expression generation method and device, electronic equipment and storage medium | |
| CN110598013B (en) | Digital music painting interactive fusion method | |
| CN110290409A (en) | Data processing method, VR equipment and system | |
| US20130336640A1 (en) | System and method for distributing computer generated 3d visual effects over a communications network | |
| CN106604087A (en) | Rendering method for panoramic live streaming | |
| WO2005088966A1 (en) | Substitute method for role head of digital tv program | |
| CN111954003A (en) | Panoramic video playing method and device | |
| CN112604279A (en) | Special effect display method and device | |
| CN111862273A (en) | Animation processing method and device, electronic equipment and storage medium | |
| KR101685505B1 (en) | Method for generating 3d image by user's participation and system and method for edutainments service using the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Feng Yi Inventor after: Cao Qiuqing Inventor after: Yin Yixiao Inventor before: Request for anonymity |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |