CN113473214B - Screen recording method and device, computing equipment and readable storage medium - Google Patents
Screen recording method and device, computing equipment and readable storage medium Download PDFInfo
- Publication number
- CN113473214B CN113473214B CN202110817455.2A CN202110817455A CN113473214B CN 113473214 B CN113473214 B CN 113473214B CN 202110817455 A CN202110817455 A CN 202110817455A CN 113473214 B CN113473214 B CN 113473214B
- Authority
- CN
- China
- Prior art keywords
- screen
- video
- screen recording
- source data
- screen image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000009877 rendering Methods 0.000 claims description 6
- 230000006835 compression Effects 0.000 claims description 3
- 238000007906 compression Methods 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 16
- 230000006870 function Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 230000002093 peripheral effect Effects 0.000 description 3
- 229920006395 saturated elastomer Polymers 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000007723 transport mechanism Effects 0.000 description 2
- 241000023320 Luma <angiosperm> Species 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4334—Recording operations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
The invention discloses a screen recording method, which is suitable for being executed in a computing device and comprises the following steps: receiving a screen recording request of a user; creating a screen recording buffer area; acquiring screen image source data from a video memory, and storing the screen image source data into a screen recording buffer area; the screen image source data is converted into a video file and saved. The invention also discloses a corresponding device, a computing device and a readable storage medium.
Description
The present application is a divisional application of patent application 2021103169717 filed on 25 th 3 rd year 2021.
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a screen recording method, apparatus, computing device, and readable storage medium.
Background
The window system commonly used in the Linux system comprises two protocols, namely X11 and wayland, and a window manager kwin can be compatible with the two protocols, however, along with the diversification of computer application programs, the demands of people are more and more, one of the longest application is screen recording, the current screen real-time image acquisition can be realized by combining the X11 with kwin, the function depends on the fact that the X11 is matched with a display card, the source data of the screen image can be acquired from a display memory, and meanwhile, the kwin is provided with a dbus screen recording interface.
The common screen recording software bypass-record is realized by monitoring the damage event of X11 to obtain redrawn damage area image data and drawing the redrawn damage area image data into an output video stream, and is essentially realized by a dbus screen recording interface and the function of reading video memory source data by X11. In addition, FFMPEG generates a recording technology of GIF or MP4 format video by reading a picture stream, and also obtains source data based on an adaptive interface of X11 and a video card. For the waiand, there is no interface for reading source data from the graphics card, so both of the above two methods cannot be directly transplanted to waiand to realize screen recording.
Therefore, how to realize the screen recording under the waiand is a problem to be solved.
Disclosure of Invention
To this end, the present invention provides a screen recording method, apparatus, computing device, and readable storage medium in an effort to solve or at least alleviate the above-identified problems.
According to one aspect of the present invention, there is provided a screen recording method adapted to be executed in a computing device, the method comprising: receiving a screen recording request of a user; creating a screen recording buffer area; acquiring screen image source data from a video memory, and storing the screen image source data into a screen recording buffer area; the screen image source data is converted into a video file and saved.
Optionally, in the screen recording method according to the present invention, the screen recording request includes a screen recording parameter, and the screen recording parameter includes a screen recording area, a recorded video frame rate, a recorded audio type, a video file format, and a video file storage path.
Optionally, in the on-screen recording method according to the present invention, the on-screen recording buffer is a ring buffer.
Optionally, in the screen recording method according to the present invention, converting the screen image source data into a video file and saving includes: acquiring screen image source data from a screen recording buffer area according to the video frame rate; converting the screen image source data into a screen image frame in a corresponding format according to the video file format; converting the screen image frames and the audio data acquired according to the recorded audio type into video files; and storing the video file according to the video file storage path.
Optionally, in the screen recording method according to the present invention, the video file format includes GIF and MP4 formats, and converting the screen image source data into the screen picture frame of the corresponding format according to the video file format includes: when the video file format is GIF, converting the screen image source data into screen picture frames in RGB format; when the video file format is MP4, the screen image source data is converted into a screen picture frame in YUV format.
Optionally, in the screen recording method according to the present invention, converting the screen picture frame and the audio data acquired according to the recorded audio type into the video file includes: compression encoding the screen picture frames into a video stream; encoding the audio data; writing the video stream and the encoded audio data into a media stream; and converting the media stream into a video file according to the screen recording parameters.
Alternatively, in the screen recording method according to the present invention, the acquisition of the screen image source data from the video memory is accomplished by the direct rendering manager.
According to another aspect of the present invention, there is provided a screen-based recording apparatus adapted to reside in a computing device, the apparatus comprising: the data receiving unit is used for receiving a screen recording request of a user; the data caching unit creates a screen recording buffer area; acquiring screen image source data from a video memory, and storing the screen image source data into a screen recording buffer area; and the video production unit is used for converting the screen image source data into a video file and storing the video file.
According to yet another aspect of the present invention, there is provided a computing device comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing any of the screen recording methods as above.
According to still another aspect of the present invention, there is provided a readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform any one of the screen recording methods described above.
According to the screen recording method, screen recording of the image frame without the image source data acquisition interface is realized, for example, the wayland is realized, so that a user can avoid the need of abandoning the specific image frame due to the limitation of screen recording application when selecting the image frame, and meanwhile, compared with the FFMPEG (flexible video stream) screen recording technology, the generated picture does not need to be subjected to frequent I/O operation of firstly storing and then reading, and the screen recording video with high recording video frame rate can be generated.
On the other hand, by locally creating the annular screen recording buffer area, the buffer data in the annular screen recording buffer area is read according to the frame rate of the recorded video to be processed, so that the buffer area is not always in a saturated state to lose data, and CPU loss caused by polling an empty queue is avoided.
Drawings
To the accomplishment of the foregoing and related ends, certain illustrative aspects are described herein in connection with the following description and the annexed drawings, which set forth the various ways in which the principles disclosed herein may be practiced, and all aspects and equivalents thereof are intended to fall within the scope of the claimed subject matter. The above, as well as additional objects, features, and advantages of the present disclosure will become more apparent from the following detailed description when read in conjunction with the accompanying drawings. Like reference numerals generally refer to like parts or elements throughout the present disclosure.
FIG. 1 illustrates a block diagram of a computing device 100 according to one embodiment of the invention;
FIG. 2 illustrates a flow chart of a screen recording method 200 according to one embodiment of the invention;
fig. 3 shows a block diagram of a screen recording apparatus 300 according to one embodiment of the present invention;
FIG. 4 illustrates a screen recording parameter setting interface according to one embodiment of the invention;
FIG. 5 illustrates a screen recording parameter setting interface according to one embodiment of the invention;
FIG. 6 illustrates a screen recording parameter setting interface according to one embodiment of the invention;
FIG. 7 illustrates a schematic diagram of a ring buffer in accordance with one embodiment of the invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The window system commonly used in Linux systems includes two protocols, X11 and waiand, where X11 is based on Client/Server network model, providing a basic framework for graphical user interfaces (Graphical User Interface, GUI), such as: a window is drawn and moved on the display device and interacted with by the mouse and keyboard. In the Linux system, kwin is a window manager in an X window (i.e. X11) system, and the combination of X11 and kwin can realize the function of screen recording, mainly using a screen recording dbus interface and an adapting interface between X11 and a display card provided by the system.
The wayland is a display mechanism for eliminating unnecessary designs in X11 and fully utilizing the modern Linux kernel graphics technology, and is used for improving the deficiency of the server in X11, but is developed to the present time, not just replacing the X server, but turning over the C/S structure of X11. However, in the waiand technology, the technology of acquiring the real-time picture of the current screen is not realized, and the screen recording cannot be realized, because there is no window image source data acquisition interface available for calling under the waiand platform, and at the same time, some operation platforms do not provide a screen recording interface, for example: kwin in the arm platform KelvinU system is without a screen-recording dbus interface.
The invention is based on the graphic frame (such as wayland) without the window image source data acquisition interface, realizes the screen recording function, and solves the problem that the graphic frame does not support screen recording software.
FIG. 1 shows a schematic diagram of a computing device 100 according to one embodiment of the invention. It should be noted that the computing device 100 shown in fig. 1 is only an example, and in practice, the computing device for implementing the screen recording method of the present invention may be any type of device, and the hardware configuration of the computing device may be the same as the computing device 100 shown in fig. 1 or may be different from the computing device 100 shown in fig. 1. In practice, the computing device for implementing the screen recording method of the present invention may add or delete hardware components of the computing device 100 shown in fig. 1, and the present invention is not limited to the specific hardware configuration of the computing device.
As shown in FIG. 1, in a basic configuration 102, a computing device 100 typically includes a system memory 106 and one or more processors 104. The memory bus 108 may be used for communication between the processor 104 and the system memory 106.
Depending on the desired configuration, the processor 104 may be any type of processing including, but not limited to: a microprocessor (μp), a microcontroller (μc), a digital information processor (DSP), or any combination thereof. The processor 104 may include one or more levels of caches, such as a first level cache 110 and a second level cache 112, a processor core 114, and registers 116. The example processor core 114 may include an Arithmetic Logic Unit (ALU), a Floating Point Unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The example memory controller 118 may be used with the processor 104, or in some implementations, the memory controller 118 may be an internal part of the processor 104.
Depending on the desired configuration, system memory 106 may be any type of memory including, but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. Physical memory in a computing device is often referred to as volatile memory, RAM, and data in disk needs to be loaded into physical memory in order to be read by processor 104. The system memory 106 may include an operating system 120, one or more applications 122, and program data 124. In some implementations, the application 122 may be arranged to execute instructions on an operating system by the one or more processors 104 using the program data 124. The operating system 120 may be, for example, linux, windows or the like, which includes program instructions for handling basic system services and performing hardware-dependent tasks. The application 122 includes program instructions for implementing various functions desired by the user, and the application 122 may be, for example, a browser, instant messaging software, a software development tool (e.g., integrated development environment IDE, compiler, etc.), or the like, but is not limited thereto. When an application 122 is installed into computing device 100, a driver module may be added to operating system 120.
When the computing device 100 starts up running, the processor 104 reads the program instructions of the operating system 120 from the memory 106 and executes them. Applications 122 run on top of operating system 120, utilizing interfaces provided by operating system 120 and underlying hardware to implement various user-desired functions. When a user launches the application 122, the application 122 is loaded into the memory 106, and the processor 104 reads and executes the program instructions of the application 122 from the memory 106.
Computing device 100 also includes storage device 132, storage device 132 including removable storage 136 and non-removable storage 138, both removable storage 136 and non-removable storage 138 being connected to storage interface bus 134.
Computing device 100 may also include an interface bus 140 that facilitates communication from various interface devices (e.g., output devices 142, peripheral interfaces 144, and communication devices 146) to basic configuration 102 via bus/interface controller 130. The example output device 142 includes a graphics processing unit 148 and an audio processing unit 150. They may be configured to facilitate communication with various external devices such as a display or speakers via one or more a/V ports 152. Example peripheral interfaces 144 may include a serial interface controller 154 and a parallel interface controller 156, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., printer, scanner, etc.) via one or more I/O ports 158. An example communication device 146 may include a network controller 160, which may be arranged to facilitate communication with one or more other computing devices 162 via one or more communication ports 164 over a network communication link.
The network communication link may be one example of a communication medium. Communication media may typically be embodied by computer readable instructions, data structures, program modules, and may include any information delivery media in a modulated data signal, such as a carrier wave or other transport mechanism. A "modulated data signal" may be a signal that has one or more of its data set or changed in such a manner as to encode information in the signal. By way of non-limiting example, communication media may include wired media such as a wired network or special purpose network, and wireless media such as acoustic, radio Frequency (RF), microwave, infrared (IR) or other wireless media. The term computer readable media as used herein may include both storage media and communication media.
In computing device 100 according to the present invention, application 122 includes instructions for performing the job screen recording method 200 of the present invention, which may instruct processor 104 to perform the screen recording method 200 of the present invention.
Fig. 2 shows a schematic diagram of a screen recording method 200 according to an embodiment of the invention, the method 200 being suitable for execution in a computing device (e.g., the computing device shown in fig. 1), the embodiment being illustrated in a waiand graphics framework.
As shown in fig. 2, the method 200 begins with step S210, receiving a user' S screen recording request. The screen recording request comprises screen recording parameters set by a user, including information such as screen recording area, video frame rate, audio type, video file format, video file storage path and the like.
According to an embodiment of the present invention, a user may complete parameter setting through a user interface, fig. 4 shows a schematic diagram of screen recording parameter setting according to an embodiment of the present invention, and the user may set a video file format generated by screen recording and a recording video frame rate through a drop-down menu of options, where the video file format includes GIF, MP4, MKV, and the like. The screen recording area may be set by dragging the rectangular frame shown in fig. 4, and the width and height of the recorded video area may be automatically generated after the screen recording area is selected. The user may also set the video file storage path in a drop down menu of options, as shown in fig. 5. The user may also set the type of recorded audio, which is divided into microphone audio and built-in audio, and fig. 5 shows a setting interface of the type of recorded audio according to an embodiment of the present invention, where the system audio is built-in audio, for example, audio of a video played on a screen. The user sets parameters, clicks a screen recording button and sends a screen recording request.
In one embodiment, a function may be defined to receive user-set parameters, such as: the parameters set by the collection user are transferred to the screen recording process through the function Wayland record (). After receiving the screen recording request of the user, the process advances to step S220.
In step S220, a screen recording buffer is created to store the screen image source data acquired from the graphic card. According to one embodiment of the invention, the buffer is declared as a ring buffer, e.g. the ring buffer may be arranged to store metadata for a predetermined number (e.g. 30) of screen image frames at most, fig. 7 shows a schematic diagram of the ring buffer according to one embodiment of the invention.
After the screen recording buffer is created, step S230 is performed to obtain the image source data from the video memory, and the image source data is inserted into the ring buffer, where the main function of the video memory is to store the image source data to be extracted by the video card chip.
According to one embodiment of the invention, the retrieval of source data from the video memory is accomplished by a direct rendering manager (Direct Rendering Manager, DRM). The DRM is a graphics rendering architecture under Linux, and is used for managing display output, and in this embodiment, screen image source data is obtained by directly operating an interface provided by DRM to an upper layer, and the rate of obtaining the image source data may be obtained according to a recorded video frame rate set by a user, for example, the recorded video frame rate set by the user is 30fps, and when obtaining the image source data, the rate of obtaining the image source data is also obtained according to a 30 frame image data rate of one second. The acquired image source data are sequentially stored in the screen recording buffer created in step S220.
In one embodiment according to the present invention, a thread is declared, the acquisition function of the underlying buffer is started, the method name may be named processBuffer (KWAyland:: client:: remoteBuffer), a lamda call method is used, and after receiving the KWAyland:: client::: remoteAccess manager:: buffer Ready signal, the received image buffer is stored in the screen recording buffer.
Audio data is also acquired while image frame data is acquired, when a user sets the recorded audio type to be a microphone, audio data is acquired from the microphone, and when the user sets the recorded audio type to be built-in audio (and system audio), built-in audio data corresponding to screen frame data is acquired.
Step S240 is then performed to acquire source data of the image frames from the screen recording buffer and convert the source data into corresponding image frames.
According to one embodiment of the present invention, when the video file format carried by the screen recording request is GIF, the step converts the source data of the image frame into a picture in RGB format, and the RGB color mode is a color standard in industry, and is obtained by changing three color channels of Red (Red), green (Green) and Blue (Blue) and overlapping them with each other, where RGB is the color representing the three channels of Red, green and Blue.
According to another embodiment of the present invention, when the video file format carried by the screen recording request is MP4, this step converts the source data of the image frame into a picture in YUV format. YUV is a color coding method. In the video processing components, YUV, which is a kind of color space (color space) of a compiled true, is a proper term of Y' UV, YUV, YCbCr, YPbPr, which may be called YUV, and overlap each other, in consideration of human perceptual ability in encoding video or images, allowing to reduce the bandwidth of chromaticity. "Y" represents brightness (Luminance or Luma), i.e., gray scale values, "U" and "V" represent Chrominance (Chroma) to describe the image color and saturation for a given pixel color.
One feature of the ring buffer is that when the ring buffer data store is saturated, the data that first enters the ring queue is pushed out of the ring queue, freeing up resources to store the newly acquired image data. When the enqueuing speed is equal to the dequeuing speed, namely, the speed of obtaining the source data from a video memory and writing the source data into the annular queue is equal to the speed of processing the image source data read from the annular queue, which is the normal state of the annular queue, namely, the enqueuing speed is approximately the same as the dequeuing speed, even if the enqueuing speed suddenly increases or the dequeuing speed suddenly decreases at a certain abrupt moment, the data can be pre-stored through the buffer area of the queue, and the data can be processed until the processing time is reached.
Yet another form is that the enqueue rate is greater than the dequeue rate, in which case the rate of queue "writing" is greater than the rate of "reading", and it is conceivable that after this state has continued for a period of time, the queue is full, and when a new element is generated, the new element drop may be dropped or placed in another buffer for storage, which is not a good condition, indicating that the efficiency or algorithm of handling dequeued elements is not good enough, and the queue is in a full state at all times.
The last form is that the enqueue rate is less than the dequeue rate, in which case the queue "read" rate is greater than the "write" rate, which indicates that the dequeue process element is fast, which is a good case, the only disadvantage is that the read queue may always poll if there is a new element in the queue, resulting in an excessively high cpu occupancy.
Therefore, in order to keep the enqueuing rate and the dequeuing rate approximately consistent, in this embodiment, the image frame source data acquired from the screen recording buffer is still set according to the recording video frame rate set by the user, for example: the user sets the recorded video frame rate to 30fps and reads the ring buffer at that rate. In a specific implementation, a writetostreamthread thread may be declared, a header picture frame in the ring queue is acquired according to a frame rate, after the picture frame is taken out, a writeToFrame method is declared, and the header picture frame is written into the method for later use.
In a specific implementation according to one embodiment of the present invention, a writeframe to stream thread may be declared, header image frame data in the ring queue may be acquired according to a frame rate, and the data may be written into another method, where the data is processed and converted into an image frame in a corresponding format.
Step S250 is then performed to generate a screen recorded video file from the obtained image frame.
According to one embodiment of the invention, image frames are compressed into a video stream format, the acquired audio data is encoded, the video stream and the encoded audio data are written into a media stream, and a video file is generated according to screen recording parameters set by a user.
According to one embodiment of the invention, this step may generate the video file from the image frames by FFMPEG. When generating the video file, the acquired audio data (including the audio data collected by the microphone and/or the built-in audio data) are combined, and the final video file is generated according to the screen recording parameters set by the user.
In a specific example, four threads may be declared, which are respectively a video thread for image acquisition, a microphone audio thread for acquisition, a built-in audio thread for acquisition, and a dual-tone channel audio mixing thread, where the threads are stored in a MainLoop event cycle, and when the event cycle is started, the threads are correspondingly started according to user parameter configuration. In the declared video capture thread, it is necessary to distinguish whether Gif or Mp4 video is generated, and if Gif video is generated, it is necessary to convert the picture format into av_pix_fmt_rgba format. If the Mp4 format video is generated, the picture format is optionally converted into the av_pix_fmt_yuv420P format. And calling an average_encode_video_2 method of FFmpeg to compress and encode the pictures into a video stream format, and writing the compressed video stream format into a media stream through an average_interleaved_write_frame method. Declaring a SetVideoCodecProp method, outputting the media stream as a real video file, wherein the parameters passed are: the parameters of video coding format, frame rate, code rate, compression quality, video width and video height are stored in a structure body in combination, the parameters are obtained by using an OpenOutputStream method, an avformat_new_stream is used for writing a media file, and an avformat_write_header is used for writing a file header of a video stream file.
Fig. 3 shows a schematic diagram of a screen recording apparatus 300 according to an embodiment of the present invention. The apparatus 300 is adapted to reside in a computing device (e.g., the computing device shown in fig. 1). As shown in fig. 3, the screen recording apparatus 300 includes a data receiving unit 310, a data buffering unit 320, and a video producing unit 330.
The data receiving unit 310 is adapted to receive a screen recording request from a user, and parse out screen recording parameters in the screen recording request.
The data buffer unit 320 is adapted to create a screen recording buffer, acquire image source data from a video memory, and store the image source data to the screen recording buffer, which may be configured as a ring buffer.
The video production unit 330 is adapted to obtain the screen image source data from the screen recording buffer and convert the screen image source data into a screen recording video file according to the screen recording parameters.
According to the screen recording method, screen recording based on the waiand graph frame is realized, so that when a user selects the image frame, the waiand graph frame can be prevented from being abandoned due to the limitation of screen recording application, and meanwhile, compared with the FFMPEG (flexible video experts group) screen recording technology, the generated pictures do not need to be subjected to frequent I/O (input/output) operations of storing and reading, and the screen recording video with high recording video frame rate can be generated.
On the other hand, by locally creating the annular screen recording buffer area, the buffer data in the annular screen recording buffer area is read according to the frame rate of the recorded video to be processed, so that the buffer area is not always in a saturated state to lose data, and CPU loss caused by polling an empty queue is avoided.
The various techniques described herein may be implemented in connection with hardware or software or, alternatively, with a combination of both. Thus, the methods and apparatus of the present invention, or certain aspects or portions of the methods and apparatus of the present invention, may take the form of program code (i.e., instructions) embodied in tangible media, such as removable hard drives, U-drives, floppy diskettes, CD-ROMs, or any other machine-readable storage medium, wherein, when the program is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.
In the case of program code execution on programmable computers, the computing device will generally include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Wherein the memory is configured to store program code; the processor is configured to perform the screen recording method of the present invention in accordance with instructions in said program code stored in the memory.
By way of example, and not limitation, readable media comprise readable storage media and communication media. The readable storage medium stores information such as computer readable instructions, data structures, program modules, or other data. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Combinations of any of the above are also included within the scope of readable media.
In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with examples of the invention. The required structure for a construction of such a system is apparent from the description above. In addition, the present invention is not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
It should be appreciated that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim.
Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment, or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or may be further divided into a plurality of sub-modules.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.
Furthermore, some of the embodiments are described herein as methods or combinations of method elements that may be implemented by a processor of a computer system or by other means of performing the functions. Thus, a processor with the necessary instructions for implementing the described method or method element forms a means for implementing the method or method element. Furthermore, the elements of the apparatus embodiments described herein are examples of the following apparatus: the apparatus is for carrying out the functions performed by the elements for carrying out the objects of the invention.
As used herein, unless otherwise specified the use of the ordinal terms "first," "second," "third," etc., to describe a general object merely denote different instances of like objects, and are not intended to imply that the objects so described must have a given order, either temporally, spatially, in ranking, or in any other manner.
While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of the above description, will appreciate that other embodiments are contemplated within the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the invention, which is defined by the appended claims.
Claims (8)
1. A screen recording method adapted to be executed in a computing device whose window system is based on a waiand protocol, the method comprising:
receiving a screen recording request of a user;
creating a screen recording buffer area, wherein the screen recording buffer area is an annular buffer area;
acquiring screen image source data from a video memory through a direct rendering manager according to a recorded video frame rate set by a user, and storing the screen image source data into the screen recording buffer area;
reading screen image source data from the screen recording buffer area according to the video recording frame rate set by the user;
and converting the screen image source data into a video file and storing the video file.
2. The method of claim 1, wherein the screen recording request includes screen recording parameters including a screen recording area, a recorded video frame rate, a recorded audio type, a video file format, and a video file storage path.
3. The method of claim 1 or 2, wherein the converting the screen image source data into a video file and saving comprises:
converting the screen image source data into screen image frames in corresponding formats according to the video file format;
converting the screen image frames and the audio data acquired according to the recorded audio type into video files;
and storing the video file according to the video file storage path.
4. The method of claim 3, wherein the video file format includes GIF and MP4 formats, and the converting the screen image source data into the screen image frames of the corresponding formats according to the video file format includes:
when the video file format is GIF, converting the screen image source data into a screen image frame in RGB format;
and when the video file format is MP4, converting the screen image source data into a screen image frame in YUV format.
5. The method of claim 3, wherein said converting the screen image frames and the audio data acquired according to the recorded audio type into a video file comprises:
compression encoding the screen image frames into a video stream;
encoding the audio data;
writing the video stream and the encoded audio data into a media stream;
and converting the media stream into a video file according to the screen recording parameters.
6. A screen-based recording apparatus adapted to reside in a computing device, a window system of the computing device being based on a waiand protocol, the apparatus comprising:
the data receiving unit is suitable for receiving a screen recording request of a user;
the data caching unit is suitable for creating a screen recording buffer zone, and the screen recording buffer zone is an annular buffer zone; acquiring screen image source data from a video memory through a direct rendering manager according to a recorded video frame rate set by a user, and storing the screen image source data into the screen recording buffer area; reading screen image source data from the screen recording buffer area according to the video recording frame rate set by the user;
and the video production unit is suitable for converting the screen image source data into a video file and storing the video file.
7. A computing device, comprising:
at least one processor; and
a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method of any of claims 1-5.
8. A readable storage medium storing program instructions which, when read and executed by a computing device, cause the computing device to perform the method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110817455.2A CN113473214B (en) | 2021-03-25 | 2021-03-25 | Screen recording method and device, computing equipment and readable storage medium |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110817455.2A CN113473214B (en) | 2021-03-25 | 2021-03-25 | Screen recording method and device, computing equipment and readable storage medium |
| CN202110316971.7A CN112714352B (en) | 2021-03-25 | 2021-03-25 | Screen recording method and device, computing equipment and readable storage medium |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110316971.7A Division CN112714352B (en) | 2021-03-25 | 2021-03-25 | Screen recording method and device, computing equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113473214A CN113473214A (en) | 2021-10-01 |
| CN113473214B true CN113473214B (en) | 2023-05-02 |
Family
ID=75550266
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110817455.2A Active CN113473214B (en) | 2021-03-25 | 2021-03-25 | Screen recording method and device, computing equipment and readable storage medium |
| CN202110316971.7A Active CN112714352B (en) | 2021-03-25 | 2021-03-25 | Screen recording method and device, computing equipment and readable storage medium |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110316971.7A Active CN112714352B (en) | 2021-03-25 | 2021-03-25 | Screen recording method and device, computing equipment and readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (2) | CN113473214B (en) |
| WO (1) | WO2022199044A1 (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113473214B (en) * | 2021-03-25 | 2023-05-02 | 统信软件技术有限公司 | Screen recording method and device, computing equipment and readable storage medium |
| CN113794926A (en) * | 2021-09-03 | 2021-12-14 | 深圳创维－Rgb电子有限公司 | Screen image acquisition method, device and computer readable storage medium |
| CN113784075B (en) * | 2021-09-28 | 2024-05-14 | 统信软件技术有限公司 | Screen video reading method, system and computing device |
| CN113709553A (en) * | 2021-09-30 | 2021-11-26 | 深圳市欢太数字科技有限公司 | Screen recording method, device, chip and terminal |
| CN115309315B (en) * | 2022-10-11 | 2023-01-20 | 麒麟软件有限公司 | Screenshot method, system and medium for X application in Wayland environment |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8510437B2 (en) * | 2000-01-27 | 2013-08-13 | Yawonba Holdings Au, Llc | Method and system for tracking screen activities |
| KR100792242B1 (en) * | 2007-04-05 | 2008-01-07 | 김재홍 | Operating method and system of remote recording |
| CN101609401B (en) * | 2009-07-02 | 2012-05-02 | 深圳市万兴软件有限公司 | Non-interference screen recording method and system thereof |
| CN102724466A (en) * | 2012-05-25 | 2012-10-10 | 深圳市万兴软件有限公司 | Video recording method and device |
| CN104424022B (en) * | 2013-08-30 | 2019-03-29 | 腾讯科技（深圳）有限公司 | Screen recording method and device |
| US9161006B1 (en) * | 2014-12-05 | 2015-10-13 | Kamcord, Inc. | Systems and methods for efficient screen capture |
| CN105763926A (en) * | 2014-12-18 | 2016-07-13 | 中兴通讯股份有限公司 | Screen recording method and device |
| CN107426606B (en) * | 2017-06-26 | 2020-07-24 | 联想(北京)有限公司 | Screen recording method and device, electronic equipment and system |
| CN111131882A (en) * | 2019-12-30 | 2020-05-08 | 联想(北京)有限公司 | Screen recording method and device and electronic equipment |
| CN112218148A (en) * | 2020-09-11 | 2021-01-12 | 杭州易现先进科技有限公司 | Screen recording method and device, computer equipment and computer readable storage medium |
| CN112423095A (en) * | 2020-11-02 | 2021-02-26 | 广州博冠信息科技有限公司 | Game video recording method and device, electronic equipment and storage medium |
| CN113473214B (en) * | 2021-03-25 | 2023-05-02 | 统信软件技术有限公司 | Screen recording method and device, computing equipment and readable storage medium |
-
2021
- 2021-03-25 CN CN202110817455.2A patent/CN113473214B/en active Active
- 2021-03-25 CN CN202110316971.7A patent/CN112714352B/en active Active
- 2021-11-10 WO PCT/CN2021/129674 patent/WO2022199044A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| CN112714352B (en) | 2021-08-20 |
| WO2022199044A1 (en) | 2022-09-29 |
| CN112714352A (en) | 2021-04-27 |
| CN113473214A (en) | 2021-10-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113473214B (en) | Screen recording method and device, computing equipment and readable storage medium | |
| US11373275B2 (en) | Method for generating high-resolution picture, computer device, and storage medium | |
| WO2020038128A1 (en) | Video processing method and device, electronic device and computer readable medium | |
| CN109685726B (en) | Game scene processing method and device, electronic equipment and storage medium | |
| WO2020107989A1 (en) | Video processing method and apparatus, and electronic device and storage medium | |
| US20210168441A1 (en) | Video-Processing Method, Electronic Device, and Computer-Readable Storage Medium | |
| CN110377263B (en) | Image synthesis method, image synthesis device, electronic equipment and storage medium | |
| WO2020108061A1 (en) | Video processing method and apparatus, electronic device and storage medium | |
| US20210281718A1 (en) | Video Processing Method, Electronic Device and Storage Medium | |
| WO2021031850A1 (en) | Image processing method and apparatus, electronic device and storage medium | |
| WO2020108092A1 (en) | Video processing method and device, and electronic device and storage medium | |
| US8713456B2 (en) | Establishing a graphical user interface (‘GUI’) theme | |
| WO2020038127A1 (en) | Decoding method and apparatus, electronic device, and storage medium | |
| WO2021008348A1 (en) | Image display method and device, mobile terminal, and storage medium | |
| WO2017202170A1 (en) | Method and device for video compression and electronic device | |
| WO2020108091A1 (en) | Video processing method and apparatus, and electronic device and storage medium | |
| KR102558385B1 (en) | Video augmentation control method, device, electronic device and storage medium | |
| JP7186877B2 (en) | Video decoding control method, device, electronic device and storage medium | |
| US11562772B2 (en) | Video processing method, electronic device, and storage medium | |
| WO2020108060A1 (en) | Video processing method and apparatus, and electronic device and storage medium | |
| WO2020108010A1 (en) | Video processing method and apparatus, electronic device and storage medium | |
| CN109587561B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN110377773B (en) | Picture processing method and device, mobile terminal and storage medium | |
| WO2023134625A1 (en) | Special effect optimization method and apparatus, and storage medium and program product | |
| CN114390307A (en) | Image quality enhancement method, device, terminal and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |