WO2021031575A1 - 媒体能力动态调整方法及装置 - Google Patents

媒体能力动态调整方法及装置 Download PDF

Info

Publication number
WO2021031575A1
WO2021031575A1 PCT/CN2020/083405 CN2020083405W WO2021031575A1 WO 2021031575 A1 WO2021031575 A1 WO 2021031575A1 CN 2020083405 W CN2020083405 W CN 2020083405W WO 2021031575 A1 WO2021031575 A1 WO 2021031575A1
Authority
WO
WIPO (PCT)
Prior art keywords
terminal
media
media capability
site corresponding
conference site
Prior art date
Application number
PCT/CN2020/083405
Other languages
English (en)
French (fr)
Inventor
高亮
周清
刘方
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2021031575A1 publication Critical patent/WO2021031575A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/24—Systems for the transmission of television signals using pulse code modulation

Definitions

  • This application relates to the field of communication technology, and in particular to a method and device for dynamically adjusting media capabilities.
  • MCU Multipoint Control Unit
  • Media capability refers to the processing capability of the media stream, which includes: forwarding, protocol conversion, codec capability adjustment, coding resolution adjustment, and coding parameter adjustment under the same coding resolution. Therefore, as the number of terminals connected to the venue increases, the media code traffic sent by the MCU receiving terminal also increases proportionally. It is easy to appear the system load bottleneck of the MCU stand-alone, and the bandwidth in the transmission network is largely occupied, and the network delay increases. This in turn leads to network congestion and affects the media quality and experience of the entire conference. Therefore, it is necessary to dynamically adjust the terminal media capabilities.
  • the adjustment process in the related technology is: the MCU detects the network information (including bandwidth and delay) from the terminal to the MCU, and increases or decreases the coding resolution according to the network information from the terminal to the MCU. For example, when the initial network bandwidth is 4M and both parties have 1080p encoding resolution, they will access the venue at the maximum encoding resolution of 1080p. As the number of terminals connected to the venue increases, the overall network bandwidth and delay increase, such as MCU When it is detected that the bandwidth from the terminal to the MCU is lower than 2M, the encoding resolution is reduced to 720p. If the network bandwidth is further reduced, the encoding resolution will be further reduced. When the network bandwidth is restored, it is raised back to the original encoding resolution according to the threshold.
  • the network information including bandwidth and delay
  • the above method is to adjust the encoding resolution according to the network information.
  • the overall network load of the entire network is high, the video effect of each venue will be reduced at the same time, and after the network bandwidth is restored, it will continue to restore to the original encoding resolution, occupying the network Bandwidth makes the MCU always maintain a high load.
  • the present application provides a method and device for dynamically adjusting media capabilities, which can adjust the media capabilities of the terminal corresponding to the conference site according to the actual needs of different terminal conference sites, improve user audio and video experience, and reduce network load.
  • this application provides a method for dynamically adjusting media capabilities, including:
  • the media enhancement device determines media information according to the code stream from the terminal, the media information includes meeting scene, participant level and network information, and the network information includes bandwidth and delay;
  • the media enhancement device sends the media information to a media capability controller
  • the media enhancement device receives the media capability adjustment strategy of the terminal sent by the media capability controller, and adjusts the media capability of the conference site corresponding to the terminal according to the media capability adjustment strategy, where the media capability adjustment strategy is The media capability controller is determined according to the media information or the service load of the network side device.
  • the media enhancement device determines the media information according to the code stream from the terminal and reports it to the media capability controller.
  • the media information includes the conference scene, the level of participants and network information, and the media capability controller Determine the media capability adjustment strategy of the terminal according to the media information or the service load of the network-side equipment, and then issue the media capability adjustment strategy of the terminal to the media enhancement device of the terminal, and the media enhancement device adjusts the corresponding terminal according to the received media capability adjustment strategy.
  • Media capabilities of the venue In this way, the media capabilities of the conference venue corresponding to the terminal can be adjusted according to the actual needs of different terminals, and the global optimal adjustment can be obtained, which improves the user's audio and video experience.
  • the network load of the entire network is lower.
  • the media enhancement device determining the media information of the terminal according to the code stream from the terminal includes:
  • the media enhancement device performs audio decoding and video decoding on the code stream from the terminal;
  • the media enhancement device performs voiceprint detection and video detection on the audio decoding result and the video decoding result, respectively, to identify the conference scene;
  • the media enhancement device performs face matching according to the video decoding result and the personnel level information database, and recognizes the level of the participants;
  • the media enhancement device detects network information according to the code stream from the terminal.
  • the method before the media enhancement apparatus determines the media information of the terminal according to the code stream from the terminal, the method further includes:
  • the media enhancement device receives and saves the participant ranking list sent by the application server;
  • the media enhancement device associates the participant level list with the user face data to obtain the personnel level information database.
  • the meeting scene includes one of a speaking scene, a listening scene, a coffee break scene, a group discussion scene, or a data sharing scene.
  • the media capability adjustment strategy includes: increase or decrease of media capability, the media capability includes at least one of resolution, encoding protocol, and encoding parameter, and the encoding parameter includes a quantization parameter QP At least one of, bandwidth, frame rate, and anti-packet redundancy.
  • the media capability adjustment strategy is to increase the media capabilities of the conference site corresponding to the terminal;
  • the media capability adjustment strategy is to reduce the media capabilities of the conference site corresponding to the terminal.
  • the media capability adjustment strategy is to improve the media capabilities of the conference site corresponding to the terminal.
  • the media capability adjustment The strategy is to reduce the media capabilities of the conference site corresponding to the terminal, so that the media capabilities of the conference site corresponding to the terminal can be adjusted according to the actual needs of different terminals, and the overall optimal adjustment can be obtained, which improves the user's audio and video experience and reduces the network load of the entire network.
  • the improving the media capability of the conference site corresponding to the terminal includes improving the media capability of the media enhancement device, or improving the media capability of the terminal;
  • the reducing the media capability of the conference site corresponding to the terminal includes reducing the media capability of the media enhancement device, or reducing the media capability of the terminal.
  • the media capability adjustment strategy is: increase the resolution of the video stream sent by the terminal corresponding to the terminal to the network side device, increase the encoding bandwidth, increase the frame rate, reduce QP, and perform protocol conversion. At least one
  • the media capability adjustment strategy is: at least one of increasing the resolution of the code stream sent by the terminal corresponding to the terminal to the network side device, increasing the encoding bandwidth, increasing the frame rate, increasing the redundancy against packet loss, and performing protocol conversion ;
  • the media capability adjustment strategy is: reducing the conference site corresponding to the terminal to send codes to the network side device Stream resolution and reduce encoding bandwidth;
  • the media capability adjustment strategy is: increasing the terminal The resolution of the code stream sent by the corresponding conference site to the network side device and the resolution of the code stream sent by the network side device to the conference site corresponding to the terminal;
  • the media capability adjustment strategy is: lower the conference site corresponding to the terminal The resolution of the code stream sent to the network side device and the resolution of the code stream sent by the network side device to the conference site corresponding to the terminal.
  • the service load includes the CPU occupancy rate of the central processing unit, the number of terminals, the number of conference sites, and the conference type.
  • this application provides a method for dynamically adjusting media capabilities, including:
  • the media capability controller receives the media information sent by the media enhancement device, the media information includes: meeting scene, participant level and network information, and the network information includes bandwidth and delay;
  • the media capability controller determines the media capability adjustment strategy of the media enhancement device of the terminal according to the media information or the service load of the network side device;
  • the media capability controller sends the media capability adjustment strategy of the terminal to the media enhancement device, where the media capability adjustment strategy is used to instruct the media enhancement device to adjust the media capability of the conference site corresponding to the terminal.
  • the media enhancement device determines the media information according to the code stream from the terminal and reports it to the media capability controller.
  • the media information includes the conference scene, the level of participants and network information, and the media capability controller Determine the media capability adjustment strategy of the terminal according to the media information or the service load of the network-side equipment, and then issue the media capability adjustment strategy of the terminal to the media enhancement device of the terminal, and the media enhancement device adjusts the corresponding terminal according to the received media capability adjustment strategy.
  • Media capabilities of the venue In this way, the media capabilities of the conference venue corresponding to the terminal can be adjusted according to the actual needs of different terminals, and the global optimal adjustment can be obtained, which improves the user's audio and video experience.
  • the network load of the entire network is lower.
  • the meeting scene includes one of a speaking scene, a listening scene, a coffee break scene, a group discussion scene, or a data sharing scene.
  • the media capability adjustment strategy includes:
  • the increase or decrease of media capability includes at least one of resolution, encoding protocol, and encoding parameter
  • the encoding parameter includes at least one of quantization parameter QP, bandwidth, frame rate, and anti-packet redundancy item.
  • the media capability controller determining the media capability adjustment strategy of the terminal according to the media information or the service load of the network side device includes:
  • the media capability controller determines that the media capability adjustment strategy is to improve the media capabilities of the conference site corresponding to the terminal;
  • the media capability controller determines that the media capability adjustment strategy is to reduce the media capability of the site corresponding to the terminal.
  • the media capability adjustment strategy is to improve the media capabilities of the conference site corresponding to the terminal.
  • the media capability adjustment The strategy is to reduce the media capabilities of the conference site corresponding to the terminal, so that the media capabilities of the conference site corresponding to the terminal can be adjusted according to the actual needs of different terminals, and the overall optimal adjustment can be obtained, which improves the user's audio and video experience and reduces the network load of the entire network.
  • the improving the media capability of the conference site corresponding to the terminal includes improving the media capability of the media enhancement device, or improving the media capability of the terminal;
  • the reducing the media capability of the conference site corresponding to the terminal includes reducing the media capability of the media enhancement device, or reducing the media capability of the terminal.
  • the media capability controller determining the media capability adjustment strategy of the terminal according to the media information or the service load of the network side device includes:
  • the media capability control The device determines that the media capability adjustment strategy is: at least one of increasing the resolution of the code stream sent by the terminal corresponding to the terminal to the network side device, increasing the encoding bandwidth, increasing the frame rate, reducing QP, and performing protocol conversion;
  • the media capability control The device determines that the media capability adjustment strategy is: increasing the resolution of the video stream sent by the terminal corresponding to the terminal to the network side device, increasing the encoding bandwidth, increasing the frame rate, increasing the redundancy against packet loss, and performing protocol conversion. At least one of
  • the media capability controller determines that the media capability adjustment strategy is: reducing the conference site direction corresponding to the terminal The resolution of the code stream sent by the network side device and the reduction of the coding bandwidth;
  • the media capability controller determines the media capability adjustment The strategy is to increase the resolution of the code stream sent by the conference site corresponding to the terminal to the network side device and the resolution of the code stream sent by the network side device to the conference site corresponding to the terminal;
  • the media capability controller determines that the media capability adjustment strategy is: Reduce the resolution of the code stream sent by the site corresponding to the terminal to the network side device and the resolution of the code stream sent by the network side device to the site corresponding to the terminal.
  • the service load includes the CPU occupancy rate of the central processing unit, the number of terminals, the number of conference sites, and the conference type.
  • this application provides a media enhancement device, including:
  • the determining module is configured to determine media information according to the code stream from the terminal, the media information includes conference scene, participant level and network information, and the network information includes bandwidth and delay;
  • a sending module for sending the media information to the media capability controller
  • a receiving module configured to receive the media capability adjustment strategy of the terminal sent by the media capability controller
  • the processing module is configured to adjust the media capability of the conference site corresponding to the terminal according to the media capability adjustment strategy, where the media capability adjustment strategy is determined by the media capability controller according to the media information or the service load of the network side device.
  • the determining module is used to:
  • the network information is detected according to the code stream from the terminal.
  • the receiving module is also used to:
  • the determining module determines the media information of the terminal according to the code stream from the terminal, receiving and saving the participant level list sent by the application server;
  • the processing module is also used to associate the participant rank list with the user's face data to obtain the staff rank information database.
  • the meeting scene includes one of a speaking scene, a listening scene, a coffee break scene, a group discussion scene, or a data sharing scene.
  • the media capability adjustment strategy includes: increase or decrease of media capability, the media capability includes at least one of resolution, encoding protocol, and encoding parameter, and the encoding parameter includes a quantization parameter QP At least one of, bandwidth, frame rate, and anti-packet redundancy.
  • the media capability adjustment strategy is to increase the media capabilities of the conference site corresponding to the terminal;
  • the media capability adjustment strategy is to reduce the media capabilities of the conference site corresponding to the terminal.
  • the improving the media capability of the conference site corresponding to the terminal includes improving the media capability of the media enhancement device, or improving the media capability of the terminal;
  • the reducing the media capability of the conference site corresponding to the terminal includes reducing the media capability of the media enhancement device, or reducing the media capability of the terminal.
  • the media capability adjustment strategy is: increase the resolution of the video stream sent by the terminal corresponding to the terminal to the network side device, increase the encoding bandwidth, increase the frame rate, reduce QP, and perform protocol conversion. At least one
  • the media capability adjustment strategy is: at least one of increasing the resolution of the code stream sent by the terminal corresponding to the terminal to the network side device, increasing the encoding bandwidth, increasing the frame rate, increasing the redundancy against packet loss, and performing protocol conversion ;
  • the media capability adjustment strategy is: reducing the conference site corresponding to the terminal to send codes to the network side device Stream resolution and reduce encoding bandwidth;
  • the media capability adjustment strategy is: increasing the terminal The resolution of the code stream sent by the corresponding conference site to the network side device and the resolution of the code stream sent by the network side device to the conference site corresponding to the terminal;
  • the media capability adjustment strategy is: lower the conference site corresponding to the terminal The resolution of the code stream sent to the network side device and the resolution of the code stream sent by the network side device to the conference site corresponding to the terminal.
  • the service load includes the CPU occupancy rate of the central processing unit, the number of terminals, the number of conference sites, and the conference type.
  • this application provides a media capability controller, including:
  • the receiving module is configured to receive media information sent by the media enhancement device, the media information includes: meeting scene, participant level and network information, and the network information includes bandwidth and delay;
  • a determining module configured to determine the media capability adjustment strategy of the media enhancement device according to the media information or the service load of the network side device;
  • the sending module is configured to send the media capability adjustment strategy to the media enhancement device, where the media capability adjustment strategy is used to instruct the media enhancement device to adjust the media capabilities of the conference site corresponding to the terminal.
  • the meeting scene includes one of a speaking scene, a listening scene, a coffee break scene, a group discussion scene, or a data sharing scene.
  • the media capability adjustment strategy includes:
  • the increase or decrease of media capability includes at least one of resolution, encoding protocol, and encoding parameter
  • the encoding parameter includes at least one of quantization parameter QP, bandwidth, frame rate, and anti-packet redundancy item.
  • the determining module is used to:
  • the media capability adjustment strategy is to reduce the media capabilities of the conference site corresponding to the terminal.
  • the improving the media capability of the conference site corresponding to the terminal includes improving the media capability of the media enhancement device, or improving the media capability of the terminal;
  • the reducing the media capability of the conference site corresponding to the terminal includes reducing the media capability of the media enhancement device, or reducing the media capability of the terminal.
  • the determining module is used to:
  • the adjustment strategy is: at least one of increasing the resolution of the code stream sent by the conference site corresponding to the terminal to the network side device, increasing the coding bandwidth, increasing the frame rate, reducing QP, and performing protocol conversion;
  • the adjustment strategy is: at least one of increasing the resolution of the code stream sent by the conference site corresponding to the terminal to the network side device, increasing the coding bandwidth, increasing the frame rate, increasing the redundancy against packet loss, and performing protocol conversion;
  • the media capability adjustment strategy is determined to be: reducing the conference site corresponding to the terminal to the network side device The resolution of the code stream and the reduction of the coding bandwidth;
  • the media capability adjustment strategy is: The resolution of the code stream sent by the conference site corresponding to the terminal to the network side device and the resolution of the code stream sent by the network side device to the conference site corresponding to the terminal;
  • the media capability adjustment strategy is: The resolution of the code stream sent by the conference site to the network side device and the resolution of the code stream sent by the network side device to the conference site corresponding to the terminal.
  • the service load includes the CPU occupancy rate of the central processing unit, the number of terminals, the number of conference sites, and the conference type.
  • this application provides a device for dynamically adjusting media capabilities, including: a memory and a processor;
  • a memory for storing executable instructions of the processor
  • the processor is configured to execute any possible design of the first aspect and the first aspect or the media capability dynamic adjustment in any one of the second aspect and the second aspect by executing the executable instruction method.
  • the present application provides a readable storage medium in which an execution instruction is stored.
  • the media capability dynamic adjustment apparatus executes the first aspect And any possible design of the first aspect or any possible design of the second aspect and a method for dynamically adjusting media capabilities in the second aspect.
  • the present application provides a program product.
  • the program product includes an execution instruction, and the execution instruction is stored in a readable storage medium.
  • At least one processor of the media capability dynamic adjustment apparatus can read the execution instruction from a readable storage medium, and at least one processor executes the execution instruction so that the media capability dynamic adjustment apparatus implements any possible design of the first aspect and the first aspect Or the second aspect and any one of the possible designs of the media capability dynamic adjustment method in the second aspect.
  • the present application provides a chip that is connected to a memory, or a memory is integrated on the chip, and when the software program stored in the memory is executed, any one of the first aspect and the first aspect is implemented.
  • Figure 1 is a schematic diagram of the system architecture applied by this application.
  • FIG. 3 is a signaling interaction flowchart of an embodiment of a method for dynamically adjusting media capabilities provided by this application;
  • Figure 4 is a schematic diagram of an initial joining state of this application.
  • Figure 5 is a schematic diagram of a state in progress of a conference
  • Figure 6 is a schematic diagram of another conference in progress
  • FIG. 7 is a schematic structural diagram of an embodiment of a media enhancement device provided by this application.
  • FIG. 8 is a schematic structural diagram of an embodiment of a media capability controller provided by this application.
  • FIG. 9 is a schematic structural diagram of a device for dynamically adjusting media capabilities provided by this application.
  • words such as “exemplary” or “for example” are used to represent examples, illustrations or illustrations, and any embodiment or solution described as “exemplary” or “for example” in the embodiments of this application It should not be construed as being more preferable or advantageous than other embodiments or solutions.
  • words such as “exemplary” or “for example” are used to present related concepts in a specific manner.
  • “at least one” refers to one or more, and “multiple” refers to two or more.
  • the MCU detects the network information from the terminal to the MCU, and adjusts the encoding resolution according to the network information to realize the dynamic adjustment of the terminal's media capabilities.
  • the overall network load of the entire network is high, it will cause the video of each venue The effect is reduced at the same time, and after the network bandwidth is restored, it will continue to return to the original encoding resolution, occupying the network bandwidth, so that the MCU always remains in a high load situation.
  • this application provides a method and device for dynamically adjusting media capabilities.
  • the media enhancement device of a terminal connected to the conference site determines media information according to the code stream from the terminal and reports it to the media capability controller.
  • the media information includes the conference According to the scene, participant level and network information
  • the media capability controller determines the media capability adjustment strategy of the terminal according to the received media information or the service load of the network side equipment, and then sends the media capability adjustment strategy of the terminal to the media enhancement device, and the media enhancement
  • the device adjusts the site media capabilities corresponding to the terminal according to the received media capability adjustment strategy. In this way, the media capabilities of the conference venue corresponding to the terminal can be adjusted according to the actual needs of different terminals, and the global optimal adjustment can be obtained, which improves the user's audio and video experience.
  • the network load of the entire network is lower.
  • FIG. 1 is a schematic diagram of the system architecture applied by this application.
  • a media enhancement device is added on the terminal side.
  • the media enhancement device can be an external module of the terminal, or It can be a built-in module of the terminal.
  • the media capability controller is deployed on the server where the MCU is located. It can be an external module of the MCU or a built-in module of the MCU to form a media network.
  • the media enhancement device of the terminal connected to the conference site determines the media information (including meeting scene, participant level and network information) according to the code stream from the terminal and reports it to the media capability controller, which is based on the received media information or network
  • the service load of the side device determines the media capability adjustment strategy of the terminal, and then delivers the media capability adjustment strategy of the terminal to the media enhancement device of the terminal, and the media enhancement device adjusts the media capability of the conference site corresponding to the terminal according to the received media capability adjustment strategy.
  • FIG. 2 is a signaling interaction flowchart of an embodiment of a method for dynamically adjusting media capabilities provided by this application. As shown in FIG. 2, the method in this embodiment may include:
  • the media enhancement device determines media information according to the code stream from the terminal.
  • the media information includes the conference scene, the level of participants and network information, and the network information includes bandwidth and delay.
  • the media enhancement device determines the media information according to the code stream from the terminal, which may be:
  • the meeting scene includes one of the speaking scene, listening scene, coffee break scene, group discussion scene or shared data scene. For example, it is recognized that the current conference scene is a speaking scene.
  • the personnel level information database is pre-stored in the terminal, and the personnel level information database stores the association relationship between the participant level list and the user's face data.
  • the level of participants can be, for example, free users and paid users, high-level leaders, mid-level supervisors, and ordinary employees, and can also be gold users, silver users, and bronze users.
  • the method in this embodiment may further include:
  • the terminal receives and saves the participant level list sent by the application server.
  • the participant level list includes participants and participant levels.
  • the terminal receives the user face data corresponding to the participant level list sent by the artificial intelligence gateway.
  • the terminal associates the participant rank list with the user's face data to obtain a staff rank information database.
  • the media enhancement device sends the media information to the media capability controller.
  • the media capability controller determines the media capability adjustment strategy of the terminal according to the media information or the service load of the network side device.
  • the service load includes a central processing unit (Central Processing Unit, CPU) occupancy rate, the number of terminals, the number of conference sites, and the type of conference.
  • the network side device is an MCU.
  • the media capability adjustment strategy includes: the increase or decrease of the media capability.
  • the media capability includes at least one of resolution, encoding protocol, and encoding parameters.
  • the encoding parameters include quantization parameters (QP), bandwidth, frame rate, and packet loss resistance. At least one item of redundancy. That is to say, the media capability adjustment strategy of the terminal can be to increase or decrease the resolution, or to perform encoding protocol conversion (such as H.264/H.265 conversion, convert H.264 to H.265, then increase Converting H.265 to H.264 will reduce the media capacity, or the conversion of AVC/SVC and converting AVC to SVC will expand the media capacity, which is equivalent to increasing the media capacity.
  • encoding protocol conversion such as H.264/H.265 conversion, convert H.264 to H.265, then increase Converting H.265 to H.264 will reduce the media capacity, or the conversion of AVC/SVC and converting AVC to SVC will expand the media capacity, which is equivalent to increasing the media capacity
  • SVC is converted to AVC, which is equivalent to reducing the media capacity), it can also be the improvement or reduction of encoding parameters (including at least one of QP, bandwidth, frame rate and anti-packet redundancy), and resolution Any combination of increase or decrease, coding protocol conversion and coding parameter increase or decrease.
  • the media capability controller determines that the media capability adjustment strategy is to improve the media capabilities of the conference site corresponding to the terminal, and the media capabilities of the conference site corresponding to the terminal include Improve the media capabilities of the media enhancement device, or improve the media capabilities of the terminal.
  • the media capability controller determines that the media capability adjustment strategy is to reduce the media capabilities of the conference site corresponding to the terminal, and reducing the media capabilities of the conference site corresponding to the terminal includes reducing the media capabilities of the media enhancement device. Or reduce the media capabilities of the terminal.
  • the media capability controller determines the media capability adjustment strategy of the terminal according to the media information or the service load of the network side device.
  • the media capability controller may determine the media capability adjustment strategy of the terminal according to the media information, or it may be based on the media information and the network side equipment.
  • the service load of the device determines the media capability adjustment strategy of the terminal, or the media capability adjustment strategy of the terminal is determined according to the service load of the network side device.
  • the adjustment strategy is: at least one of increasing the resolution of the code stream sent by the terminal corresponding to the conference site to the network side device, increasing the coding bandwidth, increasing the frame rate, reducing the QP, and performing protocol conversion.
  • the adjustment strategy is: at least one of increasing the resolution of the code stream sent by the terminal corresponding to the conference site to the network side device, increasing the coding bandwidth, increasing the frame rate, increasing the redundancy against packet loss, and performing protocol conversion.
  • the media capability controller determines the media capability adjustment strategy as: reduce the resolution of the conference site corresponding to the terminal to the network side device when sending the code stream Rate and reduce encoding bandwidth.
  • the media capability controller determines that the media capability adjustment strategy is: increase the conference site corresponding to the terminal The resolution of the code stream sent to the network side device and the resolution of the code stream sent by the network side device to the conference site corresponding to the terminal.
  • the media capability controller determines the media capability adjustment strategy as: reduce the conference site corresponding to the terminal to the network side The resolution of the code stream sent by the device and the resolution of the code stream sent by the network side device to the conference site corresponding to the terminal.
  • the speaking scenario for example, in the business strategy full announcement conference scenario (a meeting with more than 300 people, it may be 8 regional venues), such as the leadership speech session (the meeting scenario is the speaking scenario), at the same time
  • most of the meeting time in the other 7 venues is just listening.
  • the other 7 venues the meeting scene is the listening scene
  • all we need is to be able to clearly see the audio and video of the venue where the leader is.
  • the quality of the transmission code stream of the venue where the leader is located can be fully enhanced (such as improving the resolution of the code stream sent from the venue where the leader is located to the network side device, reducing QP and increasing the encoding bandwidth, etc.), and greatly reducing the transmission code of the other 7 venues Stream quality (such as reducing the resolution of the code stream sent by the other 7 sites to the network side device and reducing the encoding bandwidth, etc.), and appropriately improving the received stream quality of the 7 sites (e.g., increasing the resolution of the received stream at the site, and increasing the encoding Bandwidth, increased frame rate, and increased redundancy against packet loss, etc.), overall, the load on the network may be only half of the original or even lower.
  • each venue will focus on the questioning venue and the answering venue.
  • the meeting scenes of the questioning venue and the answering venue are both speaking scenes, and the attention to other venues that have not been asked is relatively low.
  • the conference scene of the question site terminal is a listening scene.
  • the quality of the two-way transmission and reception of the question site and the answer site should be improved (that is, the resolution of the terminal corresponding to the site to send the code stream to the network side device and the network side device to the terminal Corresponding to the conference site to send the code stream resolution, reduce QP and increase the encoding bandwidth, etc.), reduce the quality of the transmitted code stream of other sites (that is, reduce the resolution of the terminal corresponding to the site to send the code stream to the network side device and reduce the encoding bandwidth, etc.) .
  • the video conference system of each venue will not be closed, and will continue to run until the tea break is over, and there is no valuable information in the venue at this time, which can completely reduce the sending and receiving of each venue.
  • the stream quality reduces the load of the entire network, leaving more processing power for other ongoing meetings.
  • the identities of participants are inconsistent, that is, participants have different levels, including high-level leaders, middle-level supervisors, and ordinary employees.
  • the The venue where the leader is located requires better media stream quality to ensure that the audio and video quality of the venue where the high-level leader is located is the best.
  • Yes even when low-level employees are reporting scenes, they still need to be able to see the venue where the high-level leaders are located and be able to accurately understand the facial expressions of the high-level leaders.
  • there are a large number of free trial users and paying users When the network load is too heavy, it is also necessary to give priority to ensuring the experience and quality of paying users.
  • the media capability controller sends the media capability adjustment strategy to the media enhancement device.
  • the media enhancement device adjusts the media capability of the conference site corresponding to the terminal according to the media capability adjustment strategy.
  • the media enhancement device determines media information according to the code stream from the terminal and reports it to the media capability controller.
  • the media information includes meeting scenes, participant levels and network information.
  • the media information or the service load of the network side equipment determines the media capability adjustment strategy of the terminal, and then sends the media capability adjustment strategy of the terminal to the media enhancement device of the terminal, and the media enhancement device adjusts the conference site corresponding to the terminal according to the received media capability adjustment strategy Media capabilities.
  • the media capabilities of the conference venue corresponding to the terminal can be adjusted according to the actual needs of different terminals, and the global optimal adjustment can be obtained, which improves the user's audio and video experience.
  • the network load of the entire network is lower.
  • Figure 3 is a signaling interaction flow chart of an embodiment of a method for dynamic adjustment of media capabilities provided by this application.
  • the establishment of a participant level information database and how to identify conference scenes and participants are described from the interaction process.
  • a media enhancement device is added to the traditional conference terminal side, and a media capability controller is added to the network side.
  • the method of this embodiment may include:
  • the terminal joins the conference normally and the call flow.
  • S202 The application server sends a list of participant levels to the media enhancement device.
  • the media enhancement device receives and saves the participant level list.
  • the application server sends a list of participants to the artificial intelligence gateway.
  • the artificial intelligence gateway sends the user's face data corresponding to the participant level list to the media enhancement device according to the participant list.
  • the media enhancement device saves the user's face data, and associates the participant rank list with the user's face data to obtain a staff rank information database.
  • S207 The terminal sends a code stream to the media enhancement device.
  • the media enhancement device determines the media information according to the code stream from the terminal.
  • the media information includes the meeting scene, participant level and network information, specifically: audio decoding and video decoding on the code stream from the terminal, and the audio decoding result and video
  • the decoded results are respectively subjected to voiceprint detection and video detection to identify the meeting scene, match face according to the video decoding result and the personnel level information database, identify the level of participants, and detect network information based on the code stream from the terminal.
  • the media enhancement device sends the media information to the media capability controller.
  • the media capability controller determines the media capability adjustment strategy of the terminal according to the media information or the service load of the network side device.
  • the media capability controller sends respective media capability adjustment strategies to each terminal accessing the conference site.
  • the media enhancement device adjusts the media capability of the conference site corresponding to the terminal according to the received media capability adjustment strategy.
  • the media enhancement device sends the code stream after adjusting the media capability to the MCU.
  • Figure 4 is a schematic diagram of the initial joining state of the application. As shown in Figure 4, there are VIPs in the venue 1 and no VIPs in the venue 2 and 3. During the initial joining, the media capabilities were not dynamically adjusted.
  • the code streams sent by the enhancement device are all ordinary code streams, and the resolution of the sending and receiving is 4M.
  • the media enhancement devices in each venue will analyze the real-time media content of each venue in real time, and perform the following steps:
  • the media enhancement device determines media information according to the code stream from the terminal, and the media information includes the meeting scene, the level of the participants and the network information.
  • perform audio decoding and video decoding on the code stream from the terminal perform voiceprint detection and video detection on the audio decoding result and video decoding result, respectively, identify the meeting scene, and perform facial recognition based on the video decoding result and the personnel level information database.
  • Match identify the level of participants, for example, identify that there is VIP1 in the meeting place 1 (that is, the level of participants is VIP), and identify the meeting scene as the "speaking" scene; identify the meeting scene in the meeting place 2 as the "listening" scene , And no VIP is in the meeting place; it is recognized that the meeting scene of meeting place 3 is a "coffee break" scene, and no VIP is in the meeting place.
  • the root detects network information based on the code stream from the terminal.
  • the media enhancement device sends the media information to the media capability controller.
  • the media capability controller determines the media capability adjustment strategy of the terminal according to the received media information of the terminal and the service load of the network side device.
  • the media capability controller sends the media capability adjustment strategy to the media enhancement device of the terminal.
  • the media enhancement device adjusts the media capability of the conference site corresponding to the terminal according to the received media capability adjustment strategy.
  • FIG. 5 is a schematic diagram of a conference in progress. As shown in Figure 5, there is a VIP in the conference site 1, the conference scene is a "speaking" scene, and the media capability adjustment strategy is: the resolution of the transmitted code stream is 6M, and the QP value is 30. The anti-packet redundancy is 20%, and the resolution of the received code stream is 4M.
  • Venue 2 has no VIP in the venue, the conference scene is the "listening" scene, and the media capability adjustment strategy is: the resolution of the transmitted stream is 3M, the QP value is 45, the anti-packet redundancy is 5%, and the resolution of the received stream The rate is 4M.
  • the conference scene is a "coffee break" scene, and the media capability adjustment strategy is: the resolution of the sending stream is 2M, the QP value is 50, the anti-packet redundancy is 0%, the receiving stream is The resolution is 2M.
  • FIG. 6 is a schematic diagram of another conference in progress.
  • the participant level of venue 1, venue 2 and venue 3 is VIP, and the media capability adjustment strategy is: the resolution of sending code stream is 6M, and the resolution of receiving code stream is 6M ; Participants of venue 4, venue 5, and venue 6 are non-VIP, and the media capability adjustment strategy is: the resolution of the transmitted code stream is 2M, and the resolution of the received code stream is 2M. That is, priority is given to ensuring the quality of sending and receiving for paying users (ie VIP users).
  • FIG. 7 is a schematic structural diagram of an embodiment of a media enhancement device provided by this application.
  • the device of this embodiment may include: a determining module 11, a sending module 12, a receiving module 13, and a processing module 14, wherein, The determining module 11 is used to determine the media information according to the code stream from the terminal.
  • the media information includes meeting scene, participant level and network information, and the network information includes bandwidth and delay;
  • the sending module 12 is used to send the media information to the media capability controller
  • the receiving module 13 is used to receive the media ability adjustment strategy of the terminal sent by the media ability controller;
  • the processing module 14 is used to adjust the media ability of the conference site corresponding to the terminal according to the media ability adjustment strategy.
  • the media ability adjustment strategy is the media ability controller according to the media Information or the service load of the network side equipment is determined.
  • determining module 11 is used for:
  • Network information is detected based on the code stream from the terminal.
  • the receiving module 13 is also used for:
  • the determining module determines the media information of the terminal according to the code stream from the terminal, it receives and saves the participant level list sent by the application server;
  • the processing module is also used for associating the participant rank list with the user's face data to obtain a staff rank information database.
  • the meeting scene includes one of a speaking scene, a listening scene, a coffee break scene, a group discussion scene, or a data sharing scene.
  • the media capability adjustment strategy includes: increase or decrease of media capability.
  • the media capability includes at least one of resolution, encoding protocol, and encoding parameters.
  • the encoding parameters include quantization parameters QP, bandwidth, frame rate, and packet loss resistance. At least one of the redundancy.
  • the media capability adjustment strategy is to improve the media capabilities of the conference site corresponding to the terminal;
  • the media capability adjustment strategy is to reduce the media capabilities of the conference site corresponding to the terminal.
  • improving the media capability of the conference site corresponding to the terminal includes improving the media capability of the media enhancement device, or improving the media capability of the terminal;
  • Reducing the media capability of the conference site corresponding to the terminal includes reducing the media capability of the media enhancement device, or reducing the media capability of the terminal.
  • the conference site corresponding to the terminal has the highest level of participants, and the bandwidth is greater than the bandwidth threshold, and the delay is less than the delay threshold, and the conference site corresponding to the terminal has a high priority, and the media capability
  • the adjustment strategy is: at least one of increasing the resolution of the code stream sent by the terminal corresponding to the conference site to the network side device, increasing the encoding bandwidth, increasing the frame rate, reducing the QP, and performing protocol conversion;
  • the conference site corresponding to the terminal has the highest level of participants, and the bandwidth is less than the bandwidth threshold and the delay is greater than the delay threshold, the conference site corresponding to the terminal has a high priority, and the media capability adjustment strategy is: At least one of improving the resolution of the code stream sent by the terminal corresponding to the conference site to the network side device, increasing the encoding bandwidth, increasing the frame rate, increasing the redundancy against packet loss, and performing protocol conversion;
  • the media capability adjustment strategy is: reducing the resolution of the conference site corresponding to the terminal to the network side device and reducing the encoding bandwidth;
  • the media capability adjustment strategy is: increase the conference site corresponding to the terminal to send the code stream to the network side device Resolution and the resolution of the code stream sent by the network-side device to the corresponding conference site of the terminal;
  • the media capability adjustment strategy is: reduce the resolution of the conference site corresponding to the terminal to the network side device to send the code stream The resolution of the code stream sent by the network side device to the conference site corresponding to the terminal.
  • the service load includes the CPU occupancy rate of the central processing unit, the number of terminals, the number of conference sites, and the conference type.
  • the media enhancement device of this embodiment can be used to implement the technical solution of the method embodiment shown in FIG. 2, and its implementation principles are similar, and will not be repeated here.
  • the determining module determines the media information according to the code stream from the terminal.
  • the media information includes the meeting scene, the level of participants and network information.
  • the network information includes bandwidth and delay.
  • the sending module sends the media information to The media capability controller, the receiving module receives the media capability adjustment strategy of the terminal sent by the media capability controller, and the processing module adjusts the media capability of the conference site corresponding to the terminal according to the media capability adjustment strategy.
  • the media capability adjustment strategy is the media capability controller according to media information or The service load of the network side equipment is determined. In this way, the media capabilities of the conference venue corresponding to the terminal can be adjusted according to the actual needs of different terminals, and the global optimal adjustment can be obtained, which improves the user's audio and video experience.
  • the network load of the entire network is lower.
  • For the video conference system itself due to a large number of dynamic adjustments There is no need for the conference site corresponding to the terminal with the maximum media capacity to send and receive, which reduces the service load of the entire video conference system and the burden of network congestion, so that the video conference system can provide higher system capacity without changing the server-side hardware.
  • FIG. 8 is a schematic structural diagram of an embodiment of a media capability controller provided by this application.
  • the media capability controller of this embodiment may include: a receiving module 21, a determining module 22, and a sending module 23, where: The receiving module 21 is used to receive the media information sent by the media enhancement device.
  • the media information includes: meeting scene, participant level and network information.
  • the network information includes bandwidth and delay.
  • the media information is determined by the media enhancement device according to the code stream from the terminal.
  • the determining module 22 is configured to determine the media capability adjustment strategy of the terminal according to the media information or the service load of the network side device, and the sending module 23 is configured to transmit the media capability adjustment strategy to the media enhancement device, and the media capability The adjustment strategy is used to instruct the media enhancement apparatus to adjust the media capabilities of the conference site corresponding to the terminal.
  • the meeting scene includes one of a speaking scene, a listening scene, a coffee break scene, a group discussion scene, or a data sharing scene.
  • the media capability adjustment strategy includes:
  • the media capability is increased or decreased.
  • the media capability includes at least one of resolution, encoding protocol, and encoding parameter, and the encoding parameter includes at least one of quantization parameter QP, bandwidth, frame rate, and anti-packet redundancy.
  • the determining module 22 is used to:
  • the media capability adjustment strategy is to improve the media capabilities of the conference site corresponding to the terminal
  • the media capability adjustment strategy is to reduce the media capabilities of the conference site corresponding to the terminal.
  • improving the media capability of the conference site corresponding to the terminal includes improving the media capability of the media enhancement device, or improving the media capability of the terminal;
  • Reducing the media capability of the conference site corresponding to the terminal includes reducing the media capability of the media enhancement device, or reducing the media capability of the terminal.
  • the determining module 22 is used to:
  • the conference site corresponding to the terminal has high priority, and the media capability adjustment strategy is determined to be: increase the terminal
  • the corresponding conference site sends at least one of the resolution of the code stream, the encoding bandwidth, the frame rate, the QP reduction, and the protocol conversion to the network side device;
  • the conference site corresponding to the terminal has a high priority, and the media capability adjustment strategy is determined to be: increase the terminal
  • the corresponding conference site sends at least one of the resolution of the code stream, the encoding bandwidth, the frame rate, the anti-packet redundancy, and the protocol conversion to the network side device;
  • the media capability adjustment strategy is determined as: reducing the resolution of the conference site corresponding to the terminal to the network side device and reducing the encoding bandwidth;
  • the media capability adjustment strategy is determined as follows: increase the conference site corresponding to the terminal to send code to the network side device The resolution of the stream and the resolution of the code stream sent by the network side device to the corresponding conference site of the terminal;
  • the media capability adjustment strategy is determined as follows: reduce the resolution of the conference site corresponding to the terminal to the network side device when sending the code stream Rate and the resolution of the code stream sent by the network side device to the conference site corresponding to the terminal.
  • the service load includes the CPU occupancy rate of the central processing unit, the number of terminals, the number of conference sites, and the conference type.
  • the media capability controller of this embodiment may be used to execute the technical solution of the method embodiment shown in FIG. 2, and its implementation principles are similar, and will not be repeated here.
  • the media capability controller receives the media information sent by the media enhancement device through the receiving module.
  • the media information includes: meeting scene, participant level, and network information.
  • the network information includes bandwidth and delay.
  • the service load of the network side equipment determines the media capability adjustment strategy of the terminal, and the sending module sends the media capability adjustment strategy to the media enhancement device corresponding to the terminal.
  • the media capability adjustment strategy is used to instruct the media enhancement device corresponding to the terminal to adjust the media capability of the conference site corresponding to the terminal . In this way, the media capabilities of the conference venue corresponding to the terminal can be adjusted according to the actual needs of different terminals, and the global optimal adjustment can be obtained, which improves the user's audio and video experience.
  • the network load of the entire network is lower.
  • the present application may divide the function modules of the media capability dynamic adjustment device according to the foregoing method examples.
  • each function module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical function division, and there may be other division methods in actual implementation.
  • FIG. 9 is a schematic structural diagram of a media capability dynamic adjustment device provided by this application.
  • the media capability dynamic adjustment device 300 includes a memory 301 and a processor 302;
  • the memory 301 is used to store computer programs
  • the processor 302 is configured to execute a computer program stored in the memory to implement the printing method in the foregoing embodiment. For details, refer to the related description in the foregoing method embodiment.
  • the memory 301 may be independent or integrated with the processor 302.
  • the media capability dynamic adjustment apparatus 300 may further include:
  • the bus 303 is used to connect the memory 301 and the processor 302.
  • this embodiment further includes: a communication interface 304, which may be connected to the processor 302 through a bus 303.
  • the processor 302 may control the communication interface 303 to implement the aforementioned receiving and sending functions of the apparatus 300 for dynamically adjusting media capabilities.
  • the media capability dynamic adjustment device may be used to execute various steps or processes corresponding to the media enhancement device or the media capability controller in the foregoing method embodiments.
  • the present application also provides a readable storage medium in which an execution instruction is stored.
  • an execution instruction is stored.
  • the media capability dynamic adjustment device executes the above method embodiments Dynamic adjustment method of media capabilities.
  • This application also provides a program product, which includes an execution instruction, and the execution instruction is stored in a readable storage medium.
  • At least one processor of the media capability dynamic adjustment apparatus can read the execution instruction from a readable storage medium, and at least one processor executes the execution instruction to cause the media capability dynamic adjustment apparatus to implement the media capability dynamic adjustment method in the foregoing method embodiment.
  • the present application also provides a chip, which is connected to a memory, or a memory is integrated on the chip, and when a software program stored in the memory is executed, the method for dynamically adjusting media capabilities in the above method embodiment is implemented.
  • a person of ordinary skill in the art can understand that: in the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it can be implemented in the form of a computer program product in whole or in part.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website, computer, server, or data center. Transmission to another website site, computer, server or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.).
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Telephonic Communication Services (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请提供一种媒体能力动态调整方法及装置。该方法包括：媒体增强装置根据来自终端的码流确定媒体信息，媒体信息包括会议场景、与会人员等级和网络信息，网络信息包括带宽和时延，媒体增强装置将媒体信息发送给媒体能力控制器，媒体增强装置接收媒体能力控制器发送的终端的媒体能力调整策略，并根据媒体能力调整策略调整终端对应的会场的媒体能力，媒体能力调整策略为媒体能力控制器根据媒体信息或网络侧设备的业务负载确定。从而，可根据不同终端会场的实际需求调整终端对应的会场的媒体能力，提高用户音视频体验，降低网络负载。

Description

媒体能力动态调整方法及装置 技术领域
本申请涉及通信技术领域，尤其涉及一种媒体能力动态调整方法及装置。
背景技术
在视频会议***中，每接入一个新的终端，新接入的终端便和多点控制单元(Multipoint Control Unit，MCU)进行最大能力协商，然后以最大媒体能力，向MCU发送码流，其中，媒体能力是指对媒体码流的处理能力，该处理包括：转发、协议转换、编解码能力调整和编码分辨率调整，以及同一编码分辨率下编码参数的调整等。因此，随着接入会场的终端数增加，MCU接收终端所发送的媒体码流量也成比例上升，很容易出现MCU单机的***负载瓶颈，同时传输网络中的带宽被大量占用，网络延迟增加，进而导致网络拥塞，影响整个会议的媒体质量和体验。因此需要对终端媒体能力进行动态调整。
相关技术中的调整过程为：MCU探测终端到MCU的网络信息(包括带宽和时延)，根据终端到MCU的网络信息进行编码分辨率的提升或下降。如初始网络带宽为4M时，且双方具备1080p的编码分辨率时，则会以最大编码分辨率1080p接入会场，而随着接入会场的终端数增加，整体网络带宽和延迟增加，如MCU探测到终端到MCU的带宽低于2M时，则将编码分辨率降低到720p，如果网络带宽进一步降低，则会导致更进一步的调低编码分辨率。当网络带宽恢复后，再根据阈值提升回原始编码分辨率。
上述方法是根据网络信息对编码分辨率进行调整，在全网整体网络负载高时，会造成各个会场的视频效果同时降低，且在网络带宽恢复后，会继续恢复到原始编码分辨率，占用网络带宽，使得MCU始终保持在高负载的情况。
发明内容
本申请提供一种媒体能力动态调整方法及装置，可根据不同终端会场的实际需求调整终端对应的会场的媒体能力，提高用户音视频体验，降低网络负载。
第一方面，本申请提供一种媒体能力动态调整方法，包括：
媒体增强装置根据来自终端的码流确定媒体信息，所述媒体信息包括会议场景、与会人员等级和网络信息，所述网络信息包括带宽和时延；
所述媒体增强装置将所述媒体信息发送给媒体能力控制器；
所述媒体增强装置接收所述媒体能力控制器发送的所述终端的媒体能力调整策略，并根据所述媒体能力调整策略调整所述终端对应的会场的媒体能力，所述媒体能力调整策略为所述媒体能力控制器根据所述媒体信息或网络侧设备的业务负载确定。
通过第一方面提供的媒体能力动态调整方法，通过媒体增强装置根据来自终端的码流确定媒体信息并上报给媒体能力控制器，媒体信息包括会议场景、与会人员等级和网络信息，媒体能力控制器据媒体信息或网络侧设备的业务负载确定终端的媒体能力调整策略，然后将终端的媒体能力调整策略下发给终端的媒体增强装置，媒体增强装置根据接收到的媒体能力调整策略调整终端对应的会场的媒体能力。从而可实现根据不同终端的实际需求调整终端对应的会场的媒体能力，得到全局的最优调整，提高了用户音视频体验，全网网络负载更低，对于视频会议***本身，由于动态调整了大量不需要以最大媒体能力收发的 终端对应的会场，降低了整个视频会议***的业务负载和网络拥塞的负担，使得视频会议***可以在服务器侧硬件不变的情况下，提供更高的***容量。
在一种可能的设计中，所述媒体增强装置根据来自终端的码流确定终端的媒体信息，包括：
所述媒体增强装置对所述来自终端的码流进行音频解码和视频解码；
所述媒体增强装置对音频解码结果和视频解码结果分别进行声纹检测和视频检测，识别出会议场景；
所述媒体增强装置根据所述视频解码结果和人员等级信息库进行人脸匹配，识别出与会人员等级；
所述媒体增强装置根据所述来自终端的码流探测出网络信息。
在一种可能的设计中，所述媒体增强装置根据来自终端的码流确定终端的媒体信息之前，所述方法还包括：
所述媒体增强装置接收应用服务器发送的与会人员等级列表并保存；
所述媒体增强装置接收人工智能网关发送的和所述与会人员等级列表对应的用户人脸数据；
所述媒体增强装置将所述与会人员等级列表和所述用户人脸数据关联，得到所述人员等级信息库。
在一种可能的设计中，所述会议场景包括发言场景、收听场景、茶歇场景、分组讨论场景或共享数据场景中的一种。
在一种可能的设计中，所述媒体能力调整策略包括：媒体能力的提高或降低，所述媒体能力包括分辨率、编码协议和编码参数中的至少一项，所述编码参数包括量化参数QP、带宽、帧率和抗丢包冗余度中的至少一项。
在一种可能的设计中，当所述媒体信息指示所述终端对应的会场具有高优先级时，所述媒体能力调整策略为提高所述终端对应的会场的媒体能力；
当所述媒体信息指示所述终端对应的会场具有低优先级时，所述媒体能力调整策略为降低所述终端对应的会场的媒体能力。
通过该实施方式提供的媒体能力动态调整方法，终端对应的会场具有高优先级时，媒体能力调整策略为提高所终端对应的会场的媒体能力，终端对应的会场具有低优先级时，媒体能力调整策略为降低终端对应的会场的媒体能力，从而可实现根据不同终端的实际需求调整终端对应的会场的媒体能力，得到全局的最优调整，提高了用户音视频体验，全网网络负载更低。
在一种可能的设计中，所述提高所述终端对应的会场的媒体能力包括提高所述媒体增强装置的媒体能力，或提高所述终端的媒体能力；
所述降低所述终端对应的会场的媒体能力包括降低所述媒体增强装置的媒体能力，或降低所述终端的媒体能力。
在一种可能的设计中，若确定出的会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽大于带宽阈值、时延小于时延阈值时，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、降低QP和进行协议转换中的至少一项；
若确定出的会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽小于带宽阈值、时延大于时延阈值时，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、提高抗丢包冗余度和进行协议转换中的至少一项；
若所述终端的会议场景为茶歇场景或收听场景，所述终端对应的会场具有低优先级，则所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和降低编码带宽；
当所述业务负载大于预设阈值时，若所述终端对应的会场的与会人员等级为付费用户，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率；
当所述业务负载大于预设阈值时，若所述终端的与会人员等级为免费用户，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率。
在一种可能的设计中，所述业务负载包括中央处理器CPU占用率、终端数量、会场数量和会议类型。
第二方面，本申请提供一种媒体能力动态调整方法，包括：
媒体能力控制器接收媒体增强装置发送的媒体信息，所述媒体信息包括：会议场景、与会人员等级和网络信息，所述网络信息包括带宽和时延；
所述媒体能力控制器根据所述媒体信息或网络侧设备的业务负载确定所述终端的媒体增强装置的媒体能力调整策略；
所述媒体能力控制器向所述媒体增强装置发送所述终端的媒体能力调整策略，所述媒体能力调整策略用于指示所述媒体增强装置调整所述终端对应的会场的媒体能力。
通过第二方面提供的媒体能力动态调整方法，通过媒体增强装置根据来自终端的码流确定媒体信息并上报给媒体能力控制器，媒体信息包括会议场景、与会人员等级和网络信息，媒体能力控制器据媒体信息或网络侧设备的业务负载确定终端的媒体能力调整策略，然后将终端的媒体能力调整策略下发给终端的媒体增强装置，媒体增强装置根据接收到的媒体能力调整策略调整终端对应的会场的媒体能力。从而可实现根据不同终端的实际需求调整终端对应的会场的媒体能力，得到全局的最优调整，提高了用户音视频体验，全网网络负载更低，对于视频会议***本身，由于动态调整了大量不需要以最大媒体能力收发的终端对应的会场，降低了整个视频会议***的业务负载和网络拥塞的负担，使得视频会议***可以在服务器侧硬件不变的情况下，提供更高的***容量。
在一种可能的设计中，所述会议场景包括发言场景、收听场景、茶歇场景、分组讨论场景或共享数据场景中的一种。
在一种可能的设计中，所述媒体能力调整策略包括：
媒体能力的提高或降低，所述媒体能力包括分辨率、编码协议和编码参数中的至少一项，所述编码参数包括量化参数QP、带宽、帧率和抗丢包冗余度中的至少一项。
在一种可能的设计中，所述媒体能力控制器根据所述媒体信息或网络侧设备的业务负载确定所述终端的媒体能力调整策略，包括：
当所述媒体信息指示所述终端对应的会场具有高优先级时，所述媒体能力控制器确定所述媒体能力调整策略为提高所述终端对应的会场的媒体能力；
当所述媒体信息指示所述终端对应的会场具有低优先级时，所述媒体能力控制器确定所述媒体能力调整策略为降低所述终端对应的会场的媒体能力。
通过该实施方式提供的媒体能力动态调整方法，终端对应的会场具有高优先级时，媒体能力调整策略为提高所终端对应的会场的媒体能力，终端对应的会场具有低优先级时，媒体能力调整策略为降低终端对应的会场的媒体能力，从而可实现根据不同终端的实际需求调整终端对应的会场的媒体能力，得到全局的最优调整，提高了用户音视频体验，全网网络负载更低。
在一种可能的设计中，所述提高所述终端对应的会场的媒体能力包括提高所述媒体增强装置的媒体能力，或提高所述终端的媒体能力；
所述降低所述终端对应的会场的媒体能力包括降低所述媒体增强装置的媒体能力，或降低所述终端的媒体能力。
在一种可能的设计中，所述媒体能力控制器根据所述媒体信息或网络侧设备的业务负载确定所述终端的媒体能力调整策略，包括：
若会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽大于带宽阈值、时延小于时延阈值时，所述终端对应的会场具有高优先级，则所述媒体能力控制器确定所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、降低QP和进行协议转换中的至少一项；
若会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽小于带宽阈值、时延大于时延阈值时，所述终端对应的会场具有高优先级，则所述媒体能力控制器确定所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、提高抗丢包冗余度和进行协议转换中的至少一项；
若所述终端的会议场景为茶歇场景或收听场景，所述终端对应的会场具有低优先级，则所述媒体能力控制器确定所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和降低编码带宽；
当所述业务负载大于预设阈值时，若所述终端对应的会场的与会人员等级为付费用户，所述终端对应的会场具有高优先级，则所述媒体能力控制器确定所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率；
当所述业务负载大于预设阈值时，若所述终端的与会人员等级为免费用户，所述终端对应的会场具有高优先级，则所述媒体能力控制器确定所述媒体能力调整策略为：降低所 述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率。
在一种可能的设计中，所述业务负载包括中央处理器CPU占用率、终端数量、会场数量和会议类型。
第三方面，本申请提供一种媒体增强装置，包括：
确定模块，用于根据来自终端的码流确定媒体信息，所述媒体信息包括会议场景、与会人员等级和网络信息，所述网络信息包括带宽和时延；
发送模块，用于将所述媒体信息发送给媒体能力控制器；
接收模块，用于接收所述媒体能力控制器发送的所述终端的媒体能力调整策略；
处理模块，用于根据所述媒体能力调整策略调整所述终端对应的会场的媒体能力，所述媒体能力调整策略为所述媒体能力控制器根据所述媒体信息或网络侧设备的业务负载确定。
在一种可能的设计中，所述确定模块用于：
对所述来自终端的码流进行音频解码和视频解码；
对音频解码结果和视频解码结果分别进行声纹检测和视频检测，识别出会议场景；
根据所述视频解码结果和人员等级信息库进行人脸匹配，识别出与会人员等级；
根据所述来自终端的码流探测出网络信息。
在一种可能的设计中，所述接收模块还用于：
在所述确定模块根据来自终端的码流确定终端的媒体信息之前，接收应用服务器发送的与会人员等级列表并保存；
接收人工智能网关发送的和所述与会人员等级列表对应的用户人脸数据；
所述处理模块还用于：将所述与会人员等级列表和所述用户人脸数据关联，得到所述人员等级信息库。
在一种可能的设计中，所述会议场景包括发言场景、收听场景、茶歇场景、分组讨论场景或共享数据场景中的一种。
在一种可能的设计中，所述媒体能力调整策略包括：媒体能力的提高或降低，所述媒体能力包括分辨率、编码协议和编码参数中的至少一项，所述编码参数包括量化参数QP、带宽、帧率和抗丢包冗余度中的至少一项。
在一种可能的设计中，当所述媒体信息指示所述终端对应的会场具有高优先级时，所述媒体能力调整策略为提高所述终端对应的会场的媒体能力；
当所述媒体信息指示所述终端对应的会场具有低优先级时，所述媒体能力调整策略为降低所述终端对应的会场的媒体能力。
在一种可能的设计中，所述提高所述终端对应的会场的媒体能力包括提高所述媒体增强装置的媒体能力，或提高所述终端的媒体能力；
所述降低所述终端对应的会场的媒体能力包括降低所述媒体增强装置的媒体能力，或降低所述终端的媒体能力。
在一种可能的设计中，若确定出的会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽大于带宽阈值、时延小于时延阈值时，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、降低QP和进行协议转换中的至少一项；
若确定出的会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽小于带宽阈值、时延大于时延阈值时，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、提高抗丢包冗余度和进行协议转换中的至少一项；
若所述终端的会议场景为茶歇场景或收听场景，所述终端对应的会场具有低优先级，则所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和降低编码带宽；
当所述业务负载大于预设阈值时，若所述终端对应的会场的与会人员等级为付费用户，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率；
当所述业务负载大于预设阈值时，若所述终端的与会人员等级为免费用户，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率。
在一种可能的设计中，所述业务负载包括中央处理器CPU占用率、终端数量、会场数量和会议类型。
上述第三方面以及上述第三方面的各可能的设计中所提供的媒体增强装置，其有益效果可以参见上述第一方面和第一方面的各可能的实施方式所带来的有益效果，在此不再赘述。
第四方面，本申请提供一种媒体能力控制器，包括：
接收模块，用于接收媒体增强装置发送的媒体信息，所述媒体信息包括：会议场景、与会人员等级和网络信息，所述网络信息包括带宽和时延；
确定模块，用于根据所述媒体信息或网络侧设备的业务负载确定所述媒体增强装置的媒体能力调整策略；
发送模块，用于向所述媒体增强装置发送所述媒体能力调整策略，所述媒体能力调整策略用于指示所述媒体增强装置调整终端对应的会场的媒体能力。
在一种可能的设计中，所述会议场景包括发言场景、收听场景、茶歇场景、分组讨论场景或共享数据场景中的一种。
在一种可能的设计中，所述媒体能力调整策略包括：
媒体能力的提高或降低，所述媒体能力包括分辨率、编码协议和编码参数中的至少一项，所述编码参数包括量化参数QP、带宽、帧率和抗丢包冗余度中的至少一项。
在一种可能的设计中，所述确定模块用于：
当所述媒体信息指示所述终端对应的会场具有高优先级时，确定所述媒体能力调整策略为提高所述终端对应的会场的媒体能力；
当所述媒体信息指示所述终端对应的会场具有低优先级时，确定所述媒体能力调整策略为降低所述终端对应的会场的媒体能力。
在一种可能的设计中，所述提高所述终端对应的会场的媒体能力包括提高所述媒体增强装置的媒体能力，或提高所述终端的媒体能力；
所述降低所述终端对应的会场的媒体能力包括降低所述媒体增强装置的媒体能力，或降低所述终端的媒体能力。
在一种可能的设计中，所述确定模块用于：
若会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽大于带宽阈值、时延小于时延阈值时，所述终端对应的会场具有高优先级，则确定所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、降低QP和进行协议转换中的至少一项；
若会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽小于带宽阈值、时延大于时延阈值时，所述终端对应的会场具有高优先级，则确定所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、提高抗丢包冗余度和进行协议转换中的至少一项；
若所述终端的会议场景为茶歇场景或收听场景，所述终端对应的会场具有低优先级，则确定所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和降低编码带宽；
当所述业务负载大于预设阈值时，若所述终端对应的会场的与会人员等级为付费用户，所述终端对应的会场具有高优先级，则确定所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率；
当所述业务负载大于预设阈值时，若所述终端的与会人员等级为免费用户，所述终端对应的会场具有高优先级，则确定所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率。
在一种可能的设计中，所述业务负载包括中央处理器CPU占用率、终端数量、会场数量和会议类型。
上述第四方面以及上述第四方面的各可能的设计中所提供的媒体能力控制器，其有益效果可以参见上述第二方面和第二方面的各可能的实施方式所带来的有益效果，在此不再赘述。
第五方面，本申请提供一种媒体能力动态调整装置，包括：存储器和处理器；
处理器；以及
存储器，用于存储所述处理器的可执行指令；
其中，所述处理器配置为经由执行所述可执行指令来执行第一方面及第一方面任一种可能的设计或第二方面及第二方面任一种可能的设计中的媒体能力动态调整方 法。
第六方面，本申请提供一种可读存储介质，可读存储介质中存储有执行指令，当媒体能力动态调整装置的至少一个处理器执行该执行指令时，媒体能力动态调整装置执行第一方面及第一方面任一种可能的设计或第二方面及第二方面任一种可能的设计中的媒体能力动态调整方法。
第七方面，本申请提供一种程序产品，该程序产品包括执行指令，该执行指令存储在可读存储介质中。媒体能力动态调整装置的至少一个处理器可以从可读存储介质读取该执行指令，至少一个处理器执行该执行指令使得媒体能力动态调整装置实施第一方面及第一方面任一种可能的设计或第二方面及第二方面任一种可能的设计中的媒体能力动态调整方法。
第八方面，本申请提供一种芯片，所述芯片与存储器相连，或者所述芯片上集成有存储器，当所述存储器中存储的软件程序被执行时，实现第一方面及第一方面任一种可能的设计或第二方面及第二方面任一种可能的设计中的媒体能力动态调整方法。
附图说明
图1为本申请应用的***架构示意图；
图2为本申请提供的一种媒体能力动态调整方法实施例的信令交互流程图；
图3为本申请提供的一种媒体能力动态调整方法实施例的信令交互流程图；
图4为本申请的一种初始入会状态示意图；
图5为一种会议进行中状态示意图；
图6为另一种会议进行中状态示意图；
图7为本申请提供的一种媒体增强装置实施例的结构示意图；
图8为本申请提供的一种媒体能力控制器实施例的结构示意图；
图9为本申请提供的一种媒体能力动态调整装置结构示意图。
具体实施方式
本申请实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明，本申请实施例中被描述为“示例性的”或者“例如”的任何实施例或方案不应被解释为比其它实施例或方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。本申请中，“至少一个”是指一个或者多个，“多个”是指两个或两个以上。
相关技术中，通过MCU探测终端到MCU的网络信息，根据该网络信息对编码分辨率进行调整，来实现终端媒体能力的动态调整，但是在全网整体网络负载高时，会造成各个会场的视频效果同时降低，且在网络带宽恢复后，会继续恢复到原始编码分辨率，占用网络带宽，使得MCU始终保持在高负载的情况。为解决这一问题，本申请提供一种媒体能力动态调整方法及装置，通过接入会场的终端的媒体增强装置根据来自终端的码流确定媒体信息并上报给媒体能力控制器，媒体信息包括会议场景、与会人员等级和网络信息，媒体能力控制器根据接收到媒体信息或网络侧设备的业务负载确定终端的媒体能力调整策略，然后将终端的媒体能力调整策略下发给媒体增强装置，媒体增强装置根据接收到的媒体能力调整策略调整终端对应的会场媒体能力。从而可实现根据不同终端的实际需求调整终端 对应的会场的媒体能力，得到全局的最优调整，提高了用户音视频体验，全网网络负载更低，对于视频会议***本身，由于动态调整了大量不需要以最大媒体能力收发的对应的会场，降低了整个视频会议***的业务负载和网络拥塞的负担，使得视频会议***可以在服务器侧硬件不变的情况下，提供更高的***容量。
本申请可用于视频会议***中，图1为本申请应用的***架构示意图，如图1所示，本申请中通过在终端侧增加媒体增强装置，媒体增强装置可以是终端的外置模块，也可以是终端的内置模块。在网络侧增加媒体能力控制器，媒体能力控制器部署在MCU所在服务器，可以是MCU的外置模块，也可以是MCU的内置模块，形成媒体网络。通过接入会场的终端的媒体增强装置根据来自终端的码流确定媒体信息(包括会议场景、与会人员等级和网络信息)并上报给媒体能力控制器，媒体能力控制器根据接收到媒体信息或网络侧设备的业务负载确定终端的媒体能力调整策略，然后将终端的媒体能力调整策略下发给终端的媒体增强装置，媒体增强装置根据接收到的媒体能力调整策略调整终端对应的会场的媒体能力。下面结合附图详细说明本申请提供的媒体能力动态调整方法的具体过程。
图2为本申请提供的一种媒体能力动态调整方法实施例的信令交互流程图，如图2所示，本实施例的方法可以包括：
S101、媒体增强装置根据来自终端的码流确定媒体信息，媒体信息包括会议场景、与会人员等级和网络信息，网络信息包括带宽和时延。
具体地，媒体增强装置根据来自终端的码流确定媒体信息，可以为：
S1011、首先对自终端的码流进行音频解码和视频解码。
S1012、接着对音频解码结果和视频解码结果分别进行声纹检测和视频检测，识别出会议场景，其中的会议场景包括发言场景、收听场景、茶歇场景、分组讨论场景或共享数据场景中的一种，例如识别出当前会议场景为发言场景。
S1013、然后根据视频解码结果和人员等级信息库进行人脸匹配，识别出与会人员等级。
其中的人员等级信息库预先存储在终端内，人员等级信息库存储与会人员等级列表和用户人脸数据的关联关系。与会人员等级例如可以是免费用户和付费用户，还可以是高级别领导、中级别主管和普通员工等，还可以是金牌用户、银牌用户、铜牌用户等。
作为一种可实施的方式，在S101之前，本实施例的方法还可以包括：
终端接收应用服务器发送的与会人员等级列表并保存。
其中，与会人员等级列表包括与会人员和与会人员等级。
终端接收人工智能网关发送的和与会人员等级列表对应的用户人脸数据。
终端将与会人员等级列表和用户人脸数据关联，得到人员等级信息库。
S1014、最后根根据来自终端的码流探测出网络信息。
S102、媒体增强装置将媒体信息发送给媒体能力控制器。
S103、媒体能力控制器根据媒体信息或网络侧设备的业务负载确定终端的媒体能力调整策略。
其中，可选的，业务负载包括中央处理器(Central Processing Unit，CPU)占用率、终端数量、会场数量和会议类型。其中，网络侧设备如为MCU。
具体地，媒体能力调整策略包括：媒体能力的提高或降低，媒体能力包括分辨率、编码协议和编码参数中的至少一项，编码参数包括量化参数(QP)、带宽、帧率和抗丢包冗余度中的至少一项。即就是说，终端的媒体能力调整策略可以是进行分辨率提高或降低，也可以是进行编码协议转换(如H.264/H.265的转换，将H.264转换为H.265，则提高了媒体能力，将H.265转换为H.264，则降低了媒体能力，或者AVC/SVC的转换，将AVC转换为SVC，则对媒体能力做了扩充，相当于是提高了媒体能力，反之将SVC转换为AVC，相当于是降低了媒体能力)，还可以是进行编码参数的提高或降低(包括QP、带宽、帧率和抗丢包冗余度中的至少一项)，还可以进行分辨率提高或降低、编码协议转换和编码参数提高或降低这三项的任意组合。
作为一种可实施的方式，当媒体信息指示终端对应的会场具有高优先级时，媒体能力控制器确定媒体能力调整策略为提高终端对应的会场的媒体能力，提高终端对应的会场的媒体能力包括提高媒体增强装置的媒体能力，或提高终端的媒体能力。
当媒体信息指示终端对应的会场具有低优先级时，媒体能力控制器确定媒体能力调整策略为降低终端对应的会场的媒体能力，降低终端对应的会场的媒体能力包括降低媒体增强装置的媒体能力，或降低终端的媒体能力。
进一步地，媒体能力控制器根据媒体信息或网络侧设备的业务负载确定终端的媒体能力调整策略，可以是媒体能力控制器根据媒体信息确定终端的媒体能力调整策略，可以是根据媒体信息和网络侧设备的业务负载确定终端的媒体能力调整策略，还可以是根据网络侧设备的业务负载确定终端的媒体能力调整策略。可选的，在实际应用中有很多种情况，作为示例，有如下几种场景：
一、若会议场景为发言场景，终端对应的会场的与会人员等级最高，且带宽大于带宽阈值、时延小于时延阈值时，终端对应的会场具有高优先级，则媒体能力控制器确定媒体能力调整策略为：提高终端对应的会场向网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、降低QP和进行协议转换中的至少一项。
二、若会议场景为发言场景，终端对应的会场的与会人员等级最高，且带宽小于带宽阈值、时延大于时延阈值时，终端对应的会场具有高优先级，则媒体能力控制器确定媒体能力调整策略为：提高终端对应的会场向网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、提高抗丢包冗余度和进行协议转换中的至少一项。
三、若终端的会议场景为茶歇场景或收听场景，终端对应的会场具有低优先级，则媒体能力控制器确定媒体能力调整策略为：降低终端对应的会场向网络侧设备发送码流的分辨率和降低编码带宽。
四、当业务负载大于预设阈值时，若终端对应的会场的与会人员等级为付费用户，终端对应的会场具有高优先级，则媒体能力控制器确定媒体能力调整策略为：提高终端对应 的会场向网络侧设备发送码流的分辨率和网络侧设备向终端对应的会场发送码流的分辨率。
五、当业务负载大于预设阈值时，若终端的与会人员等级为免费用户，终端对应的会场具有高优先级，则媒体能力控制器确定媒体能力调整策略为：降低终端对应的会场向网络侧设备发送码流的分辨率和网络侧设备向终端对应的会场发送码流的分辨率。
例如，在发言场景下，示例性地，如在业务策略全员宣讲大会场景下(超过300人的会议，可能是8个地域会场)，如领导讲话环节(会议场景为发言场景)，同一时刻只有一个会场进行发言，没有频繁的双向互动，除领导所在的会场外，其他7个会场绝大部分会议时间仅仅是收听。其实此时，其他7个会场(会议场景为收听场景)，需要的仅仅是能够清晰的看到领导所在会场的音视频。对于其他会场可能根本不选看，或即使选看其他会场，对其他会场的画面的质量要求也是非常低的，因为当前的主要注意力均在领导讲话会场。此时，完全可以增强领导所在会场的发送码流的质量(如提高领导所在会场向网络侧设备发送码流的分辨率、降低QP和提高编码带宽等)，大幅降低其他7个会场的发送码流质量(如降低其他7个会场向网络侧设备发送码流的分辨率和降低编码带宽等)，并适当提升7个会场的接收码流质量(如提高会场接收码流的分辨率、提高编码带宽、提高帧率和提高抗丢包冗余度等)，整体综合下来，对网络的负载可能只有原来的一半甚至更低。
例如，在互动问答等环节中，各个会场都会将注意力放在提问会场和回答会场上，提问会场和回答会场的会议场景均为发言场景，对于其他未提问会场的关注度就相对降低，未提问会场终端的会议场景为收听场景，此时则更应该提升提问会场和回答会场的双向收发码流质量(即提升终端对应的会场向网络侧设备发送码流的分辨率以及网络侧设备向终端对应的会场发送码流的分辨率、降低QP和提高编码带宽等)，降低其他会场的发送码流质量(即降低终端对应的会场向网络侧设备发送码流的分辨率和降低编码带宽等)。
例如，在茶歇场景下，通常各个会场的视频会议***都不会关闭，会一直运行等到茶歇结束后继续使用，而此时的会场中没有任何有价值信息，完全可以降低各个会场的收发码流质量，降低整个网络的负载，给其他的正在进行的会议留出更多的处理能力。
又例如，还有在大型的视频会议***中与会人员的身份不一致的，即与会人员等级不同，有高级别领导，有中级别主管，也有普通员工，一般在会议***中，对于级别较高的领导所在的会场是需要更好的媒体码流质量，确保高级别领导所在的会场的音视频质量最优，同时也要保证，在高级别领导在发言时，其他会场都能够看得清，听得见，甚至是在低级别员工在汇报场景时，依然要求对高级别领导所在会场能够看的清，能够准确了解到高级别领导的面部表情。还有，在视频会议云服务场景中，有大量免费试用用户，和付费用户，在网络负载过重时，也需要优先保证付费用户的体验和质量。
S104、媒体能力控制器向媒体增强装置发送媒体能力调整策略。
S105、媒体增强装置根据媒体能力调整策略调整终端对应的会场的媒体能力。
本实施例提供的媒体能力动态调整方法，通过媒体增强装置根据来自终端的码流确定媒体信息并上报给媒体能力控制器，媒体信息包括会议场景、与会人员等级和网络信息，媒体能力控制器据媒体信息或网络侧设备的业务负载确定终端的媒体能力调整策略，然后将终端的媒体能力调整策略下发给终端的媒体增强装置，媒体增强装置根据接收到的媒体能力调整策略调整终端对应的会场的媒体能力。从而可实现根据不同终端的实际需求调整终端对应的会场的媒体能力，得到全局的最优调整，提高了用户音视频体验，全网网络负载更低，对于视频会议***本身，由于动态调整了大量不需要以最大媒体能力收发的终端对应的会场，降低了整个视频会议***的业务负载和网络拥塞的负担，使得视频会议***可以在服务器侧硬件不变的情况下，提供更高的***容量。
图3为本申请提供的一种媒体能力动态调整方法实施例的信令交互流程图，本实施例中从交互流程上描述了与会人员等级信息库的建立、以及如何进行会议场景识别和与会人员等级识别过程，本实施例中在传统会议终端侧增加媒体增强装置，在网络侧增加媒体能力控制器，如图3所示，本实施例的方法可以包括：
S201、终端正常入会及呼叫流程。
S202、应用服务器向媒体增强装置发送与会人员等级列表。
例如，与会人员等级列表为如表一所示：
表一
金牌 王总
银牌 老张
铜牌 小刘
S203、媒体增强装置接收并保存与会人员等级列表。
S204、应用服务器向人工智能网关发送与会人员列表。
S205、人工智能网关根据与会人员列表，向媒体增强装置发送和与会人员等级列表对应的用户人脸数据。
S206、媒体增强装置将用户人脸数据保存，并将与会人员等级列表和用户人脸数据关联，得到人员等级信息库。
S207、终端向媒体增强装置发送码流。
S208、媒体增强装置根据来自终端的码流确定媒体信息，媒体信息包括会议场景、与会人员等级和网络信息，具体为：对来自终端的码流进行音频解码和视频解码，对音频解码结果和视频解码结果分别进行声纹检测和视频检测，识别出会议场景，根据视频解码结果和人员等级信息库进行人脸匹配，识别出与会人员等级，并根据来自终端的码流探测出网络信息。
S209、媒体增强装置将媒体信息发送给媒体能力控制器。
S210、媒体能力控制器根据媒体信息或网络侧设备的业务负载确定终端的媒体能力调整策略。
S211、媒体能力控制器向接入会场的每个终端分别发送各自的媒体能力调整策略。
S212、媒体增强装置根据接收到的媒体能力调整策略调整终端对应的会场的媒体能力。
S213、媒体增强装置将调整媒体能力后的码流发送给MCU。
下面采用几个具体的实施例，针对不用的会议场景、与会人员等级和网络信息，对图1所示方法实施例的技术方案进行详细说明。
图4为本申请的一种初始入会状态示意图，如图4所示，会场1有VIP在会场，会场2和会场3无VIP在会场，初始入会时，未进行媒体能力的动态调整，经媒体增强装置发送的码流，均为普通码流，收发的分辨率均为4M。会议进行中，各个会场的媒体增强装置，实时解析每个会场的实时媒体内容，执行如下步骤：
S301、媒体增强装置根据来自终端的码流确定媒体信息，媒体信息包括会议场景、与会人员等级和网络信息。
具体地，对来自终端的码流进行音频解码和视频解码，对音频解码结果和视频解码结果分别进行声纹检测和视频检测，识别出会议场景，根据视频解码结果和人员等级信息库进行人脸匹配，识别出与会人员等级，例如，识别出会场1有VIP1在会场(即与会人员等级为VIP)，且识别出会议场景为“发言”场景；识别出会场2的会议场景为“收听”场景，且无VIP在会场；识别出会场3的会议场景为“茶歇”场景，且无VIP在会场。根根据来自终端的码流探测出网络信息。
S302、媒体增强装置将媒体信息发送给媒体能力控制器。
S303、媒体能力控制器根据接收到的终端的媒体信息和网络侧设备的业务负载确定该终端的媒体能力调整策略。
S304、媒体能力控制器向终端的媒体增强装置发送媒体能力调整策略。
S305、媒体增强装置根据接收到的媒体能力调整策略调整终端对应的会场的媒体能力。
例如，VIP所在会场或发言场景的会场，媒体能力调整策略为增强码流发送质量，茶歇场景的会场，媒体能力调整策略为降低会场侧网络收发质量。图5为一种会议进行中状态示意图，如图5所示，会场1有VIP在会场，会议场景为“发言”场景，媒体能力调整策略为：发送码流的分辨率为6M，QP值为30，抗丢包冗余度为20％，接收码流的分辨率为4M。会场2无VIP在会场，会议场景为“收听”场景，媒体能力调整策略为：发送码流的分辨率为3M，QP值为45，抗丢包冗余度为5％，接收码流的分辨率为4M。会场3无VIP在会场，会议场景为“茶歇”场景，媒体能力调整策略为：发送码流的分辨率为2M，QP值为50，抗丢包冗余度为0％，接收码流的分辨率为2M。
又例如，全网整体资源不足时，优先保证付费用户(即VIP用户)的收发质量，在同等带宽下，在媒体增强装置进行H.264/H.265的转换(或者AVC/SVC转换，兼容新老终端同时入会)，使得效果提升一倍，同时，降低免费用户(非VIP用户)的收发质量，以提供差异化服务。图6为另一种会议进行中状态示意图，会场1、会场2和会场3的与会人员等级为VIP，媒体能力调整策略为：发送码流的分辨率为6M，接收码流的分辨率为6M；会场4、会场5和会场6的与会人员等级为非VIP，媒体能力调整策略为：发送码流的分辨率为2M，接收码流的分辨率为2M。即优先保证付费用户(即VIP用户)的收发质量。
图7为本申请提供的一种媒体增强装置实施例的结构示意图，如图7所示，本实施例的装置可以包括：确定模块11、发送模块12、接收模块13和处理模块14，其中，确定模块11用于根据来自终端的码流确定媒体信息，媒体信息包括会议场景、与会人员等级和网络信息，网络信息包括带宽和时延；发送模块12用于将媒体信息发送给媒体能力控制器；接收模块13用于接收媒体能力控制器发送的终端的媒体能力调整策略；处理模块14用于根据媒体能力调整策略调整终端对应的会场的媒体能力，媒体能力调整策略为媒体能力控制器根据媒体信息或网络侧设备的业务负载确定。
进一步地，确定模块11用于：
对来自终端的码流进行音频解码和视频解码；
对音频解码结果和视频解码结果分别进行声纹检测和视频检测，识别出会议场景；
根据视频解码结果和人员等级信息库进行人脸匹配，识别出与会人员等级；
根据来自终端的码流探测出网络信息。
进一步地，接收模块13还用于：
在确定模块根据来自终端的码流确定终端的媒体信息之前，接收应用服务器发送的与会人员等级列表并保存；
接收人工智能网关发送的和与会人员等级列表对应的用户人脸数据；
处理模块还用于：将与会人员等级列表和用户人脸数据关联，得到人员等级信息库。
可选的，会议场景包括发言场景、收听场景、茶歇场景、分组讨论场景或共享数据场景中的一种。
可选的，媒体能力调整策略包括：媒体能力的提高或降低，媒体能力包括分辨率、编码协议和编码参数中的至少一项，编码参数包括量化参数QP、带宽、帧率和抗丢包冗余度中的至少一项。
可选的，当媒体信息指示终端对应的会场具有高优先级时，媒体能力调整策略为提高终端对应的会场的媒体能力；
当媒体信息指示终端对应的会场具有低优先级时，媒体能力调整策略为降低终端 对应的会场的媒体能力。
可选的，提高终端对应的会场的媒体能力包括提高媒体增强装置的媒体能力，或提高终端的媒体能力；
降低终端对应的会场的媒体能力包括降低媒体增强装置的媒体能力，或降低终端的媒体能力。
可选的，若确定出的会议场景为发言场景，终端对应的会场的与会人员等级最高，且带宽大于带宽阈值、时延小于时延阈值时，终端对应的会场具有高优先级，则媒体能力调整策略为：提高终端对应的会场向网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、降低QP和进行协议转换中的至少一项；
若确定出的会议场景为发言场景，终端对应的会场的与会人员等级最高，且带宽小于带宽阈值、时延大于时延阈值时，终端对应的会场具有高优先级，则媒体能力调整策略为：提高终端对应的会场向网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、提高抗丢包冗余度和进行协议转换中的至少一项；
若终端的会议场景为茶歇场景或收听场景，终端对应的会场具有低优先级，则媒体能力调整策略为：降低终端对应的会场向网络侧设备发送码流的分辨率和降低编码带宽；
当业务负载大于预设阈值时，若终端对应的会场的与会人员等级为付费用户，终端对应的会场具有高优先级，则媒体能力调整策略为：提高终端对应的会场向网络侧设备发送码流的分辨率和网络侧设备向终端对应的会场发送码流的分辨率；
当业务负载大于预设阈值时，若终端的与会人员等级为免费用户，终端对应的会场具有高优先级，则媒体能力调整策略为：降低终端对应的会场向网络侧设备发送码流的分辨率和网络侧设备向终端对应的会场发送码流的分辨率。
可选的，业务负载包括中央处理器CPU占用率、终端数量、会场数量和会议类型。
本实施例的媒体增强装置，可以用于执行图2所示方法实施例的技术方案，其实现原理类似，此处不再赘述。
本实施例提供的媒体增强装置，通过确定模块根据来自终端的码流确定媒体信息，媒体信息包括会议场景、与会人员等级和网络信息，网络信息包括带宽和时延，发送模块将媒体信息发送给媒体能力控制器，接收模块接收媒体能力控制器发送的终端的媒体能力调整策略，处理模块根据媒体能力调整策略调整终端对应的会场的媒体能力，媒体能力调整策略为媒体能力控制器根据媒体信息或网络侧设备的业务负载确定。从而可实现根据不同终端的实际需求调整终端对应的会场的媒体能力，得到全局的最优调整，提高了用户音视频体验，全网网络负载更低，对于视频会议***本身，由于动态调整了大量不需要以最大媒体能力收发的终端对应的会场，降低了整个视频会议***的业务负载和网络拥塞的负担，使得视频会议***可以在服务器侧硬件不变的情况下，提供更高的***容量。
图8为本申请提供的一种媒体能力控制器实施例的结构示意图，如图8所示，本实施例的媒体能力控制器可以包括：接收模块21、确定模块22和发送模块23，其中，接收模块21用于接收媒体增强装置发送的媒体信息，媒体信息包括：会议场景、与会人员等级和网络信息，网络信息包括带宽和时延，媒体信息为媒体增强装置根据来自终端的码流确定，确定模块22用于根据所述媒体信息或网络侧设备的业务负载确定所述终端的媒体能力调整 策略，发送模块23用于向所述媒体增强装置发送所述媒体能力调整策略，所述媒体能力调整策略用于指示所述媒体增强装置调整终端对应的会场的媒体能力。
可选的，会议场景包括发言场景、收听场景、茶歇场景、分组讨论场景或共享数据场景中的一种。
可选的，媒体能力调整策略包括：
媒体能力的提高或降低，媒体能力包括分辨率、编码协议和编码参数中的至少一项，编码参数包括量化参数QP、带宽、帧率和抗丢包冗余度中的至少一项。
可选的，确定模块22用于：
当媒体信息指示终端对应的会场具有高优先级时，确定媒体能力调整策略为提高终端对应的会场的媒体能力；
当媒体信息指示终端对应的会场具有低优先级时，确定媒体能力调整策略为降低终端对应的会场的媒体能力。
可选的，提高终端对应的会场的媒体能力包括提高媒体增强装置的媒体能力，或提高终端的媒体能力；
降低终端对应的会场的媒体能力包括降低媒体增强装置的媒体能力，或降低终端的媒体能力。
可选的，确定模块22用于：
若会议场景为发言场景，终端对应的会场的与会人员等级最高，且带宽大于带宽阈值、时延小于时延阈值时，终端对应的会场具有高优先级，则确定媒体能力调整策略为：提高终端对应的会场向网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、降低QP和进行协议转换中的至少一项；
若会议场景为发言场景，终端对应的会场的与会人员等级最高，且带宽小于带宽阈值、时延大于时延阈值时，终端对应的会场具有高优先级，则确定媒体能力调整策略为：提高终端对应的会场向网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、提高抗丢包冗余度和进行协议转换中的至少一项；
若终端的会议场景为茶歇场景或收听场景，终端对应的会场具有低优先级，则确定媒体能力调整策略为：降低终端对应的会场向网络侧设备发送码流的分辨率和降低编码带宽；
当业务负载大于预设阈值时，若终端对应的会场的与会人员等级为付费用户，终端对应的会场具有高优先级，则确定媒体能力调整策略为：提高终端对应的会场向网络侧设备发送码流的分辨率和网络侧设备向终端对应的会场发送码流的分辨率；
当业务负载大于预设阈值时，若终端的与会人员等级为免费用户，终端对应的会场具有高优先级，则确定媒体能力调整策略为：降低终端对应的会场向网络侧设备发送码流的分辨率和网络侧设备向终端对应的会场发送码流的分辨率。
可选的，业务负载包括中央处理器CPU占用率、终端数量、会场数量和会议类型。
本实施例的媒体能力控制器，可以用于执行图2所示方法实施例的技术方案，其 实现原理类似，此处不再赘述。
本实施例提供的媒体能力控制器，通过接收模块接收媒体增强装置发送的媒体信息，媒体信息包括：会议场景、与会人员等级和网络信息，网络信息包括带宽和时延，确定模块根据媒体信息或网络侧设备的业务负载确定终端的媒体能力调整策略，发送模块向终端对应的媒体增强装置发送媒体能力调整策略，媒体能力调整策略用于指示终端对应的媒体增强装置调整终端对应的会场的媒体能力。从而可实现根据不同终端的实际需求调整终端对应的会场的媒体能力，得到全局的最优调整，提高了用户音视频体验，全网网络负载更低，对于视频会议***本身，由于动态调整了大量不需要以最大媒体能力收发的终端对应的会场，降低了整个视频会议***的业务负载和网络拥塞的负担，使得视频会议***可以在服务器侧硬件不变的情况下，提供更高的***容量。
本申请可以根据上述方法示例对媒体能力动态调整装置进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请各实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
图9为本申请提供的一种媒体能力动态调整装置结构示意图，如图9所示，该媒体能力动态调整装置300包括：存储器301和处理器302；
存储器301，用于存储计算机程序；
处理器302，用于执行存储器存储的计算机程序，以实现上述实施例中的打印方法。具体可以参见前述方法实施例中的相关描述。
可选地，存储器301既可以是独立的，也可以跟处理器302集成在一起。
当存储器301是独立于处理器302之外的器件时，媒体能力动态调整装置300还可以包括：
总线303，用于连接存储器301和处理器302。
可选地，本实施例还包括：通信接口304，该通信接口304可以通过总线303与处理器302连接。处理器302可以控制通信接口303来实现媒体能力动态调整装置300的上述的接收和发送的功能。
该媒体能力动态调整装置可以用于执行上述方法实施例中媒体增强装置或媒体能力控制器对应的各个步骤或流程。
本申请还提供一种可读存储介质，可读存储介质中存储有执行指令，当媒体能力动态调整装置的至少一个处理器执行该执行指令时，媒体能力动态调整装置执行上述方法实施例中的媒体能力动态调整方法。
本申请还提供一种程序产品，该程序产品包括执行指令，该执行指令存储在可读存储介质中。媒体能力动态调整装置的至少一个处理器可以从可读存储介质读取该执行指令，至少一个处理器执行该执行指令使得媒体能力动态调整装置实施上述方法实施例中的媒体能力动态调整方法。
本申请还提供一种芯片，芯片与存储器相连，或者芯片上集成有存储器，当存储器中存储的软件程序被执行时，实现上述方法实施例中的媒体能力动态调整方法。
本领域普通技术人员可以理解：在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。

Claims (34)

  1. 一种媒体能力动态调整方法，其特征在于，包括：
    媒体增强装置根据来自终端的码流确定媒体信息，所述媒体信息包括会议场景、与会人员等级和网络信息，所述网络信息包括带宽和时延；
    所述媒体增强装置将所述媒体信息发送给媒体能力控制器；
    所述媒体增强装置接收所述媒体能力控制器发送的所述终端的媒体能力调整策略，并根据所述媒体能力调整策略调整所述终端对应的会场的媒体能力，所述媒体能力调整策略为所述媒体能力控制器根据所述媒体信息或网络侧设备的业务负载确定。
  2. 根据权利要求1所述的方法，其特征在于，所述媒体增强装置根据来自终端的码流确定终端的媒体信息，包括：
    所述媒体增强装置对所述来自终端的码流进行音频解码和视频解码；
    所述媒体增强装置对音频解码结果和视频解码结果分别进行声纹检测和视频检测，识别出会议场景；
    所述媒体增强装置根据所述视频解码结果和人员等级信息库进行人脸匹配，识别出与会人员等级；
    所述媒体增强装置根据所述来自终端的码流探测出网络信息。
  3. 根据权利要求2所述的方法，其特征在于，所述媒体增强装置根据来自终端的码流确定终端的媒体信息之前，所述方法还包括：
    所述媒体增强装置接收应用服务器发送的与会人员等级列表并保存；
    所述媒体增强装置接收人工智能网关发送的和所述与会人员等级列表对应的用户人脸数据；
    所述媒体增强装置将所述与会人员等级列表和所述用户人脸数据关联，得到所述人员等级信息库。
  4. 根据权利要求1所述的方法，其特征在于，所述会议场景包括发言场景、收听场景、茶歇场景、分组讨论场景或共享数据场景中的一种。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述媒体能力调整策略包括：媒体能力的提高或降低，所述媒体能力包括分辨率、编码协议和编码参数中的至少一项，所述编码参数包括量化参数QP、带宽、帧率和抗丢包冗余度中的至少一项。
  6. 根据权利要求5所述的方法，其特征在于，当所述媒体信息指示所述终端对应的会场具有高优先级时，所述媒体能力调整策略为提高所述终端对应的会场的媒体能力；
    当所述媒体信息指示所述终端对应的会场具有低优先级时，所述媒体能力调整策略为降低所述终端对应的会场的媒体能力。
  7. 根据权利要求6所述的方法，其特征在于，所述提高所述终端对应的会场的媒体能力包括提高所述媒体增强装置的媒体能力，或提高所述终端的媒体能 力；
    所述降低所述终端对应的会场的媒体能力包括降低所述媒体增强装置的媒体能力，或降低所述终端的媒体能力。
  8. 根据权利要求6或7所述的方法，其特征在于，
    若确定出的会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽大于带宽阈值、时延小于时延阈值时，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、降低QP和进行协议转换中的至少一项；
    若确定出的会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽小于带宽阈值、时延大于时延阈值时，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、提高抗丢包冗余度和进行协议转换中的至少一项；
    若所述终端的会议场景为茶歇场景或收听场景，所述终端对应的会场具有低优先级，则所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和降低编码带宽；
    当所述业务负载大于预设阈值时，若所述终端对应的会场的与会人员等级为付费用户，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率；
    当所述业务负载大于预设阈值时，若所述终端的与会人员等级为免费用户，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率。
  9. 根据权利要求1-8任一项所述的方法，其特征在于，所述业务负载包括中央处理器CPU占用率、终端数量、会场数量和会议类型。
  10. 一种媒体能力动态调整方法，其特征在于，包括：
    媒体能力控制器接收媒体增强装置发送的媒体信息，所述媒体信息包括：会议场景、与会人员等级和网络信息，所述网络信息包括带宽和时延；
    所述媒体能力控制器根据所述媒体信息或网络侧设备的业务负载确定所述终端的媒体能力调整策略；
    所述媒体能力控制器向所述媒体增强装置发送所述终端的媒体能力调整策略，所述媒体能力调整策略用于指示所述媒体增强装置调整所述终端对应的会场的媒体能力。
  11. 根据权利要求10所述的方法，其特征在于，所述会议场景包括发言场景、收听场景、茶歇场景、分组讨论场景或共享数据场景中的一种。
  12. 据权利要求10或11所述的方法，其特征在于，所述媒体能力调整策略包括：
    媒体能力的提高或降低，所述媒体能力包括分辨率、编码协议和编码参数中的至少一项，所述编码参数包括量化参数QP、带宽、帧率和抗丢包冗余度中的至少一项。
  13. 根据权利要求12所述的方法，其特征在于，所述媒体能力控制器根据所述媒体信息或网络侧设备的业务负载确定所述媒体增强装置的媒体能力调整策略，包括：
    当所述媒体信息指示所述终端对应的会场具有高优先级时，所述媒体能力控制器确定所述媒体能力调整策略为提高所述终端对应的会场的媒体能力；
    当所述媒体信息指示所述终端对应的会场具有低优先级时，所述媒体能力控制器确定所述媒体能力调整策略为降低所述终端对应的会场的媒体能力。
  14. 根据权利要求13所述的方法，其特征在于，所述提高所述终端对应的会场的媒体能力包括提高所述媒体增强装置的媒体能力，或提高所述终端的媒体能力；
    所述降低所述终端对应的会场的媒体能力包括降低所述媒体增强装置的媒体能力，或降低所述终端的媒体能力。
  15. 根据权利要求14所述的方法，其特征在于，所述媒体能力控制器根据所述媒体信息或网络侧设备的业务负载确定所述终端的媒体能力调整策略，包括：
    若会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽大于带宽阈值、时延小于时延阈值时，所述终端对应的会场具有高优先级，则所述媒体能力控制器确定所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、降低QP和进行协议转换中的至少一项；
    若会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽小于带宽阈值、时延大于时延阈值时，所述终端对应的会场具有高优先级，则所述媒体能力控制器确定所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、提高抗丢包冗余度和进行协议转换中的至少一项；
    若所述终端的会议场景为茶歇场景或收听场景，所述终端对应的会场具有低优先级，则所述媒体能力控制器确定所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和降低编码带宽；
    当所述业务负载大于预设阈值时，若所述终端对应的会场的与会人员等级为付费用户，所述终端对应的会场具有高优先级，则所述媒体能力控制器确定所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率；
    当所述业务负载大于预设阈值时，若所述终端的与会人员等级为免费用户，所述终端对应的会场具有高优先级，则所述媒体能力控制器确定所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率。
  16. 根据权利要求10-15任一项所述的方法，其特征在于，所述业务负载包括中央处理器CPU占用率、终端数量、会场数量和会议类型。
  17. 一种媒体增强装置，其特征在于，包括：
    确定模块，用于根据来自终端的码流确定媒体信息，所述媒体信息包括会议场景、 与会人员等级和网络信息，所述网络信息包括带宽和时延；
    发送模块，用于将所述媒体信息发送给媒体能力控制器；
    接收模块，用于接收所述媒体能力控制器发送的所述终端的媒体能力调整策略；
    处理模块，用于根据所述媒体能力调整策略调整所述终端对应的会场的媒体能力，所述媒体能力调整策略为所述媒体能力控制器根据所述媒体信息或网络侧设备的业务负载确定。
  18. 根据权利要求17所述的装置，其特征在于，所述确定模块用于：
    对所述来自终端的码流进行音频解码和视频解码；
    对音频解码结果和视频解码结果分别进行声纹检测和视频检测，识别出会议场景；
    根据所述视频解码结果和人员等级信息库进行人脸匹配，识别出与会人员等级；
    根据所述来自终端的码流探测出网络信息。
  19. 根据权利要求18所述的装置，其特征在于，所述接收模块还用于：
    在所述确定模块根据来自终端的码流确定终端的媒体信息之前，接收应用服务器发送的与会人员等级列表并保存；
    接收人工智能网关发送的和所述与会人员等级列表对应的用户人脸数据；
    所述处理模块还用于：将所述与会人员等级列表和所述用户人脸数据关联，
    得到所述人员等级信息库。
  20. 根据权利要求17所述的装置，其特征在于，所述会议场景包括发言场景、收听场景、茶歇场景、分组讨论场景或共享数据场景中的一种。
  21. 根据权利要求17-20任一项所述的装置，其特征在于，所述媒体能力调整策略包括：媒体能力的提高或降低，所述媒体能力包括分辨率、编码协议和编码参数中的至少一项，所述编码参数包括量化参数QP、带宽、帧率和抗丢包冗余度中的至少一项。
  22. 根据权利要求21所述的装置，其特征在于，当所述媒体信息指示所述终端对应的会场具有高优先级时，所述媒体能力调整策略为提高所述终端对应的会场的媒体能力；
    当所述媒体信息指示所述终端对应的会场具有低优先级时，所述媒体能力调整策略为降低所述终端对应的会场的媒体能力。
  23. 根据权利要求22所述的装置，其特征在于，所述提高所述终端对应的会场的媒体能力包括提高所述媒体增强装置的媒体能力，或提高所述终端的媒体能力；
    所述降低所述终端对应的会场的媒体能力包括降低所述媒体增强装置的媒体能力，或降低所述终端的媒体能力。
  24. 根据权利要求23所述的装置，其特征在于，
    若确定出的会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽大于带宽阈值、时延小于时延阈值时，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、降低QP和进行协议转换中的至少一项；
    若确定出的会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽小于带宽阈值、时延大于时延阈值时，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、提高抗丢包冗余度和进行协议转换中的至少一项；
    若所述终端的会议场景为茶歇场景或收听场景，所述终端对应的会场具有低优先级，则所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和降低编码带宽；
    当所述业务负载大于预设阈值时，若所述终端对应的会场的与会人员等级为付费用户，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率；
    当所述业务负载大于预设阈值时，若所述终端的与会人员等级为免费用户，所述终端对应的会场具有高优先级，则所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率。
  25. 根据权利要求17-24任一项所述的装置，其特征在于，所述业务负载包括中央处理器CPU占用率、终端数量、会场数量和会议类型。
  26. 一种媒体能力控制器，其特征在于，包括：
    接收模块，用于接收媒体增强装置发送的媒体信息，所述媒体信息包括：会议场景、与会人员等级和网络信息，所述网络信息包括带宽和时延；
    确定模块，用于根据所述媒体信息或网络侧设备的业务负载确定所述终端的的媒体能力调整策略；
    发送模块，用于向所述媒体增强装置发送所述终端的媒体能力调整策略，所述媒体能力调整策略用于指示所述媒体增强装置调整所述终端对应的会场的媒体能力。
  27. 根据权利要求26所述的媒体能力控制器，其特征在于，所述会议场景包括发言场景、收听场景、茶歇场景、分组讨论场景或共享数据场景中的一种。
  28. 据权利要求26或27所述的媒体能力控制器，其特征在于，所述媒体能力调整策略包括：
    媒体能力的提高或降低，所述媒体能力包括分辨率、编码协议和编码参数中的至少一项，所述编码参数包括量化参数QP、带宽、帧率和抗丢包冗余度中的至少一项。
  29. 根据权利要求28所述的媒体能力控制器，其特征在于，所述确定模块用于：
    当所述媒体信息指示所述终端对应的会场具有高优先级时，确定所述媒体能力调整策略为提高所述终端对应的会场的媒体能力；
    当所述媒体信息指示所述终端对应的会场具有低优先级时，确定所述媒体能 力调整策略为降低所述终端对应的会场的媒体能力。
  30. 根据权利要求29所述的媒体能力控制器，其特征在于，所述提高所述终端对应的会场的媒体能力包括提高所述媒体增强装置的媒体能力，或提高所述终端的媒体能力；
    所述降低所述终端对应的会场的媒体能力包括降低所述媒体增强装置的媒体能力，或降低所述终端的媒体能力。
  31. 根据权利要求30所述的媒体能力控制器，其特征在于，所述确定模块用于：
    若会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽大于带宽阈值、时延小于时延阈值时，所述终端对应的会场具有高优先级，则确定所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、降低QP和进行协议转换中的至少一项；
    若会议场景为发言场景，所述终端对应的会场的与会人员等级最高，且带宽小于带宽阈值、时延大于时延阈值时，所述终端对应的会场具有高优先级，则确定所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率、提高编码带宽、提高帧率、提高抗丢包冗余度和进行协议转换中的至少一项；
    若所述终端的会议场景为茶歇场景或收听场景，所述终端对应的会场具有低优先级，则确定所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和降低编码带宽；
    当所述业务负载大于预设阈值时，若所述终端对应的会场的与会人员等级为付费用户，所述终端对应的会场具有高优先级，则确定所述媒体能力调整策略为：提高所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率；
    当所述业务负载大于预设阈值时，若所述终端的与会人员等级为免费用户，所述终端对应的会场具有高优先级，则确定所述媒体能力调整策略为：降低所述终端对应的会场向所述网络侧设备发送码流的分辨率和所述网络侧设备向所述终端对应的会场发送码流的分辨率。
  32. 根据权利要求26-31任一项所述的媒体能力控制器，其特征在于，所述业务负载包括中央处理器CPU占用率、终端数量、会场数量和会议类型。
  33. 一种媒体能力动态调整装置，其特征在于，包括：
    处理器；以及
    存储器，用于存储所述处理器的可执行指令；
    其中，所述处理器配置为经由执行所述可执行指令来执行权利要求1-9或10-16任一项所述的媒体能力动态调整方法。
  34. 一种可读存储介质，所述可读存储介质中存储有执行指令，其特征在于，当媒体能力动态调整装置的至少一个处理器执行所述执行指令时，所述媒体能力动态调整装置执行权利要求1-9或10-16任一项所述的媒体能力动态调整方法。
PCT/CN2020/083405 2019-08-20 2020-04-05 媒体能力动态调整方法及装置 WO2021031575A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910767904.X 2019-08-20
CN201910767904.XA CN112422879B (zh) 2019-08-20 2019-08-20 媒体能力动态调整方法及装置

Publications (1)

Publication Number Publication Date
WO2021031575A1 true WO2021031575A1 (zh) 2021-02-25

Family

ID=74660088

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/083405 WO2021031575A1 (zh) 2019-08-20 2020-04-05 媒体能力动态调整方法及装置

Country Status (2)

Country Link
CN (1) CN112422879B (zh)
WO (1) WO2021031575A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114095308A (zh) * 2021-11-30 2022-02-25 新华三大数据技术有限公司 多媒体会议质量的增强方法、装置、设备及存储介质
EP4044615A4 (en) * 2019-10-31 2022-11-16 Huawei Technologies Co., Ltd. METHOD AND APPARATUS FOR ADJUSTING A PROPERTY OF A VIDEO STREAM
US20230007056A1 (en) * 2021-06-30 2023-01-05 Microsoft Technology Licensing, Llc Data stream prioritization for communication session

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113347472A (zh) * 2021-04-30 2021-09-03 北京奇艺世纪科技有限公司 音视频质量调节方法、装置、电子设备及存储介质
CN113824921A (zh) * 2021-10-29 2021-12-21 苏州慧工云信息科技有限公司 会议***及会议流程控制方法
CN114554136B (zh) * 2022-04-25 2022-07-29 浙江华创视讯科技有限公司 视频会议实现方法、多点控制单元、设备及可读存储介质
CN114827522B (zh) * 2022-06-22 2023-04-18 广州朗国电子科技股份有限公司 一种视频会议的分组交互方法、设备及存储介质
CN116456061B (zh) * 2023-06-15 2023-09-08 四川三思德科技有限公司 基于动态目标检测的智能社区监控管理方法、***及介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101640784A (zh) * 2008-07-28 2010-02-03 上海领世通信技术发展有限公司 一种在视频会议***中控制多画面复合的控制装置及方法
CN204316652U (zh) * 2015-01-13 2015-05-06 航天数字传媒有限公司 卫星视频会议***
US9088692B2 (en) * 2012-06-14 2015-07-21 Polycom, Inc. Managing the layout of multiple video streams displayed on a destination display screen during a videoconference
CN105100521A (zh) * 2014-05-14 2015-11-25 中兴通讯股份有限公司 远程会议中实现有序发言的方法及服务器
CN109525802A (zh) * 2018-11-27 2019-03-26 平安科技（深圳）有限公司 一种视频流传输方法和装置
CN109741754A (zh) * 2018-12-10 2019-05-10 上海思创华信信息技术有限公司 一种会议语音识别方法及***、存储介质及终端

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002176633A (ja) * 2000-12-08 2002-06-21 Nec Corp 多地点ビデオ会議システム、映像データ符号量制御方法、およびそのプログラムを記録した記録媒体
CN102256101B (zh) * 2011-07-26 2017-09-26 南京中兴软件有限责任公司 一种视频会议中媒体能力的转换方法、***及应用服务器
CN102625078B (zh) * 2012-03-08 2014-08-13 华为技术有限公司 一种远程会议控制方法、终端设备、mcu及视讯***
CN106210599B (zh) * 2015-04-30 2021-02-12 中兴通讯股份有限公司 一种多画面调整方法、装置及多点控制单元

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101640784A (zh) * 2008-07-28 2010-02-03 上海领世通信技术发展有限公司 一种在视频会议***中控制多画面复合的控制装置及方法
US9088692B2 (en) * 2012-06-14 2015-07-21 Polycom, Inc. Managing the layout of multiple video streams displayed on a destination display screen during a videoconference
CN105100521A (zh) * 2014-05-14 2015-11-25 中兴通讯股份有限公司 远程会议中实现有序发言的方法及服务器
CN204316652U (zh) * 2015-01-13 2015-05-06 航天数字传媒有限公司 卫星视频会议***
CN109525802A (zh) * 2018-11-27 2019-03-26 平安科技（深圳）有限公司 一种视频流传输方法和装置
CN109741754A (zh) * 2018-12-10 2019-05-10 上海思创华信信息技术有限公司 一种会议语音识别方法及***、存储介质及终端

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4044615A4 (en) * 2019-10-31 2022-11-16 Huawei Technologies Co., Ltd. METHOD AND APPARATUS FOR ADJUSTING A PROPERTY OF A VIDEO STREAM
US20230007056A1 (en) * 2021-06-30 2023-01-05 Microsoft Technology Licensing, Llc Data stream prioritization for communication session
US11632404B2 (en) * 2021-06-30 2023-04-18 Microsoft Technology Licensing, Llc Data stream prioritization for communication session
CN114095308A (zh) * 2021-11-30 2022-02-25 新华三大数据技术有限公司 多媒体会议质量的增强方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN112422879B (zh) 2022-10-28
CN112422879A (zh) 2021-02-26

Similar Documents

Publication Publication Date Title
WO2021031575A1 (zh) 媒体能力动态调整方法及装置
EP2863632B1 (en) System and method for real-time adaptation of a conferencing system to current conditions of a conference session
WO2018006574A1 (zh) 一种视频会议的屏幕共享实现方法
JP5320406B2 (ja) オーディオ処理の方法、システム、及び制御サーバ
US8659634B2 (en) Method and system for implementing three-party video call by mobile terminals
US20130010049A1 (en) Negotiate multi-stream continuous presence
US9311920B2 (en) Voice processing method, apparatus, and system
CN104160687A (zh) 识别并且转变到改进的voip会话
US9232244B2 (en) Efficient frame forwarding in large scale real-time screen content sharing meetings
WO2012075937A1 (zh) 视频通话方法及可视电话
US9743043B2 (en) Method and system for handling content in videoconferencing
CN103051864A (zh) 移动视频会议方法及其***
CN102883133A (zh) 基于单服务器实现大容量会议服务的***及方法
CN102915736A (zh) 混音处理方法和混音处理***
US11290685B2 (en) Call processing method and gateway
CN113612759A (zh) 一种基于sip协议的高性能高并发智能广播***及实现方法
WO2021082479A1 (zh) 调整视频流的属性的方法和装置
CN115209189B (zh) 一种视频流传输方法、***、服务器及存储介质
CN103686208B (zh) 一种视频云计算传输方法
US10924613B1 (en) Encoder pools for conferenced communications
KR20220033445A (ko) 몰입형 미디어를 처리하기 위한 방법 및 장치
CN105282473A (zh) 基于ippbx设备实现远程多媒体录制回放和监测的方法及***
WO2016206391A1 (zh) 一种视频会议网络流量控制方法和***
CN105827575B (zh) 一种传输控制方法、装置及电子设备
CN202103782U (zh) 一种具备复合功能会场终端的视频会议***

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20854019

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20854019

Country of ref document: EP

Kind code of ref document: A1