CN114268599A

CN114268599A - Method, device, equipment and medium for establishing instant messaging connection and instant messaging

Info

Publication number: CN114268599A
Application number: CN202111573699.7A
Authority: CN
Inventors: 吴蔚; 何婉; 李丽蓉
Original assignee: Beijing Qingyun Science And Technology Co ltd
Current assignee: Beijing Qingyun Science And Technology Co ltd
Priority date: 2021-12-21
Filing date: 2021-12-21
Publication date: 2022-04-01

Abstract

The invention discloses a method, a device, equipment and a medium for establishing instant messaging connection and instant messaging. The method comprises the following steps: after receiving the creation request signaling, the WebRTC server generates an offer and adds the offer to the local session description of the server; sending the offer to the WebRTC client; the WebRTC client acquires an offer and adds the offer to a client remote session description of the WebRTC client; generating an offer answer and adding the offer answer to the client local session description; sending the offer response to the WebRTC server side; the WebRTC server receives the offer answer and adds the offer answer to the server-side remote session description. The invention can realize the establishment of the instant communication connection between the WebRTC client and the WebRTC server so as to enable the media stream information of the WebRTC client to be instantly transmitted to the WebRTC server.

Description

Method, device, equipment and medium for establishing instant messaging connection and instant messaging

Technical Field

The embodiment of the invention relates to the technical field of communication, in particular to a method, a device, equipment and a medium for establishing instant messaging connection and instant messaging.

Background

With the development of the internet and the release and deployment of the HTML5, the Web client can realize Web Real-Time communication based on a Web Real-Time communication technology, which is a technology supporting audio, video and data sharing among Web Real-Time communication WebRTC clients, and the Real-Time communication capability can be achieved through a script language provided by the Web client.

In the prior art, data interaction can be performed between the WebRTC clients for web page instant messaging through the signaling server, but instant messaging of media stream information cannot be performed between the WebRTC clients for web page instant messaging and the WebRTC server.

Disclosure of Invention

Embodiments of the present invention provide a method, an apparatus, a device, and a medium for establishing an instant messaging connection and an instant messaging method, an apparatus, and a medium, so as to establish an instant messaging connection between a WebRTC client and a WebRTC server, and instantly transmit media stream information of the WebRTC client to the WebRTC server through the instant messaging connection.

In a first aspect, an embodiment of the present invention provides a method for creating an instant messaging connection, where the method is applied to an instant messaging system, and the instant messaging system includes: the method is characterized by comprising the following steps of:

after receiving a creation request signaling sent by the WebRTC client, the WebRTC server generates an offer containing a server-side session description protocol and adds the offer to a server-side local session description of the WebRTC server; sending the offer to the WebRTC client;

the WebRTC client acquires the offer and adds the offer to a client remote session description of the WebRTC client; generating an offer response containing a client session description protocol and adding the offer response to the client local session description of the WebRTC client; sending the offer response to the WebRTC server;

and the WebRTC server receives the offer response and adds the offer response to a server-side remote session description of the WebRTC server so as to complete the creation of the instant messaging connection between the WebRTC client and the WebRTC server.

Further, before receiving the creation request signaling sent by the WebRTC client, the WebRTC server further includes:

and the WebRTC client initializes and sends the creation request instruction to the WebRTC server.

Further, the WebRTC client performs initialization, including:

the WebRTC client configures information of a relay server corresponding to a network address translation protocol;

the WebRTC client adds a first information transmission channel, wherein the first information transmission channel comprises: a first media stream channel and/or a first data channel; the first media stream channel is connected with a media device of the WebRTC client.

Further, the WebRTC server serves as a signaling server and is configured to receive a creation request signaling sent by the WebRTC client.

Further, after receiving the creation request signaling sent by the WebRTC client, before generating an offer containing a server-side session description protocol, the WebRTC server further includes:

and the WebRTC server side carries out initialization.

Further, the WebRTC server side performs initialization, including:

adding a second information transmission channel to the WebRTC server, wherein the second information transmission channel comprises: a second media stream channel and/or a second data channel.

Further, the WebRTC server-side generating the offer includes:

the WebRTC server side acquires a server side session description protocol; the server-side session description protocol comprises the following steps: a server-side media codec, a server-side protocol address and a server-side network transmission protocol;

the WebRTC server generates the offer based on the server-side session description protocol.

Further, the WebRTC client generates an offer response containing a client session description protocol, including:

the WebRTC client acquires a client session description protocol; the client session description protocol comprises: a client media codec, a client protocol address and a client network transport protocol;

the WebRTC client generates an offer response based on the client session description protocol.

In a second aspect, an embodiment of the present invention provides a method for creating an instant messaging connection, where the method is applied to a WebRTC client, and the method includes:

obtaining an offer sent by the WebRTC server side, and adding the offer to a client side remote session description of the WebRTC client side;

generating an offer reply comprising a client session description protocol;

adding the offer answer to a client local session description of the WebRTC client;

and sending the offer response to the WebRTC server so that the WebRTC server completes the creation of the instant messaging connection between the WebRTC client and the WebRTC server.

In a third aspect, an embodiment of the present invention provides a method for creating an instant messaging connection, where the method is applied to a WebRTC server, and the method includes:

after receiving a creation request signaling sent by a WebRTC client, generating an offer containing a server-side session description protocol;

adding the offer to a server-side local session description of the WebRTC server side;

sending the offer to the WebRTC client;

and receiving an offer response sent by the WebRTC client, and adding the offer response into the server-side remote session description of the WebRTC server side to complete the creation of the instant messaging connection between the WebRTC client and the WebRTC server side.

In a fourth aspect, an embodiment of the present invention provides an instant messaging method, which is applied to a web instant messaging WebRTC client, and the method includes:

acquiring media stream information input by media equipment, wherein the media equipment is connected with the WebRTC client through a first media stream track;

and transmitting the media stream information to the WebRTC server through an instant messaging connection between the WebRTC client and the WebRTC server, wherein the instant messaging connection is established by adopting an instant messaging connection establishing method.

Further, the WebRTC client is provided with a network address translation device, and a relay server corresponding to the network address translation device includes: traversing STUN servers or forwarding TURN servers;

if the network address conversion equipment is asymmetric, sending a request message to a STUN server, acquiring a network address of the WebRTC client, and transmitting the media stream information and the network address of the WebRTC client to the WebRTC server;

if the NAT device is of a symmetric type, the media stream information is transmitted to a TURN server, so that the TURN server forwards the media stream information to the WebRTC server side.

Further, acquiring the media stream information input by the media device includes:

sending a permission access request to the media device;

and receiving media stream information sent by the media equipment, wherein the media stream information is acquired after the media equipment allows the permission access request.

In a fifth aspect, an embodiment of the present invention further provides a system for instant messaging connection, where the system includes: the method comprises the steps that a webpage instant messaging WebRTC client and a webpage instant messaging WebRTC server are connected;

the WebRTC server is used for generating an offer containing a server-side session description protocol and adding the offer to a server-side local session description of the WebRTC server after receiving a creation request signaling sent by the WebRTC client; sending the offer to the WebRTC client;

the WebRTC client is used for acquiring the offer and adding the offer into a client remote session description of the WebRTC client; generating an offer response containing a client session description protocol and adding the offer response to the client local session description of the WebRTC client; sending the offer response to the WebRTC server;

the WebRTC server is further configured to receive the offer response, and add the offer response to a server-side remote session description of the WebRTC server, so as to complete creation of an instant messaging connection between the WebRTC client and the WebRTC server.

In a sixth aspect, an embodiment of the present invention further provides an apparatus for creating an instant messaging connection, which is integrated in a WebRTC client for web page instant messaging, where the apparatus includes:

the offer acquisition module is used for acquiring an offer sent by the WebRTC server and adding the offer to the client remote session description of the WebRTC client;

the offer response generation module is used for generating an offer response containing the session description protocol of the client;

an offer response adding module, configured to add the offer response to a client local session description of the WebRTC client;

and the offer response sending module is used for sending the offer response to the WebRTC server so that the WebRTC server completes the creation of the instant communication connection between the WebRTC client and the WebRTC server.

In a seventh aspect, an embodiment of the present invention further provides an instant messaging device integrated in a WebRTC server for web page instant messaging, where the instant messaging device includes:

the system comprises an offer generation module, a creation module and a communication module, wherein the offer generation module is used for generating an offer containing a server-side session description protocol after receiving a creation request signaling sent by a WebRTC client;

an offer adding module, configured to add the offer to a server-side local session description of the WebRTC server side;

an offer sending module to send the offer to the WebRTC client;

and the creating module is used for receiving the offer response sent by the WebRTC client and adding the offer response into the server-side remote session description of the WebRTC server so as to complete the creation of the instant communication connection between the WebRTC client and the WebRTC server.

In an eighth aspect, an embodiment of the present invention further provides a terminal device, which includes an input device, an output device, a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the method for creating an instant messaging connection according to any one of the embodiments of the present invention, or implements the instant messaging method according to any one of the embodiments of the present invention when executing the program.

In a ninth aspect, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the method for creating an instant messaging connection according to any one of the embodiments of the present invention, or implements the instant messaging method according to any one of the embodiments of the present invention.

After receiving a creation request signaling sent by a WebRTC client, a WebRTC server generates an offer containing a server session description protocol and adds the offer to a server local session description of the WebRTC server; sending the offer to the WebRTC client; the WebRTC client acquires an offer and adds the offer to a client remote session description of the WebRTC client; generating an offer response containing a client session description protocol and adding the offer response into a client local session description of the WebRTC client; sending the offer response to the WebRTC server side; the WebRTC server receives the offer response and adds the offer response to the server remote session description of the WebRTC server to complete the creation of the instant messaging connection between the WebRTC client and the WebRTC server, so that the problem that the instant messaging of the media stream information cannot be performed between the WebRTC client and the WebRTC server is solved, the instant messaging connection between the WebRTC client and the WebRTC server is created, and the media stream information of the WebRTC client is instantly transmitted to the WebRTC server through the instant messaging connection.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.

Fig. 1 is a flowchart of a method for creating an instant messaging connection according to a first embodiment of the present invention;

FIG. 2 is an architectural diagram of an instant messaging system between a WebRTC client and a WebRTC server;

fig. 3 is a flowchart of a method for creating an instant messaging connection according to another embodiment of the present invention;

fig. 4 is a flowchart of a method for creating an instant messaging connection according to a third embodiment of the present invention;

fig. 5 is a flowchart of a method for creating an instant messaging connection according to a fourth embodiment of the present invention;

fig. 6 is a flowchart of an instant messaging method according to a fifth embodiment of the present invention;

fig. 7 is a sequence diagram of a method for creating an instant messaging connection and a method for instant messaging in an embodiment of the present invention;

fig. 8 is a timing diagram of a face recognition method based on an instant messaging connection between a WebRTC client and a WebRTC server;

fig. 9 is a schematic structural diagram of an apparatus for creating an instant messaging connection according to a sixth embodiment of the present invention;

fig. 10 is a schematic structural diagram of an instant messaging connection creation apparatus in a seventh embodiment of the present invention;

fig. 11 is a schematic structural diagram of an instant messaging connection creation apparatus in an eighth embodiment of the present invention;

fig. 12 is a schematic structural diagram of an instant messaging device according to a ninth embodiment of the present invention;

fig. 13 is a schematic structural diagram of a terminal device in the tenth embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.

Example one

Fig. 1 is a flowchart of a method for creating an instant messaging connection according to an embodiment of the present invention, where this embodiment is applicable to a case of creating an instant messaging connection between a WebRTC client and a WebRTC server, and the method can be executed by an instant messaging system according to an embodiment of the present invention. The instant communication system includes: the system comprises a web page instant messaging WebRTC client and a web page instant messaging WebRTC server.

Fig. 2 is an architecture diagram of an instant messaging system between a WebRTC client and a WebRTC server. The WebRTC client is a client supporting Web Real-Time Communication (WebRTC), and can also be simply considered as a browser, and can provide a simple script based on the browser to realize an instant Communication function of audio, video and data between the browsers; the client can be installed in a local terminal device such as a local computer; the WebRTC server is a server supporting web page instant messaging, and the server can be installed in a remote terminal device such as a server and provides services for a client.

As shown in fig. 2, a Network Address Translation (NAT) device may be disposed between an intranet where the WebRTC client is located and a public network where the WebRTC server is located, and the NAT device is commonly used in an intranet where multiple hosts access the public network only through one public IP Address to isolate the intranet from the public network, and an extranet cannot actively access the intranet. The NAT device may include a router or firewall, etc.

The WebRTC client needs to be able to communicate with the WebRTC server only through the penetration or forwarding of the relay server, i.e., the Coturn server. The Coturn server includes: penetrate STUN server or forward TURN server.

As shown in fig. 1, the method specifically includes the following steps:

s110, after receiving a creation request signaling sent by a WebRTC client, a WebRTC server generates an offer containing a server session description protocol and adds the offer to a server local session description of the WebRTC server; the offer is sent to the WebRTC client.

The WebRTC server can be a WebRTC server realized by using an asyncio library of python programming language, and an audio or video processing algorithm can be inserted into the WebRTC server to process media stream information transmitted by the client.

In the embodiment of the present invention, the WebRTC server side may be used to establish a connection with the WebRTC client to receive media stream information of the client, and may also be used as a signaling server to receive a creation request signaling sent by the WebRTC client, where the creation request signaling is used to instruct the WebRTC server side to generate an offer.

The Session Description Protocol (SDP) is a Protocol format for describing streaming media initialization parameters for the purposes of Session notification, Session invitation, multimedia Session initialization in other forms, and the like. The session description protocol may include: session name and intent, media codecs that make up the session, IP address and network transport protocol, etc.

The WebRTC server side is provided with a server side local session description and a server side remote session description, the server side local session description is used for storing a session description protocol of the server side, and the local meaning is the WebRTC server side; the server-side remote session description is used for storing a session description protocol of a WebRTC client to which the WebRTC server side is connected, and the meaning of "remote" is that the remote side of the WebRTC server side is the WebRTC client.

Specifically, the WebRTC server, as a signaling server, may receive a creation request signaling sent by the WebRTC client, and after receiving the creation request signaling, the WebRTC server performs the following three operations: the first operation is to generate an offer containing the server-side session description protocol. The second operation is to add the offer to the server-side local session description, or add the server-side session description protocol in the offer to the server-side local session description, so that the WebRTC server can store its own configuration information. The third operation is to send an offer containing the server-side session description protocol to the WebRTC client, so as to also feed back the configuration information of itself to the WebRTC client.

In the embodiment of the present invention, the execution sequence of the second operation and the third operation is not limited, and the second operation and the third operation may also be executed simultaneously.

Illustratively, a connection interface provided by a WebRTC server is called to send a creation request signaling to the WebRTC server, and in the interface, the WebRTC server generates an offer containing a server session description protocol and feeds the offer back to the WebRTC client; and adds the server-side session description protocol to the server-side local session description.

S120, the WebRTC client acquires the offer and adds the offer to the client remote session description of the WebRTC client; and generating an offer response containing a client session description protocol, sending the offer response to the WebRTC server, and adding the offer response to the client local session description of the WebRTC client.

The client session description protocol refers to a session description protocol of a client.

Similar to the WebRTC server, the WebRTC client also has a client local session description and a client remote session description, the client local session description is used for storing the session description protocol of the client itself, and the meaning of "local" is the WebRTC client itself; the client remote session description is used for saving a session description protocol of a WebRTC server side to which the client is connected, and the meaning of remote means that the client is the remote side of the WebRTC client, namely the WebRTC server side.

Specifically, when the WebRTC client receives an offer containing a server-side session description protocol fed back by the WebRTC server, the following four operations are performed: the first operation is to add the offer to the client remote session description of the WebRTC client, or add the server-side session description protocol in the offer to the client remote session description of the WebRTC client, so that the WebRTC client can know the configuration information of the WebRTC server. The second operation is to acquire the client session description protocol of the client itself and generate an offer response containing the client session description protocol. The third operation is to add the offer response to the client local session description of the WebRTC client, or add the client session description protocol in the offer response to the client local session description of the WebRTC client, so that the WebRTC client can store its own configuration information. And the fourth operation is to send an offer response containing the client session description protocol to the WebRTC server, so that the WebRTC server can store the configuration information of the WebRTC client.

In the embodiment of the present invention, the execution sequence of the third operation and the fourth operation is not limited, and the third operation and the fourth operation may also be executed simultaneously.

It should be noted that the offer response may be considered as a response of the WebRTC client to the offer sent by the WebRTC server. The offer mainly comprises the session description protocol of the server side, the offer response mainly comprises the session description protocol of the client side, and the content contained in the offer response does not depend on the content contained in the offer.

In the client local session description in which the offer response is added to the WebRTC client, it is necessary to wait for an Interactive Connectivity Establishment agent (ICE) in the client to collect information of methods that can be used for connection between the WebRTC client and the WebRTC server, such as IP addresses and available ports at both ends. Because of the multi-layer firewall and NAT device barriers between the peers, we need a mechanism to collect the IP of the public line between the peers, and the ICE agent is a good helper to do this. If the STUN server is configured, the ICE agent queries an external STUN server to obtain the public IP and port of the local end; if a TURN server is configured, ICE will use the TURN server as a candidate, and when the end-to-end connection fails, the data will be forwarded through the designated intermediary.

Illustratively, the server-side session description protocol is added to the client remote session description of the WebRTC client by calling a client remote description setup function. And generating an offer response based on the acquired session description protocol of the client by calling an offer response generation function, and calling a client local description setting function to add the client session description protocol to the client local session description of the WebRTC client.

And S130, the WebRTC server receives the offer response and adds the offer response to the server remote session description of the WebRTC server so as to complete the creation of the instant communication connection between the WebRTC client and the WebRTC server.

Specifically, after receiving the offer response, the WebRTC server adds the offer response to the server remote session description of the WebRTC server, or adds the client session description protocol in the offer response to the client remote session description of the WebRTC client, so that the WebRTC server can know the configuration information of the WebRTC client.

Illustratively, the offer response is sent to the WebRTC server by calling a remote description interface provided by the WebRTC server. In the remote description interface, the WebRTC server adds the offer response or the client session description protocol in the offer response to the server-side remote session description of the WebRTC server.

So far, the client local session description and the client remote session description of the WebRTC client, and the server local session description and the server remote session description of the WebRTC server have stored the session description protocol of the response, so as to complete the creation of the instant messaging connection between the WebRTC client and the WebRTC server, and can be used for transmitting the media stream information acquired by the WebRTC client.

According to the technical scheme of the embodiment, after receiving a creation request signaling sent by a WebRTC client, a WebRTC server generates an offer containing a server session description protocol and adds the offer to a server local session description of the WebRTC server; sending the offer to the WebRTC client; the WebRTC client acquires an offer and adds the offer to a client remote session description of the WebRTC client; generating an offer response containing a client session description protocol and adding the offer response into a client local session description of the WebRTC client; sending the offer response to the WebRTC server side; the WebRTC server receives the offer response and adds the offer response to the server remote session description of the WebRTC server so as to complete the creation of the instant messaging connection between the WebRTC client and the WebRTC server, and the instant messaging connection between the WebRTC client and the WebRTC server can be created, and the media stream information of the WebRTC client can be instantly transmitted to the WebRTC server through the instant messaging connection.

Example two

Fig. 3 is a flowchart of a method for creating an instant messaging connection according to a second embodiment of the present invention, which is optimized based on the second embodiment.

As shown in fig. 3, the method of this embodiment specifically includes the following steps:

and S210, the WebRTC client initializes and sends a creation request instruction to the WebRTC server.

Specifically, the WebRTC client is initialized, and the initialized content may include: the method comprises the steps of configuring information of a CoTurn server required by communication between a client and a server, and adding a first information transmission channel in the WebRTC client according to service requirements, wherein the first information transmission channel is used for transmitting media stream information sent by media equipment of the WebRTC client or transmitting data information sent by the WebRTC server.

It should be noted that the instant messaging connection between the WebRTC client and the WebRTC server is used to transmit the media stream information acquired by the WebRTC client, and meanwhile, the WebRTC server may also be used as a signaling server to receive the creation request signaling sent by the WebRTC client. The WebRTC server serves as a general signaling server, receives a process of creating a request signaling sent by the WebRTC client, and may be implemented by using the existing technology, which is not limited in the embodiment of the present invention.

Illustratively, for the WebRTC client, the information configuration is performed on the Coturn server required by the WebRTC client by calling the Coturn server information configuration function, and the first information transmission channel is added to the WebRTC client by calling the channel addition function, so as to complete the initialization operation of the WebRTC client.

Optionally, initializing the WebRTC client includes:

configuring information of a relay server corresponding to a network address translation protocol of a WebRTC client;

adding a first information transmission channel in the WebRTC client, wherein the first information transmission channel comprises: a first media stream channel and/or a first data channel; the first media stream channel is connected with a media device of the WebRTC client.

Specifically, the operation of initializing the client may include configuring relay server information and adding the first information transmission channel. The first information transmission channel includes: a first media stream channel and/or a first data channel. The first media stream channel is connected with the media device of the WebRTC client and used for transmitting media stream information sent by the media device; the first data channel is used for receiving data information except media stream information sent by the WebRTC server side.

The media stream information refers to media information transmitted by a streaming transmission mode, and includes: audio, video and other multimedia files, video refers to a video stream consisting of successive frame image changes. Streaming mainly refers to parsing a whole multimedia file such as audio and video into compressed packets in a specific compression manner for sequential or real-time transmission. The streaming transmission mode of downloading and playing media stream information greatly shortens the starting delay, greatly reduces the requirement on the system cache capacity, and greatly reduces the waiting time of users.

Thus, the first media stream channel may include a first audio channel for transmitting audio sent by the media device and a first video channel for transmitting video sent by the media device. According to the embodiment of the invention, the corresponding first media stream channel can be added according to the type of the media stream information required to be transmitted when the WebRTC client is connected with the WebRTC server, which is determined by the service requirement.

For example, the information configuring the relay server corresponding to the network address translation protocol of the WebRTC client may be connection information configuring the relay Coturn server according to the network address translation protocol of the WebRTC client, and the connection information may include: a transport address directly associated with the network interface, a transport address translated by the NAT, and a relay address assigned by the TURN server.

For example, the operation of adding the first information transmission channel in the WebRTC client may be: adding a first media stream channel connected with a media device in the WebRTC client by calling a media stream channel function.

And S220, initializing after receiving the creation request signaling sent by the WebRTC client by the WebRTC server.

Specifically, the initialization of the WebRTC server may include: and adding a second information transmission channel in the WebRTC server, wherein the second information transmission channel is used for transmitting media stream information sent by the media device of the WebRTC client or transmitting data information sent by the WebRTC client.

Illustratively, the second information transmission channel is added in the WebRTC server side by calling a channel adding function.

Optionally, the WebRTC server side performs initialization, including:

the WebRTC server adds a second information transmission channel, and the second information transmission channel comprises: a second media stream channel and/or a second data channel.

Specifically, for the same reason as that of adding the first transmission channel in the WebRTC client, a second information transmission channel is added in the WebRTC server, where the second information transmission channel includes: a second media stream channel and/or a second data channel; the second media stream channel is used for receiving the media stream information sent by the WebRTC client and transmitting the media stream information to the WebRTC server; and the second data channel is used for the WebRTC server side to send data information except the media stream information to the WebRTC client side.

The second media stream path may include: a second audio channel and a second video channel; according to the embodiment of the invention, the corresponding second media stream channel can be added according to the type of the media stream information required to be transmitted when the WebRTC client is connected with the WebRTC server, which is determined by the service requirement.

For example, the operation of adding the second information transmission channel in the WebRTC server side may be: and adding a second media stream channel in the WebRTC server side by calling a media stream channel function.

For example, under the service requirement of face recognition, a first video stream channel which is connected with a camera of a client is added in a WebRTC client; correspondingly, a second video stream channel is added in the WebRTC server side. And creating a first data channel in the WebRTC client by calling a function of the first additional data channel listener, wherein the first data channel is used for receiving data information transmitted by the WebRTC server side except the media stream information and transmitting the data information to the WebRTC client. Correspondingly, a second data channel is created in the WebRTC server by calling the function of the second additional data channel listener, and the second data channel is used for sending data information except the media stream information to the WebRTC client side by the WebRTC server.

S230, the WebRTC server generates an offer containing a server-side session description protocol and adds the offer to a server-side local session description of the WebRTC server; the offer is sent to the WebRTC client.

S240, the WebRTC client acquires the offer and adds the offer to the client remote session description of the WebRTC client; generating an offer response containing a client session description protocol and adding the offer response into a client local session description of the WebRTC client; and sending the offer response to the WebRTC server.

And S250, the WebRTC server side receives the offer response and adds the offer response to the server-side remote session description of the WebRTC server side so as to complete the creation of the instant communication connection between the WebRTC client and the WebRTC server side.

According to the technical scheme of the embodiment, the WebRTC client side is initialized, and a creation request instruction is sent to the WebRTC server side; the WebRTC server side initializes after receiving a creation request signaling sent by the WebRTC client side, generates an offer containing a server side session description protocol and adds the offer to a server side local session description of the WebRTC server side; sending the offer to the WebRTC client; the WebRTC client acquires an offer and adds the offer to a client remote session description of the WebRTC client; generating an offer response containing a client session description protocol and adding the offer response into a client local session description of the WebRTC client; sending the offer response to the WebRTC server side; the WebRTC server receives the offer response and adds the offer response to the server remote session description of the WebRTC server so as to complete the creation of the instant messaging connection between the WebRTC client and the WebRTC server, the instant messaging connection between the WebRTC client and the WebRTC server can be created, and information transmission channels are respectively added in the WebRTC client and the WebRTC server so as to realize the instant transmission of the media stream information of the WebRTC client to the WebRTC server.

Optionally, the WebRTC server-side generating the offer includes:

the WebRTC server side generates an offer based on a server side session description protocol.

Wherein, the offer is a session description protocol containing a WebRTC server end

Specifically, the server-side session description protocol includes: server side media codec, server side protocol address and server side network transmission protocol. The server media codec is used for coding the media stream information in the WebRTC server side; and the protocol address of the server side is the IP address of the WebRTC server side.

Illustratively, the WebRTC server generates the offer based on the server-side session description protocol by calling an offer creation function.

Optionally, the WebRTC client generates an offer response including a client session description protocol, including:

the WebRTC client acquires a client session description protocol; the client session description protocol comprises the following steps: a client media codec, a client protocol address and a client network transport protocol;

Specifically, the client session description protocol includes: a client media codec, a client protocol address, and a client network transport protocol. The client media codec is used for decoding media stream information in the WebRTC client; the client protocol address is the IP address of the WebRTC client.

Illustratively, the WebRTC client generates an offer answer based on the client session description protocol by calling an offer answer creation function.

EXAMPLE III

Fig. 4 is a flowchart of a method for creating an instant messaging connection according to a third embodiment of the present invention, where this embodiment is applicable to a case of creating an instant messaging connection between a WebRTC client and a WebRTC server, and the method may be executed by an instant messaging establishing apparatus in the third embodiment of the present invention, where the apparatus may be implemented in a software and/or hardware manner, and the apparatus may be integrated in the WebRTC client.

As shown in fig. 4, the method specifically includes the following steps:

and S310, acquiring the offer sent by the WebRTC server side, and adding the offer to the client remote session description of the WebRTC client side.

S320, generating an offer response containing the session description protocol of the client.

And S330, adding the offer response to the client local session description of the WebRTC client.

And S340, sending the offer response to the WebRTC server so that the WebRTC server completes the creation of the instant messaging connection between the WebRTC client and the WebRTC server.

The technical scheme of the embodiment is applied to the WebRTC client, and the offer is added to the client remote session description of the WebRTC client by acquiring the offer sent by the WebRTC server; generating an offer response containing a client session description protocol, and adding the offer response to a client local session description of the WebRTC client; and sending the offer response to the WebRTC server so that the WebRTC server completes the creation of the instant messaging connection between the WebRTC client and the WebRTC server, and realizes the instant transmission of the media stream information of the WebRTC client to the WebRTC server.

Example four

Fig. 5 is a flowchart of a method for creating an instant messaging connection according to a fourth embodiment of the present invention, where this embodiment is applicable to a case of creating an instant messaging connection between a WebRTC client and a WebRTC server, and the method may be executed by an instant messaging establishing apparatus according to the fourth embodiment of the present invention, where the apparatus may be implemented in a software and/or hardware manner, and the apparatus may be integrated in the WebRTC server.

As shown in fig. 5, the method specifically includes the following steps:

and S410, after receiving the creation request signaling sent by the WebRTC client, generating an offer and sending the offer to the WebRTC client.

And S420, adding the contract into the server-side local session description of the WebRTC server side.

And S430, sending the offer to the WebRTC client.

And S440, receiving the offer response sent by the WebRTC client, and adding the offer response to the server-side remote session description of the WebRTC server side to complete the creation of the instant communication connection between the WebRTC client and the WebRTC server side.

The technical scheme of the embodiment is applied to the WebRTC server, and after receiving a creation request signaling sent by the WebRTC client, the offer is generated and sent to the WebRTC client. The offer is added to the server-side local session description at the WebRTC server side. And receiving an offer response sent by the WebRTC client, and adding the offer response into a server-side remote session description of the WebRTC server so as to complete the creation of the instant communication connection between the WebRTC client and the WebRTC server and realize the instant transmission of the media stream information of the WebRTC client to the WebRTC server.

EXAMPLE five

Fig. 6 is a flowchart of an instant messaging method according to a fifth embodiment of the present invention, where this embodiment is applicable to a case where media stream information acquired by a client is transmitted based on a created instant messaging connection between a WebRTC client and a WebRTC server, and the method may be executed by an instant messaging device according to an embodiment of the present invention, where the device may be implemented in a software and/or hardware manner, and the device may be integrated in the WebRTC client.

As shown in fig. 6, the method specifically includes the following steps:

and S510, acquiring media stream information input by the media device, wherein the media device is connected with the WebRTC client through a first media stream track.

The media device refers to a device that generates media stream information in the WebRTC client, and may be, for example, an image capture device such as a video camera or a sound capture device such as a headphone. The media device is connected with the WebRTC client through a first media stream track in the process of establishing instant messaging connection between the WebRTC client and the webpage instant messaging WebRTC server, and the first media stream track is connected and accessed into the instant messaging connection.

Specifically, the media stream information input by the media device is acquired through the first media stream track, so that the media stream information is transmitted from the media device to the instant messaging connection.

In addition, if the media stream information is video information, the video information can be displayed in the WebRTC client, i.e., a browser, by calling a uniform resource locator of the client.

And S520, transmitting the media stream information to the WebRTC server through the instant messaging connection between the WebRTC client and the WebRTC server, wherein the instant messaging connection is established by adopting an instant messaging connection establishing method.

Specifically, the instant messaging connection between the WebRTC client and the WebRTC server established by the method for establishing an instant messaging connection provided in any of the embodiments transmits media stream information to the WebRTC server, so that the WebRTC server receives and processes the media stream information, thereby implementing instant messaging of the media stream information between the WebRTC client and the WebRTC server.

According to the technical scheme of the embodiment, the media device is connected with the WebRTC client through the first media stream track by acquiring the media stream information input by the media device; and transmitting the media stream information to the WebRTC server through the instant messaging connection between the WebRTC client and the WebRTC server, wherein the instant messaging connection is established by adopting an instant messaging connection establishing method, so that the instant messaging of the media stream information between the WebRTC client and the WebRTC server is realized.

Optionally, the WebRTC client is provided with a network address translation device, and a relay server corresponding to the network address translation device includes: traversing STUN servers or forwarding TURN servers;

if the network address conversion equipment is asymmetric, sending a request message to the STUN server, acquiring a network address of the WebRTC client, and transmitting the media stream information and the network address to the WebRTC server;

if the NAT device is of a symmetric type, the media stream information is transmitted to the TURN server, so that the TURN server forwards the media stream information to the WebRTC server side.

When a host where multiple WebRTC clients are located accesses a public network where a WebRTC server is located through a public IP Address, a Network Address Translation (NAT) device is usually installed in the host of the WebRTC client to isolate an internal network where the WebRTC client is located from the public network where the WebRTC server is located, so that an external network cannot actively access the internal network. The NAT device may include a router or firewall, etc. NAT devices include two types: symmetric NAT and asymmetric NAT.

When the WebRTC client needs to transmit the media stream information with the WebRTC server with the assistance of the relay server Coturn, the relay server may also include two types: penetrate STUN server or forward TURN server.

Specifically, if the network address translation device is asymmetric, a request message is sent to the STUN server, the network address of the WebRTC client is acquired, and the media stream information and the network address of the WebRTC client are transmitted to the WebRTC server.

Among them, the STUN server (Simple Traversal of UDP Through Network Address translations), i.e. a Simple Traversal method for NAT. The WebRTC client sends a request message to a STUN server outside the NAT through UDP, the STUN server receives the request message, generates a response message and sends the response message to the WebRTC client through the NAT, and the response message carries a source port of the request message, namely an external port corresponding to the WebRTC client on the NAT. The WebRTC client learns the external address on the NAT through the content in the response message body, and informs the WebRTC server side that the RTP receiving address and the port number of the WebRTC client are the address and the port number outside the NAT. Because the NAT mapping table item of the media stream is pre-established on the NAT through the STUN protocol, the media stream can smoothly pass through the NAT.

Specifically, if the network address translation device is of a symmetric type, the media stream information is transmitted to the TURN server, so that the TURN server forwards the media stream information to the WebRTC server.

If the network address translation device is of a symmetric type, the WebRTC client cannot acquire its own network address through the STUN server, and therefore, a TURN server (Traversal Using relay around NAT) is required to forward the media stream information.

Optionally, the obtaining of the media stream information input by the media device includes:

sending a permission access request to the media device;

Specifically, if the media device is provided with access rights, the WebRTC client may also send a request for access rights to the media device before acquiring the media stream information input by the media device, and if the media device allows access, receive the media stream information sent by the media device.

Fig. 7 is a timing diagram of a method of creating an instant messaging connection and instant messaging in an embodiment of the present invention. As shown in fig. 7, the specific steps of creating an instant messaging connection between the WebRTC client and the WebRTC server and performing instant messaging are as follows:

the first step is as follows: initializing the WebRTC client, wherein the initialization of the WebRTC client comprises the following steps: and adding a first information transmission channel in the WebRTC client and configuring information of the relay server.

And secondly, the WebRTC client sends a creation request instruction to the WebRTC server.

Thirdly, initializing the WebRTC server after receiving the creation request instruction, wherein the initialization of the WebRTC server comprises the following steps: a second information transmission channel is added. And generating an offer containing the server-side session description protocol, and adding the offer to the server-side local session description.

And fourthly, the WebRTC server side sends the offer to the WebRTC client side.

Fifth, the WebRTC client adds the offer to the client remote session description of the client.

And sixthly, the WebRTC client generates an offer response containing the client session description protocol, and adds the offer response into the client local session description.

And seventhly, the WebRTC client sends the offer response to the WebRTC server.

And eighthly, the WebRTC server adds the offer response to the server-side remote session description of the server side, thereby completing the establishment of the instant communication connection.

The ninth step, the WebRTC client acquires the media stream information input by the media equipment;

and step ten, the WebRTC client transmits the media stream information to the WebRTC server through the established instant messaging connection.

On the basis of the above embodiment, the technical solution of the present invention is further explained by combining the face recognition scene and the above establishment method of instant messaging connection and instant messaging method. Fig. 8 is a timing chart of a face recognition method based on an instant messaging connection between a WebRTC client and a WebRTC server, as shown in fig. 8, the face recognition system includes the WebRTC client and the WebRTC server, the WebRTC client includes an access party client and a face authentication H5 page, and the face authentication H5 page is displayed in the access party client; the WebRTC server comprises: the system comprises a face authentication server end and a butt-joint party server, wherein the face authentication server end is arranged on the butt-joint party server. The specific steps of establishing an instant messaging connection between the WebRTC client and the WebRTC server and performing face authentication may be:

in the first step, the access party client displays a face authentication H5 page. Specifically, the user may scan a face authentication two-dimensional code provided by the access client or other trigger conditions, so that the access client returns and displays the face authentication H5 page.

Secondly, the user inputs identity information in a face authentication H5 page, and whether the person of the face authentication matches with the person of the identity information is judged.

And thirdly, after confirming that the identity information is matched, the face authentication H5 page sends a permission request of the camera to the user. And after the obtained authorization of the user, opening the camera by calling a function of the user media equipment, and obtaining the face video stream of the user, which is collected by the camera.

Fourthly, according to the method for establishing the instant messaging connection between the WebRTC client and the WebRTC server provided in the first to fourth embodiments of the present invention, the instant messaging connection between the face authentication H5 page and the face authentication server is established, in the instant messaging connection, the face authentication H5 page is provided with the first media stream channel and the first data channel, the face authentication server is provided with the second media stream channel and the second data channel, and the access party client is provided with the network address translation device and the corresponding relay server. And transmitting the face video stream acquired by the face authentication H5 page to a face authentication server side based on the created instant messaging connection.

And fifthly, converting the received face video stream into video frames by the face authentication server.

Sixthly, the face authentication server sends a notification that the face is put in a prompt box to the face authentication H5 page through a data channel, wherein the data channel comprises: a second data channel arranged at the face authentication server side and a first data channel arranged in a face authentication H5 page. And then, the face authentication server side OpenCV and dlib detects a face, if the face is not detected within a first preset time, a 'detection overtime' notice is sent to prompt the user to retry the click, and the sixth step is repeated.

And seventhly, if the face is detected, the face authentication server carries out face scoring on the preset video frames of the received face video stream through the face detectors of the OpenCV and dlib, and a frame with the highest score is reserved.

And eighthly, the face authentication server randomly selects an action from an action library (shaking head, opening mouth, blinking, nodding head), sends a corresponding action prompt to a face authentication H5 page through a data channel, and the action prompt is determined according to the randomly selected action in the action library, such as 'please open mouth and leave mouth'. And then, performing living body detection through an OpenCV (open circuit vehicle) and dlib facial feature detector, if no action is detected within a second preset time, sending a detection overtime notification to prompt the user to retry the click, and repeating the sixth step.

Ninthly, if the living body detection of the face authentication server side is successful, sending an 'identity verification in progress' notice to a face authentication H5 page, then transmitting the name, the identity card number and the face picture with the highest score to a public security system for identity verification, if the identity verification fails, the user can select re-authentication, and the second step is repeated; if the identity verification passes, the face recognition server segment synchronizes the authentication information to the butt-party server; and sends a "real name authentication success" notification to the face authentication H5 page.

In the prior art, the face authentication process can only be completed in an application program APP, and the embodiment of the invention can instantly transmit media stream information (including video streams and audio streams) acquired from a WebRTC client (namely an H5 page) to a WebRTC server for processing by establishing instant communication connection between the WebRTC client and the WebRTC server, thereby realizing the completion of the face authentication in the H5 page.

Note that the above-mentioned face authentication is only one embodiment of the present invention and the technical principle applied. The present invention is not limited to the specific embodiments described herein.

EXAMPLE six

Fig. 9 is a schematic structural diagram of an instant messaging system according to a sixth embodiment of the present invention. The embodiment is applicable to the case of establishing the instant messaging connection between the WebRTC client and the WebRTC server. As shown in fig. 9, the instant communication system includes: WebRTC client 610 and WebRTC server 620;

the WebRTC server 620 is configured to generate an offer containing a server-side session description protocol and add the offer to a server-side local session description of the WebRTC server after receiving the creation request signaling sent by the WebRTC client; sending the offer to the WebRTC client;

the WebRTC client 610 is configured to obtain the offer, and add the offer to a client remote session description of the WebRTC client; generating an offer response containing a client session description protocol and adding the offer response to the client local session description of the WebRTC client; sending the offer response to the WebRTC server;

the WebRTC server 620 is further configured to receive the offer response, and add the offer response to a server-side remote session description of the WebRTC server, so as to complete creation of an instant messaging connection between the WebRTC client and the WebRTC server.

Optionally, the WebRTC server 620 is further configured to:

initializing before receiving a creation request signaling sent by the WebRTC client, and sending the creation request instruction to the WebRTC server.

Optionally, the WebRTC client 610 is specifically configured to:

Optionally, the WebRTC server 620 is used as a signaling server to receive a creation request signaling sent by the WebRTC client.

Optionally, the WebRTC server 620 is further configured to:

after receiving the creation request signaling sent by the WebRTC client, initializing before generating an offer containing a server-side session description protocol.

Optionally, the WebRTC server 620 is specifically configured to:

adding a second information transmission channel, the second information transmission channel comprising: a second media stream channel and/or a second data channel.

Optionally, the WebRTC server 620 is further configured to:

acquiring a server-side session description protocol; the server-side session description protocol comprises the following steps: a server-side media codec, a server-side protocol address and a server-side network transmission protocol; generating the offer based on the server-side session description protocol.

Optionally, the WebRTC client 610 is further configured to:

acquiring a client session description protocol; the client session description protocol comprises: a client media codec, a client protocol address and a client network transport protocol; generating an offer answer based on the client session description protocol.

The product can execute the method for establishing the instant messaging connection provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

EXAMPLE seven

Fig. 10 is a schematic structural diagram of an instant messaging creating apparatus according to a seventh embodiment of the present invention. The embodiment is applicable to the case of establishing the instant messaging connection between the WebRTC client and the WebRTC server. The device can be implemented in software and/or hardware, and the device can be integrated in a WebRTC client, as shown in fig. 10, where the device for creating instant messaging specifically includes: an offer acquisition module 710, an offer reply generation module 720, an offer reply addition module 730, and an offer reply sending module 740.

The offer acquisition module 710 is configured to acquire an offer sent by the WebRTC server, and add the offer to the client remote session description of the WebRTC client;

an offer reply generation module 720 for generating an offer reply containing the client session description protocol;

an offer answer adding module 730, configured to add the offer answer to a client local session description of the WebRTC client;

an offer response sending module 740, configured to send the offer response to the WebRTC server, so that the WebRTC server completes creation of an instant messaging connection between the WebRTC client and the WebRTC server.

Example eight

Fig. 11 is a schematic structural diagram of an instant messaging creating apparatus according to an eighth embodiment of the present invention. The embodiment is applicable to the case of establishing the instant messaging connection between the WebRTC client and the WebRTC server. The device can be implemented in software and/or hardware, and the device can be integrated in a WebRTC server, as shown in fig. 11, where the device for creating instant messaging specifically includes: an offer generation module 810, an offer addition module 820, an offer transmission module 830, and a creation module 840.

An offer generating module 810, configured to generate an offer containing a server-side session description protocol after receiving a create request signaling sent by the WebRTC client;

an offer adding module 820, configured to add the offer to a server-side local session description of the WebRTC server side;

an offer sending module 830 for sending the offer to the WebRTC client;

a creating module 840, configured to receive an offer response sent by the WebRTC client, and add the offer response to a server-side remote session description of the WebRTC server, so as to complete creation of an instant messaging connection between the WebRTC client and the WebRTC server.

Example nine

Fig. 12 is a schematic structural diagram of an instant messaging device according to a ninth embodiment of the present invention. The embodiment is applicable to the case of establishing the instant messaging connection between the WebRTC client and the WebRTC server. The device can be implemented in software and/or hardware, and the device can be integrated in a WebRTC client, as shown in fig. 12, where the device for creating instant messaging specifically includes: an acquisition module 910, and a transmission module 920.

The acquiring module 910 is configured to acquire media stream information input by a media device, where the media device is connected to the WebRTC client through a first media stream track;

the transmission module 920 is configured to transmit the media stream information to the WebRTC server through an instant messaging connection between the WebRTC client and the WebRTC server, where the instant messaging connection is created by using the method for creating an instant messaging connection provided in embodiments one to four of the present invention.

The product can execute the instant messaging connection method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

Example ten

Fig. 13 is a block diagram illustrating a terminal device according to a tenth embodiment of the present invention, where as shown in fig. 13, the terminal device includes a processor 1010, a memory 1020, an input device 1030, and an output device 1040; the number of the processors 1010 in the terminal device may be one or more, and one processor 1010 is taken as an example in fig. 13; the processor 1010, the memory 1020, the input device 1030, and the output device 1040 in the terminal apparatus may be connected by a bus or other means, and the connection by the bus is exemplified in fig. 12.

The memory 1020, which is a computer-readable storage medium, may be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the method for creating an instant messaging connection in the embodiments of the present invention (for example, the offer acquisition module 710, the offer response generation module 720, the offer response addition module 730, and the offer response transmission module 740 in the creating apparatus of an instant messaging connection, or the offer generation module 810, the offer addition module 820, the offer transmission module 830, and the creating module 840). Or program instructions/modules corresponding to the instant messaging connection method as in the embodiments of the present invention (e.g., the obtaining module 910 and the transmitting module 920 in the apparatus for creating instant messaging).

The processor 1010 executes various functional applications and data processing of the terminal device by running software programs, instructions and modules stored in the memory 1020, that is, implements the above-described method for creating an instant messaging connection.

The memory 1020 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 1020 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 1020 may further include memory located remotely from the processor 1010, which may be connected to the terminal device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input device 1030 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of a terminal device, such as a media device. Output device 1040 may include a display device such as a display screen.

EXAMPLE eleven

An eleventh embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a method for creating an instant messaging connection, according to any embodiment of the present invention: obtaining an offer sent by the WebRTC server side, and adding the offer to a client side remote session description of the WebRTC client side;

generating an offer reply comprising a client session description protocol;

Or implementing the method for creating the instant messaging connection provided by all the invention embodiments of the application: after receiving a creation request signaling sent by a WebRTC client, generating an offer containing a server-side session description protocol;

sending the offer to the WebRTC client;

Or the instant messaging method provided by all the invention embodiments of the application is realized: acquiring media stream information input by media equipment, wherein the media equipment is connected with the WebRTC client through a first media stream track;

and transmitting the media stream information to the WebRTC server through an instant messaging connection between the WebRTC client and the WebRTC server, wherein the instant messaging connection is created by using the method for creating the instant messaging connection provided in the first to fourth embodiments of the present invention.

Any combination of one or more computer-readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for creating instant communication connection is applied to an instant communication system, and the instant communication system comprises: the method is characterized by comprising the following steps of:

2. The method of claim 1, wherein before receiving the creation request signaling sent by the WebRTC client, the WebRTC server further comprises:

3. The method of claim 2, wherein the WebRTC client initializes, comprising:

4. The method of claim 1, wherein the WebRTC server serves as a signaling server for receiving a creation request signaling sent by the WebRTC client.

5. The method of claim 1, wherein the WebRTC server, after receiving the create request signaling sent by the WebRTC client, before generating the offer containing the server-side session description protocol, further comprises:

and the WebRTC server side carries out initialization.

6. The method of claim 5, wherein the WebRTC server side performs initialization, and comprises:

7. The method of claim 1, wherein the WebRTC server-side generating an offer comprises:

8. The method of claim 1, wherein the WebRTC client generates an offer answer that includes a client session description protocol, comprising:

9. A method for creating instant messaging connection, which is applied to a WebRTC client, and comprises the following steps:

generating an offer reply comprising a client session description protocol;

10. A method for creating instant messaging connection is applied to a WebRTC server side, and comprises the following steps:

sending the offer to the WebRTC client;

11. An instant messaging method is applied to a webpage instant messaging WebRTC client, and comprises the following steps:

and transmitting the media stream information to the WebRTC server through an instant messaging connection between the WebRTC client and the WebRTC server, wherein the instant messaging connection is created by using the method for creating an instant messaging connection according to any one of claims 1 to 10.

12. The method of claim 11, wherein the WebRTC client is provided with a network address translation device, and wherein a relay server corresponding to the network address translation device comprises: traversing STUN servers or forwarding TURN servers;

13. The method of claim 11, wherein obtaining media stream information input by a media device comprises:

sending a permission access request to the media device;

14. A system for instant messaging connectivity, the instant messaging system comprising: the method comprises the steps that a webpage instant messaging WebRTC client and a webpage instant messaging WebRTC server are connected;

15. An instant messaging connection creation apparatus integrated with a web page instant messaging (WebRTC) client, the apparatus comprising:

16. An instant messaging connection creating device integrated on a WebRTC server side of a web page instant messaging, the device comprising:

an offer sending module to send the offer to the WebRTC client;

17. An instant messaging apparatus, integrated with a WebRTC client, the apparatus comprising:

the acquisition module is used for acquiring media stream information input by media equipment, and the media equipment is connected with the WebRTC client through a first media stream track;

a transmission module, configured to transmit the media stream information to the WebRTC server via an instant messaging connection between the WebRTC client and the WebRTC server, where the instant messaging connection is created by using the method for creating an instant messaging connection according to any one of claims 1 to 10.

18. A terminal device comprising input means, output means, a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of creating an instant messaging connection according to claim 9 or implementing the instant messaging method according to claim 10 when executing the program.

19. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method of creating an instant messaging connection according to claim 9 or carries out the instant messaging method according to claim 10.