WO2015051816A1

WO2015051816A1 - Control of a communication session between a local and remote user of a process control system

Info

Publication number: WO2015051816A1
Application number: PCT/EP2013/070777
Authority: WO
Inventors: Elina Vartiainen; Martin Naedele; Mark Billinghurst; Seungwon Kim
Original assignee: Abb Technology Ltd
Priority date: 2013-10-07
Filing date: 2013-10-07
Publication date: 2015-04-16

Abstract

A communication handling arrangement for controlling a communication session for a number of participants, where the participants comprise a local user at a location in an industrial site where a process control system is operated and a remote user at a remote location and the communication session comprises a first media item (MI1) in the form of video captured by the local user, comprises a communication handling unit which provides the video stream to participants of the communication session for communication of the first media item (MI1), fetches graphical representations (PG1, PG2, PG3, PG4) of process control objects present in the communication of the first media item, provides the graphical representations in a second media item (MI2) for communication to the participants, receives via one of the media items, input from one of the participants concerning a first process control object present in both media items, and visually communicating to the other participants registration of the input via the other media item.

Description

CONTROL OF A COMMUNICATION SESSION BETWEEN A LOCAL AND REMOTE USER OF A PROCESS CONTROL SYSTEM

FIELD OF THE INVENTION

The present invention generally relates to process control systems. More particularly the present

invention relates to a method, communication handling arrangement and computer program product for

controlling a communication session for a number of participants .

BACKGROUND In process control systems it is very important to keep production running at all times as even a minor halt in production will result in any of wasted resources, reduced production output, or substandard product quality, wasted energy and also cost large amounts of money. Because of lost production, waste and other problems caused by unplanned stops or slowdowns, maintenance is seen as very important in order to keep the production up and running. One useful tool provided for local maintenance in a process control system is described in SE1300138.

Maintenance and repair of process equipment may thus be a critical part of production in order to ensure that the production is up and running at all times. Even a minor halt in production may waste great amounts of resources and cost large amounts of money. In many cases, external experts are needed to perform complex and advanced repair and testing work on the process equipment. In those cases, maintenance can become very expensive if these experts are brought in to the site, which might be located on the other side of the world. The regular personnel might not have the expertise to handle by themselves. Collaboration over a telephone line between the local worker and the expert is not effective enough in many cases. The expert may need to see what happens on site and may need to be able to instruct the personnel on site without the risk of any misinterpretation.

However, it can take a long time to get hold of the correct expert and bring in the expert to the site. In the case of an unexpected breakdown this can lead to long downtimes. To fly in an external expert can be very expensive and the correct expert might not always be available. Collaboration over the phone is not effective enough in all situations as the expert is not able to see what the local worker is doing. Sending pictures back and forth by email or MMS is also a slow way of sharing information. There may also be safety aspects involved. The expert may not be able or not allowed to go to the site.

Some of these problems may be addressed through

providing a special communication session between a local worker and a remote expert. EP 1657610 does for instance describe an asset management system where video and snapshots by a remote and local user are provided in a communication session. The document also discusses the presentation of circuit diagrams. However, there is still room for improvement within the field .

SUMMARY

The present invention is concerned with the problem of simplifying for a remote user to give instructions to a local user of a process control system. This object is according to a first aspect achieved through a method for controlling a communication session for a number of participants, where the

participants comprise a local user at a location in an industrial site where a process control system is operated and a remote user at a remote location, the communication session comprising a first media item in the form of video captured by the local user, the method being performed by a communication handling arrangement and comprising:

- providing the video stream to participants of the communication session for communication of the first media item,

fetching graphical representations of at least one process control object, where the graphical

representations are representations of process control objects present in the communication of the first media item,

providing the graphical representations in a second media item for communication to the participants of the session,

receiving, via one of the media items, input from one of the participants concerning a first process control object, which first process control object is present in both media items, and

visually communicating to the other

participants registration of the input via the other media item.

This object is according to a second aspect achieved through a communication handling arrangement for controlling a communication session for a number of participants, where the participants comprise a local user at a location in an industrial site where a process control system is operated and a remote user at a remote location, the communication session comprising a first media item in the form of video captured by the local user and the communication handling arrangement comprising :

a communication handling unit configured to:

provide the video stream to participants of the communication session for communication of the first media item,

fetch graphical representations of at least one process control object, where the graphical

provide the graphical representations in a second media item for communication to the participants of the session,

receive via one of the media items, input from one of the participants concerning a first process control object, which first process control object is present in both media items, and visually communicating to the other

participants registration of the input via the other media item. This object is according to a third aspect achieved through a computer program product controlling a communication session for a number of participants, where the participants comprise a local user at a location in an industrial site where a process control system is operated and a remote user at a remote location, the communication session comprising a first media item in the form of video captured by the local user, said computer program product being provided on a data carrier comprising computer program code the computer program product comprising a data carrier with computer program code which when run in a communication handling unit of the communication handling arrangement is configured to cause the communication handling unit to

- provide the video stream to participants of the communication session for communication of the first media item,

receive via one of the media items, input from one of the participants concerning a first process control object, which first process control object is present in both media items, and

visually communicating to the other

participants registration of the input via the other media item.

The invention has a number of advantages. The invention improves the ability to provide event specific and/or request dependent instructions from an expert in a fast an efficient way from virtually any place in the world without requiring travel. Thereby the downtime of a process control system can be considerably reduced. This results in cost savings. Also, safety is improved as issues can be fixed faster and more accurately.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will in the following be

described with reference being made to the accompanying drawings, where

Fig. 1 schematically shows an industrial plant with a process control system operating an industrial process together with a data presenting device,

Fig. 2 schematically shows a block schematic of units inside a housing of the data presenting device,

Fig. 3 shows a perspective view of the data presenting device in the form of the housing on a tripod,

Fig. 4 schematically shows the data presenting device communicating with a computer of a remote user in a communication session controlled by a communication handling device of the process control system, Fig. 5 schematically shows the remote user with his computer on which video captured by the data presenting device is shown,

Fig. 6 schematically shows a flow chart of method steps for mapping of images captured by the data presenting device to a model of the environment in which they are captured,

Fig. 7 schematically shows a flow chart of method steps for associating real world data of a real world process control object to a graphical representation of the same,

Fig. 8 schematically shows a session presentation area in which data of a communication session is displayed for a session participant,

Fig. 9 schematically shows a flow chart of a method for controlling a communication session, and

Fig. 10 schematically shows a data carrier with

computer program code, in the form of a CD-ROM disc, for implementing a communication handling unit of the communication handling device.

DETAILED DESCRIPTION OF THE INVENTION

This invention presents a way for a remote user to gather relevant data and provide instructions and directions for local engineers at a location of an industrial plant where a process control system

operates . Fig. 1 schematically shows an industrial plant where a process control system 10 is provided. The process control system 10 is a computerized process control system for controlling an industrial process. The process can be any type of industrial process, such as electrical power generation, transmission and

distribution processes as well as water purification and distribution processes, oil and gas production and distribution processes, petrochemical, chemical, pharmaceutical and food processes, and pulp and paper production processes. These are just some examples of processes where the system can be applied. There exist countless other industrial processes. The processes may also be other types of industrial processes such as the manufacturing of goods. A process may be monitored through one or more process monitoring computers, which communicate with a server handling monitoring and control of the process.

In fig. 1 the process control system 10 therefore includes a number of process monitoring computers 12 and 14. These computers may here also be considered to form operator terminals and are connected to a first data bus Bl . There is also a communication handling device 16 connected to this first data bus Bl, which communication handling device 16 is connected to at least one wireless network WN. The communication handling device 16 is also connected to a public data communication network, which here is the internet IN. The communication handling device 16 therefore has a wireless interface 17 for communicating with the wireless network WN, a computer network interface 18, for instance in the form of an Ethernet interface, for communicating with the Internet and a local interface 19, perhaps also in the form of an Ethernet interface, for communicating with other entities in the process control system 10. The communication handling device 16 also has a communication handling unit 20 connected to all three interfaces 17, 18 and 19. The wireless interface 17 is thereby a first communication interface for communicating with the data presenting device 32, the computer network interface 18 a second

communication interface for communicating with a device of a remote user and the local interface a third communication interface for communicating with devices of the process control system 10. The communication handling unit 20 has the function of providing a gateway between the process control system and other networks, such as the Internet or the wireless network WN, if this is external to the process control system 10. However, in some variations of the invention it also has another function and that is the function of controlling a communication session. This will be described in more detail later.

To the wireless network WN there is connected a data presenting device 32. The wireless network WN may be a local network, such as a wireless local area network (WLAN) . It may also be a Bluetooth network, i.e. a network with a number of interconnected Bluetooth nodes. It may also be a mobile communication network.

There is furthermore a second data bus B2 and between the first and second data busses Bl and B2 there is connected a server 23 providing control and protection of the process and a database 22 where data relating to control and protection of the process is stored. Such data relating to control and protection may here comprise process data such as measurements and control commands, while data relating to protection may comprise alarm and event data as well as data on which alarms and events can be generated, such as

measurements made in the process. There is furthermore an object data server 21 connected between the two buses Bl and B2. The object data server 21 comprises data about all process control objects, such as blueprints, instructions and manuals regarding the process control objects. It may also comprise face plates of process control objects, which face places may be set to access and communicate process control data from the process control object.

To the second data bus B2 there is furthermore

connected a number of further devices 24, 26, 28 and 30. These further devices 24, 26, 28 and 30 are field devices, which are devices that are interfaces to the process being controlled. A field device is typically a device comprising an interface via which measurements of the process are being made and to which control commands are given. Because of this the field devices are furthermore process control objects. In one

variation of the invention a first field device is a first process control object 24, a second field device is a second process control object 26 and a third field device is a third process control object 28.

The communication handling unit 20 is a part of a communication handling arrangement. This communication handling arrangement does in the variation in fig. 1 comprise the communication handling device 16. In one variation the only device that is a part of the

communication handling arrangement is the communication handling device 16. In some other variations also the data presenting device 32 is a part of the

communication handling arrangement. In yet other variations, the control and protection server 23, the database 22 and/or object data server 21 may also be included in the communication handling arrangement. The communication handling unit may as an alternative be provided in the data presenting device. In one

variation of this concept, the communication handling arrangement only comprises the data presenting device 32. In other variations of this latter concept, the communication handling arrangement may also comprise the control and protection server 23, the database 22 and/or object data server 21. Fig. 2 shows a block schematic of a number of units that are provided in an exemplifying data presenting device 32. The exemplifying data presenting device 32 is provided with a housing 49. In the housing 49 there is provided a bus 33, and to this bus 33 there is connected an optional short range communication unit 46 or proximity sensor, a display 48, a camera 34, a recording controller 36, a program memory 39, a

processor 40 as well as a radio communication circuit 42. The radio communication circuit 42 is furthermore connected to an antenna 44, where the radio

communication unit 42 and antenna 44 are provided for communication with the wireless network WN. The radio communication circuit 42 and antenna 44 together form one type of communication interface for communicating with the process control system as well as with other entities. It may for this reason be a WiFi or WLAN interface. It may also be a mobile communication interface. It should also be realized that there may be two communication interfaces in the data presenting device 32, one mobile communication interfaces and one WiFi interface. The recording controller 36 is in turn connected to a microphone 35. The recording controller 36 and microphone 35 together form a recording unit that may be used for recording sound in a location of the process control system. Although it is not shown, the data presenting device 32 may also comprise sound emitting units such as speakers and earphones. It is also possible that a microphone and earphones are combined into a headset connected to the data

presenting device 32. The short range communication unit 46 may also be regarded as a type of sensor, an object sensor or proximity sensor, for sensing a process control object to be serviced. This sensor may be implemented through Near Field Communication (NFC) technique .

In the program memory 39 there is provided software code which when being run by the processor 40 forms a control unit 38.

A local user equipped with the data presenting device 32 may move within the premises of the industrial plant. This means that the data presenting device 32 may be moved from one location to another location. The data presenting device may also be considered to be a portable video recording device. It may also be placed so that it will be able to capture video images. For this reason the housing 49 may be placed on an optional tripod 50, which is schematically shown in fig. 3. The camera 34 has a field of view, i.e. an area in which it detects its environment. This field of view may be changed in different ways. It may be increased through zooming out commands and it may be decreased through zoom in commands. The field of view may also be shifted or moved using various type of pan commands. In order to obtain panning, the orientation of the camera may be changed. It is of course also possible to omit the tripod and the let the local user carry the data presenting device in his or her hands. The data

presenting device may also be provided on wheels so that it can more easily be moved. The data presenting device may also be a tablet or a smart phone carried by the local user.

As can be seen in fig. 1 the data presenting device 32 may connect to the communication handling device 20 via the wireless network WN. The communication handling device 20 may also be accessed from outside of the process control system via the Internet IN. This allows the user of the data presenting device 32 to engage in a communication session with at least one other user, which other use may be a remote user. It is then possible that video captured by the data presenting device 32 is being communicated to other participants of the communication session and especially to the remote user. The communication session will then be set up between the data presenting device 32 and a computer of the remote user via the communication handling device 16, thereby making the local user of the data presenting device 32 and the remote user into

participants of the communication session.

This situation is schematically shown in fig. 4. Here it can be seen that a computer 51 of the remote user may communicate with the data presenting device 32 via the communication handling device 16. The communication of the session thus passes through the communication handling device 16, which allows the communication handling device 16 to control the communication

session. As can be seen in the figure the session comprises a number of channels between the session participants and the communication handling device 16. There is a first primary communication channel CHP1 as well as a first secondary communication channel CHS1 between the data presenting device 32 and the

communication handling device 16. There is also a second primary communication channel CHP2 as well as a second secondary communication channel CHS2 between the communication handling device 16 and the computer 51 of the remote user. The primary communication channels CHP1 and CHP2 may be unidirectional video channels in which a first media item in the form of a video stream VS is transferred from the data presenting device 32 of the local user to other session participants and in particular to the remote user computer 51. The

secondary channels CHS1 and CHS2, which are

bidirectional, are used for transferring of other media items as well as data used for controlling the

communication session. The secondary communication channels CHS1 and CHS2 may for instance be used for conveying processor graphics PG, a marker M and user inputs UI . The data in the primary channels CHP1 and CHP2 may be transparently passed through the

communication handling device 16, i.e. passed without the communication handling unit 20 acting on the data, while the communication handling unit 20 may actively act on at least some of the data of the secondary communication channels CHS1 and CHS2.

Thereby the remote 52 user may be able to obtain video images captured by the camera 34 of the data presenting device 32, which video images are then communicated to the remote user 52 and other session participants via the display of his or her computer 51, which situation is shown in fig. 5. The video may also be communicated to the local user via the data presenting device 32.

Now some variations of the invention will be described in more detail with reference being made to fig. 6, which schematically shows a flow chart of method steps for mapping of images captured by the data presenting device to a model of the environment in which they are captured, and to fig. 7, which schematically shows a flow chart of method steps for associating real world data of a real world process control object to a graphical representation of the same.

In industry, for instance in an industrial plant where a process is being run by a process control system, it is very important to keep production running at all times as even a minor halt in production may result in loss of quality, or reduced production, or wasted resources and most likely will also cost large amounts of money. Because of this, maintenance is seen as very important in order to keep the production up and running.

Maintenance and repair of process equipment is thus a critical part of production in order to ensure that the production is up and running at all times. Even a minor halt in production may result in a huge waste of resources or cost large amounts of money. In many cases, external experts are needed to perform complex and advanced repair and testing work on the process equipment. In those cases, maintenance can become very expensive if these experts are brought in to the site, which might be located in the other side of the world. The regular personnel might not have the expertise to handle by themselves. Collaboration over a telephone line between the local user or local worker and the expert is not effective enough in many cases. The expert needs to see what happens on site and needs to be able to instruct the personnel on site without the risk of any misinterpretation.

However, it can take a long time to get hold of the correct expert and bring in the expert to the site. In the case of an unexpected breakdown this can lead to long downtimes. To fly in an external expert can be very expensive and the correct expert might not always be available. Collaboration over the phone is not effective enough in all situations as the expert is not able to see what the local user or local worker is doing. Sending pictures back and forth by email or MMS is also a slow way of sharing information. There may also be Safety aspects involved. The expert may not be able or not allowed to go to the site.

Because of this it is of interest to provide a

communication session where video is transmitted from the site to the remote user. However, when this is not enough, it may be of interest for the remote user to also provide additional data of the scene shown in such a captured video stream, such as provide information about process control objects being shown in the video as well as provide further guidance to the local user in order to allow the local user to perform maintenance for example without making time-consuming mistakes, or wrongly identifying a process object as causing an abnormal process, and generally as effectively as possible.

In operation, i.e. when there is some kind of problem at a location in the plant, the data presenting device 32 is brought out to this location of the industrial site and placed at a position in the location where assistance is needed. The device may for instance be placed in the centre of a room. The data presenting device may be placed, such as on the tripod 50, at this location by the local user in order to be used for solving a problem at the location, for instance the fact that one or more of the machines or process control objects may be faulty or that the process has a strange behaviour at the location. As an alternative to using the tripod, the data presenting device may be held by the local user at the location.

When the data presenting device 32 is brought to the location a number of activities may thus be performed. In order to simplify a later giving of instructions to the local user, it is advantageous if there is a linking of the process control objects at the site as being captured by a video stream to data of and from the same process control object in the process control system.

In order to allow this to be done, the control unit 38 of the data presenting device 32 may first make the data presenting device 32 scan the area at the

location, step 54, fig. 6. In this way a three- dimensional space around the position of the data presenting device may be captured with different video images using the camera 34.

After the area has been scanned, the images may thus be forwarded to the communication handling unit 20 of the communication handling device 16, which communication handling unit 20 analyses the captured images and investigates if it recognises them with regard to a pre-existing three-dimensional model of the location and objects at this location, i.e. of the process control objects and possible other objects present at the location. If it recognizes the video images and that therefore there is a pre-existing model, step 56, then this model is fetched, step 58. The pre-existing three-dimensional model may be provided in the

communication handling unit 20 or a memory associated with the communication handling unit 20. As an

alternative the model may be obtained or fetched from a server, such as the object data server 21. If any three-dimensional model has been made of the location, then this is thus fetched. However, if there was no pre-existing model, step 56, a new three-dimensional model 3DM of the location and the various objects in it is created, step 60. A model may for instance be created using augmented reality functionality. If the data presenting device comprises an infrared sensor it is also possible to use infrared technology, such as Microsoft Kinect. A 3D map of natural features at the location can be built using a variety of feature extraction methods such as corner or edge detection both with 2D RGB data and 3D RGBD (Red, Green, Blue, Depth) data. The model may be created in the control unit 38 of the data presenting device 32. As an

alternative the captured images may be forwarded from the data presenting device 32 to the communication handling unit 20 of the communication handling device 16, which unit 20 then creates the model.

The process control objects at the site, i.e. the real world objects, which objects may be objects such as valves, motors or other actuators, sensors or measuring instruments, controllers or other control units comprising a processor, may be provided with object identifiers, such as NFC tags or bar codes. If the object identifiers are read it is possible to obtain information about what types of objects they are, for example actuators or sensors or control units. The type may be identified through the camera 34 of the data presenting device 32 detecting a visual object

identifier, like a bar code. As an alternative the short-range communication unit may be set to read a tag with the object identifier. Once such a code is detected it may then be sent from the data presenting device 32 to the communication handling unit 20 of the communication handling device 16. These codes may then be used to identify the process control objects, step 62. Such a code may thereby be used to associate the real process control object with a corresponding object in the 3D model and thereby it is possible to known what process control objects are present in a video stream that is captured by the data presenting device. When a process control object has been identified in this way, it is then possible to subscribe to real time data of the process control object. A subscription to real time data for real world process control object may thus be initiated by the communication control unit 20, step 64, which subscription may be activated at a later stage, for instance when a session is on-going. In this way a subscription to e.g. real time data for the specific process object may be established with the control system. The real time data is further

associated with a graphical representation of the process control object, step 65. This association is furthermore made in order to enable joint display of the real time data and the graphical representation of a real world object The real time data may later be made available to the communication session and may be communicated to more than one user.

The above mentioned steps may have been performed before the local user starts a communication session with the remote user 52 and optionally also with other users .

Such a session may be carried out using a TCP

connection set up between the communication handling device 16 and the data presenting device 32, where the communication handling device 16 in turn connects to the remote user via a TCP connection setup through the Internet. It is also possible that the communication handling device 16 sets up such connections with other users in order for these other users to also become participants in the communication session. The

communication session may furthermore be set up using encryption and/or a virtual private network (VPN) .

How a communication session may be set up and carried out under the control of the communication handling device will now be described with reference also being made to fig. 8 and 9, where fig. 8 shows a session presentation area in which data of a communication session is displayed for a session participant, and fig. 9 shows a flow chart of a method for controlling a communication session.

A communication session is set up between a number of participants, where the local user at a location in the industrial site where the process control system 10 is operated is one and the remote user 52 at the remote location is another. In the communication session, the communication handling device 16 provides two channels between itself and each user, where a primary channel is a video channel for the video captured by the data presenting device 32 and conveyed as a video stream VS and a secondary channel is provided for process

graphics PG, a marker M, as well as for user inputs UI, where user inputs may be requests to control the marker, positions of the marker in relation to such control and annotations to media items. The primary channel may be a unidirectional channel in which the video is transmitted from the local user towards the other participants of the session, while the secondary channels may be bidirectional channels allowing the communication handling unit to transmit data to and from every participant.

In the session there may be at least one voice

communication session between the remote user 52 and the local user via sound generating equipment and sound recording equipment of the data presenting device 32 and the computer 51. This sound may be transferred via the secondary channels CHS1 and CHS2. The session also involves transmission of a live video stream VS, which may thus be a one way video stream from the data presenting device 32 in the process control system to the other participants of the session and in particular to the computer 51 of the remote user 52. In some instances it may involve a two-way video conference, i.e. a situation where video is also provided by the computer 51 of the remote user 52 and conveyed to the data presenting device 32. The video stream is conveyed via the primary channels CHP1 and CHP2. Video images captured by the camera 34 may in this way be

transferred to the remote user 52.

A session may be started by a user of the data

presenting device 32. For this reason the control unit may comprise a communication session client, which connects to the communication handling unit 20 via the wireless communication network WN using the radio communication circuit 42 and antenna 44 when the local user wants to engage in a communication session. In order to start a session, the control unit 38 may send a request for a session and when receiving such a request, the communication handling unit 20 of the communication handling device 16 may then set up the primary communication channels CHPl and CHP2 to the various participants as well as the secondary

communication channels CHS1 and CHS2 via the wireless and computer network interfaces 17, 18 and possibly also via the local interface 19. In the example of fig. 4, the communication handling unit 20 thus sets up the first primary channel CHPl to the data presenting device 32 and the second primary channel CHP2 to the computer 51 of the remote user 52. The communication handling unit 20 also sets up the first secondary channel CHS1 to the data presenting device 32 and the second secondary channel CHS2 to the computer 51 of the remote user 52. As an alternative it is possible that the first primary and secondary channels are set up by the data presenting device 32 to the communication handling device 16, which then sets up the rest of the primary and secondary channels.

It should also be realized that a primary and a

secondary channel is set up to all the participants of the session, and especially to the remote user. The remote user may also desire other participants to engage in the communication session, why communication channels may be set up to these as well. In the

communication session, the primary communication channels CHPl and CHP2 are provided for a first media item Mil, which is provided through the video stream VS, and the secondary communication channels for other data, such as for a second MI2 and further media items.

In the session the video captured by the local user using the data presenting device 32 is provided by the communication handling unit 20 to all participants as a video stream VS, step 66. The video may then be

communicated or presented in a communication session presentation area SPA provided on all user interfaces. In relation to this, the participants are also

presented with the marker M in the session presenting area SPA, which marker M is also provided by the communication handling unit 20, but may be controlled by the participants. The video stream VS is thus conveyed via the first primary channel CHP1 to the communication handling device 16, which forwards it in the second primary channel CHP2 to the computer 51 of the remote user 52. The video stream VS is thus sent from the data presenting device 32 of the local user to the computer 51 of the remote user 51 as well as to any other party interested in viewing the video. This video is then communicated as the first media item Mil, fig. 8. As all participants have the same session

presentation area SPA, the first media item Mil is also communicated to the local user locally at the data presenting device 32. It should here be realized that it is optionally possible with also a video stream being transmitted from the remote user to the local user. If the camera view is shown on the display of the data presenting device, it is here possible that at least some of the various session presentation area items are provided on top of this camera view. As the camera view depicts the scene of the captured video, it is clear that the first media item may be omitted. The marker M may also be provided to all participants via the secondary communication channels CHSl and CHS2. The marker M is thus provided from the communication handling unit 20 of the communication handling device 16 in the first secondary communication channel CHSl to the data presenting device 32 and in the second

secondary channel CHS2 to the computer 51 of the remote user. The marker M may be sent together with

instructions on where it is to be presented in the session presentation area SPA. The marker M and first media item Mil are then communicated to all the

participants of the communication session in the session presenting area. Each participant is thus provided with a device having a screen where the presentation area SPA is shown and in this presentation area the video and the marker M are presented. The marker, which is selectable by all participants of the communication session, may be used for pointing at process control objects in media items being presented.

At the same time the communication handling unit 20 investigates if there are any user inputs via the secondary communication channels, which user inputs may be made by any of the session participants.

It may more particularly investigate if there are any user inputs that involve a selection of an object in the presented video, i.e. in the first media item Mil. A user input may for instance be a user inputs via the marker M, which is common to all participants. If no one selects an object in the video stream, the

communication handling unit 20 returns and waits for an object to be selected.

However, if an object is selected in the video stream, the communication handling unit 20 then investigates which object it was. Any of the participants of the session, and especially the remote user, may thus want to obtain some more data about the process control objects that are visible in the video stream VS. As the objects may have been identified using a virtual model, an object may easily be identified through a participant pointing at the representation of the object in the presented video, for instance using the marker M. A participant may for instance desire to obtain data of the temperature in a tank or the voltage of a transformer. In order to do this he may then select an object in the presented video. The selection of the object may then be detected through the communication handling unit 20 detecting the position of the marker in relation to the

presentation of objects in the video. If this position is the position of an object in the 3D model for which an association to a corresponding process control object exists, data of this process control object is then fetched, for instance displayed in one or more face plates from the server 21. The data being fetched may also comprise the above-mentioned real time data for which a subscription exists.

A face plate may be able to display a number of different types of real time data, why it may also be possible for the session participants to choose between different types of real time data to be communicated via a face plate of a process control object It is also possible that if a selection is detected, then all process graphics of all the process control objects covered in the video are fetched. Process graphics of all the process control objects shown in the video may thus be fetched automatically through a selection of the first media item Mil as such. When a process object has been identified then the real time data of the process control object may be thus be fetched from the process control system according to the subscription. The real time data is then made available to the communication session and may be communicated to more than one user. It is also possible that no user inputs are needed. Process graphics may be fetched automatically by the communication handling unit 20 based on the objects being covered by the camera of the data presenting device 32. It can in this way be seen that the communication handling unit 20 fetches graphical representations of at least one process control object, which graphical representations are representations of process control objects present in the presentation of the first media item, step 68.

The fetching may in some instances furthermore be made based on the communication handling unit receiving a selection of the first media item from one of the participants. The selection may furthermore be the selection of at least one of the process control objects shown in the presentation of the first media item. It can also be seen that the above mentioned user inputs may comprise participant selections made in relation to media items and/or process control objects.

The process graphics PG of one or more process control objects are then transferred in a second media item MI2 in the secondary communication channels CHS1 and CHS2 to all session participants. Also the real time data being associated with the process graphics may be transferred. The real time data may be transferred as a part of the second media item MI2 or separately. The graphical representations are thereby provided in the second media item MI2, step 69, for being communicated to the participants of the session. As can be seen in the case of the above-mentioned selection being a selection of at least one of the process control objects shown in the presentation of the first media item, the second media item may comprise a graphical representation of the selected process control object. The second media item MI2 is furthermore provided to all the participants in the communication session. In this way all the participants of the communication session are provided with the first media item Mil, i.e. the video of the video stream VS, as well as the second media item MI2 with process graphics PG. As can be seen in the example of fig. 8, there are four different graphical representations, a first piece of process graphics PG1 representing a first process control object, a second piece of process graphics PG2 representing a second process control object, a third piece of process graphics PG3 representing a third process control object and a fourth piece of process graphics PG4 representing a fourth process control object, at least some of which are also being depicted in the presentation of the first media item Mil.

As mentioned earlier, the process graphics may here be realized in the form of face plates, which in addition to graphically representing the process control object also comprise, access or subscribe to the real-time data of the process control objects. This means that the communication handling unit 20 may also fetch measurement data from the process control system being actually measured by the process control object or being determined based on data measured by the process control object and communicate this real time data together with the process graphics of the process control object. This means that in addition to seeing the captured video all the communication session participants are provided with the process graphics possibly including data measured by or determined based on measurements made by the process control object being represented by a process graphic item.

As mentioned earlier, the marker M may also be provided to all of the session participants. This marker M is also selectable by all participants. However, since there may be more than one participant, simultaneous control may have to be avoided.

A participant wanting to use the marker M may send a request to control the marker M, which request is thus sent as user inputs UI in a secondary communication channel. As an alternative the participant may select a marker control command, for instance in a menu of the presentation area.

The communication handling unit 20 therefore

investigates the user inputs UI with regard to if there are any requests for controlling of the marker M. If there is not, step 70, it returns and waits. However, if there is, step 70, the communication handling unit

20 then investigates if the control is to be granted or not . In case there is no competition for the marker, then control may be directly handed over to the participant that requests control. However, if there is contention, the communication handling unit 20 may determine which participant is to be a controlling participant

according to a priority scheme, step 72. It thus determines which participant out of the ones that simultaneously want to be controlling participant that is to be made into a controlling participant, i.e. a person that is in control of the marker M. In doing this it employs a priority scheme.

According to some variations of the invention, the remote user is given the highest priority in this scheme with the local user having the second highest priority. In case there are more participants, these may be given priority according to the order in which they join the session and with a priority that is below the priority of the local and remote users of the process control system. The participant having the highest priority is then given control of the marker M, while the other participants are denied control. A priority may also be given a user based on the

technical or security responsibility of the user in the process control system. If a user is logged in to the session or process control system with a user identity, then the priority may be linked to this user identity.

The communication handling unit 20 may thus receive at least one request for controlling the marker M from the participants of the session, and if more than one request is received from more than one participant, it may determine which participant that is to receive control according to the priority scheme and provide control to the selected participant, where the priority scheme indicates that the remote user has priority over other participants.

Once a participant is given control, the communication handling unit 20 investigates the use of the marker. It more particularly investigates if the controlling participant marks an object or not in one of the media items. If no object is marked, step 74, then the communication handling unit 20 returns and investigates if there is a request for control. However, if an object is marked, step 74, fig. 9 the object is then emphasized in the media item in question, step 76. This may be done by adding a visual emphasis such as a graphic border or ring to an image a process object. Through this marking, the communication handling unit receives, and registers via the marked media item, input from one of the participants concerning a process control object, step 74, where this process control object is present in both media items Ml and M2.

The media item in which the marking of the object by the controlling participant is determined is thus changed, which change is an indication of the selection of the object by the controlling participant, and which may be communicated to other users by recording and distributing the marked up media item. The form of emphasizing is optional. What is also done is that the object is emphasized in the other media item, step 78. It is also possible to use sound to indicate the selection to the other session participants. In this way the other participants are visually informed of the input via the other media item, i.e. via the non-marked media item. The object is thus emphasized in the non- marked media item. This means that if a participant marks for instance a graphical representation of a process control object in the video stream VS, then also the corresponding face plate is emphasized, which emphasizing may involve highlighting the representation of the object. The emphasising is more particularly made for and thus communicated to all the participants of the session. Thereafter the communication handling unit 20 goes back and investigates if a request for a control is received.

The participants may also make annotations to the media items being presented in the session presentation area SPA, which annotations are also conveyed in the

secondary communication channels. The communication handling unit may thus receive an annotation from a participant made in relation to at least one of the media items and communicate the annotation to the participants of the session. Two such annotations are shown in fig. 8, on in the first media item Mil and one in the second media item MI2. Through the above-mentioned marking made by the

controlling participant in relation to one media item, details of a process control object shown both in two media items can be viewed by all other participants of the session, and especially by the local user.

It can in this way be seen that if a participant, such as the remote user, marks an object in a media item in order to describe some aspect of the corresponding process control object, the same object is emphasized in the other media item, i.e. in the video stream. This allows instructions to be given in a more effective manner. Thereby a local user may be helped in solving a maintenance problem in the process control system in a faster and more effective way than previously.

This invention presents a way to allow an expert to remotely guide personnel on site via a live data stream. The system may use a mobile device/tablet with an integrated camera, for instance in the form of the previously described data presenting device, for the local user connected wirelessly to a PC for the remote user, who may be an expert, via the communication handling device. The expert may be located in his office in front of the computer and the local field worker may be carrying the mobile device. The local worker can then stream a live video broadcast from the site by using the camera of his mobile device. Both the expert and the local worker can make annotations on top of this video stream. They can also share still photos and process graphics values with each other. The system allows both expert and remote worker to make

annotations on still images as well. They can also use the marker, which is thus a shared pointer, to

communicate with each other.

The data presenting device of the local worker or local user uses its camera to record the scene and streams the video to the computer of the remote expert.

Therefore, the expert sitting in the office is able to look at a real-time video from the local site. The session may also comprise dual audio streams so that the expert and the local worker can communicate

verbally with each other.

Both the expert and the local worker or local user can draw and write on top of the video stream and the other party can see these annotations done in real-time. Both parties can also take high resolution snapshots of the video, draw on them and share these snapshots with each other. (Note: when taking still images, these can be substantially higher resolutions than the live video) . Furthermore, the participants can point out objects on the shared picture or video stream. Both sides can also initiate taking still pictures or capture video sequences and recall earlier (annotated) still

pictures. The communication handling unit may be provided with an image cache memory that can be

persistent between sessions. Furthermore, the expert can add pictures or video sequences from external sources into a shared cache (e.g. instructional videos or "look for this part" type of pictures) . The remote expert and local worker or local user can also share a live view of the process graphics with each other through a remote access functionality of the control system. They can annotate the process graphics view in a similar way than the live video stream. The remote expert can also take screenshots from the process graphics and send these screenshots to the local worker. Naturally, they can both make annotations to these screenshots and they are shared with both users. Furthermore, the process graphics can be

combined with the video stream to maximize the

situational awareness. The live video stream can also be used for guidance when the local user is moving within the plant. The remote expert can then explain how the local worker should navigate within the plant. The remote expert does not need to sit next to a desktop computer but he might access the tool from a mobile device as well.

Imagine the following scenario:

1. A paint factory unexpectedly experiences a severe trouble at one of their production lines. The problem is rare and technically complex, and the operators on site need support from an expert in order to restore production.

2. Bringing in an expert will take at least 24 hours as none of the experts is available close to the factory .

3. The operator contacts a support company where an expert in this area is available to instantly help the factory with the problem. 4. Operators discuss with the expert, and the expert instructs the operators on site to bring the data presenting device with has the client software or remote collaboration tool installed so that the remote expert can look at the problem through the live video stream.

5. The remote expert observes the situation by looking at the video stream and asking the local operators to show specific parts of the equipment. 6. Based on the information the remote expert can now point at specific devices in the video stream and instruct the operators on site how to perform certain operations to correct the problem. 7. The expert also looks at the process graphics together with the operators and annotates which values the operators should follow later on.

8. The expert may also select one object in the process graphics and then the same object will be highlighted in both the process graphics as well as in the live video in order communicate a selection and to better explain the actions that the local user is to take.

9. Thanks to all the information shared through the live connection, the expert found the problem in no time and together with the operators on site they managed to get production back to normal again!

Several benefits can be listed:

• Cost savings as remote experts are not needed locally anymore but they can help from the office.

• Time saving as there is no time lost waiting for the expert to travel to the remote site.

• Increased productivity as the potential production stops can be minimized. This is done by the quick maintenance work provided by a remote expert.

• Improved safety as the situational awareness of the remote expert is improved.

Remote training on real equipment. • Rapid emergency response - The system saves time and money.

The communication handling unit may be provided in the form of a processor together with memory including computer program code for performing its functions. This computer program code may also be provided on one or more data carriers which perform the functionality of the communication handling unit when the program code thereon is being loaded into the memory and run by the processor. One such data carrier 80 with computer program code 82, in the form of a CD ROM disc, is schematically shown in fig. 10. The invention can be varied in many more ways than the ones already mentioned. It should therefore be realized that the present invention is only to be limited by the following claims.

Claims

1. A method for controlling a communication session for a number of participants, where the participants comprise a local user at a location in an industrial site where a process control system (10) is operated and a remote user (52) at a remote location, the communication session comprising a first media item (Mil) in the form of video captured by the local user, the method being performed by a communication handling arrangement (20) and comprising:

providing (66) the video stream (VS) to participants of the communication session for communication of the first media item (Mil),

- fetching (68) graphical representations (PG1, PG2, PG3, PG4) of at least one process control object, where the graphical representations are

representations of process control objects present in the communication of the first media item,

- providing (69) the graphical representations in a second media item (MI2) for communication to the participants of the session,

receiving (74), via one of the media items, input from one of the participants concerning a first process control object, which first process control object is present in both media items, and

visually communicating (78) to the other

participants registration of the input via the other media item.

2. The method according to claim 1, further comprising receiving a selection of the first media item from one of the participants and fetching the graphical representations based on the selection.

3. The method according to claim 2, wherein the selection is the selection of at least one of the process control objects shown in a presentation of the first media item and the second media item (MI2) comprises a graphical representation of the selected process control object (PG2).

4. The method according to any previous claim, wherein the visual communicating is performed in a media item through visually emphasizing the first process control object.

5. The method according to any previous claim, further comprising visually communicating (76) to the participants in the communication session registration of the input also through the media item via which input is made.

6. The method according to any previous claim, further comprising at least one marker (M) , for

pointing at objects in the media items, said marker being selectable by any one participant of the

communication session, the method further comprising receiving (70) at least one request for controlling the marker from the participants, and if more than one request is received from more than one participant, determining (72) which participant that is to receive control according to a priority scheme and providing control to the selected participant, where the priority scheme indicates that the remote user has priority over other participants.

7. The method according to claim 6, wherein the first media item is provided in primary communication channels (CHP1, CHP2) of the communication session, while the second media item, said marker and

participant selections (UI), are provided in secondary communication channels (CHS1, CHS2) of the

communication session.

8. The method according to any previous claim, wherein said graphical representations of process control objects comprise retrieval and communication of real-time data from the corresponding process control obj ects .

9. The method according to any previous claim, further comprising communicating to the participants in the communication session registration of the input also through the use of sound.

10. The method according to any previous claim, further comprising receiving an annotation from a participant made in relation to at least one of the media items and communicating the annotation to the participants .

11. A communication handling arrangement (16) for controlling a communication session for a number of participants, where the participants comprise a local user at a location in an industrial site where a process control system (10) is operated and a remote user at a remote location, the communication session comprising a first media item (Mil) in the form of video captured by the local user and the communication handling arrangement comprising:

a communication handling unit (20) configured to: - provide the video stream (VS) to participants of the communication session for communication of the first media item (Mil),

fetch graphical representations (PG1, PG2, PG3, PG4) of at least one process control object, where the graphical representations are representations of process control objects present in the communication of the first media item,

provide the graphical representations in a second media item (MI2) for communication to the

participants of the session,

- visually communicating to the other participants

registration of the input via the other media item.

12. The communication handling arrangement according to claim 11, wherein the communication handling unit is further configured to receive a selection of the first media item from one of the participants and fetch the graphical representations based on the selection.

13. The communication handling arrangement according to claim 12, wherein the selection is the selection of at least one of the process control objects shown in a presentation of the first media item and the second media item comprises a graphical representation of the selected process control object.

14. The communication handling arrangement according to any of claims 11 - 13, wherein the communication handling unit (20) is configured to perform the visual communicating in a media item through visually emphasizing the first process control object.

15. The communication handling arrangement according to any of claims 11 - 14, wherein the communication handling unit (38) is further configured to visually communicate to the participants in the communication session the registration of the input also through the media item via which input is made.

16. The communication handling arrangement according to any of claims 11 - 15, further comprising at least one marker, for pointing at process control objects in the media items, said marker being

selectable by any one participant of the communication session, the communication handling unit (20) being further configured to receive at least one request for controlling the marker from the participants, and if more than one request is received from more than one participant, determine which participant that is to receive control according to a priority scheme and provide control to the selected participant, where the priority scheme indicates that the remote user has priority over other participants.

17. The communication handling arrangement

according to claim 16, wherein the communication of said graphical representations of process control objects comprise retrieval and communication of real- time data of the corresponding process control objects.

18. The communication handling arrangement

according to any of claims 11 - 17, comprising a communication handling device in the process control system, said communication handling device comprising a first communication interface (17) for communicating with a data presenting device of the local user, a second communication interface (18) for communicating with a device (51) of the remote user and the

communication handling unit (20) .

19. A computer program product controlling a communication session for a number of participants, where the participants comprise a local user at a location in an industrial site where a process control system (10) is operated and a remote user (52) at a remote location, the communication session comprising a first media item (Mil) in the form of video captured by the local user, said computer program product being provided on a data carrier (96) comprising computer program code (82) the computer program product

comprising a data carrier (80) with computer program code (82) which when run in a communication handling unit of the communication handling arrangement is configured to cause the communication handling unit to provide the video stream (VS) to participants of the communication session for communication of the first media item (Mil), fetch graphical representations (PG1, PG2, PG3, PG4) of at least one process control object, where the graphical representations are representations of process control objects present in the communication of the first media item,

participants of the session,

visually communicating to the other participants registration of the input via the other media item.