WO2011122267A1 - ネットワークシステム、通信方法、および通信端末 - Google Patents
ネットワークシステム、通信方法、および通信端末 Download PDFInfo
- Publication number
- WO2011122267A1 WO2011122267A1 PCT/JP2011/055382 JP2011055382W WO2011122267A1 WO 2011122267 A1 WO2011122267 A1 WO 2011122267A1 JP 2011055382 W JP2011055382 W JP 2011055382W WO 2011122267 A1 WO2011122267 A1 WO 2011122267A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- hand
- mobile phone
- cpu
- input
- moving image
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/4104—Peripherals receiving signals from specially adapted client devices
- H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/632—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing using a connection between clients on a wide area network, e.g. setting up a peer-to-peer communication via Internet for retrieving video segments from the hard-disk of other client devices
Definitions
- the present invention relates to a network system including at least first and second communication terminals that can communicate with each other, a communication method, and a communication terminal, and more particularly, a network system in which the first and second communication terminals reproduce the same moving image content. , A communication method, and a communication terminal.
- a network system is known in which a plurality of communication terminals that can be connected to the Internet exchange hand-drawn images.
- a server / client system For example, a server / client system, a P2P (Peer to Peer) system, etc. are mentioned.
- each communication terminal transmits and receives hand-drawn images and text data.
- Each of the communication terminals displays a hand-drawn image or text on the display based on the received data.
- a communication terminal that downloads the content from a server that stores the content including the moving image via the Internet and reproduces the content is also known.
- Patent Document 1 discloses a chat service system for mobile phones.
- a moving image display area and a character display area are displayed on the browser display screen of the terminal for a large number of mobile phone terminals and operator web terminals connected via the Internet.
- a chat server to be displayed in the character display area, and each of the operator web terminals forms an independent chat channel for each mobile phone terminal with respect to the plurality of mobile phone terminals.
- the present invention has been made to solve such a problem, and its purpose is to more effectively convey the intention of a user who transmits (inputs) information to a user who receives (views) information.
- the present invention relates to a network system, a communication method, and a communication terminal.
- a network system including first and second communication terminals.
- the first communication terminal accepts an input of a hand-drawn image via the first communication device for communicating with the second communication terminal, the first touch panel for displaying moving image content, and the first touch panel.
- the first processor transmits the hand-drawn image input during the display of the moving image content and the start information for specifying the input start time of the hand-drawn image in the moving image content via the first communication device to the second communication.
- the second communication terminal is based on the second touch panel for displaying the moving image content, the second communication device for receiving the hand-drawn image and the start information from the first communication terminal, and the start information.
- the second touch panel includes a second processor for displaying the hand-drawn image from the start of input of the hand-drawn image in the moving image content.
- the network system further includes a content server for distributing moving image content.
- the first processor acquires the moving image content from the content server in response to the download command, and transmits the moving image information for specifying the acquired moving image content to the second communication terminal via the first communication device.
- the second processor acquires moving image content from the content server based on the moving image information.
- the first processor performs hand drawing via the first communication device when a scene of the moving image content is switched and / or when an instruction to clear the input hand drawn image is received.
- a command for erasing the image is transmitted to the second communication terminal.
- the second processor calculates the time from the input start time to the time when the scene of the moving image content is switched, and determines the drawing speed of the hand-drawn image on the second touch panel based on the time.
- the second processor calculates the length of the scene of the moving image content including the input start time, and determines the drawing speed of the hand-drawn image on the second touch panel based on the length.
- a communication method in a network system including first and second communication terminals that can communicate with each other includes a step in which the first communication terminal displays the moving image content, a step in which the first communication terminal accepts input of a hand-drawn image, and the first communication terminal is input while the moving image content is being displayed. Transmitting the hand-drawn image and start information for specifying the input start time of the hand-drawn image in the moving image content to the second communication terminal, and displaying the moving image content in the second communication terminal.
- the second communication terminal receives the hand-drawn image and the start information from the first communication terminal, and the second communication terminal starts the input of the hand-drawn image in the video content based on the start information. Displaying a hand-drawn image.
- a communication terminal capable of communicating with other communication terminals.
- the communication terminal includes a communication device for communicating with another communication terminal, a touch panel for displaying moving image content, and a processor for receiving an input of a first hand-drawn image via the touch panel.
- the processor receives the first hand-drawn image input during the display of the moving image content and the first start information for specifying the input start time of the first hand-drawn image in the moving image content via the communication device.
- To the communication terminal receives the second hand-drawn image and the second start information from the other communication terminal, and, based on the second start information, displays the second hand-drawn image in the moving image content on the touch panel.
- the second hand-drawn image is displayed from the input start time.
- a communication method in a communication terminal including a communication device, a touch panel, and a processor.
- the communication method includes a step in which the processor displays the moving image content on the touch panel, a step in which the processor receives an input of the first hand-drawn image via the touch panel, and a first input by the processor during the display of the moving image content.
- the intention of the user who transmits (inputs) information can be more effectively transmitted to the user who receives (views) information by the network system, communication method, and communication terminal according to the present invention. It becomes like this.
- FIG. 7 is a flowchart showing a processing procedure of a modified example of the P2P communication processing in the mobile phone according to the first embodiment.
- 3 is a flowchart showing a processing procedure of input processing in the mobile phone according to the first embodiment. It is a flowchart which shows the process sequence of the setting process of the pen information in the mobile telephone which concerns on this Embodiment. 3 is a flowchart showing a processing procedure of hand-drawing processing in the mobile phone according to Embodiment 1; 7 is a flowchart showing a processing procedure of a modification of the input processing in the mobile phone according to the first embodiment.
- FIG. 3 is a flowchart showing a processing procedure of hand-drawn image display processing in the mobile phone according to Embodiment 1; 6 is a flowchart showing a processing procedure of first drawing processing in the mobile phone according to Embodiment 1; 6 is a first image diagram for explaining a hand-drawn image display process according to Embodiment 1.
- FIG. 6 is a flowchart showing a processing procedure of a modification of the hand-drawn image display processing in the mobile phone according to the first embodiment.
- 6 is a flowchart showing a processing procedure of second drawing processing in the mobile phone according to Embodiment 1;
- FIG. 6 is a second image diagram for explaining a hand-drawn image display process according to the first embodiment.
- FIG. 12 is a flowchart showing a processing procedure of another modification of the hand-drawn image display processing in the mobile phone according to the first embodiment.
- 7 is a flowchart showing a processing procedure of third drawing processing in the mobile phone according to Embodiment 1;
- FIG. 10 is a third image diagram for illustrating a hand-drawn image display process according to the first embodiment.
- 10 is a flowchart showing a processing procedure of P2P communication processing in the mobile phone according to the second embodiment.
- 10 is an image diagram illustrating a data structure of transmission data according to Embodiment 2.
- FIG. 10 is a flowchart showing a processing procedure of input processing in the mobile phone according to the second embodiment.
- 10 is a flowchart showing a processing procedure of hand-drawing processing in the mobile phone according to the second embodiment.
- 10 is a flowchart showing a processing procedure of display processing in the mobile phone according to the second embodiment.
- 10 is a flowchart illustrating a processing procedure of an application example of display processing in the mobile phone according to the second embodiment.
- 10 is a flowchart showing a processing procedure of hand-drawn image display processing in the mobile phone according to Embodiment 2.
- the mobile phone 100 will be described as a representative example of the “communication terminal”.
- the communication terminal is connected to a network such as a personal computer, a car navigation device (Satellite navigation system), a PND (Personal Navigation Device), a PDA (Personal Data Assistance), a game machine, an electronic dictionary, an electronic BOOK, etc.
- a network such as a personal computer, a car navigation device (Satellite navigation system), a PND (Personal Navigation Device), a PDA (Personal Data Assistance), a game machine, an electronic dictionary, an electronic BOOK, etc.
- Other information communication devices that can be connected may be used.
- FIG. 1 is a schematic diagram showing an example of a network system 1 according to the present embodiment.
- the network system 1 includes mobile phones 100A, 100B, 100C, and 100D, a chat server (first server device) 400, a content server (second server device) 600, and the Internet (first 1 network) 500 and a carrier network (second network) 700.
- the network system 1 according to the present embodiment includes a car navigation device 200 mounted on a vehicle 250 and a personal computer (PC) 300.
- PC personal computer
- the network system 1 includes a first mobile phone 100A, a second mobile phone 100B, a third mobile phone 100C, and a fourth mobile phone.
- a case where the mobile phone 100D is included will be described.
- the mobile phones 100A, 100B, 100C, and 100D are also collectively referred to as the mobile phone 100.
- a communication terminal generically.
- the mobile phone 100 is configured to be connectable to the carrier network 700.
- the car navigation device 200 is configured to be connectable to the Internet 500.
- the personal computer 300 is configured to be connectable to the Internet 500 via a LAN (Local Area Network) 350 or a WAN (Wide Area Network).
- Chat server 400 is configured to be connectable to the Internet 500.
- the content server 600 is configured to be connectable to the Internet 500.
- the first mobile phone 100A, the second mobile phone 100B, the third mobile phone 100C, the fourth mobile phone 100D, the car navigation device 200, and the personal computer 300 are connected to the Internet.
- 500, the carrier network 700, and a mail transmission server (chat server 400 in FIG. 2) can be connected to each other and can transmit and receive data to and from each other.
- identification information for example, an e-mail address or an IP (Internet Protocol) address
- identification information for example, an e-mail address or an IP (Internet Protocol) address
- the mobile phone 100, the car navigation device 200, and the personal computer 300 can store identification information of other communication terminals in an internal recording medium, and the relevant information is transmitted via the carrier network 700, the Internet 500, or the like based on the identification information. Data can be transmitted and received with other communication terminals.
- Mobile phone 100, car navigation device 200, and personal computer 300 transmit / receive data to / from other communication terminals without using servers 400 and 600 using an IP address assigned to the other terminal. It is also possible to perform. That is, the mobile phone 100, the car navigation device 200, and the personal computer 300 included in the network system 1 according to the present embodiment can constitute a so-called P2P (Peer to Peer) type network.
- P2P Peer to Peer
- chat server 400 when each communication terminal accesses the chat server 400, that is, when each communication terminal accesses the Internet, an IP address is assigned by the chat server 400 or another server device (not shown). Details of the IP address assignment process are well known, and therefore the description will not be repeated here.
- the mobile phone 100, the car navigation device 200, and the personal computer 300 receive various video contents from the content server 600 via the Internet 500.
- the users of the mobile phone 100, the car navigation device 200, and the personal computer 300 can view the moving image content from the content server 600.
- FIG. 2 is a sequence diagram showing an outline of operation in the network system 1 according to the present embodiment.
- an outline of communication processing between the first mobile phone 100A and the second mobile phone 100B will be described.
- each communication terminal needs to exchange (acquire) each other's IP address first in order to perform P2P type data transmission / reception.
- Each communication terminal after acquiring the other party's IP address, transmits a hand-drawn image message, an attached file, and the like to other communication terminals by P2P type data transmission / reception.
- chat server 400 may also serve as the content server 600.
- first mobile phone 100A requests IP registration (login) from chat server 400 (step S0002).
- First mobile phone 100A may obtain an IP address at the same time, or may obtain an IP address in advance. More specifically, the first mobile phone 100A transmits the mail address, IP address, and first address of the first mobile phone 100A to the chat server 400 via the carrier network 700, the mail transmission server (chat server 400), and the Internet 500. 2 sends the mail address of the mobile phone 100B and a message requesting the generation of a new chat room.
- Chat server 400 stores the mail address of first mobile phone 100A in association with the IP address in response to the request. Then, chat server 400 generates a room name based on the mail address of first mobile phone 100A and the mail address of second mobile phone 100B, and generates a chat room with the room name. At this time, chat server 400 may notify first mobile phone 100A that the generation of the chat room has been completed. Chat server 400 stores room names and IP addresses of participating communication terminals in association with each other.
- first mobile phone 100A generates a room name of a new chat room based on the mail address of first mobile phone 100A and the mail address of second mobile phone 100B, and chats the room name.
- Chat server 400 generates a new chat room based on the room name.
- the first mobile phone 100A transmits a P2P participation request mail indicating that a new chat room has been generated, that is, an invitation to the chat room, to the second mobile phone 100B (steps S0004 and S0006). More specifically, the first mobile phone 100A transmits a P2P participation request email to the second mobile phone 100B via the carrier network 700, a mail transmission server (chat server 400), and the Internet 500 (step S0004, Step S0006).
- the second mobile phone 100B When the second mobile phone 100B receives the P2P participation request email (step S0006), the second mobile phone 100B generates a room name based on the email address of the first mobile phone 100A and the email address of the second mobile phone 100B, and chats. A message to join the chat room having the mail address, IP address, and room name of second mobile phone 100B is transmitted to server 400 (step S0008). Second mobile phone 100B may acquire the IP address at the same time, or may access chat server 400 after acquiring the IP address first.
- Chat server 400 accepts the message, determines whether or not the email address of second mobile phone 100B corresponds to the room name, and then sets the email address of second mobile phone 100B to the IP address. Store in association with. Then, chat server 400 transmits to first mobile phone 100A the fact that second mobile phone 100B has joined the chat room and the IP address of second mobile phone 100B (step S0010). At the same time, chat server 400 transmits to second mobile phone 100B that it has accepted participation in the chat room and the IP address of first mobile phone 100A.
- the first mobile phone 100A and the second mobile phone 100B acquire each other's mail address and IP address and authenticate each other (step S0012).
- first mobile phone 100A and second mobile phone 100B start P2P communication (chat communication) (step S0014). An outline of the operation during P2P communication will be described later.
- first mobile phone 100A transmits a message to disconnect P2P communication to second mobile phone 100B (step S0016)
- second mobile phone 100B makes a request to first mobile phone 100A to disconnect.
- a message indicating acceptance is transmitted (step S0018).
- First mobile phone 100A transmits a request to delete chat room to chat server 400 (step S0020), and chat server 400 deletes the chat room.
- FIG. 3 is an image diagram showing transition of the display mode of the communication terminal along the operation outline according to the present embodiment.
- the first mobile phone 100A and the second mobile phone 100B transmit and receive hand-drawn images while displaying the content acquired from the content server 600 as the background will be described.
- the content here may be a moving image or a still image.
- the first mobile phone 100A receives and displays the content.
- first mobile phone 100A accepts a command to start chatting.
- first mobile phone 100A accepts the other user's selection command.
- first mobile phone 100A transmits information for specifying content to second mobile phone 100B via mail transmission server (chat server 400).
- second mobile phone 100B receives information from first mobile phone 100A (step S0006).
- Second mobile phone 100B receives and displays the content based on the information.
- both the first mobile phone 100A and the second mobile phone 100B may receive content from the content server 600 after the P2P communication is started, that is, during the P2P communication.
- the first mobile phone 100A can repeat the mail transmission without P2P communication with the second mobile phone 100B.
- first mobile phone 100A registers its own IP address in chat server 400, and a new one is created based on the mail address of first mobile phone 100A and the mail address of second mobile phone 100B.
- a request is made to generate a chat room (step S0002).
- second mobile phone 100B accepts a command to start chatting, and sends chat room 400 a message indicating that the room name and chat room are to be joined, and its own IP address. Are transmitted (step S0008).
- the first mobile phone 100A acquires the IP address of the second mobile phone 100B
- the second mobile phone 100B acquires the IP address of the first mobile phone 100A (step S0010), and authenticates each other. We meet (step S0012).
- the first mobile phone 100A and the second mobile phone 100B can perform P2P communication (step S0014). That is, the first mobile phone 100A and the second mobile phone 100B according to the present embodiment can transmit and receive information such as hand-drawn images while displaying the downloaded content.
- first mobile phone 100A accepts an input of a hand-drawn image from the user and displays the hand-drawn image on the content.
- first mobile phone 100A transmits the hand-drawn image to second mobile phone 100B.
- Second mobile phone 100B displays a hand-drawn image on the content based on the hand-drawn image from first mobile phone 100A.
- the second mobile phone 100B also receives an input of a hand-drawn image from the user and displays the hand-drawn image on the content. Second mobile phone 100B transmits the hand-drawn image to first mobile phone 100A. Second mobile phone 100B displays a hand-drawn image on the content based on the hand-drawn image from first mobile phone 100A.
- the second mobile phone 100B sends an e-mail to the first mobile phone 100A or the like. It can be performed. Note that it is also possible to perform P2P communication by the TCP / IP communication method and mail transmission / reception by the HTTP communication method. That is, it is possible to send and receive mail during P2P communication.
- FIG. 4 is an image diagram showing an outline of operations related to input and drawing of hand-drawn images during reproduction of moving image content.
- the first mobile phone 100A and the second mobile phone 100B start chat communication, then the third mobile phone 100C starts chat communication, and then the fourth mobile phone 100D starts chat communication. The case where it does is demonstrated.
- first mobile phone 100A, second mobile phone 100B, third mobile phone 100C, and fourth mobile phone 100D start to download moving image content from content server 600 at different timings. . Then, the first mobile phone 100A, the second mobile phone 100B, the third mobile phone 100C, and the fourth mobile phone 100D start to play the moving image content at different timings. Naturally, the first mobile phone 100A, the second mobile phone 100B, the third mobile phone 100C, and the fourth mobile phone 100D end the reproduction of the moving image content at different timings.
- first mobile phone 100A in FIG. 4 accepts input of information such as hand-drawn images during playback of moving image content.
- another mobile phone (second mobile phone 100B in FIG.
- each of the mobile phones 100A to 100D displays the hand-drawn image input to the first mobile phone 100A on the same scene of the same moving image content. In other words, each of the mobile phones 100A to 100D starts drawing the hand-drawn image input to the first mobile phone 100A on the moving image content when the same time has elapsed since the start of the moving image content.
- each communication terminal separately downloads the moving image content from the content server 600, but the hand-drawn image input to one communication terminal is It can be displayed on the same scene or on the same frame.
- the information is displayed together with the one scene on another communication terminal. That is, in the present embodiment, the intention of the user who transmits (inputs) information can be effectively communicated to the user who receives (views) information.
- FIG. 5 is an image diagram showing an appearance of mobile phone 100 according to the present embodiment.
- FIG. 6 is a block diagram showing a hardware configuration of mobile phone 100 according to the present embodiment.
- the mobile phone 100 includes a communication device 101 that transmits / receives data to / from an external network, a memory 103 that stores programs and various databases, a CPU ( Central Processing Unit) 106, display 107, microphone 108 to which sound is input from the outside, speaker 109 that outputs sound to the outside, various buttons 110 that receive input of information and commands, and communication data from the outside And a first notification unit 111 that outputs a voice indicating that the call signal has been received, and a second notification unit 112 that displays that the communication data and the call signal from the outside have been received.
- the display 107 according to the present embodiment realizes the touch panel 102 including a liquid crystal panel and a CRT. That is, in the mobile phone 100 according to the present embodiment, the pen tablet 104 is laid on the upper side (front side) of the display 107. Accordingly, the user can input graphic information or the like into the CPU 106 via the pen tablet 104 by hand using the stylus pen 120 or the like.
- the user can also perform hand-drawn input by the following method. That is, by using a special pen that outputs infrared light and sound waves, the movement of the pen is determined by a receiving unit that receives infrared light and sound waves transmitted from the pen. In this case, by connecting the receiving unit to a device that stores a trajectory, the CPU 106 can receive the trajectory output from the device as a hand-drawn input.
- the user can write a hand-drawn image on the electrostatic panel using a finger or an electrostatic-compatible pen.
- the display 107 displays an image or text based on the data output from the CPU 106.
- the display 107 displays moving image content received via the communication device 101.
- the display 107 displays the hand-drawn image superimposed on the moving image content based on the hand-drawn image received via the tablet 104 or the hand-drawn image received via the communication device 101.
- the various buttons 110 receive information from the user by a key input operation or the like.
- the various buttons 110 include a TEL button 110A for accepting a call or making a call, a mail button 110B for accepting a mail or sending a mail, accepting P2P communication, or performing P2P communication.
- P2P button 110C for issuing, address book button 110D for calling up address book data, and end button 110E for ending various processes. That is, when receiving the P2P participation request mail via the communication device 101, the various buttons 110 accept from the user an instruction to participate in the chat room, an instruction to display the contents of the mail, and the like.
- the various buttons 110 may include a button for receiving a command for starting hand-drawn input, that is, a button for receiving a first input.
- the various buttons 110 may include a button for receiving a command for ending hand-drawn input, that is, a button for receiving a second input.
- the first notification unit 111 outputs a ring tone through the speaker 109 or the like. Alternatively, the first notification unit 111 has a vibration function. The first notification unit 111 outputs a voice or vibrates the mobile phone 100 when an incoming call is received, a mail is received, or a P2P participation request mail is received.
- the second notification unit 112 includes a TEL LED (Light Emitting Diode) 112A that flashes when an incoming call is received, a mail LED 112B that flashes when a mail is received, and a P2P LED 112C that flashes when a P2P communication is received. including.
- TEL LED Light Emitting Diode
- CPU 106 controls each unit of mobile phone 100. For example, various commands are received from the user via various buttons 110, and data is transmitted / received to / from an external communication terminal via the communication device 101, the communication device 101, or the network.
- the communication device 101 converts communication data from the CPU 106 into a communication signal and transmits the communication signal to the outside.
- the communication device 101 converts communication signals received from the outside into communication data, and inputs the communication data to the CPU 106.
- the memory 103 is realized by a RAM (Random Access Memory) that functions as a working memory, a ROM (Read Only Memory) that stores a control program, a hard disk that stores image data, and the like.
- FIG. 7A is an image diagram showing the data structure of various work memories 103 ⁇ / b> A that constitute the memory 103.
- FIG. 7B is an image diagram showing the address book data 103B stored in the memory 103.
- FIG. 7C is an image diagram showing the own terminal data 103 ⁇ / b> C stored in the memory 103.
- FIG. 7D is an image diagram showing the IP address data 103D of the own terminal and the IP address data 103E of another terminal stored in the memory 103.
- the work memory 103A of the memory 103 includes an RCVTELNO area for storing a caller's telephone number, an RCVMAIL area for storing information about received mail, and a SENDMIL area for storing information about outgoing mail.
- the work memory 103A may not store a telephone number.
- the information related to the received mail includes the mail text stored in the MAIN area and the mail address of the mail transmission source stored in the RCVMAIL FROM area.
- the information regarding the outgoing mail includes the mail text stored in the MAIN area and the mail address of the mail destination stored in the TO area of RCVMAIL.
- the address book data 103B associates a memory No. for each destination (other communication terminal).
- the address book data 103B stores a name, a telephone number, a mail address, and the like in association with each other for each destination.
- the own terminal data 103C stores the name of the user of the own terminal, the telephone number of the own terminal, the mail address of the own terminal, and the like.
- the IP address data 103D of the own terminal stores the IP address of the own terminal.
- the IP address data 103E of the other terminal stores the IP address of the other terminal.
- Each of the mobile phones 100 according to the present embodiment uses the data shown in FIG. 7 to transmit data to and from other communication terminals in the manner described above (see FIGS. 1 to 3). Can be sent and received.
- chat server 400 and content server 600 Next, the hardware configuration of chat server 400 and content server 600 according to the present embodiment will be described. Below, the hardware configuration of the chat server 400 will be described first.
- FIG. 8 is a block diagram showing a hardware configuration of chat server 400 according to the present embodiment.
- chat server 400 according to the present embodiment includes a CPU 405, a memory 406, a fixed disk 407, and a communication device 409 that are mutually connected via an internal bus 408.
- the memory 406 stores various types of information. For example, the memory 406 temporarily stores data necessary for executing a program in the CPU 405.
- the fixed disk 407 stores a program executed by the CPU 405 and a database.
- the CPU 405 controls each element of the chat server 400 and is a device that performs various calculations.
- the communication device 409 converts the data output from the CPU 405 into an electric signal and transmits it to the outside, converts the electric signal received from the outside into data, and inputs the data to the CPU 405. Specifically, the communication device 409 transmits the data from the CPU 405 to the mobile phone 100, the car navigation device 200, the personal computer 300, the game machine, the electronic dictionary, the electronic BOOK, etc. via the Internet 500 or the carrier network 700. Send to a device that can be connected to the network. The communication device 409 receives data received from the mobile phone 100, the car navigation device 200, the personal computer 300, a game machine, an electronic dictionary, an electronic BOOK, and other devices connectable to the network via the Internet 500 or the carrier network 700. Is input to the CPU 405.
- FIG. 9A is a first image diagram showing the data structure of the room management table 406A stored in the memory 406 or the fixed disk 407 of the chat server 400.
- FIG. 9B shows the memory of the chat server 400.
- FIG. 9A is a first image diagram showing the data structure of the room management table 406A stored in the memory 406 or the fixed disk 407 of the chat server 400.
- FIG. 9B shows the memory of the chat server 400.
- the room management table 406A stores room names and IP addresses in association with each other. For example, at a certain point in time, as shown in FIG. 9A, a chat room having a room name R, a chat room having a room name S, and a chat room having a room name T are generated in the chat server 400. Then, a communication terminal having an IP address of A and a communication terminal having an IP address of C enter the chat room having the room name R. A communication terminal having an IP address of B enters the chat room having the room name S. A communication terminal having an IP address of D enters the chat room having the room name T.
- the room name R is determined by the CPU 406 based on the mail address of a communication terminal having an IP address of A and the mail address of a communication terminal having an IP address of B.
- the room management table 406A stores the room name S as shown in FIG. 9B.
- IP address E are stored in association with each other.
- chat server 400 when first mobile phone 100A requests generation of a new chat room (step S0002 in FIG. 2), CPU 405 determines the mail address of first mobile phone 100A.
- the room name is generated based on the mail address of the second mobile phone 100B and the room name is stored in the room management table 406A in association with the IP address of the first mobile phone 100A.
- the CPU 405 stores the room name and the second mobile phone 100B in the room management table 406A. Is stored in association with the IP address of CPU 406 reads the IP address of first mobile phone 100A corresponding to the room name from room management table 406A. CPU 406 transmits the IP address of first mobile phone 100A to each second communication terminal, and transmits the IP address of second mobile phone 100B to first mobile phone 100A.
- the content server 600 includes a CPU 605, a memory 606, a fixed disk 607, and a communication device 609 that are mutually connected via an internal bus 608.
- the memory 606 stores various types of information. For example, the memory 606 temporarily stores data necessary for execution of a program by the CPU 605.
- the fixed disk 607 stores a program executed by the CPU 605 and a database.
- the CPU 605 controls each element of the content server 600 and is a device that performs various calculations.
- the communication device 609 converts the data output from the CPU 605 into an electrical signal and transmits it to the outside, and converts the electrical signal received from the outside into data and inputs it to the CPU 605. Specifically, the communication device 609 transmits the data from the CPU 605 to the mobile phone 100, the car navigation device 200, the personal computer 300, the game machine, the electronic dictionary, the electronic BOOK, etc. via the Internet 500 or the carrier network 700. Send to a device that can be connected to the network. The communication device 609 receives data received from the mobile phone 100, the car navigation device 200, the personal computer 300, a game machine, an electronic dictionary, an electronic BOOK, and other devices connectable to the network via the Internet 500 or the carrier network 700. Is input to the CPU 605.
- the memory 606 or the fixed disk 615 of the content server 600 stores moving image content.
- the CPU 605 of the content server 600 receives content specification (such as an address indicating the storage location of the moving image content) from the first mobile phone 100A and the second mobile phone 100B via the communication device 609. Based on the content specification, the CPU 605 of the content server 600 reads the moving image content corresponding to the specification from the memory 606, and transmits the content via the communication device 609 to the first mobile phone 100A and the second mobile phone 100B. Send to.
- content specification such as an address indicating the storage location of the moving image content
- FIG. 10 is a flowchart showing a processing procedure of P2P communication processing in mobile phone 100 according to the present embodiment.
- FIG. 11 is an image diagram showing a data structure of transmission data according to the present embodiment.
- first mobile phone 100A transmits designation of moving image content, a hand-drawn image, or the like to the second mobile phone 100B.
- first mobile phone 100A and second mobile phone 100B transmit and receive data via chat server 400.
- data may be transmitted / received by P2P communication without using the chat server 400.
- first mobile phone 100A needs to store data or transmit data to second mobile phone 100B, third mobile phone 100C, or the like. .
- CPU 106 of first mobile phone 100A acquires data related to chat communication from chat server 400 via communication device 101 (step S002).
- CPU 106 of second mobile phone 100B (reception side) also acquires data related to chat communication from chat server 400 via communication device 101 (step S004).
- data related to chat communication includes chat room ID, member terminal information, notification (notification information), chat contents up to this point, and the like.
- the CPU 106 of the first mobile phone 100A displays a chat communication window on the touch panel 102 (step S006).
- CPU 106 of second mobile phone 100B displays a chat communication window on touch panel 102 (step S008).
- the CPU 106 of the first mobile phone 100A receives the moving image content via the communication device 101 based on the content reproduction command from the user (step S010). More specifically, the CPU 106 receives a command for designating moving image content from the user via the touch panel 102. The user may directly input a URL (Uniform Resource Locator) to the first mobile phone 100A, or may select a link corresponding to a desired moving image content on the displayed web page.
- a URL Uniform Resource Locator
- the CPU 106 of the first mobile phone 100A uses the communication device 101 to obtain the video information (a) for specifying the selected video content via the chat server 400 and other communications participating in the chat. Transmit to the terminal (step S012). Alternatively, the CPU 106 of the first mobile phone 100A uses the communication device 101 to obtain the video information (a) for specifying the selected video content by P2P communication and other communication terminals participating in the chat. Send directly to. As shown in FIG. 11, the moving image information (a) includes, for example, a URL indicating the storage location of the moving image content.
- the CPU 405 of the chat server 400 stores the moving image information (a) in the memory 406 for a communication terminal that participates in the chat later.
- the CPU 106 of the first mobile phone 100A starts playing the received moving image content via the touch panel 102 (step S014).
- the CPU 106 may output the sound of the moving image content via the speaker 109.
- the CPU 106 of the second mobile phone 100B receives the moving image information (a) from the chat server 400 via the communication device 101 (step S016).
- the CPU 106 analyzes the moving image information (step S018) and downloads the moving image content from the content server 600 (step S020).
- the CPU 106 starts playing the received moving image content via the touch panel 102 (step S022).
- the CPU 106 may output the sound of the moving image content via the speaker 109.
- first mobile phone 100A and the second mobile phone 100B acquire moving image information during chat communication
- present invention is not limited to this, and the first mobile phone 100A and the second mobile phone 100B Mobile phone 100B may acquire common video information before chat communication.
- CPU 106 of third mobile phone 100C acquires chat data from chat server 400 via communication device 101 (step S024).
- chat server 400 stores moving image information (a) from first mobile phone 100A.
- the CPU 405 of the chat server 400 transmits the moving image information (a) as part of the chat data to the third mobile phone 100C via the communication device 409.
- the CPU 106 of the third mobile phone 100C analyzes the chat data and acquires moving image information (step S026).
- CPU 106 acquires moving image content from content server 600 based on the moving image information (step S028).
- the CPU 106 starts playing the received moving image content via the touch panel 102 (step S030). At this time, the CPU 106 may output the sound of the moving image content via the speaker 109.
- the CPU 106 accepts a hand-drawn input by the user via the touch panel 102 while the first mobile phone 100A is reproducing the moving image content (step S032).
- the CPU 106 acquires the change (trajectory) of the contact position with respect to the touch panel 102 by sequentially receiving the contact coordinate data from the touch panel 102 every predetermined time. Then, as shown in FIG. 11, the CPU 106 performs hand-drawn clear information (b), information (c) indicating the locus of the contact position, information (d) indicating the line color, and information indicating the line width ( e) and transmission data including timing information (f) indicating the timing when the hand-drawn input is started are created (step S034).
- the hand-drawn clear information (b) includes information (true) for clearing the hand-drawn input input so far or information (false) for continuing the hand-drawn input.
- the information (c) indicating the trajectory of the contact position includes the coordinates of each vertex constituting the hand-drawn stroke and the elapsed time from the start of hand-drawn input corresponding to each vertex.
- the timing information (f) is also information indicating the timing at which drawing of a hand-drawn image should be started.
- the timing information (f) includes, for example, the time (ms) from the start of moving image content when the first mobile phone 100A accepts hand-drawn input, and information (scene number) that specifies the scene of the moving image content Etc.) and information (frame number etc.) for specifying the frame of the video content.
- step S032 the CPU 106 causes the touch panel 102 to display the hand-drawn image input on the moving image content (superimposed on the moving image content). As shown in FIGS. 4B to 4D, the CPU 106 displays the hand-drawn image on the touch panel 102 in response to the input of the hand-drawn image.
- the CPU 106 may transmit clear information (true) using the communication device 101 when the scene is switched.
- the CPU 106 repeats the processing from step S032 to step S034 every time it receives an input of a hand-drawn image. Alternatively, CPU 106 repeats the processes of steps S032 to S036 every time an input of a hand-drawn image is received. And as shown in FIG.4 (f), CPU106 complete
- the CPU 106 uses the communication device 101 to transmit the transmission data to other communication terminals participating in the chat via the chat server 400 (step S036).
- the CPU 405 of the chat server 400 stores transmission data (b) to (f) in the memory 406 for communication terminals that will participate in chat later.
- the second mobile phone 100B and the third mobile phone 100C are participating in the chat.
- CPU106 transmits the said transmission data directly to the other communication terminal which has participated in chat by P2P communication by using the communication device 101 (step S036).
- the CPU 106 of the second mobile phone 100B receives the transmission data (b) to (f) from the chat server 400 via the communication device 101 (step S038).
- CPU 106 analyzes the transmission data (step S040). As shown in FIGS. 4H to 4J, for each transmission data, the CPU 106 causes the touch panel 102 to draw a hand-drawn image on the moving image content on the basis of the timing information (f) of the transmission data (step S042).
- CPU 106 may erase the hand-drawn image based on the clear information from first mobile phone 100A. Alternatively, the CPU 106 may erase the hand-drawn image by determining by itself that the scene has been switched. Then, as shown in FIG. 4L, the CPU 106 ends the reproduction of the moving image content (step S060).
- CPU 106 of third mobile phone 100C receives transmission data from chat server 400 via communication device 101 (step S044).
- CPU 106 analyzes the transmission data (step S046).
- the CPU 106 causes the touch panel 102 to draw a hand-drawn image on the moving image content on the basis of the timing information (f) of the transmission data (step S048).
- CPU 106 may erase the hand-drawn image based on the clear information from first mobile phone 100A. Alternatively, the CPU 106 may erase the hand-drawn image by determining by itself that the scene has been switched. Then, as shown in FIG. 4 (r), the CPU 106 ends the reproduction of the moving image content (step S062).
- the fourth mobile phone 100D participates in the chat. More specifically, it is assumed that the fourth mobile phone 100D participates in the chat after the hand-drawn input is completed on the first mobile phone 100A. It does not matter whether or not the reproduction of the moving image content has ended in the first mobile phone 100A, the second mobile phone 100B, and the third mobile phone 100C.
- the CPU 106 of the fourth mobile phone 100D acquires chat data from the chat server 400 via the communication device 101 (step S050). At this time, the chat server 400 stores moving image information (a) from the first mobile phone 100A. The CPU 405 of the chat server 400 transmits the moving image information (a) and the transmission data (b) to (f) accumulated up to the present time as a part of the chat data via the communication device 409 to the fourth mobile phone. Send to phone 100D.
- the CPU 106 of the fourth mobile phone 100D analyzes the chat data and acquires moving image information and transmission data (step S052).
- CPU 106 acquires moving image content from content server 600 based on the moving image information (step S054).
- the CPU 106 starts playing the received moving image content via the touch panel 102 (step S056).
- the CPU 106 may output the sound of the moving image content via the speaker 109.
- the CPU 106 causes the touch panel 102 to draw a hand-drawn image on the moving image content based on the timing information (f) of the transmission data (step S064).
- CPU 106 may erase the hand-drawn image based on the clear information from first mobile phone 100A. Alternatively, the CPU 106 may erase the hand-drawn image by determining by itself that the scene has been switched.
- the hand-drawn image is displayed on the second mobile phone 100B, the third mobile phone 100C, and the fourth mobile phone 100D at the same timing as that in the moving image content in which the hand-drawn image is input on the first mobile phone 100A. It will be drawn. That is, in the second mobile phone 100B, the third mobile phone 100C, and the fourth mobile phone 100D, desired information is drawn in a scene desired by the user of the first mobile phone 100A.
- FIG. 12 is a flowchart showing a processing procedure of a modified example of the P2P communication processing in mobile phone 100 according to the present embodiment.
- FIG. 12 shows that after the reproduction of the moving image content and the hand-drawn input are completed in the first communication terminal, the first communication terminal transmits the moving image information (a) and the transmission data (b) to (f). Will be described as an example of collectively transmitting to other communication terminals.
- the case where moving image information and hand-drawn images are transmitted from the first mobile phone 100A to the second mobile phone 100B will be described.
- CPU 106 of first mobile phone 100A acquires data related to chat communication from chat server 400 via communication device 101 (step S102).
- CPU 106 of second mobile phone 100B (reception side) also acquires data related to chat communication from chat server 400 via communication device 101 (step S104).
- data related to chat communication includes chat room ID, member terminal information, notification (notification information), chat contents up to this point, and the like.
- the CPU 106 of the first mobile phone 100A displays a chat communication window on the touch panel 102 (step S106).
- CPU 106 of second mobile phone 100B displays a chat communication window on touch panel 102 (step S108).
- the CPU 106 of the first mobile phone 100A receives the moving image content via the communication device 101 based on the content reproduction command from the user (step S110). More specifically, the CPU 106 receives a command for designating moving image content from the user via the touch panel 102. The user may directly input the URL to the first mobile phone 100A, or may select a link corresponding to the desired moving image content on the displayed web page.
- the CPU 106 of the first mobile phone 100A starts playing the received moving image content via the touch panel 102 (step S112).
- the CPU 106 may output the sound of the moving image content via the speaker 109.
- the CPU 106 accepts a hand-drawn input by the user via the touch panel 102 while the first mobile phone 100A is reproducing the moving image content (step S114).
- the CPU 106 acquires the change (trajectory) of the contact position with respect to the touch panel 102 by sequentially receiving the contact coordinate data from the touch panel 102 every predetermined time. Then, as shown in FIG. 11, the CPU 106 performs hand-drawn clear information (b), information (c) indicating the locus of the contact position, information (d) indicating the line color, and information indicating the line width ( e) and transmission data including timing information (f) indicating the timing of hand-drawn input are created (step S116).
- the hand-drawn clear information (b) includes information (true) for clearing the hand-drawn input input so far or information (false) for continuing the hand-drawn input.
- the timing information (f) is also information indicating the timing at which hand-drawn should be drawn. More specifically, the timing information (f) includes, for example, the time (ms) from the start of moving image content when the first mobile phone 100A accepts hand-drawn input, information indicating the scene of the moving image content, Contains information indicating the frame.
- step S114 the CPU 106 causes the touch panel 102 to display the hand-drawn image input on the moving image content (superimposed on the moving image content) based on the transmission data. As shown in FIGS. 4B to 4D, the CPU 106 displays the hand-drawn image on the touch panel 102 in response to the input of the hand-drawn image.
- the CPU 106 may transmit clear information (true) using the communication device 101 when the scene is switched.
- the CPU 106 repeats the processing from step S114 to step S116 every time it accepts hand-drawn input. And as shown in FIG.4 (f), CPU106 complete
- the CPU 106 uses the communication device 101 to transmit the moving image information (a) and all the transmission data (b) to (f) already created via the chat server 400 to the other participating in the chat. It transmits to a communication terminal (step S120).
- the moving image information (a) includes, for example, a URL indicating a storage location of the moving image.
- the CPU 106 uses the communication device 101 to transmit the moving image information (a) and all of the transmission data (b) to (f) already created by P2P communication to other communication participating in the chat. It transmits directly to the terminal (step S120). In this case, the CPU 106 stores the moving image information (a) and all of the already created transmission data (b) to (f) in its own memory 103.
- the CPU 405 of the chat server 400 may leave the moving image information (a) and the transmission data (b) to (f) in the memory 406 for a communication terminal that participates in chat later.
- the second mobile phone 100B is participating in the chat.
- the CPU 106 of the second mobile phone 100B receives the video information (a) and the transmission data (b) to (f) from the chat server 400 via the communication device 101 (step S122).
- the CPU 106 analyzes the moving image information (a) and the transmission data (b) to (f) (step S124).
- CPU 106 downloads moving image content from content server 600 (step S126).
- the CPU 106 starts playing the received moving image content via the touch panel 102 (step S128). At this time, the CPU 106 may output the sound of the moving image content via the speaker 109.
- the CPU 106 causes the touch panel 102 to draw a hand-drawn image on the moving image content on the basis of the timing information (f) of the transmission data (step S130).
- CPU 106 may erase the hand-drawn image based on the clear information from first mobile phone 100A. Alternatively, the CPU 106 may erase the hand-drawn image by determining by itself that the scene has been switched. Then, as shown in FIG. 4L, the CPU 106 ends the reproduction of the moving image content (step S132).
- the hand-drawn image is drawn on the second mobile phone 100B at the same timing as that in the moving image content in which the hand-drawn image is input on the first mobile phone 100A. That is, in the second mobile phone 100B, desired information is drawn in a scene desired by the user of the first mobile phone 100A.
- FIG. 13 is a flowchart showing a processing procedure of input processing in mobile phone 100 according to the present embodiment.
- CPU 106 first executes pen information setting processing (step S300) when input to mobile phone 100 is started.
- the pen information setting process (step S300) will be described later.
- step S300 the CPU 106 determines whether the data (b) is true (step S202). If data (b) is true (YES in step S202), CPU 106 stores data (b) in memory 103 (step S204). The CPU 106 ends the input process.
- step S202 determines whether or not the stylus pen 120 has touched the touch panel 102 (step S206). That is, the CPU 106 determines whether pen-down has been detected.
- CPU 106 determines whether or not the contact position of stylus pen 120 with respect to touch panel 102 has changed (step S208). That is, the CPU 106 determines whether or not pen drag has been detected. If CPU 106 does not detect a pen drag (NO in step S208), CPU 106 ends the input process.
- CPU 106 When CPU 106 detects a pen down (YES in step S206) or detects a pen drag (YES in step S208), CPU 106 sets “false” in data (b) ( Step S210). CPU 106 executes a hand-drawing process (step S400). The hand drawing process (step S400) will be described later.
- CPU 106 stores the data (b), (c), (d), (e), and (f) in the memory 103 after completing the hand-drawing process (step S400) (step S212).
- the CPU 106 ends the input process.
- FIG. 14 is a flowchart showing a processing procedure of pen information setting processing in mobile phone 100 according to the present embodiment.
- CPU 106 determines whether or not a command for clearing a hand-drawn image is received from the user via touch panel 102 (step S302).
- CPU 106 sets “true” in data (b) (step S304).
- the CPU 106 executes processing from step S308.
- CPU 106 sets “false” in the data (e) when no instruction for clearing the hand-drawn image is received from the user (NO in step S302) (step S306).
- CPU 106 determines whether or not a command for changing the color of the pen is received from the user via touch panel 102 (step S308). If CPU 106 does not receive an instruction to change the pen color from the user (NO in step S308), CPU 106 executes the processing from step S312.
- the CPU 106 When the CPU 106 receives a command for changing the pen color from the user (YES in step S308), the CPU 106 sets the changed pen color in the data (d) (step S310). The CPU 106 determines whether or not a command for changing the pen width is received from the user via the touch panel 102 (step S312). If CPU 106 has not received a command for changing the pen width from the user (NO in step S312), CPU 106 ends the pen information setting process.
- CPU 106 When CPU 106 receives a command for changing the pen width from the user (YES in step S312), CPU 106 sets the changed pen width in data (e) (step S314). The CPU 106 ends the pen information setting process.
- FIG. 15 is a flowchart showing a processing procedure of hand-drawing processing in mobile phone 100 according to the present embodiment.
- CPU 106 determines whether stylus pen 120 is currently in contact with touch panel 102 via touch panel 102 (step S402). If stylus pen 120 is not in contact with touch panel 102 (NO in step S402), CPU 106 ends the hand-drawing process.
- the CPU 106 refers to a clock (not shown) and acquires the elapsed time from the start of the moving image content (step S404).
- CPU 106 sets the time (period) from the start of moving image content to the start of hand-drawn input in data (f) (step S406).
- the CPU 106 may set information for specifying a scene or information for specifying a frame instead of the time (period) from the start of moving image content to the start of hand-drawn input. This is because if the scene can be specified, the intention of the person who has input the hand-drawn image is easily transmitted.
- the CPU 106 acquires the contact coordinates (X, Y) of the stylus pen 120 with respect to the touch panel 102 and the current time (T) via the touch panel 102 (step S408).
- the CPU 106 sets “X, Y, T” in the data (c) (step S410).
- CPU 106 determines whether or not a predetermined time has elapsed since the previous acquisition of coordinates (step S412). If the predetermined time has not elapsed (NO in step S412), CPU 106 repeats the processing from step S308.
- CPU 106 determines whether pen drag has been detected via touch panel 102 (step S414).
- CPU 106 executes the processing from step S420 when no pen drag is detected (NO in step S414).
- step S414 If the CPU 106 detects a pen drag (YES in step S414), the touch position coordinates (X, Y) of the stylus pen 120 with respect to the touch panel 102 and the current time (T) are determined via the touch panel 102. Is acquired (step S416). The CPU 106 adds “: X, Y, T” to the data (c) (step S418). CPU 106 determines whether or not a predetermined time has elapsed since the previous acquisition of contact coordinates (step S420). If the predetermined time has not elapsed (NO in step S420), CPU 106 passes the processing from step S420.
- CPU 106 determines whether pen-up has been detected via touch panel 102 when a predetermined time has elapsed (YES in step S420) (step S422). CPU 106 repeats the processing from step S414 when pen-up is not detected (NO in step S422).
- step S422 When the CPU 106 detects pen-up (when YES in step S422), the touch coordinate (X, Y) of the stylus pen with respect to the touch panel 102 at the time of pen-up and pen-up is detected via the touch panel 102. Time (T) is acquired (step S424). The CPU 106 adds “: X, Y, T” to the data (c) (step S426). The CPU 106 ends the hand drawing process.
- FIG. 16 is a flowchart showing a processing procedure of a modification of the input processing in mobile phone 100 according to the present embodiment.
- the input process of FIG. 13 described above relates to a process for transmitting clear information (true) only when a command for clearing a hand-drawn image is received.
- the input process shown in FIG. 16 to be described later relates to a process for transmitting clear information (true) when a command for clearing a hand-drawn image is received and when a scene of a moving image content is switched. It was.
- step S300 when the input to the mobile phone 100 is started, the CPU 106 executes the above-described pen information setting process (step S300).
- step S300 the CPU 106 determines whether or not the data (b) is true (step S252). If data (b) is true (YES in step S252), CPU 106 stores data (b) in memory 103 (step S254). The CPU 106 ends the input process.
- step S252 determines whether or not the stylus pen 120 has touched the touch panel 102 (step S256). That is, the CPU 106 determines whether pen-down has been detected.
- CPU 106 determines whether or not the contact position of stylus pen 120 with respect to touch panel 102 has changed (step S258). That is, the CPU 106 determines whether or not pen drag has been detected. If CPU 106 has not detected a pen drag (NO in step S258), CPU 106 ends the input process.
- CPU 106 sets “false” in data (b) when pen-down is detected (YES in step S256) or pen drag is detected (YES in step S258) ( Step S260).
- CPU 106 executes the above-described hand-drawing process (step S400).
- step S400 determines whether or not the scene has been switched (step S262). More specifically, the CPU 106 determines whether or not the scene when the handwriting input is started is different from the current scene. However, the CPU 106 may determine whether or not a predetermined time has elapsed since the pen-up, instead of determining whether or not the scene has been switched.
- step S264 the CPU 106 adds “:” to the data (c) (step S264).
- step S266 determines whether or not a predetermined time has elapsed since the previous hand-drawn process (step S266). If the predetermined time has not elapsed (NO in step S266), CPU 106 repeats the processing from step S266. If the predetermined time has elapsed (YES in step S266), CPU 106 repeats the processing from step S400.
- the CPU 106 stores the data (b), (c), (d), (e), and (f) in the memory 103 (step S268).
- the CPU 106 ends the input process.
- FIG. 17 is a flowchart showing a processing procedure of hand-drawn image display processing in mobile phone 100 according to the present embodiment.
- the receiving communication terminal draws a hand-drawn stroke at the same speed as the transmitting communication terminal.
- CPU 106 obtains timing information time (f) from data (transmission data) received from another communication terminal (step S512).
- the CPU 106 acquires the time (period) from the start of reproduction of the moving image content to the current time, that is, the reproduction time t of the moving image content (step S514).
- the CPU 106 executes the first drawing process (step S610).
- the first drawing process (step S610) will be described later.
- the CPU 106 ends the hand drawn image display process.
- FIG. 18 is a flowchart showing a processing procedure of first drawing processing in mobile phone 100 according to the present embodiment.
- CPU 106 substitutes 1 for variable i (step S612).
- the CPU 106 determines whether or not the time Ct (i + 1) has elapsed from the time t corresponding to the reproduction time t described above (step S614). When the time Ct (i + 1) has not elapsed since time t (NO in step S614), CPU 106 repeats the processing from step S614.
- CPU 106 uses touch panel 102 to use coordinates (Cxi, Cyi) and coordinates (Cx (i + 1), Cy (i + 1). )) Are connected by a line to draw a hand-drawn stroke (step S616).
- CPU 106 increments variable i (step S618).
- CPU 106 determines whether or not the variable i is greater than or equal to the number n (step S620). CPU 106 repeats the processing from step S614 when variable i is less than the number n (NO in step S620). CPU 106 ends the first drawing process when variable i is greater than or equal to the number n (YES in step S620).
- FIG. 19 is an image diagram for explaining the hand-drawn image display process shown in FIGS. 17 and 18.
- the CPU 106 of the communication terminal (first communication terminal) to which a hand-drawn image is input every time a hand-drawn image is input (from pen-down to pen-up) or when a clear command is input, Alternatively, transmission data is created when a scene is switched. For example, when a scene is switched while a hand-drawn image is being input, transmission data indicating the hand-drawn image up to the time when the scene is switched is created.
- CPU 106 of the communication terminal (second communication terminal) that displays the hand-drawn image is based on timing information (f) and times (Ct1) to (Ct5) corresponding to the respective vertices. Then, hand-drawn strokes are drawn (Cx1, Cy1) to (Cx5, Cy5). That is, in this embodiment, the receiving communication terminal draws a hand-drawn stroke at the same speed as the transmitting communication terminal.
- FIG. 20 is a flowchart showing a processing procedure of a first modification of the hand-drawn image display processing in mobile phone 100 according to the present embodiment.
- the drawing time is shortened to reduce the drawing time.
- the image drawing can be completed. That is, a case where input of a hand-drawn image can be continued regardless of a scene change (a hand-drawn image is not cleared by a scene change) will be described.
- CPU 106 obtains timing information time (f) from the received transmission data (step S532).
- the CPU 106 acquires the reproduction time t of the moving image content (a period from the start point of the moving image content to the present) (step S534).
- the CPU 106 acquires the number n of coordinates of the vertices of the hand-drawn stroke (step S540).
- the CPU 106 refers to the moving image content and acquires the time T from the timing information time to the next scene change (step S542). CPU 106 determines whether time T is equal to or greater than time Ct ⁇ n between vertices (step S544).
- the CPU 106 executes the first drawing process (step S610) described above.
- the CPU 106 ends the hand drawn image display process. This case corresponds to a case where clear information is input before a scene change, or a case where a predetermined time has elapsed since pen-up before a scene change.
- step S630 If the time T is less than the time Ct ⁇ n between vertices (NO in step S544), the CPU 106 executes a second drawing process (step S630).
- the second drawing process (step S630) will be described later.
- the CPU 106 ends the hand drawn image display process. In this case, a case where a scene change occurs while inputting a hand-drawn image is applicable.
- FIG. 21 is a flowchart showing the procedure of the second drawing process in mobile phone 100 according to the present embodiment. As described above, a case where a scene change occurs during input of a hand-drawn image will be described.
- CPU 106 substitutes T / n for variable dt (step S632).
- the variable dt is the time between vertices at the time of drawing, and is smaller than the time Ct between vertices at the time of input.
- CPU 106 substitutes 1 for variable i (step S634).
- CPU 106 determines whether time dt ⁇ i has elapsed from time t (step S636). If time dt ⁇ i has not elapsed since time t (NO in step S636), CPU 106 repeats the processing from step S636.
- CPU 106 uses touch panel 102 to use coordinates (Cxi, Cyi) and coordinates (Cx (i + 1), Cy (i + 1)). ) Are connected by a line to draw a hand-drawn stroke (step S638).
- CPU 106 increments variable i (step S640).
- CPU 106 determines whether or not the variable i is greater than or equal to the number n (step S642). CPU 106 repeats the processing from step S636 when variable i is less than the number n (NO in step S642). If the variable i is greater than or equal to the number n (YES in step S642), the CPU 106 ends the second drawing process.
- FIG. 22 is an image diagram for explaining the hand-drawn image display processing shown in FIGS. 20 and 21.
- the CPU 106 of the communication terminal (first communication terminal) to which a hand-drawn image is input every time a hand-drawn image is input (from pen-down to pen-up), or a clear command is issued. Create transmission data when entered.
- CPU 106 of the communication terminal that displays a hand-drawn image draws a hand-drawn stroke based on timing information (f) and time dt corresponding to the vertexes. (Cx1, Cy1) to (Cx5, Cy5). That is, the communication terminal according to the present modification can reduce the drawing time to the scene change when the time required for inputting the hand-drawn image is longer than the time from the start of input of the hand-drawn image to the next scene change.
- the drawing of hand-drawn images can be completed. That is, even when the user on the transmission side inputs a hand-drawn image across scenes, the communication terminal on the reception side completes drawing the hand-drawn image in the scene desired by the user on the transmission side. Can do.
- FIG. 23 is a flowchart showing a processing procedure of a second modification of the hand-drawn image display processing in mobile phone 100 according to the present embodiment.
- the communication terminal according to the present modification draws the hand-drawn image over the entire scene including the input start time of the hand-drawn image.
- CPU 106 refers to the moving image content, and acquires periods (lengths) T1 to Tm from the start of reproduction of the moving image content to each scene change (step S552). That is, the CPU 106 acquires the time from the start of reproduction of the moving image content to the end of each scene. The CPU 106 acquires timing information time (f) from the received transmission data (step S554).
- the CPU 106 acquires a time Ti from the start of reproduction of the moving image content to a scene change immediately before the timing information time (step S556). That is, the scene corresponding to the timing information time is specified, and the length Ti from the start of reproduction of the moving image content to the end time of the scene immediately before the scene is acquired.
- CPU 106 obtains the playback time t of video content (period from the start of video content to the present) (step S558).
- the CPU 106 acquires the number n of coordinates of the vertices of the hand-drawn stroke (step S564).
- step S650 The third drawing process (step S650) will be described later.
- the CPU 106 ends the hand drawn image display process.
- FIG. 24 is a flowchart showing the procedure of the third drawing process in mobile phone 100 according to the present embodiment.
- CPU 106 substitutes (T (i + 1) ⁇ Ti) / n for variable dt (step S652).
- the variable dt is a value obtained by dividing the scene in which the hand-drawn image is input by the number of vertices.
- the CPU 106 substitutes 1 for the variable i (step S654).
- CPU 106 determines whether or not dt ⁇ i has elapsed since the reproduction time (time t) (step S656). If time dt ⁇ i has not elapsed since time t (NO in step S656), CPU 106 repeats the processing from step S656.
- CPU 106 uses touch panel 102 to use coordinates (Cxi, Cyi) and coordinates (Cx (i + 1), Cy (i + 1)). ) Are connected by a line to draw a hand-drawn stroke (step S658).
- CPU 106 increments variable i (step S660).
- the CPU 106 determines whether or not the variable i is greater than or equal to the number n (step S662). CPU 106 repeats the processing from step S656 when variable i is less than the number n (when NO in step S662). If the variable i is greater than or equal to the number n (YES in step S662), the CPU 106 ends the third drawing process.
- FIG. 25 is an image diagram for explaining the hand-drawn image display processing shown in FIGS. 23 and 24.
- the CPU 106 of the communication terminal (first communication terminal) to which a hand-drawn image is input every time a hand-drawn image is input (from pen-down to pen-up) or when a clear command is input, Create transmission data.
- CPU 106 of the communication terminal that displays a hand-drawn image draws a hand-drawn stroke based on timing information (f) and time dt corresponding to the vertexes. (Cx1, Cy1) to (Cx5, Cy5). That is, the communication terminal according to the present modification makes the input speed of the hand-drawn image as slow as possible according to the length of the scene corresponding to the hand-drawn image. The communication terminal can complete the drawing of the hand-drawn image before the scene change.
- the communication terminal on the receiving side completes drawing the hand-drawn image in the scene desired by the user on the sending side with a margin. can do.
- the receiving communication terminal draws the hand-drawn image from a timing earlier than the input start time of the hand-drawn image at the transmitting communication terminal, that is, from the start time of the scene to which the input start time of the hand-drawn image belongs. Will start.
- the communication terminals (first mobile phone 100A, second mobile phone 100B, third mobile phone 100C, and fourth mobile phone 100D) have different timings. It was intended to play video content.
- the user's intention to transmit (input) information is notified to the user who receives (views) the information by each communication terminal simultaneously starting reproduction of the moving image content. It tells effectively.
- the same number is attached
- FIG. Their functions are the same. Therefore, description of those components will not be repeated.
- the overall configuration of the network system 1 according to the present embodiment, the overall operation overview of the network system 1, the hardware configurations of the mobile phone 100, the chat server 400, and the content server 600 are the same as those of the first embodiment. Since it is the same, description is not repeated here.
- FIG. 26 is a flowchart showing a processing procedure of P2P communication processing in mobile phone 100 according to the present embodiment.
- FIG. 27 is an image diagram showing a data structure of transmission data according to the present embodiment.
- first mobile phone 100A transmits a hand-drawn image to the second mobile phone 100B
- first mobile phone 100A and second mobile phone 100B transmit and receive data via chat server 400.
- data may be transmitted / received by P2P communication without using the chat server 400.
- first mobile phone 100A needs to store data or transmit data to second mobile phone 100B, third mobile phone 100C, or the like. .
- CPU 106 of first mobile phone 100A acquires data related to chat communication from chat server 400 via communication device 101 (step S702).
- CPU 106 of second mobile phone 100B (reception side) also acquires data related to chat communication from chat server 400 via communication device 101 (step S704).
- data related to chat communication includes chat room ID, member terminal information, notification (notification information), chat contents up to this point, and the like.
- the CPU 106 of the first mobile phone 100A displays a chat communication window on the touch panel 102 (step S706).
- CPU 106 of second mobile phone 100B displays a chat communication window on touch panel 102 (step S708).
- the CPU 106 of the first mobile phone 100A receives the moving image content via the communication device 101 based on the content reproduction command from the user (step S710). More specifically, the CPU 106 receives a command for designating moving image content from the user via the touch panel 102. The user may directly input the URL to the first mobile phone 100A, or may select a link corresponding to the desired moving image content on the displayed web page.
- the CPU 106 of the first mobile phone 100A uses the communication device 101 to obtain the video information (a) for specifying the selected video content via the chat server 400 and other communications participating in the chat. It transmits to a terminal (step S712).
- the moving image information (a) includes, for example, a URL indicating the storage location of the moving image.
- the CPU 405 of the chat server 400 stores the moving image information (a) in the memory 406 for a communication terminal that participates in the chat later.
- the CPU 106 of the second mobile phone 100B receives the moving image information (a) from the chat server 400 via the communication device 101 (step S714).
- the CPU 106 analyzes the moving image information (step S716) and downloads the moving image content from the content server 600 (step S718).
- CPU 106 transmits a message indicating that preparation for reproduction of moving image content is completed to first mobile phone 100A via communication device 101 (step S720).
- CPU 106 of first mobile phone 100A receives the message from second mobile phone 100B via communication device 101 (step S722).
- the CPU 106 of the first mobile phone 100A starts playing the received moving image content via the touch panel 102 (step S724).
- the CPU 106 may output the sound of the moving image content via the speaker 109.
- CPU 106 of second mobile phone 100B starts to play the received moving image content via touch panel 102 (step S726). At this time, the CPU 106 may output the sound of the moving image content via the speaker 109.
- the CPU 106 receives a hand-drawn input by the user via the touch panel 102 while the first mobile phone 100A is reproducing the moving image content (step S728).
- the CPU 106 acquires the change (trajectory) of the contact position with respect to the touch panel 102 by sequentially receiving the contact coordinate data from the touch panel 102 every predetermined time. At this time, that is, in step S728, the CPU 106 causes the touch panel 102 to display the hand-drawn image input on the moving image content (superimposed on the moving image content). The CPU 106 displays the hand-drawn image on the touch panel 102 in response to the input of the hand-drawn image.
- the CPU 106 displays the hand-drawn clear information (b), the information (c) indicating the locus of the contact position, the information (d) indicating the line color, and the information indicating the line width (
- the transmission data including e) is created (step S730).
- the hand-drawn clear information (b) includes information (true) for clearing hand-drawn input that has been input so far, or information (false) for continuing hand-drawn input.
- the information (c) indicating the trajectory of the contact position includes the coordinates of each vertex constituting the hand-drawn stroke and the elapsed time from the start of hand-drawn input corresponding to each vertex.
- the CPU 106 of the first mobile phone 100A transmits transmission data to the second mobile phone 100B via the chat server 400 using the communication device 101 (step S732).
- CPU 106 of second mobile phone 100B receives the transmission data from first mobile phone 100A via communication device 101 (step S734).
- the CPU 106 of the second mobile phone 100B analyzes the transmission data (step S736).
- the CPU 106 of the second mobile phone 100B displays a hand-drawn image on the touch panel 102 based on the analysis result (step S738).
- the CPU 106 may transmit clear information (true) using the communication device 101 when the scene is switched. Then, CPU 106 of second mobile phone 100B may erase the hand-drawn image based on the clear information from first mobile phone 100A. Alternatively, the CPU 106 may erase the hand-drawn image by determining by itself that the scene has been switched.
- the CPU 106 of the first mobile phone 100A repeats the processing from step S728 to step S732 every time it accepts hand-drawn input.
- every time CPU 106 of second mobile phone 100B receives the transmission data it repeats the processing of steps S734 to S738.
- the CPU 106 of the first mobile phone 100A ends the reproduction of the moving image content (step S740).
- the CPU 106 of the second mobile phone 100B ends the reproduction of the moving image content (step S742).
- the hand-drawn image is drawn on the second mobile phone 100B at the same timing as that in the moving image content in which the hand-drawn image is input on the first mobile phone 100A. That is, in the second mobile phone 100B, desired information is drawn in a scene desired by the user of the first mobile phone 100A.
- FIG. 28 is a flowchart showing a processing procedure of input processing in mobile phone 100 according to the present embodiment.
- CPU 106 first executes the above-described pen information setting process (step S300) when input to mobile phone 100 is started.
- the pen information setting process (step S300) will be described later.
- step S300 the CPU 106 determines whether the data (b) is true (step S802). If data (b) is true (YES in step S802), CPU 106 stores data (b) in memory 103 (step S804). The CPU 106 ends the input process.
- step S802 determines whether or not the stylus pen 120 has touched the touch panel 102 (step S806). That is, the CPU 106 determines whether pen-down has been detected.
- CPU 106 determines whether the contact position of stylus pen 120 on touch panel 102 has changed (step S808). That is, the CPU 106 determines whether or not pen drag has been detected. If CPU 106 does not detect a pen drag (NO in step S808), CPU 106 ends the input process.
- CPU 106 sets “false” to data (b) when pen down is detected (YES in step S806) or when pen drag is detected (YES in step S808) (step S808). Step S810).
- CPU 106 executes a hand-drawing process (step S900). The hand-drawing process (step S900) will be described later.
- CPU 106 stores the data (b), (c), (d), and (e) in the memory 103 after completing the hand-drawing process (step S900) (step S812).
- the CPU 106 ends the input process.
- FIG. 29 is a flowchart showing a processing procedure of hand-drawing processing in mobile phone 100 according to the present embodiment.
- CPU 106 obtains contact coordinates (X, Y) of touch panel 102 of stylus pen 120 via touch panel 102 (step S902).
- the CPU 106 sets “X, Y” in the data (c) (step S904).
- CPU 106 determines whether or not a predetermined time has elapsed since the previous acquisition of coordinates (step S906). If the predetermined time has not elapsed (NO in step S906), CPU 106 repeats the processing from step S906.
- CPU 106 determines whether pen drag has been detected via touch panel 102 (step S908).
- CPU 106 determines whether pen-up has been detected via touch panel 102 when pen drag is not detected (NO in step S908) (step S910).
- CPU 106 repeats the processing from step S906 when pen-up is not detected (NO in step S910).
- CPU 106 When CPU 106 detects pen drag (YES in step S908) or detects pen-up (YES in step S910), CPU 106 touches touch panel 102 of stylus pen 120 via touch panel 102. Contact position coordinates (X, Y) are acquired (step S912). The CPU 106 adds “: X, Y” to the data (c) (step S914). The CPU 106 ends the hand drawing process.
- FIG. 30 is a flowchart showing a processing procedure of display processing in mobile phone 100 according to the present embodiment.
- CPU 106 determines whether or not the reproduction of the moving image content has ended (step S1002).
- CPU 106 ends the display process when the reproduction of the moving image content ends (YES in step S1002).
- step S1004 the CPU 106 obtains clear information clear (data (b)) when reproduction of moving image content has not ended (NO in step S1002) (step S1004).
- the CPU 106 determines whether or not the clear information clear is true (step S1006). If the clear information clear is true (YES in step S1006), the CPU 106 hides the hand-drawn image (step S1008). The CPU 106 ends the display process.
- step S1006 When the clear information clear is not true (NO in step S1006), the CPU 106 acquires the pen color (data (d)) (step S1010). The CPU 106 resets the pen color (step S1012). The CPU 106 acquires the pen width (data (e)) (step S1014). The CPU 106 resets the pen width (step S1016).
- CPU 106 executes hand-drawn image display processing (step S1100). The hand-drawn image display process (step S1100) will be described later. The CPU 106 ends the display process.
- FIG. 31 is a flowchart showing a processing procedure of an application example of display processing in mobile phone 100 according to the present embodiment.
- the mobile phone 100 erases (resets) not only the clear information but also the hand-drawn image displayed so far when the scene is switched.
- CPU 106 determines whether or not the reproduction of the moving image content has ended (step S1052).
- CPU 106 ends the display process when the reproduction of the moving image content ends (YES in step S1052).
- CPU 106 determines whether or not the scene of the video content has been switched (step S1054) when the playback of the video content has not ended (NO in step S1052). If the scene of the moving image content has not been switched (NO in step S1054), CPU 106 executes the processing from step S1058.
- step S1054 When the scene of the moving image content is switched (YES in step S1054), CPU 106 hides the hand-drawn image displayed so far (step S1056).
- the CPU 106 acquires clear information clear (data (b)) (step S1058).
- the CPU 106 determines whether or not the clear information clear is true (step S1060). If the clear information clear is true (YES in step S1060), the CPU 106 hides the hand-drawn image displayed so far (step S1062). The CPU 106 ends the display process.
- step S1060 the CPU 106 acquires the pen color (data (d)) (step S1064).
- the CPU 106 resets the pen color (step S1066).
- the CPU 106 acquires the pen width (data (e)) (step S1068).
- the CPU 106 resets the pen width (step S1070).
- CPU 106 executes hand-drawn image display processing (step S1100).
- the hand-drawn image display process (step S1100) will be described later.
- the CPU 106 ends the display process.
- FIG. 32 is a flowchart showing a processing procedure of hand-drawn image display processing in mobile phone 100 according to the present embodiment.
- the CPU 106 acquires the coordinates (data (c)) of the hand-drawn stroke vertex (step S1102).
- the CPU 106 acquires the latest two coordinates, that is, the coordinates (Cx1, Cy1) and the coordinates (Cx2, Cy2).
- the CPU 106 draws a hand-drawn stroke by connecting the coordinates (Cx1, Cy1) and the coordinates (Cx2, Cy2) with a line (step S1104).
- the CPU 106 ends the hand drawn image display process.
- the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.
- a storage medium for supplying the program code for example, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card (IC memory card), ROM (mask ROM, flash) EEPROM, etc.) can be used.
- the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.
- 1 network system 100, 100A, 100B, 100C, 100D mobile phone, 101 communication device, 102 touch panel, 103 memory, 103A work memory, 103B address book data, 103C own terminal data, 103D address data, 103E address data, 104 pen Tablet, 106 CPU, 107 display, 108 microphone, 109 speaker, 110 various buttons, 111 first notification unit, 112 second notification unit, 113 TV antenna, 120 stylus pen, 200 car navigation device, 250 vehicle, 300 personal Computer, 400 chat server, 406 memory, 406A room management table, 407 fixed disk, 408 internal bus 409 communication device, 500 Internet, 600 content server 606 memory, 607 a fixed disk, 608 internal bus, 609 communication device, 615 a fixed disk, 700 carrier network.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
- Telephonic Communication Services (AREA)
Abstract
Description
<ネットワークシステム1の全体構成>
まず、本実施の形態に係るネットワークシステム1の全体構成について説明する。図1は、本実施の形態に係るネットワークシステム1の一例を示す概略図である。図1に示すように、ネットワークシステム1は、携帯電話100A,100B,100C,100Dと、チャットサーバ(第1のサーバ装置)400と、コンテンツサーバ(第2のサーバ装置)600と、インターネット(第1のネットワーク)500と、キャリア網(第2のネットワーク)700とを含む。また、本実施の形態に係るネットワークシステム1は、車両250に搭載されるカーナビゲーション装置200と、パーソナルコンピュータ(PC;Personal Computer)300とを含む。
次に、本実施の形態に係るネットワークシステム1の動作概要について説明する。図2は、本実施の形態に係るネットワークシステム1における動作概要を示すシーケンス図である。以下では、説明のために、第1の携帯電話100Aと第2の携帯電話100Bとの間の通信処理の概要について説明する。
次に、動画コンテンツの再生中における手描き画像の入力と描画に関する動作概要についてさらに詳細に説明する。図4は、動画コンテンツの再生中における手描き画像の入力と描画に関する動作概要を示すイメージ図である。以下では、第1の携帯電話100Aと第2の携帯電話100Bとがチャット通信を開始し、その後第3の携帯電話100Cがチャット通信を開始し、その後第4の携帯電話100Dがチャット通信を開始する場合について説明する。
本実施の形態に係る携帯電話100のハードウェア構成について説明する。図5は、本施の形態に係る携帯電話100の外観を示すイメージ図である。図6は、本施の形態に係る携帯電話100のハードウェア構成を示すブロック図である。
次に、本実施の形態に係るチャットサーバ400およびコンテンツサーバ600のハードウェア構成について説明する。以下では、まず、チャットサーバ400のハードウェア構成について説明する。
次に、本実施の形態に係る携帯電話100におけるP2P通信処理について説明する。図10は、本実施の形態に係る携帯電話100におけるP2P通信処理の処理手順を示すフローチャートである。図11は、本実施の形態に係る送信データのデータ構造を示すイメージ図である。
次に、本実施の形態に係る携帯電話100におけるP2P通信処理の変形例について説明する。図12は、本実施の形態に係る携帯電話100におけるP2P通信処理の変形例の処理手順を示すフローチャートである。
次に、本実施の形態に係る携帯電話100における入力処理について説明する。図13は、本実施の形態に係る携帯電話100における入力処理の処理手順を示すフローチャートである。
次に、本実施の形態に係る携帯電話100におけるペン情報の設定処理について説明する。図14は、本実施の形態に係る携帯電話100におけるペン情報の設定処理の処理手順を示すフローチャートである。
次に、本実施の形態に係る携帯電話100における手描き処理について説明する。図15は、本実施の形態に係る携帯電話100における手描き処理の処理手順を示すフローチャートである。
次に、本実施の形態に係る携帯電話100における入力処理の変形例について説明する。図16は、本実施の形態に係る携帯電話100における入力処理の変形例の処理手順を示すフローチャートである。
次に、本実施の形態に係る携帯電話100における手描き画像表示処理について説明する。図17は、本実施の形態に係る携帯電話100における手描き画像表示処理の処理手順を示すフローチャートである。図17においては、受信側の通信端末が、送信側の通信端末と同じ速度で、手描きストロークを描画するものである。
次に、本実施の形態に係る携帯電話100における第1の描画処理について説明する。図18は、本実施の形態に係る携帯電話100における第1の描画処理の処理手順を示す
フローチャートである。
次に、本実施の形態に係る携帯電話100における手描き画像表示処理の第1の変形例について説明する。図20は、本実施の形態に係る携帯電話100における手描き画像表示処理の第1の変形例の処理手順を示すフローチャートである。
次に、本実施の形態に係る携帯電話100における第2の描画処理について説明する。図21は、本実施の形態に係る携帯電話100における第2の描画処理の処理手順を示すフローチャートである。前述したように、手描き画像の入力中に、シーンチェンジが発生した場合について説明する。
次に、本実施の形態に係る携帯電話100における手描き画像表示処理の第2の変形例について説明する。図23は、本実施の形態に係る携帯電話100における手描き画像表示処理の第2の変形例の処理手順を示すフローチャートである。本変形例に係る通信端末は、手描き画像を、手描き画像の入力開始時点が含まれるシーンの全部の時間をかけて描画する。
次に、本実施の形態に係る携帯電話100における第3の描画処理について説明する。図24は、本実施の形態に係る携帯電話100における第3の描画処理の処理手順を示すフローチャートである。
次に、本発明の実施の形態2について説明する。上述の実施の形態1に係るネットワークシステム1は、それぞれの通信端末(第1の携帯電話100A、第2の携帯電話100B、第3の携帯電話100C、第4の携帯電話100D)が、異なるタイミングで動画コンテンツを再生するものであった。一方、本実施の形態に係るネットワークシステム1は、それぞれの通信端末が同時に動画コンテンツの再生を開始することによって、情報を送信(入力)するユーザの意図を、情報を受信(閲覧)するユーザへと、有効に伝えるものである。
以下では、本実施の形態に係る携帯電話100におけるP2P通信処理について説明する。図26は、本実施の形態に係る携帯電話100におけるP2P通信処理の処理手順を示すフローチャートである。図27は、本実施の形態に係る送信データのデータ構造を示すイメージ図である。
次に、本実施の形態に係る携帯電話100における入力処理について説明する。図28は、本実施の形態に係る携帯電話100における入力処理の処理手順を示すフローチャートである。
次に、本実施の形態に係る携帯電話100における手描き処理について説明する。図29は、本実施の形態に係る携帯電話100における手描き処理の処理手順を示すフローチャートである。
次に、本実施の形態に係る携帯電話100における表示処理について説明する。図30は、本実施の形態に係る携帯電話100における表示処理の処理手順を示すフローチャートである。
次に、本実施の形態に係る携帯電話100における表示処理の応用例について説明する。図31は、本実施の形態に係る携帯電話100における表示処理の応用例の処理手順を示すフローチャートである。この応用例においては、携帯電話100は、クリア情報だけでなくシーンが切り換わった際にも、それまでに表示されている手描き画像を消去(リセット)するものである。
次に、本実施の形態に係る携帯電話100における手描き画像表示処理について説明する。図32は、本実施の形態に係る携帯電話100における手描き画像表示処理の処理手順を示すフローチャートである。
本発明は、システム或いは装置にプログラムを供給することによって達成される場合にも適用できることはいうまでもない。そして、本発明を達成するためのソフトウェアによって表されるプログラムを格納した記憶媒体を、システム或いは装置に供給し、そのシステム或いは装置のコンピュータ(又はCPUやMPU)が記憶媒体に格納されたプログラムコードを読出し実行することによっても、本発明の効果を享受することが可能となる。
Claims (8)
- 第1および第2の通信端末(100A,100B)とを備えるネットワークシステム(1)であって、
前記第1の通信端末は、
前記第2の通信端末と通信するための第1の通信デバイス(101)と、
動画コンテンツを表示するための第1のタッチパネル(102)と、
前記第1のタッチパネルを介して手描き画像の入力を受け付けるための第1のプロセッサ(106)とを含み、
前記第1のプロセッサは、前記動画コンテンツの表示中に入力された前記手描き画像と、前記動画コンテンツにおける前記手描き画像の入力開始時点を特定するための開始情報とを、前記第1の通信デバイスを介して前記第2の通信端末へと送信し、
前記第2の通信端末は、
前記動画コンテンツを表示するための第2のタッチパネルと、
前記第1の通信端末から前記手描き画像と前記開始情報とを受信するための第2の通信デバイスと、
前記開始情報に基づいて、前記第2のタッチパネルに、前記動画コンテンツにおける前記手描き画像の入力開始時点から当該手描き画像を表示させるための第2のプロセッサとを含む、ネットワークシステム。 - 前記ネットワークシステムは、前記動画コンテンツを配信するためのコンテンツサーバ(600)をさらに備え、
前記第1のプロセッサは、
ダウンロード命令に応じて前記コンテンツサーバから前記動画コンテンツを取得し、
取得した前記動画コンテンツを特定するための動画情報を、前記第1の通信デバイスを介して前記第2の通信端末へと送信し、
前記第2のプロセッサは、前記動画情報に基づいて、前記コンテンツサーバから前記動画コンテンツを取得する、請求項1に記載のネットワークシステム。 - 前記第1のプロセッサは、前記動画コンテンツのシーンが切り替わったときに、および/または、入力された前記手描き画像をクリアするための命令を受け付けたときに、前記第1の通信デバイスを介して、前記手描き画像を消去するための命令を前記第2の通信端末へと送信する、請求項1に記載のネットワークシステム。
- 前記第2のプロセッサは、
前記入力開始時点から前記動画コンテンツのシーンが切り替わる時点までの時間を計算し、
前記時間に基づいて、前記第2のタッチパネル上での前記手描き画像の描画速度を決定する、請求項1に記載のネットワークシステム。 - 前記第2のプロセッサは、
前記入力開始時点を含む前記動画コンテンツのシーンの長さを計算し、
前記長さに基づいて、前記第2のタッチパネル上での前記手描き画像の描画速度を決定する、請求項1に記載のネットワークシステム。 - 互いに通信可能な第1および第2の通信端末とを含むネットワークシステムにおける通信方法であって、
前記第1の通信端末が、動画コンテンツを表示するステップと、
前記第1の通信端末が、手描き画像の入力を受け付けるステップと、
前記第1の通信端末が、前記動画コンテンツの表示中に入力された前記手描き画像と、前記動画コンテンツにおける前記手描き画像の入力開始時点を特定するための開始情報とを、前記第2の通信端末へと送信するステップと、
前記第2の通信端末が、前記動画コンテンツを表示するステップと、
前記第2の通信端末が、前記第1の通信端末から前記手描き画像と前記開始情報とを受信するステップと、
前記第2の通信端末が、前記開始情報に基づいて、前記動画コンテンツにおける前記手描き画像の入力開始時点から当該手描き画像を表示するステップとを備える、通信方法。 - 他の通信端末と通信可能な通信端末であって、
前記他の通信端末と通信するための通信デバイスと、
動画コンテンツを表示するためのタッチパネルと、
前記タッチパネルを介して第1の手描き画像の入力を受け付けるためのプロセッサとを備え、
前記プロセッサは、前記動画コンテンツの表示中に入力された前記第1の手描き画像と、前記動画コンテンツにおける前記第1の手描き画像の入力開始時点を特定するための第1の開始情報とを、前記通信デバイスを介して前記他の通信端末へと送信し、
前記他の通信端末から第2の手描き画像と第2の開始情報とを受信し、
前記第2の開始情報に基づいて、前記タッチパネルに、前記動画コンテンツにおける前記第2の手描き画像の入力開始時点から当該第2の手描き画像を表示させる、通信端末。 - 通信デバイスとタッチパネルとプロセッサとを含む通信端末における通信方法であって、
前記プロセッサが、前記タッチパネルに動画コンテンツを表示させるステップと、
前記プロセッサが、前記タッチパネルを介して第1の手描き画像の入力を受け付けるステップと、
前記プロセッサが、前記動画コンテンツの表示中に入力された前記第1の手描き画像と、前記動画コンテンツにおける前記第1の手描き画像の入力開始時点を特定するための開始情報とを、前記通信デバイスを介して他の通信端末へと送信するステップと、
前記プロセッサが、前記通信デバイスを介して前記他の通信端末から第2の手描き画像と第2の開始情報とを受信するステップと、
前記プロセッサが、前記第2の開始情報に基づいて、前記タッチパネルに、前記動画コンテンツにおける前記第2の手描き画像の入力開始時点から当該第2の手描き画像を表示するステップとを備える、通信方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201180016698.0A CN102812446B (zh) | 2010-03-30 | 2011-03-08 | 网络***、通信方法以及通信终端 |
US13/638,022 US20130014022A1 (en) | 2010-03-30 | 2011-03-08 | Network system, communication method, and communication terminal |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2010077782A JP2011210052A (ja) | 2010-03-30 | 2010-03-30 | ネットワークシステム、通信方法、および通信端末 |
JP2010-077782 | 2010-03-30 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011122267A1 true WO2011122267A1 (ja) | 2011-10-06 |
Family
ID=44711993
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2011/055382 WO2011122267A1 (ja) | 2010-03-30 | 2011-03-08 | ネットワークシステム、通信方法、および通信端末 |
Country Status (4)
Country | Link |
---|---|
US (1) | US20130014022A1 (ja) |
JP (1) | JP2011210052A (ja) |
CN (1) | CN102812446B (ja) |
WO (1) | WO2011122267A1 (ja) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8594740B2 (en) | 2008-06-11 | 2013-11-26 | Pantech Co., Ltd. | Mobile communication terminal and data input method |
JP5226142B1 (ja) * | 2012-02-29 | 2013-07-03 | 株式会社東芝 | 表示制御装置、表示制御方法、電子機器、および電子機器の制御方法 |
JP5909459B2 (ja) * | 2013-05-02 | 2016-04-26 | グリー株式会社 | メッセージ送受信支援システム、メッセージ送受信支援プログラム及びメッセージ送受信支援方法 |
DE202015006141U1 (de) | 2014-09-02 | 2015-12-14 | Apple Inc. | Elektronische Touch-Kommunikation |
JP6948480B1 (ja) * | 2021-02-19 | 2021-10-13 | 一般社団法人組込みシステム技術協会 | ページサイトからチャットページを表示するプログラム、ユーザ端末、Webサーバ及び方法 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10124689A (ja) * | 1996-10-15 | 1998-05-15 | Nikon Corp | 画像記録再生装置 |
JP2003283981A (ja) * | 2002-03-20 | 2003-10-03 | Nippon Telegr & Teleph Corp <Ntt> | 映像コメント入力・表示方法およびシステムと、クライアント装置と、映像コメント入力・表示プログラム及びその記録媒体 |
JP2004118236A (ja) * | 2002-09-20 | 2004-04-15 | Ricoh Co Ltd | 画面データ管理装置、画面データ管理システム、画面データ管理方法、および画面データ管理プログラム |
JP2008252920A (ja) * | 2004-05-11 | 2008-10-16 | Matsushita Electric Ind Co Ltd | 再生装置 |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5600775A (en) * | 1994-08-26 | 1997-02-04 | Emotion, Inc. | Method and apparatus for annotating full motion video and other indexed data structures |
JPH08262965A (ja) * | 1995-03-20 | 1996-10-11 | Mitsubishi Electric Corp | 語学学習用ポーズ機能付きクローズドキャプションデコーダ |
US20020120925A1 (en) * | 2000-03-28 | 2002-08-29 | Logan James D. | Audio and video program recording, editing and playback systems using metadata |
US6006241A (en) * | 1997-03-14 | 1999-12-21 | Microsoft Corporation | Production of a video stream with synchronized annotations over a computer network |
US6584226B1 (en) * | 1997-03-14 | 2003-06-24 | Microsoft Corporation | Method and apparatus for implementing motion estimation in video compression |
US6442518B1 (en) * | 1999-07-14 | 2002-08-27 | Compaq Information Technologies Group, L.P. | Method for refining time alignments of closed captions |
US9026901B2 (en) * | 2003-06-20 | 2015-05-05 | International Business Machines Corporation | Viewing annotations across multiple applications |
CN100594527C (zh) * | 2004-11-22 | 2010-03-17 | 马里奥·皮尔基奥 | 多媒体播示中的使音频与图形同步的方法 |
KR20080096793A (ko) * | 2006-02-27 | 2008-11-03 | 교세라 가부시키가이샤 | 화상 정보 공유 시스템 |
JP5500334B2 (ja) * | 2007-01-22 | 2014-05-21 | ソニー株式会社 | 情報処理装置および方法、並びにプログラム |
US9390169B2 (en) * | 2008-06-28 | 2016-07-12 | Apple Inc. | Annotation of movies |
US20110107238A1 (en) * | 2009-10-29 | 2011-05-05 | Dong Liu | Network-Based Collaborated Telestration on Video, Images or Other Shared Visual Content |
US20110218965A1 (en) * | 2010-03-03 | 2011-09-08 | Htc Corporation | System for remotely erasing data, method, server, and mobile device thereof, and computer program product |
-
2010
- 2010-03-30 JP JP2010077782A patent/JP2011210052A/ja not_active Withdrawn
-
2011
- 2011-03-08 CN CN201180016698.0A patent/CN102812446B/zh not_active Expired - Fee Related
- 2011-03-08 WO PCT/JP2011/055382 patent/WO2011122267A1/ja active Application Filing
- 2011-03-08 US US13/638,022 patent/US20130014022A1/en not_active Abandoned
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10124689A (ja) * | 1996-10-15 | 1998-05-15 | Nikon Corp | 画像記録再生装置 |
JP2003283981A (ja) * | 2002-03-20 | 2003-10-03 | Nippon Telegr & Teleph Corp <Ntt> | 映像コメント入力・表示方法およびシステムと、クライアント装置と、映像コメント入力・表示プログラム及びその記録媒体 |
JP2004118236A (ja) * | 2002-09-20 | 2004-04-15 | Ricoh Co Ltd | 画面データ管理装置、画面データ管理システム、画面データ管理方法、および画面データ管理プログラム |
JP2008252920A (ja) * | 2004-05-11 | 2008-10-16 | Matsushita Electric Ind Co Ltd | 再生装置 |
Also Published As
Publication number | Publication date |
---|---|
JP2011210052A (ja) | 2011-10-20 |
CN102812446A (zh) | 2012-12-05 |
CN102812446B (zh) | 2016-01-20 |
US20130014022A1 (en) | 2013-01-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110917614B (zh) | 基于区块链***的云游戏***及云游戏的控制方法 | |
US11146695B2 (en) | Communication management system, communication system, communication control method, and recording medium | |
US7774505B2 (en) | Method for transmitting image data in real-time | |
JP5658547B2 (ja) | ネットワークシステム、通信方法、および通信端末 | |
WO2011122267A1 (ja) | ネットワークシステム、通信方法、および通信端末 | |
US20080252716A1 (en) | Communication Control Device and Communication Terminal | |
EP2320323A1 (en) | Communication terminal device, communication method, and communication program | |
WO2011132472A1 (ja) | 電子機器、表示方法、および表示プログラムを記憶するコンピュータ読取可能な記録媒体 | |
US20080254813A1 (en) | Control Device, Mobile Communication System, and Communication Terminal | |
JP5035852B2 (ja) | 通信端末、制御方法、および制御プログラム | |
CN110070617B (zh) | 数据同步方法、装置、硬件装置 | |
CN114443868A (zh) | 多媒体列表的生成方法、装置、存储介质和电子设备 | |
WO2011122266A1 (ja) | ネットワークシステム、通信方法、および通信端末 | |
KR100770892B1 (ko) | 실시간으로 이미지 데이터를 전송하는 방법 | |
JP5781275B2 (ja) | 電子機器、表示方法、および表示プログラム | |
JP5755843B2 (ja) | 電子機器、表示方法、および表示プログラム | |
JP6597299B2 (ja) | 共有端末、通信システム、通信方法、及びプログラム | |
JP2010183447A (ja) | 通信端末、通信方法、および通信プログラム | |
JP5523973B2 (ja) | ネットワークシステムおよび通信方法 | |
JP7476548B2 (ja) | 通信端末、通信システム、表示方法、及びプログラム | |
JP2010186400A (ja) | 通信端末、通信方法、および通信プログラム | |
JP6607034B2 (ja) | 共有端末、通信システム、通信方法、及びプログラム | |
JP5429780B2 (ja) | 通信端末、通信方法、および通信プログラム | |
CN116743851A (zh) | 程序运行方法、装置、设备、存储介质及计算机程序产品 | |
JP2011154606A (ja) | ネットワークシステム、通信方法、サーバ、および通信端末 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201180016698.0 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11762507 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13638022 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 9045/CHENP/2012 Country of ref document: IN |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 11762507 Country of ref document: EP Kind code of ref document: A1 |