AU2019101539A4 - Android-based image and text information feedback terminal system - Google Patents

Abstract

The present invention relates to the field of electronic information science technologies, and provides a design of an Android-based image and text information feedback terminal system, where the system runs in a mobile device of an Android 7.1.1 version system. The system integrates a QR code unit, a client floating button unit, a Socket communications unit, a server image and text display unit, and a server local gallery unit. A user can scan a QR code of a server through a client to obtain an IP address and a port number of the server to establish a Socket connection; and clicks a sign-in button on the client to perform sign-in on the server, clicks a floating button to capture a screen image in real time, and transmits the captured picture to the server. The server displays the image. In this design, communication between Android mobile terminal is implemented by using an existing local area network, without relying on a third-party server, so as to improve transmission security and reliability and resource utilization.

Description

2019101539 09 Dec 2019

ANDROID-BASED IMAGE AND TEXT INFORMATION FEEDBACK TERMINAL SYSTEM

TECHNICAL FIELD

The present invention provides an Android-based mobile terminal system, so as to implement transmission of image and text information from a plurality of clients to one client in a wireless local area network. The present invention belongs to the file of electronic information science technologies.

BACKGROUND

Content of this study mainly includes the application of an Internet technology in scenarios such as a conventional conference and classes. A conventional conference system implements unidirectional one-to-many information transmission, and consequently it is impossible for a speaker to know what listeners are thinking and an information reception status. In addition, a computer device has high maintenance costs and is not portable, and there is a network security risk when a conference is held by using a third-party server for communication involving important information. In this system, a mobile terminal based on Android 7.1.1 and a QR code technology are used to optimize implementation modes and means of scenarios such as a conventional conference and classes, and listeners can feed back their own ideas and the like to a speaker in a form of terminal screen-shot images. In addition, communication connection is established through QR code scanning with a client, without depending on a third-party server, so that the communication is more secure and reliable.

The Android system is a mobile terminal operating system developed based on Linux, and occupies most market shares of smart mobile terminals. The Android 7.1.1 system has characteristics such as a multiwindow mode, runtime permission, and an ART virtual machine, and has rich and powerful functions. A wireless local area network is based on the IEEE802.il standard, and can steadily and rapidly transmit data at the 2.4 GHz or 5 GHz radio frequency band, and therefore is widely applied.

SUMMARY

A core function of an Android-based image and text information feedback system is to implement multimedia communication from a plurality of clients running on Android 7.1.1 to one server only by using an existing local area network, without relying on a third-party server, so as to feed back image and text information of the clients to the server in a timely manner. A centralized conference model and a C/S (Client/Server) software system architecture is used, and

2019101539 09 Dec 2019 both the client and server use a mobile intelligent device of an Android 7.1.1 version. The Android device is based on a Qualcomm Snapdragon 625 processor, is provided with a random access memory of 4 GB, a display with a resolution of 1920*1200, and an 8-megapixel camera, and supports dual-band Wi-Fi. A specific implementation solution is as follows: a design of an Android-based image and text information feedback system includes a QR code unit, a client floating button unit, a Socket communications unit, a server image and text display unit, and a server local gallery unit.

The QR code unit generates a QR code image on a server by using an IP address of the server in a wireless local area network, and obtains information about the IP address and a port number of the server on the client by scanning the QR code, to prepare for Socket communication.

The client floating button unit can create a button to allow the button to float in a full screen and support a gesture operation such as dragging and clicking the button. The floating button is clicked to capture content currently displayed on the client and generate a Bitmap image.

The Socket communications unit establishes Socket communication after the client obtains the IP address and the port number of the server, and sends information such as a text and an image from the client to the server. After receiving an image file from the client, the server stores the image file into a storage.

The server image and text display unit displays a sign-in status of the client in a list form, displays communication transmission progress and a user name from the client through text display, and displays an image through an image display control and can view the image through amplification, shrinking, and translation.

The server local gallery unit can open and browse a local gallery through a button on the client, select an image, and display the image on a screen.

In a design scheme, the client-server QR code unit is implemented based on two parts: On the server, a core function relies on Zxing integrated in a Java core package, and is mainly implemented in GenerateQRCode and the like. A first step is to obtain the IP address of the server through a WifiManager instance, then encode the IP address to generate a black and white two-dimensional matrix, and generate a QR code image in a Bitmap format. On the client, a Google's Zxing open source library is integrated in a project, and a rear-facing camera is enabled through Intent to start a QR code scanning interface of CaptureActivity.

In a design scheme, the client floating button module can float in a full screen by obtaining highest permission of SYSTEM ALERT WINDOW of WindowManager. The floating button is

2019101539 09 Dec 2019 configured by using parameters WindowManager.LayoutParams. Support on GesturDetector is added, so that the button can monitor and recognize a gesture operation. virtualDispay is used to obtain screen information to capture an image, and the image is converted into a Bitmap format and is stored in an external storage in a png format.

In a design scheme, the server-client communications unit is implemented through Socket short connection. The server creates a ServerSocket object and waits for a connection request from the client by using an accept method. After obtaining the IP address and the port number of the server, the client sends a Socket connection request to the server to establish a Socket communication pair. The Socket communications unit mainly implements transmission of text information and an image file. After successfully receiving an image file from the client, the server stores the image file to a specified folder of an external storage through a file I/O stream.

In a design scheme, the server image and text display unit uses a List View control to display a sign-in status of the client, uses a TextView control to display Socket communication progress and a source of an image received by the server, uses Photo View as an image display control, and implements image loading through a Glide image loading frame. In this design, updating of a UI interface of a main thread in a communication thread is implemented through a Handler mechanism.

In a design scheme, a local gallery of the server local gallery unit is an Intent object that is obtained by creating ACTION and setting ACTION as ACTION PICK and that is stored in MediaStore, and the Intent object is enabled to implement opening and browsing functions. In addition, a MediaScannerConnectionClient object is used to establish a connection between the object and the gallery, and the gallery is scanned by calling an onMediaScannerConnected method, to implement real-time update of the gallery.

According to the foregoing design schemes, beneficial effects of the present invention are as follows:

A core function of an Android-based image and text information feedback system is to implement multimedia communication from a plurality of clients to one server only is implemented by using an existing local area network, without relying on a third-party server, so as to feed back screen-shot image information of the clients to the server in a timely manner. This effectively avoids a redundant operation in communication conducted by logging into the third party server, and greatly reduces network risks such as data leakage and a data loss, and makes the communication more stable and fast. In addition, an Android device used in this design is based on Qualcomm Snapdragon 625 low-end microprocessor and 4GB operation, has low costs,

2019101539 09 Dec 2019 and is easy to carry, which can be used in a wider range of scenarios while reducing system costs and has relatively high practical significance.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic structural diagram of a design of an Android-based image and text information feedback system;

FIG. 2 shows an interface displaying server QR code generation and client sign-in statistics;

FIG. 3a shows a QR code scanning interface of a client;

FIG. 3b shows an interface displayed after a client scanning result is returned;

FIG. 4 shows an image file transfer protocol customized in a Socket communications unit;

FIG. 5a shows an interface in which a client screen-shot button floating in another application;

FIG. 5b shows a display interface after an image and text display unit of a server receives a screen-shot image of a client;

FIG. 5c is a demonstration diagram of amplifying, shrinking, and translating an image by a server;

FIG. 6 is an interface diagram of browsing a local gallery by a server;

FIG. 7 is a procedure block diagram of a server; and

FIG. 8 is a procedure block diagram of a client.

DETAILED DESCRIPTION

The following further explains and describes the invention content with reference to accompanying diagrams and some examples, but purposes and objectives of these examples and accompanying diagrams are only to illustrate the present invention, and do not constitute any form of limitation to the actual protection scope of the present invention and make the protection scope of protection of the present invention limited thereto.

FIG. 1 shows a design of an Android-based image and text information feedback system, including a QR code unit, a client floating button unit, a Socket communications unit, a server image and text display unit, and a server local gallery unit.

The QR code unit is configured to scan, on a client, a QR code image generated by a server

2019101539 09 Dec 2019 and including an IP address allocated to the server in a wireless local area network, to obtain information about the IP address.

The floating button unit can float in a full screen and supports operations such as dragging and clicking through gestures. By clicking the floating button, display of a current screen can be captured.

The Socket communications unit can allow the client to send text information and an image field to the server. The text information mainly includes a user name and an IP address of the client to implement a sign-in function and display of a source of an image file, and the image file is a screen shot image of the client. The server stores the image file to an external storage after receiving the image.

The server image and text display unit mainly includes three parts: a name list of client users that has signed in, image transmission progress, an image display area. The image display area is used to display a screen shot image from the client and images in a local gallery.

The server local gallery unit may be configured to: after clicking a server button is conducted on the client to open and browse a local gallery and select an image, display the image on a screen. In addition, the gallery is proactively updated upon receiving a screen shot image from the client.

FIG. 2 shows an interface displaying server QR code generation and client sign-in statistics. The server uses a display screen with a resolution of 1920*1200; a UI interface of the display screen is divided into two sections: a left section and a right section; two buttons are arranged at an upper left part for registering a QR code generation function and an interface jump function respectively; a Text View control is configured to display a number of sign-in persons; a List View control is arranged at a left lower location of the UI interface to display a user name of a specific client in a form of a slide list; and an ImageView control is arranged on right side of the UI interface to display a generated QR code image.

A sign-in function is implemented by sending a user name and an IP address of a client to the server. A HashMap is first created in the server; the user name is placed on the Adapter and is also used a HashMap value while being displayed on List View. The IP address of the client is used as a HashMap button as a unique identifier of the client. Whether an IP address of a new client that has signed in exists in the HashMap button is determined to ensure that each client can only sign in once.

FIG. 3a shows a QR code scanning interface of a client, a button is controlled to start an

2019101539 09 Dec 2019

8-megapixel rear-facing camera for a scanning operation. An open source Zxing library is integrated into a project. CaptureActivity is an API provided by Zxing to a user for calling a scanning interface, and integrates a series of processes such as initializing the camera, decoding, and returning data. First, in AndroidManifest, camera permission is registered and dynamical application of runtime permission is conducted. After a current interface jumps to CaptureActivity through Intent, the rear-facing camera can be started for starting the scanning interface.

FIG. 3b shows an interface displayed after a client scanning result is returned, and it can be seen that, the client obtains an IP address of a server and a port number of Socket communication. A scanning result and a data result called back in CaptureActivity are transmitted to original activity through parameter data of onActivityResult, and the IP address and the port number obtained through scanning and parsing are divided into two String objects: the IP address and the port number.

FIG. 4 shows an image file transfer protocol customized in a Socket communications unit. A system designs a scheme in which transmission of a whole image file is completed in one Socket connection, and sets a simple protocol for cooperating a server and a client in transmitting an image file. The protocol stipulates that transmission content of one Socket connection is artificially divided into three parts:

A first part: a filename length is received by using DatalnputStream. After receiving InputStream from Socket of a client, a server first packages InputStream into DatalnputStream, and reads the first piece of header data through a readlnt method, and converts the header data to Int type data. In this way, a filename length of an image file is obtained.

A second part: a filename is received by first establishing a byte array with a same length as the filename and reading bytes from the second byte to the (2+filename length)^th byte of a whole input stream through a read method of InputStream, where this part of content corresponds to the name of the image file. Then, the byte array is converted into String, so as to obtain the required filename.

A third part: during image reception and display, a buffer byte array with a length of 1024 is created, and then data in InputStream is continually read by using the array as a unit until all data has been read.

FIG. 5a shows an interface in which a client screen-shot button floats in another application. SYSTEM ALERT WINDOW of WindowManager is set as a highest priority and is made to

2019101539 09 Dec 2019 float in a screen. In addition, because a floating button unit is implemented at a Service component, the floating button unit can still exist in an interface of another application.

FIG. 5b shows a display interface after an image and text display unit of a server receives a screen-shot image of a client. A Text View control displays a receiving progress and a file source of an image file (in a form of user names). Content is changed through an upDateUI method, and a UI interface is changed by using a Handler message mechanism.

FIG. 5c is a demonstration diagram of amplifying, shrinking, and translating an image by a server. An image display area of Photo View is a main bearer of image information, and ImageView controls enhancement of functions such as amplifying, shrinking, and smooth movement of an image through gesture operations. The image is loaded by using a Glide image loading frame.

FIG. 6 is an interface diagram of browsing a local gallery by a server. Opening of the local gallery is implemented through Intent. An Intent object is created and ACTION thereof is set as ACTION PICK, and stored in MediaStore, and then Intent is started. A type of a returned result of MidiaStore is Uri of an image file. This result is converted into a Bitmap image in an onActivityResult callback method; and the Bitmap image is display on a Photo View control. Each time after an image is received from a client and stored locally, a gallery needs to be manually updated.

FIG. 7 is a procedure block diagram of a server. It can be learned from the figure that, a necessary condition for establishing a Socket connection is that a client obtains an IP address and a port number of the server. The server first obtains an IP address of the server in a WLAN, integrates content of the IP address to generate an image including these information, and displays the image on an ImageView control. In addition, the server creates a ServerSocket object to wait for sign-in information of the client, and uses button non-repeatability of HashMap to set that one client can only log in once The server displays, in a ListView list form, a user name of the client that has successfully logged in, and displays, through TextView, the counted number of clients that have signed in.

After a conference start button is clicked, the server enters a conference interface. The server receives an images and text information from each client. Information such as a user name as a label of a client source is displayed through TextView; the received image is displayed on a Photo View control by using Glide image loading frame information; and the image is further stored in external storage space.

2019101539 09 Dec 2019

After a button of a local gallery is clicked, the local gallery can be opened and browsed, and after an image is selected, the image can be displayed on the Photo View control. Each time a new image is received from the client, the server updates the local gallery proactively.

FIG. 8 is a procedure block diagram of a client. It can be learned from the figure that, the client is a main producer of data and information and returns the data and information to a server through an internal network. After a scanning button is clicked, the client enters a QR code scanning interface, and can parse a scanned QR code image of the server into corresponding IP address information and a corresponding port number for establishing a Socket connection. After establishing the connection, the client sends its user name and IP address as sign-in information to the server. After a current interface jumps to a conference interface, a floating button can be created by clicking a floating button generation button, and the floating button can be deleted by clicking a delete button. The floating button mainly implements three functions:

First, SYSTEM_ALERT_WINDOW of WindowManager is set as a highest priority, so that SYSTEM ALERT WINDOW can float in a full screen. In addition, because the floating button is created and runs in a Service, it can still float in another application after an application exits and is at the forefront of the screen all the time.

Second, the floating button can be dragged on the screen: it can be used to recognize gesture operations such as onDown, onSingleTapUp, and onScroll in GesturDetector instances and process a corresponding logical response.

Finally, the floating button can be used for screen shot: a current screen can be captured by clicking the floating button. To obtain a full and complete picture, a floating button setVisibility(View.GONE) is set to disappearing at a capturing moment, a postDelayed method of a handler is called, a branch thread is created, to implement a screen shot operation in this thread. A screen shot image is obtained through a virtualDisplay display, and screenshot content is converted into a Bitmap image after screen shot is finished. The Bitmap image is stored in an external storage through file I/O, and is sent to the server through Socket communication.

The button is enabled to disappear, and is displayed on the screen after image capturing succeeds. After capturing is completed, the screenshot content is parsed into the Bitmap image for being subsequently transmitted to the server and downloaded locally for calling.

The following provides detailed description of the specific implementation in the present invention.

It should be understood that, purposes of these examples are only used to describe the present

2019101539 09 Dec 2019 invention and are not intended to limit the protection scope of the present invention. In addition, it should be also understood that, after reading technical content of the present invention, a person skilled in the art can make various modifications or variations, and all these equivalent forms shall also fall within the protection scope defined by the appended claims of this application.

Claims (5)

1. An Android-based image and text information feedback terminal, comprising a client-server QR code unit, a client floating button unit, a client-server communications unit, a server image and text display unit, and a server local gallery unit, wherein the client-server QR code unit is configured to generate a corresponding QR code image on a server by using character strings corresponding to an obtained local IP address and a Socket communication port number; and start a QR code scanning interface on the client through a rear-facing camera to scan a QR code on the server to obtain the IP address and the port number of the server; the floating button unit is configured to generate, in the client, a button that can float in a full screen and can support functions such as dragging and click event registration; clicking the floating button can capture display content of a current screen and generate a Bitmap image; the client-server communications unit implements a Socket connection with the server after the client obtains the IP address and the port number of the server, implements a function of data communication function between a plurality of clients and one server through multithreading, and can transmit content such as a text and an image file; and the local gallery unit is configured to: after clicking a button is conducted on the client to open and browse a local gallery and select an image, display the image on a screen; and a MediaStore of the server immediately updates the gallery upon receiving an image file from the client.

2. The Android-based image and text information feedback terminal according to claim 1, wherein the client-server QR code unit comprises a server IP address obtaining module, a QR code generation module, and a client QR code scanning module; the server-side IP address obtaining module obtains, through a relevant information Wifiinfo object of a local Wi-Fi connection, an IP address of the server in a WLAN connected to the server; the server QR code generation module is integrated to an Android project by using a core package Jar3.3.3 of Jar and encodes the IP address and the port number in a String format by using a Zxing service integrated by using a Java core package; an encoding result is converted into a two-dimensional array to represent pixel information, the pixel two-dimensional array is used to create the Bitmap image of the QR code comprising information about the IP address and the port number; and the Bitmap image is displayed through an ImageView control of the Android system; the client QR code scanning module uses the Google's Zxing library as a Module and integrates the Zxing library into the project, obtains permission of a camera and starts, through a rear-facing camera, a QR code scanning interface integrated in the Zxing library, and parses an obtained scanning result into character strings of the information about the IP address and the port number of the server; and the IP address and the port number scanned by the client is a basics of communication

2019101539 09 Dec 2019 between the client and the server.

3. The Android-based image and text information feedback terminal according to claim 2, wherein the client floating button unit comprises a floating button module and a screen shot module; the floating button module can float in a full screen by obtaining highest permission of SYSTEM ALERT WINDOW of WindowManager, and the unit uses an Android Service component as a carrier, so that the floating button can also exist in another program interface; a client hardware device is based on Android 7.1.1 and supports ten-point touch and a gesture operation on a screen; based on this, a dragging function is configured on the button, and the button body can be moved to a blank location when blocking key information; and after the floating button is clicked, the screen shot module is used by the system to capture image content displayed in a current screen, and the screen shot module is a core carrier for information transmission of the system, and the screen shot module converts a screen shot image into a Bitmap object, and transmits the Bitmap object to the server, and the server stores the Bitmap object to a local external storage.

4. The Android-based image and text information feedback terminal according to claim 3, wherein the server-client communications unit comprises a Socket file transmission module and a file storage module; the Socket module on the server is configured to: when the client obtains the IP address and the port number of the server through QR code scanning, establish multi-thread Socket communication connection with the server; Socket communication is mainly used to transmit content of a String text and a file through file I/O, and can implement sign-in of the client to the server and transmit a screen shot image file obtained by a screen shot unit; the file storage module is configured to: after the client successfully captures an image and the server successfully receives the image, store the image to an external storage area of the server through file I/O; and Android runtime permission is used to obtain external storage permission and establish a folder, and finally store the image file to an external storage folder through a customized file content transfer protocol, so as to implement backup of a communication record.

5. The Android-based image and text information feedback terminal according to claim 4, wherein the server display unit comprises a list display module, a text information display module, and an image display module, and the foregoing modules are displayed on a display screen of the server with a resolution of 1920*1200; the list display module of the server displays a user name of the client through a ListView control to collect sign-in statistics; the text information display module displays communication progress and received image source information through a Text View control; and the image display module uses a Photo View control

2019101539 09 Dec 2019 as a display carrier of image information, so as to obtain viewing functions of gesture amplification, shrinking, and translation; display loading of the image is implemented through a Glide image loading frame; and updating of content of the foregoing UI controls of a main thread in a branch communication thread is implemented through a Handler mechanism, avoiding cases such as system interface blocking and no response, preferably, wherein the local gallery unit comprises a gallery browsing module and a gallery update module; the gallery browsing module sets ACTION of an Intent object as ACTION PICK to obtain content Uri of a gallery, and the content is loaded through the Glide image loading frame and displayed on the display unit of the server, so as to be browsed and viewed by a user; and the gallery update module creates a MediaScannerConnectionClient object and implements gallery scanning and updating by connecting the MediaScannerConnectionClient object to MediaStore, to ensure that an image received from the client can be browsed and viewed through the gallery in real time.