WO2009112281A1

WO2009112281A1 - Garment integrated apparatus for online posture and body movement detection, analysis and feedback

Info

Publication number: WO2009112281A1
Application number: PCT/EP2009/001864
Authority: WO
Inventors: Holger Harms; Daniel Roggen; Tröster GERHARD
Original assignee: Eth Zurich
Priority date: 2008-03-14
Filing date: 2009-03-13
Publication date: 2009-09-17

Abstract

The invention is in the field of smart textiles for posture classification. It concerns a garment, in particular upper body garment, comprising an apparatus for detection of the orientation of at least one body segment integrated to the garment, wherein the apparatus comprises a plurality of sensing terminals and at least one processing unit in communication with the terminals, and wherein the garment is loose-fitting.

Description

GARMENT INTEGRATED APPARATUS FOR ONLINE POSTURE AND BODY MOVEMENT DETECTION, ANALYSIS

AND FEEDBACK

FIELD OF THE INVENTION

The invention is in the field of smart textiles for posture classification.

BACKGROUND OF THE INVENTION

Posture classification is an essential basis for activity recognition in various health- related applications. These include virtual assistants for movement rehabilitation to regain movement flexibility, or coaching support to maintain favorable upper body postures during daily activities. The common vision for virtual movement assistants is to empower the user with preventive coaching to reduce risk of hospitalization and chronic diseases and track rehabilitation progress.

On-body sensing of postures has grown to a first-choice solution for these applications, since the sensors are cheap and lightweight and do not require complex room setups. However, sensors and electronics need to be unobtrusive for a seamless integration to the body. The large amount of sensor data must be reduced in order to allow off-body transmissions, e.g. to a therapy computer or for user feedback. This typically requires an online recognition of relevant postures. Sensor and on-body processing must satisfy the stringent power and robustness constraints for daylong use of wearable systems. While different wearable sensing approaches have been proposed, few of the works actually present a full system solution for on-body sensing and recognition that fulfill these requirements.

Textile-based posture sensing solutions have been investigated for different sensors and target applications. Major research areas include the reconstruction of hand gestures with glove-attached accelerometers [1], conductive elastomers [2] or cameras [3, 4]. One pioneering project for upper body monitoring was the Georgia Tech Wearable Motherboard [5]. The system uses optical fibers to detect bullet wounds and other sensor modalities to monitor a soldier's vital conditions in combat conditions.

Several investigators attached sensors onto tight fitting clothes in order to determine postures of the upper body. Dunne et al. [6] developed a garment-integrated plastic optical fiber for monitoring seated spinal postures in one dimension. Tognetti et al.

[2] investigated a conductive elastomer, that shows piezoresistive properties when it is deformed. The material was applied in a special layout pattern to the shoulder, elbow and wrist region of a upper limb kinesthetic garment shirt. Thereon, the shirt was used by Giorgino et al. [7] for posture classification. Mattmann et al. [8] analyzed a novel elongation-sensitive yarn and classified upper body postures with a tight fitting suit called Backmanager.

The latter two approaches used textile attached or integrated strain sensors, relying upon the hypothesis, that different postures result in distinguishable elongation patterns. An integration of strain sensors fixes application to specific body regions. For the processing of sensor data, typically off-body computers are utilized. However, monitoring applications in rehabilitation and sports require free movement and on-body processing capabilities.

Junker et al. [9] implemented PadNET, a wearable data acquisition platform for the upper body. PadNET consists of a multistage sensor network with large-area sensors that was integrated into a jacket. Sensors were connected by woven wires and placed into pockets. A coupling between the sensors and the body segments is achieved by the jacket being rather stiff and the sensors having a large area covering a substantial part of the jacket's area. A crucial drawback is the bandwidth limitation of the system bus. The system proposed here, shrinks the area for garment attached sensing platforms by factor 12.5 compared to PadNET.

Gyroscopes and accelerometer-magnetometer pairs are well-established sensing solutions for posture tracking. The complementary nature of the sensors was exploited in academia and industry. Several approaches were made to determine the orientation of stand-alone sensor units using Kalman filtering [1 1, 12, 13]. All these academic investigations are not textile and rely on a defined number and location of sensors.

Moven [10] is a commercially tight fitting motion capturing system for short, but precise recordings. It consists of up to 16 body worn inertial motion sensing units and two masters, connected by a wireless link to a host PC. While these sensing solutions are very precise, the size and power consumption renders the devices infeasible for daylong recordings.

SUMMARY OF THE INVENTION It is thus an object of the invention to provide a posture sensing garment that avoids the problems of the state of the art. In particular, the garment shall be unobtrusive, wearable and user-friendly. A further object of the invention is to provide a method for measuring of the orientation of at least one body segment, preferably for detection of postures and/or body movements and/or activities, that enables daylong, unobtrusive recordings of subjects in everyday life situations.

This object is achieved by a garment comprising the features of claim 1 as well as by a method having the features of claim 13. Beneficial embodiments are described in the dependent claims, the description and the figures.

The garment according to the invention is in particular an upper body garment, e.g. a long-sleeve shirt, but could also be trousers or an all-in-one suit. It comprises an apparatus for measurement of body segment orientation which is integrated to the garment. This apparatus comprises a plurality of sensing terminals and at least one processing unit in communication with the terminals. The garment is loose-fitting and flexible. The garment substrate is made of a standard textile matter, e.g. knitted or woven fabrics or fleece.

The method for measurement of body segment orientation according to the invention comprises the steps of providing such a loose-fitting garment with an apparatus for measurement of body segment orientation integrated to the garment, and determining an orientation of at least one body segment by processing and analyzing the sensor signals in the processing unit. From the orientation information, postures and/or body movements and/or activities can be determined. Contrary to the posture sensing garments according to the prior art, where sensors are either attached to the skin or a tight fitting textile, in order to establish a direct coupling between sensor and skin, the present invention does not rely on direct coupling between sensor and the user's body. The invention is based on a loose fitting garment, which may be an everyday piece of clothing. The sensors are thus only loosely coupled to the user's body; variations in the exact positioning of the sensors with respect to the user's body occur. However, these uncertainties caused by the loose fit generally do not affect classification of several main postures. The loose fit may even make discrimination between postures easier, because a typical response of the garment can be taken into account, e.g. stretching of certain garment areas and thus displacement of corresponding sensors. For further increasing the accurateness of classification, several approaches can be made, e.g. using redundant sensors and/or knowledge about the system (model calculations, reference measurements).

According to the invention, a posture sensing garment is integrated into a loose fitting upper body garment, e.g. an everyday life garment. The garment is unobtrusive, wearable and user-friendly, and, hence, allows long term recordings in everyday life situations. That the garment is loose fitting means that the electronic components, in particular the sensors do not exert pressure to the skin and are not fixedly coupled to the body. The user is thus not hindered in its movements. This is especially helpful for motion/posture detection with children or in sports. Also, the invention may be used as a user interface for on-line control of a computer or similar device (e.g. playstation) by means of detection of certain movements or postures of the user.

Preferably, miniature sensors with dimensions less than 10 mm x 10 mm x 5 mm (e.g. having an area of 8x8mm² and a thickness of 5mm) are used. They are not obviously visible and do not affect the garment's properties and the garment's orientation. Presently, sensors with an dimensions of 8 mm x 10 mm x 5 mm are used, which exert a pressure of appr. 100 N/m². A reference for a maximum size/weight may be a typical button.

A loose-fitting garment in the context of the invention preferably means that a total area or certain dimensions of the garment of a predetermined size exceed a predetermined body area or predetermined body dimensions of a standard user having this size, by a predetermined value, e.g. by at least 10%. The predetermined body area is for example covered by the garment when worn. Predetermined garment or body dimensions can be arm length, chest length/width, wrist/neck circumference. A larger surface or larger dimensions of the garment mean that the garment is not tight fitting. There is thus enough elbowroom for the user. If the difference is not too large, e.g. less than 20-30%, there is still a sufficient spatial relation between the parts of the body and the sensors assigned thereto. An upper limit for the "looseness" of the fit may be defined by means of the deviations between the sensor signals of sensors applied to the garment (textile sensor) and reference sensors directly coupled to the associated body segment: For example, a difference in the detected angles of the gravitational vectors of 10-40°, e.g. 15°, may for certain applications still be tolerated while a larger deviation may not.

In an alternative definition, "loose-fitting" in the context of the invention preferably means that the garment does not exert compressive forces on the user's body when worn. The pressure exerted by the garment when worn does not exceed the pressure caused by the weight of the garment. The sensors and other electronic components are thus not pressed against the user's skin for the purpose of fixation.

The garment allows mobility of garment-attached sensing terminals - they are not specifically fixed to the body. There is thus only a loose coupling between the body segment which orientation is to be measured and the respective sensor on/in the garment. An upper limit for the tolerance between the position of the textile sensor and a reference position on the body segment may be 0.5-5 cm. It may also be determined depending on the result of a reference measurement as mentioned above. Contrary to state of the art systems that perform the processing of data in external hardware, the system according to the invention is preferably autonomous: Power supply, recording, processing and feedback hardware, software for measuring and analyzing sensor data, compensation of errors caused by loose fit, detection and/or classification of orientation of body segments, postures and activities, provision of feedback are integrated to the garment. These components are preferably also miniaturized. The proposed shirt does the sensing, classification and feedback- autonomously. An interface for communication with an external unit, e.g. a data processing unit, is preferably provided.

The apparatus is preferably able to provide an online feedback regarding recorded body orientation measurement, e.g. adopted postures. Postures trained by a user, e.g. in a physiotherapy or sports, can be classified online by the garment according to the invention. A feedback, e.g. an optical, vibrational, and/or acoustical signal, may be provided to the user by means of a signal unit in dependence of the adopted posture.

Preferably, the architecture of the data acquisition and processing apparatus is flexible. It can be chosen to meet the needs of a specific application. The garment according to the invention thus provides a certain flexibility of the measuring setup, i.e. number, type and positioning of the sensors. In particular sensors can generally be placed anywhere at the garment; preferably, their location is not fixed but can be adjusted to the purpose of the measurement. The sensors can preferably be attached or removed during runtime (plug 'n play capable). For error reduction, redundant sensors can be placed all over the garment, and a subset of sensors can be selected by feature ranking and multi-sensor fusion. The data acquisition and processing apparatus is integrated into the textile, preferably such that the electronic components and their connecting lines are fixedly connected to the garment. Preferably, they are encapsulated by an insulating and waterproof housing. For example silicon gel hot melted housings. The terminals and/or the power supply may be removably integrated, e.g. by providing receiving pockets.

Posture can be detected by sensors such as strain gauges, accelerometer, gyroscopes, magnetic field sensors, optical fibers, goniometer, pressure sensors, sensors for ultrasonic distance measurements and/or optical systems. Preferably, small and low- power inertial sensing units are used.

Orientation errors of sensors introduced by loose fitting clothing are preferably compensated inherently by a classifier and/or classifier fusion and/or multi-sensor fusion. For example, also generally known pattern recognition methods can be used to classify a posture or movement using a set of sensor signals or measured orientations.

Sensor orientation errors introduced by attaching terminals to the loose fitting garment can be compensated by the fusion of multiple sensing terminals. Sensor information can be fused either on signal and/or feature and/or recognition level.

On signal- or feature level one or a subset of many sensors is selected according their quality of information delivered. For example, only sensors that are less affected by garment-induced orientation errors, delivering an orientation measurement highly correlated to the actual body-segment orientation, are considered. A corresponding reference measurement with reference sensors attached to the skin can be carried out to determine this subset. Alternatively the subset of sensors can be determined by evaluating the sensor's output information conent. In a fusion of sensor information on classification level, each individual sensor's data are analyzed and used for recognition of postures or activity. In a subsequent step the individual sensor's classification results are fused to one final result, e.g. by majority vote.

An alternative method for compensating garment drape includes the simulation of the garment drape using e.g. discrete or continuous particle models. The simulation of the garment drape can be performed e.g. inside the integrated processing unit. Measured body-segment orientation data can be enhanced with simulation results for improving measurement accuracy. Instead of simulating the garment drape, the effective errors can be measured directly using body-attached reference sensors. Again, their information can be utilized for compensating the measured orientation errors.

The following general concepts for error compensation may be employed, for example:

A. Much knowledge and small number of sensors: Classification by taking into account typical (known) orientation errors introduced by the loose fit, and/or reference measurements with reference sensors attached to the skin.

B. No/little knowledge and many sensors: Adaption of the system by selecting the most exact sensors, or by multi-sensor fusion.

Some methods are listed in the following table

The invention has the following advantages with respect to state of the art: o Sensors can be placed anywhere over the body. It is detected automatically if they are attached or detached (=defect sensor). Any type of sensor can be attached due to a generic sensor interfacing. All points make the shirt to a flexible sensing and processing platform.

o Contrary to state of the art systems, sensors and electronics are integrated into an everyday piece of clothing. Contrary to textile state of the art posture sensing systems, the shirt is autonomous. No external hardware is needed for a processing of data.

o Contrary to textile state of the art posture sensing systems, the shirt is loose fitting. Subjects can move without hindrance, the shirt is feasible for usage in unsupervised everyday life. An invisible sensing and processing of data will increase the system acceptance.

o Contrary to state of the art systems, the errors introduced by the placement of sensors onto a loose fitting textile can be compensated inherently, e.g. by a classifier and a multi-sensor fusion approach.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. Ia: A garment according to the invention (inside out) showing the central data processing unit (Konnex) and three decentral preprocessing units (Gateways; each is marked with a circle, the fourth Gateway is hidden). Fig. Ib: Placement and routing of a system according to the invention with two Terminal units. The system bus was embedded on the shirt using silicone gel. Dotted lines indicate sewing lines where the system bus was routed for unloading of mass and garment strain. Double lines, perpendicular to shirt stretching zones, indicate the textile strain that was considered by routing in winding paths.

Fig. 2a: Dependence between the information flow and the system architecture.

Fig. 2b: An exemplary system architecture designed in three processing layers of standard activity recognition tasks. Sensor data is acquired and preprocessed by Terminals (first layer), features are computed by Gateways that fuse sensor data (second layer), and recognition tasks are performed by a Konnex (third layer).

Fig. 3: The worn garment. White bars at the upper arm, back and lower waist are the Gateway's connectors to Terminals (marked with a circle).

Fig. 4: The left picture shows an ADC Terminal (8x8mm) and an acceleration Terminal (8x1 Omm). On the right, a silicone integrated Gateway is shown.

Fig. 5: Experimental procedure for the system evaluation.

Fig. 6: Three different repetitions of the exercise, performed by an exemplary user.

Fig. 7: Average error in the orientation of the arm during the repetition of the experiment. Fig. 8: Twelve classified postures.

Fig. 9: Sensor placement at the wrist and upper arm (marked with a circle).

Fig. 10: Posture classification accuracy for the rehabilitation exercises and the three training modes: user-specific, user-adapted and user-independent.

Fig. 11 : Posture classification confusion matrix for the rehabilitation exercises in user-specific training mode.

Fig. 12: Posture classification confusion matrix for the rehabilitation exercises in user-independent training mode.

Fig. 13: User-specific classification accuracy variation (maximum-minimum) in relation to user arm length and body size. The dashed line indicates the result of a linear fitting.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

1 Introduction

The design and implementation of a novel posture and movement sensing platform, called SMASH (SMArt SHirt), is presented. The sensing platform is integrated to a not specifically tightened textile. SMASH is designed to operate as a comfortable monitoring garment for everyday use in movement rehabilitation or sports coaching. Sensing and processing tasks can be efficiently implemented using a distributed system architecture. Other architectures are also possible.

The following aspects of the invention are discussed below:

1. SMASH is introduced and the garment's sensing and hierarchical data processing architecture is described. The latter resembles three processing layers: sensor data acquisition, feature processing and classification. Sensing terminals are connected to a textile integrated core system, consisting of decentral preprocessing units (interface gateways) and a central system master. The master performs classification tasks on the preprocessed data from the gateways.

2. A characterization procedure to analyze the SMASH system is presented. With this approach, the system's resolution is evaluated for arm postures with five users. The posture resolution is derived from the absolute measurement error and verified by classifying 37 arm angle postures.

3. It is shown that the system can be applied in movement rehabilitation of shoulder and elbow joints. Here a study with eight healthy individuals wearing the garment and performing 12 relevant postures is presented (see Fig. 8). Good classification rates for different training modes are achieved, using simple acceleration sensors instead of full inertial systems. In addition, the garment's feasibility with regard to errors incurred from the non-tightened fitting are discussed.

Section 2 outlines the distributed system architecture along with its garment implementation. Sections 3 and 4 present characterization and movement rehabilitation investigations respectively. Finally, Section 5 summarizes the results of this work.

2. System Architecture

This chapter provides a system overview and discusses design issues. Afterwards, all units of the garment according to the invention are described in detail.

2.1 Overview and Terms

The sensing apparatus integrated into a loose-fitting garment is made of a hierarchical processing network. A picture of the garment is shown in Fig. Ia, a schematic drawing of the garment as well as the electronic compenents is shown in Fig. Ib. The system architecture is shown in Figs. 2a+b: The SMASH hierarchy is in this example aligned to a three-layer stack of standard tasks performed for activity recognition: signal sampling and sensor data preprocessing, feature processing, and pattern recognition. The corresponding functions are implemented in the SMASH units Terminals, Gateways, and a Konnex.

A central system master K, the Konnex, is connected to four decentral preprocessing units, the Gateways G, by a wired system bus 1. Together they form the core system 2, which is fully integrated to the textile 3. Each of the four Gateways G provides standardized interfaces 4 to outer peripheral platforms, called Terminals T (sensors). One kind of Terminal is the acceleration Terminal.

2.2 System Design SMASH was designed to acquire, evaluate and process signal data, perform an online classification and to report results to the user. In this flow, information are represented on different levels of abstraction, like physical observations, electrical signals, meaningful features and interpreted classification outputs. One design goal for SMASH was the distribution of processing tasks onto different computation units in a hierarchical way. Hence, data are separated and processed in three layers according to their level of abstraction: signals, features, classification. An overview is depicted in Fig. 2a+b.

Terminals convert physical observations into an electrical signal representation. Signals are filtered and translated into a standardized format for further processing. However, Terminals act on signal level and are the first processing layer. Afterwards, the Terminal sends collected data to one of the Gateways over a wired connection (terminal bus 5). Gateways acquire data from several attached Terminals and fuse them in order to extract meaningful features. Gateways are the second processing layer on the feature level. Gateways send features to the Konnex, which is the last processing layer. It takes the features as an input for an online classification and reports the results to the user. Especially in case of the terminals it was the aim to achieve a minimum PCB size. Hence, all electronic components were selected after their chip area.

Posture classes are discriminated on the Konnex by a Nearest Centroid Classifier. The unit is able to perform a sample-wise realtime classification with seven active acceleration Terminals and a sample frequency of 16Hz. SMASH can alternatively be configured to operate as a pure data acquisition hardware system. Gathered sensor data can be either stored into a non-volatile memory or sent to an outer host via an integrated IEEE 802.15.4 compatible Bluetooth module. Locally developed data recording software receives, visualizes and stores received data. 2.3 Terminals

Terminals are miniature and lightweight sensors or actuators, connected to the core system via a 2-wire I²C bus (see Fig. 4). SMASH is able to detect plugged Terminals and identify their services during runtime. For this, the available I²C address space was segmented, each segment is reserved for one type of Terminal. Gateways are polling the available address space by sending a short ping-message to every address. Newly connected Terminals respond to that ping-message and, hence, are identified. Each implemented kind of Terminal is equipped with an ATmega48 8-bit microprocessor for basic pre-processing. It offers all required functionality and interfaces with a size of 5x5mm. Following types of Terminals have been implemented:

3D-accelerometer Terminals with a size of 8x1 Omm are used to calculate the orientation of body segments by means of gravity vectors.

General purpose ADC Terminal containing four analog-digital converters with a resolution of lObit. Each ADC Terminal has a total size of 8x8mm and can be equipped with custom pull down resistors to be configured for specific measuring ranges. ADC Terminals are used to gather the temperature and resistance of the user's skin or light conditions.

An I/O interface with four input buttons (e.g. for primitive posture class labeling) has four additional LEDs to signal events. The LEDs are currently configured to shine if a button is pressed.

2.4 Gateways The main task of the Gateways is to provide an interface between the core system and remote Terminals. A special issue was the placement of the units on the garment. The goal was to permit a balanced distribution of Terminals over the whole body with a maximal cable length of 85cm. Two Gateways are located at the right and left upper arm to reach the upper body and limbs. A third Gateway was placed at the back in order to reach upper body locations and the head. A final Gateway was placed at the lower waist, to reach the legs. The positions of the Gateways are indicated in Fig. la+b, 3.

Each Gateway is equipped with four sockets, where Terminals can be connected. Hubs extend the number of Terminals attachable to a single Gateway to 127. Hence, the system can be equipped with about 500 Terminals. Here, a 3D-accelerometer is mounted on every Gateway, to give the core system a basic data gathering capability.

The device is internally handled as a virtual acceleration Terminal. It is equipped with a MSP430F1611 lόbit microcontroller, since it offers a good compromise between available computation power, necessary peripheral interfaces and power consumption. The selected model is equipped with 10KB of RAM, which allows a later porting of an operating system.

Gateways and Konnex are connected by a 4-wire Serial Peripheral Interface (SPI) bus in a redundant star topology. The Konnex, as the logical bus master, is able to detect broken signal wires and to restore the connection to unreachable devices via an associated Gateway in a static routing.

2.5 Konnex The Konnex is the system master for communication, power and data processing. The latter is done by an additional MSP430 microprocessor. For reasons of comfort, the Konnex and the battery are located near to the body's center of mass, at the lower back where the extra weight is hardly noticeable for the wearer.

A central power supply generates a system- wide distributed voltage of 3.3V. It is sourced by a flat, detachable lithium polymer battery. A virtual Terminal is located on the Konnex in order to observe the system voltage. SMASH disables all communication modules and Terminals if the system voltage drops below a critical level. A long term test to estimate the system's runtime was performed. SMASH was equipped with three acceleration Terminals and performed an on-line classification of three randomly trained posture classes. The results were sent continuously to an outer host PC via the integrated Bluetooth module. For three runs, a battery life far in excess of 14 hours was measured.

The following tables show examples for SMASH terminals and SMASH system units:

feedback LEDs.

Table 1 : SMASH Terminals that are currently implemented and tested.

Table 2: SMASH system units.

2.6 Textile Integration

The plain core system is integrated into the inside of a long sleeve garment, see Fig. Ia. The base layer of SMASH is a commercial off-the-shelf long-sleeve shirt. The shirt design was chosen as it allows to attach sensors at various locations, even at wrists, while being conveniently worn as casual cloth during daily life. The following table shows an example of a SMASH substrate garment:

Table 3: parameters of SMASH substrate garment

One goal was to keep the electronic-equipped garment comfortable, as it should not hinder any movement of the wearer. Also, the electronic components shall need to be kept save from environmental stress, technical shock, vibration and electrical shorts. While there exists a variety of integration methods, it was decided to glue the core system into the inner side of the garment with silicone gel (polymerized siloxanes, see also Fig. 4). The usage of silicone as a base material has several advantages:

Low toxicity

Electrically insulated

Thermally stable

Low chemical reactivity, resistant to oxygen and ozone

3 System characterization

The ability of SMASH to distinguish different postures depends on several factors, listed in Table 4.

Table 4: Summary of systematic errors.

The following section investigates the limits of the system's accuracy and presents the results of a study.

3.1 Characterization of Accelerometer

The accuracy of the system is limited by the electrical properties of the used ADXL330 accelerometer. The output of each Terminal's X, Y and Z axis was measured three times for -Ig, Og and Ig, respectively. All acceleration Terminals showed a similar accuracy for the different axes. Table 5 shows the average derivations for the outputs of the axes (for clearness, the deviation was converted from g [m/s²] to degrees [°]). The system's resolution is electrically limited to approx. 1°.

Table 5: Stability of the Terminals output, averaged for three readings for -90°,0°and 90°.

3.2 Vertical Angle Resolution of the System

The accuracy of the overall system was evaluated in a study. A typical movement, like the abduction of the right arm, was picked and cut it into equidistant steps of 5 degrees. The movement can be exercised in a range from approx. 0° to 180°, which ends up in 37 different posture classes. For orientation, a poster showing a semicircle and excentric beams in an respective angle of 5 degrees was printed. Two female and three male subjects put on SMASH and were equipped with an acceleration Terminal at the right wrist. Subjects were instructed to stand with their back to the poster so that the shoulder joint was aligned in front of the semicircle's center (see Fig. 5). Starting with pointing to the bottom, the right arm was abducted in steps of 5 degrees from 0° to 180°. Each of the 37 postures was held for at least one second and labeled by an assistant. The whole exercise was repeated three times. The subjects were asked to perform random activities between the repetitions in order to realign the garment in a natural way.

After all feature sets were recorded, it was tried to discriminate the posture classes with Naive Bayes classification. The classifier was trained and tested with the raw sensor data of the right Gateway and the acceleration Terminal at the right wrist. A user-specific accuracy was computed with a 3-fold cross validation, performed for every repetition of the exercise. Considering all 37 classes, an average accuracy of 41% was achieved. By looking at the classified postures it was noticed that only neighboring classes (abduction angles) are confused. When the classification was repeated for angles of 10° (19 classes), an average user-specific accuracy of 85% was achieved. One reason for the low accuracy is the shift of the garment during the exercise. The user-specific accuracy is increased from 41% to 61% if the textile is locally fixed on the forearm. A more important reason is an observed rotation of the forearm during the exercises. Fig. 6 shows the unit vectors for each 3 repetitions, for a randomly chosen subject. The rotation is indicated by the excursion of the Y- acceleration.

For a quantization of this effect, the absolute difference of the gravity vectors for the exercise's repetitions was calculated. Fig. 7 shows the average orientation difference (error), depending on the arm's abduction angle (dotted line). The rotation reaches a maximum of about 20° for a flexion of approx. 105° and decreases to 3° if the arm points up. This effect can be explained in two ways. On one hand the sleeve is shifted, if the arm is raised. It becomes tight fitting and aligns the acceleration Terminal in a similar way to the body. On the other hand the influence of the arm's rotation decreases if the arm is raised. In case the arm points up, the gravity vector is orthogonal to the rotation-sensitive Y-axis.

The solid line in Fig. 7 depicts the averaged angular deviations, if the Y-axis is masked. Hence, the system's average angular accuracy in the poster's plane is better than 7.5°. The classification was repeated without both Y-axes in order to avoid the rotation's influence. A final user-specific accuracy of 88% was reached.

4. Posture classification for rehabilitation

A study was conducted to evaluate the SMASH garment platform's feasibility for movement rehabilitation. Specifically, it was concentrated on postures of the arms, relevant for the therapy of shoulder and elbow joints. The investigation of the system's stable operation and the textile fitting impact was conducted with healthy users. Here, the experimental procedure and the classification analysis results are presented.

4.1 Experimental Procedure

Eight users (3 female, 5 male) aged between 23 and 32 years participated in the study. The users wore the SMASH system (size: large) with one sensing acceleration Terminal placed at the end of the right sleeve. The Terminal was attached to the Gateway at the right upper arm. Both Terminal and Gateway sampled their 3D- accelerometer with 16 Hz. For the purpose of this investigation the raw sensor data was transmitted from the Konnex to a remote PC for offline analysis. Fig. 9 shows the sensor positioning. A set of exercises, commonly used for rehabilitation training of the shoulder and elbow joint was selected for the investigation. Table 3 summarizes the individual postures. Each exercise begins with the user standing upright, arms relaxed. This is indicated as normal position (class 1 in Tab. 6). Fig. 8 shows all 12 postures, each was explained and shown to the users before the respective exercise. A large mirror was provided to the users, in order to verify the execution. Each user conducted three repetitions of all exercises. Between each repetition, users were asked to perform some random movements to restore the garment's natural alignment. An experiment observer annotated the individual postures for the following analysis. In total, 24 instances from 12 different postures were recorded.

Exercise 2: Flexion, Elevation of arm

4 Flexion 90° Shoulder

5 Elevation 170°

Exercise 3: 15° Adduction of arm

Exercise 4: Rotation of arm

Exercise 5: Flexion of forearm

9 Flexion 90° Elbow

10 Flexion 130°

Exercise 6: Neck-grip

Exercise 7: Flexion of forearm

Table 6: Rehabilitation exercises included in the study. Each exercise consists of one or several postures.

4.2 Classification Procedure

Using the recorded datasets a classification analysis was performed. A Nearest Centroid algorithm and a cross-validation scheme was utilized to split training and testing instances for the evaluation. The mean accelerometer readings from upper arm and wrist were used as classification features.

The classification performance for the three training modes was analyzed: user- specific, user-adapted and user-independent. For the user-specific evaluation, training and testing was performed on the posture instances from each user individually. A leave-one-out cross-validation was used. In user-adapted mode posture instances from all users were selected for training and testing set. A threefold cross-validation was used. The user-adapted evaluation includes instances from more than one subject and is an intermediate step between user-specific and user- independent modes. The user-independent is typically the hardest test. Here, the classification performance is analyzed for postures from a user who's postures were not included in the algorithm training. This test indicates the performance of the system when used with new person. For this mode, a cross-validation on the number of users (eight) was used.

4.3 Classification Results

The classification results for all three training modes were: 95% for user-specific, 94% for user-adapted and 89% for user-independent. The best rate was clearly achieved for the user-specific mode. However, the classification performs only slightly better compared to the user-adapted case.

For the user-independent mode training was performed on the data from other users. Here the performance is lower, indicating differences in the postures between the users. Fig. 10 depicts the performances of all training modes along with minimum and maximum values. The values indicate the range of results for each user in the user-specific and user-independent evaluations. For the user-adapted case, the result variance from the cross-validation folds is shown. As expected, this variance is very low, since instances from all users are used for training and testing in this mode. Figs. 1 1 and 12 show the classifier confusion matrices for the user-specific and user- independent cases respectively. This class confusion provides an indication on the misclassified postures. The overall good classification of each class is shown by the main diagonal being close to one. Confusions (all fields besides the main diagonal) for both training modes are found for posture classes 3 and 4 as well as classes 1 and 6. This effect can be explained by the visual similarity of the postures (compare Fig. 8). For the user-independent evaluation further confusions were observed.

In conclusion, the rehabilitation exercises can be successfully classified with good accuracy, even for the more challenging user-adapted and user-independent training modes. This indicates that the movement of the textile does not prevent the accurate classification of the selected exercises with simple acceleration sensors.

4.4 Influence of the Textile Fitting

While good overall classification results for the 12 postures were achieved, user dependent variations in the classification performance were observed. It is assumed that the performance could be partially explained by the user's body size and larger variations in the garment positioning during the exercised. In the experiments one garment model was worn by all users. Subsequently, the user-specific classification performance in relation to the body dimensions (size and arm length) was analyzed.

The relationship between classification rate and the user arm length and body size was investigated. Contrary to the initial assumption, no clear dependency between the variables was found. For only one user (with a height below 165 cm) a noticeably reduced accuracy was observed. The individual cross-validation runs for each user resulted in a varying classification performance. It is assumed that this variation is linked to the body dimensions. Indeed, Fig. 13 supports this assumption: lower accuracy variations were found for both longer arm length and body size. While the evaluation set of eight users is not sufficient to infer a conclusive rule, this relation is very intuitive.

5 Conclusions

In this work SMASH was presented, a novel sensing and processing platform integrated into a upper body garment. The system processes sensor data in a three- layer distributed architecture. At the signal level remote sensing terminals are used, features are extracted by local gateways and a central system master performs classification tasks. Sensor data processing and classification as well as hot-plug- capabilities were implemented in the system. In the current evaluation, accelerometer sensors were used. However, the system is designed to support different sensing modalities.

The system performance was evaluated in two independent studies. The first evaluation addressed the sensor resolution in a characterization experiment with five users. It was found that the system was able to detect angle changes at a resolution of 7.5°. This result was confirmed by posture classification in 5° and 10° resolution. The procedure has potential for the validation of similar textile systems in the future.

In the second evaluation, the classification of rehabilitation exercises was studied. Eight users conducted 12 individual postures from different exercises while wearing the garment. The feasibility of this application was investigated by a cross-validation classification scheme in different training modes. Classification rates of 95% for the user-specific and 89% for the user-independent mode were obtained. These good results indicate that the textile system can be used successfully to monitor the investigated rehabilitation exercises. Potential applications include tele- rehabilitation, where the therapist is not locally available to observe or verify the exercise execution. Concepts for these applications are currently developed, e.g. by the Special Interest Group on Telerehabilitation [15].

Moreover, the relation between user's dimensions and the classification performance was analyzed. Visual analysis of scatter plots indicated that the variations in classification performance were linked to the user's body size and arm length. For smaller users, larger performance variations were found. However, it is clear that many further garment fitting aspects could influence the classification performance of exercise postures.

While the current results are already very promising for rehabilitation and sports applications, it is planned to extend the studies to more subjects and postures. The relation of fitting aspects on the feasibility of the garment will be further investigated. Moreover, it is possible to extend the platform with additional sensing modalities through the plug-and-play design. The system can be used as a community platform to advance further research in the field of textile integrated electronics.

References

[1] J. K. Perng, B. Fisher, Seth Hollar and K. S. J. Pister. Acceleration Sensing Glove (ASG). In Proceedings of the 3rd IEEE International Symposium on Wearable Computers (ISWC), pages 178-180, 1999 [2] A. Tognetti, F. Lorussi, R. Bartalesi, S. Quaglini, M. Tesconi, G. Zupone and D. De Rossi. Wearable kinesthetic system for capturing and classifying upper limb gesture in post-stroke rehabilitation. In Journal of NeuroEngineering and Rehabilitation 2005, 2:8 doi: 10.1186/1743-0003-2-8

[3] A. Just, Y. Rodriguez and S. Marcel. Hand Posture Classification and Recognition using the Modified Census Transform. In 7th International

Conference on Automatic Face and Gesture Recognition, 2006, pages 351-356

[4] T. Starner and A. Pentland. Real-Time American Sign Language Recogni-tion From Video Using Hidden Markov Models. SCV95, 5B Systems and Applications, 1995

[5] S. Park, K. Mackenzie, and S. Jayaraman. The wearable motherboard: A framework for personalized mobile information processing (PMIP). Presented at the ACM/IEEE 39th Design Automation Conf., New Orleans, 2002

[6] L. E. Dunne, P. Walsh, B. Smyth and B.Caulfield. Design and Evaluation of a Wearable Optical Sensor for Monitoring Seated Spinal Posture. In

Proceedings of the 10th IEEE International Symposium on Wearable Computers, Oct. 2006, pages 65-68 [7] T. Giorgino, F. Lorussi, D. De Rossi and S. Quaglini. Posture Classification via Wearable Strain Sensors for Neurological Rehabilitation. Engineering in Medicine and Biology Society 2006, Aug. 2006 ,pages: 6273-6276

[8] C. Mattmann, O. Amft, H. Harms and G. Trδster. Recognizing Upper Body Postures using Textile Strain Sensors. In Proceedings of The 11th International Symposium on Wearable Computers, Boston, October 2007, pages 29-36

[9] H. Junker, P. Lukowicz and Gerhard Trδster. PadNET: Wearable Physical Activity Detection Network. In Proceedings of the 7th IEEE International Symposium on Wearable Computers, 2003, page 244

[10] Moven -wireless inertial motion capture. www.moven.com

[11] E. R. Bachmann, X. Yun and R. B. McGhee. Sourceless tracking of human posture using small inertial/magnetic sensors. In Proceedings of Computational Intelligence in Robotics and Automation, 2003, pages 822-829

[12] J. Marins, X. Yun, E. R. Bachmann, R. McGhee and M. Zyda. An Extended Kalman Filter for Quaternion-Based Orientation Estimation Using MARG Sensors., 2001

[13] D. Roetenberg, H. Luinge and P. Veltink. Inertial and magnetic sensing of human movement near ferromagnetic materials. In Proceedings of the 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003, page 268 [14] B. Clarkson, K. Mase and Alex Pentland. Recognizing User Context via Wearable Sensors. In Proceedings of the 4th IEEE International Symposium on Wearable Computers (ISWC), 2000, pages 69-76

[15] Special Interest Group on Telerehabilitation

http://www.telerehabilitation.net/index.html

Claims

WHAT IS CLAIMED TS:

1. Garment, in particular upper body garment, comprising an apparatus for detection of the orientation of at least one body segment, wherein the apparatus is integrated to the garment and comprises a plurality of sensing terminals and at least one processing unit in communication with the terminals, and wherein the garment is loose-fitting.

2. Garment according to claim 1, wherein, when worn by a user, the terminals are not fixedly coupled to the user's skin such that the position of the terminals is variable with respect to predetermined body segments.

3. Garment according to claim 1 or 2, wherein a total area or predetermined dimensions of the garment of a predetermined size exceed a predetermined body area or predetermined body dimensions of a standard user having the same size, by a predetermined value, preferably by at least 10%.

4. Garment according to one of the preceding claims, wherein the pressure exerted on the user by the garment when worn does not exceed the pressure caused by the weight of the garment.

5. Garment according to one of the preceding claims, wherein the apparatus is able to determine a posture and/or body movement and/or activity from the detected orientation of the at least one body segment.

6. Garment according to one of the preceding claims, wherein the apparatus is autonomous and comprises a power supply, recording and processing hardware, as well as software for at least one of measuring and analyzing sensor data, compensation of errors caused by loose fit, detection and/or classification of orientation of body segments, postures and activities, provision of feedback.

7. Garment according to claim 6, wherein the apparatus further comprises an actuating signal unit for providing a feedback to the user, e.g. an optical, vibrational, and/or acoustical signal.

8. Garment according to one of the preceding claims, wherein the terminal dimensions are less than 10 mm x 10 mm x 5 mm.

9. Garment according to one of the preceding claims, wherein the processing unit comprises central processing unit and a plurality of decentral preprocessing units in communication with the central processing unit and the terminals.

10. Garment according to one of the preceding claims, wherein the sensing terminals comprise strain gauges, accelerometer, gyroscopes, magnetic field sensors, optical fibers, goniometer, pressure sensors, sensors for ultrasonic distance measurements between body-segments and/or optical systems. ..

11. Garment according to one of the preceding claims, wherein the apparatus comprises an interface for communication with an external unit, e.g. a data processing unit.

12. Garment according to one of the preceding claims, wherein the processing unit is capable of compensating for errors introduced by the loose coupling between the terminals and predetermined body segments.

13. Method for detection of the orientation of at least one body segment of a user, in particular for posture and/or body movement analysis, comprising the steps of

- providing the user with a loose-fitting garment, in particular upper body garment, comprising an apparatus for detection of the orientation of at least one body segment, wherein the apparatus is integrated to the garment, and comprises a plurality of terminals and at least one processing unit in communication with the terminals;

- determining the orientation of at least one body segment by analyzing the terminal signals in the processing unit.

14. Method according to claim 13, further comprising the steps of determining from the measured body-segment orientation a posture and/or a body movement and/or activity.

15. Method according to claim 13 or 14, further comprising the steps of compensating errors caused by the loose coupling between the terminals and predetermined body segments by means of at least one of - comparison with reference measurements;

- comparison with model calculations;

- feature selection;

- boosting.

16. Method according to one of claims 13 to 15, further comprising the steps of

- determining an orientation of a body segment by analyzing sensor signals from a predetermined subset of sensors; and

- determining a posture by combining information on the orientations of different body segments.

17. Method according to one of claims 13 to 16, further comprising the step of providing a feedback to the user in dependence on the determined posture and/or body movement.

18. Method according to one of claims 13 to 17, further comprising the step of varying the locations of the terminals on/in the garment.