WO2015172229A1 - Virtual mirror systems and methods - Google Patents

Abstract

Online purchasing or electronic commerce of items relating to a user's appearance within the prior art comprises the user viewing stock images of the items provided by online retailers / enterprises which may include models or simply images of the item, sometimes from multiple viewpoints. However, the user has no way of visualizing these items upon themselves prior to ordering nor do they receive recommendations of items based upon an assessment of the predetermined portion of the user's body to which the item relates. However, embodiments of the invention provide the user with virtual mirror solutions allowing them to view a rendering of the item fitted to their personal predetermined portion of the user's body. For example, a user may establish with a virtual mirror according to the invention rendered eyewear frames fitted to their head in real-time or view recommendations presented based upon classification of the user's head shape.

Description

VIRTUAL MIRROR SYSTEMS AND METHODS

FIELD OF THE INVENTION

[001] This invention relates to virtual mirrors and more particularly to virtually fitting an item to an individual in real-time based upon a physically-based reconstructed map of the user and fitting the item to the physically-based reconstructed map with the physical constraints of the item.

BACKGROUND OF THE INVENTION

[002] Electronic commerce is rapidly increasing because of its convenience, low cost, and time savings for consumers. Whilst some products such as books. Entertainment, music and electronic equipment are well suited the electronic commerce marketplace model, other industries such as fashion and eyewear are trailing significantly as these items are inherently significantly more personal and individual to users. Despite this in the United States alone ecommerce of clothing in 2013 represented approximately $45 billion in revenue to online retailers and is projected to grow to approximately $90 billion in 2018. Unlike bricks and mortar retailing online retailing of items such as clothing, cosmetics, apparel, and other fashion suffers due to the lack of online tools such as real-time virtual mirrors allowing realtime virtual visualization of the items on the user and information relating to the feel, fit, and comfort of the product or item.

[003] Today, online apparel retailers have somewhat circumvented this issue by allowing consumers to return items that are not suitable at no or very low cost with relatively low order value thresholds. However, these costs must be fully absorbed by the retailers and still create delay, inconvenience, and frustration to the users as well as their purchase and wearing of items that do not suit or fit appropriately. As such, method and systems that can decrease the return rate of items, by for example providing better fitting information (physically-based on the user's three dimensional profile, the item's three dimensional profile, its weight, and the interaction thereof) before the purchase is made, have utility for online retailers and consumers alike.

[004] Against this backdrop the average consumer is now able to acquire their purchases from a bewildering array on online retailers in addition to the convention bricks-and-mortar retailer. The items they purchase range from must-haves, e.g. corrective lenses, to fashion items with iconic brands and overwhelming supply of hundreds and thousands of designs across a wide range of retail price points. However, online the user today is exposed to, at best a basic, and typically no recommendation system to guide or support their purchasing activity. Most prior art recommendation systems are generally procedural based on some a priori aesthetic rules and not necessarily based on real data. These rules are opaque to the user, occasionally contradicting each other and are based on rigid aesthetic principles that does not take into account current fashion, age or culture.

[005] Accordingly, it would be beneficial to provide the consumer with a more flexible, data driven recommendation framework that can aggregate large databases of faces and items of different categories such as glasses, jewelry, clothing etc.

[006] Other aspects and features of the present invention will become apparent to those ordinarily skilled in the art upon review of the following description of specific embodiments of the invention in conjunction with the accompanying figures.

SUMMARY OF THE INVENTION

[007] It is an object of the present invention to address limitations within the prior art relating to online purchasing / ecommerce by virtually fitting an item to an individual in realtime based upon a physically-based reconstructed map of the user and fitting the item to the physically-based reconstructed map with the physical constraints of the item.

[008] In accordance with an embodiment of the invention there is provided a method comprising:

acquiring an image with an image acquisition system providing location and depth information relating to a predetermined portion of a user's body forming a predetermined portion of the image;

processing the location and depth information to generate a user model of the predetermined portion of a user, the user model comprising at least one of a wireframe map and a contour map and a plurality of metrics relating to the predetermined portion of a user from the location and depth information;

receiving from the user identification of an item; retrieving an item model of the item, the item model comprising at least one of a wireframe map and a contour map and a plurality of metrics relating to physical characteristics of the item;

generating a deployable model of the item in dependence upon at least the item model and the user model;

positioning the deployable model relative to the appropriate portion of the user's body; and rendering the deployable model as part of a virtual mirror image presented to the user, the virtual mirror image comprising the rendered deployable model overlaid to the image.

[009] In accordance with an embodiment of the invention there is provided a system comprising:

an image acquisition system for acquiring an image providing location and depth information relating to a predetermined portion of a user's body forming a predetermined portion of the image;

a first microprocessor forming part of a first electronic device for processing the location and depth information to generate a user model of the predetermined portion of a user, the user model comprising at least one of a wireframe map and a contour map and a plurality of metrics relating to the predetermined portion of a user from the location and depth information;

a user interface forming part of a second electronic device for receiving from the user identification of an item;

a modelling module forming part of a third electronic device, the modelling module for: retrieving an item model of the item, the item model comprising at least one of a wireframe map and a contour map and a plurality of metrics relating to physical characteristics of the item;

generating a deployable model of the item in dependence upon at least the item model and the user model;

positioning the deployable model relative to the appropriate portion of the user's body; and

rendering the deployable model as part of a virtual mirror image presented to the user, the virtual mirror image comprising the rendered deployable model overlaid to the image; and a display forming part of a fourth electronic device for displaying to the user the virtual mirror image.

[0010] In accordance with an embodiment of the invention there is provided executable software stored upon a non-transient physical medium, wherein the executable software when executed provides a user with a virtual mirror through a series of modules, the series of modules including

a first module for acquiring image data relating to an image from an image acquisition system, the image data including location and depth information relating to a predetermined portion of a user's body forming a predetermined portion of the image; a second module providing for boundary detection based upon at least one of the acquired image data and the image, the boundary detection establishing a predetermined number of boundary points relating to the predetermined portion of the user's body; a third module providing for feature extraction based upon the boundary points established by the second module, the feature extraction comprising the generation of predetermined geometric shapes based upon first subsets of the boundary points and dimensions established upon second subsets of the boundary points;

a classification module for determining a type of the predetermined portion of the user's body in dependence upon the features extracted by the third module and a plurality of datasets, each of the datasets established from feature extraction performed upon a training set of images relating to a defined type of a plurality of defined types of the predetermined portion of the user's body;

a recommendation engine for recommending an item to the user based upon the determined type of the predetermined portion of the user's body and the results of a survey executed in respect of the plurality of defined types of the predetermined portion of the user's body relating to the type of item;

a modelling engine for retrieving an item model of the item comprising at least one of a wireframe map and a contour map and a plurality of metrics relating to physical characteristics of the item and for generating a deployable model of the item in dependence upon at least the item model and the dimensions established by the third module and positioning the deployable model relative to the predetermined portion of the user's body; and a rendering engine for rendering the deployable model as part of virtual mirror image for presentation to the user comprising the rendered deployable model in the determined position overlaid to the image acquired by the first module.

[0011] Other aspects and features of the present invention will become apparent to those ordinarily skilled in the art upon review of the following description of specific embodiments of the invention in conjunction with the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Embodiments of the present invention will now be described, by way of example only, with reference to the attached Figures, wherein:

[0013] Figure 1 depicts a network environment within which embodiments of the invention may be employed;

[0014] Figure 2 depicts a wireless portable electronic device supporting communications to a network such as depicted in Figure 1 and as supporting embodiments of the invention;

[0015] Figure 3 depicts facial scanning and facial mapping system according to the prior art;

[0016] Figure 4 depicts an exemplary process flow for providing a virtual mirror to a user according to an embodiment of the invention;

[0017] Figure 5 depicts an exemplary process flow for providing product virtualization to a user via a virtual mirror according to an embodiment of the invention;

[0018] Figure 6 depicts an exemplary process flow for providing a virtual mirror to a user according to an embodiment of the invention;

[0019] Figure 7 depicts an exemplary process flow for providing user virtualization to a user via a virtual mirror according to an embodiment of the invention;

[0020] Figure 8 depicts an exemplary process flow for providing product virtualization to a user via a virtual mirror according to an embodiment of the invention;

[0021] Figure 9 depicts schematically an eyewear recommendation system according to an embodiment of the invention;

[0022] Figure 10 depicts a Face Shape Recognition System (FSRS) process flow for a recommendation system according to an embodiment of the invention;

[0023] Figure 1 1 depicts a sample output for a detected face boundary step within an FSRS process flow for a recommendation system according to an embodiment of the invention; [0024] Figure 12 depicts the facial contour points for a detected face boundary step within an FSRS process flow for a recommendation system according to an embodiment of the invention;

[0025] Figure 13 depicts six different facial types employed within a Face Shape Recognition System (FSRS) for a recommendation system according to an embodiment of the invention; and

[0026] Figure 14 depicts an exemplary scenario of CBR for a FSRS according to an embodiment of the invention.

DETAILED DESCRIPTION

[0027] The present invention is directed to virtual mirrors and more particularly to virtually fitting an item to an individual in real-time based upon a physically-based reconstructed map of the user and fitting the item to the physically-based reconstructed map with the physical constraints of the item.

[0028] The ensuing description provides exemplary embodiment(s) only, and is not intended to limit the scope, applicability or configuration of the disclosure. Rather, the ensuing description of the exemplary embodiment(s) will provide those skilled in the art with an enabling description for implementing an exemplary embodiment. It being understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope as set forth in the appended claims.

[0029] A "portable electronic device" (PED) as used herein and throughout this disclosure, refers to a wireless device used for communications and other applications that requires a battery or other independent form of energy for power. This includes devices, but is not limited to, such as a cellular telephone, smartphone, personal digital assistant (PDA), portable computer, pager, portable multimedia player, portable gaming console, laptop computer, tablet computer, and an electronic reader.

[0030] A "fixed electronic device" (FED) as used herein and throughout this disclosure, refers to a wireless and /or wired device used for communications and other applications that requires connection to a fixed interface to obtain power. This includes, but is not limited to, a laptop computer, a personal computer, a computer server, a kiosk, a gaming console, a digital set-top box, an analog set-top box, an Internet enabled appliance, an Internet enabled television, and a multimedia player. [0031] An "application" (commonly referred to as an "app") as used herein may refer to, but is not limited to, a "software application", an element of a "software suite", a computer program designed to allow an individual to perform an activity, a computer program designed to allow an electronic device to perform an activity, and a computer program designed to communicate with local and / or remote electronic devices. An application thus differs from an operating system (which runs a computer), a utility (which performs maintenance or general-purpose chores), and a programming tools (with which computer programs are created). Generally, within the following description with respect to embodiments of the invention an application is generally presented in respect of software permanently and / or temporarily installed upon a PED and / or FED.

[0032] A "social network" or "social networking service" as used herein may refer to, but is not limited to, a platform to build social networks or social relations among people who may, for example, share interests, activities, backgrounds, or real-life connections. This includes, but is not limited to, social networks such as U.S. based services such as Facebook, Google+, Tumblr and Twitter; as well as Nexopia, Badoo, Bebo, VKontakte, Delphi, Hi5, Hyves, iWiW, Nasza-Klasa, Soup, Glocals, Skyrock, The Sphere, StudiVZ, Tagged, Tuenti, XING, Orkut, Mxit, Cyworld, Mixi, renren, weibo and Wretch.

[0033] "Social media" or "social media services" as used herein may refer to, but is not limited to, a means of interaction among people in which they create, share, and/or exchange information and ideas in virtual communities and networks. This includes, but is not limited to, social media services relating to magazines, Internet forums, weblogs, social blogs, microblogging, wikis, social networks, podcasts, photographs or pictures, video, rating and social bookmarking as well as those exploiting blogging, picture-sharing, video logs, wall- posting, music-sharing, crowdsourcing and voice over IP, to name a few. Social media services may be classified, for example, as collaborative projects (for example, Wikipedia); blogs and microblogs (for example, Twitter™); content communities (for example, YouTube and DailyMotion); social networking sites (for example, Facebook™); virtual game-worlds (e.g., World of Warcraft™); and virtual social worlds (e.g. Second Life™).

[0034] An "enterprise" as used herein may refer to, but is not limited to, a provider of a service and / or a product to a user, customer, or consumer. This includes, but is not limited to, a retail outlet, a store, a market, an online marketplace, a manufacturer, an online retailer, a charity, a utility, and a service provider. Such enterprises may be directly owned and controlled by a company or may be owned and operated by a franchisee under the direction and management of a franchiser.

[0035] A "service provider" as used herein may refer to, but is not limited to, a third party provider of a service and / or a product to an enterprise and / or individual and / or group of individuals and / or a device comprising a microprocessor. This includes, but is not limited to, a retail outlet, a store, a market, an online marketplace, a manufacturer, an online retailer, a utility, an own brand provider, and a service provider wherein the service and / or product is at least one of marketed, sold, offered, and distributed by the enterprise solely or in addition to the service provider.

[0036] A 'third party' or "third party provider" as used herein may refer to, but is not limited to, a so-called "arm's length" provider of a service and / or a product to an enterprise and / or individual and / or group of individuals and / or a device comprising a microprocessor wherein the consumer and / or customer engages the third party but the actual service and / or product that they are interested in and / or purchase and / or receive is provided through an enterprise and / or service provider.

[0037] A "user" as used herein may refer to, but is not limited to, an individual or group of individuals whose biometric data may be, but not limited to, monitored, acquired, stored, transmitted, processed and analysed either locally or remotely to the user wherein by their engagement with a service provider, third party provider, enterprise, social network, social media etc. via a dashboard, web service, website, software plug-in, software application, graphical user interface acquires, for example, electronic content. This includes, but is not limited to, private individuals, employees of organizations and / or enterprises, members of community organizations, members of charity organizations, men, women, children, and teenagers. In its broadest sense the user may further include, but not be limited to, mechanical systems, robotic systems, android systems, etc. that may be characterised by a portion of the body being identifiable to a human as a face.

[0038] "User information" as used herein may refer to, but is not limited to, user behavior information and / or user profile information. It may also include a user's biometric information, an estimation of the user's biometric information, or a projection / prediction of a user's biometric information derived from current and / or historical biometric information.

[0039] A "wearable device" or "wearable sensor" relates to miniature electronic devices that are worn by the user including those under, within, with or on top of clothing and are part of a broader general class of wearable technology which includes "wearable computers" which in contrast are directed to general or special purpose information technologies and media development. Such wearable devices and / or wearable sensors may include, but not be limited to, smartphones, smart watches, e-textiles, smart shirts, activity trackers, smart glasses, sensors, drug delivery systems, medical testing and diagnosis devices, and motion sensors.

[0040] "Electronic content" (also referred to as "content" or "digital content") as used herein may refer to, but is not limited to, any type of content that exists in the form of digital data as stored, transmitted, received and / or converted wherein one or more of these steps may be analog although generally these steps will be digital. Forms of digital content include, but are not limited to, information that is digitally broadcast, streamed or contained in discrete files. Viewed narrowly, types of digital content include popular media types such as MP3, JPG, AVI, TIFF, AAC, TXT, RTF, HTML, XHTML, PDF, XLS, SVG, WMA, MP4, FLV, and PPT, for example, as well as others, see for example http://en.wikipedia.org/wiki/List_ofJile Jbrmats. Within a broader approach digital content mat include any type of digital information, e.g. digitally updated weather forecast, a GPS map, an eBook, a photograph, a video, a Vine™, a blog posting, a Facebook™ posting, a Twitter™ tweet, online TV, etc. The digital content may be any digital data that is at least one of generated, selected, created, modified, and transmitted in response to a user request, said request may be a query, a search, a trigger, an alarm, and a message for example.

[0041] Referring to Figure 1 there is depicted a network environment 100 within which embodiments of the invention may be employed supporting publishing systems and publishing applications / platforms (VMSVMAPs) according to embodiments of the invention. Such VMSVMAPs, for example supporting multiple channels and dynamic content. As shown first and second user groups 100A and 100B respectively interface to a telecommunications network 100. Within the representative telecommunication architecture a remote central exchange 180 communicates with the remainder of a telecommunication service providers network via the network 100 which may include for example long-haul OC- 48 / OC-192 backbone elements, an OC-48 wide area network (WAN), a Passive Optical Network, and a Wireless Link. The central exchange 180 is connected via the network 100 to local, regional, and international exchanges (not shown for clarity) and therein through network 100 to first and second cellular APs 195 A and 195B respectively which provide Wi- Fi cells for first and second user groups 100A and 100B respectively. Also connected to the network 100 are first and second Wi-Fi nodes 110A and HOB, the latter of which being coupled to network 100 via router 105. Second Wi-Fi node HOB is associated with Enterprise 160, e.g. Luxotica™, within which other first and second user groups 100A and 100B are disposed. Second user group 100B may also be connected to the network 100 via wired interfaces including, but not limited to, DSL, Dial-Up, DOCSIS, Ethernet, G.hn, ISDN, MoCA, PON, and Power line communication (PLC) which may or may not be routed through a router such as router 105.

[0042] Within the cell associated with first AP 1 1 OA the first group of users 100A may employ a variety of PEDs including for example, laptop computer 155, portable gaming console 135, tablet computer 140, smartphone 150, cellular telephone 145 as well as portable multimedia player 130. Within the cell associated with second AP HOB are the second group of users 100B which may employ a variety of FEDs including for example gaming console 125, personal computer 1 15 and wireless / Internet enabled television 120 as well as cable modem 105. First and second cellular APs 195A and 195B respectively provide, for example, cellular GSM (Global System for Mobile Communications) telephony services as well as 3G and 4G evolved services with enhanced data transport support. Second cellular AP 195B provides coverage in the exemplary embodiment to first and second user groups 100A and 100B. Alternatively the first and second user groups 100A and 100B may be geographically disparate and access the network 100 through multiple APs, not shown for clarity, distributed geographically by the network operator or operators. First cellular AP 195A as show provides coverage to first user group 100A and environment 170, which comprises second user group 100B as well as first user group 100A. Accordingly, the first and second user groups 100A and 100B may according to their particular communications interfaces communicate to the network 100 through one or more wireless communications standards such as, for example, IEEE 802.1 1 , IEEE 802.15, IEEE 802.16, IEEE 802.20, UMTS, GSM 850, GSM 900, GSM 1800, GSM 1900, GPRS, ITU-R 5.138, ITU-R 5.150, ITU-R 5.280, and IMT-1000. It would be evident to one skilled in the art that many portable and fixed electronic devices may support multiple wireless protocols simultaneously, such that for example a user may employ GSM services such as telephony and SMS and Wi-Fi / WiMAX data transmission, VOIP and Internet access. Accordingly portable electronic devices within first user group 100A may form associations either through standards such as IEEE 802.15 and Bluetooth as well in an ad-hoc manner.

[0043] Also connected to the network 100 are Social Networks (SOCNETS) 165, first and second online communities 170A and 170B respectively, e.g. Facebook™ and Linkedln™, first to second retailers 175A and 175B respectively, e.g. WalMart™ and Amazon™, first and second online retailers 175C and 175D respectively, e.g. Warby Parker™ and Avon™, as well as first and second servers 190A and 190B which together with others, not shown for clarity. First and second servers 190A and 190B may host according to embodiments of the inventions multiple services associated with a provider of virtual mirror systems and virtual mirror applications / platforms (VMSVMAPs); a provider of a SOCNET or Social Media (SOME) exploiting VMSVMAP features; a provider of a SOCNET and / or SOME not exploiting VMSVMAP features; a provider of services to PEDS and / or FEDS; a provider of one or more aspects of wired and / or wireless communications; an Enterprise 160 exploiting VMSVMAP features; license databases; content databases; image databases; content libraries; customer databases; websites; and software applications for download to or access by FEDs and / or PEDs exploiting and / or hosting VMSVMAP features. First and second primary content servers 190A and 190B may also host for example other Internet services such as a search engine, financial services, third party applications and other Internet based services.

[0044] Accordingly, a user may exploit a PED and / or FED within an Enterprise 160, for example, and access one of the first or second primary content servers 190A and 190B respectively to perform an operation such as accessing / downloading an application which provides VMSVMAP features according to embodiments of the invention; execute an application already installed providing VMSVMAP features; execute a web based application providing VMSVMAP features; or access content. Similarly, a user may undertake such actions or others exploiting embodiments of the invention exploiting a PED or FED within first and second user groups 100A and 100B respectively via one of first and second cellular APs 195 A and 195B respectively and first Wi-Fi nodes 1 10A.

[0045] Now referring to Figure 2 there is depicted an electronic device 204 and network access point 207 supporting VMSVMAP features according to embodiments of the invention. Electronic device 204 may, for example, be a PED and / or FED and may include additional elements above and beyond those described and depicted. Also depicted within the electronic device 204 is the protocol architecture as part of a simplified functional diagram of a system 200 that includes an electronic device 204, such as a smartphone 155, an access point (AP) 206, such as first AP 1 10, and one or more network devices 207, such as communication servers, streaming media servers, and routers for example such as first and second servers 190 A and 190B respectively. Network devices 207 may be coupled to AP 206 via any combination of networks, wired, wireless and/or optical communication links such as discussed above in respect of Figure 1 as well as directly as indicated. Network devices 207 are coupled to network 100 and therein Social Networks (SOCNETS) 165, first and second online communities 170A and 170B respectively, e.g. Facebook™ and Linkedln™, first to second retailers 175A and 175B respectively, e.g. WalMart™ and Amazon™, first and second online retailers 175C and 175D respectively, e.g. Warby Parker™ and Avon™.

[0046] The electronic device 204 includes one or more processors 210 and a memory 212 coupled to processor(s) 210. AP 206 also includes one or more processors 21 1 and a memory 213 coupled to processor(s) 210. A non-exhaustive list of examples for any of processors 210 and 21 1 includes a central processing unit (CPU), a digital signal processor (DSP), a reduced instruction set computer (RISC), a complex instruction set computer (CISC) and the like. Furthermore, any of processors 210 and 21 1 may be part of application specific integrated circuits (ASICs) or may be a part of application specific standard products (ASSPs). A non- exhaustive list of examples for memories 212 and 213 includes any combination of the following semiconductor devices such as registers, latches, ROM, EEPROM, flash memory devices, non-volatile random access memory devices (NVRAM), SDRAM, DRAM, double data rate (DDR) memory devices, SRAM, universal serial bus (USB) removable memory, and the like.

[0047] Electronic device 204 may include an audio input element 214, for example a microphone, and an audio output element 216, for example, a speaker, coupled to any of processors 210. Electronic device 204 may include a video input element 218, for example, a video camera or camera, and a video output element 220, for example an LCD display, coupled to any of processors 210. Electronic device 204 also includes a keyboard 215 and touchpad 217 which may for example be a physical keyboard and touchpad allowing the user to enter content or select functions within one of more applications 222. Alternatively the keyboard 215 and touchpad 217 may be predetermined regions of a touch sensitive element forming part of the display within the electronic device 204. The one or more applications 222 that are typically stored in memory 212 and are executable by any combination of processors 210. Electronic device 204 also includes accelerometer 260 providing three- dimensional motion input to the process 210 and GPS 262 which provides geographical location information to processor 210.

[0048] Electronic device 204 includes a protocol stack 224 and AP 206 includes a communication stack 225. Within system 200 protocol stack 224 is shown as IEEE 802.1 1 protocol stack but alternatively may exploit other protocol stacks such as an Internet Engineering Task Force (IETF) multimedia protocol stack for example. Likewise AP stack 225 exploits a protocol stack but is not expanded for clarity. Elements of protocol stack 224 and AP stack 225 may be implemented in any combination of software, firmware and/or hardware. Protocol stack 224 includes an IEEE 802.1 1 -compatible PHY module 226 that is coupled to one or more Front-End Tx/Rx & Antenna 228, an IEEE 802.1 1 -compatible MAC module 230 coupled to an IEEE 802.2-compatible LLC module 232. Protocol stack 224 includes a network layer IP module 234, a transport layer User Datagram Protocol (UDP) module 236 and a transport layer Transmission Control Protocol (TCP) module 238.

[0049] Protocol stack 224 also includes a session layer Real Time Transport Protocol (RTP) module 240, a Session Announcement Protocol (SAP) module 242, a Session Initiation Protocol (SIP) module 244 and a Real Time Streaming Protocol (RTSP) module 246. Protocol stack 224 includes a presentation layer media negotiation module 248, a call control module 250, one or more audio codecs 252 and one or more video codecs 254. Applications 222 may be able to create maintain and/or terminate communication sessions with any of devices 207 by way of AP 206. Typically, applications 222 may activate any of the SAP, SIP, RTSP, media negotiation and call control modules for that purpose. Typically, information may propagate from the SAP, SIP, RTSP, media negotiation and call control modules to PHY module 226 through TCP module 238, IP module 234, LLC module 232 and MAC module 230.

[0050] It would be apparent to one skilled in the art that elements of the electronic device 204 may also be implemented within the AP 206 including but not limited to one or more elements of the protocol stack 224, including for example an IEEE 802.1 1 -compatible PHY module, an IEEE 802.1 1 -compatible MAC module, and an IEEE 802.2-compatible LLC module 232. The AP 206 may additionally include a network layer IP module, a transport layer User Datagram Protocol (UDP) module and a transport layer Transmission Control Protocol (TCP) module as well as a session layer Real Time Transport Protocol (RTP) module, a Session Announcement Protocol (SAP) module, a Session Initiation Protocol (SIP) module and a Real Time Streaming Protocol (RTSP) module, media negotiation module, and a call control module. Portable and fixed electronic devices represented by electronic device 204 may include one or more additional wireless or wired interfaces in addition to the depicted IEEE 802.1 1 interface which may be selected from the group comprising IEEE 802.15, IEEE 802.16, IEEE 802.20, UMTS, GSM 850, GSM 900, GSM 1800, GSM 1900, GPRS, ITU-R 5.138, ITU-R 5.150, ITU-R 5.280, IMT-1000, DSL, Dial-Up, DOCSIS, Ethernet, G.hn, ISDN, MoCA, PON, and Power line communication (PLC).

[0051] Also connected to Electronic Device 204 is Motion Capture (MC) 205 comprising a Motion Capture (MC) Stack 275 coupled to Antenna 278 supporting bidirectional communications with the Electronic Device 204. MC 205 may exploit the same protocol as communications between the Electronic Device 204 and AP 206 or it may exploit an alternate protocol wherein Electronic Device 204 may comprise a second Protocol Stack other than Protocol Stack 224 which is not shown for clarity. The MC Stack 275 is in communication with MC Microprocessor (μΡ) 271 which communicates with Memory 272, Light 273, and first and second Video In 274 and 275. Light 273 may, for example, be a visible light source, an infrared light source, or a light source switchable between visible and infrared. These may be continuous and / or pulsed optical sources. Similarly, first and second Video In 274 and 275 may be continuously or periodically capturing images within the visible and / or infrared regions of the electromagnetic spectrum.

[0052] Examples of such MC 205 devices include, but are not limited to, Microsoft™ Kinect™, PLAYSTATION Eye™, PlayStation™ Camera, an infrared motion sensing device, a laser scanning device, a time-of-flight scanning device, an optical motion sensing device, a plurality of cameras, and combinations thereof. For example, the Microsoft™ Kinect™ device provides a depth sensor by combining an infrared laser projector with a monochrome CMOS sensor, which captures video data in 3D under any ambient light conditions. Accordingly, MC 205 transmits image data to the Electronic Device 204 including, but not limited to, image data in a standard picture and / or video file format and image depth data. Optionally, the sensing range of the depth sensor may be adjusted under control of MC μΡ 271 of MC 205 or Processor 210 of Electronic Device 204. [0053] Optionally, MC 205 may alternatively be connected to AP 206 rather than Electronic Device 204 and therein to Electronic Device 204 or the network 100 and thereafter an Electronic Device 204 via another AP 206 such that the MC 205 and Electronic Device 206 are remote to one another. Within other embodiments of the invention the MC 205 is connected to a remote server such as first and second servers 190A and 190B such that the acquired data is transmitted via Network 100 to the remoter server from MC 205 and then processed data from the remote server transmitted to the Electronic Device 204.

[0054] Now referring to Figure 3 there is depicted facial mapping system according to the prior art wherein a user 310 has their image captured and processed by a Microsoft™ Kinect™ gaming controller (MKGC) 320 wherein the captured data is then processed in step 330 to generate:

• 87 two-dimensional (2D) points of features of the user' face, e.g. eyebrows, eye shape as defined by eyelids, mouth shape, and outline of the face from ear-chin-ear;

• 13 points defining center of the eyes, corners of the mouth, center of the nose, and a bounding box around the head; and

• 121 three-dimensional (3D) shape points representing mesh vertices for a wireframe 3D face model.

[0055] An example of the bounding box and rendered wireframe mesh are depicted in image 340 of the user exploiting a software development kit (SDK) provided by Microsoft™. With further processing decisions may be made as to identity of user using previously acquired data from the MKGC 320 on registered account holders and to provide input to gaming software. Similarly, processing of acquired data from the MKGC 320 of the user's body can be used to provide gesture recognition, body posture identification, etc. to similarly provide input to software such as that in execution upon Electronic Device 204, for example, in Figure 2 which would typically be a Microsoft™ Xbox™ gaming console within the prior art.

[0056] Now referring to Figure 4 there is depicted an exemplary process flow for providing a virtual mirror to a user via a VMSVMAP according to an embodiment of the invention. Accordingly, the process begins with a user as depicted in first image 405 in association with a first image acquisition system (IAS) 410, e.g. Motion Capture (MC) 205 in Figure 2 or MKGC 320 in Figure 3. Accordingly, first IAS 410 captures a single image or a sequence of images (video frames) which can be captured live or not, stored locally or transferred through a network such as the Internet or Network 100. Subsequently, facial features are established and tracked online and in real time through either 2D or 3D mapping through mapping process 415 yielding a wireframe model and core facial dimensions such as depicted in second image 420. Within some embodiments of the invention the wireframe and / or other data may be generated in real time, offline within a predetermined latency, and offline. Optionally, the wireframe and / or other data may be generated fully per acquired image or partially per acquired image based upon prior image wireframe and / or other data.

[0057] Subsequently, within some embodiments of the invention, a denser 3D mesh (dense contour map) may be generated in combination with or in isolation from a facial template for each image in dense mapping process 425 yielding, if presented to the user, third image 430. Depending upon the MC 205 and it's processing capabilities together with those of the other devices to which it is connected, e.g. Electronic Device 204, remote server, etc. these meshes, contour maps etc. and their associated tracking of facial orientation, facial features, etc. can be tracked locally and / or remotely and in real time or offline. Optionally, an anatomically correct and dense generic 3D template mesh can be fitted to the user facial features such that the selected features match although if the IAS 410 provided sufficiently high resolution the process may be simplified as no fitting and / or interpolation / extrapolation from a first fitted wireframe (mesh) to a denser mesh may be required.

[0058] Accordingly, the denser 3D mesh generated in dense mapping process 425 and depicted in third image 430 has been scaled and rigidly transformed to roughly fit the facial features obtained from the initial tracking step, mapping process 415 and depicted in second image 420. For example, this may be singular value decomposition of the covariance matrix obtained from the feature points' correspondence. Subsequently in deformation process 445 an offline, delayed and or real-time deformation technique is applied to deform the 3D template mesh generated in dense mapping process 425 to accurately match the face of the user, depicted in first image 405. This deformed facial mask is depicted in fourth image 450 together with a 3D model of an eyeglasses frame selected by the user within a parallel item selection process step 435 wherein upon selection of an item within a database 430 a 3D model of the selected item is extracted and fitted to the deformed facial mask based upon the core facial dimensions of the user and dimensions of the selected item. Generation of the deformed facial mask of the user may, for example, be performed in a manner such that every vertex of the dense 3D template mesh is deformed using a Laplacian operator and constraints provided by the IAS 410.

[0059] Optionally, facial orientation information employed during the generation of the deformed facial mask tracker with or without mapping of a selected item may be obtained from a second IAS 440 such that the output of the dense mapping process 425 is stored after generation and extracted subsequently. Optionally, the 3D model of the selected item, e.g. eyewear frame, may be retrieved with associated physical properties weight, options, dimensional tolerances dimensional variations supported, friction coefficients, material properties, etc. Optionally, where additional properties are not provided, these may be estimated. Subsequently, in virtual mirror process 455 the deformed facial mask is removed and the selected item rendered such that the user is provided with an image, fifth image 460, of themselves with the rendered selected item.

[0060] Optionally, the placement of the selected item upon the user's deformed facial mask may be established through a two-stage process wherein the first stage of the process comprises roughly positioning the selected item in a pre-processing stage with respect to the 3D deformed facial mask. This pre-processing stage may, for example, be manually placing the item by the user and / or using an automatic method based on the position of the facial features. Optionally, the automated method may itself be a two-step process wherein initially a generic mesh template is employed followed by the deformed facial mask. Alternatively, the automated method may be a two-step process wherein initially a mesh / contour template generated in mapping process 415 or dense mapping process 425 is employed followed by the deformed facial mask.

[0061] The second stage of the process comprises a physics-based simulation of the selected item to determine its accurate position either in a single image or in every video frame of a series of images such that the user can visualize the item from a plurality of perspectives. The physics based simulation may, for example, exploit rigid body motion and gravitational forces, friction forces and collision forces. For example, an earring may be simulated to hang from the user's ear and its position relative to the user's ear, neck, etc. depicted including, for long dangling earrings its touching the user's body at certain head positions. The user, may in some embodiments, fine-tune the position of the selected item, e.g. eyewear frame glasses on their nose using a user interface (keyboard, mouse or a hand gesture recognition interface) as minor adjustments may have limited impact on the user's vision but substantial differences visually.

[0062] Within embodiments of the invention, should face tracking fail for during a sequence of frames, the process may recover through a mechanism of iterating back to a previous stage within the sequence as discussed with respect to Figure 4. This iterating back may be, for example, to a predetermined step in dependence upon the length of the failed tracking or sequentially iterating stages and regressing upon determination of an error established, for example, as a measure of distance between the user's face and the generated contour.

[0063] Optionally, the VMSVMAP may provide the user with information relating to an estimated comfort level and / or fit level relating to the selected item, where appropriate. For example, consider an eyewear frame selected by the user then an estimated comfort fit may be established by applying a skin deformation model to the digitally mapped and rendered nose using real or estimated measurements for the weight of the glasses as well as the elasticity of the user's skin, again real or estimated based upon demographic data such as age, race, etc. The comfort level can be conveyed to the user using visual feedback (i.e. color highlighting on the rendering of the skin) or using a numerical scale. Additionally, mismatch between the width of the user's head across their temples derived from the measurements extracted for the user's head from the initial IAS scan may imply pressure to the temple area of the user as the frames are too narrow and / or deformation of the eyewear frame with time and hence a variation in the visual aesthetic as the eyewear frame become loose. Similarly an estimated fit level for the eyewear frame may be derived by calculations of dimensions of the selected item relative to the mapped and rendered body. In the instances that the selected item lacks substantial physical structure, e.g. an item of apparel versus the eyewear frame then physical modelling may be provided to simulate the hang or lay of the selected item.

[0064] Further the VMSVMAP according to embodiments of the invention may, as discussed below, in respect of Figures 5 and 8, for example, provide recommendations based upon the user's physical dimensions and features as well as, optionally, accumulated user taste data, and user social network recommendations for example. Examples of facial fitting and recommendations with respect of eyewear frames are presented in Figures 9 to 18 although similar processes exploiting user data capture, ratio generation, reference aethesthic data sets etc. may be applied to other regions and / or portions of the user's body without departing from the scope of the invention. [0065] It would also be evident that a single image and / or video of the user view within the virtual mirror using a VMSVMAP according to embodiments of the invention may be posted to one or more SOCNETs by the user for their friends to provide feedback, comments, etc. and perhaps even vote on their favorite item of a range of selected item the user is selecting between. The generated single image and / or video of the user may be posted from their PED / FED directly or from a remote server according to the VMSVMAP the user is accessing. Optionally, the user may make a series of initial selections which are rendered at a first resolution and all or a selected subset of these may then be submitted for rendering offline and / or remotely for further processing to a higher resolution.

[0066] Now referring to Figure 5 there is depicted an exemplary process flow for providing product virtual ization to a user via a virtual mirror according to an embodiment of the invention. As depicted a user 505 employs an image acquisition system (IAS) 515, e.g. Microsoft™ Kinect™ connected to a FED 510, e.g. Microsoft™ Xbox™ One console, which is coupled to VMSVMAP 580 via Network 100. Accordingly, the user 505 may execute a process such as described above in respect of Figure 4 and may select via the VMSVMAP 580 to view a series of items, e.g. eyewear frames, from an enterprise, e.g. from Warby Parker 175C in Figure 1. The user selecting items from a first database 525, these being depicted as first to fourth sunglasses 570A to 570D respectively. However, the VMSVMAP 580 accesses a corresponding second database 520 which stores wireframe models of the items within the first database 525, depicted as first to fourth wireframes 560A to 560D which each correspond to one of the first to fourth retail items 570A to 570D respectively.

[0067] Accordingly, the user has selected three items in series as depicted with first to third virtual mirror (VM) images 530, 540, and 550 respectively wherein the user is depicted with first to third selected items 535, 545, and 555 respectively which have been presented using wireframe models to try and focus the user's 505 attention to the overall style / design rather than aesthetic elements such as frame colour etc. First to third selected items 535, 545, 555 being based upon first to third sunglasses 560A to 560C respectively associated with first to third retail items 570A to 570C respectively. Subsequently, the user selects to view a fully rendered selected item, e.g. first sunglasses 570A, and is presented with fourth VM image 590 wherein the selected item is fully rendered as rendered item 595 wherein the user may subsequently adjust aesthetic aspects of the selected item, e.g. frame colour, tint of sunglasses etc. [0068] Now referring to Figure 6 there is depicted an exemplary process flow for a VMSVMAP providing a virtual mirror to a user according to an embodiment of the invention. As depicted the process begins at step 605 with a user initiating a virtual mirror process upon a local and / or remote VMSVMAP and then progresses to step 610 wherein the user determines whether a new rendering process should be initiated or a prior rendered image retrieved for use. In the event of a new rendering process the process proceeds from step 610 to steps 620 to 650 wherein these comprise:

• Step 620 wherein the user sits to establish the scan via an image acquisition system (AIS) under control of the VMSVMAP wherein the VMSVMAP knowing the characteristics of the AIS provides information to the user with respect to the distance that they should be away from the AIS to allow the AIS to define the physical dimensions of the user's body as acquired;

• Step 630 wherein the captured image(s) are processed to generate coarse contour map with core facial dimensions that may be referenced to the known scale of the image;

• Steps 635 wherein the VMSVMAP generates an overlay to the image presented to user with facial recognition template and dense contour map generated by VMSVMAP in step 640;

• Step 645 wherein the facial type of the user is define and key metrics are established for subsequent use in recommendation processes and / or manipulating selected items for overlap to the user's image; and

• Step 650 wherein the facial type, coarse and dense contour maps, and key metrics are stored within a database against user credentials allowing the user to subsequently retrieve and exploit them.

[0069] Optionally, the AIS may ask the user to hold an object of defined dimensions in order to define the magnification of the acquired scan(s). From step 650 the process proceeds to step 615. If the user in step 610 elects to retrieve and employ a previously stored mapping then the process also progresses to step 615 from step 610 wherein the user is provided with a series of options for displaying and exploiting their stored facial profile. These options being depicted through first to third rendering flows 655, 665, and 675 which are associated with user options, Controller, Preview and Full respectively. Within first rendering flow 655, Controller, the user may employ a controller 656 separate to or integrated with their PED / FED they are accessing the VMSVMAP upon. Accordingly, they can through the controller 656 perform tilts, turns, rolls, etc. of the facial rendering based upon their stored profile information. In this manner the user may exploit their rendered profile without requiring access to an image acquisition system every time they wish to employ the virtual mirror.

[0070] Within second rendering flow 665 the user when exploiting the virtual mirror through the VMSVMAP upon their PED / FED is presented with images, including first to third images 670A to 670C, that comprise a view of the user with recognition frame and dense wireframe together with rendered facial images overlaid to the image. In third rendering flow 675 the user when exploiting the virtual mirror through the VMSVMAP upon their PED / FED is presented with images, including fourth to sixth images 680A to 680C, that comprise a view of the user together with rendered facial images overlaid to the image. It would be evident that the user may also be presented with a further rendering flow, not shown for clarity, wherein the only images presented to the user are rendered facial images overlaid to the captured image. Such a rendering flow in combination with first and second rendering flows 665 and 675 may be performed upon a single image, a short video file, a video file, and / or a live video camera stream according to the selection of the user. Optionally, first or second rendering flows 665 and 675 may provide a series of still images on the left hand side and a raw video flow in the right hand side or vice versa. As such the still images may be automatically selected at limits of user head motion or selected by the user with the raw processed video feed also presented.

[0071] Figure 7 depicts an exemplary process flow for providing user virtualization to a user via a virtual mirror provided by a VMSVMAP according to an embodiment of the invention. As depicted the process begins at step 705 with a user initiating a virtual mirror process upon a local and / or remote VMSVMAP and then progresses to step 710 wherein the user determines whether a new rendering process should be initiated or a prior rendered image retrieved for use. In the event of a new rendering process the process proceeds to sub-flow 720 comprising, for example, steps 620 to 650 of Figure 6 above resulting in rendered user 725 before the process flows to step 715. If the user in step 710 elects to retrieve and employ a previously stored mapping then the process also progresses to step 715 from step 710 wherein the user is provided with a series of options for displaying and exploiting their stored facial profile. These options being depicted through first to third rendering flows 730A, 730B, and 730C respectively which are associated with user options, Controller and Retrieve, wherein two different users, User A and User B, respectively exploit the VMSVMAP. First renderings flow 730A may for example be one of the first to third rendering flows 655, 665, and 675 in Figure 6, for example. As depicted second rendering flow 73 OB relates to first user 735, User A, wherein the user applies two different processes from a menu of processing options 740 with respect to their stored profile. Within first process 745 the user has selected a styling option wherein they wish to see what they look like bald and accordingly facial profile information relating to their facial shape is employed on render the upper surface of their head. Within second process 750 the user has elected to adjust their hair colour wherein facial profile information relating to their facial shape may be used to aid derivation of the hair line.

[0072] In third rendering flow 730C, relating to a second user 755, User B, wherein stages in provisioning of virtual mirror images to the second user 755 are depicted. These stages being a result of the user selection options from a menu of processing options 760 with respect to their stored profile. Within first process a template 765 is retrieved before being processed to generate rendered profile 770. In the second process the user's profile is retrieved for a process wherein the user has selected a styling option wherein they wish to see what they look like with different hair styles and accordingly facial profile information relating to their facial shape is employed on the template to define the upper surface of their head with modified template 775 which is then rendered as modified profile 780. As discussed supra in respect of Figure 5 the user may then select a hair style which is rendered onto their modified profile 780.

[0073] Now referring to Figure 8 there is depicted an exemplary process flow for providing product virtualization to a user via a virtual mirror generated using a VMSVMAP according to an embodiment of the invention. In essence the process flow in Figure 8 is similar to that depicted and described with respect to Figure 6 except that rather than the user profile being facial it is their full body. Accordingly, as depicted the process begins at step 805 with a user initiating a virtual mirror process upon a local and / or remote VMSVMAP and then progresses to step 810 wherein the user determines whether a new rendering process should be initiated or a prior rendered image retrieved for use. In the event of a new rendering process the process proceeds from step 810 to steps 825 to 845 wherein these comprise: • Step 825 wherein the user stands to establish the scan via an image acquisition system (AIS) under control of the VMSVMAP wherein the VMSVMAP knowing the characteristics of the AIS provides information to the user with respect to the distance that they should be away from the AIS to allow the AIS to define the physical dimensions of the user's body as acquired. In contrast to the facial recognition the AIS captures images with and without the user as depicted by first and second images 825A and 825B respectively. When the user stands in underwear, lingerie, swimming costume, tight top, tight pants, etc. or naked, for example, then the first image 825A captures their body profile;

• Step 830 wherein the captured first and second image(s) are processed to generate an isolated image 830A of the user;

• Steps 835 wherein the VMSVMAP generates from the isolate body image 830A of the user a coarse contour map with core body dimensions ratios and thereafter a dense contour map 835A;

• Step 840 wherein based upon the characteristics of the AIS and the dense contour map 835 A a body type of the user is defined and key metrics are established for subsequent use in recommendation processes and / or manipulating selected items for overlap to the user's image; and

• Step 845 wherein the body type, coarse and dense contour maps, isolate image and key metrics are stored within a database against user credentials allowing the user to subsequently retrieve and exploit them.

[0074] If the user in step 810 elects to retrieve and employ a previously stored mapping then the process also progresses to step 815 from step 810 wherein the user is provided with a series of options for displaying and exploiting their stored facial profile. These options being depicted through first to third rendering flows 820A, 820B and 820C respectively. First and second rendering flows 820A and 820B are not depicted in detail and end at step 875A. Third rendering flow 820C proceeds with the user selecting an option from a series of options in step 880 and selecting one or more retail items from one or more enterprises through processes known within the prior art. However, the retail items presented to the user are those for which wireframe models exist allowing the VMSVMAP to position and align the items selected by the user upon the wireframe mapping of the user and once rendered depict these atop the image of the user. Accordingly, as depicted the user may select from a plurality of databases including, for example, first to third databases 850A to 850C respectively. Based upon the selected items these are rendered to the user's wireframe(s), aligned / positioned / fitted to their wireframe according to the key metrics and therein presented to the user. Exemplary presented images to the user are depicted in first to fourth images 855 to 870 respectively before the process ends at step 875B.

[0075] Optionally, the placement of the selected item upon the user's rendered wireframe contour map may be established through a two-stage process wherein the first stage of the process comprises roughly positioning the selected item in a pre-processing stage with respect to the 3D body wireframe contour map. This pre-processing stage may, for example, be manually placing the item by the user and / or using an automatic method based on the position of the user's body / physical features. Optionally, the automated method may itself be a two-step process wherein initially a generic mesh template is employed followed by the body wireframe contour map. Alternatively, the automated method may be a two-step process wherein initially a mesh / contour template generated in mapping process 415 or dense mapping process 425 is employed followed by the body wireframe contour map.

[0076] Optionally, the 3D model of the selected item, e.g. dress, may be retrieved with associated physical properties weight, options, dimensional tolerances, dimensional variations supported, friction coefficients, material properties, etc. Optionally, where additional properties are not provided, these may be estimated. Accordingly, a further stage of the process may comprise a physics-based simulation of the selected item to determine its accurate position either in a single image or in every video frame of a series of images such that the user can visualize the item from a plurality of perspectives. The physics based simulation may, for example, exploit rigid body motion and gravitational forces, friction forces and collision forces. The user, may in some embodiments, fine-tune the position of the selected item on their body using a user interface (keyboard, mouse or a hand gesture recognition interface) as minor adjustments may have limited impact on the user's vision but substantial differences visually.

[0077] Optionally, the VMSVMAP may provide the user with information relating to an estimated comfort level and / or fit level relating to the selected item, where appropriate. For example, consider an elasticated item of apparel selected by the user then an estimated comfort fit may be established by applying a deformation model on the elasticated item of apparel's base dimension and user's body metrics to establish pressure applied to the user's body. The comfort level can be conveyed to the user using visual feedback (i.e. color highlighting on the rendering of the skin) or using a numerical scale. Where only portions of the item of apparel in this instance are elastic then the visual feedback may include the location of the localized pressure. In the instances that the selected item lacks substantial physical structure, e.g. a dress or skirt then physical modelling may be provided to simulate the hang or lay of the selected item. Accordingly, the VMSVMAP according to embodiments of the invention may in addition to eyewear and jewelry allow virtual mirrors to be provided for a range of items including, but not limited to, underwear, lingerie, shirts, blouses, skirts, trousers, pants, jackets, hats, sports equipment, and shoes. Optionally, the user may establish a short video sequence, e.g. walking towards IAS, turning around, sitting and standing, etc. In this manner the user's body is isolated and wireframed frame by frame allowing the virtual mirroring process with the VMSVMAP to map the selected item(s) onto the user's wireframe and therein rendered as part of the virtual mirror.

[0078] It would be evident that according to embodiments of the invention the user may feel comfortable providing to a remote server some categories of images relating to portions of the user's body and uncomfortable with respect to other categories. For example, an image of the user's head may be acceptable but an image of the user standing naked or in underwear / lingerie unacceptable. Accordingly, embodiments of the invention may provide for user based selection of local and / or remote processing of the images in order to provide the virtual mirror to the user. It would be evident that acquired, stored, processed images and their associated wireframes, contours, characteristics, dimensions etc. in addition to images rendered with items selected and / or recommended to the user may be stored encrypted within some embodiments of the invention, unencrypted within some embodiments of the invention, and protected by security measures based upon user credentials, device identity, etc. as known within the prior art.

[0079] Within the embodiments of the invention described supra in respect of Figures 1 to 8 a VMSVMAP generates one or more contour maps / wireframes from acquired image data with an image acquisition system. Embodiments of the invention may employ, for example, the Canny Edge Detection algorithm in order to provide an "optimal" edge detector by which the inventors mean one that has a low error rate, meaning good detection of only existent edges; good localization, such that the distance between edge pixels detected and real edge pixels is minimized; and minimal response, such that there is only on detected response per edge. To achieve this Canny used the calculus of variations which finds the function which optimizes a given functional. The optimal function in Canny's detector is described by the sum of four exponential terms, but it can be approximated by the first derivative of a Gaussian. Accordingly, Canny edge detection is a four step process:

- A Gaussian blur is applied to clear any speckles and free the image of noise;

- A gradient operator is applied for obtaining the gradients' intensity and direction;

- Non-maximum suppression determines if the pixel is a better candidate for an edge than its neighbours; and

- Hysteresis thresholding finds where edges begin and end.

[0080] Additionally, embodiments of the invention may generate contours, i.e. curves joining all the continuous points (along the boundary), having same color or intensity. Such contours being useful tools for shape analysis, object detection, and object recognition. Within OpenCV the method given by Equation (4) calculates the contours of an input array, e.g. the image processed by the Canny edge detection algorithm. Further, it would be evident that within embodiments of the invention that detection of edges, e.g. face edge, body edge, neck edge, etc. are a necessary step within the establishment of wireframes, contour maps, and rendered surfaces for VMSVMAPs. Within the prior art as noted supra there are many algorithms offering edge detection. However, based upon the mathematical model and calculations involved, the algorithm adopted by the inventors within initial implemented software providing VMSVMAP functionality was the Gradient Edge Detection (GED) Predictor Template Algorithm (GED Algorithm). The GED Algorithm uses 5 neighbouring pixels to determine local gradient and predict the current pixel value. These 5 adjacent pixels are A and D in the same row, E and B in the same column and C which is a diagonal pixel. The GED Algorithm uses fixed single direction, e.g. from left to right or from top to bottom, to analyze the image but it may not predict the proper pixel values. Accordingly, a multi - directional template of the GED Algorithm was employed with the image divided into 4 parts and each part is processed individually. This is because the central part of the image covers the most of the information about the picture and the regional characteristics and local gradient directions are mainly considered.

[0081] The GED Algorithm steps employed were follows:

[0082] 1. Calculate the Local Gradients (Horizontal and Vertical) using Equations (1) and (2) respectively.

dh = \D - A\ + \C - B\{HorizontalGradient) ( 1 ) dv = \C - A\ + \E - B\(VerticalGradient) (2)

[0083] 2. Prediction Pixel Value is calculated using the following routine:

if (dv-dh>S0) then

Γ=Α

else

if (dv-dh<-S ) then

I'=B

else

/' = [3(A + B)/2 + (C + D + E) 112]

end

end

[0084] 3. Calculate the Forecast Error Value as given by Equation (3).

2<(/,J) = |/-/'| (3)

[0085] 4. Determine Edges of the Image

[0086] To calculate the edges, an between-cluster variance method is used which follows the following steps:

E(I, J) gray value of the original image

r separated threshold value between foreground and background

Gl e E(I, J)≤T foreground

G2 e E(I, J)>T background

Pr^)_^ {n_q/n) rate of pixels number in gray scale r_q for image pixels

(<7 = 0,1,2,...,L-1)

[0087] When σ² value is maximum for some value of T , then T is the best segmentation threshold and edges of the image are identified as below:

if E(i,j)>r then

Mark the Pixel as edge point

else

Mark the Pixel as non-edge point

end

[0088] 5. Apply Thinning Algorithm [0089] Edge image pixel matrix (from previous step) contains multi-pixel wide edges. It has to be skeletonised to the single-pixel wide edge so that the contour points of the face can be determined. Table 1 below provides the terms used within the Edge Detection mechanism.

Table 1 : Terms used in Edge Detection Mechanism

[0090] Following Thinning algorithm is applied to the edge image pixel matrix as:

[0091] (a) Create two matrices ( H_m for horizontal scan matrix and V_m for vertical scan matrix) with all default values set to 1.

[0092] (b) Scan the edge image pixel matrix horizontally and check for any transition between pixels, such as 0 to 1 or 1 to 0. If found then keep the tag the value of 0 to 0 in new matrix. Rest of the values are left to default 1.

[0093] (c) Repeat the same procedure from previous step for vertical scan ( V_m ).

[0094] (d) At the end, by simple "and" logical operation, both H_m and V_m are combined together to give final face edge matrix.

[0095] Accordingly, embodiments of the invention exploit a state of the art edge tracker which tracks with all of the above rules and definitions. The output of the tracker algorithm then provides the input to subsequent steps of VMSVMAP algorithms.

[0096] Within the embodiments of the described supra in respect of Figures 1 to 8 a VMSVMAP is described as presenting virtual mirror images to the user which include rendered images of items selected by the user. In many instances a user may be faced with a bewildering array of options and the VMSVMAP may incorporate a recommendation engine providing options to the user or options may be filtered based upon a classification of the user's body, a region of the user's body, face, etc. Accordingly, embodiments of the invention exploit classification methods of faces and frame types into different categories. The inventors selected the "Case-Based Reasoning" (CBR) classification method to classify face and frame shapes. A CBR solver in essence solves new problems by adapting solutions that were used to solve old problems and can be summarized as having two phases, storing previous cases in memory and solving new problems where the later exploits a four step- process wherein these steps are:

• 1. Retrieve As per the problem definition, retrieve from memory cases that are relevant to solving it. A case consists of a problem, its solution and annotations about how the solution was derived.

• 2. Reuse After retrieving the best possible solution from the previous case, map the case to the problem definition. This may involve adapting the solution as needed to fit the new situation.

• 3. Revise Having mapped the previous solution to the problem, test the new solution in the real world and, if necessary, revise.

• 4. Retain After the solution has been successfully adapted to the problem definition, the resulting experience is stored as a new case in memory that can be used for future learning.

[0097] Accordingly, the recommendation engine forming part of VMSVMAPs according to embodiments of the invention exploit a two-step process wherein in the first part, the user's body (in part or its entirety) is classified by a classification procedure and in the second part the recommendation system provides recommendations to the user. Referring to Figure 9 there is depicted schematically an overview of the overall system for such a recommendation engine with respect to eyewear frames and user's facial geometry. However, the processes and procedures described may be applied to other item / body combinations such as for example, apparel necklines and user's shoulders / neck, earrings and user's ears, and lower body apparel such as skirts, dresses, and trousers and user's hips / legs. As depicted in Figure 9 the recommendation comprises a first portion 900A relates to the data collection pipeline and 900B the recommendation pipeline.

[0098] For each image, the system undertakes Face Shape Recognition and Frame shape Recognition. Each of these algorithms contains pre-processing followed by feature extraction and classification phases. Referring to first portion 900A an image database 910 of subjects wearing eyeglasses are presented to the system as input. Accordingly, the geometric features 920 are extracted from the faces and automatic frame shapes 930 are extracted based on these processes. Subsequently, facial classification 940 and frame shape classification 950 are executed and the results stored within a first database 970A. Now, having mechanisms to recognise the shape of face and frame of images automatically, we can enable the system to produce data-driven surveys in order to rank the most fitted face/frame combinations base on users' perception. Accordingly, such a survey or surveys may be executed by pulling combinations of faces and frames from the first database 970A, obtaining user's votes through a variety of mechanisms including crowdsourcing, social media, dedicated web marketing, dedicated surveys, etc. Accordingly, the ratings of these images from the survey(s) are stored together with the face and eyeframe classifications within the second database 970B from which we can extract the most popular combinations. It would be evident that surveys and their results may be generated for a range of scenarios such that popular combinations are generated and stored for a variety of demographic and / or socioeconomic and / or geographical scenarios. Accordingly, the voting process 960 may be performed in different parts of a single country or region to capture variations within the country and / or region (e.g. California versus Texas, Bavaria versus Schleswig-Holstein, 18- 25 year old Italian men, 55-70 year old North American women, etc.)

[0099] Second portion 900B now addresses the eyeframe recommendation system. When an image 975 is provided of a user without glasses then face shape detection is performed on the image and based upon the results from this a recommendation is made from different types of the glasses available in the system 985. The user is presented with a frame of the recommended type overlapped to their face wherein as their facial profile has been analysed the frame can be aligned and / or scaled relative to their facial image or vice-versa. Optionally, the user may be presented with a couple of recommendations.

[00100] Within the embodiments of the invention described below various efficient and reliable algorithms are used to compute face shapes by comparing selected facial features and their respective inclination of face edges with variants of face shapes in a facial database forming a Face Shape Recognition System (FSRS) forming a module within a VMSVMAP according to embodiments of the invention. The FSRS employs a typical sequence of phases for pattern recognition wherein data sets are required for building categories and to compare similarities between the test data and each category. Accordingly input data is passing through a pre-processing stage of the stored raw data wherein a sequence of data preprocessing operations are applied to the images in order to put them into a suitable format ready for feature extraction. After this each raw data item within the data sets is transformed into a set of features, and the classifier is mainly trained on these feature representations. When a query is received similar data pre-processing is carried out followed by the same sequence of operations and then features are added into the trained classifier. The output of the classifier will be the optimal class label (sometimes with the classification accuracy) or a rejection note (return to manual classification).

[00101] A method employed by the inventors utilizes an analysis of different face shapes based on the boundary points across the edges of the face. The inventors have established that such an algorithm yields good results using a subset of feature points than many other typical applications. The image database used for these experiments was an adequate number of images retrieved from online resources although it would be evident that other databases may be employed such as social media profiles, Government photo-identity databases, etc. A subset of these images were chosen to reduce problems associated with lighting, facial expressions and facial details although it would be evident that embodiments of the invention may be extended to exploit image processing / filtering techniques as described supra with respect to removing background images etc. The image files used were in .PNG format and an initial training data set of over 300 different face shapes were employed that were recognized by domain experts i.e. Fashion Designers, Hair Stylists, etc. In testing data the inventors took 100 new face shapes and 50 randomly selected from training data.

[00102] Embodiments of the invention implemented by the inventors for face shape recognition as a method to identify a person through a photograph of their face employs four modules. The first module in face recognition processing is known as face detection which provides an initial estimate of the location and scale of specified boundary points on the face. The second module in the face recognition processing is face point creation wherein accurate localization is required. The third module for processing is feature extraction, where we seek to extract effective information that will potentially distinguish faces of different persons wherein it is important in this module to obtain stable geometrical representations of the extracted features. In the final module, feature matching, the extracted feature vectors of a particular image are compared to the face images stored in the database to determine the classification to apply to the face in the image. Referring to Figure 10 this process flow of an FSRS system according to an embodiment of the invention is outlined wherein the steps presented are:

• Step 1010 Image is input;

• Step 1020 Face boundary detection;

• Step 1030 Feature extraction;

• Step 1040 Classification; and

• Step 1050 Output facial shape.

[00103] As depicted the classification step 1040 employs a Case Based Reasoning (CBR) 1060 classification methodology comprising steps of retrieve 1080A, re-use 1080B, revise 1080C, and retain 1080D wherein classifications 1070 for the output are defined as being Oval, Round, Square, Oblong, Diamond and Heart. It would be evident other classifications may be employed and that the number of classifications may also be adjusted within other embodiments of the invention.

[00104] Based upon the algorithms and procedures established by the user's with an image acquisition system exploiting a Microsoft™ Kinect™ then the seventeen (17) points established by the Microsoft™ Kinect™ within the 87 two-dimensional (2D) points of features of the user' face were extracted based upon approximation and the use of natural symmetry of the face, which help to identify the features of the face. Referring to Figure 1 1 there is depicted an image of a face after processing in first image 1 100A. Second image 1 100B has been digitally processed using a graphics engine to increase contrast and overlaid with markers allowing the feature points 1 1 10 to be viewed more clearly along with the symmetric boundary 1 120 is depicted with blue curved line. Figure 12 shows the resulting feature points 1 1 10 and boundary 1 120 absent the image indicating their numbering and position.

[00105] Referring to Figure 13 there are depicted representative images of the six facial types employed within the classifications of the FSRS processes according to embodiments of the invention by the inventors. This classification procedure represents the third step 1040 in FSRS process flow depicted in Figure 10. Accordingly, within Figure 13 there are depicted first to sixth images 1310A to 1360A representing oval, round, square, oblong, diamond, and heart face types with their associated extracted geometric feature points as discussed supra and as would be generated by a feature extraction process representing the second step 1030 in FSRS process flow depicted in Figure 10. Also depicted are seventh to twelfth images 131 OB to 1360B representing the same 6 individuals but now their image has been overlaid with generated geometric features exploiting the 17 extracted feature points depicted in first to sixth images 1310A to 1360A respectively. For each face 6 geometric features are depicted, these being:

• First ellipse that best fits at the boundary points;

• Second ellipse that best fits the chin points;

• Third ellipse that best fits the cheek boundary points;

• First line being diagonal lines on lower face;

• Second line being jaw line; and

• Third line being the eye line.

[00106] Based upon the detected 17 points from the face the inventors extract different features. For comparisons within the embodiments of the invention discussed below 8 different face features are considered. Three of these are captured by generating ellipses from different points of the face and others are determined by calculating distances. The 6 main elements of face utilized to distinguish different face shapes are:

[00107] 1. Ellipse Generation. By drawing ellipses with face points, wherein, initially, three ellipses are drawn considering the different facial points:

[00108] (a) A first ellipse fitted such that it accommodates most of the feature points. This ellipse is used to derive the height of the individual's face.

[00109] (b) A second ellipse fitted such that it accommodates only the chin points. This ellipse is used to derive the identify if the chin shape as being pointed (for heart shape) or round (for oblong shape) or other (for square shape);

[00110] (c) A third ellipse that considers the remaining points. This is used to identify if the width of the face is same at eye line, cheekbone and jaw line which is beneficial in identifying if the facial shape is oblong or square.

[00111] 2. Distances. By taking the distances of facial points from drawn ellipses in previous step (1) where these are calculated from the respective ellipse to recognise which type of ellipse covers maximum points. Moreover, a threshold value is set for the distances during training phase. For example, consider the ellipse that covers chin points. If the chin is pointed, then the distances of the chin points from ellipse will be more than the threshold value such that these type of faces can be categorised in heart or diamond face shape. However, if the distances are less than threshold value, then face may be categorized in round or oval face shape.

[00112] 3. Eye Line Length. Eye Line Length is calculated by joining PI and P17. This property length is compared with cheek bone line and jaw line. It will help to determine if these lengths are equal for oval or round faces, or differ substantially as with heart and diamond face shapes.

[00113] 4. Jaw Line Length. Calculation of Jaw line may not be accurate because few face shapes have explicitly defined jaw lines such as round or oval. Therefore, the Jaw Line is defined by averaging 3 different lines that join face points P6 - P12, P7— PI 1 , and P8- P10.

[00114] 5. Symmetry Lines. A series of diagonal line lengths that connect jaw points with face symmetry point P9. Four diagonal lines are drawn from points, P4, P5, PI 3, P14 to point P9 and the lengths of these lines are used to decide if the face is elongated. Moreover, these lines are used to calculate distances of the remaining face points, as explained in the next step.

[00115] 6. Distances of Points from Diagonal Lines. Although the distances from the different ellipses generated in step (1) above are compared to identify the face shapes, results will not be accurate as it may miss the angles present at the jaw line. In order to overcome this limitation, the inventors exploit the diagonal lines generated in step (5) in order to calculate the distances of the face points near the jaw. These distances more accurately determine if the face shape is wider near the jaw.

[00116] Normalization: As the user may input an image of any size the extracted image features may not be directly compared otherwise the results will not be accurate. Although the image is normalized to a specific fixed sized matrix, it cannot avoid the errors with the images having zooming effects (either zoomed in or zoomed out). In order to avoid such type of discrepancies, the natural symmetry of the face is utilized again to determine special features. Such features are described in Table 2.

4 Distance (all) / Width (all) Distance of each facial point from respective ellipses drawn.

5 Distance (chin) / Width (chin)

6 Distance (other) / Width (other)

7 Eye-line / Jaw line Lines extracted from feature extraction steps 3 and 4.

8 Distance from bottom points to Lines extracted from feature extraction diagonal lines / Width (all) step 6.

Table 2: FSRS Normalized Features

[00117] Classification: Within the FSRS according to embodiments of the invention six different types of face shapes are considered for identification. However, in other embodiments of the invention more or less types of face shapes may be employed within the identification process. Within the current embodiment these are Heart, Oblong, Oval, Diamond, Square and Round as depicted in Figure 13. Facial properties for each shape are summarized in Table 3. The variables employed within these determinations are listed Table 4.

line, broad forehead and square 2. L_j « L_ch « L_j

chin. Proportional length and

width.

Table 3: Facial Shape Characteristics and Definitions

Table 4: Facial Comparison Variables

[00118] Once all features have been extracted then the next step is to perform classification based on the retrieved features. The main purpose of the classification is to assign each point in the space with a class label. On the basis of a training set of data containing several observations, we identify to which of a set of categories a new observation belongs. The term "classifier" is an algorithm that implements a classification and a variety of classification methods can be applied depends on the problem definition. Based on the features extracted for the Face Shape recognition, we have used as discussed supra CBR as the classification method to classify face shape into different categories. It would be evident that within other embodiments of the invention that other classification methods may be employed. Referring to Figure 14 there is depicted an exemplary scenario of CBR for FSRS.

[00119] Accordingly, within an embodiment of the invention the inventors defined the distance with the CBR as that given by Equation (3) where T is the feature vector extracted from using the test image, F^k is the feature vector of the k 's element of the training images set, the subscript i denotes the components of the 8 dimensional feature vector and w_j is a weight associated to each of the features in order to compensate for the relative magnitude and importance of each feature. These weights were determined experimentally and were set only once. These wei hts being (1.2,1.0,1.0,1.2,1.1,1.2,1.1,1. l)^r .

[00120] However, in the training only the dominant face shape was considered. Within the training process the inventors employed a set of 100 images extracted from the Internet and evaluated the accuracy of the classification process upon a different set of 300 pictures.

[00121] Within the embodiments of the invention described supra in respect of Figures 9 to 14 a user may classify their facial type and determine appropriate recommendations with respect to eyewear. However, it would be apparent that based upon the characterization, measurement, and assessment of the user's face that additional recommendation engines may exploit the core facial shape determination engine and leverage this to other aspects of the user's appearance and / or purchases. For example, recommendation engines fed from a facial shape determination engine may include those for hairstyles, hair accessories, wigs, sunglasses, hats, earrings, necklaces, beards, moustaches, tattoos, piercings, makeup, etc. Further, it would be evident that the facial shape determination, recommendation engine etc. may be provided to a user locally upon a PED, FED, terminal, etc. or that the user may exploit such engines remotely via a network or the Internet through remote server based provisioning of the required engines, classifications, etc.

[00122] Accordingly, it would be evident to one skilled in the art that similar classifications may be established for other features / body parts / body types etc. exploiting the techniques described above or variants thereof. Similarly, recommendations of other items relative to different classifications may also be provided.

[00123] Similarly, surveys as presented and discussed supra may be supported and enabled through networks such as the Internet either as managed and controlled by manufacturers, providers, third party providers, etc. or as driven through other network based activities such as crowd sourcing, social media, social networks, etc. Such online surveys may rapidly establish results across geographical boundaries, across wide demographics, etc.

[00124] Within embodiments of the invention a user may submit an image, receive a recommendation, and post that to their social network. Their social network may provide feedback that is added to the survey results such that over time the survey results are augmented / expanded. As the embodiments of the invention provide for normalized dimensional analysis of the user's face then the recommendations / simulations of the user with an item of apparel / style etc., e.g. eyewear, may include the appropriately dimensioned augmentation of their uploaded / captured image rather than the crude inaccurate overlays within the prior art. [00125] Within embodiments of the invention the user may, for example, acquire an image upon their PED, transfer this to a remote server via a network for processing and receive back a recommendation. In this manner the classification engine etc. may be remotely hosted reducing the processing requirements upon the user's PED. In embodiments of the invention the overall system may be contextually aware such that the user's location, for example, within an eyewear retailer leads to recommendations driven by the available eyewear frames, for example, available from that retailer. Optionally, the user may establish a profile with a provider of classification based recommendation services such that upon subsequent online visits the user does not to provide an image for facial type analysis as the results of their previous visit(s) are stored against their unique account credentials.

[00126] Within the embodiments of the invention presented and discussed supra in respect of Figures 2 to 29 the recommendations, of eyewear frames for example, are essentially broad classes, e.g. oblong versus round. However, it would be evident to one skilled in the art that the methodologies and embodiments of the invention may be applied at finer granularity with respect to different items, for example eyewear frames, such that, for example, different oblong eyeframes are ranked / rated against a particular facial type. Accordingly, a granular hierarchy of recommendations may be presented to the user such that upon picking, for example, oblong eyeframes they are then presented with recommendations with respect to other aspects of the eyewear frame such as visible frame, invisible frame, colour, arms attaching middle of frame or high on frame, etc. It would be evident that some finer granularity recommendations may require additional characterization of the user's facial profile, e.g. large nose, low nose, wide bridge, high ears, low ears, etc.

[00127] Specific details are given in the above description to provide a thorough understanding of the embodiments. However, it is understood that the embodiments may be practiced without these specific details. For example, circuits may be shown in block diagrams in order not to obscure the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.

[00128] Implementation of the techniques, blocks, steps and means described above may be done in various ways. For example, these techniques, blocks, steps and means may be implemented in hardware, software, or a combination thereof. For a hardware implementation, the processing units may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described above and/or a combination thereof.

[00129] Also, it is noted that the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process is terminated when its operations are completed, but could have additional steps not included in the figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function.

[00130] Furthermore, embodiments may be implemented by hardware, software, scripting languages, firmware, middleware, microcode, hardware description languages and/or any combination thereof. When implemented in software, firmware, middleware, scripting language and/or microcode, the program code or code segments to perform the necessary tasks may be stored in a machine readable medium, such as a storage medium. A code segment or machine-executable instruction may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a script, a class, or any combination of instructions, data structures and/or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters and/or memory content. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[00131] For a firmware and/or software implementation, the methodologies may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. Any machine-readable medium tangibly embodying instructions may be used in implementing the methodologies described herein. For example, software codes may be stored in a memory. Memory may be implemented within the processor or external to the processor and may vary in implementation where the memory is employed in storing software codes for subsequent execution to that when the memory is employed in executing the software codes. As used herein the term "memory" refers to any type of long term, short term, volatile, nonvolatile, or other storage medium and is not to be limited to any particular type of memory or number of memories, or type of media upon which memory is stored.

[00132] Moreover, as disclosed herein, the term "storage medium" may represent one or more devices for storing data, including read only memory (ROM), random access memory (RAM), magnetic RAM, core memory, magnetic disk storage mediums, optical storage mediums, flash memory devices and/or other machine readable mediums for storing information. The term "machine-readable medium" includes, but is not limited to portable or fixed storage devices, optical storage devices, wireless channels and/or various other mediums capable of storing, containing or carrying instruction(s) and/or data.

[00133] The methodologies described herein are, in one or more embodiments, performable by a machine which includes one or more processors that accept code segments containing instructions. For any of the methods described herein, when the instructions are executed by the machine, the machine performs the method. Any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine are included. Thus, a typical machine may be exemplified by a typical processing system that includes one or more processors. Each processor may include one or more of a CPU, a graphics-processing unit, and a programmable DSP unit. The processing system further may include a memory subsystem including main RAM and/or a static RAM, and/or ROM. A bus subsystem may be included for communicating between the components. If the processing system requires a display, such a display may be included, e.g., a liquid crystal display (LCD). If manual data entry is required, the processing system also includes an input device such as one or more of an alphanumeric input unit such as a keyboard, a pointing control device such as a mouse, and so forth.

[00134] The memory includes machine-readable code segments (e.g. software or software code) including instructions for performing, when executed by the processing system, one of more of the methods described herein. The software may reside entirely in the memory, or may also reside, completely or at least partially, within the RAM and/or within the processor during execution thereof by the computer system. Thus, the memory and the processor also constitute a system comprising machine-readable code.

[00135] In alternative embodiments, the machine operates as a standalone device or may be connected, e.g., networked to other machines, in a networked deployment, the machine may operate in the capacity of a server or a client machine in server-client network environment, or as a peer machine in a peer-to-peer or distributed network environment. The machine may be, for example, a computer, a server, a cluster of servers, a cluster of computers, a web appliance, a distributed computing environment, a cloud computing environment, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. The term "machine" may also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[00136] The foregoing disclosure of the exemplary embodiments of the present invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many variations and modifications of the embodiments described herein will be apparent to one of ordinary skill in the art in light of the above disclosure. The scope of the invention is to be defined only by the claims appended hereto, and by their equivalents.

[00137] Further, in describing representative embodiments of the present invention, the specification may have presented the method and/or process of the present invention as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps set forth herein, the method or process should not be limited to the particular sequence of steps described. As one of ordinary skill in the art would appreciate, other sequences of steps may be possible. Therefore, the particular order of the steps set forth in the specification should not be construed as limitations on the claims. In addition, the claims directed to the method and/or process of the present invention should not be limited to the performance of their steps in the order written, and one skilled in the art can readily appreciate that the sequences may be varied and still remain within the spirit and scope of the present invention.

Claims

CLAIMS What is claimed is:

1. A method comprising:

acquiring an image with an image acquisition system providing location and depth information relating to a predetermined portion of a user's body forming a predetermined portion of the image;

processing the location and depth information to generate a user model of the predetermined portion of a user, the user model comprising at least one of a wireframe map and a contour map and a plurality of metrics relating to the predetermined portion of a user from the location and depth information;

receiving from the user identification of an item;

retrieving an item model of the item, the item model comprising at least one of a wireframe map and a contour map and a plurality of metrics relating to physical characteristics of the item;

generating a deployable model of the item in dependence upon at least the item model and the user model;

positioning the deployable model relative to the appropriate portion of the user's body;

rendering the deployable model as part of a virtual mirror image presented to the user, the virtual mirror image comprising the rendered deployable model overlaid to the image.

2. The method according to claim 1, wherein positioning the deployable model comprises at least one of receiving an indication of the position from the user and automatically establishing the position in dependence upon the user model, the item and the deployable model.

3. The method according to claim 1, wherein receiving from the user an identification of an item comprises receiving a selection made by the user from a plurality of variants of the item provided to the user based upon a recommendation process.

4. The method according to claim 3, wherein the recommendation process comprises establishing a type associated with the predetermined portion of the user's body and selecting a recommended item in dependence upon the determined type and the results of a survey executed in respect of a plurality of classes of the item and a plurality of types including the predetermined type.

5. The method according to claim 1, further comprising;

modifying the positioning of the deployable model relative to the appropriate portion of the user's body based upon applying a physical modelling, the physical modelling employing a predetermined subset of the plurality of metrics relating to physical characteristics of the item.

6. A system comprising:

an image acquisition system for acquiring an image providing location and depth information relating to a predetermined portion of a user's body forming a predetermined portion of the image;

a first microprocessor forming part of a first electronic device for processing the location and depth information to generate a user model of the predetermined portion of a user, the user model comprising at least one of a wireframe map and a contour map and a plurality of metrics relating to the predetermined portion of a user from the location and depth information;

a user interface forming part of a second electronic device for receiving from the user identification of an item;

a modelling module forming part of a third electronic device, the modelling module for:

retrieving an item model of the item, the item model comprising at least one of a wireframe map and a contour map and a plurality of metrics relating to physical characteristics of the item;

generating a deployable model of the item in dependence upon at least the item model and the user model;

positioning the deployable model relative to the appropriate portion of the user's body; and rendering the deployable model as part of a virtual mirror image presented to the user, the virtual mirror image comprising the rendered deployable model overlaid to the image; and

a display forming part of a fourth electronic device for displaying to the user the virtual mirror image.

7. The system according to claim 6, wherein the modelling module positions the deployable model based upon at least one of receiving an indication of the position from the user and automatically establishing the position in dependence upon the user model, the item and the deployable model.

8. The system according to claim 6, wherein receiving from the user an identification of an item comprises receiving a selection made by the user from a plurality of variants of the item provided to the user upon the display based upon a recommendation process.

9. The system according to claim 8, wherein the recommendation process comprises establishing a type associated with the predetermined portion of the user's body and selecting a recommended item in dependence upon the determined type and the results of a survey executed in respect of a plurality of classes of the item and a plurality of types including the predetermined type.

10. The system according to claim 6, further comprising;

a physical modelling module for modifying the position of the deployable model relative to the appropriate portion of the user's body based upon applying a physical process to the item, the physical process employing a predetermined subset of the plurality of metrics relating to physical characteristics of the item.

1 1. Executable software stored upon a non-transient physical medium, wherein the executable software when executed provides a user with a virtual mirror through a series of modules, the series of modules including

a first module for acquiring image data relating to an image from an image acquisition system, the image data including location and depth information relating to a predetermined portion of a user's body forming a predetermined portion of the image;

second module providing for boundary detection based upon at least one of the acquired image data and the image, the boundary detection establishing a predetermined number of boundary points relating to the predetermined portion of the user's body; third module providing for feature extraction based upon the boundary points established by the second module, the feature extraction comprising the generation of predetermined geometric shapes based upon first subsets of the boundary points and dimensions established upon second subsets of the boundary points;

classification module for determining a type of the predetermined portion of the user's body in dependence upon the features extracted by the third module and a plurality of datasets, each of the datasets established from feature extraction performed upon a training set of images relating to a defined type of a plurality of defined types of the predetermined portion of the user's body;

recommendation engine for recommending an item to the user based upon the determined type of the predetermined portion of the user's body and the results of a survey executed in respect of the plurality of defined types of the predetermined portion of the user's body relating to the type of item;

modelling engine for retrieving an item model of the item comprising at least one of a wireframe map and a contour map and a plurality of metrics relating to physical characteristics of the item and for generating a deployable model of the item in dependence upon at least the item model and the dimensions established by the third module and positioning the deployable model relative to the predetermined portion of the user's body; and

rendering engine for rendering the deployable model as part of virtual mirror image for presentation to the user comprising the rendered deployable model in the determined position overlaid to the image acquired by the first module.