WO2024062912A1 - Display control device, method for operating display control device, and program for operating display control device - Google Patents

Display control device, method for operating display control device, and program for operating display control device Download PDF

Info

Publication number
WO2024062912A1
WO2024062912A1 PCT/JP2023/032375 JP2023032375W WO2024062912A1 WO 2024062912 A1 WO2024062912 A1 WO 2024062912A1 JP 2023032375 W JP2023032375 W JP 2023032375W WO 2024062912 A1 WO2024062912 A1 WO 2024062912A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
images
feature
display
control device
Prior art date
Application number
PCT/JP2023/032375
Other languages
French (fr)
Japanese (ja)
Inventor
正志 藏之下
Original Assignee
富士フイルム株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 富士フイルム株式会社 filed Critical 富士フイルム株式会社
Publication of WO2024062912A1 publication Critical patent/WO2024062912A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing

Definitions

  • the technology of the present disclosure relates to a display control device, a display control device operating method, and a display control device operating program.
  • Japanese Patent Laid-Open No. 2001-005822 describes a method for displaying images of a group of search results obtained as a result of searching an image database by an arbitrary method.
  • the display method is equipped with a database in which feature values for a group of images in the entire image database are associated with keywords or symbols that can be assigned to images and image feature values through statistical processing.
  • a maximum of a specified number of keywords or symbol features that express differences between features are dynamically and automatically selected for each search, in order of sensitivity to differences between the features.
  • Japanese Patent Application Laid-Open No. 05-282375 discloses a method for searching for images with arbitrary dimension attribute information from an image database that stores images with n-dimensional attribute information, and displaying the searched images in a reduced size.
  • An image database system is described in which a device performs browsing display, and when an image of interest is selected from among the displayed images, that image is displayed on the entire screen of the display device.
  • the image database system searches for images that have features within a specified distance space from each feature of the focused two-dimensional attribute information of the sample image.
  • the browsing display is performed using coordinates of two-dimensional attribute information.
  • Japanese Patent Application Publication No. 2003-271665 describes a graphical user interface for searching that includes a first display control means, a movement command means, and a second display control means.
  • the first display control means includes a coordinate axis on which a scale is formed, an area for displaying search results of target information corresponding to a search target range set corresponding to the scale, and an operator located at a reference position on the coordinate axis.
  • the display device is controlled to display on the display screen.
  • the movement command means gives a horizontal or vertical movement command to the operator.
  • the second display control means changes the scale of the scale when the movement command means gives the operator a movement command in either the horizontal or vertical direction, and the movement command means changes the scale of the scale to the operator.
  • the display of the coordinate axes is controlled so that the values of the scales are changed when a movement command is given in either the horizontal direction or the vertical direction.
  • One embodiment of the technology of the present disclosure provides a display control device capable of displaying a list of a wide variety of images, a method for operating the display control device, and an operating program for the display control device.
  • a display control device of the present disclosure includes a processor, the processor acquires a plurality of images, acquires a plurality of types of feature amounts for the plurality of images, and displays a two-dimensional or three-dimensional display space related to the feature amounts. , a plurality of images are displayed as a list by arranging the images at positions corresponding to the feature amounts in a display space that has at least one axis that indicates the magnitude of a numerical value that integrates two or more types of feature amounts.
  • the processor receives a user's instruction to change the specifications of the display space, and changes the specifications of the display space in accordance with the change instruction.
  • the change instruction is an instruction to change a weighting coefficient that determines the ratio of integrating two or more types of feature amounts.
  • the change instruction is an instruction to change the feature amount that constitutes the axis.
  • the processor displays the representative image of the two or more images at the top, and displays the other images below the representative image. It is preferable to display in a layered manner.
  • the processor receives an instruction to select an image from the user on the display space, and performs specified processing only on the image selected by the selection instruction.
  • the processor receives a user's selection instruction for at least one of the similar images and the dissimilar images, and, in accordance with the selection instruction, moves the similar images in a direction in which the feature amounts match, and the dissimilar images in a direction in which the feature amounts match. It is preferable to change the weighting coefficients applied to the plurality of types of feature quantities in the direction in which the quantities deviate.
  • the operating method of the display control device disclosed herein includes acquiring a plurality of images, acquiring a plurality of types of feature quantities for the plurality of images, and displaying a list of the plurality of images by arranging the images at positions according to the feature quantities in a two-dimensional or three-dimensional display space relating to the feature quantities and having at least one axis indicating the magnitude of a numerical value obtained by integrating two or more types of feature quantities.
  • the operating program of the display control device of the present disclosure includes acquiring a plurality of images, acquiring a plurality of types of feature amounts for the plurality of images, and a two-dimensional or three-dimensional display space related to the feature amounts. , displaying a plurality of images as a list by arranging the images at positions corresponding to the feature amounts in a display space having at least one axis indicating the magnitude of a numerical value that integrates two or more types of feature amounts; cause a computer to perform processing including
  • FIG. 2 is a diagram showing a user terminal and an image management server.
  • FIG. 2 is a block diagram showing a computer that constitutes a user terminal and an image management server.
  • FIG. 2 is a block diagram showing a processing unit of a CPU of a user terminal.
  • FIG. 2 is a block diagram showing a processing unit of a CPU of an image management server. It is a diagram showing data stored in an image DB. It is a diagram showing supplementary information.
  • FIG. 3 is a diagram showing feature amount information and feature amount vectors.
  • FIG. 2 is an explanatory diagram of a method for deriving date and time features.
  • FIG. 13 is a diagram showing a process of deriving image quality features using an image quality feature model.
  • FIG. 13 is a diagram showing a process of deriving image quality features using an image quality feature model.
  • FIG. 3 is a diagram illustrating a process of deriving model features using a model feature derivation model.
  • FIG. 7 is a diagram illustrating a process of deriving a subject feature using a subject feature deriving model.
  • FIG. 3 is a diagram showing a subject discrimination model.
  • FIG. 3 is a diagram showing the processing of each processing unit of the image management server when an image storage request is sent from a user terminal.
  • FIG. 3 is a diagram showing a display setting screen. 13 is a diagram showing a process of the browser control unit when a setting button is pressed on the display setting screen.
  • FIG. FIG. 6 is a diagram illustrating the processing of each processing unit of the image management server when an image distribution request is transmitted from a user terminal.
  • FIG. 6 is a diagram illustrating a process of arranging an image at a position in a display space according to a feature amount.
  • FIG. 7 is a diagram illustrating a process of displaying images whose feature amounts are within a preset range in a layered manner.
  • FIG. 3 is a diagram showing an image list display screen.
  • FIG. 7 is a diagram illustrating a process of changing the specifications of a display space in response to an instruction to change feature amounts that constitute an axis.
  • FIG. 7 is a diagram illustrating a process of changing the specifications of a display space in response to an instruction to change feature amounts that constitute an axis.
  • FIG. 3 is a diagram illustrating how a user instructs to select an image in a display space.
  • FIG. 6 is a diagram illustrating a state in which a processing instruction menu for instructing processing on an image selected by a selection instruction is displayed.
  • FIG. 3 is a diagram showing an album creation screen on which an album created from images selected by a selection instruction is displayed.
  • 5 is a flowchart showing the processing procedure of the image management server. 5 is a flowchart showing the processing procedure of the image management server. It is a flowchart which shows the processing procedure of a user terminal.
  • FIG. 7 is a diagram illustrating an image list display screen in which the feature amounts configuring an axis can be changed using a pull-down menu.
  • FIG. 7 is a diagram showing a display setting screen having a function of instructing a change in weighting coefficients.
  • FIG. 6 is a diagram illustrating a process of arranging an image at a position in a display space according to a feature amount.
  • FIG. 7 is a diagram showing how an image is moved to an arbitrary position in the display space.
  • FIG. 7 is a diagram showing an image list display screen including a display space in which other images are arranged around one image moved to an arbitrary position.
  • FIG. 7 is a diagram showing an image list display screen including a display space in which other images are arranged around two images moved to arbitrary positions. It is a figure showing a similar image selection screen.
  • FIG. 13 is a diagram illustrating a process of correcting weighting coefficients by solving an optimization problem for obtaining weighting coefficients that make the distance of feature vectors of similar images less than a first threshold value.
  • FIG. 7 is a diagram illustrating a process of correcting weighting coefficients by solving an optimization problem for finding weighting coefficients for which the distance between feature vectors of dissimilar images is equal to or greater than a second threshold
  • FIG. 3 is a diagram showing an example of arranging images in a three-dimensional display space.
  • a user U owns a user terminal 10.
  • the user terminal 10 is a device having a camera function, an image playback/display function, an image transmission/reception function, and the like.
  • the camera function of the user terminal 10 has an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, and forms an image of the object 46 (see FIG. 5) by capturing object light from a lens and focusing it on the image sensor. get.
  • the user terminal 10 is a smartphone, a tablet terminal, a compact digital camera, a mirrorless single-lens camera, a notebook personal computer, or the like.
  • the user terminal 10 is an example of a "display control device" according to the technology of the present disclosure.
  • the user terminal 10 is connected to the image management server 12 via the network 11 so that they can communicate with each other.
  • the network 11 is, for example, a WAN (Wide Area Network) such as the Internet or a public communication network.
  • the user terminal 10 transmits (uploads) the image 46 to the image management server 12. Further, the user terminal 10 receives (downloads) an image 46 from the image management server 12.
  • the image management server 12 is, for example, a server computer, a workstation, etc., and together with the user terminal 10 is an example of a "display control device" according to the technology of the present disclosure. In this way, the "display control device" according to the technology of the present disclosure may be realized across multiple devices.
  • a plurality of user terminals 10 of a plurality of users U are connected to the image management server 12 via a network 11.
  • the computers configuring the user terminal 10 and the image management server 12 basically have the same configuration, including a storage 20, a memory 21, a CPU (Central Processing Unit) 22, a communication unit 23, It includes a display 24 and an input device 25. These are interconnected via a bus line 26.
  • a storage 20 a storage 20
  • a memory 21 a main memory 22
  • a CPU 22 Central Processing Unit 22
  • a communication unit 23 It includes a display 24 and an input device 25.
  • a bus line 26 interconnected via a bus line 26.
  • the storage 20 is a hard disk drive that is built into a computer that constitutes the user terminal 10 and the image management server 12, or is connected via a cable or a network.
  • the storage 20 is a disk array in which a plurality of hard disk drives are connected in series.
  • the storage 20 stores control programs such as an operating system, various application programs (hereinafter abbreviated as AP (Application Program)), and various data accompanying these programs.
  • AP Application Program
  • a solid state drive may be used instead of the hard disk drive.
  • the memory 21 is a work memory for the CPU 22 to execute processing.
  • the CPU 22 loads the program stored in the storage 20 into the memory 21 and executes processing according to the program. Thereby, the CPU 22 centrally controls each part of the computer.
  • the CPU 22 is an example of a "processor" according to the technology of the present disclosure. Note that the memory 21 may be built into the CPU 22.
  • the communication unit 23 is a network interface that controls transmission of various information via the network 11 and the like.
  • the display 24 displays various screens. Various screens are provided with operation functions using a GUI (Graphical User Interface).
  • the computers constituting the user terminal 10 and the image management server 12 accept input of operation instructions from the input device 25 through various screens.
  • the input device 25 is a keyboard, a mouse, a touch panel, a microphone for voice input, or the like.
  • each part of the computer that makes up the user terminal 10 storage 20, CPU 22, display 24, and input device 25
  • each part of the computer that makes up the image management server 12 is indicated by the suffix "A”
  • each part of the computer that makes up the image management server 12 is indicated by the suffix "A.”
  • Storage 20 and CPU 22 are distinguished by adding a suffix "B" to their respective symbols.
  • an image AP30 is stored in the storage 20A of the user terminal 10.
  • Image AP 30 is installed on user terminal 10 by user U.
  • the image AP 30 is an AP for causing a computer forming the user terminal 10 to function as a "display control device" according to the technology of the present disclosure. That is, the image AP30 is an example of the "display control device operation program" according to the technology of the present disclosure.
  • the CPU 22A of the user terminal 10 functions as the browser control unit 32 in cooperation with the memory 21 and the like.
  • the browser control unit 32 controls the operation of the dedicated web browser of the image AP 30.
  • the browser control unit 32 generates various screens.
  • the browser control unit 32 displays various generated screens on the display 24A.
  • the browser control unit 32 also receives various operation instructions input by the user U from the input device 25A through various screens.
  • the browser control unit 32 sends various requests to the image management server 12 according to operation instructions.
  • an operating program 35 is stored in the storage 20B of the image management server 12.
  • the operating program 35 is an AP for causing the computer forming the image management server 12 to function as a "display control device" according to the technology of the present disclosure. That is, like the image AP 30, the operation program 35 is an example of the "operation program for the display control device" according to the technology of the present disclosure.
  • An image database (hereinafter referred to as DB (Data Base)) 36 is also stored in the storage 20B.
  • the storage 20B contains a user ID (Identification Data) for uniquely identifying the user U, a password set by the user U, and a terminal ID for uniquely identifying the user terminal 10. is stored as user U's account information.
  • the CPU 22B of the image management server 12 cooperates with the memory 21 and the like to execute the reception unit 40, feature value derivation unit 41, and read/write (hereinafter referred to as RW (Read Write)). It functions as a control section 42 and a distribution control section 43.
  • the reception unit 40 receives various requests from the user terminal 10.
  • the reception unit 40 outputs the various requests to the feature derivation unit 41, the RW control unit 42, and the distribution control unit 43.
  • the feature derivation unit 41 derives multiple types of features of the image 46.
  • the feature derivation unit 41 outputs feature information 48 (see FIG. 5) consisting of the multiple types of derived features to the RW control unit 42.
  • the RW control unit 42 controls storage of various data in the storage 20B and reading of various data from the storage 20B.
  • the RW control unit 42 particularly controls the storage of the image 46 in the image DB 36 and the reading of the image 46 from the image DB 36. Further, the RW control unit 42 controls the storage of the feature amount information 48 in the image DB 36 and the reading of the feature amount information 48 from the image DB 36.
  • the distribution control unit 43 controls distribution of various data to the user terminal 10.
  • a storage area 45 is provided for each user U in the image DB 36.
  • a user ID is registered in the storage area 45.
  • an image 46, additional information 47, and feature amount information 48 are stored in association with each other.
  • the additional information 47 includes multiple items such as shooting date and time, shooting location, aperture value, ISO (International Organization for Standardization) sensitivity, shutter speed, focal length, presence or absence of flash, and tags. .
  • the date and time when the image 46 was photographed using the camera function of the user terminal 10 is registered in the photographing date and time.
  • an address and/or a landmark name determined from latitude and longitude information obtained by the GPS (Global Positioning System) function of the user terminal 10 is registered.
  • the tag is a word that simply represents the subject in the image 46.
  • the tags include those manually input by the user U or those derived using the subject discrimination model 60 (see FIG. 12).
  • the feature information 48 includes a plurality of items such as a date and time feature, a color feature, a brightness feature, an image quality feature, a model feature, and a subject feature.
  • the image 46 can be represented by a feature vector VI in which a plurality of types of feature amounts constituting the feature information 48 are listed. There are hundreds to thousands of feature amounts included in the feature information 48. Therefore, the number of dimensions of the feature quantity vector VI also ranges from several hundred to several thousand.
  • the date and time feature is a feature related to the date and time when the image 46 was taken.
  • the feature amount deriving unit 41 sets the date and time feature amount to 0 for the image 46O with the oldest shooting date and time among the images 46 stored in the storage area 45, and the date and time feature amount of the image 46O with the latest shooting date and time.
  • the date and time feature of image 46N is derived.
  • the color feature amount is a feature amount related to each color of RGB (red, green, and blue) of the image 46.
  • the feature amount deriving unit 41 derives a value obtained by normalizing the average value of the pixel values of each color of RGB of the image 46, for example, with the maximum value of the pixel value being 1, as the color feature amount.
  • the brightness feature amount is a feature amount related to the brightness (luminance) of the image 46.
  • the feature amount deriving unit 41 derives a value obtained by normalizing the average brightness value of the image 46, with the maximum brightness value being 1, as a brightness feature amount.
  • the image quality feature quantity is a feature quantity related to the image quality of the image 46.
  • the image quality feature amount is a numerical value of 0 or more and 1 or less. The closer the image quality feature quantity is to 0, the lower the image quality of the image 46 is, and the closer the image quality feature quantity is to 1, the higher the image quality of the image 46 is.
  • the feature amount derivation unit 41 derives the image quality feature amount using the image quality feature amount derivation model 55.
  • the image quality feature amount derivation model 55 is a machine learning model that outputs an image quality feature amount in response to the input of the image 46.
  • the correct image quality is used to match the learning image with the learning image quality feature output from the image quality feature derivation model 55 in response to the input of the learning image.
  • the feature amounts are prepared as training data.
  • the correct image quality features are calculated based on the results of the training data creator's evaluation of multiple evaluation items such as sharpness, exposure, gradation expression, saturation, white balance, presence of blur, and presence of blur. Ru.
  • Model features are features obtained from machine learning models. Like other feature quantities, the model feature quantity is also a value normalized with a maximum value of 1.
  • the feature amount derivation unit 41 derives the model feature amount using the model feature amount derivation model 56.
  • the model feature derivation model 56 is a machine learning model that outputs model features in response to the input of the image 46.
  • the subject feature quantity is a feature quantity that indicates the existence probability of a subject appearing in the image 46 as a numerical value between 0 and 1 for each type of subject.
  • the subject includes everything that can be the subject of the image 46, in addition to the illustrated mountains, rivers, oceans, lakes, men, women, etc.
  • the feature amount derivation unit 41 derives the object feature amount using the object feature amount derivation model 57.
  • the subject feature amount derivation model 57 is a machine learning model that outputs a subject feature amount in response to the input of the image 46.
  • the image quality feature derivation model 55, model feature derivation model 56, and subject feature derivation model 57 are stored in the storage 20B.
  • the RW control unit 42 reads these derived models 55 to 57 from the storage 20B and outputs them to the feature amount deriving unit 41.
  • the object discrimination model 60 is, for example, a convolutional neural network (CNN) such as U-Net or ResNet (Residual Network), and includes an encoder section 61 and an output section 62.
  • An image 46 is input to the encoder section 61 .
  • the encoder unit 61 derives the feature amount 63 by performing well-known convolution processing using a filter, pooling processing, skip layer processing, etc. on the image 46.
  • the encoder section 61 outputs the feature amount 63 to the output section 62.
  • the feature amount 63 is nothing but a model feature amount. In other words, the encoder unit 61 is used as the model feature derivation model 56.
  • the output section 62 includes a decoder section 64, a probability calculation section 65, and a subject discrimination section 66.
  • the feature amount 63 is input to the decoder section 64 .
  • the decoder unit 64 derives the final feature amount 63E by subjecting the feature amount 63 to upsampling processing accompanied by convolution processing, merging processing, and the like.
  • the probability calculation unit 65 generates a probability calculation result 67 from the final feature amount 63E using a well-known activation function such as a softmax function or a sigmoid function.
  • the probability calculation section 65 outputs the probability calculation result 67 to the subject discrimination section 66.
  • the probability calculation result 67 is the existence probability of each subject such as a mountain, a river, the sea, and a lake in the image 46.
  • the probability calculation result 67 is nothing but a subject feature amount. That is, the encoder section 61, the decoder section 64, and the probability calculation section 65 are used as the subject feature derivation model 57.
  • the subject discrimination unit 66 outputs the subject with the highest existence probability in the probability calculation result 67 as the subject discrimination result 68.
  • a part of the class discrimination model called the subject discrimination model 60 is used as the model feature derivation model 56 and the subject feature derivation model 57
  • the present invention is not limited to this.
  • An encoder section of an autoencoder that outputs a restored image of the image 46 in response to the input of the image 46 may be used as the model feature derivation model 56.
  • the encoder section, decoder section, and probability calculation section of a semantic segmentation model that identifies a subject in the image 46 in pixel units may be used as the subject feature derivation model 57.
  • the browser control unit 32 transmits an image storage request 70 to the image management server 12.
  • the image storage request 70 includes a user ID, an image 46, and additional information 47.
  • the reception unit 40 accepts the image storage request 70 and outputs the image storage request 70 to the feature quantity derivation unit 41 and the RW control unit 42.
  • the feature amount deriving unit 41 derives various feature amounts as shown in FIGS. 8 to 11, and outputs the obtained feature amount information 48 to the RW control unit 42.
  • the RW control unit 42 stores the image 46, additional information 47, and feature amount information 48 in the storage area 45 of the image DB 36 corresponding to the user ID.
  • the image management server 12 acquires a plurality of images 46 and acquires a plurality of types of feature amounts for the plurality of images 46.
  • the browser control unit 32 displays a display setting screen 75 for displaying a list of images 46 on the display 24A in response to an instruction from the user U.
  • the display setting screen 75 includes a pull-down menu 76 for selecting the feature amount forming the X-axis of the two-dimensional display space 90 (see FIG. 17) used for displaying a list of images 46, and a pull-down menu 76 for selecting the feature amount forming the Y-axis.
  • a pull-down menu 77 and a settings button 78 are provided for selection.
  • Two pull-down menus 76 and 77 are provided in the initial display state.
  • feature amount addition buttons 79 and 80 are provided at the bottom of the pull-down menus 76 and 77.
  • Features that can be selected from the pull-down menus 76 and 77 are date and time features, color features, brightness features, image quality features, and subject features.
  • the plurality of pull-down menus 76 only one of the pull-down menus 76 must select some feature amount.
  • the X-axis and Y-axis of the display space 90 serve as axes that indicate the numerical value of at least one type of feature amount.
  • the X axis of the display space 90 becomes an axis indicating the magnitude of a numerical value that integrates two or more types of feature quantities.
  • the Y-axis of the display space 90 becomes an axis indicating the magnitude of a numerical value that integrates two or more types of feature quantities.
  • the browser control unit 32 when the setting button 78 is pressed on the display setting screen 75, the browser control unit 32 generates an image distribution request 85.
  • the image distribution request 85 includes the user ID and information according to the feature amount selected from the pull-down menus 76 and 77.
  • the information corresponding to the feature amount is a keyword representing the object feature amount when the feature amount selected from the pull-down menus 76 and 77 is the object feature amount.
  • FIG. 15 a case is illustrated in which subject features of a dog and a child are selected in the pull-down menu 76, and an image distribution request 85 including the keywords dog and child is generated.
  • period designation information that specifies images 46 from a preset period such as the past year is used. is registered in the image distribution request 85 as information corresponding to the feature amount.
  • the browser control unit 32 transmits an image distribution request 85 to the image management server 12.
  • the reception unit 40 accepts the image distribution request 85 and outputs the image distribution request 85 to the RW control unit 42 .
  • the RW control unit 42 searches for an image 46 corresponding to information according to the feature amount of the image distribution request 85 from among the images 46 stored in the storage area 45 corresponding to the user ID of the image distribution request 85.
  • the RW control unit 42 outputs the searched image 46, its accompanying information 47 (not shown in FIG. 16), and feature amount information 48 to the distribution control unit 43.
  • the distribution control unit 43 distributes the image 46 and the like from the RW control unit 42 to the user terminal 10 that has requested the image distribution request 85 .
  • the distribution control unit 43 identifies the user terminal 10 that is the source of the image distribution request 85 based on the user ID included in the image distribution request 85.
  • FIG. 16 shows an example of the image distribution request 85 in FIG. 15 that includes the words dog and child as keywords.
  • the RW control unit 42 searches for an image 46 in which the subject feature amount of the dog is greater than 0, and an image 46 in which the subject feature amount of the child is greater than 0.
  • the browser control unit 32 calculates an integrated feature IF for an image 46 delivered from the image management server 12 in response to an image delivery request 85.
  • the integrated feature IF is an arithmetic average value of the multiple types of feature to be integrated.
  • FIG. 17 illustrates an example of an image delivery request 85 including keywords for dog and child as shown in FIG. 15 and FIG. 16.
  • the browser control unit 32 calculates the arithmetic average value of the subject feature of the dog and the subject feature of the child as the integrated feature IF.
  • the browser control unit 32 calculates the integrated feature IF as the arithmetic average value.
  • the browser control unit 32 places the image 46 at a position according to the feature in a two-dimensional display space 90 related to the feature having an X axis and a Y axis. When images 46 overlap in the display space 90, the browser control unit 32 displays the image 46 with the most recent shooting date and time in front.
  • a position according to the feature in the display space 90 refers to a position according to at least three types of feature out of the hundreds to thousands of types of feature, and does not necessarily have to be a position according to all of the hundreds to thousands of types of feature.
  • the X axis is an axis indicating a numerical value that integrates the subject feature quantities of the dog and the child, that is, the integrated feature quantity IF
  • the Y axis is the axis indicating the image quality feature quantity.
  • the X-axis is an example of "an axis that indicates the magnitude of a numerical value that integrates two or more types of feature amounts" according to the technology of the present disclosure. Note that not only the X-axis but also the Y-axis may be an axis indicating the magnitude of a numerical value that integrates two or more types of feature amounts.
  • the browser control unit 32 places the image 461 whose integrated feature quantity IF is 0.345 and whose image quality feature quantity is 0.88 at the position of coordinates ⁇ 0.345, 0.88 ⁇ in the display space 90. Similarly, the browser control unit 32 places the image 462 whose integrated feature quantity IF is 0.43 and whose image quality feature quantity is 0.56 at the position of coordinates ⁇ 0.43, 0.56 ⁇ in the display space 90. . Further, the browser control unit 32 arranges the image 463 whose integrated feature amount IF is 0.48 and whose image quality feature amount is 0.72 at the position of coordinates ⁇ 0.48, 0.72 ⁇ in the display space 90. In the display space 90 of FIG.
  • the image 46 has a relatively high subject feature of the dog and a child, and has a relatively high image quality feature, in other words, the dog and/or the child are clearly the subject.
  • the image 46 that appears more clearly and has better image quality is placed in the upper right area.
  • the browser control unit 32 displays a list of the plurality of images 46 by arranging the images 46 at positions corresponding to the feature amounts in the display space 90 in this manner.
  • the browser control unit 32 selects the most representative image 46RP of the two or more images 46. Display at the top. Then, the other images 46 are displayed in a stacked manner under the representative image 46RP.
  • the representative image 46RP is, for example, the image 46 with the oldest photographing date and time among the two or more images 46 whose feature amounts are within a preset range.
  • two or more images 46 whose feature amounts are within a preset range are images 46 whose feature amount vectors VI are less than a preset threshold.
  • the threshold value is set to a value that allows continuous shots taken at intervals of several milliseconds to several seconds to be determined as an image 46 whose feature amount falls within a preset range.
  • FIG. 18 shows an example in which three images 46 whose feature amounts are within a preset range are collectively displayed in a stacked manner as a representative image 46RP.
  • the browser control unit 32 displays an image list display screen 95 including a display space 90 in which a plurality of images 46 are arranged on the display 24A.
  • a setting change button 96 is provided on the image list display screen 95. When the settings change button 96 is pressed, the browser control unit 32 changes the display from the image list display screen 95 to the display settings screen 75.
  • FIG. 20 and 21 show an example of a process for changing the specifications of the display space 90 in response to an instruction to change the feature quantities constituting the axes of the display space 90.
  • FIG. 20 shows a case where the setting change button 96 is pressed on the image list display screen 95 including the display space 90 shown in FIG. 19, and furthermore, on the display setting screen 75, the feature quantities constituting the X-axis are changed to date and time features (the Y-axis remains unchanged as image quality features).
  • the browser control unit 32 rearranges the images 46 in the display space 90 with the X-axis being the date and time features and the Y-axis being the image quality features, and displays on the display 24A the image list display screen 95 including the display space 90 with the X-axis being the date and time features and the Y-axis being the image quality features.
  • the images 46 with relatively high date and time features and relatively high image quality features are arranged in the upper right area.
  • the browser control unit 32 rearranges the images 46 in the display space 90 where the X-axis is the subject feature quantity of alpine plants and the Y-axis is the red feature quantity, and displays the image list display screen 95 including the display space 90 where the X-axis is the subject feature quantity of alpine plants and the Y-axis is the red feature quantity on the display 24A.
  • images 46 with relatively high subject feature quantities of alpine plants and relatively high red feature quantities, in other words images 46 in which red alpine plants are clearly depicted as subjects, are arranged in the upper right region.
  • the browser control unit 32 receives an instruction to change the feature amount forming the axis of the display space 90 as an instruction from the user U to change the specifications of the display space 90. Then, the specifications of the display space 90 are changed according to the change instruction.
  • FIGS. 20 and 21 show an example in which the feature quantities configuring the X-axis and Y-axis are changed to one type, it is of course possible to change the feature quantities configuring the X-axis and Y-axis to multiple types. Good too.
  • FIG. 22 illustrates a case where the display 24A is a touch panel display.
  • the user U specifies the selection area 100 by tracing the surface of the display 24A with the finger F.
  • the image 46 existing within the designated selection area 100 is thus selected.
  • the image 46 selected by the user U will be referred to as a selected image 46S.
  • a processing instruction menu 101 is pop-up displayed on the image list display screen 95.
  • the processing instruction menu 101 has three options: album creation, batch editing, and batch deletion.
  • the browser control unit 32 displays an album creation screen 105 on the display 24A, as shown in FIG. 24 as an example.
  • the album creation screen 105 displays an album 106 created using the selected image 46S.
  • a save button 107 and a cancel button 108 are provided.
  • the browser control unit 32 transmits a storage request for the album 106 to the image management server 12.
  • the RW control unit 42 of the image management server 12 stores the album 106 in the storage area 45 of the image DB 36.
  • the cancel button 108 is pressed, the browser control unit 32 returns the display to the image list display screen 95 in which the selected image 46S is not selected.
  • the browser control unit 32 applies a specified display effect such as black and white, sepia, vivid, soft focus, light leakage, whitening, etc. to the selected image 46S. Apply all at once. Further, when the option of batch deletion in the processing instruction menu 101 is selected, the browser control unit 32 deletes the selected images 46S in a batch.
  • the browser control unit 32 receives an instruction from the user U to select the image 46 on the display space 90. Then, designated processes such as album creation, batch editing, and batch deletion are performed only on the selected images 46S selected by the selection instruction.
  • the CPU 22A of the user terminal 10 functions as the browser control unit 32 by activating the image AP 30.
  • the CPU 22B of the image management server 12 functions as a reception unit 40, a feature value derivation unit 41, an RW control unit 42, and a distribution control unit 43 by starting the operation program 35.
  • the user U uses the camera function of the user terminal 10 to photograph the image 46.
  • an image storage request 70 including an image 46 and supplementary information 47 is transmitted to the image management server 12.
  • the image storage request 70 is accepted by the reception unit 40 (YES in step ST100 in FIG. 25).
  • the image storage request 70 is output from the reception unit 40 to the feature quantity derivation unit 41 and the RW control unit 42.
  • the feature amount deriving unit 41 derives various feature amounts as shown in FIGS. 8 to 11 (step ST110).
  • Feature amount information 48 composed of various feature amounts is output from the feature amount deriving section 41 to the RW control section 42. Then, under the control of the RW control unit 42, the image 46, additional information 47, and feature information 48 are stored in the image DB 36 (step ST120).
  • the user U selects the feature quantities forming the X-axis and Y-axis of the display space 90 from the pull-down menus 76 and 77, and then presses the setting button 78.
  • the browser control unit 32 generates an image distribution request 85, and transmits the image distribution request 85 to the image management server 12.
  • the image distribution request 85 is received by the reception unit 40 (YES in step ST200 in FIG. 26).
  • the image distribution request 85 is output from the reception unit 40 to the RW control unit 42.
  • the image 46 corresponding to the image distribution request 85 is read from the image DB 36 (step ST210).
  • the image 46 is outputted from the RW control section 42 to the distribution control section 43 along with additional information 47 and feature amount information 48 .
  • the image 46, the supplementary information 47, and the feature amount information 48 are distributed to the user terminal 10 that requested the image distribution request 85 under the control of the distribution control unit 43 (step ST220).
  • the browser control unit 32 receives the images 46 and the like distributed from the image management server 12 (YES in step ST300 in FIG. 27). As shown in FIG. 17, the image 46 is placed at a position corresponding to the feature amount in the display space 90 set on the display setting screen 75 under the control of the browser control unit 32 (step ST310). Then, under the control of the browser control unit 32, an image list display screen 95 including the display space 90 is displayed on the display 24A (step ST320). At this time, among the two or more images 46 whose feature amounts are within a preset range, as shown in FIG. 18, the representative image 46RP is displayed at the top, and the other images are displayed below the representative image 46RP. 46 are displayed in a stacked manner.
  • the RW control unit 42 of the CPU 22B of the image management server 12 acquires multiple images 46 and acquires multiple types of feature amounts for the multiple images 46.
  • the browser control unit 32 of the CPU 22A of the user terminal 10 displays a list of the multiple images 46 by arranging the images 46 at positions according to the feature amounts in a two-dimensional display space 90 related to the feature amounts, the display space 90 having at least one axis indicating the magnitude of a numerical value obtained by integrating two or more types of feature amounts. Therefore, it is possible to display a list of a wide variety of images 46 compared to display spaces having X and Y axes indicating one type of feature amount, such as the display space 90 shown in FIG.
  • the X axis indicates the date and time feature amount and the Y axis indicates the image quality feature amount
  • the display space 90 shown in FIG. 21 in which the X axis indicates the subject feature amount of alpine plants and the Y axis indicates the red color feature amount.
  • the browser control unit 32 accepts an instruction from the user U to change the specifications of the display space 90, and changes the specifications of the display space 90 in accordance with the change instruction. This makes it easy to change the specifications of the display space 90 to match the intentions of the user U. As a result, it becomes easier for the user U to find the desired image 46, and the time required to search for the desired image 46 can be reduced.
  • the change instruction is an instruction to change the feature amount that constitutes the axis of the display space 90. Therefore, the axis of the display space 90 can be easily changed as the user U desires. As a result, an unexpected image 46 may be discovered, such as displaying an image 46 that has been taken but forgotten, thereby attracting the user U's interest.
  • the browser control unit 32 displays the representative image 46RP of the two or more images 46 at the top, and The other images 46 are displayed in a layered manner below. Therefore, the display of two or more images 46 whose feature amounts are within a preset range can be clearly displayed, and the list of images 46 can be displayed more easily.
  • the browser control unit 32 receives an instruction from the user U to select the image 46 on the display space 90. Then, the designated process is performed only on the selected image 46S selected by the selection instruction. For this reason, as shown in FIG. 23, for example, an album 106 may be created by selecting an image 46 in which a dog and/or child is clearly shown as the subject and with good image quality, or an image 46 with relatively poor image quality.
  • the images 46 can be selected and deleted all at once, and the user U's work on organizing and editing the images 46 can be greatly facilitated.
  • an image list display screen 110 shown in FIG. 28 may be adopted.
  • the image list display screen 110 is a screen in which the functions of the display setting screen 75 are added to the image list display screen 95.
  • the image list display screen 110 is provided with a pull-down menu 111 for selecting the feature amounts forming the X-axis and a pull-down menu 112 for selecting the feature amounts forming the Y-axis.
  • Two pull-down menus 111 and 112 are provided in the initial display state.
  • feature amount addition buttons 113 and 114 are provided on the right side of the pull-down menus 111 and 112. When the feature amount addition button 113 is pressed, the pull-down menu 111 is added.
  • the pull-down menu 112 is added.
  • Pull-down menus 111 and 112 correspond to pull-down menus 76 and 77 on display setting screen 75.
  • the feature value addition buttons 113 and 114 correspond to the feature value addition buttons 79 and 80 on the display setting screen 75. According to this image list display screen 110, there is no need to press the setting change button 96 to change the display to the display setting screen 75, so that the user U's operation effort can be saved.
  • the display setting screen 120 of the second embodiment is provided with weighting factor setting bars 121 and 122 in addition to the pull-down menus 76 and 77 of the first embodiment.
  • the weighting factor setting bar 121 is a GUI for setting the weighting factor to be applied to the feature amount constituting the X axis selected by the pull-down menu 76 to a value of 0.1 or more and 1 or less.
  • the weighting factor setting bar 122 is a GUI for setting the weighting factor to be applied to the feature amount constituting the Y axis selected by the pull-down menu 77 to a value of 0.1 or more and 1 or less.
  • two weighting factor setting bars 121 and 122 are provided in the initial display state.
  • the weighting coefficients of the weighting coefficient setting bars 121 and 122 are set to 1 in the initial display state.
  • FIG. 29 a case is illustrated in which subject features of a dog and a child are selected in the pull-down menu 76, and image quality features are selected in the pull-down menu 77.
  • the weighting factor setting bar 121 0.5 is set for the weighting coefficient multiplied by the dog's subject feature, and 1 is set for the weighting coefficient multiplied by the child's subject feature, and in the weighting factor setting bar 122.
  • the weighting coefficient multiplied by the image quality feature is set to 1.
  • FIG. 30 shows the processing of the browser control unit 32 in the case of the display settings shown in FIG. 29.
  • the browser control unit 32 calculates the integrated feature amount IF for the image 46 distributed from the image management server 12.
  • the browser control unit 32 calculates the average of the multiple types of feature amounts to be integrated as the integrated feature amount IF in the case of the first embodiment, whereas the browser control unit 32 calculates the weight of the multiple types of feature amounts to be integrated.
  • the weighted average (weighted average) is calculated as the integrated feature amount IF.
  • the browser control unit 32 uses the weighting coefficients set in the weighting coefficient setting bars 121 and 122 to calculate the weighted average.
  • the browser control unit 32 sets a weighting coefficient of 0 for the subject feature amount of a dog set in the weighting factor setting bar 121.
  • the instruction to change the specifications of the display space 90 by the user U is an instruction to change the weighting coefficient that determines the ratio of integrating two or more types of feature amounts. Therefore, it is possible to display a list of images 46 with even greater variety. For example, you might want to create an album 106 that gives top priority to images 46 with good image quality, but you also want to take the date and time of shooting into account, or you want to create an album 106 that mainly includes images 46 of children, but there are also images 46 of dogs here and there. It is possible to respond to user U's detailed requests, such as wanting to include
  • the image list display screen 125 of the third embodiment includes an area 126 in which images 46 are displayed as a list in a matrix, and a display space 90.
  • the image 46 in the area 126 can be moved to any position in the display space 90 by a drag-and-drop operation using the finger F of the user U.
  • the image 46 moved from the area 126 to the display space 90 by the drag-and-drop operation will be referred to as a reference image 46R.
  • the display space 90 of the third embodiment converts feature quantities having more dimensions than two dimensions into two dimensions using a well-known dimension reduction technique such as t-SNE (T-distributed Stochastic Neighbor Embedding). This is the space gained through reduction. Therefore, the feature amounts forming the X-axis and Y-axis of the display space 90 in the third embodiment are not determined to be a specific type of feature amount.
  • t-SNE T-distributed Stochastic Neighbor Embedding
  • the browser control unit 32 adjusts the distance between the reference image 46R and the feature vector VI, centering on the reference image 46R. Another image 46 is placed at the corresponding position. Therefore, the image 46 whose feature vector VI is similar to the reference image 46R and whose distance to the feature vector VI of the reference image 46R is close is placed at a position close to the reference image 46R. On the other hand, an image 46 in which the reference image 46R and the feature vector VI deviate from each other, and the distance between the reference image 46R and the feature vector VI is long, is placed at a position far from the reference image 46R.
  • an image whose feature vector VI is similar to the reference image 46R is an image similar to the reference image 46R.
  • an image in which the reference image 46R and the feature amount vector VI are different from each other is, simply put, an image that is dissimilar to the reference image 46R.
  • the browser control unit 32 adds a color, for example, a red circle 128, to the reference image 46R, and displays the reference image 46R to distinguish it from other images 46. Note that FIG. 32 shows a case where there is one reference image 46R, and FIG. 33 shows a case where there are two reference images 46R.
  • the feature quantity vector VI of the image 46 is obtained by multiplying the number of dimensions of VI by the weighting coefficient Wi.
  • the browser control unit 32 displays a similar image selection screen 130 on the display 24A in response to an instruction from the user U.
  • the similar image selection screen 130 is a screen for selecting an image 46 that the user U considers to be similar.
  • the images 46 are displayed in a matrix format on the similar image selection screen 130.
  • the user U selects two or more images that the user U considers to be similar, as shown by hatching, from the images 46 displayed in a list. After that, the user U presses the OK button 131.
  • the browser control unit 32 transmits a weighting coefficient correction request including the image 46 selected by the user U to the image management server 12.
  • the image 46 selected by the user U considering it to be similar will be referred to as a similar image 46SM.
  • the CPU 22B of the image management server 12 of the fourth embodiment functions as a weighting factor correction unit 135 in addition to each of the processing units 40 to 43 of the first embodiment.
  • the feature information 48SM of the similar image 46SM is input to the weighting coefficient correction unit 135.
  • the weighting factor correction unit 135 calculates the corrected weighting factor WiC by solving an optimization problem for finding a weighting factor Wi such that the distance between the feature vectors VI of the plurality of similar images 46SM is less than a first preset threshold. calculate.
  • the first threshold is set to a value close to 0, such as 0.1, for example. That is, the weighting coefficient correction unit 135 changes the weighting coefficient Wi in a direction in which the feature amounts of the similar images 46SM match each other, and sets the weighting coefficient Wi to a corrected weighting coefficient WiC.
  • the browser control unit 32 displays a dissimilar image selection screen 140 on the display 24A in response to an instruction from the user U.
  • the dissimilar image selection screen 140 is a screen for selecting an image 46 that the user U considers to be completely different from the similar image selection screen 130. Similar to the similar image selection screen 130, the dissimilar image selection screen 140 displays a list of images 46 in a matrix.
  • the user U selects two or more images that the user U considers to be completely different, as shown by hatching, from among the images 46 displayed in a list. After that, the user U presses the OK button 141.
  • the browser control unit 32 transmits a weighting coefficient correction request including the image 46 selected by the user U to the image management server 12.
  • the image 46 selected by the user U considering it to be completely different will be referred to as a dissimilar image 46DSM.
  • the feature amount information 48DSM of the dissimilar image 46DSM is input to the weighting coefficient correction unit 135.
  • the weighting coefficient correction unit 135 solves an optimization problem to obtain a weighting coefficient Wi such that the distance between the feature vectors VI of the plurality of dissimilar images 46DSM is equal to or greater than a preset second threshold value, thereby obtaining a corrected weighting coefficient WiC.
  • the second threshold value is set to a value such that the placement position of the dissimilar image 46DSM is farthest apart, such as the position of the lower left origin and the position where the upper right feature amount is maximum, in the display space 90 shown in the third embodiment, for example. Set. That is, the weighting coefficient correction unit 135 changes the weighting coefficient Wi in a direction in which the feature amounts of the dissimilar images 46DSM deviate from each other, and sets it as a corrected weighting coefficient WiC.
  • the browser control unit 32 receives an instruction from the user U to select at least one of the similar image 46SM and the dissimilar image 46DSM.
  • the weighting coefficient correction unit 135 applies weights to multiple types of feature amounts in a direction in which the feature amounts match for similar images 46SM, and in a direction in which feature amounts deviate from each other for dissimilar images 46DSM. Change the coefficient Wi. Therefore, in the display space 90, images 46 that the user U considers to be similar are placed relatively close to each other, and images 46 that the user U considers to be completely different are placed relatively far apart. It happens. Therefore, it is possible to display a list of images 46 that suits the sense of the user U.
  • the user U may be allowed to select the similar image 46SM or the dissimilar image 46DSM on the display space 90.
  • possible selection methods include surrounding the similar images 46SM with a frame, connecting the similar images 46SM with a line, or connecting the dissimilar images 46DSM with a line.
  • the images 46 are configured to be movable within the display space 90 by drag-and-drop operations, etc., and the operation of re-arranging the images 46 that the user U considers to be similar to each other is performed as a selection operation of the similar images 46SM. It may be considered as On the other hand, an operation of rearranging images 46 that the user U considers to be completely different to distant positions may be regarded as an operation of selecting dissimilar images 46DSM.
  • a list of images 46 may be displayed in a three-dimensional display space 150.
  • a display space 150 is illustrated in which the X-axis is a date and time feature, the Y-axis is a mountain subject feature and a forest subject feature, and the Z-axis is image quality.
  • the display size of the image 46 may be changed depending on the position of the display space 90. For example, in the display space 90 shown in FIGS. 19 to 21, the display size of the image 46 placed in the upper right area is increased. In this way, images 46 that are likely to be included in the album 106 can be shown to the user U in more detail.
  • the image 46 is not limited to an image taken using the camera function of the illustrated user terminal 10.
  • the image may be an image that the user U has downloaded from a web page, an image that the user U has received from a family member, a friend, or the like.
  • the image may be an image captured by the camera function of the user terminal 10 from an instant film printed out from an instant camera.
  • the image management server 12 may be responsible for all or part of the functions of the browser control unit 32 of the user terminal 10. Specifically, various screens such as a display setting screen 75 and an image list display screen 95 are generated in the image management server 12, and screen data for web distribution created in a markup language such as XML (Extensible Markup Language) is generated. It is distributed and output to the user terminal 10 in this format. In this case, the browser control unit 32 of the user terminal 10 reproduces various screens to be displayed on the web browser based on the screen data and displays them on the display 24A. Note that other data description languages such as JSON (JavaScript (registered trademark) Object Notation) may be used instead of XML.
  • JSON JavaScript (registered trademark) Object Notation
  • the hardware configuration of the computer that constitutes the image management server 12 can be modified in various ways.
  • the image management server 12 can be configured with multiple computers separated as hardware in order to improve processing power and reliability.
  • the functions of the reception unit 40 and feature derivation unit 41 and the functions of the RW control unit 42 and distribution control unit 43 can be distributed and assigned to two computers.
  • the image management server 12 is configured with two computers.
  • all or part of the functions of the image management server 12 may be assigned to the user terminal 10.
  • the hardware configurations of the computers of the user terminal 10 and the image management server 12 can be changed as appropriate depending on required performance such as processing capacity, safety, and reliability.
  • APs such as the image AP 30 and the operating program 35 can of course be duplicated or distributed and stored in multiple storages for the purpose of ensuring safety and reliability. be.
  • processing units that execute various processes such as the browser control unit 32, the reception unit 40, the feature derivation unit 41, the RW control unit 42, the distribution control unit 43, and the weighting coefficient correction unit 135 are used.
  • Various processors include CPUs 22A and 22B, which are general-purpose processors that execute software (image AP 30 and operating program 35) and function as various processing units, as well as FPGAs (Field Programmable Gate Array), etc.
  • a processor that has a circuit configuration specifically designed to execute a specific process such as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed, and/or an ASIC (Application Specific Integrated Circuit). This includes dedicated electrical circuits, etc.
  • PLD programmable logic device
  • ASIC Application Specific Integrated Circuit
  • One processing unit may be composed of one of these various processors, or a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs and/or a CPU and (in combination with FPGA). Further, the plurality of processing units may be configured with one processor.
  • one processor is configured with a combination of one or more CPUs and software, as typified by computers such as clients and servers.
  • a processor functions as multiple processing units.
  • SoC system-on-chip
  • various processing units are configured using one or more of the various processors described above as a hardware structure.
  • circuitry that is a combination of circuit elements such as semiconductor elements can be used.
  • the processor includes: Get multiple images, Obtaining multiple types of feature amounts for the multiple images, In a two-dimensional or three-dimensional display space related to the feature amount, which has at least one axis indicating the magnitude of a numerical value that integrates two or more types of the feature amount, displaying a list of a plurality of images by arranging the images at positions; Display control device.
  • the processor includes: accept a user's instruction to change the specifications of the display space; The display control device according to supplementary note 1, wherein the specification of the display space is changed in accordance with the change instruction.
  • the display control device according to appendix 2, wherein the change instruction is an instruction to change a weighting coefficient that determines a ratio of integrating two or more types of the feature amounts.
  • the display control device according to appendix 2 or 3, wherein the change instruction is an instruction to change the feature amount constituting the axis.
  • the processor includes: If there are two or more images whose feature amounts are within a preset range, a representative image of the two or more images is displayed at the top, and the other images are stacked below the representative image.
  • the display control device displays a display in a mode in which: [Additional note 6]
  • the processor includes: accepting a user's instruction to select the image on the display space;
  • the display control device according to any one of Supplementary Notes 1 to 5, which performs specified processing only on the image selected by the selection instruction.
  • the processor includes: Accepting a selection instruction from a user of at least one of a similar image and a dissimilar image, In response to the selection instruction, weighting coefficients applied to the plurality of types of feature quantities are changed in a direction in which the feature quantities match for the similar images, and in a direction in which the feature quantities diverge for the dissimilar images.
  • the display control device according to any one of Supplementary Notes 1 to 6.
  • the technology of the present disclosure can also be combined as appropriate with the various embodiments and/or various modifications described above. Furthermore, it goes without saying that the present invention is not limited to the above embodiments, and that various configurations can be adopted as long as they do not depart from the gist of the invention. Furthermore, the technology of the present disclosure extends not only to programs but also to storage media that non-temporarily store programs.
  • a and/or B has the same meaning as “at least one of A and B.” That is, “A and/or B” means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, even when three or more items are expressed in conjunction with “and/or”, the same concept as “A and/or B" is applied.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Processing Or Creating Images (AREA)

Abstract

This display control device comprises a processor. The processor: acquires a plurality of images; acquires a plurality of types of feature amounts relating to the images; and displays a collection of the images by arranging the images at positions corresponding to the feature amounts in a two-dimensional or three-dimensional display space regarding the feature amounts and having at least one axis indicating the magnitude of a numerical value obtained by integrating two or more types of the feature amounts.

Description

表示制御装置、表示制御装置の作動方法、および表示制御装置の作動プログラムDISPLAY CONTROL DEVICE, OPERATION METHOD OF DISPLAY CONTROL DEVICE, AND OPERATION PROGRAM OF DISPLAY CONTROL DEVICE
　本開示の技術は、表示制御装置、表示制御装置の作動方法、および表示制御装置の作動プログラムに関する。 The technology of the present disclosure relates to a display control device, a display control device operating method, and a display control device operating program.
　特開２００１－００５８２２号公報には、画像データベースから任意の方法によって検索した結果として得られる検索結果の画像群の画像の表示方法が記載されている。表示方法は、画像データベース全体の画像群に対する特徴量と、画像に付与し得るキーワードあるいはシンボルと画像特徴量の対応関係をあらかじめ統計処理によって対応付けたデータベースを備えて、検索結果の複数の画像の特徴量間の違いを表現するキーワードあるいはシンボルの特徴量を、その特徴量間の違いに敏感な順に、上位から最大で指定した個数だけが検索毎にダイナミックに自動選択されること、選択されたキーワードあるいはシンボルを検索者へ提示すること、並びに、提示されたキーワードあるいはシンボルから検索者によって選択されたキーワードあるいはシンボルから構成される表示空間上に検索結果の複数の画像の縮小画像を散布図として配置し表示することを含む。 Japanese Patent Laid-Open No. 2001-005822 describes a method for displaying images of a group of search results obtained as a result of searching an image database by an arbitrary method. The display method is equipped with a database in which feature values for a group of images in the entire image database are associated with keywords or symbols that can be assigned to images and image feature values through statistical processing. A maximum of a specified number of keywords or symbol features that express differences between features are dynamically and automatically selected for each search, in order of sensitivity to differences between the features. Presenting keywords or symbols to a searcher, and displaying reduced images of multiple images of search results as a scatter plot on a display space composed of the keywords or symbols selected by the searcher from the presented keywords or symbols. Including arranging and displaying.
　特開平０５－２８２３７５号公報には、ｎ次元の属性情報をもつ画像を蓄積する画像データベースの中から、任意次元の属性情報をもつ画像を検索し、検索した複数の画像については縮小して表示装置にブラウジング表示し、表示した画像の中から着目する画像を選択すると、その画像を表示装置の全体の画面に表示する画像データベース・システムが記載されている。画像データベース・システムは、任意の標本画像に類似した画像を検索するにあたって、標本画像の着目する２次元の属性情報の各特徴量から指定された範囲内の距離空間にある特徴量をもつ画像を検索し、検索した複数の画像を縮小して表示装置にブラウジング表示するにあたっては２次元の属性情報の座標でブラウジング表示を行う。 Japanese Patent Application Laid-Open No. 05-282375 discloses a method for searching for images with arbitrary dimension attribute information from an image database that stores images with n-dimensional attribute information, and displaying the searched images in a reduced size. An image database system is described in which a device performs browsing display, and when an image of interest is selected from among the displayed images, that image is displayed on the entire screen of the display device. When searching for images similar to a given sample image, the image database system searches for images that have features within a specified distance space from each feature of the focused two-dimensional attribute information of the sample image. When searching, reducing the size of a plurality of searched images, and displaying them for browsing on a display device, the browsing display is performed using coordinates of two-dimensional attribute information.
　特開２００３－２７１６６５号公報には、第１の表示制御手段、移動指令手段、および第２の表示制御手段を備える検索用グラフィカル・ユーザ・インターフェイスが記載されている。第１の表示制御手段は、目盛りが形成されている座標軸、目盛りに対応して設定される検索対象範囲に該当する対象情報の検索結果を表示する領域、および座標軸上における基準位置にある操作子を表示画面上に表示するように表示装置を制御する。移動指令手段は、操作子に横方向または縦方向への移動指令を与える。第２の表示制御手段は、移動指令手段から操作子に横方向または縦方向のいずれか一方への移動指令が与えられたことにより、目盛りの尺度が変更され、かつ移動指令手段から操作子に横方向または縦方向のいずれか他方への移動指令が与えられたことにより、目盛りの値が変更されるように座標軸の表示を制御する。 Japanese Patent Application Publication No. 2003-271665 describes a graphical user interface for searching that includes a first display control means, a movement command means, and a second display control means. The first display control means includes a coordinate axis on which a scale is formed, an area for displaying search results of target information corresponding to a search target range set corresponding to the scale, and an operator located at a reference position on the coordinate axis. The display device is controlled to display on the display screen. The movement command means gives a horizontal or vertical movement command to the operator. The second display control means changes the scale of the scale when the movement command means gives the operator a movement command in either the horizontal or vertical direction, and the movement command means changes the scale of the scale to the operator. The display of the coordinate axes is controlled so that the values of the scales are changed when a movement command is given in either the horizontal direction or the vertical direction.
　本開示の技術に係る１つの実施形態は、バラエティーに富んだ画像の一覧表示を行うことが可能な表示制御装置、表示制御装置の作動方法、および表示制御装置の作動プログラムを提供する。 One embodiment of the technology of the present disclosure provides a display control device capable of displaying a list of a wide variety of images, a method for operating the display control device, and an operating program for the display control device.
　本開示の表示制御装置は、プロセッサを備え、プロセッサは、複数の画像を取得し、複数の画像について複数種類の特徴量を取得し、特徴量に係る２次元または３次元の表示空間であって、２種類以上の特徴量を統合した数値の大きさを示す軸を少なくとも１軸以上有する表示空間において、特徴量に応じた位置に画像を配置することで、複数の画像を一覧表示する。 A display control device of the present disclosure includes a processor, the processor acquires a plurality of images, acquires a plurality of types of feature amounts for the plurality of images, and displays a two-dimensional or three-dimensional display space related to the feature amounts. , a plurality of images are displayed as a list by arranging the images at positions corresponding to the feature amounts in a display space that has at least one axis that indicates the magnitude of a numerical value that integrates two or more types of feature amounts.
　プロセッサは、表示空間の仕様のユーザによる変更指示を受け付け、変更指示にしたがって表示空間の仕様を変更することが好ましい。 Preferably, the processor receives a user's instruction to change the specifications of the display space, and changes the specifications of the display space in accordance with the change instruction.
　変更指示は、２種類以上の特徴量を統合する割合を決める重み係数の変更指示であることが好ましい。 Preferably, the change instruction is an instruction to change a weighting coefficient that determines the ratio of integrating two or more types of feature amounts.
　変更指示は、軸を構成する特徴量の変更指示であることが好ましい。 It is preferable that the change instruction is an instruction to change the feature amount that constitutes the axis.
　プロセッサは、特徴量が予め設定された範囲内にある２つ以上の画像がある場合、２つ以上の画像のうちの代表画像を最上位に表示し、代表画像の下にそれ以外の画像を積層した態様で表示することが好ましい。 If there are two or more images with feature values within a preset range, the processor displays the representative image of the two or more images at the top, and displays the other images below the representative image. It is preferable to display in a layered manner.
　プロセッサは、表示空間上でユーザによる画像の選択指示を受け付け、選択指示によって選択された画像に対してのみ、指定された処理を行うことが好ましい。 Preferably, the processor receives an instruction to select an image from the user on the display space, and performs specified processing only on the image selected by the selection instruction.
　プロセッサは、類似画像、および非類似画像のうちの少なくともいずれかのユーザによる選択指示を受け付け、選択指示に応じて、類似画像同士については特徴量が一致する方向に、非類似画像同士については特徴量が乖離する方向に、複数種類の特徴量に掛ける重み係数を変更することが好ましい。 The processor receives a user's selection instruction for at least one of the similar images and the dissimilar images, and, in accordance with the selection instruction, moves the similar images in a direction in which the feature amounts match, and the dissimilar images in a direction in which the feature amounts match. It is preferable to change the weighting coefficients applied to the plurality of types of feature quantities in the direction in which the quantities deviate.
　本開示の表示制御装置の作動方法は、複数の画像を取得すること、複数の画像について複数種類の特徴量を取得すること、並びに、特徴量に係る２次元または３次元の表示空間であって、２種類以上の特徴量を統合した数値の大きさを示す軸を少なくとも１軸以上有する表示空間において、特徴量に応じた位置に画像を配置することで、複数の画像を一覧表示すること、を含む。 The operating method of the display control device disclosed herein includes acquiring a plurality of images, acquiring a plurality of types of feature quantities for the plurality of images, and displaying a list of the plurality of images by arranging the images at positions according to the feature quantities in a two-dimensional or three-dimensional display space relating to the feature quantities and having at least one axis indicating the magnitude of a numerical value obtained by integrating two or more types of feature quantities.
　本開示の表示制御装置の作動プログラムは、複数の画像を取得すること、複数の画像について複数種類の特徴量を取得すること、並びに、特徴量に係る２次元または３次元の表示空間であって、２種類以上の特徴量を統合した数値の大きさを示す軸を少なくとも１軸以上有する表示空間において、特徴量に応じた位置に画像を配置することで、複数の画像を一覧表示すること、を含む処理をコンピュータに実行させる。 The operating program of the display control device of the present disclosure includes acquiring a plurality of images, acquiring a plurality of types of feature amounts for the plurality of images, and a two-dimensional or three-dimensional display space related to the feature amounts. , displaying a plurality of images as a list by arranging the images at positions corresponding to the feature amounts in a display space having at least one axis indicating the magnitude of a numerical value that integrates two or more types of feature amounts; cause a computer to perform processing including
ユーザ端末および画像管理サーバを示す図である。FIG. 2 is a diagram showing a user terminal and an image management server. ユーザ端末および画像管理サーバを構成するコンピュータを示すブロック図である。FIG. 2 is a block diagram showing a computer that constitutes a user terminal and an image management server. ユーザ端末のＣＰＵの処理部を示すブロック図である。FIG. 2 is a block diagram showing a processing unit of a CPU of a user terminal. 画像管理サーバのＣＰＵの処理部を示すブロック図である。FIG. 2 is a block diagram showing a processing unit of a CPU of an image management server. 画像ＤＢに記憶されるデータを示す図である。It is a diagram showing data stored in an image DB. 付帯情報を示す図である。It is a diagram showing supplementary information. 特徴量情報および特徴量ベクトルを示す図である。FIG. 3 is a diagram showing feature amount information and feature amount vectors. 日時特徴量の導出方法の説明図である。FIG. 2 is an explanatory diagram of a method for deriving date and time features. 画質特徴量モデルを用いて画質特徴量を導出する処理を示す図である。FIG. 13 is a diagram showing a process of deriving image quality features using an image quality feature model. モデル特徴量導出モデルを用いてモデル特徴量を導出する処理を示す図である。FIG. 3 is a diagram illustrating a process of deriving model features using a model feature derivation model. 被写体特徴量導出モデルを用いて被写体特徴量を導出する処理を示す図である。FIG. 7 is a diagram illustrating a process of deriving a subject feature using a subject feature deriving model. 被写体判別モデルを示す図である。FIG. 3 is a diagram showing a subject discrimination model. ユーザ端末から画像記憶要求が送信された場合の画像管理サーバの各処理部の処理を示す図である。FIG. 3 is a diagram showing the processing of each processing unit of the image management server when an image storage request is sent from a user terminal. 表示設定画面を示す図である。FIG. 3 is a diagram showing a display setting screen. 表示設定画面において設定ボタンが押された場合のブラウザ制御部の処理を示す図である。13 is a diagram showing a process of the browser control unit when a setting button is pressed on the display setting screen. FIG. ユーザ端末から画像配信要求が送信された場合の画像管理サーバの各処理部の処理を示す図である。FIG. 6 is a diagram illustrating the processing of each processing unit of the image management server when an image distribution request is transmitted from a user terminal. 特徴量に応じた表示空間の位置に画像を配置する処理を示す図である。FIG. 6 is a diagram illustrating a process of arranging an image at a position in a display space according to a feature amount. 特徴量が予め設定された範囲内にある画像を積層表示する処理を示す図である。FIG. 7 is a diagram illustrating a process of displaying images whose feature amounts are within a preset range in a layered manner. 画像一覧表示画面を示す図である。FIG. 3 is a diagram showing an image list display screen. 軸を構成する特徴量の変更指示に応じて、表示空間の仕様を変更する処理を示す図である。FIG. 7 is a diagram illustrating a process of changing the specifications of a display space in response to an instruction to change feature amounts that constitute an axis. 軸を構成する特徴量の変更指示に応じて、表示空間の仕様を変更する処理を示す図である。FIG. 7 is a diagram illustrating a process of changing the specifications of a display space in response to an instruction to change feature amounts that constitute an axis. 表示空間上でのユーザによる画像の選択指示の様子を示す図である。FIG. 3 is a diagram illustrating how a user instructs to select an image in a display space. 選択指示によって選択された画像への処理を指示するための処理指示メニューが表示された状態を示す図である。FIG. 6 is a diagram illustrating a state in which a processing instruction menu for instructing processing on an image selected by a selection instruction is displayed. 選択指示によって選択された画像から作成したアルバムが表示されたアルバム作成画面を示す図である。FIG. 3 is a diagram showing an album creation screen on which an album created from images selected by a selection instruction is displayed. 画像管理サーバの処理手順を示すフローチャートである。5 is a flowchart showing the processing procedure of the image management server. 画像管理サーバの処理手順を示すフローチャートである。5 is a flowchart showing the processing procedure of the image management server. ユーザ端末の処理手順を示すフローチャートである。It is a flowchart which shows the processing procedure of a user terminal. 軸を構成する特徴量をプルダウンメニューで変更可能な画像一覧表示画面を示す図である。FIG. 7 is a diagram illustrating an image list display screen in which the feature amounts configuring an axis can be changed using a pull-down menu. 重み係数の変更指示機能を有する表示設定画面を示す図である。FIG. 7 is a diagram showing a display setting screen having a function of instructing a change in weighting coefficients. 特徴量に応じた表示空間の位置に画像を配置する処理を示す図である。FIG. 6 is a diagram illustrating a process of arranging an image at a position in a display space according to a feature amount. 表示空間の任意の位置に画像を移動させる様子を示す図である。FIG. 7 is a diagram showing how an image is moved to an arbitrary position in the display space. 任意の位置に移動された１つの画像を中心として、他の画像を配置した表示空間を含む画像一覧表示画面を示す図である。FIG. 7 is a diagram showing an image list display screen including a display space in which other images are arranged around one image moved to an arbitrary position. 任意の位置に移動された２つの画像を中心として、他の画像を配置した表示空間を含む画像一覧表示画面を示す図である。FIG. 7 is a diagram showing an image list display screen including a display space in which other images are arranged around two images moved to arbitrary positions. 類似画像選択画面を示す図である。It is a figure showing a similar image selection screen. 類似画像の特徴量ベクトルの距離が第１閾値未満となる重み係数を求める最適化問題を解くことで、重み係数を補正する処理を示す図である。FIG. 13 is a diagram illustrating a process of correcting weighting coefficients by solving an optimization problem for obtaining weighting coefficients that make the distance of feature vectors of similar images less than a first threshold value. 非類似画像選択画面を示す図である。It is a figure showing a dissimilar image selection screen. 非類似画像の特徴量ベクトルの距離が第２閾値以上となる重み係数を求める最適化問題を解くことで、重み係数を補正する処理を示す図である。FIG. 7 is a diagram illustrating a process of correcting weighting coefficients by solving an optimization problem for finding weighting coefficients for which the distance between feature vectors of dissimilar images is equal to or greater than a second threshold; ３次元の表示空間に画像を配置した例を示す図である。FIG. 3 is a diagram showing an example of arranging images in a three-dimensional display space.
　一例として図１に示すように、ユーザＵはユーザ端末１０を所有する。ユーザ端末１０は、カメラ機能、画像再生表示機能、および画像送受信機能等を有する機器である。ユーザ端末１０のカメラ機能は、ＣＭＯＳ（Ｃｏｍｐｌｅｍｅｎｔａｒｙ　Ｍｅｔａｌ－Ｏｘｉｄｅ－Ｓｅｍｉｃｏｎｄｕｃｔｏｒ）イメージセンサ等の撮像素子をもち、レンズから取り込んだ被写体光を撮像素子に結像することで被写体の画像４６（図５参照）を得る。ユーザ端末１０は、具体的にはスマートフォン、タブレット端末、コンパクトデジタルカメラ、ミラーレス一眼カメラ、あるいはノート型のパーソナルコンピュータ等である。ユーザ端末１０は、本開示の技術に係る「表示制御装置」の一例である。 As an example, as shown in FIG. 1, a user U owns a user terminal 10. The user terminal 10 is a device having a camera function, an image playback/display function, an image transmission/reception function, and the like. The camera function of the user terminal 10 has an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, and forms an image of the object 46 (see FIG. 5) by capturing object light from a lens and focusing it on the image sensor. get. Specifically, the user terminal 10 is a smartphone, a tablet terminal, a compact digital camera, a mirrorless single-lens camera, a notebook personal computer, or the like. The user terminal 10 is an example of a "display control device" according to the technology of the present disclosure.
　ユーザ端末１０は、ネットワーク１１を介して画像管理サーバ１２と相互通信可能に接続されている。ネットワーク１１は、例えばインターネットまたは公衆通信網等のＷＡＮ（Ｗｉｄｅ　Ａｒｅａ　Ｎｅｔｗｏｒｋ）である。ユーザ端末１０は画像４６を画像管理サーバ１２に送信（アップロード）する。また、ユーザ端末１０は画像管理サーバ１２からの画像４６を受信（ダウンロード）する。 The user terminal 10 is connected to the image management server 12 via the network 11 so that they can communicate with each other. The network 11 is, for example, a WAN (Wide Area Network) such as the Internet or a public communication network. The user terminal 10 transmits (uploads) the image 46 to the image management server 12. Further, the user terminal 10 receives (downloads) an image 46 from the image management server 12.
　画像管理サーバ１２は、例えばサーバコンピュータ、ワークステーション等であり、ユーザ端末１０と併せて、本開示の技術に係る「表示制御装置」の一例である。このように、本開示の技術に係る「表示制御装置」は、複数の装置に跨って実現されてもよい。画像管理サーバ１２には、ネットワーク１１を介して複数のユーザＵの複数のユーザ端末１０が接続されている。 The image management server 12 is, for example, a server computer, a workstation, etc., and together with the user terminal 10 is an example of a "display control device" according to the technology of the present disclosure. In this way, the "display control device" according to the technology of the present disclosure may be realized across multiple devices. A plurality of user terminals 10 of a plurality of users U are connected to the image management server 12 via a network 11.
　一例として図２に示すように、ユーザ端末１０および画像管理サーバ１２を構成するコンピュータは、基本的には同じ構成であり、ストレージ２０、メモリ２１、ＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）２２、通信部２３、ディスプレイ２４、および入力デバイス２５を備えている。これらはバスライン２６を介して相互接続されている。 As an example, as shown in FIG. 2, the computers configuring the user terminal 10 and the image management server 12 basically have the same configuration, including a storage 20, a memory 21, a CPU (Central Processing Unit) 22, a communication unit 23, It includes a display 24 and an input device 25. These are interconnected via a bus line 26.
　ストレージ２０は、ユーザ端末１０および画像管理サーバ１２を構成するコンピュータに内蔵、またはケーブル、ネットワークを通じて接続されたハードディスクドライブである。もしくはストレージ２０は、ハードディスクドライブを複数台連装したディスクアレイである。ストレージ２０には、オペレーティングシステム等の制御プログラム、各種アプリケーションプログラム（以下、ＡＰ（Ａｐｐｌｉｃａｔｉｏｎ　Ｐｒｏｇｒａｍ）と略す）、およびこれらのプログラムに付随する各種データ等が記憶されている。なお、ハードディスクドライブに代えてソリッドステートドライブを用いてもよい。 The storage 20 is a hard disk drive that is built into a computer that constitutes the user terminal 10 and the image management server 12, or is connected via a cable or a network. Alternatively, the storage 20 is a disk array in which a plurality of hard disk drives are connected in series. The storage 20 stores control programs such as an operating system, various application programs (hereinafter abbreviated as AP (Application Program)), and various data accompanying these programs. Note that a solid state drive may be used instead of the hard disk drive.
　メモリ２１は、ＣＰＵ２２が処理を実行するためのワークメモリである。ＣＰＵ２２は、ストレージ２０に記憶されたプログラムをメモリ２１へロードして、プログラムにしたがった処理を実行する。これによりＣＰＵ２２はコンピュータの各部を統括的に制御する。ＣＰＵ２２は、本開示の技術に係る「プロセッサ」の一例である。なお、メモリ２１は、ＣＰＵ２２に内蔵されていてもよい。 The memory 21 is a work memory for the CPU 22 to execute processing. The CPU 22 loads the program stored in the storage 20 into the memory 21 and executes processing according to the program. Thereby, the CPU 22 centrally controls each part of the computer. The CPU 22 is an example of a "processor" according to the technology of the present disclosure. Note that the memory 21 may be built into the CPU 22.
　通信部２３は、ネットワーク１１等を介した各種情報の伝送制御を行うネットワークインターフェースである。ディスプレイ２４は各種画面を表示する。各種画面にはＧＵＩ(Ｇｒａｐｈｉｃａｌ　Ｕｓｅｒ　Ｉｎｔｅｒｆａｃｅ)による操作機能が備えられる。ユーザ端末１０および画像管理サーバ１２を構成するコンピュータは、各種画面を通じて、入力デバイス２５からの操作指示の入力を受け付ける。入力デバイス２５は、キーボード、マウス、タッチパネル、および音声入力用のマイク等である。 The communication unit 23 is a network interface that controls transmission of various information via the network 11 and the like. The display 24 displays various screens. Various screens are provided with operation functions using a GUI (Graphical User Interface). The computers constituting the user terminal 10 and the image management server 12 accept input of operation instructions from the input device 25 through various screens. The input device 25 is a keyboard, a mouse, a touch panel, a microphone for voice input, or the like.
　なお、以下の説明では、ユーザ端末１０を構成するコンピュータの各部（ストレージ２０、ＣＰＵ２２、ディスプレイ２４、および入力デバイス２５）には添え字の「Ａ」を、画像管理サーバ１２を構成するコンピュータの各部（ストレージ２０およびＣＰＵ２２）には添え字の「Ｂ」をそれぞれ符号に付して区別する。 In the following description, each part of the computer that makes up the user terminal 10 (storage 20, CPU 22, display 24, and input device 25) is indicated by the suffix "A," and each part of the computer that makes up the image management server 12 is indicated by the suffix "A." (Storage 20 and CPU 22) are distinguished by adding a suffix "B" to their respective symbols.
　一例として図３に示すように、ユーザ端末１０のストレージ２０Ａには、画像ＡＰ３０が記憶されている。画像ＡＰ３０は、ユーザＵによってユーザ端末１０にインストールされる。画像ＡＰ３０は、ユーザ端末１０を構成するコンピュータを、本開示の技術に係る「表示制御装置」として機能させるためのＡＰである。すなわち、画像ＡＰ３０は、本開示の技術に係る「表示制御装置の作動プログラム」の一例である。画像ＡＰ３０が起動された場合、ユーザ端末１０のＣＰＵ２２Ａは、メモリ２１等と協働して、ブラウザ制御部３２として機能する。ブラウザ制御部３２は、画像ＡＰ３０の専用のウェブブラウザの動作を制御する。 As an example, as shown in FIG. 3, an image AP30 is stored in the storage 20A of the user terminal 10. Image AP 30 is installed on user terminal 10 by user U. The image AP 30 is an AP for causing a computer forming the user terminal 10 to function as a "display control device" according to the technology of the present disclosure. That is, the image AP30 is an example of the "display control device operation program" according to the technology of the present disclosure. When the image AP 30 is activated, the CPU 22A of the user terminal 10 functions as the browser control unit 32 in cooperation with the memory 21 and the like. The browser control unit 32 controls the operation of the dedicated web browser of the image AP 30.
　ブラウザ制御部３２は各種画面を生成する。ブラウザ制御部３２は、生成した各種画面をディスプレイ２４Ａに表示する。また、ブラウザ制御部３２は、各種画面を通じて、ユーザＵによって入力デバイス２５Ａから入力される様々な操作指示を受け付ける。ブラウザ制御部３２は、操作指示に応じた様々な要求を画像管理サーバ１２に送信する。 The browser control unit 32 generates various screens. The browser control unit 32 displays various generated screens on the display 24A. The browser control unit 32 also receives various operation instructions input by the user U from the input device 25A through various screens. The browser control unit 32 sends various requests to the image management server 12 according to operation instructions.
　一例として図４に示すように、画像管理サーバ１２のストレージ２０Ｂには、作動プログラム３５が記憶されている。作動プログラム３５は、画像管理サーバ１２を構成するコンピュータを、本開示の技術に係る「表示制御装置」として機能させるためのＡＰである。すなわち、作動プログラム３５は、画像ＡＰ３０と同じく、本開示の技術に係る「表示制御装置の作動プログラム」の一例である。ストレージ２０Ｂには、画像データベース（以下、ＤＢ（Ｄａｔａ　Ｂａｓｅ）と表記する）３６も記憶されている。また、図示は省略したが、ストレージ２０Ｂには、ユーザＵを一意に識別するためのユーザＩＤ（Ｉｄｅｎｔｉｆｉｃａｔｉｏｎ　Ｄａｔａ）と、ユーザＵが設定したパスワードと、ユーザ端末１０を一意に識別するための端末ＩＤとが、ユーザＵのアカウント情報として記憶されている。 As an example, as shown in FIG. 4, an operating program 35 is stored in the storage 20B of the image management server 12. The operating program 35 is an AP for causing the computer forming the image management server 12 to function as a "display control device" according to the technology of the present disclosure. That is, like the image AP 30, the operation program 35 is an example of the "operation program for the display control device" according to the technology of the present disclosure. An image database (hereinafter referred to as DB (Data Base)) 36 is also stored in the storage 20B. Although not shown, the storage 20B contains a user ID (Identification Data) for uniquely identifying the user U, a password set by the user U, and a terminal ID for uniquely identifying the user terminal 10. is stored as user U's account information.
　作動プログラム３５が起動されると、画像管理サーバ１２のＣＰＵ２２Ｂは、メモリ２１等と協働して、受付部４０、特徴量導出部４１、リードライト（以下、ＲＷ（Ｒｅａｄ　Ｗｒｉｔｅ）と表記する）制御部４２、および配信制御部４３として機能する。 When the operating program 35 is started, the CPU 22B of the image management server 12 cooperates with the memory 21 and the like to execute the reception unit 40, feature value derivation unit 41, and read/write (hereinafter referred to as RW (Read Write)). It functions as a control section 42 and a distribution control section 43.
　受付部４０は、ユーザ端末１０からの各種要求を受け付ける。受付部４０は、各種要求を、特徴量導出部４１、ＲＷ制御部４２、および配信制御部４３に出力する。特徴量導出部４１は、画像４６の複数種類の特徴量を導出する。特徴量導出部４１は、導出した複数種類の特徴量で構成される特徴量情報４８（図５参照）をＲＷ制御部４２に出力する。 The reception unit 40 receives various requests from the user terminal 10. The reception unit 40 outputs the various requests to the feature derivation unit 41, the RW control unit 42, and the distribution control unit 43. The feature derivation unit 41 derives multiple types of features of the image 46. The feature derivation unit 41 outputs feature information 48 (see FIG. 5) consisting of the multiple types of derived features to the RW control unit 42.
　ＲＷ制御部４２は、ストレージ２０Ｂへの各種データの記憶、およびストレージ２０Ｂからの各種データの読み出しを制御する。ＲＷ制御部４２は、特に、画像ＤＢ３６への画像４６の記憶、並びに画像ＤＢ３６からの画像４６の読み出しを制御する。また、ＲＷ制御部４２は、画像ＤＢ３６への特徴量情報４８の記憶、並びに画像ＤＢ３６からの特徴量情報４８の読み出しを制御する。配信制御部４３は、ユーザ端末１０への各種データの配信を制御する。 The RW control unit 42 controls storage of various data in the storage 20B and reading of various data from the storage 20B. The RW control unit 42 particularly controls the storage of the image 46 in the image DB 36 and the reading of the image 46 from the image DB 36. Further, the RW control unit 42 controls the storage of the feature amount information 48 in the image DB 36 and the reading of the feature amount information 48 from the image DB 36. The distribution control unit 43 controls distribution of various data to the user terminal 10.
　一例として図５に示すように、画像ＤＢ３６には、ユーザＵ毎に記憶領域４５が設けられている。記憶領域４５にはユーザＩＤが登録されている。また、記憶領域４５には、画像４６、付帯情報４７、および特徴量情報４８が関連付けて記憶されている。 As an example, as shown in FIG. 5, a storage area 45 is provided for each user U in the image DB 36. A user ID is registered in the storage area 45. Further, in the storage area 45, an image 46, additional information 47, and feature amount information 48 are stored in association with each other.
　一例として図６に示すように、付帯情報４７は、撮影日時、撮影場所、絞り値、ＩＳＯ（Ｉｎｔｅｒｎａｔｉｏｎａｌ　Ｏｒｇａｎｉｚａｔｉｏｎ　ｆｏｒ　Ｓｔａｎｄａｒｄｉｚａｔｉｏｎ）感度、シャッター速度、焦点距離、フラッシュの有無、およびタグといった複数の項目を含む。撮影日時には、ユーザ端末１０のカメラ機能にて画像４６を撮影した日時が登録される。撮影場所には、ユーザ端末１０のＧＰＳ（Ｇｌｏｂａｌ　Ｐｏｓｉｔｉｏｎｉｎｇ　Ｓｙｓｔｅｍ）機能で得られた経緯度情報から割り出された住所および／またはランドマーク名が登録される。タグは、画像４６に写る被写体を端的に表す単語である。タグは、ユーザＵが手入力したもの、あるいは、被写体判別モデル６０(図１２参照）を用いて導出したものを含む。 As an example, as shown in FIG. 6, the additional information 47 includes multiple items such as shooting date and time, shooting location, aperture value, ISO (International Organization for Standardization) sensitivity, shutter speed, focal length, presence or absence of flash, and tags. . The date and time when the image 46 was photographed using the camera function of the user terminal 10 is registered in the photographing date and time. As the shooting location, an address and/or a landmark name determined from latitude and longitude information obtained by the GPS (Global Positioning System) function of the user terminal 10 is registered. The tag is a word that simply represents the subject in the image 46. The tags include those manually input by the user U or those derived using the subject discrimination model 60 (see FIG. 12).
　一例として図７に示すように、特徴量情報４８は、日時特徴量、色特徴量、明るさ特徴量、画質特徴量、モデル特徴量、および被写体特徴量といった複数の項目を含む。これら特徴量情報４８を構成する複数種類の特徴量を羅列した特徴量ベクトルＶＩによって、画像４６を表すことができる。特徴量情報４８に含まれる特徴量は数百～数千ある。このため、特徴量ベクトルＶＩの次元数も数百～数千となる。 As shown in FIG. 7 as an example, the feature information 48 includes a plurality of items such as a date and time feature, a color feature, a brightness feature, an image quality feature, a model feature, and a subject feature. The image 46 can be represented by a feature vector VI in which a plurality of types of feature amounts constituting the feature information 48 are listed. There are hundreds to thousands of feature amounts included in the feature information 48. Therefore, the number of dimensions of the feature quantity vector VI also ranges from several hundred to several thousand.
　日時特徴量は画像４６の撮影日時に関する特徴量である。一例として図８に示すように、特徴量導出部４１は、記憶領域４５に記憶された画像４６のうちで、撮影日時が最古の画像４６Ｏの日時特徴量を０、撮影日時が最新の画像４６Ｎの日時特徴量を１として、画像４６の日時特徴量を導出する。 The date and time feature is a feature related to the date and time when the image 46 was taken. As an example, as shown in FIG. 8, the feature amount deriving unit 41 sets the date and time feature amount to 0 for the image 46O with the oldest shooting date and time among the images 46 stored in the storage area 45, and the date and time feature amount of the image 46O with the latest shooting date and time. By setting the date and time feature of image 46N to 1, the date and time feature of image 46 is derived.
　色特徴量は画像４６のＲＧＢ（赤緑青）の各色に関する特徴量である。特徴量導出部４１は、例えば、画素値の最大値を１として、画像４６のＲＧＢの各色の画素値の平均値を正規化した値を、色特徴量として導出する。明るさ特徴量は画像４６の明るさ（輝度）に関する特徴量である。特徴量導出部４１は、明るさの最大値を１として、画像４６の明るさの平均値を正規化した値を、明るさ特徴量として導出する。 The color feature amount is a feature amount related to each color of RGB (red, green, and blue) of the image 46. The feature amount deriving unit 41 derives a value obtained by normalizing the average value of the pixel values of each color of RGB of the image 46, for example, with the maximum value of the pixel value being 1, as the color feature amount. The brightness feature amount is a feature amount related to the brightness (luminance) of the image 46. The feature amount deriving unit 41 derives a value obtained by normalizing the average brightness value of the image 46, with the maximum brightness value being 1, as a brightness feature amount.
　画質特徴量は画像４６の画質に関する特徴量である。画質特徴量は０以上１以下の数値である。画質特徴量が０に近い程画像４６の画質は低く、画質特徴量が１に近い程画像４６の画質は高い。 The image quality feature quantity is a feature quantity related to the image quality of the image 46. The image quality feature amount is a numerical value of 0 or more and 1 or less. The closer the image quality feature quantity is to 0, the lower the image quality of the image 46 is, and the closer the image quality feature quantity is to 1, the higher the image quality of the image 46 is.
　一例として図９に示すように、特徴量導出部４１は、画質特徴量導出モデル５５を用いて画質特徴量を導出する。画質特徴量導出モデル５５は、画像４６の入力に応じて画質特徴量を出力する機械学習モデルである。 As an example, as shown in FIG. 9, the feature amount derivation unit 41 derives the image quality feature amount using the image quality feature amount derivation model 55. The image quality feature amount derivation model 55 is a machine learning model that outputs an image quality feature amount in response to the input of the image 46.
　画質特徴量導出モデル５５の学習フェーズにおいては、学習用画像と、学習用画像の入力に応じて画質特徴量導出モデル５５から出力された学習用画質特徴量との答え合わせをするための正解画質特徴量とが教師データとして用意される。正解画質特徴量は、鮮鋭度、露出、階調表現、彩度、ホワイトバランス、ボケの有無、およびブレの有無といった複数の評価項目について、教師データの作成者が評価した結果に基づいて算出される。 In the learning phase of the image quality feature derivation model 55, the correct image quality is used to match the learning image with the learning image quality feature output from the image quality feature derivation model 55 in response to the input of the learning image. The feature amounts are prepared as training data. The correct image quality features are calculated based on the results of the training data creator's evaluation of multiple evaluation items such as sharpness, exposure, gradation expression, saturation, white balance, presence of blur, and presence of blur. Ru.
　モデル特徴量は機械学習モデルから得られる特徴量である。モデル特徴量も他の特徴量と同様に、最大値を１として正規化した値である。 Model features are features obtained from machine learning models. Like other feature quantities, the model feature quantity is also a value normalized with a maximum value of 1.
　一例として図１０に示すように、特徴量導出部４１は、モデル特徴量導出モデル５６を用いてモデル特徴量を導出する。モデル特徴量導出モデル５６は、画像４６の入力に応じてモデル特徴量を出力する機械学習モデルである。 As an example, as shown in FIG. 10, the feature amount derivation unit 41 derives the model feature amount using the model feature amount derivation model 56. The model feature derivation model 56 is a machine learning model that outputs model features in response to the input of the image 46.
　被写体特徴量は、画像４６に写る被写体の存在確率を、被写体の種類毎に０以上１以下の数値で示す特徴量である。被写体としては、例示の山、川、海、湖、男性、女性等の他に、画像４６の被写体となり得るあらゆるものが含まれている。 The subject feature quantity is a feature quantity that indicates the existence probability of a subject appearing in the image 46 as a numerical value between 0 and 1 for each type of subject. The subject includes everything that can be the subject of the image 46, in addition to the illustrated mountains, rivers, oceans, lakes, men, women, etc.
　一例として図１１に示すように、特徴量導出部４１は、被写体特徴量導出モデル５７を用いて被写体特徴量を導出する。被写体特徴量導出モデル５７は、画像４６の入力に応じて被写体特徴量を出力する機械学習モデルである。 As an example, as shown in FIG. 11, the feature amount derivation unit 41 derives the object feature amount using the object feature amount derivation model 57. The subject feature amount derivation model 57 is a machine learning model that outputs a subject feature amount in response to the input of the image 46.
　画質特徴量導出モデル５５、モデル特徴量導出モデル５６、および被写体特徴量導出モデル５７は、ストレージ２０Ｂに記憶されている。ＲＷ制御部４２は、これらの導出モデル５５～５７をストレージ２０Ｂから読み出し、特徴量導出部４１に出力する。 The image quality feature derivation model 55, model feature derivation model 56, and subject feature derivation model 57 are stored in the storage 20B. The RW control unit 42 reads these derived models 55 to 57 from the storage 20B and outputs them to the feature amount deriving unit 41.
　一例として図１２に示すように、モデル特徴量導出モデル５６、および被写体特徴量導出モデル５７には、被写体判別モデル６０の一部が転用される。被写体判別モデル６０は、例えばＵ－Ｎｅｔ、ＲｅｓＮｅｔ（Ｒｅｓｉｄｕａｌ　Ｎｅｔｗｏｒｋ）といった畳み込みニューラルネットワーク（ＣＮＮ：Ｃｏｎｖｏｌｕｔｉｏｎａｌ　Ｎｅｕｒａｌ　Ｎｅｔｗｏｒｋ）であり、エンコーダ部６１および出力部６２により構成される。エンコーダ部６１には画像４６が入力される。エンコーダ部６１は、フィルタを用いた周知の畳み込み演算処理、プーリング処理、およびスキップレイヤー処理等を画像４６に対して施すことで、特徴量６３を導出する。エンコーダ部６１は特徴量６３を出力部６２に出力する。特徴量６３はモデル特徴量に他ならない。つまり、エンコーダ部６１がモデル特徴量導出モデル５６に転用される。 As an example, as shown in FIG. 12, a part of the subject discrimination model 60 is used as the model feature derivation model 56 and the subject feature derivation model 57. The object discrimination model 60 is, for example, a convolutional neural network (CNN) such as U-Net or ResNet (Residual Network), and includes an encoder section 61 and an output section 62. An image 46 is input to the encoder section 61 . The encoder unit 61 derives the feature amount 63 by performing well-known convolution processing using a filter, pooling processing, skip layer processing, etc. on the image 46. The encoder section 61 outputs the feature amount 63 to the output section 62. The feature amount 63 is nothing but a model feature amount. In other words, the encoder unit 61 is used as the model feature derivation model 56.
　出力部６２は、デコーダ部６４、確率算出部６５、および被写体判別部６６を有する。デコーダ部６４には特徴量６３が入力される。デコーダ部６４は、畳み込み演算処理を伴うアップサンプリング処理、およびマージ処理等を特徴量６３に対して施すことで、最終特徴量６３Ｅを導出する。確率算出部６５は、ソフトマックス関数、あるいはシグモイド関数といった周知の活性化関数を用いて、最終特徴量６３Ｅから確率算出結果６７を生成する。確率算出部６５は、確率算出結果６７を被写体判別部６６に出力する。確率算出結果６７は、画像４６における、山、川、海、湖といった各被写体の存在確率である。確率算出結果６７は被写体特徴量に他ならない。つまり、エンコーダ部６１、デコーダ部６４、および確率算出部６５が、被写体特徴量導出モデル５７に転用される。 The output section 62 includes a decoder section 64, a probability calculation section 65, and a subject discrimination section 66. The feature amount 63 is input to the decoder section 64 . The decoder unit 64 derives the final feature amount 63E by subjecting the feature amount 63 to upsampling processing accompanied by convolution processing, merging processing, and the like. The probability calculation unit 65 generates a probability calculation result 67 from the final feature amount 63E using a well-known activation function such as a softmax function or a sigmoid function. The probability calculation section 65 outputs the probability calculation result 67 to the subject discrimination section 66. The probability calculation result 67 is the existence probability of each subject such as a mountain, a river, the sea, and a lake in the image 46. The probability calculation result 67 is nothing but a subject feature amount. That is, the encoder section 61, the decoder section 64, and the probability calculation section 65 are used as the subject feature derivation model 57.
　被写体判別部６６は、確率算出結果６７における存在確率が最も高い被写体を、被写体判別結果６８として出力する。なお、ここでは被写体判別モデル６０というクラス判別モデルの一部を、モデル特徴量導出モデル５６および被写体特徴量導出モデル５７に転用する例を示したが、これに限らない。画像４６の入力に応じて、画像４６の復元画像を出力するオートエンコーダのエンコーダ部を、モデル特徴量導出モデル５６に転用してもよい。あるいは、画像４６に写る被写体を画素単位で識別するセマンティックセグメンテーションモデルのエンコーダ部、デコーダ部、および確率算出部を、被写体特徴量導出モデル５７に転用してもよい。 The subject discrimination unit 66 outputs the subject with the highest existence probability in the probability calculation result 67 as the subject discrimination result 68. Although an example has been shown in which a part of the class discrimination model called the subject discrimination model 60 is used as the model feature derivation model 56 and the subject feature derivation model 57, the present invention is not limited to this. An encoder section of an autoencoder that outputs a restored image of the image 46 in response to the input of the image 46 may be used as the model feature derivation model 56. Alternatively, the encoder section, decoder section, and probability calculation section of a semantic segmentation model that identifies a subject in the image 46 in pixel units may be used as the subject feature derivation model 57.
　一例として図１３に示すように、ブラウザ制御部３２は、画像記憶要求７０を画像管理サーバ１２に送信する。画像記憶要求７０は、ユーザＩＤと、画像４６および付帯情報４７とを含む。 As an example, as shown in FIG. 13, the browser control unit 32 transmits an image storage request 70 to the image management server 12. The image storage request 70 includes a user ID, an image 46, and additional information 47.
　受付部４０は画像記憶要求７０を受け付け、画像記憶要求７０を特徴量導出部４１およびＲＷ制御部４２に出力する。特徴量導出部４１は、図８～図１１で示したように各種特徴量を導出し、これにより得られた特徴量情報４８をＲＷ制御部４２に出力する。ＲＷ制御部４２は、ユーザＩＤに対応する画像ＤＢ３６の記憶領域４５に、画像４６、付帯情報４７、および特徴量情報４８を記憶する。これにより、画像管理サーバ１２は、複数の画像４６を取得し、かつ、複数の画像４６について複数種類の特徴量を取得する。 The reception unit 40 accepts the image storage request 70 and outputs the image storage request 70 to the feature quantity derivation unit 41 and the RW control unit 42. The feature amount deriving unit 41 derives various feature amounts as shown in FIGS. 8 to 11, and outputs the obtained feature amount information 48 to the RW control unit 42. The RW control unit 42 stores the image 46, additional information 47, and feature amount information 48 in the storage area 45 of the image DB 36 corresponding to the user ID. Thereby, the image management server 12 acquires a plurality of images 46 and acquires a plurality of types of feature amounts for the plurality of images 46.
　一例として図１４に示すように、ブラウザ制御部３２は、ユーザＵの指示に応じて、画像４６の一覧表示の表示設定画面７５をディスプレイ２４Ａに表示する。表示設定画面７５には、画像４６の一覧表示に用いる２次元の表示空間９０（図１７参照）のＸ軸を構成する特徴量を選択するためのプルダウンメニュー７６、Ｙ軸を構成する特徴量を選択するためのプルダウンメニュー７７、および設定ボタン７８が設けられている。プルダウンメニュー７６および７７は、初期表示状態においては２つずつ用意されている。プルダウンメニュー７６および７７の下部には、特徴量追加ボタン７９および８０が設けられている。特徴量追加ボタン７９が押された場合、プルダウンメニュー７６が追加される。同様に、特徴量追加ボタン８０が押された場合、プルダウンメニュー７７が追加される。 As an example, as shown in FIG. 14, the browser control unit 32 displays a display setting screen 75 for displaying a list of images 46 on the display 24A in response to an instruction from the user U. The display setting screen 75 includes a pull-down menu 76 for selecting the feature amount forming the X-axis of the two-dimensional display space 90 (see FIG. 17) used for displaying a list of images 46, and a pull-down menu 76 for selecting the feature amount forming the Y-axis. A pull-down menu 77 and a settings button 78 are provided for selection. Two pull-down menus 76 and 77 are provided in the initial display state. At the bottom of the pull-down menus 76 and 77, feature amount addition buttons 79 and 80 are provided. When the feature amount addition button 79 is pressed, a pull-down menu 76 is added. Similarly, when the feature amount addition button 80 is pressed, the pull-down menu 77 is added.
　プルダウンメニュー７６および７７で選択可能な特徴量は、日時特徴量、色特徴量、明るさ特徴量、画質特徴量、および被写体特徴量である。複数のプルダウンメニュー７６のうち、必ず１つのプルダウンメニュー７６だけは何らかの特徴量を選択しなければならない。同様に、複数のプルダウンメニュー７７のうち、必ず１つのプルダウンメニュー７７だけは何らかの特徴量を選択しなければならない。このため表示空間９０のＸ軸およびＹ軸は、少なくとも１種類の特徴量の数値の大きさを示す軸となる。一方で、複数のプルダウンメニュー７６において特徴量が選択された場合、表示空間９０のＸ軸は、２種類以上の特徴量を統合した数値の大きさを示す軸となる。同様に、複数のプルダウンメニュー７７において特徴量が選択された場合、表示空間９０のＹ軸は、２種類以上の特徴量を統合した数値の大きさを示す軸となる。 Features that can be selected from the pull-down menus 76 and 77 are date and time features, color features, brightness features, image quality features, and subject features. Among the plurality of pull-down menus 76, only one of the pull-down menus 76 must select some feature amount. Similarly, among the plurality of pull-down menus 77, only one of the pull-down menus 77 must select some feature amount. Therefore, the X-axis and Y-axis of the display space 90 serve as axes that indicate the numerical value of at least one type of feature amount. On the other hand, when feature quantities are selected from the plurality of pull-down menus 76, the X axis of the display space 90 becomes an axis indicating the magnitude of a numerical value that integrates two or more types of feature quantities. Similarly, when a feature quantity is selected from a plurality of pull-down menus 77, the Y-axis of the display space 90 becomes an axis indicating the magnitude of a numerical value that integrates two or more types of feature quantities.
　一例として図１５に示すように、表示設定画面７５において設定ボタン７８が押された場合、ブラウザ制御部３２は画像配信要求８５を生成する。画像配信要求８５は、ユーザＩＤ、およびプルダウンメニュー７６および７７にて選択された特徴量に応じた情報を含む。特徴量に応じた情報は、プルダウンメニュー７６および７７にて選択された特徴量が被写体特徴量であった場合は、被写体特徴量を表すキーワードである。図１５においては、プルダウンメニュー７６にて犬および子どもの被写体特徴量が選択され、犬および子どもをキーワードとして含む画像配信要求８５が生成された場合を例示している。なお、プルダウンメニュー７６および７７にて選択された特徴量が、いずれも被写体特徴量以外の特徴量であった場合は、過去１年間等の予め設定された期間の画像４６を指定する期間指定情報が、特徴量に応じた情報として画像配信要求８５に登録される。 As an example, as shown in FIG. 15, when the setting button 78 is pressed on the display setting screen 75, the browser control unit 32 generates an image distribution request 85. The image distribution request 85 includes the user ID and information according to the feature amount selected from the pull-down menus 76 and 77. The information corresponding to the feature amount is a keyword representing the object feature amount when the feature amount selected from the pull-down menus 76 and 77 is the object feature amount. In FIG. 15, a case is illustrated in which subject features of a dog and a child are selected in the pull-down menu 76, and an image distribution request 85 including the keywords dog and child is generated. Note that if the feature amounts selected in the pull-down menus 76 and 77 are all feature amounts other than the subject feature amount, period designation information that specifies images 46 from a preset period such as the past year is used. is registered in the image distribution request 85 as information corresponding to the feature amount.
　一例として図１６に示すように、ブラウザ制御部３２は、画像配信要求８５を画像管理サーバ１２に送信する。受付部４０は画像配信要求８５を受け付け、画像配信要求８５をＲＷ制御部４２に出力する。ＲＷ制御部４２は、画像配信要求８５のユーザＩＤに対応する記憶領域４５に記憶された画像４６のうち、画像配信要求８５の特徴量に応じた情報に該当する画像４６を検索する。ＲＷ制御部４２は、検索した画像４６と、その付帯情報４７（図１６においては不図示）および特徴量情報４８とを配信制御部４３に出力する。配信制御部４３は、ＲＷ制御部４２からの画像４６等を、画像配信要求８５の要求元のユーザ端末１０に配信する。配信制御部４３は、画像配信要求８５の要求元のユーザ端末１０を、画像配信要求８５に含まれるユーザＩＤを元に特定する。 As an example, as shown in FIG. 16, the browser control unit 32 transmits an image distribution request 85 to the image management server 12. The reception unit 40 accepts the image distribution request 85 and outputs the image distribution request 85 to the RW control unit 42 . The RW control unit 42 searches for an image 46 corresponding to information according to the feature amount of the image distribution request 85 from among the images 46 stored in the storage area 45 corresponding to the user ID of the image distribution request 85. The RW control unit 42 outputs the searched image 46, its accompanying information 47 (not shown in FIG. 16), and feature amount information 48 to the distribution control unit 43. The distribution control unit 43 distributes the image 46 and the like from the RW control unit 42 to the user terminal 10 that has requested the image distribution request 85 . The distribution control unit 43 identifies the user terminal 10 that is the source of the image distribution request 85 based on the user ID included in the image distribution request 85.
　図１６においては、図１５の犬および子どもをキーワードとして含む画像配信要求８５の場合を例示している。この場合、ＲＷ制御部４２は、犬の被写体特徴量が０より大きい画像４６、および子どもの被写体特徴量が０より大きい画像４６を検索する。 FIG. 16 shows an example of the image distribution request 85 in FIG. 15 that includes the words dog and child as keywords. In this case, the RW control unit 42 searches for an image 46 in which the subject feature amount of the dog is greater than 0, and an image 46 in which the subject feature amount of the child is greater than 0.
　一例として図１７に示すように、ブラウザ制御部３２は、複数のプルダウンメニュー７６または７７において特徴量が選択され、２種類以上の特徴量を統合する必要がある場合、画像配信要求８５に応じて画像管理サーバ１２から配信された画像４６について、統合特徴量ＩＦを算出する。統合特徴量ＩＦは、具体的には、統合する複数種類の特徴量の加算平均値である。図１７においては、図１５および図１６の犬および子どもをキーワードとして含む画像配信要求８５の場合を例示している。この場合、ブラウザ制御部３２は、犬の被写体特徴量と子どもの被写体特徴量の加算平均値を、統合特徴量ＩＦとして算出する。例えば犬の被写体特徴量が０．６４、子どもの被写体特徴量が０．２２の画像４６２の場合、ブラウザ制御部３２は、
　ＩＦ＝（０．６４＋０．２２）／２＝０．４３
と算出する。 As an example, as shown in FIG. 17, when features are selected in multiple pull-down menus 76 or 77 and two or more types of features need to be integrated, the browser control unit 32 calculates an integrated feature IF for an image 46 delivered from the image management server 12 in response to an image delivery request 85. Specifically, the integrated feature IF is an arithmetic average value of the multiple types of feature to be integrated. FIG. 17 illustrates an example of an image delivery request 85 including keywords for dog and child as shown in FIG. 15 and FIG. 16. In this case, the browser control unit 32 calculates the arithmetic average value of the subject feature of the dog and the subject feature of the child as the integrated feature IF. For example, in the case of an image 462 in which the subject feature of the dog is 0.64 and the subject feature of the child is 0.22, the browser control unit 32 calculates the integrated feature IF as the arithmetic average value.
IF = (0.64 + 0.22) / 2 = 0.43
It is calculated as follows.
　ブラウザ制御部３２は、Ｘ軸およびＹ軸を有する特徴量に係る２次元の表示空間９０において、特徴量に応じた位置に画像４６を配置する。ブラウザ制御部３２は、表示空間９０において画像４６が重なり合う場合、撮影日時が新しい画像４６を手前に表示する。なお、本実施形態において、表示空間９０の特徴量に応じた位置とは、場合により数百～数千種類ある特徴量のうちの少なくとも３種の特徴量に応じた位置を指し、数百～数千種類ある特徴量の全てに応じた位置であることを必要としない。 The browser control unit 32 places the image 46 at a position according to the feature in a two-dimensional display space 90 related to the feature having an X axis and a Y axis. When images 46 overlap in the display space 90, the browser control unit 32 displays the image 46 with the most recent shooting date and time in front. Note that in this embodiment, a position according to the feature in the display space 90 refers to a position according to at least three types of feature out of the hundreds to thousands of types of feature, and does not necessarily have to be a position according to all of the hundreds to thousands of types of feature.
　図１７における表示空間９０は、Ｘ軸が、犬および子どもの被写体特徴量を統合した数値、すなわち統合特徴量ＩＦを示す軸であり、Ｙ軸が画質特徴量を示す軸である。Ｘ軸は、本開示の技術に係る「２種類以上の特徴量を統合した数値の大きさを示す軸」の一例である。なお、Ｘ軸だけでなくＹ軸も、２種類以上の特徴量を統合した数値の大きさを示す軸としてもよい。 In the display space 90 in FIG. 17, the X axis is an axis indicating a numerical value that integrates the subject feature quantities of the dog and the child, that is, the integrated feature quantity IF, and the Y axis is the axis indicating the image quality feature quantity. The X-axis is an example of "an axis that indicates the magnitude of a numerical value that integrates two or more types of feature amounts" according to the technology of the present disclosure. Note that not only the X-axis but also the Y-axis may be an axis indicating the magnitude of a numerical value that integrates two or more types of feature amounts.
　ブラウザ制御部３２は、統合特徴量ＩＦが０．３４５、画質特徴量が０．８８である画像４６１を、表示空間９０の座標｛０．３４５、０．８８｝の位置に配置する。同様に、ブラウザ制御部３２は、統合特徴量ＩＦが０．４３、画質特徴量が０．５６である画像４６２を、表示空間９０の座標｛０．４３、０．５６｝の位置に配置する。また、ブラウザ制御部３２は、統合特徴量ＩＦが０．４８、画質特徴量が０．７２である画像４６３を、表示空間９０の座標｛０．４８、０．７２｝の位置に配置する。図１７の表示空間９０においては、犬の被写体特徴量、および子どもの被写体特徴量が相対的に高く、かつ画質特徴量が相対的に高い画像４６、言い換えれば犬および／または子どもが明確に被写体として写り、かつ画質が良好な画像４６程、右上の領域に配置される。ブラウザ制御部３２は、こうして表示空間９０の特徴量に応じた位置に画像４６を配置することで、複数の画像４６を一覧表示する。 The browser control unit 32 places the image 461 whose integrated feature quantity IF is 0.345 and whose image quality feature quantity is 0.88 at the position of coordinates {0.345, 0.88} in the display space 90. Similarly, the browser control unit 32 places the image 462 whose integrated feature quantity IF is 0.43 and whose image quality feature quantity is 0.56 at the position of coordinates {0.43, 0.56} in the display space 90. . Further, the browser control unit 32 arranges the image 463 whose integrated feature amount IF is 0.48 and whose image quality feature amount is 0.72 at the position of coordinates {0.48, 0.72} in the display space 90. In the display space 90 of FIG. 17, the image 46 has a relatively high subject feature of the dog and a child, and has a relatively high image quality feature, in other words, the dog and/or the child are clearly the subject. The image 46 that appears more clearly and has better image quality is placed in the upper right area. The browser control unit 32 displays a list of the plurality of images 46 by arranging the images 46 at positions corresponding to the feature amounts in the display space 90 in this manner.
　一例として図１８に示すように、ブラウザ制御部３２は、特徴量が予め設定された範囲内にある２つ以上の画像４６がある場合、２つ以上の画像４６のうちの代表画像４６ＲＰを最上位に表示する。そして、代表画像４６ＲＰの下にそれ以外の画像４６を積層した態様で表示する。代表画像４６ＲＰは、例えば、特徴量が予め設定された範囲内にある２つ以上の画像４６のうちで撮影日時が最古の画像４６である。ここで、特徴量が予め設定された範囲内にある２つ以上の画像４６とは、互いの特徴量ベクトルＶＩの距離が、予め設定された閾値未満の画像４６同士である。閾値には、例えば、数ミリ秒～数秒の間隔で撮影した連写画像を、特徴量が予め設定された範囲内にある画像４６として判定可能な値が設定されている。図１８においては、特徴量が予め設定された範囲内にある３つの画像４６を、代表画像４６ＲＰでまとめて積層表示した例を示している。 As an example, as shown in FIG. 18, when there are two or more images 46 whose feature amounts are within a preset range, the browser control unit 32 selects the most representative image 46RP of the two or more images 46. Display at the top. Then, the other images 46 are displayed in a stacked manner under the representative image 46RP. The representative image 46RP is, for example, the image 46 with the oldest photographing date and time among the two or more images 46 whose feature amounts are within a preset range. Here, two or more images 46 whose feature amounts are within a preset range are images 46 whose feature amount vectors VI are less than a preset threshold. For example, the threshold value is set to a value that allows continuous shots taken at intervals of several milliseconds to several seconds to be determined as an image 46 whose feature amount falls within a preset range. FIG. 18 shows an example in which three images 46 whose feature amounts are within a preset range are collectively displayed in a stacked manner as a representative image 46RP.
　一例として図１９に示すように、ブラウザ制御部３２は、複数の画像４６が配置された表示空間９０を含む画像一覧表示画面９５をディスプレイ２４Ａに表示する。画像一覧表示画面９５には設定変更ボタン９６が設けられている。設定変更ボタン９６が押された場合、ブラウザ制御部３２は、画像一覧表示画面９５から表示設定画面７５に表示を遷移させる。 As an example, as shown in FIG. 19, the browser control unit 32 displays an image list display screen 95 including a display space 90 in which a plurality of images 46 are arranged on the display 24A. A setting change button 96 is provided on the image list display screen 95. When the settings change button 96 is pressed, the browser control unit 32 changes the display from the image list display screen 95 to the display settings screen 75.
　図２０および図２１は、表示空間９０の軸を構成する特徴量の変更指示に応じて、表示空間９０の仕様を変更する処理の一例を示す。図２０は、図１９で示した表示空間９０を含む画像一覧表示画面９５において設定変更ボタン９６が押され、さらに表示設定画面７５において、Ｘ軸を構成する特徴量が日時特徴量に変更（Ｙ軸は画質特徴量のまま変更なし）された場合を示す。この場合、ブラウザ制御部３２は、Ｘ軸が日時特徴量、Ｙ軸が画質特徴量の表示空間９０に画像４６を配置し直し、Ｘ軸が日時特徴量、Ｙ軸が画質特徴量の表示空間９０を含む画像一覧表示画面９５をディスプレイ２４Ａに表示する。Ｘ軸が日時特徴量、Ｙ軸が画質特徴量の表示空間９０においては、日時特徴量が相対的に高く、かつ画質特徴量が相対的に高い画像４６、言い換えれば撮影日時が新しく、かつ画質が良好な画像４６程、右上の領域に配置される。 20 and 21 show an example of a process for changing the specifications of the display space 90 in response to an instruction to change the feature quantities constituting the axes of the display space 90. FIG. 20 shows a case where the setting change button 96 is pressed on the image list display screen 95 including the display space 90 shown in FIG. 19, and furthermore, on the display setting screen 75, the feature quantities constituting the X-axis are changed to date and time features (the Y-axis remains unchanged as image quality features). In this case, the browser control unit 32 rearranges the images 46 in the display space 90 with the X-axis being the date and time features and the Y-axis being the image quality features, and displays on the display 24A the image list display screen 95 including the display space 90 with the X-axis being the date and time features and the Y-axis being the image quality features. In the display space 90 with the X-axis being the date and time features and the Y-axis being the image quality features, the images 46 with relatively high date and time features and relatively high image quality features, in other words, the images 46 with newer shooting dates and better image quality, are arranged in the upper right area.
　図２１は、図２０で示した表示空間９０を含む画像一覧表示画面９５において設定変更ボタン９６が押され、さらに表示設定画面７５において、Ｘ軸を構成する特徴量が高山植物の被写体特徴量に変更され、Ｙ軸を構成する特徴量が色特徴量のうちの赤色特徴量に変更された場合を示す。この場合、ブラウザ制御部３２は、Ｘ軸が高山植物の被写体特徴量、Ｙ軸が赤色特徴量の表示空間９０に画像４６を配置し直し、Ｘ軸が高山植物の被写体特徴量、Ｙ軸が赤色特徴量の表示空間９０を含む画像一覧表示画面９５をディスプレイ２４Ａに表示する。Ｘ軸が高山植物の被写体特徴量、Ｙ軸が赤色特徴量の表示空間９０においては、高山植物の被写体特徴量が相対的に高く、かつ赤色特徴量が相対的に高い画像４６、言い換えれば赤色の高山植物が明確に被写体として写った画像４６程、右上の領域に配置される。 21 shows a case where the setting change button 96 is pressed on the image list display screen 95 including the display space 90 shown in FIG. 20, and further, on the display setting screen 75, the feature quantity constituting the X-axis is changed to the subject feature quantity of alpine plants, and the feature quantity constituting the Y-axis is changed to the red feature quantity among the color features. In this case, the browser control unit 32 rearranges the images 46 in the display space 90 where the X-axis is the subject feature quantity of alpine plants and the Y-axis is the red feature quantity, and displays the image list display screen 95 including the display space 90 where the X-axis is the subject feature quantity of alpine plants and the Y-axis is the red feature quantity on the display 24A. In the display space 90 where the X-axis is the subject feature quantity of alpine plants and the Y-axis is the red feature quantity, images 46 with relatively high subject feature quantities of alpine plants and relatively high red feature quantities, in other words images 46 in which red alpine plants are clearly depicted as subjects, are arranged in the upper right region.
　このように、ブラウザ制御部３２は、表示空間９０の仕様のユーザＵによる変更指示として、表示空間９０の軸を構成する特徴量の変更指示を受け付ける。そして、変更指示にしたがって表示空間９０の仕様を変更する。なお、図２０および図２１では、Ｘ軸およびＹ軸を構成する特徴量を１種類に変更する例を示したが、もちろん、Ｘ軸およびＹ軸を構成する特徴量を複数種類に変更してもよい。 In this manner, the browser control unit 32 receives an instruction to change the feature amount forming the axis of the display space 90 as an instruction from the user U to change the specifications of the display space 90. Then, the specifications of the display space 90 are changed according to the change instruction. Note that although FIGS. 20 and 21 show an example in which the feature quantities configuring the X-axis and Y-axis are changed to one type, it is of course possible to change the feature quantities configuring the X-axis and Y-axis to multiple types. Good too.
　一例として図２２に示すように、画像一覧表示画面９５においては、表示空間９０上でユーザＵによる画像４６の選択指示が可能となっている。図２２は、ディスプレイ２４Ａがタッチパネルディスプレイであった場合を例示している。この場合、ユーザＵは、指Ｆでディスプレイ２４Ａの表面をなぞることで選択領域１００を指定する。こうして指定された選択領域１００内に存在する画像４６が選択される。以下、こうしてユーザＵにより選択された画像４６を、選択画像４６Ｓと表記する。 As an example, as shown in FIG. 22, on the image list display screen 95, the user U can select an image 46 on the display space 90. FIG. 22 illustrates a case where the display 24A is a touch panel display. In this case, the user U specifies the selection area 100 by tracing the surface of the display 24A with the finger F. The image 46 existing within the designated selection area 100 is thus selected. Hereinafter, the image 46 selected by the user U will be referred to as a selected image 46S.
　一例として図２３に示すように、選択領域１００が指定された場合、画像一覧表示画面９５には処理指示メニュー１０１がポップアップ表示される。処理指示メニュー１０１には、アルバム作成、一括編集、および一括削除の３つの選択肢が用意されている。 As an example, as shown in FIG. 23, when the selection area 100 is specified, a processing instruction menu 101 is pop-up displayed on the image list display screen 95. The processing instruction menu 101 has three options: album creation, batch editing, and batch deletion.
　処理指示メニュー１０１のアルバム作成の選択肢が選択された場合、一例として図２４に示すように、ブラウザ制御部３２は、アルバム作成画面１０５をディスプレイ２４Ａに表示する。アルバム作成画面１０５には、選択画像４６Ｓを用いて作成されたアルバム１０６が表示される。アルバム作成画面１０５においては、アルバム１０６の選択画像４６Ｓの配置を変更したり、新たな選択画像４６Ｓをアルバム１０６に加えたり、逆にアルバム１０６の選択画像４６Ｓを削除したりすることが可能である。 When the album creation option of the processing instruction menu 101 is selected, the browser control unit 32 displays an album creation screen 105 on the display 24A, as shown in FIG. 24 as an example. The album creation screen 105 displays an album 106 created using the selected image 46S. On the album creation screen 105, it is possible to change the arrangement of the selected images 46S in the album 106, add a new selected image 46S to the album 106, or conversely delete the selected images 46S in the album 106. .
　アルバム作成画面１０５の下部には、保存ボタン１０７およびキャンセルボタン１０８が設けられている。保存ボタン１０７が押された場合、ブラウザ制御部３２は、アルバム１０６の記憶要求を画像管理サーバ１２に送信する。画像管理サーバ１２のＲＷ制御部４２は、アルバム１０６を画像ＤＢ３６の記憶領域４５に記憶する。キャンセルボタン１０８が押された場合、ブラウザ制御部３２は、選択画像４６Ｓが選択されていない状態の画像一覧表示画面９５に表示を戻す。 At the bottom of the album creation screen 105, a save button 107 and a cancel button 108 are provided. When the save button 107 is pressed, the browser control unit 32 transmits a storage request for the album 106 to the image management server 12. The RW control unit 42 of the image management server 12 stores the album 106 in the storage area 45 of the image DB 36. When the cancel button 108 is pressed, the browser control unit 32 returns the display to the image list display screen 95 in which the selected image 46S is not selected.
　なお、処理指示メニュー１０１の一括編集の選択肢が選択された場合、ブラウザ制御部３２は、白黒、セピア、ビビット、ソフトフォーカス、光漏れ、美白等のうちの指定された表示エフェクトを、選択画像４６Ｓに対して一括して施す。また、処理指示メニュー１０１の一括削除の選択肢が選択された場合、ブラウザ制御部３２は選択画像４６Ｓを一括削除する。 Note that when the batch editing option of the processing instruction menu 101 is selected, the browser control unit 32 applies a specified display effect such as black and white, sepia, vivid, soft focus, light leakage, whitening, etc. to the selected image 46S. Apply all at once. Further, when the option of batch deletion in the processing instruction menu 101 is selected, the browser control unit 32 deletes the selected images 46S in a batch.
　このように、ブラウザ制御部３２は、表示空間９０上でユーザＵによる画像４６の選択指示を受け付ける。そして、選択指示によって選択された選択画像４６Ｓに対してのみ、アルバム作成、一括編集、および一括削除といった指定された処理を行う。 In this way, the browser control unit 32 receives an instruction from the user U to select the image 46 on the display space 90. Then, designated processes such as album creation, batch editing, and batch deletion are performed only on the selected images 46S selected by the selection instruction.
　次に、上記構成による作用について、一例として図２５、図２６、および図２７に示すフローチャートを参照して説明する。ユーザ端末１０のＣＰＵ２２Ａは、図３で示したように、画像ＡＰ３０の起動により、ブラウザ制御部３２として機能される。また、画像管理サーバ１２のＣＰＵ２２Ｂは、図４で示したように、作動プログラム３５の起動により、受付部４０、特徴量導出部４１、ＲＷ制御部４２、および配信制御部４３として機能される。 Next, the operation of the above configuration will be described as an example with reference to flowcharts shown in FIGS. 25, 26, and 27. As shown in FIG. 3, the CPU 22A of the user terminal 10 functions as the browser control unit 32 by activating the image AP 30. Further, as shown in FIG. 4, the CPU 22B of the image management server 12 functions as a reception unit 40, a feature value derivation unit 41, an RW control unit 42, and a distribution control unit 43 by starting the operation program 35.
　ユーザＵは、ユーザ端末１０のカメラ機能を用いて画像４６を撮影する。図１３で示したように、ブラウザ制御部３２の制御の下、画像４６および付帯情報４７を含む画像記憶要求７０が画像管理サーバ１２に送信される。 The user U uses the camera function of the user terminal 10 to photograph the image 46. As shown in FIG. 13, under the control of the browser control unit 32, an image storage request 70 including an image 46 and supplementary information 47 is transmitted to the image management server 12.
　画像管理サーバ１２においては、画像記憶要求７０が受付部４０にて受け付けられる（図２５のステップＳＴ１００でＹＥＳ）。画像記憶要求７０は受付部４０から特徴量導出部４１およびＲＷ制御部４２に出力される。 In the image management server 12, the image storage request 70 is accepted by the reception unit 40 (YES in step ST100 in FIG. 25). The image storage request 70 is output from the reception unit 40 to the feature quantity derivation unit 41 and the RW control unit 42.
　特徴量導出部４１においては、図８～図１１で示したように各種特徴量が導出される（ステップＳＴ１１０）。各種特徴量で構成される特徴量情報４８は、特徴量導出部４１からＲＷ制御部４２に出力される。そして、ＲＷ制御部４２の制御の下、画像４６、付帯情報４７、および特徴量情報４８が画像ＤＢ３６に記憶される（ステップＳＴ１２０）。 The feature amount deriving unit 41 derives various feature amounts as shown in FIGS. 8 to 11 (step ST110). Feature amount information 48 composed of various feature amounts is output from the feature amount deriving section 41 to the RW control section 42. Then, under the control of the RW control unit 42, the image 46, additional information 47, and feature information 48 are stored in the image DB 36 (step ST120).
　図１５で示したように、ユーザＵは、表示設定画面７５において、プルダウンメニュー７６および７７により表示空間９０のＸ軸およびＹ軸を構成する特徴量を選択した後、設定ボタン７８を押す。これによりブラウザ制御部３２によって画像配信要求８５が生成され、画像配信要求８５が画像管理サーバ１２に送信される。 As shown in FIG. 15, on the display setting screen 75, the user U selects the feature quantities forming the X-axis and Y-axis of the display space 90 from the pull-down menus 76 and 77, and then presses the setting button 78. As a result, the browser control unit 32 generates an image distribution request 85, and transmits the image distribution request 85 to the image management server 12.
　図１６で示したように、画像管理サーバ１２においては、画像配信要求８５が受付部４０にて受け付けられる（図２６のステップＳＴ２００でＹＥＳ）。画像配信要求８５は受付部４０からＲＷ制御部４２に出力される。そして、ＲＷ制御部４２の制御の下、画像ＤＢ３６から画像配信要求８５に応じた画像４６が読み出される（ステップＳＴ２１０）。画像４６は、付帯情報４７および特徴量情報４８とともに、ＲＷ制御部４２から配信制御部４３に出力される。画像４６、付帯情報４７、および特徴量情報４８は、配信制御部４３の制御の下、画像配信要求８５の要求元のユーザ端末１０に配信される（ステップＳＴ２２０）。 As shown in FIG. 16, in the image management server 12, the image distribution request 85 is received by the reception unit 40 (YES in step ST200 in FIG. 26). The image distribution request 85 is output from the reception unit 40 to the RW control unit 42. Then, under the control of the RW control unit 42, the image 46 corresponding to the image distribution request 85 is read from the image DB 36 (step ST210). The image 46 is outputted from the RW control section 42 to the distribution control section 43 along with additional information 47 and feature amount information 48 . The image 46, the supplementary information 47, and the feature amount information 48 are distributed to the user terminal 10 that requested the image distribution request 85 under the control of the distribution control unit 43 (step ST220).
　ユーザ端末１０においては、画像管理サーバ１２から配信された画像４６等がブラウザ制御部３２にて受信される（図２７のステップＳＴ３００でＹＥＳ）。図１７で示したように、画像４６は、ブラウザ制御部３２の制御の下、表示設定画面７５で設定された表示空間９０において、特徴量に応じた位置に配置される（ステップＳＴ３１０）。そして、ブラウザ制御部３２の制御の下、表示空間９０を含む画像一覧表示画面９５がディスプレイ２４Ａに表示される（ステップＳＴ３２０）。このとき、特徴量が予め設定された範囲内にある２つ以上の画像４６は、図１８で示したように、代表画像４６ＲＰが最上位に表示され、代表画像４６ＲＰの下にそれ以外の画像４６が積層された態様で表示される。 In the user terminal 10, the browser control unit 32 receives the images 46 and the like distributed from the image management server 12 (YES in step ST300 in FIG. 27). As shown in FIG. 17, the image 46 is placed at a position corresponding to the feature amount in the display space 90 set on the display setting screen 75 under the control of the browser control unit 32 (step ST310). Then, under the control of the browser control unit 32, an image list display screen 95 including the display space 90 is displayed on the display 24A (step ST320). At this time, among the two or more images 46 whose feature amounts are within a preset range, as shown in FIG. 18, the representative image 46RP is displayed at the top, and the other images are displayed below the representative image 46RP. 46 are displayed in a stacked manner.
　以上説明したように、画像管理サーバ１２のＣＰＵ２２ＢのＲＷ制御部４２は、複数の画像４６を取得し、複数の画像４６について複数種類の特徴量を取得する。ユーザ端末１０のＣＰＵ２２Ａのブラウザ制御部３２は、特徴量に係る２次元の表示空間９０であって、２種類以上の特徴量を統合した数値の大きさを示す軸を少なくとも１軸以上有する表示空間９０において、特徴量に応じた位置に画像４６を配置することで、複数の画像４６を一覧表示する。したがって、図２０で示したＸ軸が日時特徴量、Ｙ軸が画質特徴量の表示空間９０、および図２１で示したＸ軸が高山植物の被写体特徴量、Ｙ軸が赤色特徴量の表示空間９０等、１種類の特徴量を示すＸ軸およびＹ軸を有する表示空間と比べて、バラエティーに富んだ画像４６の一覧表示を行うことが可能となる。 As described above, the RW control unit 42 of the CPU 22B of the image management server 12 acquires multiple images 46 and acquires multiple types of feature amounts for the multiple images 46. The browser control unit 32 of the CPU 22A of the user terminal 10 displays a list of the multiple images 46 by arranging the images 46 at positions according to the feature amounts in a two-dimensional display space 90 related to the feature amounts, the display space 90 having at least one axis indicating the magnitude of a numerical value obtained by integrating two or more types of feature amounts. Therefore, it is possible to display a list of a wide variety of images 46 compared to display spaces having X and Y axes indicating one type of feature amount, such as the display space 90 shown in FIG. 20, in which the X axis indicates the date and time feature amount and the Y axis indicates the image quality feature amount, and the display space 90 shown in FIG. 21, in which the X axis indicates the subject feature amount of alpine plants and the Y axis indicates the red color feature amount.
　ブラウザ制御部３２は、表示空間９０の仕様のユーザＵによる変更指示を受け付け、変更指示にしたがって表示空間９０の仕様を変更する。このため、容易にユーザＵの意図に合った表示空間９０の仕様に変更することができる。結果として、ユーザＵが所望の画像４６を見つけやすくなり、所望の画像４６の検索に要する時間を短縮することができる。 The browser control unit 32 accepts an instruction from the user U to change the specifications of the display space 90, and changes the specifications of the display space 90 in accordance with the change instruction. This makes it easy to change the specifications of the display space 90 to match the intentions of the user U. As a result, it becomes easier for the user U to find the desired image 46, and the time required to search for the desired image 46 can be reduced.
　変更指示は、表示空間９０の軸を構成する特徴量の変更指示である。このため、ユーザＵの気持ちの赴くままに、気軽に表示空間９０の軸を変更することができる。これにより、今まで撮影しただけで忘れていた画像４６が表示される等、思わぬ画像４６の発見があったりして、ユーザＵの興趣を引き付けることができる。 The change instruction is an instruction to change the feature amount that constitutes the axis of the display space 90. Therefore, the axis of the display space 90 can be easily changed as the user U desires. As a result, an unexpected image 46 may be discovered, such as displaying an image 46 that has been taken but forgotten, thereby attracting the user U's interest.
　ブラウザ制御部３２は、特徴量が予め設定された範囲内にある２つ以上の画像４６がある場合、２つ以上の画像４６のうちの代表画像４６ＲＰを最上位に表示し、代表画像４６ＲＰの下にそれ以外の画像４６を積層した態様で表示する。このため、特徴量が予め設定された範囲内にある２つ以上の画像４６の表示がすっきりとし、画像４６の一覧表示を一段と見やすくすることができる。 If there are two or more images 46 whose feature amounts are within a preset range, the browser control unit 32 displays the representative image 46RP of the two or more images 46 at the top, and The other images 46 are displayed in a layered manner below. Therefore, the display of two or more images 46 whose feature amounts are within a preset range can be clearly displayed, and the list of images 46 can be displayed more easily.
　ブラウザ制御部３２は、表示空間９０上でユーザＵによる画像４６の選択指示を受け付ける。そして、選択指示によって選択された選択画像４６Ｓに対してのみ、指定された処理を行う。このため、例えば図２３等で示したように、犬および／または子どもが明確に被写体として写り、かつ画質が良好な画像４６を選択してアルバム１０６を作成したり、あるいは画質が相対的に悪い画像４６を選択して一括削除したりすることができ、ユーザＵによる画像４６の整理、編集作業を大いに捗らせることができる。 The browser control unit 32 receives an instruction from the user U to select the image 46 on the display space 90. Then, the designated process is performed only on the selected image 46S selected by the selection instruction. For this reason, as shown in FIG. 23, for example, an album 106 may be created by selecting an image 46 in which a dog and/or child is clearly shown as the subject and with good image quality, or an image 46 with relatively poor image quality. The images 46 can be selected and deleted all at once, and the user U's work on organizing and editing the images 46 can be greatly facilitated.
　なお、一例として図２８に示す画像一覧表示画面１１０を採用してもよい。画像一覧表示画面１１０は、表示設定画面７５の機能を画像一覧表示画面９５に付与した画面である。具体的には、画像一覧表示画面１１０には、Ｘ軸を構成する特徴量を選択するためのプルダウンメニュー１１１、およびＹ軸を構成する特徴量を選択するためのプルダウンメニュー１１２が設けられている。プルダウンメニュー１１１および１１２は、初期表示状態においては２つずつ用意されている。プルダウンメニュー１１１および１１２の右側には、特徴量追加ボタン１１３および１１４が設けられている。特徴量追加ボタン１１３が押された場合、プルダウンメニュー１１１が追加される。同様に、特徴量追加ボタン１１４が押された場合、プルダウンメニュー１１２が追加される。プルダウンメニュー１１１および１１２は、表示設定画面７５のプルダウンメニュー７６および７７に相当する。また、特徴量追加ボタン１１３および１１４は、表示設定画面７５の特徴量追加ボタン７９および８０に相当する。この画像一覧表示画面１１０によれば、設定変更ボタン９６を押して表示設定画面７５に表示を遷移させる必要がなくなるため、ユーザＵの操作の手間を省くことができる。 Incidentally, as an example, an image list display screen 110 shown in FIG. 28 may be adopted. The image list display screen 110 is a screen in which the functions of the display setting screen 75 are added to the image list display screen 95. Specifically, the image list display screen 110 is provided with a pull-down menu 111 for selecting the feature amounts forming the X-axis and a pull-down menu 112 for selecting the feature amounts forming the Y-axis. . Two pull-down menus 111 and 112 are provided in the initial display state. On the right side of the pull-down menus 111 and 112, feature amount addition buttons 113 and 114 are provided. When the feature amount addition button 113 is pressed, the pull-down menu 111 is added. Similarly, when the feature quantity addition button 114 is pressed, the pull-down menu 112 is added. Pull-down menus 111 and 112 correspond to pull-down menus 76 and 77 on display setting screen 75. Further, the feature value addition buttons 113 and 114 correspond to the feature value addition buttons 79 and 80 on the display setting screen 75. According to this image list display screen 110, there is no need to press the setting change button 96 to change the display to the display setting screen 75, so that the user U's operation effort can be saved.
　［第２実施形態］
　一例として図２９に示すように、第２実施形態の表示設定画面１２０には、上記第１実施形態のプルダウンメニュー７６および７７等に加えて、重み係数設定バー１２１および１２２が設けられている。重み係数設定バー１２１は、プルダウンメニュー７６により選択されたＸ軸を構成する特徴量に掛ける重み係数を、０．１以上１以下の値に設定するためのＧＵＩである。重み係数設定バー１２２は、プルダウンメニュー７７により選択されたＹ軸を構成する特徴量に掛ける重み係数を、０．１以上１以下の値に設定するためのＧＵＩである。重み係数設定バー１２１および１２２は、プルダウンメニュー７６および７７と同じく、初期表示状態においては２つずつ用意されている。また、重み係数設定バー１２１および１２２は、初期表示状態においては重み係数が１に設定されている。 [Second embodiment]
As an example, as shown in FIG. 29, the display setting screen 120 of the second embodiment is provided with weighting factor setting bars 121 and 122 in addition to the pull-down menus 76 and 77 of the first embodiment. The weighting factor setting bar 121 is a GUI for setting the weighting factor to be applied to the feature amount constituting the X axis selected by the pull-down menu 76 to a value of 0.1 or more and 1 or less. The weighting factor setting bar 122 is a GUI for setting the weighting factor to be applied to the feature amount constituting the Y axis selected by the pull-down menu 77 to a value of 0.1 or more and 1 or less. Like the pull-down menus 76 and 77, two weighting factor setting bars 121 and 122 are provided in the initial display state. Furthermore, the weighting coefficients of the weighting coefficient setting bars 121 and 122 are set to 1 in the initial display state.
　図２９においては、プルダウンメニュー７６にて犬および子どもの被写体特徴量が選択され、プルダウンメニュー７７にて画質特徴量が選択された場合を例示している。また、図２９においては、重み係数設定バー１２１にて犬の被写体特徴量に掛ける重み係数に０．５、子どもの被写体特徴量に掛ける重み係数に１がそれぞれ設定され、重み係数設定バー１２２にて画質特徴量に掛ける重み係数に１が設定された場合を例示している。 In FIG. 29, a case is illustrated in which subject features of a dog and a child are selected in the pull-down menu 76, and image quality features are selected in the pull-down menu 77. In addition, in FIG. 29, in the weighting factor setting bar 121, 0.5 is set for the weighting coefficient multiplied by the dog's subject feature, and 1 is set for the weighting coefficient multiplied by the child's subject feature, and in the weighting factor setting bar 122. In this example, the weighting coefficient multiplied by the image quality feature is set to 1.
　図３０は、図２９で示した表示設定の場合のブラウザ制御部３２の処理を示す。この場合も上記第１実施形態の図１７で示した場合と同様に、ブラウザ制御部３２は、画像管理サーバ１２から配信された画像４６について、統合特徴量ＩＦを算出する。ただし、この場合、ブラウザ制御部３２は、上記第１実施形態の場合は統合する複数種類の特徴量の加算平均を統合特徴量ＩＦとして算出したのに対し、統合する複数種類の特徴量の重み付き平均（加重平均）を統合特徴量ＩＦとして算出する。ブラウザ制御部３２は、重み付き平均の算出に、重み係数設定バー１２１および１２２で設定された重み係数を用いる。例えば犬の被写体特徴量が０．６４、子どもの被写体特徴量が０．２２の画像４６２の場合、ブラウザ制御部３２は、重み係数設定バー１２１で設定された犬の被写体特徴量に対する重み係数０．５および子どもの被写体特徴量１を用いて、
　ＩＦ＝（０．６４×０．５＋０．２２×１）／２＝０．２７
と算出する。以降、ブラウザ制御部３２は、上記第１実施形態の場合と同じく、表示空間９０において、特徴量に応じた位置に画像４６を配置する。 FIG. 30 shows the processing of the browser control unit 32 in the case of the display settings shown in FIG. 29. In this case as well, similarly to the case shown in FIG. 17 of the first embodiment, the browser control unit 32 calculates the integrated feature amount IF for the image 46 distributed from the image management server 12. However, in this case, the browser control unit 32 calculates the average of the multiple types of feature amounts to be integrated as the integrated feature amount IF in the case of the first embodiment, whereas the browser control unit 32 calculates the weight of the multiple types of feature amounts to be integrated. The weighted average (weighted average) is calculated as the integrated feature amount IF. The browser control unit 32 uses the weighting coefficients set in the weighting coefficient setting bars 121 and 122 to calculate the weighted average. For example, in the case of an image 462 in which the subject feature amount of a dog is 0.64 and the subject feature amount of a child is 0.22, the browser control unit 32 sets a weighting coefficient of 0 for the subject feature amount of a dog set in the weighting factor setting bar 121. Using .5 and the child's subject feature amount 1,
IF=(0.64×0.5+0.22×1)/2=0.27
It is calculated as follows. Thereafter, the browser control unit 32 arranges the image 46 at a position corresponding to the feature amount in the display space 90, as in the first embodiment.
　このように、第２実施形態においては、表示空間９０の仕様のユーザＵによる変更指示は、２種類以上の特徴量を統合する割合を決める重み係数の変更指示である。このため、さらにバラエティーに富んだ画像４６の一覧表示を行うことが可能となる。例えば、画質の良好な画像４６を最優先にアルバム１０６を作成したいが、撮影日時も多少は加味したい、あるいは、子どもの画像４６をメインにしたアルバム１０６を作成したいが、犬の画像４６も所々に入れたい、といったユーザＵの細かな要望に応えることができる。 As described above, in the second embodiment, the instruction to change the specifications of the display space 90 by the user U is an instruction to change the weighting coefficient that determines the ratio of integrating two or more types of feature amounts. Therefore, it is possible to display a list of images 46 with even greater variety. For example, you might want to create an album 106 that gives top priority to images 46 with good image quality, but you also want to take the date and time of shooting into account, or you want to create an album 106 that mainly includes images 46 of children, but there are also images 46 of dogs here and there. It is possible to respond to user U's detailed requests, such as wanting to include
　［第３実施形態］
　一例として図３１に示すように、第３実施形態の画像一覧表示画面１２５は、画像４６が行列状に一覧表示される領域１２６と、表示空間９０とを含む。領域１２６の画像４６は、ユーザＵの指Ｆによるドラッグアンドドロップ操作により、表示空間９０の任意の位置に移動させることが可能である。以下、ドラッグアンドドロップ操作により領域１２６から表示空間９０に移動させられた画像４６を、基準画像４６Ｒと表記する。 [Third embodiment]
As shown in FIG. 31 as an example, the image list display screen 125 of the third embodiment includes an area 126 in which images 46 are displayed as a list in a matrix, and a display space 90. The image 46 in the area 126 can be moved to any position in the display space 90 by a drag-and-drop operation using the finger F of the user U. Hereinafter, the image 46 moved from the area 126 to the display space 90 by the drag-and-drop operation will be referred to as a reference image 46R.
　ここで、第３実施形態の表示空間９０は、２次元よりも多い次元数を有する特徴量を、ｔ－ＳＮＥ（Ｔ－ｄｉｓｔｒｉｂｕｔｅｄ　Ｓｔｏｃｈａｓｔｉｃ　Ｎｅｉｇｈｂｏｒ　Ｅｍｂｅｄｄｉｎｇ）といった周知の次元削減技術を用いて２次元に次元削減することで得られた空間である。このため、第３実施形態の表示空間９０のＸ軸およびＹ軸を構成する特徴量は、特定の種類の特徴量に決まらない。 Here, the display space 90 of the third embodiment converts feature quantities having more dimensions than two dimensions into two dimensions using a well-known dimension reduction technique such as t-SNE (T-distributed Stochastic Neighbor Embedding). This is the space gained through reduction. Therefore, the feature amounts forming the X-axis and Y-axis of the display space 90 in the third embodiment are not determined to be a specific type of feature amount.
　一例として図３２および図３３に示すように、ブラウザ制御部３２は、基準画像４６Ｒのドラッグアンドドロップ操作がなされた場合、基準画像４６Ｒを中心として、基準画像４６Ｒの特徴量ベクトルＶＩとの距離に応じた位置に他の画像４６を配置する。このため、基準画像４６Ｒと特徴量ベクトルＶＩが類似し、基準画像４６Ｒの特徴量ベクトルＶＩとの距離が近い画像４６は、基準画像４６Ｒに近い位置に配置される。反対に、基準画像４６Ｒと特徴量ベクトルＶＩが乖離し、基準画像４６Ｒの特徴量ベクトルＶＩとの距離が遠い画像４６は、基準画像４６Ｒから遠い位置に配置される。基準画像４６Ｒと特徴量ベクトルＶＩが類似している画像とは、端的に言えば基準画像４６Ｒの類似画像である。また、基準画像４６Ｒと特徴量ベクトルＶＩが乖離している画像とは、端的に言えば基準画像４６Ｒの非類似画像である。 As an example, as shown in FIGS. 32 and 33, when a drag-and-drop operation is performed on the reference image 46R, the browser control unit 32 adjusts the distance between the reference image 46R and the feature vector VI, centering on the reference image 46R. Another image 46 is placed at the corresponding position. Therefore, the image 46 whose feature vector VI is similar to the reference image 46R and whose distance to the feature vector VI of the reference image 46R is close is placed at a position close to the reference image 46R. On the other hand, an image 46 in which the reference image 46R and the feature vector VI deviate from each other, and the distance between the reference image 46R and the feature vector VI is long, is placed at a position far from the reference image 46R. Simply put, an image whose feature vector VI is similar to the reference image 46R is an image similar to the reference image 46R. Moreover, an image in which the reference image 46R and the feature amount vector VI are different from each other is, simply put, an image that is dissimilar to the reference image 46R.
　ブラウザ制御部３２は、基準画像４６Ｒに色付き、例えば赤色の丸印１２８を付し、基準画像４６Ｒを他の画像４６と区別して表示する。なお、図３２は基準画像４６Ｒが１つの場合、図３３は基準画像４６Ｒが２つの場合をそれぞれ示す。 The browser control unit 32 adds a color, for example, a red circle 128, to the reference image 46R, and displays the reference image 46R to distinguish it from other images 46. Note that FIG. 32 shows a case where there is one reference image 46R, and FIG. 33 shows a case where there are two reference images 46R.
　このように、第３実施形態においては、表示空間９０の任意の位置に配置された基準画像４６Ｒを中心として、基準画像４６Ｒの特徴量ベクトルＶＩとの距離に応じた位置に他の画像４６を配置する。このため、さらにバラエティーに富んだ画像４６の一覧表示を行うことが可能となる。ユーザＵが気になる基準画像４６Ｒの類似画像および非類似画像が一目で分かるので、ユーザＵによる画像４６の整理、編集作業を大いに捗らせることができる。 In this way, in the third embodiment, with the reference image 46R placed at an arbitrary position in the display space 90 as the center, other images 46 are placed at positions corresponding to the distance from the feature vector VI of the reference image 46R. Deploy. Therefore, it is possible to display a list of images 46 with even greater variety. Since images similar to and dissimilar to the reference image 46R that the user U is interested in can be seen at a glance, the user U can greatly facilitate the organization and editing of the images 46.
　［第４実施形態］
　第４実施形態では、下記の式（１）に示すように、各特徴量Ｚｉ（ｉ＝１、２、３、・・・、ｎ－１、ｎ　ｎは特徴量の数、すなわち特徴量ベクトルＶＩの次元数）に重み係数Ｗｉを掛けたものを画像４６の特徴量ベクトルＶＩとする。
　ＶＩ＝｛Ｗｉ×Ｚｉ｝＝｛Ｗ１×Ｚ１、Ｗ２×Ｚ２、Ｗ３×Ｚ３、・・・、Ｗｎ－１×Ｚｎ－１、Ｗｎ×Ｚｎ｝・・・（１） [Fourth embodiment]
In the fourth embodiment, each feature amount Zi (i=1, 2, 3, ..., n-1, n is the number of feature amounts, that is, the feature vector The feature quantity vector VI of the image 46 is obtained by multiplying the number of dimensions of VI by the weighting coefficient Wi.
VI={Wi×Zi}={W1×Z1, W2×Z2, W3×Z3,..., Wn-1×Zn-1, Wn×Zn}...(1)
　一例として図３４に示すように、ブラウザ制御部３２は、ユーザＵの指示に応じて、類似画像選択画面１３０をディスプレイ２４Ａに表示する。類似画像選択画面１３０は、ユーザＵが類似していると考える画像４６を選択するための画面である。類似画像選択画面１３０には画像４６が行列状に一覧表示される。 As an example, as shown in FIG. 34, the browser control unit 32 displays a similar image selection screen 130 on the display 24A in response to an instruction from the user U. The similar image selection screen 130 is a screen for selecting an image 46 that the user U considers to be similar. The images 46 are displayed in a matrix format on the similar image selection screen 130.
　ユーザＵは、一覧表示された画像４６の中から、ハッチングで示すように、自らが類似していると考える画像を２つ以上選択する。その後、ユーザＵはＯＫボタン１３１を押す。ＯＫボタン１３１が押された場合、ブラウザ制御部３２は、ユーザＵが選択した画像４６等を含む重み係数補正要求を画像管理サーバ１２に送信する。以下、ユーザＵが類似していると考えて選択した画像４６を、類似画像４６ＳＭと表記する。 The user U selects two or more images that the user U considers to be similar, as shown by hatching, from the images 46 displayed in a list. After that, the user U presses the OK button 131. When the OK button 131 is pressed, the browser control unit 32 transmits a weighting coefficient correction request including the image 46 selected by the user U to the image management server 12. Hereinafter, the image 46 selected by the user U considering it to be similar will be referred to as a similar image 46SM.
　一例として図３５に示すように、第４実施形態の画像管理サーバ１２のＣＰＵ２２Ｂは、上記第１実施形態の各処理部４０～４３に加えて、重み係数補正部１３５として機能する。重み係数補正部１３５には、類似画像４６ＳＭの特徴量情報４８ＳＭが入力される。重み係数補正部１３５は、複数の類似画像４６ＳＭの特徴量ベクトルＶＩの距離が、予め設定された第１閾値未満となる重み係数Ｗｉを求める最適化問題を解くことで、補正後重み係数ＷｉＣを算出する。第１閾値には、例えば０．１等、０に近い値が設定される。すなわち、重み係数補正部１３５は、類似画像４６ＳＭ同士の特徴量が一致する方向に重み係数Ｗｉを変更し、補正後重み係数ＷｉＣとする。 As an example, as shown in FIG. 35, the CPU 22B of the image management server 12 of the fourth embodiment functions as a weighting factor correction unit 135 in addition to each of the processing units 40 to 43 of the first embodiment. The feature information 48SM of the similar image 46SM is input to the weighting coefficient correction unit 135. The weighting factor correction unit 135 calculates the corrected weighting factor WiC by solving an optimization problem for finding a weighting factor Wi such that the distance between the feature vectors VI of the plurality of similar images 46SM is less than a first preset threshold. calculate. The first threshold is set to a value close to 0, such as 0.1, for example. That is, the weighting coefficient correction unit 135 changes the weighting coefficient Wi in a direction in which the feature amounts of the similar images 46SM match each other, and sets the weighting coefficient Wi to a corrected weighting coefficient WiC.
　また、一例として図３６に示すように、ブラウザ制御部３２は、ユーザＵの指示に応じて、非類似画像選択画面１４０をディスプレイ２４Ａに表示する。非類似画像選択画面１４０は、類似画像選択画面１３０とは逆に、ユーザＵが全く異なると考える画像４６を選択するための画面である。非類似画像選択画面１４０には、類似画像選択画面１３０と同じく、画像４６が行列状に一覧表示される。 Furthermore, as shown in FIG. 36 as an example, the browser control unit 32 displays a dissimilar image selection screen 140 on the display 24A in response to an instruction from the user U. The dissimilar image selection screen 140 is a screen for selecting an image 46 that the user U considers to be completely different from the similar image selection screen 130. Similar to the similar image selection screen 130, the dissimilar image selection screen 140 displays a list of images 46 in a matrix.
　ユーザＵは、一覧表示された画像４６の中から、ハッチングで示すように、自らが全く異なると考える画像を２つ以上選択する。その後、ユーザＵはＯＫボタン１４１を押す。ＯＫボタン１４１が押された場合、ブラウザ制御部３２は、ユーザＵが選択した画像４６等を含む重み係数補正要求を画像管理サーバ１２に送信する。以下、ユーザＵが全く異なると考えて選択した画像４６を、非類似画像４６ＤＳＭと表記する。 The user U selects two or more images that the user U considers to be completely different, as shown by hatching, from among the images 46 displayed in a list. After that, the user U presses the OK button 141. When the OK button 141 is pressed, the browser control unit 32 transmits a weighting coefficient correction request including the image 46 selected by the user U to the image management server 12. Hereinafter, the image 46 selected by the user U considering it to be completely different will be referred to as a dissimilar image 46DSM.
　一例として図３７に示すように、この場合、重み係数補正部１３５には、非類似画像４６ＤＳＭの特徴量情報４８ＤＳＭが入力される。重み係数補正部１３５は、複数の非類似画像４６ＤＳＭの特徴量ベクトルＶＩの距離が、予め設定された第２閾値以上となる重み係数Ｗｉを求める最適化問題を解くことで、補正後重み係数ＷｉＣを算出する。第２閾値には、例えば上記第３実施形態で示した表示空間９０において、左下の原点の位置と右上の特徴量が最大の位置といった、非類似画像４６ＤＳＭの配置位置が最も離れるような値が設定される。すなわち、重み係数補正部１３５は、非類似画像４６ＤＳＭ同士の特徴量が乖離する方向に重み係数Ｗｉを変更し、補正後重み係数ＷｉＣとする。 As shown in FIG. 37 as an example, in this case, the feature amount information 48DSM of the dissimilar image 46DSM is input to the weighting coefficient correction unit 135. The weighting coefficient correction unit 135 solves an optimization problem to obtain a weighting coefficient Wi such that the distance between the feature vectors VI of the plurality of dissimilar images 46DSM is equal to or greater than a preset second threshold value, thereby obtaining a corrected weighting coefficient WiC. Calculate. The second threshold value is set to a value such that the placement position of the dissimilar image 46DSM is farthest apart, such as the position of the lower left origin and the position where the upper right feature amount is maximum, in the display space 90 shown in the third embodiment, for example. Set. That is, the weighting coefficient correction unit 135 changes the weighting coefficient Wi in a direction in which the feature amounts of the dissimilar images 46DSM deviate from each other, and sets it as a corrected weighting coefficient WiC.
　このように、第４実施形態においては、ブラウザ制御部３２は、類似画像４６ＳＭ、および非類似画像４６ＤＳＭのうちの少なくともいずれかのユーザＵによる選択指示を受け付ける。重み係数補正部１３５は、選択指示に応じて、類似画像４６ＳＭ同士については特徴量が一致する方向に、非類似画像４６ＤＳＭ同士については特徴量が乖離する方向に、複数種類の特徴量に掛ける重み係数Ｗｉを変更する。このため、表示空間９０において、ユーザＵが類似していると考える画像４６同士が相対的に近い位置に配置され、ユーザＵが全く異なると考える画像４６同士が相対的に遠い位置に配置されることとなる。したがって、ユーザＵの感覚に合った画像４６の一覧表示を行うことができる。 In this manner, in the fourth embodiment, the browser control unit 32 receives an instruction from the user U to select at least one of the similar image 46SM and the dissimilar image 46DSM. In response to a selection instruction, the weighting coefficient correction unit 135 applies weights to multiple types of feature amounts in a direction in which the feature amounts match for similar images 46SM, and in a direction in which feature amounts deviate from each other for dissimilar images 46DSM. Change the coefficient Wi. Therefore, in the display space 90, images 46 that the user U considers to be similar are placed relatively close to each other, and images 46 that the user U considers to be completely different are placed relatively far apart. It happens. Therefore, it is possible to display a list of images 46 that suits the sense of the user U.
　なお、表示空間９０上でユーザＵに類似画像４６ＳＭまたは非類似画像４６ＤＳＭを選択させてもよい。この場合、類似画像４６ＳＭを枠で囲む、類似画像４６ＳＭを線で繋ぐ、あるいは非類似画像４６ＤＳＭを線で繋ぐといった選択方法が考えられる。また、表示空間９０内でドラッグアンドドロップ操作等により画像４６を移動可能に構成し、ユーザＵが類似していると考える画像４６同士を近い位置に配置し直す操作を、類似画像４６ＳＭの選択操作と見なしてもよい。反対に、ユーザＵが全く異なると考える画像４６同士を遠い位置に配置し直す操作を、非類似画像４６ＤＳＭの選択操作と見なしてもよい。 Note that the user U may be allowed to select the similar image 46SM or the dissimilar image 46DSM on the display space 90. In this case, possible selection methods include surrounding the similar images 46SM with a frame, connecting the similar images 46SM with a line, or connecting the dissimilar images 46DSM with a line. In addition, the images 46 are configured to be movable within the display space 90 by drag-and-drop operations, etc., and the operation of re-arranging the images 46 that the user U considers to be similar to each other is performed as a selection operation of the similar images 46SM. It may be considered as On the other hand, an operation of rearranging images 46 that the user U considers to be completely different to distant positions may be regarded as an operation of selecting dissimilar images 46DSM.
　上記各実施形態では、２次元の表示空間９０に画像４６を一覧表示する例を示したが、これに限らない。一例として図３８に示すように、３次元の表示空間１５０に画像４６を一覧表示してもよい。図３８においては、Ｘ軸を日時特徴量、Ｙ軸を山の被写体特徴量および森の被写体特徴量、Ｚ軸を画質とした表示空間１５０を例示している。 In each of the above embodiments, an example was shown in which the images 46 are displayed as a list in the two-dimensional display space 90, but the present invention is not limited to this. As an example, as shown in FIG. 38, a list of images 46 may be displayed in a three-dimensional display space 150. In FIG. 38, a display space 150 is illustrated in which the X-axis is a date and time feature, the Y-axis is a mountain subject feature and a forest subject feature, and the Z-axis is image quality.
　表示空間９０の位置に応じて、画像４６の表示サイズを変更してもよい。例えば図１９～図２１で示した表示空間９０においては、右上の領域に配置された画像４６程、表示サイズを大きくする。こうすれば、アルバム１０６に採用されやすい画像４６を、より詳細にユーザＵに見せたりすることができる。 The display size of the image 46 may be changed depending on the position of the display space 90. For example, in the display space 90 shown in FIGS. 19 to 21, the display size of the image 46 placed in the upper right area is increased. In this way, images 46 that are likely to be included in the album 106 can be shown to the user U in more detail.
　画像４６は、例示のユーザ端末１０のカメラ機能で撮影した画像に限らない。ユーザＵがウェブページからダウンロードした画像、ユーザＵが家族および友達等から譲り受けた画像等でもよい。また、インスタントカメラから印刷出力されたインスタントフイルムを、ユーザ端末１０のカメラ機能で撮影した画像等でもよい。 The image 46 is not limited to an image taken using the camera function of the illustrated user terminal 10. The image may be an image that the user U has downloaded from a web page, an image that the user U has received from a family member, a friend, or the like. Alternatively, the image may be an image captured by the camera function of the user terminal 10 from an instant film printed out from an instant camera.
　ユーザ端末１０のブラウザ制御部３２の機能の全部または一部を画像管理サーバ１２に担わせてもよい。具体的には、画像管理サーバ１２において表示設定画面７５、画像一覧表示画面９５等の各種画面を生成し、例えばＸＭＬ（Ｅｘｔｅｎｓｉｂｌｅ　Ｍａｒｋｕｐ　Ｌａｎｇｕａｇｅ）等のマークアップ言語によって作成されるウェブ配信用の画面データの形式でユーザ端末１０に配信出力する。この場合、ユーザ端末１０のブラウザ制御部３２は、画面データに基づきウェブブラウザ上に表示する各種画面を再現し、これをディスプレイ２４Ａに表示する。なお、ＸＭＬに代えて、ＪＳＯＮ（Ｊａｖａｓｃｒｉｐｔ（登録商標）　Ｏｂｊｅｃｔ　Ｎｏｔａｔｉｏｎ）等の他のデータ記述言語を利用してもよい。 The image management server 12 may be responsible for all or part of the functions of the browser control unit 32 of the user terminal 10. Specifically, various screens such as a display setting screen 75 and an image list display screen 95 are generated in the image management server 12, and screen data for web distribution created in a markup language such as XML (Extensible Markup Language) is generated. It is distributed and output to the user terminal 10 in this format. In this case, the browser control unit 32 of the user terminal 10 reproduces various screens to be displayed on the web browser based on the screen data and displays them on the display 24A. Note that other data description languages such as JSON (JavaScript (registered trademark) Object Notation) may be used instead of XML.
　画像管理サーバ１２を構成するコンピュータのハードウェア構成は種々の変形が可能である。例えば、画像管理サーバ１２を、処理能力および信頼性の向上を目的として、ハードウェアとして分離された複数台のコンピュータで構成することも可能である。例えば、受付部４０および特徴量導出部４１の機能と、ＲＷ制御部４２および配信制御部４３の機能とを、２台のコンピュータに分散して担わせる。この場合は２台のコンピュータで画像管理サーバ１２を構成する。また、画像管理サーバ１２の全部または一部の機能を、ユーザ端末１０が担ってもよい。 The hardware configuration of the computer that constitutes the image management server 12 can be modified in various ways. For example, the image management server 12 can be configured with multiple computers separated as hardware in order to improve processing power and reliability. For example, the functions of the reception unit 40 and feature derivation unit 41 and the functions of the RW control unit 42 and distribution control unit 43 can be distributed and assigned to two computers. In this case, the image management server 12 is configured with two computers. In addition, all or part of the functions of the image management server 12 may be assigned to the user terminal 10.
　このように、ユーザ端末１０および画像管理サーバ１２のコンピュータのハードウェア構成は、処理能力、安全性、信頼性等の要求される性能に応じて適宜変更することができる。さらに、ハードウェアに限らず、画像ＡＰ３０および作動プログラム３５といったＡＰについても、安全性および信頼性の確保を目的として、二重化したり、あるいは、複数のストレージに分散して格納することももちろん可能である。 In this way, the hardware configurations of the computers of the user terminal 10 and the image management server 12 can be changed as appropriate depending on required performance such as processing capacity, safety, and reliability. Furthermore, not only the hardware but also APs such as the image AP 30 and the operating program 35 can of course be duplicated or distributed and stored in multiple storages for the purpose of ensuring safety and reliability. be.
　上記各実施形態において、例えば、ブラウザ制御部３２、受付部４０、特徴量導出部４１、ＲＷ制御部４２、配信制御部４３、および重み係数補正部１３５といった各種の処理を実行する処理部（Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）のハードウェア的な構造としては、次に示す各種のプロセッサ（Ｐｒｏｃｅｓｓｏｒ）を用いることができる。各種のプロセッサには、ソフトウェア（画像ＡＰ３０および作動プログラム３５）を実行して各種の処理部として機能する汎用的なプロセッサであるＣＰＵ２２Ａおよび２２Ｂに加えて、ＦＰＧＡ（Ｆｉｅｌｄ　Ｐｒｏｇｒａｍｍａｂｌｅ　Ｇａｔｅ　Ａｒｒａｙ）等の製造後に回路構成を変更可能なプロセッサであるプログラマブルロジックデバイス（Ｐｒｏｇｒａｍｍａｂｌｅ　Ｌｏｇｉｃ　Ｄｅｖｉｃｅ:ＰＬＤ）、および／またはＡＳＩＣ（Ａｐｐｌｉｃａｔｉｏｎ　Ｓｐｅｃｉｆｉｃ　Ｉｎｔｅｇｒａｔｅｄ　Ｃｉｒｃｕｉｔ）等の特定の処理を実行させるために専用に設計された回路構成を有するプロセッサである専用電気回路等が含まれる。 In each of the above embodiments, processing units that execute various processes such as the browser control unit 32, the reception unit 40, the feature derivation unit 41, the RW control unit 42, the distribution control unit 43, and the weighting coefficient correction unit 135 are used. As the hardware structure of the unit, the following various processors can be used. Various processors include CPUs 22A and 22B, which are general-purpose processors that execute software (image AP 30 and operating program 35) and function as various processing units, as well as FPGAs (Field Programmable Gate Array), etc. A processor that has a circuit configuration specifically designed to execute a specific process, such as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed, and/or an ASIC (Application Specific Integrated Circuit). This includes dedicated electrical circuits, etc.
　１つの処理部は、これらの各種のプロセッサのうちの１つで構成されてもよいし、同種または異種の２つ以上のプロセッサの組み合わせ（例えば、複数のＦＰＧＡの組み合わせ、および／または、ＣＰＵとＦＰＧＡとの組み合わせ）で構成されてもよい。また、複数の処理部を１つのプロセッサで構成してもよい。 One processing unit may be composed of one of these various processors, or a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs and/or a CPU and (in combination with FPGA). Further, the plurality of processing units may be configured with one processor.
　複数の処理部を１つのプロセッサで構成する例としては、第１に、クライアントおよびサーバ等のコンピュータに代表されるように、１つ以上のＣＰＵとソフトウェアの組み合わせで１つのプロセッサを構成し、このプロセッサが複数の処理部として機能する形態がある。第２に、システムオンチップ（Ｓｙｓｔｅｍ　Ｏｎ　Ｃｈｉｐ:ＳｏＣ）等に代表されるように、複数の処理部を含むシステム全体の機能を１つのＩＣ（Ｉｎｔｅｇｒａｔｅｄ　Ｃｉｒｃｕｉｔ）チップで実現するプロセッサを使用する形態がある。このように、各種の処理部は、ハードウェア的な構造として、上記各種のプロセッサの１つ以上を用いて構成される。 As an example of configuring multiple processing units with one processor, first, one processor is configured with a combination of one or more CPUs and software, as typified by computers such as clients and servers. There is a form in which a processor functions as multiple processing units. Second, as typified by system-on-chip (SoC), there are forms that use processors that realize the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip. be. In this way, various processing units are configured using one or more of the various processors described above as a hardware structure.
　さらに、これらの各種のプロセッサのハードウェア的な構造としては、より具体的には、半導体素子等の回路素子を組み合わせた電気回路（Ｃｉｒｃｕｉｔｒｙ）を用いることができる。 Furthermore, as the hardware structure of these various processors, more specifically, an electric circuit (circuitry) that is a combination of circuit elements such as semiconductor elements can be used.
　以上の記載から、下記の付記項に記載の技術を把握することができる。 From the above description, the technology described in the appendix below can be understood.
　［付記項１］
　プロセッサを備え、
　前記プロセッサは、
　複数の画像を取得し、
　複数の前記画像について複数種類の特徴量を取得し、
　前記特徴量に係る２次元または３次元の表示空間であって、２種類以上の前記特徴量を統合した数値の大きさを示す軸を少なくとも１軸以上有する表示空間において、前記特徴量に応じた位置に前記画像を配置することで、複数の前記画像を一覧表示する、
表示制御装置。
　［付記項２］
　前記プロセッサは、
　前記表示空間の仕様のユーザによる変更指示を受け付け、
　前記変更指示にしたがって前記表示空間の仕様を変更する付記項１に記載の表示制御装置。
　［付記項３］
　前記変更指示は、２種類以上の前記特徴量を統合する割合を決める重み係数の変更指示である付記項２に記載の表示制御装置。
　［付記項４］
　前記変更指示は、前記軸を構成する前記特徴量の変更指示である付記項２または付記項３に記載の表示制御装置。
　［付記項５］
　前記プロセッサは、
　前記特徴量が予め設定された範囲内にある２つ以上の画像がある場合、２つ以上の画像のうちの代表画像を最上位に表示し、前記代表画像の下にそれ以外の画像を積層した態様で表示する付記項１から付記項４のいずれか１項に記載の表示制御装置。
　［付記項６］
　前記プロセッサは、
　前記表示空間上でユーザによる前記画像の選択指示を受け付け、
　前記選択指示によって選択された前記画像に対してのみ、指定された処理を行う付記項１から付記項５のいずれか１項に記載の表示制御装置。
　［付記項７］
　前記プロセッサは、
　類似画像、および非類似画像のうちの少なくともいずれかのユーザによる選択指示を受け付け、
　前記選択指示に応じて、前記類似画像同士については前記特徴量が一致する方向に、前記非類似画像同士については前記特徴量が乖離する方向に、複数種類の前記特徴量に掛ける重み係数を変更する付記項１から付記項６のいずれか１項に記載の表示制御装置。 [Additional note 1]
Equipped with a processor,
The processor includes:
Get multiple images,
Obtaining multiple types of feature amounts for the multiple images,
In a two-dimensional or three-dimensional display space related to the feature amount, which has at least one axis indicating the magnitude of a numerical value that integrates two or more types of the feature amount, displaying a list of a plurality of images by arranging the images at positions;
Display control device.
[Additional note 2]
The processor includes:
accept a user's instruction to change the specifications of the display space;
The display control device according to supplementary note 1, wherein the specification of the display space is changed in accordance with the change instruction.
[Additional note 3]
The display control device according to appendix 2, wherein the change instruction is an instruction to change a weighting coefficient that determines a ratio of integrating two or more types of the feature amounts.
[Additional note 4]
The display control device according to appendix 2 or 3, wherein the change instruction is an instruction to change the feature amount constituting the axis.
[Additional note 5]
The processor includes:
If there are two or more images whose feature amounts are within a preset range, a representative image of the two or more images is displayed at the top, and the other images are stacked below the representative image. The display control device according to any one of Supplementary Notes 1 to 4, wherein the display control device displays a display in a mode in which:
[Additional note 6]
The processor includes:
accepting a user's instruction to select the image on the display space;
The display control device according to any one of Supplementary Notes 1 to 5, which performs specified processing only on the image selected by the selection instruction.
[Additional note 7]
The processor includes:
Accepting a selection instruction from a user of at least one of a similar image and a dissimilar image,
In response to the selection instruction, weighting coefficients applied to the plurality of types of feature quantities are changed in a direction in which the feature quantities match for the similar images, and in a direction in which the feature quantities diverge for the dissimilar images. The display control device according to any one of Supplementary Notes 1 to 6.
　本開示の技術は、上述の種々の実施形態および／または種々の変形例を適宜組み合わせることも可能である。また、上記各実施形態に限らず、要旨を逸脱しない限り種々の構成を採用し得ることはもちろんである。さらに、本開示の技術は、プログラムに加えて、プログラムを非一時的に記憶する記憶媒体にもおよぶ。 The technology of the present disclosure can also be combined as appropriate with the various embodiments and/or various modifications described above. Furthermore, it goes without saying that the present invention is not limited to the above embodiments, and that various configurations can be adopted as long as they do not depart from the gist of the invention. Furthermore, the technology of the present disclosure extends not only to programs but also to storage media that non-temporarily store programs.
　以上に示した記載内容および図示内容は、本開示の技術に係る部分についての詳細な説明であり、本開示の技術の一例に過ぎない。例えば、上記の構成、機能、作用、および効果に関する説明は、本開示の技術に係る部分の構成、機能、作用、および効果の一例に関する説明である。よって、本開示の技術の主旨を逸脱しない範囲内において、以上に示した記載内容および図示内容に対して、不要な部分を削除したり、新たな要素を追加したり、置き換えたりしてもよいことはいうまでもない。また、錯綜を回避し、本開示の技術に係る部分の理解を容易にするために、以上に示した記載内容および図示内容では、本開示の技術の実施を可能にする上で特に説明を要しない技術常識等に関する説明は省略されている。 The described content and illustrated content shown above are detailed explanations of the portions related to the technology of the present disclosure, and are merely examples of the technology of the present disclosure. For example, the above description regarding the configuration, function, operation, and effect is an example of the configuration, function, operation, and effect of the part related to the technology of the present disclosure. Therefore, unnecessary parts may be deleted, new elements may be added, or replacements may be made to the written and illustrated contents shown above without departing from the gist of the technology of the present disclosure. Needless to say. In addition, in order to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the descriptions and illustrations shown above do not include parts that require particular explanation in order to enable implementation of the technology of the present disclosure. Explanations regarding common technical knowledge, etc. that do not apply are omitted.
　本明細書において、「Ａおよび／またはＢ」は、「ＡおよびＢのうちの少なくとも１つ」と同義である。つまり、「Ａおよび／またはＢ」は、Ａだけであってもよいし、Ｂだけであってもよいし、ＡおよびＢの組み合わせであってもよい、という意味である。また、本明細書において、３つ以上の事柄を「および／または」で結び付けて表現する場合も、「Ａおよび／またはＢ」と同様の考え方が適用される。 In this specification, "A and/or B" has the same meaning as "at least one of A and B." That is, "A and/or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, even when three or more items are expressed in conjunction with "and/or", the same concept as "A and/or B" is applied.
　本明細書に記載された全ての文献、特許出願および技術規格は、個々の文献、特許出願および技術規格が参照により取り込まれることが具体的かつ個々に記された場合と同程度に、本明細書中に参照により取り込まれる。 All documents, patent applications, and technical standards mentioned herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference. Incorporated by reference into this book.

Claims (9)

  1. 　プロセッサを備え、
    　前記プロセッサは、
    　複数の画像を取得し、
    　複数の前記画像について複数種類の特徴量を取得し、
    　前記特徴量に係る２次元または３次元の表示空間であって、２種類以上の前記特徴量を統合した数値の大きさを示す軸を少なくとも１軸以上有する表示空間において、前記特徴量に応じた位置に前記画像を配置することで、複数の前記画像を一覧表示する、
    表示制御装置。     Equipped with a processor,
    The processor includes:
    Get multiple images,
    Obtaining multiple types of feature amounts for the multiple images,
    In a two-dimensional or three-dimensional display space related to the feature amount, which has at least one axis indicating the magnitude of a numerical value that integrates two or more types of the feature amount, displaying a list of a plurality of images by arranging the images at positions;
    Display control device.
  2. 　前記プロセッサは、
    　前記表示空間の仕様のユーザによる変更指示を受け付け、
    　前記変更指示にしたがって前記表示空間の仕様を変更する請求項１に記載の表示制御装置。     The processor includes:
    accept a user's instruction to change the specifications of the display space;
    The display control device according to claim 1, wherein specifications of the display space are changed in accordance with the change instruction.
  3. 　前記変更指示は、２種類以上の前記特徴量を統合する割合を決める重み係数の変更指示である請求項２に記載の表示制御装置。 The display control device according to claim 2, wherein the change instruction is an instruction to change a weighting coefficient that determines a ratio of integrating two or more types of the feature amounts.
  4. 　前記変更指示は、前記軸を構成する前記特徴量の変更指示である請求項２に記載の表示制御装置。 The display control device according to claim 2, wherein the change instruction is an instruction to change the feature amount forming the axis.
  5. 　前記プロセッサは、
    　前記特徴量が予め設定された範囲内にある２つ以上の画像がある場合、２つ以上の画像のうちの代表画像を最上位に表示し、前記代表画像の下にそれ以外の画像を積層した態様で表示する請求項１に記載の表示制御装置。     The processor includes:
    If there are two or more images whose feature amounts are within a preset range, a representative image of the two or more images is displayed at the top, and the other images are stacked below the representative image. 2. The display control device according to claim 1, wherein the display control device displays the image in a manner as described above.
  6. 　前記プロセッサは、
    　前記表示空間上でユーザによる前記画像の選択指示を受け付け、
    　前記選択指示によって選択された前記画像に対してのみ、指定された処理を行う請求項１に記載の表示制御装置。     The processor includes:
    accepting a user's instruction to select the image on the display space;
    The display control device according to claim 1, wherein specified processing is performed only on the image selected by the selection instruction.
  7. 　前記プロセッサは、
    　類似画像、および非類似画像のうちの少なくともいずれかのユーザによる選択指示を受け付け、
    　前記選択指示に応じて、前記類似画像同士については前記特徴量が一致する方向に、前記非類似画像同士については前記特徴量が乖離する方向に、複数種類の前記特徴量に掛ける重み係数を変更する請求項１に記載の表示制御装置。     The processor includes:
    Accepting an instruction from a user to select at least one of a similar image and a dissimilar image,
    In response to the selection instruction, weighting coefficients applied to the plurality of types of feature quantities are changed in a direction in which the feature quantities match for the similar images, and in a direction in which the feature quantities diverge for the dissimilar images. The display control device according to claim 1.
  8. 　複数の画像を取得すること、
    　複数の前記画像について複数種類の特徴量を取得すること、並びに、
    　前記特徴量に係る２次元または３次元の表示空間であって、２種類以上の前記特徴量を統合した数値の大きさを示す軸を少なくとも１軸以上有する表示空間において、前記特徴量に応じた位置に前記画像を配置することで、複数の前記画像を一覧表示すること、
    を含む表示制御装置の作動方法。     acquiring multiple images,
    obtaining a plurality of types of feature amounts for the plurality of images, and
    In a two-dimensional or three-dimensional display space related to the feature amount, which has at least one axis indicating the magnitude of a numerical value that integrates two or more types of the feature amount, displaying a list of a plurality of images by arranging the images at positions;
    A method of operating a display control device including:
  9. 　複数の画像を取得すること、
    　複数の前記画像について複数種類の特徴量を取得すること、並びに、
    　前記特徴量に係る２次元または３次元の表示空間であって、２種類以上の前記特徴量を統合した数値の大きさを示す軸を少なくとも１軸以上有する表示空間において、前記特徴量に応じた位置に前記画像を配置することで、複数の前記画像を一覧表示すること、
    を含む処理をコンピュータに実行させる表示制御装置の作動プログラム。     acquiring multiple images,
    acquiring a plurality of types of feature amounts for the plurality of images, and
    In a two-dimensional or three-dimensional display space related to the feature amount, which has at least one axis indicating the magnitude of a numerical value that integrates two or more types of the feature amount, displaying a list of a plurality of images by arranging the images at positions;
    An operating program for a display control device that causes a computer to execute processing including.
PCT/JP2023/032375 2022-09-20 2023-09-05 Display control device, method for operating display control device, and program for operating display control device WO2024062912A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-149485 2022-09-20
JP2022149485 2022-09-20

Publications (1)

Publication Number Publication Date
WO2024062912A1 true WO2024062912A1 (en) 2024-03-28

Family

ID=90454219

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/032375 WO2024062912A1 (en) 2022-09-20 2023-09-05 Display control device, method for operating display control device, and program for operating display control device

Country Status (1)

Country Link
WO (1) WO2024062912A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001325294A (en) * 2000-05-17 2001-11-22 Olympus Optical Co Ltd Method and device for retrieving similar image
JP2005196298A (en) * 2003-12-26 2005-07-21 Canon Software Inc Information processor, image data display control method, program and recording medium
JP2006004062A (en) * 2004-06-16 2006-01-05 Canon Inc Image database creation device and image search method
JP2013114597A (en) * 2011-11-30 2013-06-10 Canon Marketing Japan Inc Information processing device, control method thereof, and program
JP2013200591A (en) * 2012-03-23 2013-10-03 Fujifilm Corp Database search device, method, and program

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001325294A (en) * 2000-05-17 2001-11-22 Olympus Optical Co Ltd Method and device for retrieving similar image
JP2005196298A (en) * 2003-12-26 2005-07-21 Canon Software Inc Information processor, image data display control method, program and recording medium
JP2006004062A (en) * 2004-06-16 2006-01-05 Canon Inc Image database creation device and image search method
JP2013114597A (en) * 2011-11-30 2013-06-10 Canon Marketing Japan Inc Information processing device, control method thereof, and program
JP2013200591A (en) * 2012-03-23 2013-10-03 Fujifilm Corp Database search device, method, and program

Similar Documents

Publication Publication Date Title
EP3823261B1 (en) Method and system for providing recommendation information related to photography
CN108369633B (en) Visual representation of photo album
US9491366B2 (en) Electronic device and image composition method thereof
US11256904B2 (en) Image candidate determination apparatus, image candidate determination method, program for controlling image candidate determination apparatus, and recording medium storing program
US20110219329A1 (en) Parameter setting superimposed upon an image
US11281940B2 (en) Image file generating device and image file generating method
US9690980B2 (en) Automatic curation of digital images
US10755487B1 (en) Techniques for using perception profiles with augmented reality systems
US20150169944A1 (en) Image evaluation apparatus, image evaluation method, and non-transitory computer readable medium
JP2022526053A (en) Techniques for capturing and editing dynamic depth images
KR20230021144A (en) Machine learning-based image compression settings reflecting user preferences
US11854178B2 (en) Photography session assistant
CN114926351A (en) Image processing method, electronic device, and computer storage medium
WO2024062912A1 (en) Display control device, method for operating display control device, and program for operating display control device
WO2023149135A1 (en) Image processing device, image processing method, and program
JP2014085814A (en) Information processing device, control method therefor, and program
WO2019065582A1 (en) Image data discrimination system, image data discrimination program, image data discrimination method and imaging system
US9736380B2 (en) Display control apparatus, control method, and storage medium
WO2022104181A1 (en) Systems, apparatus, and methods for improving composition of images
US10552888B1 (en) System for determining resources from image data
CN110235174B (en) Image evaluation device, image evaluation method, and recording medium
JP2006172090A (en) Representative image selecting device and method and program
JP2006171942A (en) Significance setting device and method and program
JP2020194472A (en) Server, display method, creation method, and program
KR20150096552A (en) System and method for providing online photo gallery service by using photo album or photo frame

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23868036

Country of ref document: EP

Kind code of ref document: A1