IROS2019國際學術(shù)會議論文集 0902

Are You With Me Determining the Association of Individuals and the Collective Social Space Alan D G SilvaandDouglas G Macharet Abstract The increasing use of autonomous mobile robots in different parts of society and not restricted only to industrial environments makes it important to propose techniques that will allow them to behave in the most socially acceptable way as possible In most real world scenarios individuals in the environment are interacting with each other and are arranged into groups Therefore it is paramount the proposition of techniques to effi ciently and correctly identify and represent such groups This information can be useful in different tasks such as approaching and initiating an interaction escorting and the navigation itself In this work we propose a novel graph based approach to evaluate the possible association of individuals in the environment based on their position and body orientation Next based on this association we propose a representation of the combined social space of individuals in the same group The methodology was evaluated using synthetic and real world datasets showing that it achieves results comparable to or better than the state of the art I INTRODUCTION The use of autonomous robots is changing from more controlled environments to unconstrained environments in which people are an integral part In this context recent advances on the research of Human Robot Interaction HRI have uncovered a broad range of new problems and prompted challenges to several classes of already known problems The need for socially acceptable behaviors crosses many domains and are especially concerned about the level of acceptance or discomfort of people in sharing the environ ment with such autonomous agents In this sense there are some implicit nonverbal social rules that must be respected especially those related to the personal space of individuals which are referred as proxemics 1 An individual personal space represents how a person perceives and relates with distances in an interaction con sidering the surrounding area Generally the personal space can be divided into four social zones considering from a more intimate space to the public one However in most everyday situations people are inter acting with each other in groups of two or more people In this case not only the individuals social spaces should be considered but a new collective social space that encloses the whole group Tasks such as navigation approaching and escorting can benefi t from when considering this combined entity instead of each person individually In this work initially we propose a methodology to evaluate the association of individuals in the environment TheauthorsarewiththeComputerVisionandRobotics Laboratory VeRLab DepartmentofComputerScience UniversidadeFederaldeMinasGerais MG Brazil E mails alandeivite doug dcc ufmg br based on their position and body orientation in order to identify and cluster them into groups Next we present a mathematical model build upon the use of a density function to effi ciently determine the social space of groups and in dividuals according to different proxemic zones Finally we validate the presented methodology with several experiments in a simulated scenario and also compare it with the state of the art considering available real world datasets II RELATED WORK A Personal Space Several studies in psychology have shown that there are some implicit nonverbal social rules that need to be respected in a human human interaction in particular interest of this paper those related to the personal space of individuals called proxemics 1 Different works suggest that the same proxemic zones observed in human human interaction can also be considered in human robot interaction scenarios This notion of personal social space is one of the most used aspects in several HRI research topics for example human aware autonomous navigation 2 approaching 3 and person following 4 An individual s personal space can be generally repre sented using different models 5 but mainly are based on the use Gaussian functions 6 B Group Identifi cation In many real world scenarios people in the environment are interacting with each other for example engaged in a conversation and consequently forming groups This percep tion of such social interactions is a very useful information for a robot The problem of identifying and correctly associate groups of people is a challenge in itself We assume at this moment that the visual detection of the position and orientation of people in the scene as not being part of our scope 7 An important feature to be observed during social interac tions is the spatial arrangement of the individuals towards the group The spatial arrangement during a social interaction aims to meet two main needs 8 the fi rst is to provide all those involved with the opportunity to participate in the interaction the second is to establish a separation between an interaction group and external individuals in case there are other people in the environment Patterns formed during social interactions which describe how people adjust their position and orientation to jointly interact and jointly manage their attention are mostly assigned to the F formation system designed by 9 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 978 1 7281 4003 2 19 31 00 2019 IEEE313 In 10 a method of estimating F formations is proposed using a graph clustering algorithm by individuating maximal cliques in weighted graphs where each node in the graph was a person and the edges between them measure the affi nity between pairs The approach presented in 11 also consid ered a similar graph based clustering algorithm inspired by the principle of dominant set of graphs to discover socially interacting groups By defi nition a cluster should have high internal homogeneity and should have high inhomogeneity between the entities in the cluster and those outside In 12 it is proposed a graph cuts based framework for F formation detection based on the position and orientation estimates of targets The o space of an F formation it s found assigning to it those individuals whose transactional segments do overlap without focusing on a particular arrangement C Collective Social Space Another issue is how to determine and represent the social boundaries taking into account this new combined entity In 13 is presented a methodology that can instantiate diverse social spacing models depending on the context and further as a function of uncertainty and robot perception capacity Their method is based on the use of non stationary skew normal probability density functions for the space of individuals and on treating multi person space interactions through social mapping applying the potential fi eld algo rithm to avoid visits into the individuals comfortable space by the robot In 14 is proposed a human safety framework that takes the humans states and social constraints into account for building up a completed estimation model using a bivariate Gaussian distribution function so called Dynamic Social Zone DSZ The DSZ is incorporated into the motion planning system in order to ensure human safety in human robot shared workspaces III METHODOLOGY A Atomic Social Space In this work we model an individual personal space as an Asymmetric Gaussian Function AGF 6 which is obtained by composing two standard 2D Gaussian functions with equal xand different yvalues The model proposed in 6 is limited to represent the changes in the format of the social space considering as single factor the speed of displacement of the individual However studies mention that an individual personal space is commonly larger in the direction of the body orientation 15 In this case we use the AGF considering a constant linear velocity of displacement of 1 0m s even for static persons to obtain a personal space larger in the front area The interpersonal distance is defi ned according to the Gaussian axis of greatest length in which case the axis is aligned towards the subject s torso orientation In this direction the standard deviation of the Gaussian function is defi ned as h max 2 1 2 The standard deviation for the sides 2 direction are given by s 2 3 h and rear direction by r 1 2 h Then we can formally represent the personal space as G AGF x y s r h 1 Furthermore a reformulation is proposed to include the interpersonal distances from proxemics 1 no longer mod ifying the format but rather the size of the social space A threshold is used for partitioning the aforementioned AGF into four distinct zones according to the following distances i Intimate 0 00 0 45 m ii Personal 0 45 1 20 m iii Social 1 20 3 60 m and iv Public 3 60 m For this specifi c case the Gaussian function will be segmented at its widest point Thus we fi nd the intersection value between a unidimensional symmetric Gaussian and the AGF in order to fi nd the desired cut point This is accomplished by replace the standard Gaussian notation x by the desired interpersonal distance represented by and the by h the standard deviation of the AGF in its axis of greater amplitude front The threshold which would segment the cost function according to one of the previously defi ned interpersonal zones will be given by e 2 2 2 h B Interacting People Determination The detection of possible interactions among individuals in the environment is modeled following a graph based approach built on proxemics rules and estimates of body position and orientation Let G hV Ei be an undirected graph with vertices vi Vthe set of people to be segmented as a interaction group and edges vi vj E corresponding to pairs of neighboring interacting people in the same predefi ned proxemic zone Figure 1 shows the edge set E initially constructed by connecting pairs of neighboring individuals in the same proxemic zone Subsequently each candidate edge vi vj E receives a corresponding weight w vi vj Equation 2 which is a non negative measure representing the possible Level of Interaction LoI between neighboring elements vi and vj Figure 1a This procedure is employed successively until each vertex in V has been analyzed generating the fi nal result shown in Figure 1b vi a b Fig 1 Group determination a The red dashed circle represents the search region of and individual vigiven a social distance The Level of Interaction for candidates partners are calculated according to Equation 2 b Complete interaction graph for all individuals in the scene 314 As mentioned we use an edge weight function to estimate the LoI between neighboring individuals viand vj Similarly to the approach performed by 16 this function is based on the focus of attention established for each pair of persons ac cording to their body orientation However in their approach several poses are sampled from Gaussian distributions to deal with the uncertainty of real scenarios As our methodology aims at applications directed to human robot interaction we opted for an analytic and less time consuming approach Given as input the position and orientation of each person we can express the relationship between all node pairs of people in the scene by a weighted function normalized between 0 and 1 Specifi cally this range represents the disparity distance in the focus of attention between a pair of candidates to establish a social interaction Figure 2 v i v j i j Fig 2 Illustration of the disparity distance in the focus of attention between a pair of individuals viand vj Their main centers of focus are iand j respectively Initially the center point between vi vj is selected as reference Next this centroid is rotated towards the axis of orientation corresponding to each of the individuals where iand j are respectively defi ned as the main center of focus next the absolute distance to the focus of attention is com puted The addition of two constraints becomes necessary to bring robustness to the model in cases where one element does not have direct access to the focus of attention of the other and to deal with converging confi gurations like in a vis a vis F formation The fi rst constraint aims to ensure that the LoI gradually decreases its relevance whenever k i jk reaches a value higher than min kvi jk kvj ik The second constraint produces the same effect of decreasing the LoI whenever the line produced by the points iand j intercept the line produced by the points viand vj The weight of an edge is then given by w vi vj 1 k i jk 2kvi vjk k 2 where k k denotes the Euclidean norm viand vjrepresent the position of a pair of individuals sharing the same social space iand j are defi ned as the respective position of the focus of attention attributed to each individual and k is a term initially assigned value 1 The regularization term k is defi ned empirically as shown on Algorithm 1 After computing all the edges weights in the graph a cut will be executed given a certain threshold w where values below the threshold e g w 0 5 are discarded and all equal to or greater are retained As a fi nal result will be obtained all estimated neighboring interacting people in the same zone of proxemics through the subgraphs of G Our method is robust in identifying the main F formation confi gurations With just one cost function and two con straints we are able to consider side to side face to face corner to corner among others e g N H V L C and I Algorithm 1 CalculateLevelofInteraction vi vj 1 k 1 0 2 if vivj i jthen 3 k 2 0 4 end if 5 if k i jk min kvi jk kvj ik then 6 k k 0 5 7 end if 8 return w vi vj Equation 2 C Collective Social Space Considering groups of people and individual persons in the environment we must then determine the social spaces for each one of these entities In this paper since the personal space is represented by an AGF we will use a density function to obtain its boundary and a Gaussian mixture strategy to determine the combined social space of a group This is accomplished by using a method inspired in 17 Initially we compute the AGF for each vertex in the subgraphs G0belonging to G In a singleton vertex the AGF is generated only in the surroundings of each vertex within the limits corresponding to the selected personal distance e g 1 2m For cases where the subgraph is formed by more than a singleton vertex the AGF is generated in the area corresponding to max d 3 with d max vi vj E G0 kvi vjk 4 where E G0 denotes all pairs of edges belonging to the subgraph G0 i e a group of interacting individuals Afterwards we must defi ne the boundary of the collective social space This is accomplished by identifying the weights that correspond to the Gaussian mixture expressed as V G0 X i iGi 5 where V G0 denotes all vertex belonging to the subgraph G0 that is the number of Gaussian functions being mixed and irepresents the mixture weight of the ithAGF Thus we compute the average weight to be used in the determination of threshold h Equation 6 Since the Gaussian function is defi ned only in the space corresponding to the social zone of all the individuals be longing to the group the local average point belonging to the function will correspond to the limit which includes the ma jority of individuals belonging to the group In cases where there is only one individual their social space boundary is defi ned as previously explained in Section III A 315 Finally we can formalize the choice of the threshold responsible for segmenting people according to their social space as follows h singleton subgraph otherwise 6 With the restrictions imposed on h we guarantee that the atomic social space aligned to the subject s torso will never be within a smaller distance than the personal distance For the collective social space due to the mixture of Gaussian the selected boundary will be as close as possible to the individual but never less than the personal distance IV EXPERIMENTS A Datasets We have used fi ve publicly available datasets which main characteristics are summarized in Table I 12 TABLE I Main characteristics of the datasets DatasetData typeDetectionDetection quality Syntheticsynthetic perfect SALSArealmanual semi automatichigh Cocktail Partyrealautomatichigh Coffee Breakrealautomaticlow GDetrealautomaticvery low B Group Determination The proposed methodology was compared to fi ve state of the art approaches i Hough voting 16 HVFF lin ii its non linear variant 18 HVFF ent and iii multi scale extensions 19 HVFF ms iv the game theoretic approach CTCG 20 and v the graph cut approach GCFF 12 We adopt the accuracy metrics proposed in 16 and ex tended in 18 A group is considered as correctly estimated if at least d T G e of their members are correctly detected by the algorithm and if no more than d 1 T G e false subjects are identifi ed where G is the cardinality of the labeled group G and T 0 1 is an arbitrary tolerance threshold In particular we focus on T 1 Table II and T 2 3 Table III The number of false positives FP and false negatives FN rates are derived by subtracting the true positives TP from the cardinality of the detected groups and of the ground truth groups respectively Based on this metrics we compute precision recall and F1 score per frame averaging these values over the frames gives the fi nal scores It is not in the scope of this article the visual detection problem in this sense we used the ground truth position and orientation annotations and considered all the annotated frames We empirically adopt as parameter for our approach a threshold w 0 5 to cut the graph The methodology lacks performance for T 1 on real datasets mainly due to temporal inconsistencies in the automatic detection however achieved the best results in the Synthetic dataset For the T 2 3 threshold fi nd at least 2 3 of the members of a group we also obtained better results the higher is the detection quality of the dataset Figure 3 presents a temporal distribution analysis of the F1 score values in order to identify specifi cally t