how to group photos with similar faces together

Question

In most face recognition SDK, it only provides two major functions

1. detecting faces and extracting templates from photos, this is called detection.
2. comparing two templates and returning the similar score, this is called recognition.

However, beyond those two functions, what I am looking for is an algorithm or SDK for grouping photos with similar faces together, e.g. based on similar scores.

Thanks

Answer 1

First, perform step 1 to extract the templates, then compare each template with all the others by applying step two on all the possible pairs, obtaining their similarity scores.

Sort the matches based on this similarity score, decide on a threshold and group together those templates that exceed it.

Take, for instance, the following case:

Ten templates: A, B, C, D, E, F, G, H, I, J.
Scores between: 0 and 100.
Similarity threshold: 80.

Similarity table:

   A   B   C   D   E   F   G   H   I   J
A  100 85  8   0   1   50  55  88  90  10

B  85  100 5   30  99  60  15  23  8   2 

C  8   5   100 60  16  80  29  33  5   8

D  0   30  60  100 50  50  34  18  2   66

E  1   99  16  50  100 8   3   2   19  6

F  50  60  80  50  8   100 20  55  13  90

G  55  15  29  34  3   20  100 51  57  16

H  88  23  33  18  2   55  51  100 8   0

I  90  8   5   2   19  13  57  8   100 3

J  10  2   8   66  6   90  16  0   3   100

Sorted matches list:

AI 90
FJ 90
BE 99
AH 88
AB 85
CF 80
------- <-- Threshold cutoff line
DJ 66
.......

Iterate through the list until the threshold cutoff point, where the values no longer exceed it, maintain a full templates set and association sets for each template, obtaining the final groups:

// Empty initial full templates set
fullSet = {};

// Iterate through the pairs list
foreach (templatePair : pairList)
{
  // If the full set contains the first template from the pair
  if (fullSet.contains(templatePair.first))
  {
     // Add the second template to its group
     templatePair.first.addTemplateToGroup(templatePair.second);

     // If the full set also contains the second template
     if (fullSet.contains(templatePair.second))
     {
        // The second template is removed from the full set
        fullSet.remove(templatePair.second);

        // The second template's group is added to the first template's group
        templatePair.first.addGroupToGroup(templatePair.second.group);
     }
  }
  else
  {
    // If the full set contains only the second template from the pair
    if (fullSet.contains(templatePair.second))
    {
      // Add the first template to its group
      templatePair.second.addTemplateToGroup(templatePair.first);
    }
  }
  else
  {
     // If none of the templates are present in the full set, add the first one
     // to the full set and the second one to the first one's group
     fullSet.add(templatePair.first);
     templatePair.first.addTemplateToGroup(templatePair.second);
  } 
}

Execution details on the list:

AI: fullSet.add(A); A.addTemplateToGroup(I);
FJ: fullSet.add(F); F.addTemplateToGroup(J);
BE: fullSet.add(B); B.addTemplateToGroup(E);
AH: A.addTemplateToGroup(H);
AB: A.addTemplateToGroup(B); fullSet.remove(B); A.addGroupToGroup(B.group);
CF: C.addTemplateToGroup(F);

In the end, you end up with the following similarity groups:

A - I, H, B, E
C - F, J