Photos.app automatically recognizes faces in photos and groups them by person. Are photos sent to Apple for processing or is everything done locally on the machine?
Facial recognition and other image analysis is done on device. Never done on Apple’s servers. Also the results of facial recognition and other images analysis are never send to Apple.
Apple’s system is more focused on privacy than Facebook’s or Google’s,
with stricter limits on how the data is collected and stored. Onstage,
Craig Federighi insisted that Apple’s new system only uses local data, which
means the company isn’t storing faceprints on company servers — a
crucial point for anyone worried about their faceprint being used for
more than just cataloging photos. Faceprints are still being created
and used, but it’s all happening on your phone, where Apple and the
rest of the world can’t access it. That’s different from Google and
Facebook, which store data on the cloud and get significant scale
benefits as a result.
Starting with iOS 11 and macOS High Sierra 10.13 however, facial recognition data will be synced across iCloud Photo Library.
Interview John Gruber with Craig Federighi on how this works:
No, we’ve been working on it. And it’s actually something that’s
harder to get right right, because when you go through the process of
classifying your photos, we offer up and we say “Hey, is this John?”
And y’know, “Is this John?” And you say “Yes, yes.”
But when you say yes to one face, we’re actually saying “Okay, well,
there are a thousand other photos we saw that we think are the same
person as this, so we’re going to count those all as John — but all
you really told us is about that one photo.
And so what we actually sync, and support, and so that we get this
right — we sync only the specific photos you told us about, and then
each… ‘cos that’s really the truth, y’know, that’s what the user
really told us.
And then we sync that information, and then each device, because
they’re independently doing photo recognition are talking that, and
— coming to the final result on all your machines to synchronize. So
synchronize is actually very little data, and it’s all just the truth
data, and we won’t hold back the right solution.
But getting that architecture completely right was a — took a little
longer than we would have liked, and so we’re super excited to have it