Samsung recently filed a patent for an app that will detect a user's skin tone using a smartphone's camera. The app shown in the patent application seems to be focused on giving users tips to avoid sunburns and other environmental hazards. It works by sorting users into one of six predetermined skin types based on how fair or tanned their skin is, along with considerations for pigmentation issues, wounds and other abnormalities. The description from Samsung says that the app will provide information on how to manage a subject's specific skin type, along with specific info on what kind of sun protection to use, how long the subject can be out in the sun and other such information.
The way it works is by taking a photo of some portion of the user's body that shows both skin and non-skin, such as a fingernail or the user's eyes. The app figures out what it's looking at, then compares the coloration in the two parts of the photo it chooses to examine in order to determine how fair a subject's skin is. It also uses the front-facing camera during normal use to see abnormalities in the skin, as well as any prominent marks or issues with the subject's face. This information, in turn, is used to fetch tips for the subject as detailed above. Tips for managing sun exposure are shown in the patent drawings as seen below, but the scope of the app may extend beyond that in the future, juding by the language used in the patent description.
While this is certainly an unconventional app, it's not that big of a surprise to see something like this showing up on a Samsung device. The company has always prided itself on innovation and offering features that nobody else has, and the onboard machine learning technology packed into modern mobile processors allows Samsung to create something like this, capable of examining a subject in detail and comparing those details to both known and new values in order to reach a classification. Given the timing of the patent, if this app is going to be real, it may show up in the Galaxy Note 9 or Galaxy S10.