Apple acquired Zurich-based Faceshift in September/October 2015, and launched Animoji and Memoji after digestion and absorption. According to a list published by the United States Trademark and Patent Office (USPTO) on January 26, 2023, Apple was awarded a patent dating back to 2013 from Faceshift.
The inventors of the patent include Apple senior software engineer Mark Pauley and former Faceshift engineer Sofien Bouaziz. According to the latter’s profile, he “designed, developed, and productized a real-time face tracking algorithm for iPhone X Animoji, and made it available to third-party developers through ARKit.” After leaving Apple in 2018, Bouaziz worked at Google and is now Meta’s director of XR Presence.
This patent relates to a real-time facial animation production method and a real-time facial animation processing device. The method includes providing a dynamic expression model, receiving tracking data corresponding to user facial expressions, estimating tracking parameters based on the dynamic expression model and the tracking data, and improving the dynamic expression model based on the tracking data and the estimated tracking parameters.
The system can be dynamically simulated on a grunt monster similar to “Lord of the Rings” based on the face, and can produce different expressions according to changes in the face.
