Search

EP-4742620-A2 - MODIFYING IMAGES WITH SUPPLEMENTAL CONTENT FOR MESSAGING

EP4742620A2EP 4742620 A2EP4742620 A2EP 4742620A2EP-4742620-A2

Abstract

The subject technology provides a messaging application in which an image can be captured and supplemented with supplemental content such as stickers, animations, etc., from within an active messaging thread. In this manner, a user participating in a messaging conversation with a remote user can add stickers, animations, and/or adaptive content to an image captured by the user, without having to locally cache/store the captured image before editing and without having to leave the messaging conversion (or the messaging application) to access an image editing application.

Inventors

  • GARRIDO, CHRISTOPHER M.
  • CHIEN, ERIC L.
  • SHYU, AUSTIN W.
  • JIN, MING
  • YANG, YAN
  • BAIRD, Ian J.
  • ABUAN, JOE S.

Assignees

  • Apple Inc.

Dates

Publication Date
20260513
Application Date
20190313

Claims (15)

  1. A method, comprising: receiving, by an application running on an electronic device, an image from a camera of the electronic device; and receiving, by the application, a request to send the image combined with supplemental content; and providing, by the application responsive to the request, a package that includes: the image, first metadata that includes instructions for obtaining the supplemental content, and second metadata that includes a location of at least one feature of a user in the image, for transmission to a remote device for rendering of the image combined with the supplemental content at the remote device using the instructions, the first metadata, and the second metadata.
  2. The method of claim 1, further comprising, displaying the image combined with the supplemental content in a message stream view of the application.
  3. The method of claim 1, wherein receiving the image comprises receiving the image in a continuous stream of images from the camera, and wherein the method further comprises displaying, by the application, the supplemental content combined with the continuous stream of images.
  4. The method of claim 3, wherein the supplemental content includes static supplemental content, animated supplemental content, or adaptive supplemental content.
  5. The method of claim 4, wherein the adaptive supplemental content comprises facial modification content, and wherein the method further comprises: detecting, by the application, a face in the image; applying, by the application, the facial modification content to the detected face; and actively modifying, by the application, the facial modification content based on changes to the detected face.
  6. The method of claim 3, further comprising: receiving, at a time prior to receiving the request and by the application, a selection of an image capture option provided by the application while displaying the supplemental content combined with the continuous stream of images from the camera; capturing, by the application, the image as a still image responsive to the selection of the image capture option; and displaying, by the application prior to receiving the request, the still image combined the supplemental content as displayed at the time of the selection of the image capture option.
  7. The method of claim 1, further comprising providing, by the application within an image capture display view of the application, a send option selectable for requesting to send the image.
  8. The method of claim 7, further comprising providing, by the application for transmission to the remote device, instructions for rendering the image and the supplemental content to the remote device.
  9. The method of claim 1, further comprising: reducing, by the application, a resolution of a portion of the image to be covered with the supplemental content without reducing a resolution of other portions of the image; and providing, by the application, the image with the reduced resolution portion in the package for transmission to the remote device.
  10. A method, comprising: receiving, by an application running at a device, a package that includes an image and metadata, the metadata including: first metadata that includes instructions for obtaining supplemental content for the image, and second metadata that includes a location of at least one feature of a face in the image; obtaining, by the application, the supplemental content using the first metadata; rendering, by the application, the image combined with the supplemental content using the second metadata; and displaying, by the application, the image combined with the supplemental content in a message stream view of the application.
  11. The method of claim 10, wherein the second metadata includes depth information associated with the image.
  12. The method of claim 11, wherein the supplemental content includes animated supplemental content that is displayed over the image.
  13. The method of claim 10, wherein the package further comprises instructions for rendering the image and the supplemental content at the device.
  14. The method of claim 10, wherein the image comprises a first portion have a first resolution and a second portion having a second resolution that is reduced relative to the first resolution, and wherein rendering the image combined with the supplemental content using the second metadata comprises rendering the supplemental content over the second portion of the image.
  15. The method of claim 10, wherein obtaining the supplemental content comprises obtaining the supplemental content from within the first metadata or from a storage location specified in the first metadata.

Description

The present application claims the benefit of U.S. Provisional Patent Application Serial No. 62/668,115, entitled "MODIFYING IMAGES WITH SUPPLEMENTAL CONTENT FOR MESSAGING," filed May 7, 2018, which is hereby incorporated herein by reference in its entirety and made part of the present U.S. Utility Patent Application for all purposes. TECHNICAL FIELD The present description relates generally to modifying electronic images, and more particularly to modifying images with supplemental content for messaging. BACKGROUND Messaging applications can provide the ability for person-to-person communication for users of electronic devices. Texts messages and images can be transmitted using messaging applications. Messaging applications for electronic devices commonly provide a keyboard interface and a message stream display in which a user of an electronic device can conduct a messaging conversation with a user of a remote device. Some messaging applications also provide an option for a user to capture an image with a device camera for sending in the messaging conversation. BRIEF DESCRIPTION OF THE DRAWINGS Certain features of the subject technology are set forth in the appended claims. However, for purpose of explanation, several embodiments of the subject technology are set forth in the following figures. FIG. 1 illustrates an example network environment in which modifying images with supplemental content for messaging may be implemented in accordance with one or more implementations.FIG. 2 illustrates a schematic diagram showing exemplary user interface views of a sending device during modifying images with supplemental content for messaging in accordance with one or more implementations.FIG. 3 illustrates a schematic diagram showing exemplary user interface views of a receiving device for receipt of images with supplemental content for messaging in accordance with one or more implementations.FIG. 4 illustrates a schematic diagram showing an exemplary data flow for integration of supplemental content with a captured image for transmission in a message in accordance with one or more implementations.FIG. 5 illustrates a schematic diagram showing an exemplary data flow for packaging of supplemental content metadata to accompany a captured image for transmission in a message in accordance with one or more implementations.FIG. 6 illustrates a schematic diagram showing an exemplary data flow for integration of supplemental content with a camera feed for transmission in accordance with one or more implementations.FIG. 7 illustrates a schematic diagram showing an exemplary data flow for packaging of supplemental content metadata to accompany a camera feed for transmission in accordance with one or more implementations.FIG. 8 illustrates a flow diagram of an example process for modifying images with supplemental content for messaging in accordance with one or more implementations.FIG. 9 illustrates a flow diagram of an example process for sending a message including a captured image with supplemental content in accordance with one or more implementations.FIG. 10 illustrates a flow diagram of another example process for sending a message including a captured image with supplemental content in accordance with one or more implementations.FIG. 11 illustrates a flow diagram of an example process for displaying images modified with supplemental content in a message stream in accordance with one or more implementations.FIG. 12 illustrates an example user interface of a selection of a camera mode option in a messaging application in accordance with one or more implementations.FIG. 13 illustrates an example user interface of a camera feed displayed with supplemental content options after selection of a camera mode option in accordance with one or more implementations.FIG. 14 illustrates an example user interface of supplemental content added to a camera feed in accordance with one or more implementations.FIG. 15 illustrates an example user interface of stickers that can be added to a camera feed in accordance with one or more implementations.FIG. 16 illustrates an electronic system with which one or more implementations of the subject technology may be implemented. DETAILED DESCRIPTION The detailed description set forth below is intended as a description of various configurations of the subject technology and is not intended to represent the only configurations in which the subject technology can be practiced. The appended drawings are incorporated herein and constitute a part of the detailed description. The detailed description includes specific details for the purpose of providing a thorough understanding of the subject technology. However, the subject technology is not limited to the specific details set forth herein and can be practiced using one or more other implementations. In one or more implementations, structures and components are shown in block diagram form in order to avoid obscuring the concepts of the subject technolo