Search

EP-4459951-B1 - MODIFYING IMAGES WITH SUPPLEMENTAL CONTENT FOR MESSAGING

EP4459951B1EP 4459951 B1EP4459951 B1EP 4459951B1EP-4459951-B1

Inventors

  • GARRIDO, CHRISTOPHER M.
  • CHIEN, ERIC L.
  • SHYU, AUSTIN W.
  • JIN, MING
  • YANG, YAN
  • BAIRD, Ian J.
  • ABUAN, JOE S.

Dates

Publication Date
20260513
Application Date
20190313

Claims (15)

  1. A device, comprising: a memory; and at least one processor communicatively coupled to the memory and configured to: receive, from a sender device, a package that includes an image and metadata, the metadata including: supplemental content metadata that includes instructions for obtaining supplemental content for the image, and augmented reality metadata that includes a location of at least one feature of a user of the sender device in the image; provide the image and the metadata to a messaging application at the device; obtain the supplemental content using the supplemental content metadata; render the image combined with the supplemental content using the augmented reality metadata; and display the image combined with the supplemental content in a message stream view of the messaging application.
  2. The device of claim 1, wherein the augmented reality metadata includes depth information associated with the image.
  3. The device of claim 1 or 2, wherein the supplemental content includes animated supplemental content that is displayed over the image.
  4. The device of any preceding claim, wherein the package further comprises instructions for rendering the image and the supplemental content at the device.
  5. The device of any preceding claim, wherein the image comprises a first portion have a first resolution and a second portion having a resolution that is reduced relative to the first resolution, and wherein the at least one processor is configured to render the image combined with the supplemental content using the augmented reality metadata by rendering the supplemental content over the second portion of the image.
  6. The device of any preceding claim, wherein the at least one processor is configured to provide the image to the messaging application via a decoder.
  7. The device of any preceding claim, wherein the at least one processor is configured to obtain the supplemental content from the supplemental content metadata.
  8. The device of any of claims 1 to 6, wherein the at least one processor is configured to obtain the supplemental content from a storage location specified in the supplemental content metadata.
  9. The device of any preceding claim, wherein the at least one processor is configured to receive the package with a transport protocol engine that unpacks the image and the metadata from the package.
  10. A method, comprising: receiving, at a device from a sender device, a package that includes an image and metadata, the metadata including: supplemental content metadata that includes instructions for obtaining supplemental content for the image, and augmented reality metadata that includes a location of at least one feature of a user of the sender device in the image; providing the image and the metadata to a messaging application at the device; obtaining the supplemental content using the supplemental content metadata; rendering the image combined with the supplemental content using the augmented reality metadata; and displaying the image combined with the supplemental content in a message stream view of the messaging application.
  11. The method of claim 10, wherein the augmented reality metadata includes depth information associated with the image.
  12. The method of claim 10 or 11, wherein the supplemental content includes animated supplemental content that is displayed over the image.
  13. The method of any of claims 10 to 12, wherein the package further comprises instructions for rendering the image and the supplemental content at the device.
  14. The method of any of claims 10 to 13, wherein the image comprises a first portion have a first resolution and a second portion having a resolution that is reduced relative to the first resolution, and wherein rendering the image combined with the supplemental content using the augmented reality metadata comprises rendering the supplemental content over the second portion of the image.
  15. A non-transitory computer-readable medium comprising instructions, which when executed by at least one computing device, cause the at least one computing device to perform the method of any of claims 10-14.

Description

The present application claims the benefit of U.S. Provisional Patent Application Serial No. 62/668,115, entitled "MODIFYING IMAGES WITH SUPPLEMENTAL CONTENT FOR MESSAGING," filed May 7, 2018. TECHNICAL FIELD The present description relates generally to modifying electronic images, and more particularly to modifying images with supplemental content for messaging. BACKGROUND Messaging applications can provide the ability for person-to-person communication for users of electronic devices. Texts messages and images can be transmitted using messaging applications. Messaging applications for electronic devices commonly provide a keyboard interface and a message stream display in which a user of an electronic device can conduct a messaging conversation with a user of a remote device. Some messaging applications also provide an option for a user to capture an image with a device camera for sending in the messaging conversation. Chaim Gartenberg "How to use Snapchat in 2017 - The Verge" 27 May 2017(2017-05-27). discusses various features of the application Snapchat. EP 2887686 discloses a system for sharing images, games, and/or other types of content. The system includes activating a camera directly from a messaging session on a computing device, and sending an image to one or more recipients on at least one other computer device, directly from the messaging session. BRIEF DESCRIPTION OF THE DRAWINGS Certain features of the subject technology are set forth in the appended claims. However, for purpose of explanation, several embodiments of the subject technology are set forth in the following figures. FIG. 1 illustrates an example network environment in which modifying images with supplemental content for messaging may be implemented in accordance with one or more implementations.FIG. 2 illustrates a schematic diagram showing exemplary user interface views of a sending device during modifying images with supplemental content for messaging in accordance with one or more implementations.FIG. 3 illustrates a schematic diagram showing exemplary user interface views of a receiving device for receipt of images with supplemental content for messaging in accordance with one or more implementations.FIG. 4 illustrates a schematic diagram showing an exemplary data flow for integration of supplemental content with a captured image for transmission in a message in accordance with one or more implementations.FIG. 5 illustrates a schematic diagram showing an exemplary data flow for packaging of supplemental content metadata to accompany a captured image for transmission in a message in accordance with one or more implementations.FIG. 6 illustrates a schematic diagram showing an exemplary data flow for integration of supplemental content with a camera feed for transmission in accordance with one or more implementations.FIG. 7 illustrates a schematic diagram showing an exemplary data flow for packaging of supplemental content metadata to accompany a camera feed for transmission in accordance with one or more implementations.FIG. 8 illustrates a flow diagram of an example process for modifying images with supplemental content for messaging in accordance with one or more implementations.FIG. 9 illustrates a flow diagram of an example process for sending a message including a captured image with supplemental content in accordance with one or more implementations.FIG. 10 illustrates a flow diagram of another example process for sending a message including a captured image with supplemental content in accordance with one or more implementations.FIG. 11 illustrates a flow diagram of an example process for displaying images modified with supplemental content in a message stream in accordance with one or more implementations.FIG. 12 illustrates an example user interface of a selection of a camera mode option in a messaging application in accordance with one or more implementations.FIG. 13 illustrates an example user interface of a camera feed displayed with supplemental content options after selection of a camera mode option in accordance with one or more implementations.FIG. 14 illustrates an example user interface of supplemental content added to a camera feed in accordance with one or more implementations.FIG. 15 illustrates an example user interface of stickers that can be added to a camera feed in accordance with one or more implementations.FIG. 16 illustrates an electronic system with which one or more implementations of the subject technology may be implemented. The embodiments of figs. 6 and 9 where the image is integrated with supplemental content are not covered by the claims. DETAILED DESCRIPTION The detailed description set forth below is intended as a description of various configurations of the subject technology and is not intended to represent the only configurations in which the subject technology can be practiced. The appended drawings are incorporated herein and constitute a part of the detailed description. The detailed