EP-4742144-A1 - METHOD AND APPARATUS FOR INTERACTION AND PRODUCING SPECIAL EFFECT, AND DEVICE AND STORAGE MEDIUM

EP4742144A1EP 4742144 A1EP4742144 A1EP 4742144A1EP-4742144-A1

Abstract

The embodiments of the disclosure relates to a method, apparatus, device and storage medium for interaction and effect generating. The interaction method provided in the disclosure includes: detecting a plurality of reference key points associated with a virtual object to be mounted from a target image; determining a mounting position of the virtual object based on the position information of the plurality of reference key points in the target image; and mounting the virtual object to a mounting position in the target image. Based on the above approach, the embodiments of the disclosure can realize more accurate mounting of the virtual object and improve the interaction experience of the user.

Inventors

YUAN, QI

Assignees

Beijing Zitiao Network Technology Co., Ltd.

Dates

Publication Date: 20260513
Application Date: 20240630

Claims (20)

An interaction method, comprising: detecting a plurality of reference key points associated with a virtual object to be mounted from a target image; determining a mounting position of the virtual object based on position information of the plurality of reference key points in the target image; and mounting the virtual object to the mounting position in the target image.
The method of claim 1, wherein the mounting position is a target mounting position, and detecting the plurality of reference key points associated with the virtual object to be mounted from the target image comprises: determining the plurality of reference key points from the target image based on an expected mounting position of the virtual object, distances from the plurality of reference key points to the expected mounting position being less than a first threshold.
The method of claim 1, wherein the plurality of reference key points comprises at least a first reference key point and a second reference key point, and determining the mounting position of the virtual object based on the position information of the plurality of reference key points in the target image comprises: determining a first reference line based on the first reference key point and the second reference key point; and determining the mounting position of the virtual object based on the first reference line.
The method of claim 3, wherein determining the mounting position of the virtual object based on the first reference line comprises: determining a target position on the first reference line as the mounting position according to preset distance description information, wherein the distance description information indicates a relationship between a first distance from the target position to the first reference key point and a second distance from the target position to the second reference key point.
The method of claim 3, wherein the plurality of reference key points further comprises a third reference key point, and determining the mounting position of the virtual object based on the first reference line comprises: determining a second reference line based on the third reference key point and target direction information; and determining the mounting position based on an intersection of the first reference line and the second reference line.
The method of claim 5, wherein the third reference key point is a reference key point in the plurality of reference key points that has the shortest distance to the expected mounting position of the virtual object.
The method of claim 5, wherein the target direction information comprises: preset first direction information; or second direction information determined based on a gesture of a target object in the target image.
The method of claim 1, wherein mounting the virtual object to the mounting position in the target image comprises: determining a target orientation of the virtual object based on the position information of the plurality of reference key points; and mounting the virtual object to the mounting position in the target image according to the target orientation.
The method of claim 8, wherein determining the target orientation of the virtual object based on the position information of the plurality of reference key points comprises: determining at least one reference plane based on the plurality of reference key points, wherein each reference plane is determined based on at least three reference key points in the plurality of reference key points; and determining the target orientation of the virtual object based on first orientation information of the at least one reference plane.
The method of claim 9, wherein determining the target orientation of the virtual object based on the first orientation information of the at least one reference plane comprises: determining second orientation information of a target reference key point in the plurality of reference key points, a distance from the target reference key point to the expected mounting position of the virtual object being less than a preset distance; and determining the target orientation of the virtual object based on the first orientation information and the second orientation information.
The method of claim 1, wherein determining the mounting position of the virtual object based on the position information of the plurality of reference key points in the target image comprises: detecting a facial region from the target image; and determining the mounting position of the virtual object based on the position information of the plurality of reference key points in the target image, in response to a proportion of the facial region in the target image being less than or equal to a threshold.
The method of claim 11, further comprising: determining a position of a predetermined reference key point in the plurality of reference key points as the mounting position of the virtual object, in response to the proportion of the facial region in the target image being greater than the threshold; and determining an orientation of the predetermined reference key point in the plurality of reference key points as the target orientation of the virtual object; wherein the predetermined reference key point is a reference key point in the plurality of refrence key points that has the shortest distance to the expected mounting position of the virtual object.
A method for generating an effect, comprising: presenting an editing interface for generating a mounting effect; determining an expected mounting position of a virtual object to be mounted based on input information received in the editing interface; determining a set of reference key points associated with the expected mounting position, the set of reference key points being used for determining a target mounting position of the virtual object during execution of the mounting effect; and generating an effect file corresponding to the mounting effect based on the set of reference key points.
The method of claim 13, wherein the editing interface presents a reference model and a plurality of key points corresponding to the reference model, and determining the set of reference key points associated with the expected mounting position comprises: determining, from the plurality of key points, the set of reference key points having distances to the expected mounting position less than a threshold distance, wherein the threshold distance is determined based on user input.
The method of claim 14, further comprising: highlighting the set of reference key points in the plurality of key points in the editing interface.
The method of claim 13, further comprising: determining a set of distances from the expected mounting position to the set of reference key points; and generating, based on the set of distances, distance description information to indicate relationships among the set of distances, the distance description information being used for determining the target mounting position of the virtual object during execution of the mounting effect.
An interaction apparatus, comprising: a key point determining module configured to detect a plurality of reference key points associated with a virtual object to be mounted from a target image; a position determining module configured to determine a mounting position of the virtual object based on position information of the plurality of reference key points in the target image; and a mounting module configured to mount the virtual object to the mounting position in the target image.
An apparatus for generating an effect, comprising: an interface presenting module configured to present an editing interface for generating a mounting effect; a position marking module configured to determine an expected mounting position of a virtual object to be mounted based on input information received in the editing interface; a key point marking module configured to determine a set of reference key points associated with the expected mounting position, the set of reference key points being used for determining a target mounting position of the virtual object during execution of the mounting effect; and an effect generating module configured to generate an effect file corresponding to the mount effect based on the set of reference key points.
An electronic device, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, cause the electronic device to perform the method of any of claims 1 to 12 or 13 to 16.
A computer-readable storage medium having stored thereon a computer program executable by a processor to implement the method of any of claims 1 to 12 or 13 to 16.

Description

This application claims the benefit of Chinese Patent Application No. 202310807910.X, entitled "Method, Apparatus, Device And Storage Medium for Interaction and Effect Generation," filed on 07. 03, 2023, the entire content of which is incorporated herein by reference. FIELD Example embodiments of the present disclosure generally relate to the field of computers, and in particular, to methods, apparatuses, devices, and computer-readable storage media for interactions and effect generations. BACKGROUND With the development of computer technologies, the Internet has become an important platform for creating and sharing media content. In a process of creating the media content, video effects and picture effects are creation means frequently used by a user in an authoring process. These effects can enrich the media content created by the user and improve the level of interest of the content. SUMMARY In a first aspect of the present disclosure, an interaction method is provided. The method comprises the following steps: detecting a plurality of reference key points associated with a virtual object to be mounted from a target image; determining a mounting position of the virtual object based on the position information of the plurality of reference key points in the target image; and mounting the virtual object to the mounting position in the target image. In a second aspect of the present disclosure, a method for generating an effect is provided. The method comprises the following steps: presenting an editing interface for generating a mounting effect; determining an expected mounting position of a virtual object to be mounted based on input information received in the editing interface; determining a plurality of reference key points associated with the expected mounting position, the plurality of reference key points being used for determining a target mounting position of the virtual object during execution of the mounting effect; and generating an effect file corresponding to the mounting effect based on the plurality of reference key points. In a third aspect of the present disclosure, an interaction apparatus is provided. The apparatus includes: a key point determining module configured to detect a plurality of reference key points associated with a virtual object to be mounted from a target image; a position determining module configured to determine a mounting position of the virtual object based on position information of the plurality of reference key points in the target image; and a mounting module configured to mount the virtual object to the mounting position in the target image. In a fourth aspect of the present disclosure, an apparatus for generating an effect is provided. The apparatus comprises: an interface presenting module configured to present an editing interface for generating a mounting effect; a position marking module configured to determine an expected mounting position of the virtual object to be mounted based on input information received in the editing interface; a key point marking module configured to determine a plurality of reference key points associated with the expected mounting position, the plurality of reference key points being used for determining a target mounting position of the virtual object during execution of the mounting effect; and a file generating module configured to generate an effect file corresponding to the mounting effect based on the plurality of reference key points. In a fifth aspect of the present disclosure, an electronic device is provided. The apparatus includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the device to perform the method of the first aspect and/or the second aspect. In a sixth aspect of the present disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and the computer program is executable by the processor to implement the method of the first aspect and/or the second aspect. It should be understood that the content described in this content section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood from the following description. BRIEF DESCRIPTION OF DRAWINGS The abovementioned and other features, advantages, and aspects of various embodiments of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. In the drawings, the same or similar reference numbers refer to the same or similar elements, wherein: FIG. 1A and FIG. 1B illustrate schematic diagrams of a traditional mounting virtual object;F