CN-122023741-A - Animation character three-dimensional model interactive display method based on augmented reality

CN122023741ACN 122023741 ACN122023741 ACN 122023741ACN-122023741-A

Abstract

The invention discloses an interactive display method of an animation character three-dimensional model based on augmented reality, which relates to the technical field of augmented reality model interaction, and comprises the steps of collecting physical space video stream data through an image sensor, and constructing a physical space coordinate frame through space feature recognition and mark point detection, and aligning the coordinate origin of the three-dimensional model of the animation character with the position space anchor of the mark point. And acquiring a user voice command signal, analyzing an interaction command through keyword matching and semantic structure analysis, rendering a three-dimensional model to a real scene picture in the augmented reality display device to form a virtual-real fusion picture, executing behavior logic operation according to the type of the interaction command, and driving the model to make corresponding dynamic behaviors. According to the invention, the space anchoring mode of the three-dimensional model and the voice interaction analysis logic are optimized, and the space precision and the interaction effect of the augmented reality animation character display are improved.

Inventors

LI YANG
LUO NA
LIU MAN
CHEN SHENGGUI
CHEN YIGUO
CHEN YIREN

Assignees

湖南科技职业学院

Dates

Publication Date: 20260512
Application Date: 20260409

Claims (10)

1. The method for interactive display of the three-dimensional model of the animated character based on the augmented reality is characterized by comprising the following steps: Acquiring video stream data of a physical space where a user is located in real time through an image sensor, and carrying out space feature identification and mark point detection on the video stream data to construct a physical space coordinate frame; Loading an animation character preset three-dimensional model file, and carrying out space anchoring alignment on the coordinate origin of the animation character preset three-dimensional model file and the position established by the detected mark point in the physical space coordinate frame; Acquiring a voice command signal from a voice input channel of a user, performing voice keyword matching and semantic structure analysis on the voice command signal, and analyzing an interaction command representing the interaction intention of the user; Rendering a model corresponding to the animation character preset three-dimensional model file into a real scene picture generated by the video stream data on an augmented reality display device by taking the position of the marking point as a reference to form a virtual-real combined composite augmented reality picture; And executing corresponding behavior logic operation on the preset three-dimensional model file of the animation character according to the type of the interaction instruction so as to drive the preset three-dimensional model file of the animation character to make corresponding dynamic behaviors in the composite augmented reality picture.
2. The augmented reality-based animated character three-dimensional model interactive presentation method of claim 1, wherein performing spatial feature recognition and marker point detection on the video stream data to construct a physical spatial coordinate frame comprises: Performing corner extraction and edge contour detection on each frame of picture in the video stream data to generate a scene key feature point cloud; carrying out characteristic point tracking and matching on the scene key characteristic point clouds of a plurality of continuous picture frames, and calculating the spatial displacement and visual angle change parameters of the characteristic points between adjacent picture frames; Reconstructing a three-dimensional point cloud map of a physical space by utilizing a motion restoration structure algorithm based on the spatial displacement and the visual angle change parameters, and estimating the motion pose of the image sensor; Searching an image area meeting preset geometric shapes and pattern characteristics in the three-dimensional point cloud map, judging the image area as available augmented reality mark points, and distributing a unique space identifier and three-dimensional position coordinates in the three-dimensional point cloud map for each augmented reality mark point; And integrating the spatial position coordinates and identifiers of all the augmented reality mark points, and combining the real-time motion pose of the image sensor to form a physical spatial coordinate frame containing the mapping relation of an absolute coordinate system, the mark point coordinate system and the screen coordinate system.
3. The augmented reality-based animated character three-dimensional model interactive presentation method of claim 2, wherein spatially anchoring the origin of coordinates of the animated character preset three-dimensional model file to the location of the detected marker points established in the physical space coordinate frame comprises: Reading the model center point coordinate data of the preset three-dimensional model file of the animation character, and acquiring the coordinate value of the model center point coordinate data under the local coordinate system of the preset three-dimensional model file of the animation character; Querying a spatial identifier of an augmented reality marking point selected as a display reference from the physical space coordinate frame, and acquiring three-dimensional world coordinates of the augmented reality marking point under the physical space coordinate frame; Calculating a coordinate conversion matrix from a model center point coordinate of the animation character preset three-dimensional model file to the augmented reality mark point, wherein the coordinate conversion matrix comprises a translation vector, a rotation vector and a scaling factor; Applying the coordinate transformation matrix to vertex data of the preset three-dimensional model file of the animation character, so that a model center point of the preset three-dimensional model file of the animation character and a spatial position of the augmented reality mark point are completely overlapped; And establishing a dynamic coordinate mapping relation between a local coordinate system and a physical space coordinate system of the animation character preset three-dimensional model file, so that the animation character preset three-dimensional model file can keep an anchoring relation with a physical mark point in an augmented reality picture according to the dynamic coordinate mapping relation when the pose of the image sensor changes.
4. The augmented reality-based animated character three-dimensional model interactive presentation method of claim 3, wherein performing a voice keyword matching and a semantic structure analysis on the voice command signal resolves an interactive command representing a user's interactive intention, comprising: noise reduction and gain control processing are carried out on the collected original voice command signals, and pure voice waveform signals are extracted; performing voice endpoint detection on the pure voice waveform signal to segment out voice fragments containing effective instructions; Extracting acoustic feature vectors from the voice fragments, inputting the extracted acoustic feature vectors into a pre-trained acoustic model, and calculating to obtain probability distribution of a phoneme sequence; decoding the phoneme sequence based on a statistical language model to generate a text instruction character string corresponding to the voice instruction signal; Matching the text instruction character string with a preset interactive instruction keyword library, and identifying a keyword combination comprising a core action verb and a target object noun; And performing dependency syntactic analysis on the identified keyword combination, and determining the grammatical relation of the action and the object so as to analyze a structured interaction instruction, wherein the interaction instruction comprises the action type, the action target object and optional parameters.
5. The augmented reality-based animated character three-dimensional model interactive display method of claim 4, wherein matching the text command string with a preset interactive command keyword library identifies a keyword combination comprising a core action verb and a target object noun, comprising: loading an interactive instruction keyword library containing multi-category action instructions, wherein the interactive instruction keyword library contains instruction trigger phrases, an action verb list, a model component name list and a parameter descriptor list; scanning the text instruction character string by adopting a forward maximum matching algorithm, and searching the longest matching phrase in the interactive instruction keyword library, wherein the phrase is marked as an instruction trigger phrase; After the text instruction character string fragment of the instruction trigger phrase is identified, continuing to scan the subsequent character strings, and searching a matched action verb in the action verb list; After the action verbs are identified, searching matched model component names in a model component name list in the adjacent character string intervals; if the matched model component name is found, further searching description words possibly accompanying with the moving direction, the action amplitude or the times in a parameter description word list in the nearby character strings; And combining the successfully matched instruction trigger phrase, action verb, model component name and parameter description word to form the keyword combination of the current voice interaction.
6. The method for interactive display of three-dimensional model of animated character based on augmented reality according to claim 5, wherein the step of rendering the model corresponding to the preset three-dimensional model file of the animated character to the real scene generated from the video stream data on the augmented reality display device with the position of the marker as a reference to form a virtual-real fusion composite augmented reality picture comprises: receiving the latest video stream data frame captured by the image sensor in real time, and decoding the latest video stream data frame into a live-action texture bitmap; Calculating a virtual camera view matrix and a projection matrix corresponding to the current video frame according to the physical space coordinate frame and the current pose of the image sensor; According to the anchoring position of the preset three-dimensional model file of the animation character in the physical space and the dynamic coordinate mapping relation, calculating vertex coordinate transformation data of the preset three-dimensional model file of the animation character under the current virtual camera view angle; inputting the calculated vertex coordinate transformation data into a graphic rendering pipeline, and generating a rendering image of the preset three-dimensional model file of the animation character under the current view angle by combining the loaded texture map and skeleton animation data of the preset three-dimensional model file of the animation character; And carrying out pixel-level fusion on the generated rendering image and the real scene texture bitmap, wherein opaque pixels in the rendering image cover corresponding pixels of the real scene texture bitmap, and transparent pixels display original contents of the real scene texture bitmap, so that the composite augmented reality picture including the virtual animation character model superimposed on the real scene is synthesized and output to an augmented reality display device.
7. The augmented reality-based animated character three-dimensional model interactive display method of claim 6, wherein performing corresponding behavioral logic operations on the animated character preset three-dimensional model file according to the type of the interactive instruction to drive the animated character preset three-dimensional model file to make corresponding dynamic behaviors in the composite augmented reality picture comprises: Inquiring a preset behavior logic rule base according to the action type in the analyzed interaction instruction to obtain a corresponding model behavior logic script, wherein the model behavior logic script defines a behavior sequence, behavior parameters and behavior triggering conditions which are required to be executed by the three-dimensional model; Extracting an action target object from the interaction instruction, wherein the action target object is a specific body part, prop or preset action segment of the three-dimensional model of the animation character; Calculating a numerical value required by a behavior parameter according to the model behavior logic script, wherein the numerical value is derived from a parameter carried in an interaction instruction or a default value obtained from a current model state; Invoking a three-dimensional model animation engine, and dispatching a skeleton animation controller, a material converter or a space converter of the animation character preset three-dimensional model file according to the model behavior logic script, the action target object and the calculated behavior parameters; The skeletal animation controller drives a skeletal system of the model to generate motion, the texture transformer changes the texture or color of the surface of the model, and the space transformer changes the position, orientation or scaling of the model; And applying the execution results of the controller and the converter to the animation character preset three-dimensional model file being rendered in real time, so that the animation character preset three-dimensional model file shows dynamic behavior change corresponding to the interaction instruction in the composite augmented reality picture.
8. The augmented reality-based three-dimensional model interactive presentation method of claim 7, wherein invoking a three-dimensional model animation engine, based on the model behavior logic script, the action target object, and the calculated behavior parameters, dispatches a skeletal animation controller, a texture transformer, or a space transformer of the animated character preset three-dimensional model file, comprises: Analyzing a behavior sequence definition in the model behavior logic script, decomposing the behavior sequence into a plurality of independent behavior units sequenced according to time axes, wherein each behavior unit is associated with a specific controller type identifier; Instantiating, for each behavior unit, a corresponding controller object in the three-dimensional model animation engine according to its associated controller type identifier, the controller object comprising a skeletal animation controller object, a texture transformer object, or a space transformer object; Mapping the action target object into an internal node tree structure of the animation character preset three-dimensional model file, and positioning to a skeletal joint node, a grid material node or a space transformation node corresponding to the action target object; Injecting the calculated behavior parameters into an instantiated controller object, wherein bone rotation quaternion or displacement vector is injected for the bone animation controller object, texture coordinate offset or color mixing factor is injected for the material converter object, and world coordinate translation matrix or Euler angle rotation data is injected for the space converter object; starting a scheduling queue of the three-dimensional model animation engine, sequentially activating corresponding controller objects according to preset time stamps of each behavior unit in the behavior sequence, and triggering the controller objects to execute data writing operation on the bound model nodes; Before each frame of picture is rendered, synchronously reading intermediate state data output by all the controller objects in an activated state, merging the intermediate state data, and uniformly writing the merged intermediate state data into a rendering state buffer zone of the preset three-dimensional model file of the animation character so as to complete real-time driving of the preset three-dimensional model file of the animation character.
9. The augmented reality-based animated character three-dimensional model interactive presentation method of claim 8, further comprising collision detection and response processing of the model in physical space interacting with the environment: In the physical space coordinate frame, a three-dimensional model file is preset for the animation character to generate a simplified collision body grid matched with the outer surface of the model; before the augmented reality display device renders each frame of composite augmented reality picture, acquiring expected position and posture data of the animation character preset three-dimensional model file at the next moment; extracting a high-density point cloud region representing a physical obstacle from the three-dimensional point cloud map, and converting the high-density point cloud region into an obstacle collision body grid; Performing space intersection detection calculation on the simplified collision body grids and the obstacle collision body grids, and judging whether the animation character preset three-dimensional model file is subjected to mold penetration or intersection with a physical obstacle at the next moment; If the expected collision is detected, calculating a model position offset vector and a rotation adjustment amount required for avoiding penetration according to the collision position and the surface normal information; And applying the model position offset vector and the rotation adjustment amount to a coordinate conversion matrix of the animation character preset three-dimensional model file, so that the model avoids physical barriers or slides along the surfaces of the barriers during final rendering, and interactive display conforming to the physical laws is realized.
10. The augmented reality-based animated character three-dimensional model interactive presentation method of claim 9, further comprising model interactive focus adaptive adjustment based on user gaze tracking: acquiring the gaze point coordinates of a user on the composite augmented reality screen in real time through an eyeball tracking sensor integrated in the augmented reality display device; converting the sight-gaze point coordinate from a screen coordinate system to a physical space coordinate system to obtain a focus position coordinate of the current sight of the user in a three-dimensional physical space; Calculating the distance between the focus position coordinate and each component part of the animation character preset three-dimensional model file in a three-dimensional space, and identifying a model component closest to the focus position coordinate as a potential interaction focus; Judging whether the potential interaction focus is continuously positioned in a user sight focus area and exceeds a preset gazing time threshold; If the gazing time threshold is exceeded, performing visual enhancement processing on the model component corresponding to the potential interaction focus, and unlocking a deep interaction instruction set related to the model component; when the interaction instruction is subsequently received, the interaction action corresponding to the model component which is currently visually enhanced is preferentially matched from the deep interaction instruction set.

Description

Animation character three-dimensional model interactive display method based on augmented reality Technical Field The invention belongs to the technical field of augmented reality model interaction, and particularly relates to an interactive display method of an animation character three-dimensional model based on augmented reality. Background The three-dimensional model display of the animation character in the existing augmented reality scene mostly adopts a mode of directly loading the three-dimensional model and matching single mark points to realize virtual-real superposition, the voice interaction link only completes instruction identification through simple keyword matching, and the dynamic behavior of the model is triggered by a preset fixed program. The technology does not construct a systematic coordinate frame aiming at the physical space, only completes model positioning through single-point marking, does not carry out deep analysis on a semantic level on a voice instruction, and only completes simple instruction correspondence by depending on keywords, and has defects. The anchoring of the three-dimensional model and the physical space only depends on single-point matching, spatial feature recognition and coordinate frame support are lacked, the model is easy to generate spatial deviation and inaccurate in positioning in a live-action picture, virtual-real fusion effect is poor, voice interaction only does keyword matching, semantic structures cannot be analyzed, real interaction intention of a user is difficult to accurately capture, the model cannot execute corresponding behavior logic operation according to different instruction types, dynamic behavior forms are single, and flexibility and accuracy of interactive display are insufficient. The physical space coordinate frame is required to be constructed, the space anchoring alignment of the coordinate origin of the model and the mark point is completed, meanwhile, the interaction intention is analyzed through voice keyword matching and semantic structure analysis, and the behavior logic operation is executed according to the instruction type to drive the dynamic behavior of the model, so that the problems of positioning and interaction in the prior art are solved, and the defects in the prior art are overcome. Disclosure of Invention The present invention aims to solve at least one of the technical problems existing in the prior art; Therefore, the invention provides an augmented reality-based three-dimensional model interactive display method for an animated character, which comprises the following steps: Acquiring video stream data of a physical space where a user is located in real time through an image sensor, and carrying out space feature identification and mark point detection on the video stream data to construct a physical space coordinate frame; Loading an animation character preset three-dimensional model file, and carrying out space anchoring alignment on the coordinate origin of the animation character preset three-dimensional model file and the position established by the detected mark point in the physical space coordinate frame; Acquiring a voice command signal from a voice input channel of a user, performing voice keyword matching and semantic structure analysis on the voice command signal, and analyzing an interaction command representing the interaction intention of the user; Rendering a model corresponding to the animation character preset three-dimensional model file into a real scene picture generated by the video stream data on an augmented reality display device by taking the position of the marking point as a reference to form a virtual-real combined composite augmented reality picture; And executing corresponding behavior logic operation on the preset three-dimensional model file of the animation character according to the type of the interaction instruction so as to drive the preset three-dimensional model file of the animation character to make corresponding dynamic behaviors in the composite augmented reality picture. Further, performing spatial feature recognition and mark point detection on the video stream data to construct a physical spatial coordinate frame, including: Performing corner extraction and edge contour detection on each frame of picture in the video stream data to generate a scene key feature point cloud; carrying out characteristic point tracking and matching on the scene key characteristic point clouds of a plurality of continuous picture frames, and calculating the spatial displacement and visual angle change parameters of the characteristic points between adjacent picture frames; Reconstructing a three-dimensional point cloud map of a physical space by utilizing a motion restoration structure algorithm based on the spatial displacement and the visual angle change parameters, and estimating the motion pose of the image sensor; Searching an image area meeting preset geometric shapes and pattern