Search

CN-122018687-A - Gaze interaction system based on gravity vector constraint and space topological association

CN122018687ACN 122018687 ACN122018687 ACN 122018687ACN-122018687-A

Abstract

The invention provides a gazing interaction system based on gravity vector constraint and spatial topological association, which comprises a gazing acquisition module, a gesture correction module, a spatial object index module, a collision judgment and locking module and a protocol communication module, wherein the gazing acquisition module is used for acquiring user gazing information and generating an original gazing direction vector, the gesture correction module is used for acquiring a gravity vector and constructing a gesture correction operator based on the gravity vector, the spatial object index module is used for maintaining a spatial object library, the collision judgment and locking module is used for constructing gazing rays based on the gazing direction vector aligned by gravity, and the protocol communication module is used for distributing control instructions to target spatial objects and receiving execution receipts according to a standardized interaction protocol.

Inventors

  • CHEN HUICHONG

Assignees

  • 武汉华创全息影像技术有限公司

Dates

Publication Date
20260512
Application Date
20260128

Claims (10)

  1. 1. A gaze interaction system based on gravity vector constraint and spatial topological association is characterized by comprising the following modules: The system comprises a gaze acquisition module, a gesture correction module and a gesture correction module, wherein the gaze acquisition module is used for acquiring user gaze information and generating an original gaze direction vector; The system comprises a space object index module, a collision judgment and locking module, a target locking module and a target locking module, wherein the space object index module is used for maintaining a space object library, and the space object at least comprises an object identifier, a space position and geometric bounding volume information; and the protocol communication module is used for distributing control instructions to the target space object according to the standardized interaction protocol and receiving and executing the receipt.
  2. 2. A gaze interaction system based on gravity vector constraints associated with a spatial topology according to claim 1, wherein the gaze interaction system is adapted to implement the steps of: s1, collecting user fixation information by a space computing terminal and generating an original fixation direction vector; S2, acquiring a gravity vector by an inertia measurement unit of the space computing terminal, constructing a posture correction operator based on the gravity vector, correcting the original gazing direction vector, and obtaining a gazing direction vector with aligned gravity; S3, constructing a gazing ray based on the gazing direction vector aligned by the gravity, and acquiring geometrical bounding volume information of at least one space object in a space object library; S4, performing space collision judgment of the gazing rays and the geometric bounding volume information, and generating a target locking result when a hit condition and a gazing residence time threshold are met; And S5, distributing control instructions to the space objects corresponding to the target locking result according to a standardized interaction protocol, and receiving an execution receipt fed back by the space objects so as to complete fixation interaction closed loop.
  3. 3. The system for gaze interaction associated with a spatial topology based on gravity vector constraints of claim 1, further comprising a gaze interaction protocol defining a data interaction flow and a message format for gaze interaction between a spatial computing terminal and a spatial object, said protocol comprising at least: The space object broadcast message is used for externally announcing the existence and control capability of the space object, and at least comprises any one or more of an object identifier, space anchor point information, a semantic tag, a geometric bounding volume parameter and an executable instruction set; The system comprises a space computing terminal, a fixation vector flow message, a target locking receipt message and a user terminal, wherein the space computing terminal is used for continuously outputting fixation ray information, and the fixation vector flow message at least comprises any one or more of fixation ray starting point information, fixation direction vector information, gravity alignment identification and timestamp information; The instruction distribution message is used for sending a control instruction to the target object after the target object is locked; executing receipt information, wherein the receipt information is used for feeding back an execution result after the target object executes a control instruction, and the protocol further provides that the space computing terminal executes gesture correction based on a gravity vector on the gaze direction vector, and performs space collision judgment based on corrected gaze rays and the geometric bounding volume parameters so as to complete gaze interaction closed loop.
  4. 4. The gaze interactive system associated with a spatial topology based on gravity vector constraints of claim 3, wherein said gaze interactive system is further configured to implement a spatial object broadcasting and registration method comprising: acquiring object identification, semantic tags, geometric bounding volume parameters and an executable instruction set of a space object; Acquiring space anchor point information of a space object, wherein the space anchor point information comprises any one or more of an absolute coordinate anchor point, a local coordinate anchor point or a visual characteristic anchor point; generating a space object broadcast message and broadcasting the space object broadcast message to a space index center or a space computing terminal; Writing, by the spatial index center, a spatial object to a spatial object library based on the spatial object broadcast message, thereby enabling a spatial computing terminal to perform collision determination with the spatial object library based on gaze rays and lock the spatial object.
  5. 5. The gaze-interactive system associated with a spatial topology based on gravity vector constraints of claim 2, wherein said raw gaze-direction vector comprises a compensation term for differences between the individual visual axis and the optical axis of the user, said compensation term being obtained by user calibration parameters, online adaptive estimation or preset model parameters; in the step S2, a yaw alignment operator is constructed based on magnetometer output or visual azimuth reference, so that the gazing direction vector after the gravity alignment is horizontally and azimuthally locked; The direction vector of the gazing ray is obtained by the combined action of a gravity alignment operator and a yaw alignment operator, so that the gazing ray meets the three-degree-of-freedom gesture stability; the geometric bounding volume information of the spatial object includes, but is not limited to, any one or more of an axis alignment bounding box AABB, an orientation bounding box OBB, a sphere, a cylinder, or a polyhedron.
  6. 6. The system of claim 5, wherein when the gaze ray hits a plurality of spatial objects, the depth ordering is performed according to the closest intersection distance of the ray and the spatial object, and the spatial object with the smallest distance is preferentially locked; The multi-target hit further performs occlusion culling in combination with depth sensors, binocular vision, structured light, toF, UWB ranging, or visual depth estimation results to reduce the locking probability of the occluded object or cull the occluded object; The multi-target hit further performs lock disambiguation in combination with spatial object semantic relevance determination including any one or more of a degree of matching of target class with current task, relevance of target with user action, topological adjacency relationship of target with history lock object, or visual semantic confidence consistency.
  7. 7. The gaze interaction system associated with spatial topology based on gravity vector constraints of claim 2, wherein said gaze dwell time threshold in S4 is used to gate switching between candidate hit status and steady lock status; the step of confirming intention is further included before the step S5, wherein the intention confirmation is realized by any one of voice confirmation, blink confirmation, gesture confirmation, secondary fixation confirmation or multi-mode combination confirmation; The space position of the space object is obtained by fusion of multi-source positioning/sensing information, wherein the multi-source positioning/sensing information comprises any one or more of satellite positioning, RTK, UWB, wiFi-FTM, bluetooth AoA or visual SLAM; The multisource positioning/sensing information fusion is realized through Kalman filtering or factor graph optimization, so that position jump or jitter caused by heterogeneous positioning source switching is restrained; the object identification of the space object library is generated by a space domain name resolution mechanism, and the space domain name resolution mechanism is used for uniformly mapping heterogeneous coordinate representations into space semantic object identifications; The standardized interaction protocol comprises a spatial object broadcast message which at least contains any one or more of an object identifier, spatial anchor point information, a semantic tag, a geometric shape parameter and an executable instruction set; when the space object is a dynamic object, executing time sequence alignment according to the object pose time stamp and the gazing ray time stamp, and executing space collision judgment in the step S4 after performing motion prediction compensation based on the object speed or acceleration information; And executing space authority verification before distributing the control instruction in S5, wherein the space authority verification at least comprises any one or more of gazing hit validity verification, operator identity authentication verification and operator space legality verification.
  8. 8. The gravity vector constraint and space topology correlation-based gaze interaction system of claim 2, wherein when abnormal instruction triggering, high-risk false triggering, impact event or authority rejection event is detected, triggering an XR space interaction black box memory certificate, wherein the interaction black box memory certificate at least records any one or more of physical sensing data, interaction track data and semantic evidence data; generating a data abstract in the terminal side trusted execution environment TEE and carrying out signature packaging on the data abstract, and asynchronously synchronizing the data abstract to an edge node or a trusted memory certificate service to form a multi-copy memory certificate; The semantic evidence data supports a data desensitization process that includes storing only any one or more of a target object feature descriptor, a semantic segmentation ROI, target frame information, or semantic category confidence, but not a complete original image or video stream; The XR space interaction black box certificate obtains a trusted time stamp through a satellite time service, a network time service or a clock synchronization server, and binds the time stamp with the data abstract to realize verifiable evidence chain integrity.
  9. 9. The gaze interactive system based on gravity vector constraint and spatial topological association of claim 8, wherein when a dynamic target is blocked, semantic recognition fails or a positioning signal is instantaneously interrupted, the system performs trajectory extrapolation based on a pose sequence before target disappearance and a motion vector within a preset out-of-lock time threshold delta T_hold to keep a target locking state; The space domain name resolution mechanism supports a multi-level resolution mode of cloud full resolution, edge node area cache and terminal local prefetching cache, so that a space computing terminal can finish local collision judgment and target locking based on a pre-downloaded space object topology subset under a weak network or offline condition, and can find temporarily-appearing local space objects in real time through a broadcast protocol.
  10. 10. The gaze interactive system based on gravity vector constraint and spatial topological association of claim 9, wherein the spatial domain name resolution mechanism (sDNS) supports pose subscription update of dynamic spatial objects, when the spatial objects are dynamic objects, geometric bounding volume parameters of the spatial objects are pushed to a terminal cache along with the change of object poses in a preset update frequency or threshold triggering mode, and object pose timestamps are carried for time sequence alignment with gaze ray timestamps, so that consistency of spatial collision judgment is ensured; When a plurality of space computing terminals lock the same space object at the same time, the space domain name resolution mechanism sDNS or the edge node issues an object locking state according to a preset authority priority, preemption logic or queuing strategy, and returns an occupation prompt or refuses receipt to the non-occupation terminal so as to avoid the concurrent conflict and misoperation of instructions.

Description

Gaze interaction system based on gravity vector constraint and space topological association Technical Field The invention relates to the technical fields of space calculation, body intelligence, man-machine interaction and safety control of the Internet of things, in particular to a gazing interaction system based on gravity vector constraint and space topology association. Background With the development of space computing (Spatial Computing) and body intelligence, space computing terminals (including but not limited to AI glasses, AR glasses, head-mounted devices, mobile terminals, and body-intelligent robotic terminals, etc.) are gradually moving from "information display/content consumption" to "physical world operation portal". The gazing interaction is considered as one of the most intuitive input modes, namely, a user can express selection, attention and operation intention only by looking at a certain object, so that the natural interaction experience of what you see is what you get is realized. Starting from the first principle, the fixation interaction is to be a physical world operation entrance, and three underlying conditions, namely geometric alignment, fixation rays, object addressing, fixation entities, instruction executable, interaction results and protocol closed loops, must be calculated, landed and reproduced in a three-dimensional space, and the fixation entities must have unique identities and spatial indexes in the system. The prior art cannot meet the conditions at the same time, namely, if a stable reference system which does not change along with the environment is absent in the prior art, fixation interaction cannot become an executable physical control mode and can only stay on a weak interaction level of prompt/display, the prior art generally depends on GNSS/RTK outdoors and UWB/WiFi and the like indoors for local positioning, and the visual SLAM forms another set of local coordinate system. If heterogeneous coordinates cannot be unified into the same object addressing mechanism, fixation interaction cannot work across scenes and cannot become an industry universal protocol, the existing internet of things (IoT) protocol emphasizes network connectivity instead of spatial semantics, a spatial index layer is not provided, and a unified entry does not exist for physical world interaction. Most of the existing interaction schemes lack a 'security protocol layer' oriented to high-risk scenes, and lack an 'XR space interaction black box' mechanism similar to an aviation/automobile event recorder, so that the system cannot enter into strong supervision and high-risk industry application. Spatial interaction protocols without audit certificates are difficult to become an industry infrastructure. Disclosure of Invention Aiming at the defects existing in the prior art, the invention aims to provide a fixation interaction system based on the association of gravity vector constraint and space topology so as to solve the problems in the background art, the invention can be used for solving the problems in different indoor and outdoor space environments, the unified space coordinate expression and object addressing mechanism is constructed through the multisource positioning/sensing information, and space object broadcasting, gazing vector stream transmission, interactive locking receipt and instruction distribution are completed through a standardized protocol, so that closed-loop interaction of 'what you see is what you control' of a physical entity is finally realized. In order to achieve the aim, the invention is realized through the following technical scheme that the gazing interaction system based on the association of gravity vector constraint and space topology comprises the following modules: The system comprises a gaze acquisition module, a gesture correction module and a gesture correction module, wherein the gaze acquisition module is used for acquiring user gaze information and generating an original gaze direction vector; The system comprises a space object index module, a collision judgment and locking module, a target locking module and a target locking module, wherein the space object index module is used for maintaining a space object library, and the space object at least comprises an object identifier, a space position and geometric bounding volume information; and the protocol communication module is used for distributing control instructions to the target space object according to the standardized interaction protocol and receiving and executing the receipt. Further, the gaze interaction system is configured to implement the steps of: s1, collecting user fixation information by a space computing terminal and generating an original fixation direction vector; S2, acquiring a gravity vector by an inertia measurement unit of the space computing terminal, constructing a posture correction operator based on the gravity vector, correcting the original gazing direction vecto