Search

CN-121433490-B - Eye movement precise tracking and identifying system and method based on artificial intelligence

CN121433490BCN 121433490 BCN121433490 BCN 121433490BCN-121433490-B

Abstract

The invention discloses an eye movement precise tracking and identifying system and method based on artificial intelligence, an intelligent interaction tracking expansion module, an intelligent interaction tracking and calibrating module, a video synchronization and eye movement analysis module, a data visualization and cloud storage module, a thermodynamic diagram generation rendering, composite screen snapshot generation, eye movement structured data generation and composite screen snapshot cloud storage module, a monitoring library loading state, a database connection state, eye movement data uploading result and tracking data acquisition process reliability, wherein the intelligent interaction tracking expansion module integrates a multisource database to encapsulate an HTML instruction file set to construct a video eye movement tracking data collection platform, the eye movement tracking and calibration module controls eye movement tracking and triggers calibration, controls camera access, eye movement key point identification, gaze point prediction calibration and precision test, the video synchronization and eye movement analysis module constructs an immersive video playing environment to correlate eye movement data with a video timestamp, and performs gaze point eye movement behavior identification statistics.

Inventors

  • QI MENG

Assignees

  • 山东新宇科技发展有限公司

Dates

Publication Date
20260508
Application Date
20251021

Claims (6)

  1. 1. An artificial intelligence based eye movement precision tracking and identifying system, comprising: The intelligent interactive tracking expansion module integrates and packages the HTML instruction file set with the multi-source database, constructs a video eye tracking data collection platform, and intelligently interactively expands eye tracking information and high-precision eye tracking; the eye movement tracking and calibrating module is used for controlling eye movement tracking and triggering calibration, and performing eye movement tracking precision calibration test by controlling camera access, eye movement key point identification, gaze point prediction calibration and precision test; a video synchronization and eye movement analysis module comprising: The immersive Video playing unit automatically calls a full-screen playing instruction to enter a full-screen mode through a Video tag of the interface Video player to construct an immersive Video playing environment; the eye movement data real-time acquisition unit continuously receives the screen gaze point coordinate output by the eye movement tracking JS library by registering a high-frequency callback function; The data time stamp association unit monitors a video time update event, records a video playing time stamp according to a set video recording period, binds the video playing time stamp with the currently acquired gaze point coordinates, and forms an eye movement basic data set; The dynamic statistics panel updating unit is used for carrying out fixation point eye jump behavior identification statistics, dynamic statistics calculation and displaying eye movement real-time data information; data visualization and high in the clouds storage module includes: The real-time thermodynamic diagram generating unit calls a thermodynamic diagram visualization JavaScript library according to the eye movement real-time data information, and accumulates all the gaze points based on screen coordinates to form gaze point density distribution; The composite screen snapshot capturing unit automatically triggers snapshot generation, captures composite screen snapshot information of a video playing area, an accumulated thermodynamic diagram, a user information floating layer and an eye movement detection frame, associates the composite screen snapshot information with a corresponding fixation SessionId, generates HTTP read-write snapshot URL, and uploads the HTTP read-write snapshot URL to the cloud for storage; automatically structuring the eye movement data unit, automatically structuring the eye movement real-time data information to form structured eye movement data, and automatically uploading the structured eye movement data to a cloud NoSQL document database; The eye movement tracking and debugging monitoring unit is used for tracking an eye movement data test calibration execution state, a test data uploading state and an updated thermodynamic diagram state by an interface, monitoring a library loading state, a database connecting state and an eye movement data uploading result, and tracking the reliability of a data acquisition process.
  2. 2. The artificial intelligence based eye movement precision tracking and identification system of claim 1, wherein the intelligent interactive tracking expansion module comprises: the video eye tracking data collection platform is used for integrating and packaging an eye tracking JavaScript library, a thermodynamic diagram visual Javascript library and a cloud NoSQL document database in an independent HTML file to form an integrated packaging HTML instruction file set, and constructing the video eye tracking data collection platform; the eye tracking library error-tolerant loading unit dynamically loads an eye tracking JavaScript library from a CDN source by monitoring onload events according to an integrated package HTML instruction file set to realize a fault-tolerant mechanism; The cloud database connection unit synchronously initializes the connection with the cloud NoSQL document database and verifies the access authority of the database; The user session identifier generating unit verifies the access authority setting of the database and generates a user information floating layer, wherein the user information floating layer comprises a user UserId and a session SessionId, and a basic eye tracking interface control button is constructed.
  3. 3. The artificial intelligence based eye movement precision tracking and identification system of claim 1, wherein the eye movement tracking and calibration module comprises: The camera access authority application unit is used for calling an eye tracking JavaScript library API, applying the access authority of the network camera to the user and starting the network camera; the camera control unit is used for controlling the network camera to acquire an eye movement tracking real-time video stream; And the nine-point calibration execution unit sequentially displays the nine-point calibration execution unit according to the set flash time length through presetting the calibration points, acquires the eye movement key point data and the screen click coordinates during clicking, inputs a ridge regression model for parameter training and optimization, and improves the gaze point prediction precision.
  4. 4. An eye movement precise tracking and identifying method based on artificial intelligence is characterized by comprising the following steps: P100, integrating and packaging an HTML instruction file set with a multi-source database, constructing a video eye tracking data collection platform, and intelligently interactively expanding eye tracking information and high-precision eye tracking; P200, controlling eye movement tracking and triggering calibration, and performing eye movement tracking precision calibration test by controlling camera access, eye movement key point identification, gaze point prediction calibration and precision test; P300, comprising: P301, automatically calling a full-screen playing instruction to enter a full-screen mode through a Video tag of an interface Video player to construct an immersive Video playing environment; P302, continuously receiving the screen gaze point coordinate output by the eye movement tracking JS library by registering a high-frequency callback function, applying Kalman filtering to the screen gaze point coordinate data, setting a process noise covariance and a measurement noise covariance, smoothing the data and reducing environmental noise interference; P303, monitoring a video time update event, recording a video playing time stamp according to a set video recording period, and binding the video playing time stamp with the currently acquired fixation point coordinates to form an eye movement basic data set; p304, performing fixation point eye jump behavior identification statistics, dynamic statistics calculation and displaying eye movement real-time data information; P400, comprising: P401, calling a thermodynamic diagram visual JavaScript library according to the eye movement real-time data information, and accumulating all the gaze points based on screen coordinates to form gaze point density distribution; p402, automatically triggering snapshot generation, capturing composite screen snapshot information of a video playing area, an accumulated thermodynamic diagram, a user information floating layer and an eye movement detection frame, associating the composite screen snapshot information with a corresponding fixation SessionId, generating HTTP read-write snapshot URL, and uploading the HTTP read-write snapshot URL to a cloud for storage; p403, automatically structuring the eye movement real-time data information to form structured eye movement data, and automatically uploading the structured eye movement data to a cloud NoSQL document database; P404, the interface tracks the eye movement data test calibration execution state, test data uploading state, updated thermodynamic diagram state, monitoring library loading state, database connection state, eye movement data uploading result, and tracks the reliability of the data acquisition process.
  5. 5. The artificial intelligence based eye movement precision tracking and identifying method according to claim 4, wherein P100 comprises: p101, integrating and packaging an eye tracking JavaScript library, a thermodynamic diagram visual Javascript library and a cloud NoSQL document database in an independent HTML file to form an integrated packaging HTML instruction file set, and constructing a video eye tracking data collection platform; P102, dynamically loading an eye tracking JavaScript library from a CDN source according to an integrated package HTML instruction file set by monitoring onload events to realize a fault-tolerant mechanism; P103, synchronously initializing connection with the cloud NoSQL document database, and verifying the access authority of the database; And P104, verifying the access authority setting of the database to generate a user information floating layer, wherein the user information floating layer comprises a user UserId and a session SessionId, and a basic eye tracking interface control button is constructed.
  6. 6. The artificial intelligence based eye movement precision tracking and identifying method according to claim 4, wherein P200 comprises: P201, calling an eye tracking JavaScript library API, applying access rights of a network camera to a user, and starting the network camera; P202, controlling a network camera to acquire an eye movement tracking real-time video stream; and P203, displaying according to the set flash time sequentially through preset calibration points, collecting eye movement key point data and screen click coordinates during clicking, inputting a ridge regression model for parameter training and optimization, and improving the gaze point prediction precision.

Description

Eye movement precise tracking and identifying system and method based on artificial intelligence Technical Field The invention relates to the technical field of intelligent machine learning precise tracking and monitoring systems, in particular to an eye movement precise tracking and identifying system and method based on artificial intelligence. Background At present, along with the development of computer vision and man-machine interaction technology, eye tracking technology has been widely applied to the technical fields of research, experience test, popularization effect evaluation and the like; the prior eye tracking system mainly has the technical defects of high deployment complexity, poor portability, incapability of quickly adapting to different terminals (such as mobile phones, PCs and notebook computers) or experimental scenes, insufficient data association precision, incapability of establishing accurate time synchronization of eye movement data and video playing processes, incapability of establishing a one-to-one correspondence relationship between video content frames and user gaze points, incapability of carrying out depth association analysis of video content and user attention, limited gaze point prediction precision, incapability of partially light-weight eye tracking schemes, incapability of effectively calibrating and optimizing mechanisms, incapacitating key points of human faces only depending on basic computer vision algorithms, easiness of being influenced by environmental light and user gesture changes, large gaze point prediction noise, difficulty of guaranteeing data quality, incapability of ensuring that most systems only store original eye movement coordinate data, lack of real-time visual display (such as thermodynamic diagram), incapability of associating visual results with the original data, incapability of combining visual distribution and structure analysis, incapability of simplifying the following problems, difficulty of judging how to acquire data by using a comprehensive data, and debugging tool, difficulty of improving the accuracy of the development of the system, and debugging the development of the comprehensive data, and debugging tool is difficult to be improved by aiming at the problems of the research of the accuracy of the development of the data, such as the method is difficult to be easily verified by the research of the data is difficult to develop, the eye tracking system integrating visualization and storage and being convenient for debugging is still needed to solve the problems of light weight, high adaptability, user attention analysis requirement and the like, so that an eye precise tracking and identifying system and method based on artificial intelligence are needed to at least partially solve the problems in the prior art. Disclosure of Invention The summary of the invention is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to identify the scope of the claimed subject matter, since a series of concepts in a simplified form are included in the summary of the invention, which is described in further detail in the detailed description. To at least partially solve the above problems, the present invention provides an artificial intelligence-based eye movement precision tracking recognition system, comprising: The intelligent interactive tracking expansion module integrates and packages the HTML instruction file set with the multi-source database, constructs a video eye tracking data collection platform, and intelligently interactively expands eye tracking information and high-precision eye tracking; the eye movement tracking and calibrating module is used for controlling eye movement tracking and triggering calibration, and performing eye movement tracking precision calibration test by controlling camera access, eye movement key point identification, gaze point prediction calibration and precision test; the Video synchronization and eye movement analysis module is used for constructing an immersive Video playing environment, controlling the full-screen playing of the Video through the full screen of the Video tag, correlating the eye movement data with the Video time stamp, and carrying out the gaze point eye jump behavior identification statistics; The data visualization and cloud storage module is used for performing thermodynamic diagram generation rendering, composite screen snapshot generation, eye movement structured data generation and composite screen snapshot cloud storage, monitoring a database loading state, a database connection state and an eye movement data uploading result, and tracking the reliability of a data acquisition process. Preferably, the intelligent interactive tracking expansion module comprises: the video eye tracking data collection platform is used for integrating and packaging an eye tracking JavaScript library, a thermodynamic diagram visual Javascript libra