Search

CN-122008245-A - Visual self-calibration system and method for automatic micro-needle patch equipment

CN122008245ACN 122008245 ACN122008245 ACN 122008245ACN-122008245-A

Abstract

The invention discloses a visual self-calibration system of automatic micro-needle patch equipment and a calibration method thereof, and relates to the technical field of device testing, comprising a data acquisition module, a calibration module and a calibration module, wherein the data acquisition module is used for measuring a composite calibration target and calibrating the spatial relationship between a visual camera, a laser profiler and a manipulator coordinate system; the system comprises a compensation model construction module, a drift prediction module, a deformation pre-compensation module, a deformation compensation correction module, a final compensation control instruction and a manipulator, wherein the compensation model construction module is used for establishing a corresponding relation model of grabbing conditions and pose compensation amounts based on patch deformation data acquired by a laser profiler, the drift prediction module is used for monitoring the temperature and the movement mileage of a motor and establishing a drift prediction model, the deformation pre-compensation module is used for acquiring deformation characteristics after the patch is picked and acquiring the pose pre-compensation amounts by using compensation model processing, the deformation compensation correction module is used for fusing the pose pre-compensation amounts and the drift pre-compensation amounts to generate the final compensation control instruction and drive the pose pre-compensation amounts to complete compensation movement, and the deformation compensation model shows great adaptability and robustness when facing complex operation conditions.

Inventors

  • ZENG JINSUI
  • YANG ZHOUJU
  • QIU ZIHAN

Assignees

  • 南通新世元生物科技有限公司

Dates

Publication Date
20260512
Application Date
20260407

Claims (9)

  1. 1. The utility model provides a microneedle automatic patch equipment vision self calibration system which characterized in that specifically includes: The data acquisition module is used for controlling the equipment manipulator to measure the composite calibration target fixed in the working space so as to finish the calibration of the spatial relationship between the vision camera, the laser profiler and the equipment manipulator coordinate system, and the tail end of the equipment manipulator is provided with an actuator, the vision camera and the laser profiler; the compensation model construction module is used for controlling the equipment manipulator to carry out multiple pick-up and placement operations, and based on patch deformation data acquired by the laser profiler, building a corresponding relation model of patch deformation characteristics and pose compensation amounts under different main grabbing conditions to serve as a deformation compensation model; the drift prediction module is used for establishing a drift prediction model for describing the association relation between the tail end drift amount of the manipulator and the state and time of the equipment based on the temperature of the motor, the movement mileage and the reference position feedback monitored by the equipment manipulator in the idle running process; The deformation pre-compensation module is used for acquiring the deformation characteristics of the patch adsorbed on the end effector after the equipment manipulator performs the patch picking operation, and processing the patch deformation characteristics by using a corresponding deformation compensation model according to the main grabbing condition of the patch picking operation to obtain the pose pre-compensation quantity; And the deformation compensation correction module is used for calling the drift prediction model according to the current equipment state, obtaining a drift pre-compensation quantity, fusing the pose pre-compensation quantity and the drift pre-compensation quantity, generating a final compensation control instruction, and driving the manipulator to complete compensation movement according to the instruction.
  2. 2. The system for self-calibration of vision of automatic microneedle patch equipment according to claim 1, wherein the logic based on which the spatial relationship between a vision camera, a laser profiler and an equipment manipulator coordinate system is calibrated is that the equipment manipulator is controlled to move, so that the vision camera and the laser profiler at the tail end of the equipment manipulator are sequentially aligned with a composite calibration target fixed on a workbench surface, and the surface of the target is provided with a precise circle center array with known world coordinates; The method comprises the specific steps of controlling a vision camera to shoot a target image, obtaining an internal reference matrix of the camera and initial camera external parameters relative to a target coordinate system through solving a vision camera calibration algorithm, controlling a laser profiler to scan the surface of the target, obtaining a measuring point cloud of the laser profiler, calculating a transformation relation from the laser profiler coordinate system to the target coordinate system through a fitting algorithm of a least square method; The internal reference matrix comprises the focal length and principal point position of the vision camera, and the initial camera external parameters specifically comprise the spatial position and the gesture of the camera relative to the target.
  3. 3. The visual self-calibration system of the automatic microneedle patch equipment is characterized by comprising the specific method for controlling the equipment manipulator to pick and place for multiple times, wherein a plurality of groups of grabbing conditions are set, the vacuum pressure values or the manipulator picking postures of the grabbing conditions are different, the logic for specifically setting the grabbing conditions is that for any vacuum pressure value, a plurality of groups of manipulator picking postures are set under the vacuum pressure value, the vacuum pressure value and the manipulator picking postures are combined to obtain a plurality of groups of grabbing conditions, and the grabbing conditions with the same vacuum pressure value are summarized into the same type of main grabbing conditions; The method comprises the steps of controlling an actuator at the tail end of a mechanical arm of equipment to perform actions of picking up a patch sample under each grabbing condition, after each picking up, controlling the mechanical arm of equipment to move the patch above a preset placement reference surface, controlling a laser profiler to perform full-width scanning on the lower surface of the suspended patch sample to obtain a high-density deformation point cloud of the lower surface of the patch sample, and recording a current vacuum pressure value and a mechanical arm picking gesture, wherein the mechanical arm picking gesture specifically comprises a picking up position and a mechanical arm picking up angle; the method comprises the specific steps of controlling a manipulator to move a patch sample downwards to a preset height from a placement reference surface for suspension, cutting off a vacuum pressure value of an actuator, synchronously recording analysis videos of the patch sample which is separated from the actuator and then freely falls down to contact the placement reference surface by a high-speed camera from the side surface, determining the maximum amplitude, oscillation frequency and damping attenuation time constant of the patch sample before the patch sample is stable from the analysis videos, and recording the preset height and the manipulator placement gesture during placement, wherein the manipulator placement gesture specifically comprises a placement position and a manipulator placement angle.
  4. 4. The visual self-calibration system of automatic micro-needle patch equipment, according to claim 3, wherein for a high-density deformation point cloud of the lower surface of a patch sample under any grabbing condition, a normal vector is set to point to a basic reference plane on the back surface of the patch sample, a directed distance from the patch sample to the basic reference plane is calculated, a two-dimensional deformation field is formed, wherein a positive value represents a bulge, a negative value represents a recess, patch deformation characteristics of the lower surface of the patch sample under the grabbing condition are calculated based on the formed two-dimensional deformation field, the patch deformation characteristics specifically comprise integral flatness, integral bias and integral kurtosis, the integral flatness is specifically characterized by root mean square error of the directed distance, a pick-up screening coefficient under the grabbing condition is calculated based on the patch deformation characteristics, the grabbing condition is screened based on the pick-up screening coefficient, and the method based on which the pick-up screening coefficient is calculated specifically is as follows: taking the reciprocal of the sum of the corresponding overall flatness and a preset compensation constant under the combination of the vacuum pressure value and the mechanical arm pick-up gesture as a first evaluation coefficient; taking the reciprocal of the sum of the corresponding integral deflection and a preset compensation constant under the combination of different vacuum pressure values and the picking gesture of the manipulator as a second evaluation coefficient; Taking the reciprocal of the sum of the corresponding overall kurtosis and a preset compensation constant under the combination of different vacuum pressure values and the picking gesture of the manipulator as a third evaluation coefficient; Calculating products of the first evaluation coefficient, the second evaluation coefficient and the third evaluation coefficient and the set authority coefficient respectively, and taking the sum of the three products as a picking and screening coefficient of a combination of a vacuum pressure value and a picking gesture of the manipulator; Based on the picking and screening coefficients, training a corresponding picking and screening coefficient prediction model, constructing the picking and screening coefficient prediction model based on a neural network model, taking set grabbing conditions as input values, namely a vacuum pressure value and a manipulator picking gesture as corresponding input values, taking the picking and screening coefficients corresponding to the grabbing conditions as labels, training the picking and screening coefficient prediction model, obtaining the picking and screening coefficient prediction model with the grabbing conditions as input values, and outputting the picking and screening coefficient prediction model as the corresponding picking and screening coefficients.
  5. 5. The visual self-calibration system of automatic microneedle patch equipment according to claim 4, wherein the method comprises the steps of screening grabbing conditions based on pick-up screening coefficients, marking grabbing conditions with pick-up screening coefficients not larger than a pick-up quality threshold, and eliminating grabbing conditions which are not marked; Aiming at any marked grabbing condition, taking the corresponding preset height and manipulator placing gesture as optimization variables, taking the maximum amplitude, the oscillation frequency, the damping decay time constant and the difference between the maximum amplitude, the oscillation frequency and the target placing position as optimization targets, determining the optimal preset height and the optimal manipulator placing gesture corresponding to the grabbing condition through an optimization algorithm, and calculating the adjustment quantity for adjusting the preset height and the manipulator placing gesture corresponding to the grabbing condition to the optimal preset height and the optimal manipulator placing gesture to serve as the pose compensation quantity corresponding to the grabbing condition, wherein the optimization algorithm specifically refers to a multi-target genetic algorithm, and the difference between the target placing positions specifically refers to the distance between the midpoint of a patch sample and the midpoint of the target placing position after the patch sample is placed.
  6. 6. The visual self-calibration system of automatic microneedle patch equipment according to claim 5, wherein the method for training the deformation compensation model is characterized by constructing a corresponding deformation compensation model for any main grabbing condition, extracting vacuum pressure values from all grabbing conditions to be consistent with the main grabbing condition, taking the marked grabbing condition as a target grabbing condition, taking patch deformation characteristics of the target grabbing condition as input, taking corresponding pose compensation quantity as a label to train the deformation compensation model, and obtaining a deformation compensation model corresponding to the main grabbing condition; The specific method for determining the pose pre-compensation quantity comprises the steps of obtaining the grabbing condition of the current picking patch operation, inputting the grabbing condition into a picking and screening coefficient prediction model which is trained, outputting a picking and screening coefficient prediction value of the grabbing condition, judging whether the current grabbing condition is not greater than a picking quality threshold value according to the picking and screening coefficient prediction value, if so, optimizing, determining the corresponding pose pre-compensation quantity, otherwise, re-grabbing; determining a corresponding deformation compensation model according to a main grabbing condition corresponding to the current pick-up patch operation, extracting patch deformation characteristics of the current grabbing condition, inputting the patch deformation characteristics into the corresponding deformation compensation model, and outputting corresponding pose compensation quantity by the deformation compensation model; if the main grabbing condition corresponding to the current pick-up patch operation does not exist, calculating the similarity between the current vacuum pressure value and the vacuum pressure value corresponding to the existing main grabbing condition, wherein the similarity of the vacuum pressure values is represented by Euclidean distance between the vacuum pressure values, determining the main grabbing condition with the maximum similarity, and taking the main grabbing condition as the main grabbing condition corresponding to the current pick-up patch operation.
  7. 7. The visual self-calibration system of the automatic microneedle patch device according to claim 6, wherein the logic based on which a drift prediction model is specifically built is that a device manipulator is controlled to continuously run for a plurality of hours at a classical working speed in a no-load state, the temperature of a device manipulator servo motor in the continuous running process is collected, the real-time temperature of the X, Y, Z-axis servo motor is specifically included, meanwhile, the accumulated movement position of each axis is determined based on the grating ruler reading of each axis of the device manipulator, the accumulated movement mileage of the device manipulator is calculated based on the accumulated movement position of each axis, the absolute position drift of each axis at the corresponding moment is collected, and the method based on which the accumulated movement mileage of the device manipulator is specifically calculated is as follows: Setting a plurality of sampling moments in the operation process of the equipment manipulator, acquiring grating ruler readings of the equipment manipulator X, Y and a Z axis at any sampling moment, and accumulating grating ruler readings of the equipment manipulator X, Y and the Z axis corresponding to the sampling moments to obtain accumulated movement mileage of the equipment manipulator at the sampling moment; Based on the accumulated movement mileage of the equipment manipulator, the temperature of each axis servo motor and the absolute position drift of each corresponding axis, a drift prediction model is established, specifically, the absolute position drift data of each axis is taken as dependent variables, the accumulated movement mileage of the equipment manipulator and the temperature of each axis servo motor are taken as independent variables, and a drift prediction model expression is established through a multiple linear regression analysis method; And for the drift prediction models of the X axis and the Y axis of the equipment manipulator, the method for establishing the drift prediction models is the same as the method for establishing the Z axis drift prediction models, and the drift prediction values of the X axis, the Y axis and the Z axis of the equipment manipulator are determined according to the drift prediction models of the X axis, the Y axis and the Z axis of the equipment manipulator.
  8. 8. The visual self-calibration system of the automatic microneedle patch device according to claim 7, wherein the logic based on which the pose pre-compensation amount and the drift pre-compensation amount are integrated is that a compensation value and a direction of a preset height and a manipulator placement posture are output based on a deformation compensation model, the preset height is compensated through a drift predicted value of a Z axis of the device manipulator, specifically expressed as a sum of the drift predicted value of the Z axis of the device manipulator and the preset height, the placement position in the manipulator placement posture is compensated through drift predicted values of X axis and Y axis of the device manipulator, so that a final compensation control instruction is generated, and the manipulator is driven to complete compensation movement according to the instruction.
  9. 9. A method for self-calibrating vision of automatic microneedle patch equipment, for controlling a system for self-calibrating vision of automatic microneedle patch equipment according to any one of claims 1 to 8, comprising: The method comprises the steps that a mechanical arm of the equipment is controlled to measure a composite calibration target fixed in a working space, so that the calibration of the spatial relationship between a visual camera, a laser profiler and a coordinate system of the mechanical arm of the equipment is completed, and an actuator, the visual camera and the laser profiler are arranged at the tail end of the mechanical arm of the equipment; The control equipment manipulator performs multiple pick-up and placement operations, and establishes corresponding relation models of patch deformation characteristics and pose compensation amounts under different main grabbing conditions based on patch deformation data acquired by the laser profiler to serve as deformation compensation models; the method comprises the steps of (1) feeding back the temperature, the movement mileage and a reference position of a motor monitored by a mechanical arm of equipment in the no-load operation process, and establishing a drift prediction model for describing the association relation between the drift amount of the tail end of the mechanical arm and the state of the equipment and time; after the equipment manipulator performs the patch picking operation, patch deformation characteristics of patches adsorbed on the end effector are obtained, and according to main grabbing conditions of the patch picking operation, the patch deformation characteristics are processed by using a corresponding deformation compensation model, so that pose pre-compensation quantity is obtained; And calling the drift prediction model according to the current equipment state, acquiring a drift pre-compensation quantity, fusing the pose pre-compensation quantity and the drift pre-compensation quantity, generating a final compensation control instruction, and driving the manipulator to complete compensation movement according to the instruction.

Description

Visual self-calibration system and method for automatic micro-needle patch equipment Technical Field The invention relates to the technical field of device testing, in particular to a visual self-calibration system and a visual self-calibration method for automatic micro-needle patch equipment. Background In modern medical and cosmetic industries, automatic microneedle patch devices are widely used in the fields of drug delivery and the like. The device accurately attaches the medicine or other therapeutic components to the target by utilizing the microneedle technology, however, due to the deformation of the microneedle patch and the influence of external factors, how to ensure the accurate placement and deformation compensation of the patch in the operation process becomes a key problem for improving the performance and the use safety of the device; The prior art generally relies on static calibration and simple machine vision algorithms to achieve positioning of the device and placement of the patch. These methods are generally operated by pre-set parameters and empirical models, ensuring the accuracy of the patch to some extent. In addition, some devices attempt to monitor environmental changes using sensors to achieve dynamic compensation. However, most of the technologies have single functions, often cannot adapt to different operation conditions in real time, and analysis on deformation characteristics is simple and lacks pertinence; although the prior art reduces errors in the patch attachment process to some extent, significant disadvantages remain. The main problems include lack of deep analysis and dynamic compensation capability for deformation characteristics of the patch, which results in limited placement accuracy and consistency of the patch in practical applications. In addition, the prior art fails to fully consider the influence of the running state of the equipment on the operation precision, so that the drift of the manipulator cannot be effectively predicted and compensated in the long-time use process. Therefore, it is particularly desirable to develop a self-calibrating system that can monitor, analyze, and compensate for deformation and drift in real time. The above information disclosed in the background section is only for enhancement of understanding of the background of the disclosure and therefore it may include information that does not form the prior art that is already known to a person of ordinary skill in the art. Disclosure of Invention The invention aims to provide a visual self-calibration system and a visual self-calibration method for automatic micro-needle patch equipment, which are used for solving the problems in the background technology. In order to achieve the above purpose, the present invention provides the following technical solutions: a visual self-calibration system of automatic micro-needle patch equipment specifically comprises: The data acquisition module is used for controlling the equipment manipulator to measure the composite calibration target fixed in the working space so as to finish the calibration of the spatial relationship between the vision camera, the laser profiler and the equipment manipulator coordinate system, and the tail end of the equipment manipulator is provided with an actuator, the vision camera and the laser profiler; the compensation model construction module is used for controlling the equipment manipulator to carry out multiple pick-up and placement operations, and based on patch deformation data acquired by the laser profiler, building a corresponding relation model of patch deformation characteristics and pose compensation amounts under different main grabbing conditions to serve as a deformation compensation model; the drift prediction module is used for establishing a drift prediction model for describing the association relation between the tail end drift amount of the manipulator and the state and time of the equipment based on the temperature of the motor, the movement mileage and the reference position feedback monitored by the equipment manipulator in the idle running process; The deformation pre-compensation module is used for acquiring the deformation characteristics of the patch adsorbed on the end effector after the equipment manipulator performs the patch picking operation, and processing the patch deformation characteristics by using a corresponding deformation compensation model according to the main grabbing condition of the patch picking operation to obtain the pose pre-compensation quantity; And the deformation compensation correction module is used for calling the drift prediction model according to the current equipment state, obtaining a drift pre-compensation quantity, fusing the pose pre-compensation quantity and the drift pre-compensation quantity, generating a final compensation control instruction, and driving the manipulator to complete compensation movement according to the instruction. The method