Search

US-12617079-B2 - Robotics system for verification and validation of smart connected HMI devices and method thereof

US12617079B2US 12617079 B2US12617079 B2US 12617079B2US-12617079-B2

Abstract

Disclosed is a robotic system and method designed for the verification and validation of smart connected Human-Machine Interface (HMI) devices. The robotic system integrates a multi-finger modular robotic module, a haptic module, a finger actuator, vision cameras, vision lights, a finger rotary module, a Cartesian robotic arm, an onboard user interface, and an L-shaped door. The method involves replicating human finger interactions, simulating touch sensations on a device under test (DUT), ensuring continuous and reliable operation, utilizing vision cameras for navigation and verification, optimizing vision with controlled lighting, positioning cameras strategically, executing various tasks with a robotic arm, and facilitating user interaction through an onboard interface.

Inventors

  • Aronin Ponnappan
  • Akhil Asokan
  • Asif Dadamiya Ismail
  • Premkumar Sivankutty
  • Ann Mary Sebastian
  • Rony Augustian

Assignees

  • SGBI Inc.

Dates

Publication Date
20260505
Application Date
20240403

Claims (20)

  1. 1 . A robotic system for verification and validation of one or more smart connected human-machine interface (HMI) devices, comprising: a multi-finger modular robotic module to replicate bare human fingers, and one or more of: human fingers with glove, water, sweat, and impurities; a haptic module to simulate a human finger over a device under test (DUT) by applying a precise force required for a predefined duration and determine the performance of the robotics system based on the precise force and haptic feedback; a finger actuator to provide speed, accuracy, and reliable finger movement throughout a life cycle which makes the robotics system capable of running continuously; a plurality of vision cameras configured for vision-guided navigation and visual verifications; a plurality of vision lights configured for the vision cameras to control the ambient light for better accuracy for a computer vision as well as controllability in a test chamber environmental light; a finger rotary module to rotate the multi-finger modular robotic module to a specific angle and aesthetically position the vision cameras on a specific region for the maximum field of view coverage; a cartesian robotic arm compatible with one or more of the multi-finger modular robotic module, a plurality of pipetting modules, a pick and place module, a vision inspection module, or a combination thereof; an onboard user interface enables monitoring and adjustment of various settings on a test rig; and a L-shaped door to provide wide openings to the robotics system providing an operating access to a user to mount a test device.
  2. 2 . The robotic system as claimed in claim 1 , further comprising a plurality of universal mounting guides for vertical and horizontal cartesian robots mounting without any hardware change or additions.
  3. 3 . The robotic system as claimed in claim 1 , further comprising a finger module cable chain to provide reliability in cable routing and prevent tangling and hanging cable occlusion for the vision cameras.
  4. 4 . The robotic system as claimed in claim 1 , further comprising a homing module to provide a positional accuracy and repeatability for the finger actions.
  5. 5 . The robotic system as claimed in claim 1 , further comprising a status indicator using multicolor light sources positioned on the top of the test rig.
  6. 6 . The robotic system as claimed in claim 1 , further comprising a heavy-duty rig table to provide one or more of: a solid mounting provision for the robot rig; cable additional test support equipment such as API controllable power supply, signal generators, analyzers; a multi-functionality anti-vibrating caster wheel with manual thumb wheel nut open-ended wrench for leveling adjustment and on-premise test system movements; an inbuilt power distribution unit to support any standard power socket along with electrical isolator unit; a wide open front and back door for flexibility of device mounting, organizing, and cable routing; a plurality of customizable cabling racks and grommets; and an adjustable rack-mountable equipment mounting rail.
  7. 7 . The robotic system as claimed in claim 1 , further comprising a plurality of rack mounting rack plates to provide convenient positioning of a plurality of test support devices.
  8. 8 . The robotic system as claimed in claim 1 , further comprising a plurality of multi-functional anti-vibrating caster wheels to provide solid placement of the test device and its manual thumb wheel nut open-ended wrench helps leveling adjustment.
  9. 9 . The robotic system as claimed in claim 1 , further comprising an emergency stop to halt the robotics system manually if any unforeseen issues happen to the DUT of the equipment.
  10. 10 . The robotic system as claimed in claim 1 , further comprising a test device barcode printer to create a unique barcode for each test device while creating the device library using the software suite.
  11. 11 . The robotic system as claimed in claim 1 , further comprising a selective compliance assembly robot arm (SCARA) with a single finger module configured for an HMI testing automation to accommodate the plurality of end effectors.
  12. 12 . The robotic system as claimed in claim 1 , further comprising a plurality of manipulator connectors to connect the robotic manipulator to the test rig.
  13. 13 . The robotic system as claimed in claim 1 , further comprising a SCARA robot with a two-finger module for HMI testing automation to accommodate the plurality end effectors.
  14. 14 . The robotic system as claimed in claim 1 , further comprising a robot height adjustment module to adjust an operating height of the robots with respect to the DUT bed to mount a plurality types of DUTs to the test rig.
  15. 15 . The robotic system as claimed in claim 1 , further comprising a voice assistant test simulator to simulate human kind of voice feedback for voice assistance testing as well as capture voice response from the test device.
  16. 16 . The robotic system as claimed in claim 1 , further comprising an in-built DUT power supply for the DUTs (test device).
  17. 17 . The robotic system as claimed in claim 1 , further comprising a local camera mounting bracket for the SCARA to mount an additional vision camera to monitor the changes happening in the test device.
  18. 18 . A method for verifying and validating smart connected Human-Machine Interface (HMI) devices using a robotic system, the method comprising: replicating human finger interactions by utilizing a multi-finger modular robotic module with variations such as a glove, water, sweat, and impurities; simulating a human finger's interaction with a device under test (DUT) through a haptic module by applying precise force for predefined durations, allowing performance assessment based on the applied force; ensuring continuous and reliable operation of the robotics system by utilizing a finger actuator that delivers speed, accuracy, and dependable finger movements throughout its life cycle; employing a plurality of vision cameras for vision-guided navigation and visual verifications during the verification and validation process; enhancing the accuracy of computer vision through controlled ambient light within the test chamber environment using vision lights; strategically positioning vision cameras by rotating the multi-finger modular robotic module with a finger rotary module to specific angles for optimal field-of-view coverage; executing a plurality of tasks related to the verification and validation process, wherein the plurality of tasks comprising, pipetting, pick and place operations, and vision inspections using a Cartesian robotic arm; and monitoring and adjusting settings on a test rig through an onboard user interface for efficient operation during the verification and validation process.
  19. 19 . The method as claimed in claim 18 , further comprising a step of involving an incorporation of a plurality of universal mounting guides, facilitating the mounting of vertical and horizontal Cartesian robots without necessitating any hardware modifications or additions.
  20. 20 . The method as claimed in claim 18 , further comprising a step of involving an integration of a finger module cable chain, enhancing cable routing reliability and preventing tangling and hanging cable occlusion for vision cameras.

Description

TECHNICAL FIELD The specification is generally directed toward a complete robotics platform for verification and validation of smart connected Human-Machine Interface (HMI) devices. More particularly, but not limited to, a robotics system and method for verification and validation of one or more smart connected human-machine interface (HMI) devices. DESCRIPTION OF THE RELATED ART Companies, particularly in automotive in-vehicle infotainment, consumer electronics (smart appliances, smart meter systems, etc.), Mission-critical medical equipment, Avionics cockpit & inflight entertainment systems, and Smart gadget manufacturers (smartphones, tablets & wearable devices), face significant challenges with product recalls and quality-related lawsuits due to issues in their human-machine interfaces (HMI). These interfaces encompass touch screens, combinations of screens & buttons, knobs, rocker switches, voice interfaces, and hand gesture feedback. The primary reason for these issues is the manual, expensive, ineffective, and error-prone nature of conventional HMI hardware-in-loop testing. According to the World Quality Report by Capgemini, a significant percentage of industries lack suitable automation tools, and skilled labor, and face challenges with manual efforts in testing, contributing to substantial overhead costs. An example of a robotic device is recited in U.S. Pat. No. 9,652,077B2. This patent discloses a touch-screen testing platform may be used to perform repeatable testing of a touch-screen-enabled device using a robotic device tester and a controller. The platform may use various types of conductive tips that engage the touch screen, thereby simulating human behavior. The platform may perform multi-touch operations by employing multiple tips that can engage the touch screen simultaneously. The tips activate a touch screen from at least a trace of conductive coating located on nonconductive components of the robotic device tester. Another example of a touch screen testing platform for use with a robotic device is recited in a patent application US20160320889A1. This patent discloses a touchscreen testing platform may be used to engage a dynamically positioned target feature being displayed on a touchscreen-enabled device during a testing protocol. The platform may record imagery displayed by the touchscreen device and then analyze the imagery to locate the target feature within a reference coordinate system. The platform may recognize that the target feature is missing from the imagery and respond by causing the touch screen device to scroll through a command menu and/or toggle through virtual screens. Once located, the platform may instruct a robotic device tester to select the target feature by contacting the touch screen at the identified location using a conductive tip designed to simulate a user's fingertip. Before running a test, the camera may be focused to a point that is offset from the display screen of the touchscreen device. Various attempts to address these issues have been made, but prior art products have significant drawbacks. There is a lack of standardized equipment, modularity, and platform independence in existing tools, along with shortcomings in smart finger technology, multi-finger modules, robot arm design, safety features, and remote operating capabilities. Furthermore, the market lacks a specialized rig that accommodates larger test devices, provides mechanical mounting capabilities for various robots, and supports a range of functionalities crucial for HMI testing automation. These challenges highlight the need for a robotic system and method to address the shortcomings of current HMI testing methods, ensuring efficient verification and validation of smart connected HMI devices in various industries. Thus, in view of the above, there is a long-felt need in the industry to address the aforementioned deficiencies and inadequacies. Further limitations and disadvantages of conventional approaches will become apparent to one of skill in the art through the comparison of described systems with some aspects of the present disclosure, as set forth in the remainder of the present application and with reference to the drawings. SUMMARY An aspect of the present invention relates to a robotic system and method designed for the verification and validation of smart connected Human-Machine Interface (HMI) devices. The system integrates a multi-finger modular robotic module, a haptic module, a finger actuator, vision cameras, vision lights, a finger rotary module, a Cartesian robotic arm, an onboard user interface, and an L-shaped door. The multi-finger modular robotic module replicates bare human fingers, and one or more of: human fingers with glove, water, sweat, and impurities. The haptic module simulates a human finger over a device under test (DUT) by applying a precise force required for a predefined duration and determining the performance of the DUT based on the precise force. The f