US-12625091-B2 - Systems and methods for automatically generating synthetic x-ray scan data of objects in a plurality of orientations

US12625091B2US 12625091 B2US12625091 B2US 12625091B2US-12625091-B2

Abstract

Systems and methods of automatically generating synthetic X-ray scan data include generating scan data corresponding to a frame holding or supporting an object, wherein the frame and hence the object is manipulated, without manual intervention, to be positioned in a plurality of orientations in three dimensional space. Subsequently, X-ray scan data corresponding to the object is isolated and extracted from the X-ray scan data corresponding to the frame, the X-ray scan data corresponding to the object is adjusted and finally each of the adjusted X-ray scan data corresponding to the object is inserted into X-ray scan data of a cargo container in order to generate a plurality of X-ray scan data of the cargo container embedded with the object.

Inventors

Mark Procter
James Ollier

Assignees

RAPISCAN HOLDINGS, INC.

Dates

Publication Date: 20260512
Application Date: 20240418

Claims (20)

1 . A system for automatically generating a plurality of X-ray scan data of a cargo container embedded with an object, wherein the object is embedded in a plurality of orientations in three dimensional space within the cargo container, and wherein the three dimensional space is defined by first, second and third mutually orthogonal axes, comprising: a frame for holding the object; a base plate for supporting the frame, wherein the frame is positioned in an initial orientation with respect to the first, second and third axes; a first table for supporting the base plate; a second table for supporting the first table, wherein the second table is capable of imparting linear motion to the frame, and wherein the first table is capable of imparting rotational motion to the frame around the first axis independent of the second table; first and second robotic arms and associated cameras configured to locate and rotate the frame around the second and third axes respectively; an X-ray source for generating an X-ray beam that impinges on the frame and a detector array for capturing resultant X-ray scan data; and a computing device having a memory and a processor, wherein the computing device controls movements of the first table, second table and the first and second robotic arms, and wherein the memory stores a plurality of programmatic instructions which when executed cause the processor to: sequentially implement first, second, third, fourth and fifth set of steps in order to generate X-ray scan data corresponding to the frame; isolate and extract X-ray scan data corresponding to the object from the X-ray scan data corresponding to the frame; adjust the X-ray scan data corresponding to the object; and insert each of the adjusted X-ray scan data corresponding to the object into X-ray scan data of the cargo container in order to generate the plurality of X-ray scan data of the cargo container embedded with the object.
2 . The system of claim 1 , wherein the second set of steps is implemented only after completion of the first set of steps, wherein the third set of steps is implemented only after completion of the second set of steps, and wherein the fourth and fifth set of steps are implemented only after completion of the third set of steps.
3 . The system of claim 2 , wherein the first set of steps includes causing the first table to incrementally rotate the frame around the first axis by a predetermined first angle until one full rotation around the first axis is completed, wherein for each unique incremental rotational orientation of the frame around the first axis the second table moves the frame through the X-ray beam in first and second mutually opposing directions in order to generate a pair of scan image data.
4 . The system of claim 3 , wherein the second set of steps includes causing the first robotic arm to incrementally rotate the frame around the second axis by a predetermined second angle until one full rotation around the second axis is completed, wherein for each unique incremental rotational orientation of the frame around the second axis the first set of steps are repeated.
5 . The system of claim 4 , wherein the third set of steps includes causing the second robotic arm to incrementally rotate the frame around the third axis by a predetermined third angle until one full rotation around the third axis is completed, wherein for each unique incremental rotational orientation of the frame around the third axis the first set of steps are repeated.
6 . The system of claim 5 , wherein the fifth set of steps includes causing the first table to incrementally rotate the frame around the second axis by the predetermined second angle until one full rotation around the second axis is completed, wherein for each unique incremental rotational orientation of the frame around the second axis the second table moves the frame through the X-ray beam in first and second mutually opposing directions in order to generate a pair of scan image data.
7 . The system of claim 6 , wherein the fourth set of steps includes causing the second robotic arm to incrementally rotate the frame around the third axis by the predetermined third angle until one full rotation around the third axis is completed, wherein for each unique incremental rotational orientation of the frame around the third axis the fifth set of steps are repeated.
8 . The system of claim 7 , wherein each of the first, second and third angles is the same.
9 . The system of claim 7 , wherein each of the first, second and third angles is 15 degrees.
10 . The system of claim 7 , wherein each of the first, second and third angles ranges from 1 to 90 degrees.
11 . The system of claim 1 , wherein the frame is positioned at a first height of a plurality of predefined heights in order to generate the X-ray scan data corresponding to the frame.
12 . The system of claim 11 , wherein the frame is positioned at a second height of the plurality of predefined heights and the first, second, third, fourth and fifth set of steps are sequentially implemented again in order to generate another set of X-ray scan data corresponding to the frame at the second height.
13 . The system of claim 1 , wherein adjustment of the X-ray scan data corresponding to the object comprises one or more of the introduction of salt and pepper noise to mimic the noise distribution of the X-ray scan data of the cargo container, modulating the intensity level to align with the intensity scaling of the X-ray scan data of the cargo container, dimensional scaling to account for a change in magnification for near and far positions within the X-ray scan data of the cargo container, or ensuring that the X-ray scan data corresponding to the object resides within the boundaries of the cargo container in the X-ray scan data of the cargo container.
14 . The system of claim 1 , wherein a shape of the frame is one of spherical, cubical, regular polygon or a cylindrical tube with or without hemispherical ends.
15 . The system of claim 1 , wherein the frame is made from polystyrene.
16 . The system of claim 1 , wherein each of a plurality of scintillating crystals of the detector array has different vertical and horizontal crystal resolutions.
17 . A system for automatically generating a plurality of X-ray scan data of a cargo container embedded with an object, wherein the object is embedded in a plurality of orientations in three dimensional space within the cargo container, and wherein the three dimensional space is defined by first, second and third mutually orthogonal axes, comprising: a frame for holding the object; a base plate for supporting the frame, wherein the frame is positioned in an initial orientation with respect to the first, second and third axes; a first table for supporting the base plate; a second table for supporting the first table, wherein the second table is capable of imparting linear motion to the frame, and wherein the first table is capable of imparting rotational motion to the frame around the first axis independent of the second table; a robotic arm and associated camera configured to locate and rotate the frame around the second axis; an X-ray source for generating an X-ray beam that impinges on the frame and a detector array for capturing resultant X-ray scan data; and a computing device having a memory and a processor, wherein the computing device controls movements of the first table, second table and the robotic arm, and wherein the memory stores a plurality of programmatic instructions which when executed cause the processor to: sequentially implement first and second set of steps in order to generate X-ray scan data corresponding to the frame; isolate and extract X-ray scan data corresponding to the object from the X-ray scan data corresponding to the frame; adjust the X-ray scan data corresponding to the object; and insert each of the adjusted X-ray scan data corresponding to the object into X-ray scan data of the cargo container in order to generate the plurality of X-ray scan data of the cargo container embedded with the object.
18 . The system of claim 17 , wherein the second set of steps is implemented only after completion of the first set of steps.
19 . The system of claim 18 , wherein the first set of steps includes causing the first table to incrementally rotate the frame around the first axis by a predetermined first angle until one full rotation around the first axis is completed, wherein for each unique incremental rotational orientation of the frame around the first axis the second table moves the frame through the X-ray beam in first and second mutually opposing directions in order to generate a pair of scan image data.
20 . The system of claim 19 , wherein the second set of steps includes causing the robotic arm to incrementally rotate the frame around the second axis by a predetermined second angle until one full rotation around the second axis is completed, wherein for each unique incremental rotational orientation of the frame around the second axis the first set of steps are repeated.

Description

CROSS-REFERENCE The present specification relies on U.S. Patent Provisional Application No. 63/505,670, titled “Systems and Method for Automatically Generating Synthetic X-Ray Scan Data of Objects in a Plurality of Orientations”, filed on Jun. 1, 2023, for priority, which is herein incorporated by reference in its entirety. FIELD The present specification is related generally to the field of X-ray scanning. More specifically, the present specification is related to systems and methods for generating artificial or synthetic X-ray scan data for objects that are automatically manipulated into a plurality of orientations in three-dimensional space. BACKGROUND In recent years there has been an increased need for tools that assist operators in their inspection of X-ray images. This is driven by the demand for higher throughput scanning systems, where the bottleneck in scanning high volumes rapidly is typically due to the image adjudication time. Many tools, such as material classification techniques, already exist for assisting operators. However, such tools are limited in their function, require an operator to manually select buttons to display the results of the tool, and provide no definitive measure of object or material presence within the image. In order to significantly reduce the inspection time of X-ray images, targeted material detection algorithms are required. Such algorithms aim to identify the presence of specific items or groups of items based on targeted item lists. In many cases, while the object to be identified may be a well-known, ubiquitous object, the number of real-world scans that contain examples of the object are low. To compound this problem even further, the relatively little image data representative of this very low number are not usually available for sharing outside of the customs authority where they were captured. In addition, it is well-known that within the field of supervised machine learning, insufficient amounts of training data results in a poor approximation. An over-constrained machine learning model will underfit the relatively small training dataset, whereas an under-constrained machine learning model, in turn, will likely overfit the training data, both of which result in poor performance. Stated differently, a small amount of training or test data will result in an optimistic and high variance estimation of machine learning model performance. In order to overcome the hurdle of insufficient training data, one option is to generate artificial or synthetic X-ray scan data of the objects to be identified. The synthetic X-ray scan data can be used in training machine learning (ML) algorithms in a number of ways, including the use of X-ray scan data as it is captured using a production system, isolation of the particular threat, and injection of a threat item into other stream-of-commerce images for subsequent training. For example, consider a weapon as a threat item. The aim is to arrive at a process that can identify the weapon within an X-ray image of a cargo container. The manifestation of such a weapon within the X-ray image will depend upon several factors, including: i) the output energy of the X-ray source used to generate the image as different X-ray energies result in differing amounts of attenuation and corresponding image pixel intensity; ii) the output dose of the X-ray source used to generate the image; iii) the relative location of the object being scanned to the source and detector array, resulting in different magnification factors and corresponding coverage of the detector array; iv) the presence of occluding materials which impact the spectral composition of the X-ray beam passing through the weapon, which, in turn, impacts the resulting intensity distribution, resolution, and overall appearance of the image; and v) the orientation of the weapon itself, because depending upon the orientation of the weapon, its intensity profile within the X-ray image will vary drastically. In the field of low-energy X-ray baggage scanning, the effort required to build a library of synthetic X-ray scan images of a particular item is time-consuming, albeit achievable given the short scan time, the ability to readily access the machines for high throughput scanning, and the limited number of sizes/orientations/occlusions that an item can experience in small packages. However, this approach is significantly complicated in the field of cargo and vehicle inspection as the possible number of orientations and clutter quantities and material types are vast. In addition, individual threat items may potentially be placed in any orientation relative to the container within which they are located. A manual approach to taking scans, adjusting threat item orientation, and rescanning is prohibitively time consuming and labor intensive further complicated by the impact of exposure restrictions on accessibility to the equipment from one scan to another. Accordingly, there is a need for sys