US-12621414-B2 - Identifying, recording and assessing geospatial assets with data collection, image processing, and selective pruning

US12621414B2US 12621414 B2US12621414 B2US 12621414B2US-12621414-B2

Abstract

The present invention is intended to provide enhanced asset data collection functions, asset analysis, and/or asset alerts. The system assists in updating existing asset records, adding new assets to asset inventories, identifying maintenance issues, and/or identifying assets which are no longer present. The system is composed of a data collection device(s) which can collect images and location sensor data, server(s), and client interface(s) for interacting with the collected and/or processed data. The system also includes image processing operations and data pruning and selection functions for the smart asset data collection and processing, allowing to obtain the appropriate data for the appropriate asset at the appropriate time.

Inventors

Royi Tal
Thomas Bakonyi
Redenthor Ibana
Omri Artman

Assignees

VISUAL DEFENCE INC.

Dates

Publication Date: 20260505
Application Date: 20230911

Claims (20)

1 . A system for optimized collection and/or processing of asset data to generate selected data, and for transmitting the selected data over a communications network to one or more servers, the one or more servers located remotely from one or more data collection devices, the system comprising: the one or more collection devices mounted in one or more vehicles and having: at least one camera for obtaining digital images including image data having at least a portion of the asset data; one or more sensors for collecting sensor data, including at least one sensor for acquiring geolocation information associated with the digital images; at least one working memory and at least one processor for executing data collection and/or processing instructions, the data collection instructions defining operation of the at least one camera and the one or more sensors in order to generate collected data including the image data and the sensor data for the asset data; storage media for storage of the asset data including temporary storage of collected and/or processed portions of the asset data prior to sending the selected data to the one or more servers over the communications network; and a network communications interface for sending the selected data to the one or more servers over the communications network, and for receiving collection and/or processing instructions and updates from the one or more servers over the communications network; and at least one asset database representing the asset data of one or more assets and their geolocation, the database working in conjunction with at least one image processing function and at least one data pruning function; and at least one image processing function for generating derived data from the digital images, the at least one image processing function operating on the one or more data collection devices and/or the one or more servers; and at least one data pruning function for optimizing an amount of at least one of the image data, the sensor data and the derived data; the least one data pruning function operating on the one or more data collection devices and/or the one or more servers.
2 . The system of claim 1 wherein the communications network includes the internet.
3 . The system of claim 1 wherein the system collects the image data for the one or more assets from one or more perspectives.
4 . The system of claim 1 wherein the system can be operated autonomously, semi-autonomously and/or manually.
5 . The system of claim 1 wherein the database is synchronized with one or more 3 rd party asset databases.
6 . The system of claim 1 wherein at least some of the collected data is matched to a particular asset in the database using one or more matching algorithms.
7 . The system of claim 1 wherein the system supports collection of the asset data for one or more of the following asset types selected from a group; the group including at least one of the following asset types: (a) pavement markers, vertical markers, longitudinal markers, horizontal markers, transversal markers, lane markers, vertical lines, horizontal lines, zig zagged lines, crossed lines, solid lines, patterned lines, dashed lines, double lines, multiple lines, lines of different thickness, lines of different colors, bars, crosswalks, arrows (left, right, forward, backwards, multi-arrow), numbers, letters, line drawings (handicap, family, expectant mother), and any other pavement marking of any shape, pattern, color or combination, whether regulatory, warning, informative or decorative; (b) fire hydrants, manholes, grates, basins, pipe access points, access ports, drains, or any other surface based points that provide ingress, egress, and/or access for water, equipment or people to an underground water system, whether for multi-purpose, combined purpose or specific purpose; (c) Trees, gardens, planters, shrubbery, greenery, flowers beds, and otherwise planned, planted vegetation considered an asset under the responsibility of an entity, whether planted in ground or in planters; (d) highways, roads, rails, guardrails, hand rails, tramlines, lanes, trails, bicycle lanes, parking lanes, parking spots, lots, intersections, sidewalks, pads, walkways, trails, easements, right-of-way, and/or other paths (including concrete, asphalt, gravel, aggregate, composite, dirt, or other material and/or a combination thereof) meant to provide a surface for movement of vehicles (of one or more types), equipment, and/or people; (e) curbs, bridges, bridge decks, bridge joists, columns, retaining walls, spalls, tracks, bars, bolts, nuts, panels, cables, wires, anchors, frames, joints, or other road, bridge and/or rail infrastructure, and/or related items thereof; (f) traffic control devices, such as signs (such as regulatory, warning, construction, temporary, informational, or other signs), traffic lights, signaling lights, pedestrian warning systems, and/or other traffic control devices; (g) barriers, barricades, gates, guiderails, fences, bollards, cones, pylons, rocks, and/or other assets meant to restrict or redirect traffic of vehicles (of one or more types) and/or people; (h) various types of poles, such as sign poles, lighting poles, electrical poles, phone poles, telecommunication poles, utility poles, multi-purpose poles, load bearing poles, and/or other poles as well as transformers, boxes, wires, and/or accessories related thereof; (i) cabinets of one or more types, such as cabinets for power distribution, electrical distribution, utility, telecomm equipment, computing equipment, network equipment, communications equipment, control equipment, phone systems, signaling equipment, multi-purpose cabinets, and/or cabinets for other purposes; (j) transit related asset, such as a transit stop, transit station, transit pole, transit stop sign, bench, bollard, ashtray, clock, digital displays, advertising display, information post, stands, ticketing systems, lighting, glass panels, maps, news stands, payphones, advertising surface, bike rack, shelter, solar power source, washroom, garbage bin, and/or other transit assets and/or components thereof; (k) asset fixtures, such as brackets, anchors, clamps, mounting plates, fasteners, hooks, adhesives, hangers, clips, straps, pins, braces, standoffs, angles, bolts, nuts and/or other fixtures and/or components thereof; and/or (l) other assets, such as mailboxes, donation bins, public waste bins, playgrounds, ramps, water fountains, statues, decorations, structures, electronic devices, charging stations, parking meters, parking spots (whether dedicated or time limited), meters, pipes, lawns, parks, shuttles, other vehicles, and/or components thereof.
8 . The system of claim 1 wherein the at least one image processing function generates the derived data for the image data of the digital images.
9 . The system of claim 8 wherein the at least one image processing function is performed on the data collection device and at least some of the derived data is sent to the one or more servers as the selected data, the selected data excluding the image data used to generate the derived data.
10 . The system of claim 8 wherein the at least one image processing function includes at least one neural network.
11 . The system of claim 8 wherein the derived data confirms the presence of one or more assets of the asset data in one or more images of the image data.
12 . The system of claim 8 wherein the derived data includes visual assessment results of a condition of one or more assets of the asset data in one or more images of the image data.
13 . The system of claim 8 wherein the derived data includes identification of one or more issues in the one or more images of the image data.
14 . The system of claim 13 wherein the system initiates one or more actions selected from a group when an issue is identified; the group includes at least one of the following actions: (a) generate notification; (b) send SMS message; (c) initiate alert; (d) send email; (f) create service request; (g) create work order; (h) generate report; (i) initiate integration task; (j) assign to user; (k) display image; (l) display text; (m) play sound; (n) offer recommendation; (o) populate interface component; (p) generate file; (q) update database tables; (r) update views; (s) update corresponding record; and/or (t) related actions thereof.
15 . The system of claim 8 wherein the at least one image processing function redacts personal information determined in the image data.
16 . The system of claim 8 wherein the asset inspection issues include issues selected from a group; the group includes at least one of the following issues: (a) presence issues, missing assets, misplaced assets, and/or assets located elsewhere; (b) structural issues or physical damage issues, whereas the asset or one or more of its components is broken, shattered, chipped, bent, cracked, deformed, punctured, disintegrated, sheared, cut, rutted, warped, bleeding, heaved, depressed, mangled, twisted, shattered, smashed, shot or otherwise physically damaged; (c) orientation or continuity issues, whereas the asset or one or more of its components are leaning, fallen over, twisted, warped, uneven, mis-aligned, non-aligned, facing the wrong way, rotated, tilted, flipped, sunken, raised, recessed, protruding, dropped-off, or otherwise not positioned properly; (d) visibility, obstruction and/or occlusion issues, whether in full or in part, whereas the asset or one or more components of the asset are being covered, obscured, obstructed, blocked, clogged, and/or occluded, including by one or more objects, resulting in lack of visibility, and/or physical obstruction; (e) use, wear, tear and exposure issues, including issues encountered due to asset's use, or asset's exposure to the environment or weather, whereas the asset or one or more of its components can experience failures, whether mechanical, electrical, material, degradation, corrosion, oxidization, decomposition, rusting, erosion, tarnishing, discoloration, change of color, changes in texture, staining, chips, cracks, swelling, peeling of paint or coatings, pitting, leaking, flaking, bubbling, fading, failure of a mechanical component, and/or other such failures, whether absolute, or partial; (g) vandalism issues, graffiti, paint, stickers, drawings, dents, scratches, etchings, stencils, adhesives, splattering, breaking, smashing, or other vandalizing, defacing and/or other damaging actions; (h) day and/or night visibility issues, where an asset or one or more components of the asset are not adequately visible at day or at night, dimming, flickering, broken lighting, poor material reflectiveness, poor retro-reflectivity score, fade of material, wear and tear of material, orientation, and/or other to electrical, mechanical or material malfunction which compromise the asset's visibility at day and/or at night; (i) capacity issues, over-filling, under-filling issues, whereas the asset or one or more of its components, are used beyond the intended capacity or under the intended capacity, (j) accessibility issues, whereas the asset or one or more of its components, is inaccessible due to environmental conditions, manmade obstructions, asset failures and/or a combination thereof; (k) chemical, biological, environmental, toxic and/or hazardous issues in proximity or in contact with the asset or one or more of its components, dead animals, carcasses, excrement, urine, invasive plant species, dangerous animals, infestation, fungi, blight, garbage piles, used needles, used contraception products, used personal hygiene products, medical waste, chemical waste, radioactive materials, food waste, spills, dumping, broken glass, exposed wires, trip edges, and/or other such products or use cases that can deter use or be hazardous to people, animals, plants and/or the environment; (l) cleanliness issues, in which an asset or one or more of its components are dirty due to accumulation of dust, pollution particles, salt, or other particulates which coat the asset or otherwise form film or coating, whether organic or inorganic, whether human originated or otherwise; (m) compliance issues, where an asset or one or more components of the asset fail to comply with a standard, regulation, policy and/or best practices, accessibility issue, safety issue, maintenance issue, and/or asset management issue, illegal parking, encroachment issues, excessive vegetation growth, encampments, and/or other compliance issues; and/or (n) abnormal status issues, whereas an asset is in an abnormal state.
17 . The system of claim 1 wherein the least one data pruning function includes a pruning operation selected from a group of operations; the group of operations includes at least one of the following operations: discarding of data, grouping of data, matching of data to asset, exclusion of data, inclusion of data, extraction of data, throttling of data, compression of data, retention of data, deletion of data, prioritizing of data, assembling of data, verification of data, requesting of data, selection of data, and/or user review of data.
18 . The system of claim 1 wherein at least some of the selected data is available to a user through a client interface.
19 . The system of claim 1 wherein the vehicle is a public transit vehicle, and assets of the asset data are transit stops; the transit stops including one or more asset types.
20 . The system of claim 19 wherein the system visually inspects the transit stops assets for issues.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a C-I-P application of U.S. patent application Ser. No. 17/853,918 filed on Jun. 30, 2022; which is a C-I-P application of U.S. patent application Ser. No. 17/559,315 filed on Dec. 22, 2021; which is a C-I-P application of U.S. patent application Ser. No. 16/930,070 filed Jul. 15, 2020; This application is a C-I-P application of U.S. patent application Ser. No. 17/559,315 filed on Dec. 22, 2021; which is a C-I-P application of U.S. patent application Ser. No. 16/930,070 filed Jul. 15, 2020; This application is a C-I-P application of U.S. patent application Ser. No. 17/373,514 filed on Jul. 12, 2021; which is a C-I-P application of U.S. patent application Ser. No. 16/930,070 filed Jul. 15, 2020; and This application is a C-I-P application of U.S. patent application Ser. No. 17/534,816 filed on Nov. 24, 2021, the contents of which, all of the above, are hereby incorporated by reference. FIELD The present invention is related to asset management and maintenance for automating aspects of identification, recording, monitoring and assessing the condition of infrastructure and assets. BACKGROUND The management and maintenance of infrastructure, including road and transit assets is key to safety for commuters, vehicle owners and pedestrians. Well maintained road and transit infrastructure reduce risks and liabilities and assist in the efficient flow of people and goods. In the long term an effective asset management program can reduce cost, increase efficiency and improve organizational performance. Additionally, asset management programs will generate information about assets which can be used for more strategic decision making. Asset Management refers to all activities that are undertaken within an asset's lifecycle from development and procurement to disposal. Asset management activities are undertaken by various parts of an organization for example finance, logistics, maintenance and repair, customer service and reporting. The involvement of many different business functions makes the asset management process onerous and challenging. Furthermore, the challenges faced by asset managers is knowing the condition and location of assets in real time. Failure to detect missing, damaged or obstructed assets can create hazardous situations for road users and increase an organization's liability. It can also provide for unsightly, unpleasant or otherwise discouraging user experience to commuters. The public can therefore, report and complain about issues they encounter in relation to the assets. In the event of a property damage, injury or death, it can also result in claims and/or litigation. As computing technologies become easier to make, cheaper to produce and more available, they inevitably become incorporated into everyday applications. One such application is connected and/or autonomous vehicle application. With respect to vehicles, more sensors are continuously added to vehicles that improve the performance, comfort and safety of vehicles for its drivers. Many cars are currently equipped with rear cameras to assist drivers in reversing and the installation of stock forward-facing cameras or third-party dashboard cameras is increasing. Early warning systems such as collision alert which can use a camera, radar or laser sensor that make the vehicle aware of its surroundings. Public transit vehicles can be equipped with CCTV systems for security and risk management functions. Sensors are already present in vehicles that can identify, detect road conditions, markers signs and other right of way objects, for example, for collision avoidance, lane keeping or autonomous or semi-autonomous driving. An organization such as a town, municipality, city, transit authority, asset operator or other private or governmental entities may be responsible for or have ownership over hundreds, thousands, or many thousands (or more) of assets which can include, one or more categories, for example roads (or road sections), manhole covers, signs, fire hydrants, trees, transit stops, concrete pads and other assets related to roads, accessible by road, or visible from road. The organization may be required to demonstrate their adherence to maintenance standards or its own internal maintenance guidelines. The responsible organization may performs manually scheduled inspections by trained staff who travel to inspect the asset(s) and document the asset(s) condition. When asset(s) are found to be in noncompliance or defective, the inspectors escalate the issue to the management or maintenance team. A work crew then performs the repair or replacement and on completion, create a report or mark the issue as closed. Factors such as the availability of money, staff, equipment/material, the asset importance and its conditions, and repair/maintenance priority are considered when planning and scheduling maintenance and repair exercises. Inspectors can reach assets by vehicle. They can also wa