Search

CN-122002008-A - Inspection bicycle system based on multi-sensor fusion

CN122002008ACN 122002008 ACN122002008 ACN 122002008ACN-122002008-A

Abstract

The invention discloses a multi-sensor fusion-based inspection bicycle system and a multi-sensor fusion-based inspection bicycle method, and relates to the technical field of smart city sensing equipment and city management informatization. The system comprises a mobile acquisition module, a data processing and event judging module and an event grading and responding module, wherein the mobile acquisition module integrates an image acquisition device, an ultrahigh frequency RFID reader-writer, a millimeter wave radar and a vehicle-mounted control unit, the mobile acquisition module is mounted on a bicycle platform to realize multi-mode data synchronous acquisition, the vehicle-mounted control unit guarantees that the time stamp deviation of three types of signals is less than or equal to 100 mu s through a PTP precise time service module, and the data is uploaded to a cloud through a wireless communication module. The data processing and event judging module executes fourth-order fusion processing, comprehensive confidence coefficient is calculated according to the weight of 35%, 40% and 25%, a structured city management event is generated, and the event grading and responding module distributes responding level according to the confidence coefficient and triggers corresponding disposal flow. The invention solves the problems of insufficient coverage, low recognition precision, weak rights association and the like of the existing inspection scheme, has the advantages of high mobility, high reliability and full-flow automation, can be widely applied to the scenes of municipal facility inspection, shared bicycle standard management and the like, and greatly improves the urban management refinement and intelligence level.

Inventors

  • MEI HAO
  • DU YIFENG
  • BI DING

Assignees

  • 浙江本源质品信息科技股份有限公司

Dates

Publication Date
20260508
Application Date
20260127

Claims (8)

  1. 1. The system is characterized by comprising a mobile acquisition module, a data processing and event judging module and an event grading and responding module; The mobile acquisition module synchronously acquires multi-mode data through a plurality of sensors, and comprises an image acquisition device, an ultrahigh frequency RFID reader-writer, a radar and a vehicle-mounted control unit, and is arranged on a bicycle; The image acquisition device is fixedly arranged at the front part of the bicycle body and is used for acquiring visual data in a forward or lateral environment; the RFID reader is fixedly arranged at the front part or the middle part of the bicycle body and connected with a high-frequency or ultra-high-frequency antenna to read RFID tag information deployed on urban vehicles or public facilities; the radar is fixedly arranged at the front part of the bicycle body, is adjacent to the space position of the image acquisition device, jointly covers a detection area in front, and measures the distance, speed and angle information of a target; the vehicle-mounted control unit is in communication connection with the image acquisition device, the RFID reader-writer and the radar, and synchronously receives and temporarily stores the visual data, the RFID tag information and the radar data; the event grading and response module is used for distributing response grades to the urban management events according to preset rules and automatically triggering the disposal flow corresponding to the grades.
  2. 2. The system according to claim 1, wherein the RFID reader is an ultra-high frequency reader, and the radio frequency field configured by the RFID reader covers a main field area of the image acquisition device, and RFID tag information in the field of view is read in batch during the moving process; the transverse deviation between the optical center of the image acquisition device and the millimeter wave radar and the beam center is less than or equal to 2 cm, the pitch angle deviation is less than or equal to 0.5 degrees, and the engineering realization foundation with the field of view overlapping degree of more than or equal to 90 percent is ensured.
  3. 3. The system according to claim 1, wherein the vehicle-mounted control unit is internally provided with a PTP precision time service module for synchronizing an image acquisition frame start signal, an RFID command trigger signal and a radar sampling trigger signal, and the deviation of the time stamps of the three signals is less than or equal to 100 μs.
  4. 4. The system of claim 1, wherein the ultra-high frequency RFID reader has an operating frequency range of 902-928 MHz, and an effective reading radius of the radio frequency field is 3.0-3.5 m in a moving state, so as to satisfy batch reading of tags in a main field of view of an image; The millimeter wave radar has a working frequency range of 76-81 GHz, a distance precision of +/-0.1 m and an angle resolution of +/-1 DEG, and the output frame rate of point cloud is more than or equal to 10 Hz, so that continuous tracking of a dynamic target is supported.
  5. 5. The system of claim 1, wherein the vehicle control unit and the data processing and event determination module interact with each other via a wireless communication network, and wherein the data processing and event determination module and the event ranking and response module are deployed on a remote server or cloud platform.
  6. 6. The system of claim 1, wherein the data processing and event determination module is disposed on a remote server or cloud platform, and is configured to receive the synchronized visual data, RFID tag information, and radar data, and perform the following fourth-order fusion process: The first step is to take an image recognition result as a trigger, extract pixel coordinates, sizes, color histograms and YOLOv s confidence coefficients of potential event targets; calling an RFID-ID mapping database, and returning the EPC code, the unit of the EPC code, the maintenance contact person and other management attributes according to pixel coordinate matching to realize form-identity semantic association; Thirdly, accessing target distance, speed and azimuth data measured by the millimeter wave radar, and performing state verification on an image-RFID matching result; And the fourth step of calculating comprehensive confidence coefficient according to a weighted mode, wherein the image attribute contribution weight is 35%, the RFID matching degree contribution weight is 40%, the radar state consistency contribution weight is 25%, the total confidence coefficient is used as a core field of a structured city management event, and the event grading and response module is deployed on a remote server or a cloud platform and is used for distributing one-level to four-level response grades according to the comprehensive confidence coefficient and event type and automatically triggering corresponding disposal flows.
  7. 7. The system of claim 1, wherein the data processing and event determination module, when performing fusion analysis, identifies a potential event object and its image attributes from the visual data, obtains identification information or management attributes of the potential event object from the RFID information, and matches and associates the image attributes with the identification information or management attributes to generate a structured urban management event.
  8. 8. The method for inspecting the bicycle based on the multi-sensor fusion is characterized by comprising the following steps of: s1, acquiring a video stream through an image acquisition device, identifying a potential event target through a YOLOv S model, and extracting pixel coordinates, image attributes and YOLOv S confidence coefficient; s2, controlling the ultrahigh frequency RFID reader-writer to directionally scan the coordinate area obtained in the S1, and attempting to acquire RFID tag information associated with the target; s3, synchronously acquiring distance measurement, speed measurement and azimuth angle data of the millimeter wave radar to the target; S4, performing fourth-order fusion: (a) Taking an image recognition result as a first-order trigger; (b) Taking the RFID matching result as a second order responsibilities; (c) Taking radar state data as third-order verification; (d) Calculating comprehensive confidence coefficient according to the weight of 35% +40% +25%, and generating a structured city management event; s5, distributing response levels by an event grading and response module according to the confidence coefficient and the event type obtained in the S4; And S6, automatically executing a corresponding treatment flow, including short message pushing, voice reminding, worksheet dispatching, GIS thermodynamic diagram updating or archiving and checking.

Description

Inspection bicycle system based on multi-sensor fusion Technical Field The invention relates to the technical field of smart city sensing equipment and city management informatization, in particular to a lightweight inspection bicycle system and method based on image, ultrahigh frequency RFID and millimeter wave radar multi-sensor fusion. Background With the continuous deep urban process in China, the urban scale is continuously enlarged, the system is increasingly complex, and the unprecedented higher requirements on the levels of refinement, intellectualization and normalization of urban operation management are provided. The effectiveness of urban operation management inspection relates to the health safety and orderly operation of cities. However, the traditional manual inspection mode faces serious challenges, on one hand, the management elements are numerous and complicated, the management elements are from pavement manhole covers to aerial cables, the urban order is safe to facilities, the contradiction between heavy tasks and limited human resources is prominent, inspection blind areas are unavoidable, on the other hand, the repeated problems repeatedly occur, and the fundamental transition from passive treatment to active prevention is difficult to realize. The existing intelligent inspection equipment has poor environmental adaptability and extensive bottleneck for event judgment. Patent CN110378352a discloses a four-wheel electric inspection vehicle carrying a high-definition camera and a GPU edge computing unit, and targets such as well lid missing, road surface damage, illegal advertisements and the like are identified through a YOLOv5 model. However, the patent relies on a single visual mode, the identification accuracy rate suddenly drops to below 62% under the conditions of rain fog, night, strong reflection or shielding such as green belts and parked vehicles, and the unique identity and the ownership information of the target cannot be obtained, only a broken well cover at a certain position can be output, and the unique identity and ownership information cannot be associated to specific road section numbers, property units and maintenance responsibilities, so that the event work order distribution is delayed and the responsibilities are unclear. The four-wheel vehicle has large volume and turning radius of more than 3.5m, is difficult to enter old lanes and non-motor lanes with the width of less than 2.5 m, and has coverage rate of less than 45 percent of the total length of the urban road network. Patent CN109886242B proposes to deploy UHF RFID tags on facilities such as bus stop boards, garbage cans, fire hydrants, etc., and patrol personnel scan along a route by holding a handheld reader-writer, and acquire the facility status by matching tag IDs with a background database. The inspection mode completely depends on RFID, when the tag is stained, shielded or fallen, the system cannot judge whether a physical facility still exists at the position, the inspection omission is easy to occur, and the situation that the facility is still intact and temporarily shielded by a vehicle cannot be distinguished due to lack of target positioning and motion state sensing. The prior art does not solve the core problem of how to synchronously acquire the three-dimensional information of the urban management target and automatically generate the executable, traceable and hierarchical structured event worksheets by a low-cost and high-mobility platform Disclosure of Invention The invention aims to provide a bicycle inspection system and method with reliable identification, accurate judgment and intelligent response. To solve the problems set forth in the background art. The invention aims at realizing the technical scheme that the multi-sensor fusion-based inspection bicycle system is characterized by comprising a mobile acquisition module, a data processing and event judging module and an event grading and responding module; The mobile acquisition module synchronously acquires multi-mode data through a plurality of sensors, and comprises an image acquisition device, an ultrahigh frequency RFID reader-writer, a radar and a vehicle-mounted control unit, and is arranged on a bicycle; the image acquisition device is fixedly arranged at the front part of the bicycle body and is used for acquiring visual data in the front or side environment; the RFID reader is fixedly arranged at the front part or the middle part of the bicycle body and connected with a high-frequency or ultrahigh-frequency antenna to read RFID tag information deployed on urban vehicles or public facilities; the radar is fixedly arranged at the front part of the bicycle body, is adjacent to the space position of the image acquisition device, jointly covers a detection area in front, and measures the distance, speed and angle information of a target; the vehicle-mounted control unit is in communication connection with the image acquisition de