Business Ledger and Agent Interaction Design for 3D Smart Parking

In the previous 3D rendering optimization note, we reduced the campus model footprint and added patrolling vehicles, rotating wheels, and a fixed-distance chase camera across desktop and mobile browsers.

With rendering settled, the next step was connecting concrete parking workflows—visitor bay recommendations, security alert checks and work orders, operator revenue and utilization reports, and natural-language camera/panel coordination—while keeping the 3D canvas fullscreen and reusing the existing Enterprise AI Cockpit backend (Spring Boot + MySQL + pgvector) without deploying a separate agent service.

Desktop scene · Mobile landscape scene · Frontend source · Backend source

Generating a 365-day conservation-checked ledger from public data

If chat answers, dashboard charts, and 3D bay states disagree on basic numbers, users lose trust immediately. Because live campus gate and sensor logs were not available, we extracted daytime occupancy patterns from UCI Parking Birmingham (compiled by Birmingham City Council/NCP and Daniel Stolfi from partial 2016 daytime observations; source hash and license notes are kept in the repository).

After removing invalid capacities and out-of-range counts, weekday and weekend curves were grouped into 15-minute slots and combined with explicit synthetic night curves, seasonal factors, zone weights, and a fixed random seed to produce 365 days from 2025-10-03 through 2026-10-02:

Zone capacities are A=120, B=100, and C=80, obeying strict occupancy conservation across consecutive intervals:

current occupancy = previous occupancy + arrivals - departures
0 <= occupancy <= zone capacity

Arrivals open anonymous stays and departures settle them in integer cents (paid_cents) under a standalone simulated tariff (not a real hospital fee schedule). Revenue reports aggregate settled paid_cents directly in SQL rather than multiplying occupancy by an estimated ticket price:

SELECT COUNT(*) AS settled_stays,
       COALESCE(SUM(paid_cents), 0) AS paid_cents
FROM parking_stays
WHERE dataset_id = 'parking-year-v1'
  AND exited_at >= :from
  AND exited_at < :to_exclusive;

Fixed input hashes, seeds, and date ranges make generation deterministic; database imports back up beforehand, enforce idempotent keys, and verify row counts. Every UI panel labels the synthetic source and sampling timestamp rather than presenting historical values as live sensor feeds.

Enforcing role permissions on backend endpoints

Access control cannot rely on system prompt instructions. Every parking GET and write endpoint verifies a server-signed session principal and rechecks named staff accounts against the database, using BCrypt password hashes and 30-minute sessions:

Modifying role metadata in the browser console does not elevate server permissions.

Routing and recommendations over a versioned POI road graph

Language models do not output raw (x, y, z) coordinates, which easily drift inside buildings or cut through walls. Instead, a versioned POI road graph maps discrete semantic IDs (such as outpatient or zone_c) to validated scene coordinates:

1. Semantic target selection: the model selects stable POI IDs; graph saves check unique IDs, finite coordinates, valid edge endpoints, and optimistic versions.
2. Dijkstra pathfinding: routes run only across open nodes and edges, labeling model-scale distance estimates and returning explicit errors when a path is closed or disconnected.
3. Multi-factor scoring: full, closed, unreachable, or preference-mismatched zones are filtered out first, then scored by walking distance, open bays, charging support (Zone B), and a 10-bay emergency reserve in Zone C (zone-level demo capabilities rather than per-bay sensor feeds).

Human-confirmed work-order workflow

workorder.prepare only creates a draft; staff review the alert, add notes, and explicitly confirm before writing to the database:

Pending review → Operator approval → Security assignment
  → On-site verification → Operator closure

Idempotency keys prevent duplicate creation, open orders on the same alert are reused, and state transitions check row locks and optimistic versions inside transactions. The model handles summarization and drafting, while operational decisions and physical gate controls remain strictly with staff.

Pre-aggregated reports and AG-UI event streaming

Operations views filter by date and zone, showing capacity-weighted utilization, settled stays, and simulated receipts, with full daily CSV export alongside an 18-row table preview.

Day, week, month, and year summaries are anchored to the dataset’s latest date and pre-aggregated by the cockpit process at 02:15 Asia/Shanghai via SQL without model calls.

For chat-to-scene coordination, /api/parking/ag-ui streams standard SSE lifecycle, text, tool-call, and state events, carrying reports, routes, drafts, and citations in named CUSTOM events:

{
  "type": "TOOL_CALL_ARGS",
  "toolCallId": "action-0",
  "delta": "{\"target\":\"outpatient\"}"
}

The frontend validates every event with @ag-ui/core schemas before running its local tool allowlist, disables Nginx SSE buffering, halts pending camera actions on cancellation, and preserves multiple report cards within a single reply.

Voice interruption and 3D canvas snapshot Q&A

1. Voice wake word and interruption: single-shot, wake-word, and continuous modes share the same role-bound tools via browser SpeechRecognition and speechSynthesis. Saying the wake phrase interrupts active narration, echo matching reduces self-triggering, and the microphone defaults to off and stops when backgrounded.
2. Canvas snapshot inspection: capturing a visual query previews only the 3D WebGL canvas (excluding chat and account UI) for staff confirmation, validates JPEG format and size limits on the backend, and calls vision-enabled deepseek-flash to describe visible road and bay conditions without triggering scene mutations or storing images.

Closing reflections

From lightweight 3D rendering to integrating with the Enterprise AI Cockpit backend, one engineering thread runs through the project: a 3D digital twin needs consistent, verifiable operational data beneath it, and an AI agent needs explicit topology constraints, server-side permission boundaries, and structured UI protocols to participate safely in real workflows. Keeping conservation-checked data, role checks, road topology, and AG-UI events cleanly separated also makes it straightforward to swap in real IoT adapters, surveyed maps, and hospital tariffs later without rewriting the workflow layer.

Explore the live desktop and mobile landscape scenes at 3D Smart Parking.

References

• UCI Parking Birmingham dataset and license
• Dataset DOI
• AG-UI event specification
• DeepSeek vision documentation