Project facts & technologies
A snapshot of the TubeSight project — entities, technologies, scale, and outcomes — formatted for quick scanning by readers, search engines, and AI answer engines.
- Project name
- TubeSight, heat exchanger tubesheet inspection
- Client
- Confidential, process industry operator
- Provider
- AiSPRY
- Industry
- Metals, Mining & Commodities
- Use case
- Counting and classifying tube openings in heat-exchanger tubesheets; plugging-limit compliance reporting
- Core technology
- RF-DETR Small object detector (31.8M parameters), fine-tuned from pretrained weights
- Detection classes
- Unplugged (open), plugged (blocked), uncertain (needs human review)
- Input
- Photos of the front or rear tubesheet, from thumbnails to roughly 4,000 px
- Image pipeline
- Tube-size estimate, rescale so a tube is about 32 px, overlapping 512x512 tiles, per-tile detection, merge into one result per photo
- Training data
- 448 annotated photos (419 real, 29 AI-generated used for training only), about 75,000 tubes, annotated in Label Studio
- Validation
- Duplicate-aware split; 3-fold cross-validation plus a frozen test set of 95 real photos
- Platform
- Next.js web app, FastAPI backend, PostgreSQL, object storage, JWT sign-in with two-factor and lockout
- MLOps
- MLflow (runs and registry), DVC (dataset versions), Marquez/OpenLineage (lineage), Evidently (drift), Prometheus and Grafana (monitoring)
- Languages
- English and Arabic interface, light and dark themes
- Outputs
- Annotated tube map, per-class counts, plugged % versus allowable limit, PDF/CSV reports
Why does tubesheet inspection matter in process plants?
Shell-and-tube heat exchangers sit at the heart of refining, fertiliser, power and metals processing. Their tubes corrode, foul and leak over time. When a tube leaks, the usual fix is to plug it at both ends. Each plugged tube removes heat-transfer area, so every exchanger has an allowable plugging limit. Beyond that limit, performance drops and a retube or replacement has to be planned.
Knowing how many tubes are plugged is therefore a maintenance and safety decision. During a shutdown the answer comes from a person walking up to the tubesheet and counting openings, often thousands per sheet, under poor light and time pressure. Miscounts mean either unnecessary retubing or running an exchanger past its limit.
What was wrong with manual tubesheet counting?
Key challenges
- Very high tube density. Sheets can hold more than 2,000 tubes, so counting is slow and error-prone.
- Hard imaging conditions. Photos range from small thumbnails to 4,000 px shots, with different distances, angles and lighting.
- Plugged versus open is subtle. Some plugs are flush, dirty or partly visible. Even annotators are unsure about a small share of tubes.
- No consistent record. Counts live in notes and spreadsheets, with no link to the photo, the inspector or the limit that applied.
- Accountability gaps. Nobody can later show who decided a tube was plugged, or which model or rule was used.
- Different readers need different views. An inspector needs the tube map, a manager needs one compliance verdict and an auditor needs the trail.
How does TubeSight work?
Automated tube detection and classification
- The pipeline estimates tube size and rescales the photo, so one tube is about 32 px.
- It cuts the photo into overlapping 512x512 tiles, runs the detector on each, then merges and de-duplicates the boxes.
- Each tube is labelled unplugged, plugged or uncertain, with a per-class confidence threshold tuned on validation data only.
Human-in-the-loop review
- Uncertain and low-confidence tubes go to a reviewer queue.
- A reviewer confirms a tube as open or plugged, and every change is saved and audited.
- Completing the review moves the inspection to the reviewed state.
Compliance reporting
- TubeSight calculates plugged % and compares it with the exchanger's allowable limit.
- A reviewer issues the report. The inspector of that inspection cannot issue it, which keeps the roles separate.
- Each issued report is versioned and immutable, carries a SHA-256 fingerprint and verification code, and records the model version used.
Role-based reports
- Seven roles: inspector, reviewer, planner, client manager, auditor, ML engineer and admin.
- The server decides which reports and fields each role sees. A client manager gets one page with a verdict, and an auditor gets the trail without images.
See TubeSight in action
A 1-minute walkthrough from sign-in to an issued report. The tubesheet is a sample photo, and the detections come from the trained proof-of-concept model.
TubeSight — tubesheet inspection from upload to issued report
- Secure sign-in through the inspector portal, with two-factor and lockout support.
- Upload the front or rear tubesheet photo for an exchanger.
- Processing and results: tubes are detected and shown on the tube map, with filters for unplugged, plugged and uncertain.
- Report and dashboard: open the inspection report, then see the fleet view and the reports library.
What is the architecture of TubeSight?
Photos are uploaded through the web app to the FastAPI backend and stored encrypted in object storage. The inference service tiles each photo, runs the RF-DETR detector and merges the tile results into a single list of tubes, each with a class, a confidence and a position. The results are written to PostgreSQL. The reviewer works on them in the web app, and each override is written to a hash-chained audit log. Issuing a report freezes the numbers, the model version and the limit into a PDF whose fingerprint is stored with it. MLflow, DVC and Marquez record every training run, dataset version and lineage, and Prometheus and Grafana watch the service.
How is TubeSight engineered for real plants?
Honest evaluation
- Near-identical photos are grouped before splitting, so the same sheet never appears in both training and test.
- A test set of 95 real photos is locked away and scored once.
- AI-generated images are used for training only and never for validation or test.
- Thresholds are tuned on validation only.
Data and model governance
- Weights, data and checkpoints stay outside Git, in client-controlled storage.
- Model promotion needs explicit reviewer approval, and the registry records checksums and licence records.
- Drift is monitored, and data lineage is emitted for every dataset preparation.
Security and accountability
- JWT sign-in, two-factor authentication, account lockout and password-strength rules.
- Role-based access enforced on the server, with 404 rather than 403 for out-of-scope records.
- Every generation, export, review override and issue action is audited, and the audit log is hash-chained.
- Rate limiting, security headers, a strict content-security policy and CSV formula-injection protection.
- An information-security document set and a requirements traceability matrix with automated impact analysis in CI.
Practical operation
- Early stopping and per-epoch checkpoints, so a crash resumes from the last finished epoch.
- Training time per epoch was cut about 5x by moving work to the GPU.
- The platform runs in Docker Compose, so it can sit on-premises or in a client-controlled cloud.
What measurable results has TubeSight delivered?
TubeSight is a working proof of concept. All detection figures below are measured on photos the model had not seen, and they are being improved with more labelled data.
Detection performance
- Tube recall: 0.974 in 3-fold cross-validation, 0.959 on the held-out test set of 95 photos.
- Plugged F1: 0.921 in cross-validation, 0.889 on the held-out test set.
- Plugged precision / recall: 0.931 / 0.913 in cross-validation, 0.917 / 0.861 on the held-out test set.
- Unplugged F1: 0.938 on the held-out test set.
- Median plugged-count error per sheet: 0 to 1 in cross-validation, 0 on the held-out test set.
- 1 of 3 folds already reached the plugged F1 target of 0.93, at 0.942.
- The uncertain class is the weak spot, with F1 of about 0.2, because it makes up only about 1% of examples. This is why uncertain tubes go to human review.
- Training for longer or tuning settings plateaued at the same level, so the limit is the amount of labelled data.
Platform outcomes
- Seven role-specific report types, with PDF, CSV and verified issued versions.
- A complete audit trail from upload to issued report.
- Reviewer-in-the-loop workflow: confirm, complete and issue.
- English and Arabic interface.
Next steps
- Retrain on about 230 newly annotated images, which the current model has not seen. The labelled set has grown by about 55% and its classes are now roughly balanced.
- Export to ONNX and deploy as an inference endpoint on the client's devices.
- Move to site scoping and digital signatures for issued reports.
TubeSight: frequently asked questions
What does TubeSight detect?
It finds every tube opening in a tubesheet photo and classifies each one as unplugged, plugged or uncertain. It then reports the counts and the plugged share per exchanger.
How accurate is it?
On a held-out test set of 95 real photos, it found 95.9% of tubes and scored 0.889 F1 on plugged tubes. Cross-validated plugged F1 was 0.921. These results are from a proof of concept, and they are being improved with more labelled data.
What happens with tubes the model is not sure about?
They go to a reviewer queue. A reviewer confirms each as open or plugged, and the change is saved with the reviewer's identity in the audit log.
Can the platform run inside our own network?
Yes. It is containerised with Docker Compose, and its data, weights and checkpoints stay in client-controlled storage.
How is the final report protected against tampering?
An issued report is immutable. It has a version number, a SHA-256 fingerprint and a verification code, and it records the model version and the limit that applied. A change creates a new version and supersedes the old one.