Comparative object detection system
The same image through two machine learning ecosystems, answered in one format.
- Year
- Context
- Domain
How one request runs
- Entry: React frontend - React
- Image, multipart/form-data
- Orchestration: NestJS gateway - NestJS
- In parallel
- FastAPI service - Ultralytics YOLOv8m - Python, FastAPI
- ASP.NET Core service - ML.NET model - C#, ASP.NET Core
- One JSON shape: boxes, classes, confidence
Demo
Overview
The thesis asks what changes when the same detection task is built twice: once in Python, the ecosystem that research reaches for, and once in .NET, the one that already runs the backend. The system trains a YOLOv8m model and an ML.NET model on the same annotated images, puts both behind one API, and runs an image through either or both.
The domain is personal protective equipment on a work site, over six classes: helmet, vest and gloves, each as present or missing. The negative classes are the point. A system like this is only useful if it can say that someone is not wearing a helmet, not merely that a helmet is somewhere in the frame.
Architecture
The frontend posts the image
React sends the file as multipart/form-data with the chosen model as a second field. Ask for both and it fires the two requests in parallel, so the comparison is over one image at one moment.
The gateway routes it
NestJS is the only address the frontend knows. It takes the upload and forwards it to the FastAPI service or the ASP.NET Core one, which leaves the two ML services free to change independently of the client.
The service runs inference
Python reads the image into RGB and hands it to YOLOv8m, holding the loaded weights in memory so the .pt file is not read again on every request. The .NET side converts to MLImage and selects the ML.NET model by id.
Both answer in the same shape
Class, score, and a box as x1, y1, x2, y2, alongside the model id, the annotation set it was trained on, and the original image dimensions. The .NET side clamps its coordinates to those dimensions first, so a box can never fall outside the image.
The frontend draws it
Because the response is identical either way, there is no YOLO branch and no ML.NET branch in the interface. It scales the coordinates from the original size to the displayed size and paints the boxes, labels and confidences over the image.
What it does
One model or both
Either service on its own, or both at once in two panels over the same image, with the four trained models selectable by name.
A confidence threshold you control
A slider hides everything under the threshold, the way a real deployment acts only on predictions above a cutoff rather than on everything the model returns.
Every detection, listed
Class and score per detection, and selecting one isolates it in the image, which is how you check a frame with eight overlapping boxes in it.
Outlines, fill, zoom and pan
Boxes as outlines or as filled regions, one detection or all of them, with zoom and pan for checking a box against the pixels under it.
The two answers side by side
Request duration, detection count, average confidence, the strongest detection, and which service answered first.
Health checks
Gateway, Python API and .NET API each report status and response time, because three processes fail independently and a blank panel should say which one is down.
Dataset and training
The dataset was assembled in Roboflow out of two public sets, then reviewed and merged into one: 2,911 images split 2,374 for training, 290 for validation and 247 for test, auto-oriented and resized to 640x640. The Python side takes it in YOLO format and the .NET side in COCO, which Model Builder expects for detection, so the two tools get the formats they want over identical images and identical classes.
Then the part I did not plan for. The first pass held 8,813 annotations, and going back through it brought that to 17,942, which means more than half the objects had been unlabelled. An object that is present in the image but missing from the labels teaches the model during training that it is background, and then counts against the model during evaluation when it detects it anyway.
Both models trained for 50 epochs at 640x640 from pretrained weights, with the parameters held steady across the two dataset versions so a difference in results could be read as a difference in annotation quality rather than in configuration. Model Builder stops at the trained model, so the .NET side needed an evaluation service of its own: it loads the COCO annotations, predicts over the validation set, and computes precision, recall, F1 and mAP@0.5 with confusion matrices and precision-recall curves. That gave ML.NET the same output Ultralytics prints by itself.
Results
| Model | Annotations | Precision | Recall | F1 | mAP@0.5 | mAP@0.5:0.95 |
|---|---|---|---|---|---|---|
| YOLOv8m | 8,813 | 0.738 | 0.800 | - | 0.790 | 0.455 |
| YOLOv8m | 17,942 | 0.894 | 0.867 | - | 0.916 | 0.552 |
| ML.NET | 8,813 | 0.669 | 0.645 | 0.657 | 0.580 | - |
| ML.NET | 17,942 | 0.862 | 0.709 | 0.778 | 0.748 | - |
Reannotating moved every metric for both models. YOLOv8m went from 0.790 to 0.916 mAP@0.5 and ML.NET from 0.580 to 0.748, so in relative terms the weaker model gained more, 29 per cent against 16. The first pass had been holding it back hardest.
YOLOv8m is ahead on the numbers, and the gap that matters is recall: 0.867 against 0.709 on the corrected set, at almost the same precision. For protective equipment that asymmetry is the whole story, because a missed detection is a person the system quietly reports as fine, and precision on its own cannot tell you it happened.
The comparison is fair because the setup was the same on both sides: one image domain, the same six classes, and comparable parameters instead of separate tuning for each model.
What I took from it
The outcome sat in the data more than in the choice of framework. Reannotating the same 2,911 images moved both models further than the gap between the two ecosystems set ML.NET back, and that is not the conclusion I expected to be writing up.
The engineering half was more practical. Python gave me room to experiment and produced the evaluation material for free, .NET gave me a model that drops into an ASP.NET Core service with no bridge in between, and the standardized response is the only reason one frontend can treat them as interchangeable.