Custom Weights Guide

How to use Ollama vision as an add-on, and how to train or plug in your own detector weights for this project.
Settings Custom Weights Guide

Short Answer: Can Ollama Vision Models Be Used?

Yes, but as an add-on layer, not as the main detector. This app's video pipeline is built around Ultralytics object detection, which is fast enough to scan full videos frame-by-frame. Ollama vision models are better for reasoning over selected frames or short clips.

Good add-on uses: reviewing candidate events, summarizing a clip, comparing reruns, explaining why a possession was tagged a certain way, or labeling a handful of already-detected moments. Poor fit: scanning an entire game video frame-by-frame as the primary detector.

Recommended Ollama Vision Use Cases

Good Fit
Clip explanation
Event verification
Human-readable summaries
Side-by-side rerun comparisons
Not a Good Fit
Full-game detection pass
High-FPS object tracking
Replacing YOLO for bounding boxes
Real-time frame-by-frame inference

Examples of vision-capable Ollama families to evaluate for add-on workflows include models like llava, qwen2.5vl, or minicpm-v when they are available in your Ollama environment. Treat them as reviewers or describers, not as the detector that populates detections.

Important Limitation in the Current App

The current analyzer only persists person and ball detections into the database. If you train a custom model with extra classes like rim, referee, made_shot, or screen, those classes will not automatically show up in the app until the analyzer/event pipeline is expanded to store and use them.

So today, the safest custom-weight strategy is to train a model that is still excellent at person and ball detection in your gym and camera angles. That alone can improve downstream tracking and event heuristics.

How to Create Your Own Custom Weights

  1. Choose the exact classes you want. For the current app, start with person and ball. Add more classes only if you also plan to expand the code that consumes them.
  2. Collect representative footage. Use clips from your actual gym, scoreboard lighting, camera height, sideline angle, uniforms, and ball color. Variety matters more than total file count.
  3. Extract frames. Pull frames from games, practices, crowded possessions, fast breaks, rebounds, and bad lighting. Include misses, occlusion, and empty frames so the model learns what not to detect.
  4. Annotate in YOLO format. Tools like CVAT, Label Studio, or Roboflow can export YOLO detection labels. Keep labeling rules consistent, especially for the ball when it is partially occluded.
  5. Split the dataset. Create train, validation, and test sets. Keep some games fully out of training so you can tell whether the model generalizes.
  6. Create a dataset.yaml. Point it at your train/val folders and define the class names in the exact order used by the labels.
  7. Train from a reasonable base model. Start from a small or medium detector close to your hardware budget, then compare.
  8. Review the validation output. Look at missed balls, false positives on shoes or logos, and whether player boxes stay stable when defenders overlap.
  9. Export the best weights. Ultralytics training typically writes a best.pt file under the run directory.
  10. Load that weight file into this app. In Settings, choose Custom Ultralytics weights and set the custom model field to the local .pt path or a supported model name.
  11. Rerun AI and compare. Use the compare workflow on the same video to measure whether your custom detector produced better detections or downstream events.

Example Training Command

yolo detect train data=/path/to/dataset.yaml model=yolo11s.pt imgsz=1280 epochs=100 batch=8 device=0 project=training_runs name=liberty-ball-player

If your GPU is small, reduce imgsz or batch size, or start from yolo11n.pt. If you are CPU-only, training will be much slower; consider training elsewhere and copying the resulting best.pt onto this machine.

Suggested Dataset Structure

dataset/
  images/train/
  images/val/
  images/test/
  labels/train/
  labels/val/
  labels/test/
  dataset.yaml

Example dataset.yaml

path: /path/to/dataset
train: images/train
val: images/val
test: images/test

names:
  0: person
  1: ball

What Usually Improves Results the Most

  1. More examples from your real gym and camera angle
  2. Cleaner ball labels during occlusion and motion blur
  3. Frames from difficult possessions, not just easy ones
  4. Comparing small vs medium models on the same uploaded video
  5. Using the compare page to test changes one rerun at a time

Practical Workflow for This Project

  1. Start with yolo11n.pt or yolo11s.pt in Settings.
  2. If stock weights still miss the ball or merge players too often, train a custom detector focused on your footage.
  3. Load the resulting best.pt through the custom detector field.
  4. Rerun the same uploaded video and compare counts.
  5. If you want richer labels later, add Ollama vision as a second-stage reviewer on top of detections rather than replacing the detector.