CoreML matting vs Vision person segmentation: which one for a live camera?

Short version: start with Vision, measure, and only ship your own model if hair edges are actually the thing users notice.

Vision VNGeneratePersonSegmentationRequest

• Zero model shipping, Apple maintains it, runs on the Neural Engine.
• Three quality levels; .balanced comfortably does 1080p at 60 fps on M2, .accurate is more like 30 fps.
• Output is a mask, not an alpha matte: hard-ish edges, hair becomes a blob. You can soften with a small guided filter pass in Metal, which gets you 80% of the way for talking-head shots.
• Also has VNGeneratePersonInstanceMaskRequest if you need multiple people separated.

Your own matting model (e.g. an RVM/MODNet-style network converted to CoreML)

• Produces a real alpha matte: individual hair strands, semi-transparent edges, glasses frames.
• You own the performance work: 1080p at 60 fps means running the network at a lower resolution (e.g. 512 px on the long side) and upsampling the alpha with the guided filter — that’s how the video matting papers hit real-time too.
• Temporal stability is the hidden problem: per-frame models flicker at the edges; recurrent ones (RVM) keep state between frames and look far calmer. Budget time for this.
• Model size adds 10–30 MB to the app and you maintain conversions across OS updates.

Practical path

1. Ship Vision .balanced + Metal guided filter. Measure GPU time per frame; you’ll likely be at 4–6 ms.
2. Record 20 clips of real users (different hair, lighting, backgrounds). If the mask edges are the top complaint, then
3. Add a matting model behind a “High quality edges” toggle so the safe path still exists on lower-end machines.

One trick either way: composite in linear light, not sRGB, and pre-multiply alpha before blending. Half of “bad hair edges” complaints are actually gamma-space blending halos.