Repository navigation
Requesting for guidance on running Visual Anomaly Detection on Edge devices #3804
|
Hi all, I'm working on an I'm deploying on an However, feeding a full 4K frame directly into a ML model immediately causes an Out-of-Memory (OOM) crash, during inference. I'm not sure about the best way to approach this. I understand that techniques such as tiling can help by splitting the image into smaller patches and running inference on each tile separately, which significantly reduces memory usage. My concern is that a purely tile-based approach may
Thanks! |
Replies: 1 comment 2 replies
|
Some things that come to my mind if you have to keep RAM and resolution
but really you should use a machine with a more memory |
I agree with the ideas from @alexriedel1
Additionally, it is better to go with optimised exported models like onnx with onnx runtime instead of pure torch.
OpenVINO (although primarly used for x86 , also supports ) is another option but better to benchmark it on the target device.
4K frame processing sounds a lot especially for on 2GB memory.
On the image size :
About the tiling part,
You could a…