I like your project. I'm a SWE working on something completely different by day but tinkering with several other side projects, including this, https://github.com/ratsbane/panda-mcp/ (which is accelerating my computer vision learning rapidly) I'm interested in your choice of the OAK cameras because I'm trying to select a camera model also. I started this with a simple USB webcam and added a PhotoNeo camera. I've just switched to RealSense and I've got a couple of ZED cameras on order but now I'm looking at the OAK-D and thinking that it makes a lot of sense to run a nn directly on the camera.
well these days inference can be done on CPUs depending on the size of a model. the OAK-D camera's especially are literally just computer. They are linux and you can SSH directly into them which is insane. The camera itself is shaped like a heat sink to dissipate heat. it's honestly phenomenal. I talked to the CEO of the company as well and I'm really impressed with their machine and supply chain. so all in all, high hopes for them.