by blobbers 11 hours ago

Can someone help me understand when these neural engines kick in in open source software?

I typically use python ML libraries like lightgbm, sklearn, xgboost etc.

I also use numpy for large correlation matrices, covariance etc.

Are these operations accelerated? Is there a simple way to benchmark?

I see a lot of benchmarks on what look like C functions, but today in my jobs I rely on higher level libraries. I don't know if they perform any better on apple HW, and unless they have a flag like use_ane I'm inclined to think they do better.

Of course chatgpt suggested I benchmark an Intel Mac vs. newer apple silicon. Thanks chatgpt, there's a reason people still hate AI.

zozbot234 11 hours ago | [-4 more]

> when these neural engines kick in in open source software?

It mostly doesn't because NPUs are bespoke and vendor-specific (which incents neglect by software devs working on open source numerics and ML/AI infrastructure), and the Apple ANE is no exception. Part of this effort is most likely about fixing that for the specific case of the Apple ANE.

blobbers 10 hours ago | [-3 more]

Part of which effort? The Reverse engineering is so it can be used blog article?

I just think: great it seems like I'm paying for a hardware accelerator that makes Siri go faster. And I use siri on my laptop exactly 0 times in the last infinite years.

bri3d 8 hours ago | [-2 more]

It also makes a lot of really useful features like on device OCR, captions, voice isolation, temporal antialiasing in metalfx, an enormous host of things in the apple pro apps, etc. work

blobbers 2 hours ago | [-1 more]

Yeah, I don't use any of those features. So it sounds like its for folks who are creatives running lightroom or apple movie, or some kind of apple sound program?

I'm a dev, not a creative, unfortunately. I don't use other people's software, I generally write my own (or used to before Claude took over my world).

saagarjha 42 minutes ago | [-0 more]

You bought a truck. Obviously there will be some part of it you don’t use.

this-is-why 4 hours ago | [-1 more]

Yes. Numpy will accelerate if it detects hardware that it supports.

blobbers 2 hours ago | [-0 more]