by Ldorigo 4 hours ago
How do the economics of your statement work out? Clearly inference providers don't have a time to ROI of 10 years on their hardware costs; and that's without even taking ongoing energy costs into account. What's missing here?
The inference providers are running batch sizes much larger than 10