The inference providers are running batch sizes much larger than 10