by cpncrunch 5 hours ago
There shouldn't be any noticeable hit on your cpu from bots from a site like that. Are you sure it's not a DDoS?
Obviously it depends on the bot, and you can't block the scammy ones. I was really just referring to the major legitimate companies (which might not include Perplexity).
There is a noticeable hit, there's also a noticeable cost, and it's not a ddos.
Not all sites can have full caching, we've tried.
I was referring to the community wiki.