by mplewis 7 hours ago
No you cannot! I blocked all of the user agents on a community wiki I run, and the traffic came back hours later masquerading as Firefox and Chrome. They just fucking lie to you and continue vacuuming your CPU.
There shouldn't be any noticeable hit on your cpu from bots from a site like that. Are you sure it's not a DDoS?
Obviously it depends on the bot, and you can't block the scammy ones. I was really just referring to the major legitimate companies (which might not include Perplexity).
There is a noticeable hit, there's also a noticeable cost, and it's not a ddos.
Not all sites can have full caching, we've tried.
I was referring to the community wiki.