by fidotron 13 hours ago

> HugOps

This childish nonsense needs to end.

Ops are heavily rewarded because they're supposed to be responsible. If they're not then the associated rewards for it need to stop as well.

denysvitali 13 hours ago | [-6 more]

I have never seen an Ops team being rewarded for avoiding incidents (focusing in tech debt reduction), but instead they get the opposite - blamed when things go wrong.

I think it's human nature (it's hard to realize something is going well until it breaks), but still has a very negative psychological effect. I can barely imagine the stress the team is going through right now.

fidotron 13 hours ago | [-5 more]

> I have never seen an Ops team being rewarded for avoiding incidents

That's why their salaries are so high.

denysvitali 13 hours ago | [-1 more]

Depending on the tech debt, the ops team might just be in "survival mode" and not have the time to fix every single issue.

In this particular case, they seem to be doing two things: - Phasing out the old proxy (Lua based) which is replaced by FL2 (Rust based, the one that caused the previous incident) - Reacting to an actively exploited vulnerability in React by deploying WAF rules - and they're doing them in a relatively careful way (test rules) to avoid fuckups, which caused this unknown state, which triggered the issue

fidotron 13 hours ago | [-0 more]

They deliberately ignored an internal tool that started erroring out at the given deployment and rolled it out anyway without further investigation.

That's not deserving of sympathy.

esseph 12 hours ago | [-1 more]

Ops salaries are high??? Where?!?!

hnthrowaway0328 12 hours ago | [-0 more]

Definitely commands better salaries than us pitty DEs.

agoodusername63 11 hours ago | [-0 more]

news to me.

esseph 12 hours ago | [-0 more]

Ops has never been "rewarded" at any org I've ever been at or heard about, including physical infra companies.