by thepasch 5 hours ago
I know quite well what an LLM is and how it works! I've captured activation patterns and written scripts to analyze how they compare to one another in response to a set of controlled and curated prompts; in particular, trying to replicate the functional emotional vector findings from the Anthropic paper (https://transformer-circuits.pub/2026/emotions/index.html) on various open source models; successfully on some, less so on others. FWIW, Gemma 4 31B was among those where clear patterns did emerge.
What I don't know quite as much about is how cognition works in biological computers - and I suspect you know just as little as most of the rest of us do in that regard! So I think it's not entirely appropriate to make sweeping claims about what artificial neural networks, fundamentally, can and cannot do. Most of what we can do is poke and prod at them and see what happens, which is exactly what this piece is about.
I see, this experiment is a fun thing to do. My comment wasn't concerned much on why we do this, but rather on the fact that many people are starting to see LLMs as genuine entities, and i really don't think they are (also, i feed bad about using the word "genuine" after it has been abused so badly)
That's fair. FWIW I don't think they are either, but I specifically don't think they're fundamentally incapable of it, and I think that as models grow, we're going to see more and more concepts and behaviors emerge that might, one day, with enough parameters and enough training, approach the parts that a genuine entity requires to be a genuine entity. Whatever those are.
No idea if that's true or if there's some sort of "special sauce" required that you just can't get from artificial trained networks. But I've been a functionalist since long before LLMs emerged, so the signs of these behaviors that we are already seeing in the models of today aren't very surprising to me!