by rfergie 5 hours ago
> there are students whose performance on such assessments is not a good signal of their intellectual ability
Is there a form of assessment that is a good signal of the intellectual ability of all students?
Fruit of labor? Good students of science produce novel science..as in testable ideas without precedent.. a thing llms struggle with, as they at best can interpolate and mash up previous ideas
Many good students don't produce such artifacts. That's one of the big problems in academia -- a smart person with good ideas can inadvertently pursue a less promising path and not produce ground-breaking research, so how do you tease that apart from somebody scamming the system? The current approach is to measure unrelated bullshit like citation metrics. I'd hate to see those sorts of perverse incentives pushed down into the student body as well.