> Fast students are smarter.
Assuming this is true (others have already addressed a few of the many reasons it might not be), wouldn't that imply that practically all existing tests are already flawed? If you want to grade on time, every exam should be graded with a formula taking both the time and the correctness into account. A binary "fast enough" vs "not fast enough" is about as useful as a pass/fail class grade.
The exams that felt like the fairest reflections of my own knowledge were proctored in-person, closed book, and time unlimited. Of course, being time unlimited works better for quantitative/engineering exams. I haven't put much thought into more qualitative/liberal arts type exams.
In my experience, it was common to test speed - my favorite was solving 100 arithmetic problems in, I think it was 3 minutes? Perhaps shorter. The point was that it was basically impossible to solve them all, so it evaluated both your speed and your self-assessment: if you move too quickly, you'll make errors. If you spend too much time second-guessing yourself, you won't get enough problems finished.
It always struck me as an ingenious way to get a feel for a new group of students, since it's quick to administer and equally quick to grade, while revealing quite a lot of information.
Certainly we hold speed to some regard, since a lot of academic accommodations involve granting extra time. If we aren't testing on speed, then surely we should give that to anyone that asks?