If you run usability tests, you need to read Usability Testing: You Get What You Pay For. I’m not sure about many of the conclusions she drew from the fact that different usability companies charge differently and find different problems (Jared, maybe you have some thoughts on this). It seems to mostly be “People are charging too little so therefore they must be doing it wrong.” I have no doubt a lot of people are doing it wrong; I’ve seen most of the mistakes she lists as well. But I suspect there is more to the difference in results and cost than just wrong and right.
Her assessment of common mistakes made by novice usability testers and how to correct them is dead on. It is well worth reading and seriously asking yourself Am I guilty of this? It’s rare to find a practical article with applicable advice. This is one. Reminds me of one of my favorite books, By People, For People. If you don’t have this book, I encourage you to pick it up. It deals with many of the questions Mayhew brings up like sample size and how to avoid influencing Think Aloud protocols.