Error messages as metric

Here's an idea for evaluating an organization: measure their error messages.

I picked the word "measure" in that sentence to let you define the appropriate metric. It could be a raw count or the complexity of messages. Or the absence of them. (An organization with lots of error messages in its code says one thing. An organization with little or no messages says something else.)

This task is perhaps more thought experiment than feasible. I suspect that most organizations are ill-prepared to simply hand over their error messages, even if they so desired. Error messages get tucked away in the darndest places, and any large system (comprising of multiple programs) will have messages in different forms. Messages can be hard-coded, stored in resource files, read from external text files, and generated on-the-fly.

Yet an evaluation of error messages may be of value. Error messages are presented to the user, and are a form of communication. I suspect that they are less regulated by the marketing arms of organizations than other forms of communication, such as web pages and e-mail updates. They are sent to the user only when something goes wrong (usually the fault of the user, but not always). They are not in the forefront of marketing.

Here are the aspects that I would look at:

- Are the messages accurate? Do they present the correct information for the situation?
- Are they spelled correctly? Do they have correct grammar?
- Are they specific? Do they present details, or do they present general text such as "Required field missing"?
- Do they recommend an action? Or do they assume that the user will know what to do?
- Are there lots of them? Too many, perhaps?
- Are there too few?
- Are there messages for situations that can never occur (much like "dead source code")?

Error messages may tell us a lot about the organization's view of its relationship to its users. The home page will have pleasant (or impressive) descriptions and pictures of smiling people. But anyone can have a welcoming or impressive home page.

Show me the messages!