08 March 2011

Who will test the tester?

Have you ever been asked to undertake a test as part of your job application? In the absence of any professional certification, it’s a reasonable approach for employers to take. Especially given that the majority of recruitment agents who short-list candidates are not equipped to filter expert from journeyman from “wannabe”.

As it happens I’ve done two tests in the last year, after a gap of maybe a decade. I wasn’t successful in either of those, but unfortunately neither of them was actually  a test of technical writing skills!

There’s a rather infamous test that used to be conducted by a company that does high-rate managed projects as well as placements. Maybe they still use it, although I’d say most candidates in Sydney would have done it by now. I was placed quite a few times by that company.

Anyway, the test stands out as being the most appropriate for technical writing that I’ve seen. You have to write instructions for the use of an everyday device, for an audience that has never heard of it.

This is a great test for some important reasons:

* Everybody has the required prior knowledge of the device, so there’s no issue of technical or domain exposure.

* It is genuinely a test of working with information, as distinct from working with content (i.e editing).

* It does not require expertise with any particular software. I did it with a pen and paper!

I did another test about 10 years ago that presented me with an excerpt from an actual passage of text from one of the prospective company’s engineers. I can understand the motivation for offering that as a test, and it had valid elements of working with information, but it was in the end more an editing test than a writing test.

When I began to refine the text, it became apparent it was describing two interdependent products. The extent of the interdependencies was not given in the text, so it required domain knowledge to process properly. That should never happen in a test! I must have done enough, because I did get that contract.

Skipping ahead 10 years, I was asked to demonstrate my competence in mapping processes. The example was completing a leave request form. The interviewer probably assumed everybody knows about such forms, which was, well, presumptuous! I’m actually not sure if I’ve ever used one, but it would certainly have been more than 20 years ago.

I took a punt at how I thought people used such forms in a bureaucratic organisation. In fact that was my undoing. I should have answered “that’s not a suitable process for me to demonstrate, could we use a different one please?”. Or else I should have written the process that I actually use which is “Advise client I will not be available in second week of April. End”.

I started thinking of steps that might be included in a large organisation as soon as I got in the lift, and kept thinking of others for the next two days!

Also last year I took another test that I think was prepared by a tech writing team, again using what looked like an extract from an engineer’s draft. But this time, it revolved around a very big screen capture image.

This created an instant dilemma: was the image an actual reflection of the software, or had it been created as a composite for the test? I suspected that it was a trick, and I was supposed to resolve that dilemma. I did as much of the test as possible without the resolution, and also created a template, and applied styles (wonder if they noticed?).

On the way out I pointed out that the instructions were not sufficient. The minder cut off my sentence and said “Yeh, there’s just not quite enough time to finish it, is there?”. As it turned out, I had given the testers more credit than was due. They just wanted candidates to break up the image into smaller parts. To make matters worse for me, it used Word 2007, which I had not used for three years except with my own customised toolbar (and only then at one client’s office that I visit maybe 10 days per year).

Let’s look at the ways this test was a failure: it was about editing rather than writing; the candidate’s objective was ambiguous; it assumed currency with a particular tool (which had not been given as a required skill); the main activity was to manipulate an image (and it was not a role that required graphical skills). That’s about as far away from the ideal  “everyday object” test as you could get.

Truly, we need some kind of certification for tech writing, to ensure a minimum standard of competency in candidates, and to protect candidates from unqualified skills testing. I've noticed that the Guru freelancing web site has some writing and editing tests, along with tests for just about every other kind of freelance skill. I'd like to evaluate them soon - stay tuned!