Can you think of a[n interview] when……

As a contractor, I reckon I go through the job interview process about once every 18 months to 2 years. Frankly, it can be pretty tiring stuff, particularly if the market is doing well (yes, I know, I shouldn’t really be grumbling because that’s what any job seeker hopes for). However, the most tiring thing is going through the same interview format every time; what strikes me as odd is how the whole thing hasn’t particularly changed since I applied for my first (real, no-longer-a-student) job, especially given the speed at which the industry in which I work changes.

Classic job interview question cartoon

One may argue that it’s because the methods are tried and tested. One may also argue that the process is as painful for interviewers as it is the interviewees, so why spend more time thinking about it than you have to? I actually went through one recently where the interviewer actually said, “you know what? I’m just going to read these out parrot-fashion”. One size fits all.

Here’s an example of the sort of thing I see on your standard job spec for a contract BA:

  • Experience of ‘As-Is’ and ‘To-Be’ process mapping
  • Use of BPMN and UML
  • Stakeholder management across all levels
  • Workshop facilitation
  • Requirements gathering and documentation

Or as I may rewrite it:

  • Must be a half-decent BA

Now I can’t speak for developers or testers or even project managers but I can almost guarantee that no matter what variation there is on the job spec above, I will turn up to a “competency-based interview” where I WILL get asked the following questions as a couple of examples:

“Can you think of a time when you had to influence a particularly difficult stakeholder?”

“Can you think of an example where you’ve had to deliver to demanding timescales?”

I can see why these questions are being asked, because the best interviewees will pluck the example out of their experience and relate it to the context of the organisation for which they are interviewing. But what about all those other things you asked for? The “product” skills? The BPMNs and UMLs and Agiles of this world? When I’ve looked at CVs in the past, I’ve typically seen all the buzzwords on the front page that means the CV’s been picked up by the recruitment consultant’s database. I then try to find the evidence in the body of the CV. But that still doesn’t tell me whether the person can do them. My view is that these skills can be tested at interview, so prioritise the skills you’re after and test them.

Footballers go for trials where they showcase their skills; they don’t just talk about the different games they’ve played in and the great goals they may (or may not) have scored. If someone wants to go on X-Factor (hmm), they have to sing, they don’t just talk about how they’ve sung at karaoke in the past and how well their mum’s reacted to the delivery of their latest rendition of Lady Gaga.

You say you can write an Agile user story? Show me. You say you’ve done a use case or a UML statechart diagram? Show me. Not quite as open-ended as that, but I’ll explain what I mean.

I was sat in a meeting room about 5 years ago with a couple of guys that were explaining the background of an upcoming project. Whilst I was in the meeting room, I started drawing up a process model to visualise what they were telling me. One of them went, “this is great, we should make people do this in interviews!” Whilst that was an off-the-cuff comment at the time, it resonated with me; yes, why not?

A couple of contracts later and I was in the position of doing some interviewing myself, for another contract business analyst. The client wanted someone with good process mapping skills (ideally BPMN as that was the adopted standard) to come in and look at the document handling processes within the organisation. There was a specific skill required and a specific project, so why not examine how the analyst might go about tackling the problem domain? To that end, I worked up a small exercise. In use case brief-style language, I wrote a paragraph explaining a simple end-to-end process for the receipt, scanning and distribution of documentation received, which went something like this:

Documentation is received into the postroom where it is opened, examined and any barcoded documentation is scanned into the imaging system. The electronic document is then assigned to a work queue where it will be picked up by the processing team.

I deliberately left out bits of detail; I’d mapped the process out in advance and then chose the bits to leave out in the description. When gathering requirements, people often leave out detail because either they’ve forgotten about it, or it seems unimportant. So I didn’t just want to test the ability of the candidate to model the process I’d written down, it was to test their approach to analysis; their ability to ask the questions that drive out the detail required; their ability to play back their understanding to their stakeholders and gain acceptance of what they’re delivering. It wasn’t even essential that the diagram was done using BPMN in this particular instance; BPMN has set semantics and can be picked up quite quickly, but the analytical skills required to get the information out and down on paper was far more important. As I was writing this, it brought to mind the “technical challenge” part of the Great British Bake-Off; contestants are given the same recipe, but details are left out that mean they need to use their knowledge and skills to deliver what’s required.

The results were very interesting; the first two candidates mapped the process out, not specifically using BPMN (I asked them to use whatever they were comfortable with) and by-and-large got the process nailed. They stopped and asked questions “what if the barcode can’t be read?” “what do you do with documentation that has no barcode?” The last guy focussed so much on the notation that he forgot to focus on the outcomes. To make matters worse, he got the notation so wrong that the final result was not just an incomplete process, it was meaningless. So he demonstrated that not only could he not actually use BPMN, fundamental analysis and communication skills were just missing.

Candidate number 1 got through btw 😉

To use an Agile example; why not think of a process or function in your problem domain, describe it and then let them have a crack at breaking it down into stories? Candidate number 3 also had Agile on the CV and this is one I love to test, because it’s another big buzzword, along with Systems Thinking. How many people are actually dressing up iterative requirements gathering as Agile? How many people are on small projects with frequent releases and calling it Agile delivery? How many people actually understand the Agile principles?

This may sound like a lot to fit into the interview time. This is just another test of business skills: the ability of the candidate to effectively time-box and prioritise their work. You’ve got a timescale to meet; you need to interview them and get onto the next candidate so let’s see how well they work within it.

It requires a bit more planning, but I think it will pay off, because you’re truly honing in on the ability of the candidate to do the role you’re advertising for. I’d say a lot of the above may be more relevant to contractor interviews, because permanent positions are not always as targetted as the gaps that contractors are there to fill. It also may not work as well if there is a large-scale recruitment drive and it may also depend whether you are recruiting for skills you don’t currently have so are effectively unable to test the candidate in the ways described. But you may have other thoughts? Is this approach just as flawed as the standard template of the competency-based interview?

2 thoughts on “Can you think of a[n interview] when……

  1. Good post Nick,

    I’ve had a similar complaint about competency based interviews; from both recruiter and recruit side.

    As a rescruit I don’t feel that my qualities and skills have always been fully utilised in the contract (or permanent) roles I’ve been in. The truth is that everywhere I’ve worked to date have actively pursued me to stay, and in some cases, to return. They rely on the service, and behaviours I exhibit. I’m not saying that I’m the best person in the room, but the truth is, a series of standard “tell me a situation where you’ve done x” doesn’t really fulfill the explanation of all the other [modesty alert] awesome things I have delivered. On the flipside, the two interviews I’ve had where I’ve been – actively tested on the competencies I’ve needed – have proven to assess not only whether I have the key skills required, but also where I need to be positioned within that role (FYI, one of these jobs was a permanent position, so I guess it still applies).

    As a recruiter, I was interviewing testers. In my opinion being a good tester is an art form. Unfortunately, this doesn’t always seem to be the case to employers – and some of the testers out there give the field of Quality Assurance a bad name. To combat the string of incompetence I’d seen, I came up with some very specific scenarios – ultimately the output was:
    a) what would you look to test?
    b) how would you look for it?
    c) what edge-cases would you consider looking for?
    The result was a damn good candidate who then went on to further develop the skills of the company as a whole – the extra time paid Dividends!

    Like you, I think competency based interviews are perhaps suitable for behaviours (not, in fact, competencies) and should be accompanied by some good hard skills testing.


    • Hi Chris,

      Thanks for your comments and it’s good to see this applies more widely than just the contract market.

      Your final sentence nails it for me; like my LinkedIn reply to Keith, I have no issue with competency-based questions and they are necessary to establish desirable behavioural characteristics. But a focus on skills testing is a must in my view.

      For example, I recently had an interview where I was asked why I chose to use BPMN over UML Activity Diagrams and where I saw Use Cases fitting into the picture. This is much closer to the sort of thing I’m on about, but it’s still lacking. It’s a good test in that it challenges why one technique is preferable to another – something you can only really talk about from experience. But the slightly odd thing was the fact that the organisation uses BPMN and not UML Activity Diagrams. So are they testing that I really know what BPMN is, or whether I’d fit into their ways of working? Not sure. And whilst I can talk about it from experience, they don’t know that I’m actually doing it right.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s