This is the third and final post on the subject of recruiting testers.
The selection of interview candidates and the interviews are done so its time to check out interview feed back to try to identify the strongest candidate. Back in the second post on this subject i mentioned that i was scoring the candidates out of 10 for each of the items in each sections 2 and 3 of the interview. There is a bit more to the feedback than just noting the candidates scores however, the best way to show this is to look at some
See below for the format and example content from an interview write-up:
The interview plan (Didnt quite keep the timings!)
- 20 minutes for questions on their CV
- 20 minutes on questions around their competencies based on the things we required on the job ad.
- 20 minute for the test site test that **** wrote for me to see how many bugs they find and then get them to pick there favourite and write a bug report for it. Will go thru these results after the interview is complete.
CV Questions (Scores out of 10 next to questions only questions that i managed to ask will be scored!):
- What sort of tests did you create with selenium IDE / TestNG / Webdriver, any Limitations you found? (0)
- Which do you prefer Agile or Waterfall (5)
- What did you get from doing the ISTQB certification (5)
- Can you explain some test cases based on Behaviour-driven development (5)
- How did you tackle cross browser testing (5)
- How do you create your test requirements (0)
- How did you use data driven testing (0)
- How do you tackle testing time estimation (5)
Competency based questions:
- Talk about what process you would use when a new feature is being development and you will need to test it. (5)
- Talk about a time when you worked with end users to successfully UAT a new feature / product
- Talk about a time where you dealt with a developer to fix a bug you had raised.
- Give an example of when you improved a process that you were working with. (5)
- Talk about a time when you had to flag up issues to management that were impacting the testing phase of a project.(5)
Score out of 80 = 40
- Lots of experience but sometimes they did not explain the things they had done very clearly.
- Even though selenium featured a great deal in his cv they did not really know much about it.
- Didnt seem to know much about the buzzword in their cv when questioned eg Behaviour driven development.
- Did have management experience and wanted to learn more, not sure they would be able to fulfill this in the short-term with the role on offer.
- They made positive improvements to processes and defined some when there were none in place.
- They had worked at the start of projects to improve requirements
- They had used to having a large amount of documentation around testing, but is fine with having less.
- They had suggested tools to use as they had experience of them in the past, not because they did a POC…
- Didnt get a feel of how they worked with developers, what their relationship was like with them…
- Couldn’t really get much of a feel for the applications they had tested in his previous roles.
QA web site test notes transposed from written notes of candidate:
- I gave card number as ‘AAAA’, then application raised a bug as card number is too short but as per functionality card number should accept numeric values. this is high severity bug
- Application focusing on only one bug at a time. Action: I gave card number as “12345678912345” and name on card as “1234” and Expiry date as “10/15”. I gave 3 fields with wrong data, but application focusing on card number only. It raised a bug “card number is invalid”, but it did not give errors on name on the card, Ti should contain numeric values and Expiry data, It should contain numeric values and Expiry date, it should be 11/15.
- Post code: For post code i gave input as “777” As per functionality the application should rise a bug. It is not the standard format.
- Email: I gave input as “uuuu”. Application accepting the input. As per functionality. The email should be xyz@gmail etc
- I kept the “last name” text field empty and clicked on the pay now. “Application raised a bug “Please enter first name” but as per functionality application should raise “Please enter first name” but as per functionality application should raise “please enter last name”
- Card No: I gave card NO as “ 4444 4444 4444 4444” as per functionality application should contain numbers but text field card NO Not working properly for the number as well.
- FIRST Name: Should accept numeric values, but accepting numbers. It is wrong
- Last name: Should accept numeric values only but accepted non numbers. This is wrong.
Example bug report :
My thoughts on QA Test output from candidate:
The output from the test shows a fairly detailed testing of the validation of the customer details page but there was very little focus on the other pages. There were bugs to find on those other pages. Didn’t see the 404 page when you click the link for contact us, should really of at least clicked all the buttons for the pages there were not many!. Also did not notice that the basket does not add up the products correctly and that the currency converter does not work.
Even though i asked the candidate to pick a bug found and create a bug report from it they didn’t do this and instead gave me some things that a bug report should contain.
Candidate seemed to have good knowledge of testing processes but seemed to rather hide behind buzzwords which when asked about revealed a lack of knowledge of them. During explanations reiterated a lot of the same points again. They were keen to expand into automation which is encouraging. Slightly surprised they missed some rather easy to spot bugs in the test but he did do some good testing of the form validation. My conclusion is that they didn’t quite match up to their CV but would still be able to do an adequate job.
After the all the interviews has taken place i had a number of these candidate feedback documents to go through. Having this level of feedback really helped to make a fair and informed decision on who we wanted to offer the position.
I hope this series of posts will help you if your recruiting for your team.
Thanks for reading!