A very simple automation framework using selenium builder

In one of my previous posts i wrote that i was going explain how i set up a very simple test automation framework using selenium builder, well here goes!

These instructions are for windozzzzz by the way!

First thing you going to need is firefox and the selenium builder plug in. Once you have these installed you can record you first script, instructions on how to do this are here.

Next we need to grab Se Interpreter, this will allow us to run our new script created using selenium builder via the cmd line.

Create a new folder and pop in the files you downloaded for Se Interpreter into it, then also pop in this folder the script you created using the selenium builder.

Now we want to run our recorded script from the cmd line, to do this your going to need to run a cmd from the folder you have you script file and your Se-builder files in.  The cmd should look something like this:

java -jar SeInterpreter.jar <path_to_script.json>

If you run this you will see firefox start up the script will run and then Firefox will close. You should see the logging in the cmd window, this will tell you whether the test passed or failed.

The next step is to get a log file created when you run a script, if we are going to be scheduling and running these tests we will need to check the log files after the test run to check our results.

We are going to need to add something to the cmd line in order to this:

java -jar SeInterpreter.jar <path_to_script.json>>C:\Path to logfile\MyLog.log 2>&1

If you now run this you will get a log file created in the folder you stipulate in the cmd above with the name you have given the log.

You can also create a test suite file, this has details of individual test files in it. This means that you can run a bunch of scripts but you only have to tell selenium interpreter to run one test suite file. You just have to switch the script name in the cmd to the test suite file and let the tool do the rest.  In your log file you will get the results of each script that is run (The ones you have stipulated in your test suite file).

Now we have the scripts and our cmd line sorted we need to schedule the scripts to run at a certain time. I like my scripts to run over night against the latest development build so i can check the next morning how the build did. I am running my test on windows so i created a bat file using the cmd line and scheduled this to run at 11pm each weekday night. I could have used the windows scheduler to do this but i found that it ran the scripts in headless mode (You could not see firefox opening up when the scripts were running on the desktop) This caused me a problem as some of my scripts tested pop up error messages and these need the browser gui to be open. The free tool i used to do this is here.

You can add more test scripts and add them to your test suite and these will then get run at the time you have scheduled. You can also add more suites to the bat file as new lines e.g.:

java -jar SeInterpreter.jar <path_to_test_suite1.json>>C:\Path to logfile\MyLog1.log 2>&1
java - jar SeInterpreter.jar <path_to_test_suite2>>C:\Path to logfile\MyLog2.log 2>&1

Each one of these lines will create its own log file for you to use to check the results.

There you have it, possibly the easiest automation frame work possible and you can get something like this set up in a few hours or less then extend it as you think of more tests!


Web form validation testing part 1

Most web sites will have forms, places were a user can add details in order to provoke an action from the web site. Stuff like signing up to a customer account or a checkout were you have to add your address and payment details. Most forms will have some sort of validation so that the data entered by the user is usable by the web site and also to help the user to avoid mistakes.

The eCommerce site that my company users has lots of different forms and i wanted to write some comprehensive automated regression tests to check that the validation was working after we did any changes to the site. I was specifically interested in testing the input combinations that should be allowed and ones that should provoke an error that is displayed to the user.

I was not looking at how the form should handle the incorrect format of an input e.g. wrong format of email address, but rather the combinations of good data that could be entered into the form.

For example we have on the site a contact us form,  see screen shot below for the form fields:

Screen Shot 2014-09-06 at 18.56.41

There are a number of fields on this form that are compulsory and also there are different combinations of entries that are allowed and other ones that are not allowed.

The first thing i needed to do was to get all the possible combinations of inputs for this form. Not all of these will be valid but we can start to narrow down the test cases once we have created them all.

luckily there were a few tools available that will create a truth table of results when you give a number of different input fields. I used this one:


For the contact us form that has 9 input fields you just have to add nine fields into the truth table solver and hit solve. It will then spit out a matrix that should include every combination of inputs that are possible for the form. Next pop this into spreadsheet application, you should end up with something like this:

Screen Shot 2014-09-06 at 19.31.01

The ‘1’ in a cell indicates a valid input and the ‘0’ in the cell indicates no input.

You can see from the screen shot that the test results for these cells (This is just the first few rows in the sheet, there are 513 rows in the actual sheet!) are N/A that means that the combination of inputs for that row is not actually possible. You have to go thru your results and identify these cases along with the ones that will allow the form to be submitted and the ones that will provoke an error and shouldn’t allow the user to submit the form.

Now you will have a list of tests that will cover all the possible combinations of good data input, next thing is to create scripts that will test all your identifed tests that pass or fail.

Part 2 of this blog will focus on creating those automated scripts using the json format script that can be run using selenium builder!