Favourite Quote

Every artist was first an amateur. - Ralph Waldo Emerson The more difficulties one has to encounter, within and without, the more significant and the higher in inspiration his life will be. - Horace Bushnell

Monday, February 25, 2013

Selenium WebDriver and Sikuli

With Introduction of more & more gadgets ,HTML5, Flash objects and many more technologies to come in near future

It often becomes annoying when you can't incorporate those in your test suite for some weird reason

I was assigned a Microsoft DX Project as most of you are aware those annoying Lookup's which you can't even know what objects you are going to work with.

In traversing through the DOM.Which is crucial for WebDriver/Selenium RC

Here's how i dealt with it by implementing sikuli within my webDriver scripts

What/Why is Sikuli now ?

Sikuli Script automates anything you see on the screen. It uses image recognition to identify and control GUI components.

It is useful when there is no easy access to a GUI's internal or source code so that you interact with it or complex flash objects list the is not limited to only these many more cross browser,operating system compatible

Download the latest version based on your Operating System from http://www.sikuli.org/download.html

For windows here are steps with Pre-requisites & installation :

Only 32-bit version is provided. But this version should run on both 32-bit and 64-bit Windows systems.


1. Make sure you have installed the official Sun Java 6 JRE 32-bit version (Java 7 or 64-bit are not supported).

2. Make sure you have uninstalled all previous versions of Sikuli (especially 0.10.x versions).

3. Make sure you have restarted your system after you installed JRE and uninstalled old Sikuli installations.

Installation Steps:

1. Download and install Sikuli using the self-extracting installer: Sikuli-X-1.0rc3 (r905)-win32.exe.

After installation is completed, a folder named Sikuli X should be created on your system.

2. Do not start using Sikuli X now because it has some bugs.

3. Download the following zip file: Sikuli X r930. This contains important bug fixes

4. Open the downloaded zip file and locate the folder called SIKULI-IDE.

5. Copy the content in SIKULI-IDE to Sikuli X. The purpose of this step is to replace the files associated with r905 (the buggy version)

by the files associated with r930 that has the most recent bug fixes.

After all the above steps you should be able to see sikuli installed in your All programs or whereever you have installed

Now,Let's get started launching Sikuli if you've installed in your All Programs > Sikuli X > Sikuli IDE

Sikuli IDE Editor will be launched - Just, like the one shown in below screen shot

On the left hand side of the IDE you see different commands available on the right side is where you would see commands copied

Below is an example with some random click an object on the web page,find a object on the web page ,doubleClick an object on the webpage

You can build as complex a script as you want.

An amazing feature Sikuli has to offer other than being open source and matching images is that one can use Sikuli features in your Java program through the Sikuli Script Java API

Here's are steps how you add Sikuli API to your existing library

I am using Eclipse IDE so steps mentioned will be what needs to be done with Eclipse

How do i add the jar to eclipse ?

Right click on your project > Build Path > Configure Build Path

Now that you see a popup with Properties to Java Build

Click on the Libraries tab then click on Add External Jars

Add sikuli-api-1.0.2-standalone.jar that you have downloaded earlier

import org.sikuli.script.FindFailed;
import org.sikuli.script.Screen;

public class SomeTest 
public static Screen screen;

	public static void setUp() 
			System.setProperty("webdriver.ie.driver", "C:\\Eclipse\\Selenium\\IEDriverServer.exe");
			screen = new Screen();

        public void popUp() throws FindFailed, InterruptedException
		   screen.exists("C:\LookupRecords.png", 10); //here you will specify the location of your image file
		   screen.click("C:\\OK.png",0);  //here you will specify the location of your image file

        public void someSikulitest()
          // do something ...
          // call the above method

	public static void tearDown()


feel free to let me know how it goes , your comments are valuable

Tuesday, February 19, 2013

Selenium WebDriver and AutoIT

It is often possible in this web world that we get to work with objects which is either out of box,flash or webpages that are added with extra layer of security like a single sign-on for example windows credentials

Also another example is if one works with Microsoft technologies or Oracle CRM Web Applications which are only compatible with IE browsers there LOV(List of Values) popup , not to forget the modal popup

In the above situations it's hard to achieve or progress with automation with Selenium RC/Selenium WebDriver

Hence the simplest and fairly easy option is AutoIT

What is AutoIt ?

AutoIt v3 is a freeware BASIC-like scripting language designed for automating the Windows GUI and general scripting. It uses a combination of simulated keystrokes, mouse movement and window/control manipulation in order to automate tasks in a way not possible or reliable with other languages (e.g. VBScript and SendKeys). AutoIt is also very small, self-contained and will run on all versions of Windows out-of-the-box with no annoying "runtimes" required!

Let's get started with having AutoIt installed on your local box

Download latest version of AutoIt from the official release page here

It's better you download AutoIt Full Installation which will help you in scripting & compiling as well

Once installation is completed you should see the application being installed in your All Programs or which ever location you have installed

Now, let's get started - What is that we are trying to automate or use AutoIt for

Given this Scenario : Of Logging in the windows Authentication to your webapplication

Just like the below screen shot

Now that is quiet a common to have a windows authentication if it's a product built by Microsoft themselves ;)

Here is how the AutoIt Script should look

WinWaitActive("Connect to yourservername","","60") If WinExists("Connect to yourservername") Then ControlSend("Connect to yourservername","", "[CLASS:Edit; INSTANCE:2]","DomainName\username") ControlSend("Connect to yourservername","", "[CLASS:Edit; INSTANCE:3]","Password") ControlClick("Connect to yourservername","","[CLASS:Button; INSTANCE:3,button[,clicks[,153[,223]]]]") EndIf

Once you have the above script compiled - You are all ready to use them in your webDriver / selenium RC script

I assume it should be able to work with both .Net & Java

For now i have just tried only on with WebDriver Java Bindings

So here's the code that goes in your webDriver script

@BeforeClass public static void setUp() { System.setProperty("webdriver.ie.driver", "C:\\Eclipse\\Selenium\\IEDriverServer.exe"); driver = new InternetExplorerDriver(); driver.get("your url"); try { Runtime.getRuntime().exec("C:\\Eclipse\\Selenium\\Autentication.exe"); } catch (IOException e) { e.printStackTrace(); } }

You are all set now : both the code snippets together should help in working on Windows Authentication

If you are keen on learning about how & when should i use working on Image recognition

Read my next post on using Sikuli for doing the same functionality or different scenario