Automated Mobile Apps Testing

Today I attended this conference on automation of mobile application testing by one of the major players in the industry, Jamo solutions.

Testing of mobile applications can be envisioned in two perspectives. One through the bitmap or image based and the other object based, that is by recognizing and performing actions on objects.

The primary aim during the development of Jamo solutions was:

• Testers to be abstracted from knowing the complexity of multiple devices with same operating system. Example: iPhone, iPad and iTouch based on common operating system iOS. Similarly Droid and Nexus devices based on android operating system.
• The ability to create tests to test applications on multiple versions of mobile operating systems independent of devices. Example iOS3, 4 and android 2.2, 3 or 3.1 versions.
• Ease of learning for those testers with some level of skills already acquired on testing tools such as QTP, Visual Studio or Eclipse and of course the knowledge of the application.

As mentioned above, Jamo solution aids tester to develop test scripts from QTP, Microsoft Visual Studio 2008 or 2009 and Eclipse.

Android devices can be tested using an emulator provided by Jamo and with set of suggested setup and configurations on the device, application and the PC [which I will talk about shortly below]. For testing iPhone applications on iOS, Eclipse solution is suggested on a mac machine to develop scripts.

Applications on Blackberry operating systems too can be tested with this solution be it Bold, Curve or the latest Storm operating system.


Here is how the Jamo solution works:

• Firstly an add-in has to be obtained for the testing tool. For QTP or VB or Eclipse, Jamo has add-ins that are licensed.
• Secondly a device manager should be installed on the computer to which the device is connected and from which the apps will be tested. The device can be tested by connecting physically via USB or via WAN. In case of WAN a licensed machine can be in one state while the device can be physically present anywhere as long as they both are on the same WAN. This enables testing of different scenarios such as low network connectivity (low signal areas) and also supports distributed testing team concept.
• Thirdly an agent must be deployed on the device under test. The procedure will be provided in detail by Jamo once license is purchased. The primary difference in testing different device operating systems is the security rules. Jamo has made it clear that it is possible to test iPhone, android or Blackberry applications without breaking these security rules.

Also the speaker did mention that Jamo solutions is having a tie up with HP in development of mobile application testing solutions. It is among the 4 companies that have the tie up with HP. Three others being Zap, Device anywhere and one other which I don't remember right now. But the other 3 are mainly bitmap based and are not as popular as the object based Jamo solutions.

The license cost of Jamo solutions depends on a matrix depending on various parameters including:

1> Seat or concurrent license 

2> No. of types of Operating support needed: One OS, Two or All OS.

Roughly if someone purchases license for everything it is slated at about $18000 and 18% maintenance support from Jamo.

I might sound like a promoter of Jamo solutions, although that is not entirely true. I am penning details about these as I myself am going to test a major retailer's website on iPhone and Android phones using Jamo solutions.

To know more:

1> Jamo Solutions Website: http://www.jamosolutions.com

2> BizExc Website: http://www.bizexc.com - the distributors of Jamo solutions.

3> DFWQA Association (DFWQAA) - http://www.dfwqaa.org

Testing Tools supported:



Operating Systems supported:

Flex Objects not recognized by QTP

This is a commonly seen issue that QTP is not recognizing a particular flex object even though you have installed the Adobe Flex Plug-in and the library files are deployed with the flex build. There could be many ways to resolve this. The object could be recognized as a win object, macromedia flash object or a basic flex object like the one shown below.

I have a checklist that may help in resolving this:

1. Have you installed the flex plug in correctly? My blog below should help you verify that:
http://qtpadvanced.blogspot.com/2011/01/verify-install-of-flex-plug-in-for-qtp.html

2. Have you selected the required add ins for your application? Typically for a flex based web application requires flex, web and activex plug ins selected in the QTP's Add-in manager. Remember that QTP add-in manager loads at run time only and if you are not seeing this at start-up then it could be disabled within QTP Tools-> Options -> General.

3. If the object is only partially recognized by QTP and if it is an input field that can accept keyboard inputs they try the QTP SendKey method on that object. This uses the object.Type method to send keyboard inputs. If this works then this should knock off most of the text input objects.

4. The object could have been configured to work based on user action. For example a small window could open with options when user selects that object/ clicks on it. There could be two such possibilities:
   a> An input field coded by developers by combining more than one standard flex components. Example: A spark.components.TextInput can be combined with spark.components.Group, spark.components.PopUpAnchor and a spark.components.List making it kind of a search-able intelligent field that loads the list based on user input.
   b> The component is a custom component that has inherited the standard flex component. In such cases this may require "Instrumentation" of this component. This necessitates the developers to create a delegate class for this component which sends flex automation events that QTP can intercept while recording.
This adobe site link should give you insight into the instrumenting custom components:
http://livedocs.adobe.com/flex/3/html/functest_components2_11.html

Also check to make sure that the object in question has a class definition entry in the automation xml file. This link should give you more information on the class definition entry:
http://livedocs.adobe.com/flex/3/html/functest_components2_15.html#178953


5. There are a few cases available in the HP knowledge base which may be worth looking at but this may or may not help you resolve your problem. HP however states clearly that the plug-in is developed, maintained and supported by Adobe and that Adobe is the point of contact for any issues with respect to QTP and Flex automation.

6. Check in the Adobe Bug and Issue management system which is meant for public to log in and create or search issues. Searching this database could be very useful since there are a huge number of bugs logged and resolved by Adobe.
Here is the link: http://bugs.adobe.com/flex/
This requires registration to create or search bugs logged with Adobe. Just register and you can look into them.

Verify Install of Flex Plug-In for QTP

1. Install the FLEX plug-in as instructed by Adobe. Refer PDF for installation instructions given along with it.

2. Restart the machine and opened QTP and see that the Flex 4.00 add-in is available in the list.

 

3. Then verify that the install is correct by searching for entry of TEAPluginIE.dll and TEAPluginQTP.dll. If those dlls are not in the registry, then the plug-in may not have been installed correctly.  

4. Verify the flash player version should be 9 or above.

 

     5. Verify the compatibility of QTP and IE version. They both should be compatible.

        The table below shows the versions supported for the Flex plugins for QTP

QTP Version	IE6	IE7	IE8
QTP 9.2	Yes	No	No
QTP 9.5	Yes	Yes	No
QTP 10	No	Yes	Yes

 QTP10 and 1E6 also theoretically seem to work. But it is not a supported combination from Adobe as we have not done extensive testing on this combination 


     6. Open a sample website by Adobe: http://examples.adobe.com/flex2/inproduct/lcds/flexstore/flexstore.html and try to spy objects and record steps on that site. The website prompts on the browser to install an ActiveX plugin. Allow it to install. QTP should recognize the objects now on the example application.

Useful Links:

Plug-in Download Location (also located on ATC SharePoint Knowledge Share folder):

https://www.adobe.com/cfusion/entitlement/index.cfm?e=flex4_automation_plugin

Sample Application to test on:

http://www.adobe.com/devnet/flex/samples/flex_store_v2.html

Installing Flex 4 Plug-In for QTP

Download Plug-In:

Adobe Flex 4 Plug-in for QTP Download Location on Adobe site::

https://www.adobe.com/cfusion/entitlement/index.cfm?e=flex4_automation_plugin

Installation:
1. Install Flash Player for Microsoft Internet Explorer. This is currently the only supported browser/player.
2. Restart your computer.

3. Get the Adobe Flex 4 Plug-in for HP QuickTest Pro zip file and unzip it on the machine where you want to install the plug-in.
2. Double Click on the Install_QTP_Plugin.bat.
3. Restart your machine.

The plug-in installer will include the following in the installation directory:
• AIR folder which will have the AIR related dlls
• FLEX folder which will have the Flex related dlls
• Uninstall_QTP_Plugin.bat Double clicking on this bat file will uninstall the QTP Plug-in.
• ReadMe.txt file.
The zip file also contains a Demo folder that contains a Flash movie that describes the basics of using the plug-in. (Be sure to enable audio on your computer.

Using the Plug-in
1. Start QTP again after installing the plug-in. The Add-in Manager lists the Flex and AIR plug-in.
2. Select the Flex and Web plug-in in the Add-in Manager if you want to automate Flex applications or select the AIR Plug-in in the Add-in Manager if you want to automate AIR applications.
3. Select New > Test and click the Record button.

Note: Flex application testing with QTP currently supports only Microsoft Internet Explorer with the ActiveX Flash Player. For more information on these tasks and using QTP to test Flex applications, see Testing with QTP. For information on the operations and properties of Flex objects in QTP, see QTP Object Type Information.
Samples for Automated Testing Sample custom agents are available at Custom Automation Agents.
An application ready for testing with QTP can be found at Flexstore AT. This sample can be used to test if the QTP plugin installation was successful. An example for automating custom components can be found at Automating Custom Component.

Testing Adobe Flex 4 with QTP

 Adobe has a technique specified on their site that in order to automate Flex applications with QTP, it is necessary that the developer force include a set of library files during deployment. These library files enable the communication between Flex and QTP layers.

Successful deployment of the libraries and correct installation of the plug-in on the testing machine are two important things to consider. The developers should deploy the application including the framework swc files necessary for QTP to communicate with Flex and tester should install Adobe provided QTP plug-in for Flex on the tester's machine.

Note that the developer may have standard controls used such as FlexDateGrid, FlexButton, SparkButton, SparkLabel, SparkCheckBox, etc. And also may have custom controls developed.