Appendix 9: Surface Spy Tutorial

Appendix 9: Surface Spy Tutorial



Appendix 9: Surface Spy Tutorial

Introduction

Process Studio Surface Spy is used to capture images/regions on the screen. This image can then be located based on a matching pattern specified. It is particularly useful on remote machines where we do not have any handle to the content and everything is an image or pattern.

Start Using Surface Spy

Follow the Steps below to use Surface Spy.

  1. We have created a Project ‘Spy’. Now create a folder.

  1. Name the folder Surface Spy.




  1. Surface Spy folder is visible in the workspace as below.




  1. To launch Surface Spy click ToolsSpy Surface SPY menu option.       

 


  1. Respond to the warning message to create a workflow.

 





  1. The default workflow name is Workflow_1. Click OK.

 



  1. Surface Spy opens in the foreground to Process Studio as seen in the snapshot below.  

 


  1. The two icons on Surface Spy are explained below and implemented in the steps that follow.

Icon on Surface Spy

Description



On hovering over the icon it shows the name of the icon - Capture Image


If you click on the drop down arrow it shows three options,

  1. Capture now

If you click on the Capture Image icon or select Capture now from the drop down list you see a + mouse pointer that can be used to outline any area on the screen.

  1. Capture in 3 Seconds

If you select Capture in 3 Seconds from the drop down list you see a + mouse pointer after 3 seconds, that can be used to outline any area on the screen. This is useful if the area to outline is not on the screen immediately but you need to navigate to a desired screen.

  1. Capture in 10 Seconds

If you select Capture in 10 Seconds from the drop down list you see a + mouse pointer after 10 seconds, that can be used to outline any area on the screen. This is useful if the area to outline is not on the screen immediately but you need more than seconds to navigate to a desired screen.


On hovering over the icon it shows the name of the icon - Locate Images

Once you outline an area using Capture Image the next step is to Locate Images. Clicking on Locate images icon searches for matching patterns(/images) based on the Matching Pattern selection from:

  1. Retrieve Single closest match
  2. Retrieve Multiple close matches
  3. Retrieve All matches




  1. Let us Capture Image on the paint document as shown in the screenshot below. Let us assume the image on the top left corner to be the base image with a size 100%.  The images on the right are smaller images with 10% reduction in size each. The images in the bottom row are in ascending order of size ascending by 10% each.


  1. Once you capture an image or pattern the AutomationEdge Surface Spy window expands as shown below. In this case we have captured the third image from left in the top line.  


  1. Generate Step configuration screen options differ depending on the Action chosen.
  2. The following screen shows the Generate Step configuration options when Action chosen is Click as seen above also.




  1. The following screen shows the Generate Step configuration options when Action chosen is Action Set Value





  1. The following screen shows the Generate Step configuration options when Action chosen is Action Get Value


  1. The following screen shows the Generate Step configuration options when Action chosen is Action Find Image.



  1. The following table summarizes all the Generate step options for all Actions types.


    Table 2 : Surface Spy Generate Step Options

No.

Option

Description

1

Action

The Action Drop Down has the following options available.

No

Click Types

Description

1

Click

It performs aclick action.

2

Set Value


It performs Set Value on a region/image.

3

Get Value


It Gets Value from a region/image.

4

Find Image

It checks for an image and returns true or false.


2

Click Type

The Click Type Drop Down has the following options available. This option is available if Action chosen in Click.

No

Click Types

Description

1

Click

It performs left click action.

2

Right Click

It performs right click action.

3

Double Click

It performs double-click action.

4

Middle Click

It performs left Middle click action.

5

Mouse Move

It performs Mouse Move action.


3

Value

Specify a Value to be set on a region/image.

4

Output field name (for Action Get Value)

This field is active when Action chosen is Get Value.  Provide a field name to hold the value retrieved from a region.

5

Output field name (for Action Find Image)

This field is active when Action chosen is Find Image.  Provide a field name to hold a Boolean value(Y/N) indicating if the image was found or not.

6

Position

The Position Drop Down has the following options available. The Action chosen above is performed at the position selected here.

No

Positions

Description

1

Left

Center Left position of the Image.

2

Right

Center Right position of the Image.

3

Center

Center of the Image.

4

Custom

Relative position from the entire Image. It could be anywhere on the screen. If the relative position of the relative element to click is closer it is preferable.


7

Image Index

Image Index is retrieved once an image is clicked from the selected images.

8

Timeout

This is the Timeout for step execution. By default timeout is 5 seconds

9

Match Pattern

The Match Pattern Drop Down has the following options available.

  1. Retrieve Single closest match
  2. Retrieve Multiple close match
  3. Retrieve All close match

10

Button: Generate

Click Generate button to generate the step with the configurations above.


  1. The Match Pattern drop down field has the following options as shown below,
  1. Retrieve Single closest match
  2. Retrieve Multiple close match
  3. Retrieve All close match




  1. Once you outline an area using Capture Image the next step is to Locate Images. Clicking on Locate images icon searches for matching images (/patterns) depending on the Match Pattern chosen as mentioned above.
  2. Let us select Retrieve Single closest match in the match Pattern drop down. Now click on Locate Images () icon. It gives an error Multiple Match Found as it has found more than one close matches. 

Note: Click on an image to select and press Esc key to exit.

 



  1. Let us select Retrieve Multiple close matches in the match Pattern drop down. Now click on Locate Images () icon. It locates Multiple Matches as seen below.









  1. Let us select Retrieve All matches in the match Pattern drop down. Now click on Locate Images   () icon. It locates All Matches as seen below (not below 50% or above 150% of the original size).




  1. In this Appendix 3 we have seen how Surface Spy is used Capture and Locate Images and the various options available. 


Close Surface Spy

  1. You may click the X button on the AutomationEdge Surface Spy window to close Surface Spy. 

 


  1. You may also click X on the expanded AutomationEdge Surface Spy window to close Surface Spy.  


  1. Surface Spy also closes when you close Process Studio.


  Note:

Refer Project 8: Spy & Folder Surface Spy in AutomationEdge_Process_Studio_Getting_Started_Guide_R6.0.0 for a sample workflow using Surface Spy.





      Links to better reach 

            Bot Store

             EPD