Appendix 9: Surface Spy Tutorial

Introduction

Process Studio Surface Spy is used to capture images/regions on the screen. This image can then be located based on a matching pattern specified. It is particularly useful on remote machines where we do not have any handle to the content and everything is an image or pattern.

Start Using Surface Spy

Follow the Steps below to use Surface Spy.

1. We have created a Project ‘Spy’. Now create a folder.
   

2. Name the folder Surface Spy.
   

3. Surface Spy folder is visible in the workspace as below.
   

4. To launch Surface Spy click ToolsSpy Surface SPY menu option.       
   

 

5. Respond to the warning message to create a workflow.
   

 

6. The default workflow name is Workflow_1. Click OK.
   

 

7. Surface Spy opens in the foreground to Process Studio as seen in the snapshot below.  
   

 

8. The two icons on Surface Spy are explained below and implemented in the steps that follow.
   

Icon on Surface Spy	Description
	On hovering over the icon it shows the name of the icon - Capture Image If you click on the drop down arrow it shows three options, Capture now If you click on the Capture Image icon or select Capture now from the drop down list you see a + mouse pointer that can be used to outline any area on the screen. Capture in 3 Seconds If you select Capture in 3 Seconds from the drop down list you see a + mouse pointer after 3 seconds, that can be used to outline any area on the screen. This is useful if the area to outline is not on the screen immediately but you need to navigate to a desired screen. Capture in 10 Seconds If you select Capture in 10 Seconds from the drop down list you see a + mouse pointer after 10 seconds, that can be used to outline any area on the screen. This is useful if the area to outline is not on the screen immediately but you need more than seconds to navigate to a desired screen.
	On hovering over the icon it shows the name of the icon - Locate Images Once you outline an area using Capture Image the next step is to Locate Images. Clicking on Locate images icon searches for matching patterns(/images) based on the Matching Pattern selection from: Retrieve Single closest match Retrieve Multiple close matches Retrieve All matches

9. Let us Capture Image on the paint document as shown in the screenshot below. Let us assume the image on the top left corner to be the base image with a size 100%.  The images on the right are smaller images with 10% reduction in size each. The images in the bottom row are in ascending order of size ascending by 10% each.
   

10. Once you capture an image or pattern the AutomationEdge Surface Spy window expands as shown below. In this case we have captured the third image from left in the top line.  
    

11. Generate Step configuration screen options differ depending on the Action chosen.
    
12. The following screen shows the Generate Step configuration options when Action chosen is Click as seen above also.
    

13. The following screen shows the Generate Step configuration options when Action chosen is Action Set Value
    

14. The following screen shows the Generate Step configuration options when Action chosen is Action Get Value
    

15. The following screen shows the Generate Step configuration options when Action chosen is Action Find Image.
    

16. The following table summarizes all the Generate step options for all Actions types.
    

    Table 2 : Surface Spy Generate Step Options

No.	Option	Description
1	Action	The Action Drop Down has the following options available. No Click Types Description 1 Click It performs aclick action. 2 Set Value It performs Set Value on a region/image. 3 Get Value It Gets Value from a region/image. 4 Find Image It checks for an image and returns true or false.
2	Click Type	The Click Type Drop Down has the following options available. This option is available if Action chosen in Click. No Click Types Description 1 Click It performs left click action. 2 Right Click It performs right click action. 3 Double Click It performs double-click action. 4 Middle Click It performs left Middle click action. 5 Mouse Move It performs Mouse Move action.
3	Value	Specify a Value to be set on a region/image.
4	Output field name (for Action Get Value)	This field is active when Action chosen is Get Value.  Provide a field name to hold the value retrieved from a region.
5	Output field name (for Action Find Image)	This field is active when Action chosen is Find Image.  Provide a field name to hold a Boolean value(Y/N) indicating if the image was found or not.
6	Position	The Position Drop Down has the following options available. The Action chosen above is performed at the position selected here. No Positions Description 1 Left Center Left position of the Image. 2 Right Center Right position of the Image. 3 Center Center of the Image. 4 Custom Relative position from the entire Image. It could be anywhere on the screen. If the relative position of the relative element to click is closer it is preferable.
7	Image Index	Image Index is retrieved once an image is clicked from the selected images.
8	Timeout	This is the Timeout for step execution. By default timeout is 5 seconds
9	Match Pattern	The Match Pattern Drop Down has the following options available. Retrieve Single closest match Retrieve Multiple close match Retrieve All close match
10	Button: Generate	Click Generate button to generate the step with the configurations above.

17. The Match Pattern drop down field has the following options as shown below,
    

3. Retrieve Single closest match
   
4. Retrieve Multiple close match
   
5. Retrieve All close match
   

18. Once you outline an area using Capture Image the next step is to Locate Images. Clicking on Locate images icon searches for matching images (/patterns) depending on the Match Pattern chosen as mentioned above.
    
19. Let us select Retrieve Single closest match in the match Pattern drop down. Now click on Locate Images () icon. It gives an error Multiple Match Found as it has found more than one close matches. 
    

Note: Click on an image to select and press Esc key to exit.

 

20. Let us select Retrieve Multiple close matches in the match Pattern drop down. Now click on Locate Images () icon. It locates Multiple Matches as seen below.
    

21. Let us select Retrieve All matches in the match Pattern drop down. Now click on Locate Images   () icon. It locates All Matches as seen below (not below 50% or above 150% of the original size).
    

22. In this Appendix 3 we have seen how Surface Spy is used Capture and Locate Images and the various options available. 
    

Close Surface Spy

1. You may click the X button on the AutomationEdge Surface Spy window to close Surface Spy. 
   

 

2. You may also click X on the expanded AutomationEdge Surface Spy window to close Surface Spy.  
   

3. Surface Spy also closes when you close Process Studio.