Windows Application GUI

Windows Application GUI



Windows Application GUI

Windows Application GUI plugin steps are used for Windows Applications automation.

Compatibility:

  1. Windows 7/8.1/10, Windows Server 2008/2012/2016


Prerequisites: 

  1. Microsoft .NET Framework 4.5 or above


Check Plugin Support:

  1. You may check if your target application elements are supported by Windows Application GUI Plugin steps.
  2. Verify the target application elements against “Accessibility Insights for Windows” application.
  3. You may check only for Element accessibility.  Choose criterion Element from the drop down list as seen in the image below


  1. Get more details in the following video, 

https://www.youtube.com/watch?v=BIu9ONdMGGg


Notes:

  1. You need to use Windows Spy to build a workflow with Windows Application GUI plugin steps. Refer Appendix 10: Windows Application GUI Tutorial (including best practices/workarounds) and a sample workflow with Windows Spy.
  2. AutomationEdge Workflows with Windows Application GUI steps should be marked Sequential. 



Following are the Plugin steps,


Element Action 

Description

Windows: Element Action step performs an action on the specified element (based on the Element Property chosen).  

Configurations 

No.

Field Name

Description

1

Step Name

Name of the step.  This name has to be unique in a single workflow.

2

ActionType

Possible Action Types are, 

  1. Get Property Value
  2. Wait Until (some condition (element property value) is satisfied).

3

Windows Path

Windows Path is the relative path of the desktop application on which element exists. It is captured by Windows Spy in the Generate Step dialog. It is then populated in the step configuration upon Step Generation. You may also enter or modify Windows path manually on the ‘Step Generation’ window or step configuration. 


Windows path may be provided as a String or regex. 

Sample of Windows Path as string:

w[Desktop]/w[Application Name] 

or

Sample of Windows Path as regex:

w[Desktop]/w[/Application N/], w[Desktop]/w[/Application .*/], 

If you are providing partial application name then put it as a regex between / and /. Also if the Application name changes dynamically it is a good idea to provide it as a regex.

4

Criteria

‘Criteria’ is the criterion to locate element on Windows Desktop. It is captured by Windows Spy in the Generate Step dialog. Upon Step Generation it is populated in the step configuration. You may change criteria by choosing from the drop down list in ‘Step Generation’ window or step configuration.


Note: It is similar as in WebGUI plugin steps.

5

Criteria Value

Specify relevant criteria value for the criteria provided above.

6

Element index(Starts with 0)

For Multiple elements with same criteria, specify index of the element you want to locate.

Provide index 0 if you want first element from the located elements, increment index by 1 for the next elements.

7

Element Property

The possible Element Properties are as follows, 

  1. Is Exist – for all elements, 
  2. Is Checked – for checkbox, 
  3. Is Visible – for all elements,
  4. Is Enable – for all elements,
  5. Is Selected – for all elements but specifically designed for radio button


Note: 

  1. Is Checked Element Property is not supported for Radio button elements.
  2. With element property – Is Exist or Is Visible avoid ‘Absolute Path’ criteria.
  3. In a workflow it is recommended to first check if element exists (Wait Until – Is Exist) followed by Get Property Values steps.

8

Reverse above Element Property Condition

Enable checkbox to reverse the Element Property chosen above.


Dynamic fields:


9

Output Field

This field is visible if Get Property Value is chosen in Action Type.

Specify an output field name to hold the output of step execution (The output is True/False).

10

Timeout (In Seconds)

This field is visible if Wait Until is chosen in Action Type.

Specify a Timeout value for the Element Property chosen.


Common Buttons:

No.

Field Name

Description


Buttons:


1

OK

On click of this button field values are checked. If any required field values are missing then validation error message is displayed.

If all the required field values are provided then it will save the field values.

2

Cancel

Click this button to cancel the window without saving any values.

3

Button: Edit Element

Click Edit Element to start spying and change step details as desired.

Opening spy from step is restricted to the scope of the step. i.e. Only relevant elements are detected with active spy. Use Hotkeys (‘Shift’ or ’~’)  to detect the element. We may call it as a ‘Step Restricted Spy’ for understanding purposes. e.g. Suppose spy is started from ‘Element Action’ step. Now spy is restricted to elements that support ‘Element Action’. All elements that support ‘Element Action’ will be highlighted in blue color and the rest will be highlighted in ‘Red’ color upon hover. Notice Step dropdown is disabled and value selected is ‘Element Action’


Get Value 

Description

Windows: Get Value plugin step gets value of element type textbox.

Configurations 

No.

Field Name

Description

1

Step Name

Name of the step.  This name has to be unique in a single workflow.


Read Parameters Tab: 

No.

Field Name

Description

1

Tabular Columns:


i

Windows Path

Windows Path is the relative path of the desktop application on which element exists. It is captured by Windows Spy in the Generate Step dialog. It is then populated in the step configuration upon Step Generation. You may also enter or modify Windows path manually on the ‘Step Generation’ window or step configuration. 


Windows path may be provided as a String or regex. 

Sample of Windows Path as string:

w[Desktop]/w[Application Name] 

or

Sample of Windows Path as regex:

w[Desktop]/w[/Application N/], w[Desktop]/w[/Application .*/], 

If you are putting partial application name then put ii as a regex between / and /. Also if the Application name changes dynamically it is a good idea to provide it as a regex.

ii

Criteria

‘Criteria’ is the criterion to locate element on Windows Desktop. It is captured by Windows Spy in the Generate Step dialog. Upon Step Generation it is populated in the step configuration. You may change criteria by choosing from the drop down list in ‘Step Generation’ window or step configuration.


Note: It is similar as in WebGUI plugin steps.

iii

Criteria Value

Specify relevant criteria value for the criteria provided above.

iv

Element index(Starts with 0)

For Multiple elements with same criteria, specify index of the element you want to locate.

Provide index 0 if you want first element from the located elements, increment index by 1 for the next elements.

V

Read Type

Choose a Read Types from the drop down list,

  1. ‘Value’ –from text box or
  2. ‘Text’    -from label


Note: Get value is not applicable to encrypted values like password fields.

vi

Output field name

Specify an Output field to hold the value of the textbox element.

2

Timeout(In seconds)

Specify a Timeout to find the element on the Desktop application.


Common Buttons:

No.

Field Name

Description


Buttons:


1

OK

On click of this button field values are checked. If any required field values are missing then validation error message is displayed.

If all the required field values are provided then it will save the field values.

2

Cancel

Click this button to cancel the window without saving any values.

3

Edit Element

Click Edit Element to start spying.

Opening spy from step is restricted to the step, i.e. Only relevant elements are detected from active spy. Use Hotkeys to detect the element (‘Shift’ or ’~’). We may call it as a ‘Step Restricted Spy’ for understanding purposes. E.g. Suppose spy is started from ‘Get Value’ step. Now spy is restricted to elements that support ‘Get Value’. All elements that are supported will be highlighted in blue color and others will be highlighted in ‘Red’ color upon hover. Step dropdown is disabled and value selected is ‘Get Value’. 


Launch Application 

Description

Windows Application GUI: Launch Application plugin step starts new application at runtime. 


Configurations 

Input Tab:

No.

Field Name

Description

1

Step Name

Name of the step.  This name has to be unique in a single workflow.

2

Path of Application

Specify the full path (including full path and filename with extention) of an application to launch. 

3

Button: Browse

Click the button to browse the Application path. 

4

Checkbox:

Fail if error occurs

Enable checkbox to enable the Properties for application validation below.


Properties for validation:

The following fields are enabled only if ‘Fail if error occurs’ checkbox is enabled. 

Provide values in the fields below to be validated.

5

Windows Path/Title

Windows Path and Title are explained below,

  1. Windows Path is the relative path of the desktop application on which element exists. Enter Windows path manually in Step Configuration. 


Windows path may be provided as a String or regex. 

Sample of Windows Path as string:

w[Desktop]/w[Application Name] 

or

Sample of Windows Path as regex:

w[Desktop]/w[/Application N/], w[Desktop]/w[/Application .*/], 

If you are putting partial application name then put ii as a regex between / and /. Also if the Application name changes dynamically it is a good idea to provide it as a regex.


  1. Window Title is the title of the Windows Application. (i.e. Application Name)


Note: 

  1. When specifying ‘Window Path/Title’ make sure that no other application, window or folder with the same title is open, else it may lead to confusion at runtime.
  2. If you get an error like “Window is not available or not loaded in specified time”, make sure that the Windows Path/Title, Path and application startup time are correctly specified.

6

Application startup time (Minutes)

Specify a waiting period in minutes for application startup as specified in the Windows Path/Title above.


Note: You may manually check time taken by the application to load and come up with a better approximation of Application startup time.

7

Checkbox: 

close application on failure

You may enable this checkbox to close the application if the validations above are not successful.


Common Buttons:

No.

Field Name

Description


Buttons:


1

OK

On click of this button field values are checked. If any required field values are missing, then validation error message is displayed.

If all the required field values are provided then it will save the field values.

2

Cancel

Click this button to cancel the window without saving any values.



Mouse Action 

Windows Application GUI: Mouse Action plugin step performs simple mouse actions such as hover, click etc. to complex operations such as drag and drop.

Configurations 

Input Tab:

No.

Field Name

Description

1

Step Name

Name of the step.  This name has to be unique in a single workflow.

2

Action Type

Select an Action Type from the list below,



  1. Hover, Click, Double Click, Right Click actions are available with all element types.
  2. Expand, Collapse are useful for tree items, Dropdown element. In case expand is not available you may use Click Action Type.
  3. Toggle is useful for setting value to checkbox.
  4. Scroll option is available for scrollable elements. 

You need to detect the immediate window containing the scroll bar, possibly by hovering on the intersection of the right and bottom scroll bars as marked in yellow below. 


  1. If Action type is Drag Drop first Spy the Source Element and generate step. In the step click on any field in the Target Element and click Edit Destination Element button to spy the destination element.


Note: In Mouse action In case drag drop is not working you may use the Set Value step.  


Source Element:


3

Windows Path

Windows Path is the relative path of the desktop application on which element exists. It is captured by Windows Spy in the Generate Step dialog. It is then populated in the step configuration upon Step Generation. You may also enter or modify Windows path manually on the ‘Step Generation’ window or step configuration. 


Windows path may be provided as a String or regex. 

Sample of Windows Path as string:

w[Desktop]/w[Application Name] 

or

Sample of Windows Path as regex:

w[Desktop]/w[/Application N/], w[Desktop]/w[/Application .*/], 

If you are putting partial application name then put ii as a regex between / and /. Also if the Application name changes dynamically it is a good idea to provide it as a regex.

4

Criteria

‘Criteria’ is the criterion to locate element on Windows Desktop. It is captured by Windows Spy in the Generate Step dialog. Upon Step Generation it is populated in the step configuration. You may change criteria by choosing from the drop down list in ‘Step Generation’ window or step configuration.


Note: It is similar as in WebGUI plugin steps.

5

Criteria Value

Specify relevant criteria value for the criteria provided above.

6

Element index(Starts with 0)

For Multiple elements with same criteria, specify index of the element you want to locate.

Provide index 0 if you want first element from the located elements, increment index by 1 for the next elements.





Destination Element:

The section is enabled only for the Action Type Drag Drop.

1

Windows Path

Windows Path is the relative path of the desktop application on which element exists. It is captured by Windows Spy in the Generate Step dialog. It is then populated in the step configuration upon Step Generation. You may also enter or modify Windows path manually on the ‘Step Generation’ window or step configuration. 


Windows path may be provided as a String or regex. 

Sample of Windows Path as string:

w[Desktop]/w[Application Name] 

or

Sample of Windows Path as regex:

w[Desktop]/w[/Application N/], w[Desktop]/w[/Application .*/], 

If you are putting partial application name then put ii as a regex between / and /. Also if the Application name changes dynamically it is a good idea to provide it as a regex.

2

Criteria

‘Criteria’ is the criterion to locate element on Windows Desktop. It is captured by Windows Spy in the Generate Step dialog. Upon Step Generation it is populated in the step configuration. You may change criteria by choosing from the drop down list in ‘Step Generation’ window or step configuration.


Note: It is similar as in WebGUI plugin steps.

3

Criteria Value

Specify relevant criteria value for the criteria provided above.

4

Element index(Starts with 0)

For Multiple elements with same criteria, specify index of the element you want to locate.

Provide index 0 if you want first element from the located elements, increment index by 1 for the next elements.

5

Timeout(In seconds)

Specify a Timeout to find the element on the Desktop application.


Common Buttons:

No.

Field Name

Description


Buttons:


1

OK

On click of this button field values are checked. If any required field values are missing then validation error message is displayed.

If all the required field values are provided then it will save the field values.

2

Cancel

Click this button to cancel the window without saving any values.

3

Button: 

Edit Source Element/

Edit Destination Element

Click Edit Element to start spying.


Either Edit Source Element or  Edit Destination Element button is visible depending on any field chosen on ‘Source element’ or Destination element’ group.


Opening spy from step is restricted to the step, i.e. Only relevant elements are detected from active spy. Use Hotkeys (‘Shift’ or ’~’) to detect the element. We may call it as a ‘Step Restricted Spy’ for understanding purposes.  E.g. is restricted to elements that support ‘Mouse Action’. All elements that are supported will be highlighted in blue color and others will be highlighted in ‘Red’ color upon hover. Step dropdown is disabled and value selected is ‘Mouse Action’.




Set Value 

Windows: Set Value plugin step sets value to element of type – Text box, Text area, document or pane. 


Configurations 

No.

Field Name

Description

1

Step Name

Name of the step.  This name has to be unique in a single workflow.


Write Parameter Tab 

No.

Field Name

Description

1

Tabular Columns:


i

Windows Path

Windows Path is the relative path of the desktop application on which element exists. It is captured by Windows Spy in the Generate Step dialog. It is then populated in the step configuration upon Step Generation. You may also enter or modify Windows path manually on the ‘Step Generation’ window or step configuration. 


Windows path may be provided as a String or regex. 

Sample of Windows Path as string:

w[Desktop]/w[Application Name] 

or

Sample of Windows Path as regex:

w[Desktop]/w[/Application N/], w[Desktop]/w[/Application .*/], 

If you are putting partial application name then put ii as a regex between / and /. Also if the Application name changes dynamically it is a good idea to provide it as a regex.

ii

Criteria

‘Criteria’ is the criterion to locate element on Windows Desktop. It is captured by Windows Spy in the Generate Step dialog. Upon Step Generation it is populated in the step configuration. You may change criteria by choosing from the drop down list in ‘Step Generation’ window or step configuration.


Note: It is similar as in WebGUI plugin steps.

iii

Criteria Value

Specify relevant criteria value for the criteria provided above.

iv

Element index(Starts with 0)

For Multiple elements with same criteria, specify index of the element you want to locate.

Provide index 0 if you want first element from the located elements, increment index by 1 for the next elements.

v

Write Type

Specify a Write type – Append Text or Set New Text

vi

Input Text

Specify the text to set value of the element- textbox or  text area

2

Timeout(In Seconds)

Specify a Timeout to find the element on the Desktop application.


Common Buttons:

No.

Field Name

Description


Buttons:


1

OK

On click of this button field values are checked. If any required field values are missing then validation error message is displayed.

If all the required field values are provided then it will save the field values.

2

Cancel

Click this button to cancel the window without saving any values.

3

Edit Element

Click Edit Element to start spying.

Opening spy from step is restricted to the step, i.e. Only relevant elements are detected from active spy. Use Hotkeys (‘Shift’ or ’~’) to detect the element. We may call it as a ‘Step Restricted Spy’ for understanding purposes. E.g. Suppose spy is started from ‘Set Value’ step. Now spy is restricted to elements that support ‘Set Value’. All elements that are supported will be highlighted in blue color and others will be highlighted in ‘Red’ color upon hover. Step dropdown is disabled and value selected is ‘et Value’.



Windows Action 

Windows Application GUI: Windows Action plugin step does window related actions such as Maximize, Minimize, Normal and Close.

Note: 

  1. For Windows Action step you need to hover over the edges of the window to detect the window.
  2. Windows Action cannot be used, on pop-up windows, instead Mouse Action- Click, or Surface plugin –Surface Click or Robot Handing steps could be used.


Configurations 

Input Tab:

No.

Field Name

Description

1

Step Name

Name of the step.  This name has to be unique in a single workflow.

2

ActionType

Select an ActionType from one of the following,


3

Windows Path

Windows Path is the relative path of the desktop application on which element exists. It is captured by Windows Spy in the Generate Step dialog. It is then populated in the step configuration upon Step Generation. You may also enter or modify Windows path manually on the ‘Step Generation’ window or step configuration. 


Windows path may be provided as a String or regex. 

Sample of Windows Path as string:

w[Desktop]/w[Application Name] 

or

Sample of Windows Path as regex:

w[Desktop]/w[/Application N/], w[Desktop]/w[/Application .*/], 

If you are putting partial application name then put ii as a regex between / and /. Also if the Application name changes dynamically it is a good idea to provide it as a regex.

4

Timeout(In Seconds)

Specify a Timeout to find the element on the Desktop application.


Common Buttons:

No.

Field Name

Description


Buttons:


1

OK

On click of this button field values are checked. If any required field values are missing then validation error message is displayed.

If all the required field values are provided then it will save the field values.

2

Cancel

Click this button to cancel the window without saving any values.

3

Button: Edit Element

Click Edit Element to start spying.

Opening spy from step will be restricted to the step i.e. Only relevant elements will be detected from active spy. We can call it as a ‘Step Restricted Spy’ just for understanding purposes. Use Hotkeys (‘Shift’ or ’~’) to detect elements. E.g. Suppose spy is started from ‘Window Action’ step. Now spy is restricted to elements of type ‘Window’ only. All element of type ‘Window’ will be highlighted in blue color. If we try to detect element other than of type ‘Window’ it will be highlighted in ‘Red’ color upon hover. Notice Step dropdown is disabled and value selected is ‘Window Action’



Get Table Data [Beta]

Description

Windows: Get Table Data plugin step gets table from elements of type - List, Grid, Table or Data grid, filtered based on criteria provided.

Configurations 

No.

Field Name

Description

1

Step Name

Name of the step.  This name has to be unique in a single workflow.

2

Action Type

Select an Action Type from the drop down list.


3

Windows Path

Windows Path is the relative path of the desktop application on which element exists. It is captured by Windows Spy in the Generate Step dialog. It is then populated in the step configuration upon Step Generation. You may also enter or modify Windows path manually on the ‘Step Generation’ window or step configuration. 


Windows path may be provided as a String or regex. 

Sample of Windows Path as string:

w[Desktop]/w[Application Name] 

or

Sample of Windows Path as regex:

w[Desktop]/w[/Application N/], w[Desktop]/w[/Application .*/], 

If you are putting partial application name then put ii as a regex between / and /. Also if the Application name changes dynamically it is a good idea to provide it as a regex.

4

Criteria

‘Criteria’ is the criterion to locate element on Windows Desktop. It is captured by Windows Spy in the Generate Step dialog. Upon Step Generation it is populated in the step configuration. You may change criteria by choosing from the drop down list in ‘Step Generation’ window or step configuration.


Note: It is similar as in WebGUI plugin steps.

5

Criteria Value

Specify relevant criteria value for the criteria provided above.

6

Element index(Starts with 0)

For Multiple elements with same criteria, specify index of the element you want to locate.

Provide index 0 if you want first element from the located elements, increment index by 1 for the next elements.

7

Row Index

Specify the Row Index to fetch data.

Row index is enabled only if one of the following is chosen in Action Type above,

  1. One Row With Specific Columns
  2. One Cell

8

Get Values (Tabular):


i

Column Index

Specify the Column Indices to fetch data.

ii

Output Variable

Specify an output field to hold the value of Step execution.

9

Timeout(In seconds)

Specify a Timeout to find the element on the Desktop application.


Common Buttons:

No.

Field Name

Description


Buttons:


1

OK

On click of this button field values are checked. If any required field values are missing, then validation error message is displayed.

If all the required field values are provided then it will save the field values.

2

Cancel

Click this button to cancel the window without saving any values.

3

Button: Edit Element

Click Edit Element to start spying.

Opening spy from step is restricted to the step, i.e. Only relevant elements are detected from active spy. Use Hotkeys (‘Shift’ or ’~’) to detect the element. We may call it as a ‘Step Restricted Spy’ for understanding purposes. E.g. Suppose spy is started from ‘Get Table Data’ step. Now spy is restricted to elements of type ‘Table’ only. All element of type ‘Table’ will be highlighted in blue color. If we try to detect element other than of type ‘Table’ it will be highlighted in ‘Red’ color upon hover. Step dropdown is disabled and value selected is ‘Get Table Data’.







      Links to better reach 

            Bot Store

             EPD