Contents

WhiteLibrary Support

WhiteLibrary Support

RobotKernel can include dedicated support for selected libraries. One of these libraries is WhiteLibrary – the current library of choice for testing and automating Windows applications.

*** Settings ***

Library  WhiteLibrary

State management

RobotKernel remembers the current WhiteLibrary context between executions. That’s why it is possible to launch and attach application once and then move on to work on keywors testing or automating that application.

*** Keywords ***

Attach Notepad window
    Launch application  notepad.exe
    Attach application by name  notepad
    Attach window  title:Nimetön – Muistio

Element picker

RobotKernel has also an interactive built-in element picker to help selecting elements for WhiteLibrary keywords.

This element picker is activated by typing the special pseudo selector prefix ae: (automation element) and pressing <TAB> (similarly to triggering autocompletion suggestions in Jupyter Notebook and JupyterLab).

Once picker is activated, it changes mouse pointer into crosshair. After the desired element is clicked with that crosshair pointer, RobotKernel will suggest supported selector completions (like text:, id: or class_name:).

*** Keywords ***

Input into Notepad
    [Arguments]  ${text}=Here be dragons
    Input text to textbox  class_name=Edit  ${text}\n

Because not all elements, like menu items for example, can be selected with a simple click, there is one more thing: long press selection.

Once mouse button is kept pressed for a few seconds, single click is emulated to the underlying element. This should be enough to let the menu be opened. Then it is possible to keep the mouse button pressed, move its pointer above the desired menu element, wait a second to see the menu item focused, and finally release the mouse button to get supported selectors for the element.

*** Keywords ***

Clear notepad
    Click menu button  text=Muokkaa
    Click menu button  text=Valitse kaikki
    Click menu button  text=Muokkaa
    Click menu button  text=Poista

*** Keywords ***

Save file
    [Arguments]  ${filename}=hello-world.txt
    Click menu button  text=Tiedosto
    Click menu button  text=Tallenna nimellä...
    Input text to textbox  text=Tiedostonimi:  ${filename}
    Select listbox value  text=Koodaus:  Unicode
    Click button  text=Tallenna

OpenCV template grabber and matcher

Finally, RobotKernel ships with an experimental image recognition based Windows automation support based on OpenCV computer vision library, if that’s available on the current Python environment.

*** Settings ***

Library  WhiteLibrary
Library  robotkernel.WhiteLibraryCompanion

The special keyword library robotkernel.WhiteLibraryCompanion provides a single keyword: Click template with arguments template and similarity (with default value 0.95 to require exact match).

The magic is hidden within the keyword argument template as follows:

  1. type Click template  template= and

  2. press <TAB>.

Once again the mouse cursor is changed into crosshair, but this time it expects a rectangular area to be painted with dragging the default mouse button pressed. Once the mouse button is released:

  1. a screenshot is captured from the painted area

  2. it is saved in to the current working directory of Jupyter process and

  3. template= part of the keyword is replaced with template=${EXECDIR}\\[TIMESTAMP].png

and now the keyword is ready to be called. Call of the keyword captures screenshot of the current default screen, looks for captured area using OpenCV matchTemplate and click the center of the found match using WhiteLibrary mouse automation functions.

*** Keywords ***

Open calc
    Launch application  calc.exe
    Attach window  Laskin
    
1
    Click template  template=${EXECDIR}\\1554037865.png
    
+
    Click template  template=${EXECDIR}\\1554037877.png
    
2
    Click template  template=${EXECDIR}\\1554037888.png

previous

Prototyping Keywords

next

Prototyping Libraries