findImage

ScreenRectangle findImage(imageFile, [parameterMap], [searchRegion])

This function will perform a search of the template image as stored in imageFile on the display also showing the currently running application. On success the location of the found image is returned as a ScreenRectangle. Functions like mouseClick() or tapObject() can then emulate a user interaction on the detected location.

If a directory is passed for imageFile, it is treated as an Image Group.

The search for the sub-image is performed left to right, top to bottom. The first successful match is returned. For selection of other matches use the occurrence parameter as described below.

All available screens making up the desktop will be searched. At least on Windows and X11-based systems. For searches on multi-screen setup on macOS please inquire with technical support.

The search behavior can be configured through entries passed via the optional parameterMap argument.

• occurrence - The occurrence index. By default the first match (with index 1) will be returned. Index values 2 and higher will select consecutive matches as far as available.
• tolerant - Boolean switch that enables tolerant image search mode. In this mode some differences between the template image and the desktop screenshot are allowed. The default value is the value of testSettings.imageSearchTolerant property. The comparison algorithm used is Correlation.
• threshold - Threshold on the correlation between template image and a desktop screenshot, expressed as a percentage. A correlation higher than the threshold is considered a match against the template image. Usable values for this parameter usually fall between 99.0 and 100.0. The default value is the value of the testSettings.imageSearchThreshold property. This parameter is ignored if the tolerant parameter is set to false.
• multiscale - Boolean switch that enables scaled image search mode. In this mode the template image is expanded into a series of images of different sizes. Those image are subsequently used as template images for the image search. The default value is the value of the testSettings.imageSearchMultiscale property. This parameter is ignored if the tolerant parameter is set to false.
• minScale - Lower bound on the template image series sizes, expressed as a percentage. The resized template image sizes will span between minScale and maxScale of the original image size. The default value is the value of the testSettings.imageSearchMinScale property. This parameter is ignored if either the tolerant or the multiscale parameters are set to false.
• maxScale Upper bound on the template image series sizes, expressed as a percentage. The resized template image sizes will span between minScale and maxScale of the original image size. The default value is the value of the testSettings.imageSearchMaxScale property. This parameter is ignored if either the tolerant or the multiscale parameters are set to false.

# Click on icon.png
mouseClick( findImage("icon.png") )
# Click on second occurrence of icon.png
mouseClick( findImage("icon.png", {"occurrence": 2}) )

// Click on icon.png
mouseClick( findImage("icon.png") );
// Click on second occurrence of icon.png
mouseClick( findImage("icon.png", {"occurrence": 2}) );

# Click on icon.png
mouseClick( findImage("icon.png") );
# Click on second occurrence of icon.png
mouseClick( findImage("icon.png", {"occurrence" => 2}) );

# Click on icon.png
mouseClick( findImage("icon.png") )
# Click on second occurrence of icon.png
mouseClick( findImage("icon.png", {"occurrence" => 2}) )

# Click on icon.png
invoke mouseClick [waitForImage "icon.png"]
# Click on second occurrence of icon.png
invoke mouseClick [waitForImage "icon.png" {occurrence 2 timeout 10000}]

This function will perform a single check of the screen content only. If it fails to find the sub-image an exception will be thrown. A call is useful in case a) the presence of the sub-image is almost certain or b) a quick check of the presence is wanted. In all other cases the ScreenRectangle waitForImage(imageFile, [parameterMap], [searchRegion]) function is preferred for timing-independent look-ups.

The matching algorithm performs a pixel-by-pixel color value comparison. The screen content and sub-image content therefore have to match 100% unless the tolerant search mode is enabled.

# Click on icon.png
mouseClick( findImage("icon.png", {'tolerant': True,
                                   'threshold': 99.7,
                                   'multiscale': True,
                                   'minScale': 100.0,
                                   'maxScale': 150.0}) )

// Click on icon.png
mouseClick( findImage("icon.png", {tolerant: true,
                                   threshold: 99.7,
                                   multiscale: true,
                                   minScale: 100.0,
                                   maxScale: 150.0}) );

# Click on icon.png
mouseClick( findImage("icon.png", {tolerant => true,
                                   threshold => 99.7,
                                   multiscale => true,
                                   minScale => 100.0,
                                   maxScale => 150.0}) );

# Click on icon.png
mouseClick( findImage("icon.png", {"tolerant" => true,
                                   "threshold" => 99.7,
                                   "multiscale" => true,
                                   "minScale" => 100.0,
                                   "maxScale" => 150.0}) )

# Check whether icon.png is displayed on screen
invoke mouseClick [ findImage "icon.png" {tolerant true \
                                          threshold 99.7 \
                                          multiscale true \
                                          minScale 100.0 \
                                          maxScale 150.0} ]

In case a searchRegion is specified as the last argument, the search is performed only within the area corresponding to it. The search region can be an AUT object, a ScreenRectangle object or an array of such objects. It can also be a special string value "all", in which case all top-level objects are used.

The searchRegion can also be an Image Object object. In such case no new screenshots are taken and the specified image is used instead. Currently it is not possible to limit the search scope to a portion of an Image object; please use the Image.copy() to cut the relevant section.

# Find icon.png on the MyWidget object
findImage("icon.png", {}, waitForObject( names.MyWidget ) )

# Find icon.png on multiple objects
findImage("icon.png", {}, [ waitForObject( names.MyWidget ),
                            waitForObject( names.MyOtherWidget ) ] )

# Find icon.png on the left half of the MyWidget object
bounds = object.globalBounds( waitForObject( names.MyWidget ) )
bounds.width = bounds.width / 2
findImage("icon.png", {}, bounds )

# Find icon.png in the specified area
findImage("icon.png", {}, UiTypes.ScreenRectangle( 100, 100, 200, 300 ) )

# Find icon.png on all top-level objects
findImage("icon.png", {}, "all" )

// Find icon.png on the MyWidget object
findImage("icon.png", {}, waitForObject( names.MyWidget ) );

// Find icon.png on multiple objects
findImage("icon.png", {}, [ waitForObject( names.MyWidget ),
                            waitForObject( names.MyOtherWidget ) ] );

// Find icon.png on the left half of the MyWidget object
var bounds = object.globalBounds( waitForObject( names.MyWidget ) );
bounds.width = bounds.width / 2
findImage("icon.png", {}, bounds );

// Find icon.png in the specified area
findImage("icon.png", {}, UiTypes.ScreenRectangle( 100, 100, 200, 300 ) );

// Find icon.png on all top-level objects
findImage("icon.png", {}, "all" );

# Find icon.png on the MyWidget object
findImage("icon.png", {}, waitForObject( names::MyWidget ) );

# Find icon.png on multiple objects
findImage("icon.png", {}, ( waitForObject( names::MyWidget ),
                            waitForObject( names::MyOtherWidget ) ) );

# Find icon.png on the left half of the MyWidget object
my $bounds = object::globalBounds( waitForObject( names::MyWidget ) );
$bounds->width = $bounds->width / 2;
findImage("icon.png", {}, bounds );

# Find icon.png in the specified area
findImage("icon.png", {}, new UiTypes::ScreenRectangle( 100, 100, 200, 300 ) );

# Find icon.png on all top-level objects
findImage("icon.png", {}, "all" )

# Find icon.png on the MyWidget object
findImage("icon.png", {}, waitForObject( names.MyWidget ) );

# Find icon.png on multiple objects
findImage("icon.png", {}, [ waitForObject( names.MyWidget ),
                            waitForObject( names.MyOtherWidget ) ] );

# Find icon.png on the left half of the MyWidget object
bounds = Squish::Object.globalBounds( waitForObject( names.MyWidget ) );
bounds.width = bounds.width / 2
findImage("icon.png", {}, bounds );

# Find icon.png in the specified area
findImage("icon.png", {}, UiTypes::ScreenRectangle.new( 100, 100, 200, 300 ) );

# Find icon.png on all top-level objects
findImage("icon.png", {}, "all" );

# Find icon.png on the MyWidget object
findImage "icon.png" {} [ waitForObject $names::MyWidget ]

# Find icon.png on multiple objects
findImage "icon.png" {} { [ waitForObject $names::MyWidget ],
                          [ waitForObject $names::MyOtherWidget ] }

# Find icon.png on the left half of the MyWidget object
set bounds [object globalBounds [ waitForObject $names::MyWidget ] ]
property set $bounds width [ expr [ property get $bounds width ] / 2 ]
findImage "icon.png" {} $bounds

# Find icon.png in the specified area
findImage "icon.png" {} [ construct ScreenRectangle 100 100 200 300 ]

# Find icon.png on all top-level objects
findImage "icon.png" {} "all"