findOcrText

ScreenRectangle findOcrText(text, [parameterMap], [searchRegion])

This function will perform optical character recognition on the display also showing the currently running application and search for the specific text within the result set. It will return a ScreenRectangle object corresponding to the location of the text.

The OCR will be performed over all available screens making up the desktop. At least on Windows and X11-based systems. For OCR on multi-screen setup on macOS please inquire with technical support.

The search behavior can be configured through entries passed via the optional parameterMap argument.

• occurrence - The occurrence index. By default the first text match (with index 1) will be returned. Index values 2 and higher will select consecutive matches as far as available.
• language - The language hint for OCR engine. Use an empty string to use the OCR engine defaults. The default value is the value of testSettings.defaultOcrLanguage property.
• profiles - An array of OCR engine specific profile names. This parameter is used only with engines that support profiles.
• options - A map of engine-specific option names and values. These options are passed to the OCR engine after loading the specified profiles.
• scaleFactor - Image up-scale factor. The source image is upscaled by that factor before it is passed to the OCR engine. The default value is 3.5. It can be set to a lower value to improve the OCR performance in case the text is rendered with a large font.

# Click on text
mouseClick( findOcrText( "Test Text" ) )
# Click on text in German
mouseClick( findOcrText( "Test Text", { "language": "German" } ) )

// Click on text
mouseClick( findOcrText( "Test Text" ) );
// Click on text in German
mouseClick( findOcrText( "Test Text", { language: "German" } ) );

# Click on text
mouseClick( findOcrText( "Test Text" ) );
# Click on text in German
mouseClick( findOcrText( "Test Text", { "language" => "German" } ) );

# Click on text
mouseClick( findOcrText( "Test Text" ) )
# Click on text in German
mouseClick( findOcrText( "Test Text", { "language" => "German" } ) )

# Click on text
invoke mouseClick[ findOcrText "Test Text" ]
# Click on text in German
invoke mouseClick[ findOcrText "Test Text" { language "German" } ]

This function will perform a single OCR of the screen content only. If it fails to find the specified text an exception will be thrown. A call is useful in case a) the presence of the text is almost certain or b) a quick check of the presence is wanted. In all other cases the ScreenRectangle waitForOcrText(text, [parameterMap], [searchRegion]) function is preferred for timing-independent look-ups.

In case a searchRegion is specified as the last argument, the OCR is performed only within the area corresponding to it. The search region can be an AUT object, a ScreenRectangle object or an array of such objects. It can also be a special string value "all", in which case all top-level objects are used.

The searchRegion can also be an Image Object object. In such case no new screenshots are taken and the specified image is used instead. Currently it is not possible to limit the OCR scope to a portion of an Image object; please use the Image.copy() to cut the relevant section.

# Click on text on the MyWidget object
mouseClick( findOcrText( "Click me", {}, waitForObject( names.MyWidget ) )

# Click on text on multiple objects
mouseClick( findOcrText( "Click me", {}, [ waitForObject( names.MyWidget ),
                                                   waitForObject( names.MyOtherWidget ) ] ) )

# Click on text on the left half of the MyWidget object
bounds = object.globalBounds( waitForObject( names.MyWidget ) )
bounds.width = bounds.width / 2
mouseClick( findOcrText( "Click me", {}, bounds ) )

# Click on text in the specified area
mouseClick( findOcrText( "Click me", {}, UiTypes.ScreenRectangle( 100, 100, 200, 300 ) ) )

# Click on text on all top-level objects
mouseClick( findOcrText( "Click me", {}, "all" ) )

# Click on text on the MyWidget object
mouseClick( findOcrText( "Click me", {}, waitForObject( names.MyWidget ) ) );

// Click on text on multiple objects
mouseClick( findOcrText( "Click me", {}, [ waitForObject( names.MyWidget ),
                                                   waitForObject( names.MyOtherWidget ) ] ) );

// Click on text on the left half of the MyWidget object
var bounds = object.globalBounds( waitForObject( names.MyWidget ) );
bounds.width = bounds.width / 2
mouseClick( findOcrText( "Click me", {}, bounds ) );

// Click on text in the specified area
mouseClick( findOcrText( "Click me", {}, new UiTypes.ScreenRectangle( 100, 100, 200, 300 ) ) );

// Click on text on all top-level objects
mouseClick( findOcrText( "Click me", {}, "all" ) );

# Click on text on the MyWidget object
mouseClick( findOcrText( "Click me", {}, waitForObject( names::MyWidget ) ) );

# Click on text on multiple objects
mouseClick( findOcrText( "Click me", {}, ( waitForObject( names::MyWidget ),
                                                   waitForObject( names::MyOtherWidget ) ) ) );

# Click on text on the left half of the MyWidget object
my $bounds = object::globalBounds( waitForObject( names::MyWidget ) );
$bounds->width = $bounds->width / 2;
mouseClick( findOcrText( "Click me", {}, bounds ) );

# Click on text in the specified area
mouseClick( findOcrText( "Click me", {}, new UiTypes::ScreenRectangle( 100, 100, 200, 300 ) ) );

# Click on text on all top-level objects
mouseClick( findOcrText( "Click me", {}, "all" ) )

# Click on text on the MyWidget object
mouseClick( findOcrText( "Click me", {}, waitForObject( names.MyWidget ) ) );

# Click on text on multiple objects
mouseClick( findOcrText( "Click me", {}, [ waitForObject( names.MyWidget ),
                                                   waitForObject( names.MyOtherWidget ) ] ) );

# Click on text on the left half of the MyWidget object
bounds = Squish::Object.globalBounds( waitForObject( names.MyWidget ) );
bounds.width = bounds.width / 2
mouseClick( findOcrText( "Click me", {}, bounds ) );

# Click on text in the specified area
mouseClick( findOcrText( "Click me", {}, UiTypes::ScreenRectangle.new( 100, 100, 200, 300 ) ) );

# Click on text on all top-level objects
mouseClick( findOcrText( "Click me", {}, "all" ) );

# Click on text on the MyWidget object
invoke mouseClick [ findOcrText "Click me" {} [ waitForObject $names::MyWidget ] ]

# Click on text on multiple objects
invoke mouseClick [ findOcrText "Click me" {} { [ waitForObject $names::MyWidget ],
                                                        [ waitForObject $names::MyOtherWidget ] } ]

# Click on text on the left half of the MyWidget object
set bounds [object globalBounds [ waitForObject $names::MyWidget ] ]
property set $bounds width [ expr [ property get $bounds width ] / 2 ]
invoke mouseClick [ findOcrText "Click me" {} $bounds ]

# Click on text in the specified area
invoke mouseClick [ findOcrText "Click me" {} [ construct ScreenRectangle 100 100 200 300 ] ]

# Click on text on all top-level objects
invoke mouseClick [ findOcrText "Click me" {} "all" ]