Checking Verifications

Alumnium can verify statements on the web page when you instruct it to check something. It analyzes the current state of the web page, optionally including its screenshot, and decides whether the verification successfully passes or fails.

For example, after performing a Google search for “Mercury element”, you might want to check that search results contain a link to the Wikipedia:

al.check("search results contain Wikipedia article")

If the search results don’t contain a link to Wikipedia, Alumnium raises an assertion error and explains:

AssertionError: The search results do not include Wikipedia article. The results shown in the ARIA tree include links to 'foobar2000', 'selenium.dev', and other related topics, but there is no mention of 'Wikipedia'.

Specific Verifications

Similarly to actions, Alumnium works better when the verifications are concrete.

For example, if you are writing a test for completing tasks in the To-Do application and you don’t check the exact task’s state, you might end up with contradictory verifications passing at the same time!

al.check("task is completed")
al.check("task is not completed")

To avoid false positives in the tests, write more concrete verifications:

al.check("task 'buy milk' is completed")
al.check("task 'buy bread' is not completed")

Vision

Caution

Vision verifications are significantly more expensive and slower, so resort to them only when needed.

Note

Meta Llama 3.2 does not support vision verifications yet.

Note

A screenshot is taken for the visible part of the page.

Occasionally, the web page state is not enough for Alumnium to perform the check. In this case, instruct it to take a screenshot of the page and include it in the verification decision. This is useful when you need to check the visual representation of elements or their spatial relationships.

For example, in your test for the To-Do application, you might need a check that a completed task is shown with a strikethrough style.

al.check("'buy milk' title font style is strikethrough")

Without a screenshot, this assertion can fail because there is no indication of font style in the web page itself:

AssertionError: The ARIA tree does not provide any information regarding the font style of the 'buy milk' task, including whether it is strikethrough or not.

To make the check more reliable, add a screenshot to it:

al.check("'buy milk' title font style is strikethrough", vision=True)

Flakiness

Alumnium automatically waits for the following conditions before attempting to check any verification:

1. The HTML document is loaded and ready.
2. Resources on the page are loaded.
3. Document mutations are finished.
4. XHR/fetch requests are finished.