Writing First Test

Let’s see how we can write a simple test that would open Google, search for a Mercury element, ensure that search results are loaded, include the Wikipedia article about the element, and check that its symbol is presented correctly on the page.

Note

Alumnium can be used with any test runner you like. For the purpose of simplicity, this guide uses a built-in unittest module.

Setup Browser

Start by creating a Python script file for your tests and instantiating a browser instance:

Selenium

google_test.py

import unittest
from selenium.webdriver import Chrome


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        driver = Chrome()
        driver.get("https://google.com")

    def test_search(self):
        pass

Playwright

google_test.py

import unittest
from playwright.sync_api import sync_playwright


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        page = sync_playwright().start().chromium.launch(headless=False).new_page()
        page.goto("https://google.com")

    def test_search(self):
        pass

Now run the test. You should see a Chrome browser window with Google opened.

Running test...

$ python -m unittest google_test.py
.
----------------------------------------------------------------------
Ran 1 test in 3.809s

OK

Setup Alumnium

Now let’s add the code that would instantiate Alumnium using the browser:

Selenium

google_test.py

import unittest
from alumnium import Alumni
from selenium.webdriver import Chrome


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        driver = Chrome()
        driver.get("https://google.com")
        self.al = Alumni(driver)

    def test_search(self):
        pass

Playwright

google_test.py

import unittest
from alumnium import Alumni
from playwright.sync_api import sync_playwright


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        page = sync_playwright().start().chromium.launch(headless=False).new_page()
        page.goto("https://google.com")
        self.al = Alumni(page)

    def test_search(self):
        pass

The test should still work fine, let’s re-run it to make sure:

Running test...

$ python -m unittest google_test.py
.
----------------------------------------------------------------------
Ran 1 test in 3.044s

OK

Note

If you are getting errors about missing API keys, make sure you configured Alumnium and try again!

Add Actions

Now let’s add some actions that Alumnium should do on the page. Our test needs to search for a Mercury element, so let’s use this exact command:

Selenium

google_test.py

import unittest
from alumnium import Alumni
from selenium.webdriver import Chrome


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        driver = Chrome()
        driver.get("https://google.com")
        self.al = Alumni(driver)

    def test_search(self):
        pass
        self.al.do("search for 'Mercury element'")

Playwright

google_test.py

import unittest
from alumnium import Alumni
from playwright.sync_api import sync_playwright


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        page = sync_playwright().start().chromium.launch(headless=False).new_page()
        page.goto("https://google.com")
        self.al = Alumni(page)

    def test_search(self):
        pass
        self.al.do("search for 'Mercury element'")

Run the test, you should now see “Mercury element” typed in the search box and the page with results loaded.

Running test...

$ python -m unittest google_test.py
.
----------------------------------------------------------------------
Ran 1 test in 4.133s

OK

Add Verifications

The next step is to add some verifications that Alumnium should check on the page. We are going to add two of them, one that checks that the page title contains the search keyword and one to see if the Wikipedia article is present in the results.

Common wisdom says to never trust a test you haven’t seen fail. Let’s add the first verification and see it fail!

Selenium

google_test.py

import unittest
from alumnium import Alumni
from selenium.webdriver import Chrome


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        driver = Chrome()
        driver.get("https://google.com")
        self.al = Alumni(driver)

    def test_search(self):
        self.al.do("search for 'Mercury element'")
        self.al.check("page title contains Alumnium word")

Playwright

google_test.py

import unittest
from alumnium import Alumni
from playwright.sync_api import sync_playwright


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        page = sync_playwright().start().chromium.launch(headless=False).new_page()
        page.goto("https://google.com")
        self.al = Alumni(page)

    def test_search(self):
        self.al.do("search for 'Mercury element'")
        self.al.check("page title contains Alumnium word")

Now let’s run our test:

Running test...

$ python -m unittest google_test.py
F
======================================================================
FAIL: test_search (test.TestGoogleSearch.test_search)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/alumnium/google_test.py", line 14, in test_search
    self.al.check("page title contains Alumnium word")
  File "/alumnium/alumnium/alumni.py", line 90, in check
    assert result.value, result.explanation
           ^^^^^^^^^^^^
AssertionError: The page title is 'Mercury element - Google Search', which does not contain the word 'Alumnium'.

----------------------------------------------------------------------
Ran 1 test in 7.599s

FAILED (failures=1)

Our test failed as we expected and provided a meaningful explanation of what went wrong.

Let’s fix the first check and add another one, expecting it to fail again:

Selenium

google_test.py

import unittest
from alumnium import Alumni
from selenium.webdriver import Chrome


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        driver = Chrome()
        driver.get("https://google.com")
        self.al = Alumni(driver)

    def test_search(self):
        self.al.do("search for 'Mercury element'")
        self.al.check("page title contains Alumnium word")
        self.al.check("page title contains Mercury word")
        self.al.check("search results do not contain Wikipedia articles")

Playwright

google_test.py

import unittest
from alumnium import Alumni
from playwright.sync_api import sync_playwright


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        page = sync_playwright().start().chromium.launch(headless=False).new_page()
        page.goto("https://google.com")
        self.al = Alumni(page)

    def test_search(self):
        self.al.do("search for 'Mercury element'")
        self.al.check("page title contains Alumnium word")
        self.al.check("page title contains Mercury word")
        self.al.check("search results do not contain Wikipedia articles")

Time to re-run the test:

Running test...

$ python -m unittest google_test.py
F
======================================================================
FAIL: test_search (test.TestGoogleSearch.test_search)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/alumnium/google_test.py", line 15, in test_search
    self.al.check("search results do not contain Wikipedia article")
  File "/alumnium/alumnium/alumni.py", line 90, in check
    assert result.value, result.explanation
           ^^^^^^^^^^^^
AssertionError: The ARIA tree contains multiple links to Wikipedia articles related to the Mercury element, indicating that the search results do include Wikipedia articles.

----------------------------------------------------------------------
Ran 1 test in 10.201s

FAILED (failures=1)

Ok, the test failed as we wanted it to, let’s fix it:

Selenium

google_test.py

import unittest
from alumnium import Alumni
from selenium.webdriver import Chrome


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        driver = Chrome()
        driver.get("https://google.com")
        self.al = Alumni(driver)

    def test_search(self):
        self.al.do("search for Mercury element")
        self.al.check("page title contains Mercury word")
        self.al.check("search results do not contain Wikipedia articles")
        self.al.check("search results contain Wikipedia article")

Playwright

google_test.py

import unittest
from alumnium import Alumni
from playwright.sync_api import sync_playwright


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        page = sync_playwright().start().chromium.launch(headless=False).new_page()
        page.goto("https://google.com")
        self.al = Alumni(page)

    def test_search(self):
        self.al.do("search for 'Mercury element'")
        self.al.check("page title contains Mercury word")
        self.al.check("search results do not contain Wikipedia articles")
        self.al.check("search results contain Wikipedia article")

Now, re-run to make sure it passes:

Running test

$ python -m unittest google_test.py
.
----------------------------------------------------------------------
Ran 1 test in 14.088s

OK

Add Data Retrieval

Finally, let’s verify some data that Alumnium can get from the page. We are going to ensure the Mercury element card shows its symbol correctly.

Let’s add a failing verification first:

Selenium

google_test.py

import unittest
from alumnium import Alumni
from selenium.webdriver import Chrome


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        driver = Chrome()
        page.goto("https://duckduckgo.com")
        self.al = Alumni(driver)

    def test_search(self):
        self.al.do("search for 'Mercury element'")
        self.al.check("page title contains Mercury word")
        self.al.check("search results contain Wikipedia article")
        symbol = self.al.get("chemical symbol")
        self.assertEqual(symbol, "Al")

Playwright

google_test.py

import unittest
from alumnium import Alumni
from playwright.sync_api import sync_playwright


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        page = sync_playwright().start().chromium.launch(headless=False).new_page()
        page.goto("https://duckduckgo.com")
        self.al = Alumni(page)

    def test_search(self):
        self.al.do("search for 'Mercury element'")
        self.al.check("page title contains Mercury word")
        self.al.check("search results contain Wikipedia article")
        symbol = self.al.get("chemical symbol")
        self.assertEqual(symbol, "Al")

Time to re-run the test:

Running test

$ python -m unittest google_test.py
F
======================================================================
FAIL: test_search (test.TestGoogleSearch.test_search)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/alumnium/google_test.py", line 17, in test_search
    self.assertEqual(symbol, "Al")
AssertionError: 'Hg' != 'Al'
- Hg
+ Al


----------------------------------------------------------------------
Ran 1 test in 14.394s

FAILED (failures=1)

The test failed as expected, let’s fix it and re-run to make sure it’s passing:

Selenium

google_test.py

import unittest
from alumnium import Alumni
from selenium.webdriver import Chrome


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        driver = Chrome()
        driver.get("https://google.com")
        self.al = Alumni(driver)

    def test_search(self):
        self.al.do("search for 'Mercury element'")
        self.al.check("page title contains Mercury word")
        self.al.check("search results contain Wikipedia article")
        symbol = self.al.get("chemical symbol")
        self.assertEqual(symbol, "Al")
        self.assertEqual(symbol, "Hg")

Playwright

google_test.py

import unittest
from alumnium import Alumni
from playwright.sync_api import sync_playwright


class TestGoogleSearch(unittest.TestCase):
    def setUp(self):
        page = sync_playwright().start().chromium.launch(headless=False).new_page()
        page.goto("https://google.com")
        self.al = Alumni(page)

    def test_search(self):
        self.al.do("search for 'Mercury element'")
        self.al.check("page title contains Mercury word")
        self.al.check("search results contain Wikipedia article")
        symbol = self.al.get("chemical symbol")
        self.assertEqual(symbol, "Al")
        self.assertEqual(symbol, "Hg")

Running test

$ python -m unittest google_test.py
.
----------------------------------------------------------------------
Ran 1 test in 15.162s

OK

Congratulations, we have completed our first test!