Writing First Test with Appium
Let’s see how we can write a simple test that would use Appium to open DuckDuckGo, search for a Mercury element, ensure that search results are loaded, include the Wikipedia article about the element, and check that its symbol is presented correctly on the page.
Install Dependencies
Section titled “Install Dependencies”Make sure you have Alumnium installed in your project.
plugins { id 'java'}
java { toolchain { languageVersion = JavaLanguageVersion.of(21) }}
repositories { mavenCentral()}
dependencies { testImplementation 'ai.alumnium:alumnium:0.21.0' testRuntimeOnly 'ai.alumnium:alumnium-cli-darwin-arm64:0.21.0' // adjust as needed testImplementation 'io.appium:java-client:10.1.1' testImplementation 'org.junit.jupiter:junit-jupiter:5.11.4' testRuntimeOnly 'org.junit.platform:junit-platform-launcher'}
tasks.withType(Test).configureEach { useJUnitPlatform()}Now let’s install Appium server to communicate with the mobile device. You can do it using npm in a terminal:
$ npm install -g appium$ appium driver install xcuitest$ pip install alumnium pytestNow let’s start Appium server to communicate with the mobile device. You can do it using npm in a terminal:
$ npx appium driver install xcuitest$ npx appium$ npm install alumniumNow initialize WebdriverIO in your project if you haven’t done it yet. Select the options to test mobile applications on iOS platform with Mocha framework. You don’t need to use visual testing and it’s better to remove autogenerated test files.
$ npm init wdio$ rm test/specs/test.e2e.tsSetup Browser
Section titled “Setup Browser”Start by creating a test file for your tests and instantiating a browser instance:
import io.appium.java_client.ios.IOSDriver;import io.appium.java_client.ios.options.XCUITestOptions;import org.junit.jupiter.api.AfterEach;import org.junit.jupiter.api.BeforeEach;import org.junit.jupiter.api.Test;
class SearchTest { private IOSDriver driver;
@BeforeEach void setUp() throws Exception { XCUITestOptions options = new XCUITestOptions() .setBundleId("com.apple.mobilesafari") .setDeviceName("iPhone 17") .setPlatformVersion("26.5") .setNoReset(true); driver = new IOSDriver(options); }
@AfterEach void tearDown() { driver.quit(); }
@Test void shouldSearch() { driver.get("https://duckduckgo.com"); }}Now run the test. You should see an iOS simulator with a browser window with DuckDuckGo opened.
$ ./gradlew test --info...BUILD SUCCESSFUL in 5sfrom appium.options.ios import XCUITestOptionsfrom appium.webdriver.webdriver import WebDriverfrom pytest import fixture
@fixturedef driver(): options = XCUITestOptions() options.automation_name = "XCUITest" options.bundle_id = "com.apple.mobilesafari" options.device_name = "iPhone 17" options.no_reset = True options.platform_name = "iOS" options.platform_version = "26.5" driver = WebDriver(options=options) yield driver
def test_search(driver: WebDriver): driver.get("https://duckduckgo.com")Now run the test. You should see an iOS simulator with a browser window with DuckDuckGo opened.
$ pytest --quiet...1 passed in 3.73simport { browser, expect } from "@wdio/globals";
describe("TestSearch", () => { it("should search", async () => { await browser.url("https://duckduckgo.com"); });});You might need to adjust your wdio.conf.ts file to include the desired capabilities for iOS Safari:
export const config: WebdriverIO.Config = { // other options capabilities: [ { platformName: "iOS", browserName: "Safari", "appium:automationName": "XCUITest", "appium:deviceName": "iPhone 17", "appium:platformVersion": "26.5", }, ], // other options};Now run the test. You should see an iOS simulator with a browser window with DuckDuckGo opened.
$ npx wdio run wdio.conf.ts
[Safari iOS #0-0] Running: Safari on iOS[Safari iOS #0-0] Session ID: 29eb39f3-1ca6-4349-bb07-6ea545e8a3e9[Safari iOS #0-0][Safari iOS #0-0] » test/specs/search.e2e.ts[Safari iOS #0-0] TestSearch[Safari iOS #0-0] ✓ should search[Safari iOS #0-0][Safari iOS #0-0] 1 passing (1.6s)
Spec Files: 1 passed, 1 total (100% completed) in 00:00:09Setup Alumnium
Section titled “Setup Alumnium”Now let’s add the code that would instantiate Alumnium using the browser. We can use a test fixture to ensure proper setup and teardown of the Alumnium instance:
import ai.alumnium.Alumni;import io.appium.java_client.ios.IOSDriver;import io.appium.java_client.ios.options.XCUITestOptions;import org.junit.jupiter.api.AfterEach;import org.junit.jupiter.api.BeforeEach;import org.junit.jupiter.api.Test;
class SearchTest { private Alumni al; private IOSDriver driver;
@BeforeEach void setUp() throws Exception { XCUITestOptions options = new XCUITestOptions() .setBundleId("com.apple.mobilesafari") .setDeviceName("iPhone 17") .setPlatformVersion("26.5") .setNoReset(true); driver = new IOSDriver(options); al = new Alumni(driver); }
@AfterEach void tearDown() { driver.quit(); al.quit(); }
@Test void shouldSearch() { driver.get("https://duckduckgo.com"); }}Alumni shuts down both the AI session and the wrapped driver when quit() is called, so an explicit driver.quit() is no longer needed.
The test should still work fine, let’s re-run it to make sure:
$ ./gradlew test --info...BUILD SUCCESSFUL in 9sfrom alumnium import Alumnifrom appium.options.ios import XCUITestOptionsfrom appium.webdriver.webdriver import WebDriverfrom pytest import fixture
@fixturedef driver(): options = XCUITestOptions() options.automation_name = "XCUITest" options.bundle_id = "com.apple.mobilesafari" options.device_name = "iPhone 17" options.no_reset = True options.platform_name = "iOS" options.platform_version = "26.5" driver = WebDriver(options=options) yield driver
@fixturedef al(driver: WebDriver): al = Alumni(driver) yield al al.quit()
def test_search(al: Alumni, driver: WebDriver): driver.get("https://duckduckgo.com")The test should still work fine, let’s re-run it to make sure:
$ pytest --quiet...1 passed in 3.70simport { Alumni } from "alumnium";import { browser, expect } from "@wdio/globals";
describe("TestSearch", () => { let al: Alumni;
before(async () => { al = new Alumni(browser); });
after(async () => { await al.quit(); });
it("should search", async () => { await browser.url("https://duckduckgo.com"); });});The test should still work fine, let’s re-run it to make sure:
$ npx wdio run wdio.conf.ts...Spec Files: 1 passed, 1 total (100% completed) in 00:00:08Add Actions
Section titled “Add Actions”Now let’s add some actions that Alumnium should do on the page. Our test needs to search for a Mercury element, so let’s use this exact command:
import ai.alumnium.Alumni;import io.appium.java_client.ios.IOSDriver;import io.appium.java_client.ios.options.XCUITestOptions;import org.junit.jupiter.api.AfterEach;import org.junit.jupiter.api.BeforeEach;import org.junit.jupiter.api.Test;
class SearchTest { private Alumni al; private IOSDriver driver;
@BeforeEach void setUp() throws Exception { XCUITestOptions options = new XCUITestOptions() .setBundleId("com.apple.mobilesafari") .setDeviceName("iPhone 17") .setPlatformVersion("26.5") .setNoReset(true); driver = new IOSDriver(options); al = new Alumni(driver); }
@AfterEach void tearDown() { al.quit(); }
@Test void shouldSearch() { driver.get("https://duckduckgo.com"); al.act("search for 'Mercury element' and press Enter"); }}The Java client uses act because do is a reserved word in the language.
Run the test, you should now see “Mercury element” typed in the search box and the page with results loaded.
$ ./gradlew test --info...BUILD SUCCESSFUL in 52sfrom alumnium import Alumnifrom appium.options.ios import XCUITestOptionsfrom appium.webdriver.webdriver import WebDriverfrom pytest import fixture
@fixturedef driver(): options = XCUITestOptions() options.automation_name = "XCUITest" options.bundle_id = "com.apple.mobilesafari" options.device_name = "iPhone 17" options.no_reset = True options.platform_name = "iOS" options.platform_version = "26.5" driver = WebDriver(options=options) yield driver
@fixturedef al(driver: WebDriver): al = Alumni(driver) yield al al.quit()
def test_search(al: Alumni, driver: WebDriver): driver.get("https://duckduckgo.com") al.do("search for 'Mercury element' and press Enter")Run the test, you should now see “Mercury element” typed in the search box and the page with results loaded.
$ pytest --quiet...1 passed in 26.88simport { Alumni } from "alumnium";import { browser, expect } from "@wdio/globals";
describe("TestSearch", () => { let al: Alumni;
before(async () => { al = new Alumni(browser); });
after(async () => { await al.quit(); });
it("should search", async () => { await browser.url("https://duckduckgo.com"); await al.do("search for 'Mercury element' and press Enter"); });});Run the test, you should now see “Mercury element” typed in the search box and the page with results loaded.
$ npx wdio run wdio.conf.ts...Spec Files: 1 passed, 1 total (100% completed) in 00:00:33Add Verifications
Section titled “Add Verifications”The next step is to add some verifications that Alumnium should check on the page. We are going to add two of them, one that checks that the page title contains the search keyword and one to see if the Wikipedia article is present in the results.
Common wisdom says to never trust a test you haven’t seen fail. Let’s add the first verification and see it fail!
import ai.alumnium.Alumni;import io.appium.java_client.ios.IOSDriver;import io.appium.java_client.ios.options.XCUITestOptions;import org.junit.jupiter.api.AfterEach;import org.junit.jupiter.api.BeforeEach;import org.junit.jupiter.api.Test;
class SearchTest { private Alumni al; private IOSDriver driver;
@BeforeEach void setUp() throws Exception { XCUITestOptions options = new XCUITestOptions() .setBundleId("com.apple.mobilesafari") .setDeviceName("iPhone 17") .setPlatformVersion("26.5") .setNoReset(true); driver = new IOSDriver(options); al = new Alumni(driver); }
@AfterEach void tearDown() { al.quit(); }
@Test void shouldSearch() { driver.get("https://duckduckgo.com"); al.act("search for 'Mercury element' and press Enter"); al.check("page title contains Aluminium word"); }}Now let’s run our test:
$ ./gradlew test --info...java.lang.AssertionError: The requested information is whether the page title contains the word 'Aluminium'. The title of the webpage is 'Mercury element at DuckDuckGo', which does not include the word 'Aluminium'. Therefore, the statement is false....BUILD FAILED in 47sfrom alumnium import Alumnifrom appium.options.ios import XCUITestOptionsfrom appium.webdriver.webdriver import WebDriverfrom pytest import fixture
@fixturedef driver(): options = XCUITestOptions() options.automation_name = "XCUITest" options.bundle_id = "com.apple.mobilesafari" options.device_name = "iPhone 17" options.no_reset = True options.platform_name = "iOS" options.platform_version = "26.5" driver = WebDriver(options=options) yield driver
@fixturedef al(driver: WebDriver): al = Alumni(driver) yield al al.quit()
def test_search(al: Alumni, driver: WebDriver): driver.get("https://duckduckgo.com") al.do("search for 'Mercury element' and press Enter") al.check("page title contains Aluminium word")Now let’s run our test:
$ pytest --quietF...E AssertionError: The requested information is whether the page title contains the word 'Aluminium'. The title of the webpage is 'Mercury element at DuckDuckGo', which does not include the word 'Aluminium'. Therefore, the statement is false....1 failed in 45.21simport { Alumni } from "alumnium";import { browser, expect } from "@wdio/globals";
describe("TestSearch", () => { let al: Alumni;
before(async () => { al = new Alumni(browser); });
after(async () => { await al.quit(); });
it("should search", async () => { await browser.url("https://duckduckgo.com"); await al.do("search for 'Mercury element' and press Enter"); await al.check("page title contains Aluminium word"); });});Now let’s run our test:
$ npx wdio run wdio.conf.ts...AssertionError: The requested information is about whether the page title contains the word 'Aluminium'. The page title is 'Mercury element at DuckDuckGo', which does not include the word 'Aluminium'. Therefore, the statement is false....Spec Files: 0 passed, 1 failed, 1 total (100% completed) in 00:00:35Our test failed as we expected and provided a meaningful explanation of what went wrong.
Let’s fix the first check and add another one to fail again:
import ai.alumnium.Alumni;import io.appium.java_client.ios.IOSDriver;import io.appium.java_client.ios.options.XCUITestOptions;import org.junit.jupiter.api.AfterEach;import org.junit.jupiter.api.BeforeEach;import org.junit.jupiter.api.Test;
class SearchTest { private Alumni al; private IOSDriver driver;
@BeforeEach void setUp() throws Exception { XCUITestOptions options = new XCUITestOptions() .setBundleId("com.apple.mobilesafari") .setDeviceName("iPhone 17") .setPlatformVersion("26.5") .setNoReset(true); driver = new IOSDriver(options); al = new Alumni(driver); }
@AfterEach void tearDown() { al.quit(); }
@Test void shouldSearch() { driver.get("https://duckduckgo.com"); al.act("search for 'Mercury element' and press Enter"); al.check("page title contains Aluminium word"); al.check("page title contains Mercury word"); al.check("search results do not contain Wikipedia articles"); }}Time to re-run the test:
$ ./gradlew test --info...java.lang.AssertionError: The accessibility tree contains multiple links to Wikipedia articles related to the Mercury element, indicating that search results do include Wikipedia articles. Therefore, the statement is false....BUILD FAILED in 54sfrom alumnium import Alumnifrom appium.options.ios import XCUITestOptionsfrom appium.webdriver.webdriver import WebDriverfrom pytest import fixture
@fixturedef driver(): options = XCUITestOptions() options.automation_name = "XCUITest" options.bundle_id = "com.apple.mobilesafari" options.device_name = "iPhone 17" options.no_reset = True options.platform_name = "iOS" options.platform_version = "26.5" driver = WebDriver(options=options) yield driver
@fixturedef al(driver: WebDriver): al = Alumni(driver) yield al al.quit()
def test_search(al: Alumni, driver: WebDriver): driver.get("https://duckduckgo.com") al.do("search for 'Mercury element' and press Enter") al.check("page title contains Aluminium word") al.check("page title contains Mercury word") al.check("search results do not contain Wikipedia articles")Time to re-run the test:
$ pytest --quietF...E AssertionError: The accessibility tree contains multiple links to Wikipedia articles related to the Mercury element, indicating that search results do include Wikipedia articles. Therefore, the statement is false....1 failed in 76.08s (0:01:16)import { Alumni } from "alumnium";import { browser, expect } from "@wdio/globals";
describe("TestSearch", () => { let al: Alumni;
before(async () => { al = new Alumni(browser); });
after(async () => { await al.quit(); });
it("should search", async () => { await browser.url("https://duckduckgo.com"); await al.do("search for 'Mercury element' and press Enter"); await al.check("page title contains Aluminium word"); await al.check("page title contains Mercury word"); await al.check("search results do not contain Wikipedia articles"); });});Time to re-run the test:
$ npx wdio run wdio.conf.ts...AssertionError: The search results include multiple links to Wikipedia articles, such as 'More at Wikipedia' and 'Mercury (element) - Wikipedia'. Therefore, the statement that search results do not contain Wikipedia articles is false....Spec Files: 0 passed, 1 failed, 1 total (100% completed) in 00:00:36Ok, the test failed as we wanted it to, let’s fix it:
import ai.alumnium.Alumni;import io.appium.java_client.ios.IOSDriver;import io.appium.java_client.ios.options.XCUITestOptions;import org.junit.jupiter.api.AfterEach;import org.junit.jupiter.api.BeforeEach;import org.junit.jupiter.api.Test;
class SearchTest { private Alumni al; private IOSDriver driver;
@BeforeEach void setUp() throws Exception { XCUITestOptions options = new XCUITestOptions() .setBundleId("com.apple.mobilesafari") .setDeviceName("iPhone 17") .setPlatformVersion("26.5") .setNoReset(true); driver = new IOSDriver(options); al = new Alumni(driver); }
@AfterEach void tearDown() { al.quit(); }
@Test void shouldSearch() { driver.get("https://duckduckgo.com"); al.act("search for 'Mercury element' and press Enter"); al.check("page title contains Mercury word"); al.check("search results do not contain Wikipedia articles"); al.check("search results contain Wikipedia articles"); }}Now, re-run to make sure it passes:
$ ./gradlew test --info...BUILD SUCCESSFUL in 58sfrom alumnium import Alumnifrom appium.options.ios import XCUITestOptionsfrom appium.webdriver.webdriver import WebDriverfrom pytest import fixture
@fixturedef driver(): options = XCUITestOptions() options.automation_name = "XCUITest" options.bundle_id = "com.apple.mobilesafari" options.device_name = "iPhone 17" options.no_reset = True options.platform_name = "iOS" options.platform_version = "26.5" driver = WebDriver(options=options) yield driver
@fixturedef al(driver: WebDriver): al = Alumni(driver) yield al al.quit()
def test_search(al: Alumni, driver: WebDriver): driver.get("https://duckduckgo.com") al.do("search for 'Mercury element' and press Enter") al.check("page title contains Mercury word") al.check("search results do not contain Wikipedia articles") al.check("search results contain Wikipedia articles")Now, re-run to make sure it passes:
$ pytest --quiet...1 passed in 91.62s (0:01:31)import { Alumni } from "alumnium";import { browser, expect } from "@wdio/globals";
describe("TestSearch", () => { let al: Alumni;
before(async () => { al = new Alumni(browser); });
after(async () => { await al.quit(); });
it("should search", async () => { await browser.url("https://duckduckgo.com"); await al.do("search for 'Mercury element' and press Enter"); await al.check("page title contains Mercury word"); await al.check("search results do not contain Wikipedia articles"); await al.check("search results contain Wikipedia articles"); });});Now, re-run to make sure it passes:
$ npx wdio run wdio.conf.ts...Spec Files: 1 passed, 1 total (100% completed) in 00:00:37Add Data Retrieval
Section titled “Add Data Retrieval”Finally, let’s verify some data that Alumnium can get from the page. We are going to ensure the Mercury element card shows its symbol correctly.
Let’s add a failing verification first:
import ai.alumnium.Alumni;import io.appium.java_client.ios.IOSDriver;import io.appium.java_client.ios.options.XCUITestOptions;import org.junit.jupiter.api.AfterEach;import org.junit.jupiter.api.Assertions;import org.junit.jupiter.api.BeforeEach;import org.junit.jupiter.api.Test;
class SearchTest { private Alumni al; private IOSDriver driver;
@BeforeEach void setUp() throws Exception { XCUITestOptions options = new XCUITestOptions() .setBundleId("com.apple.mobilesafari") .setDeviceName("iPhone 17") .setPlatformVersion("26.5") .setNoReset(true); driver = new IOSDriver(options); al = new Alumni(driver); }
@AfterEach void tearDown() { al.quit(); }
@Test void shouldSearch() { driver.get("https://duckduckgo.com"); al.act("search for 'Mercury element' and press Enter"); al.check("page title contains Mercury word"); al.check("search results contain Wikipedia articles"); String symbol = (String) al.get("chemical symbol"); Assertions.assertEquals("Al", symbol); }}al.get() returns Object and Alumnium supports the following types — String, Long, Double, Boolean, or a List of those. A simple cast on the call site is enough.
Time to re-run the test:
$ ./gradlew test --info...org.opentest4j.AssertionFailedError: expected: <Al> but was: <Hg>...BUILD FAILED in 56sfrom alumnium import Alumnifrom appium.options.ios import XCUITestOptionsfrom appium.webdriver.webdriver import WebDriverfrom pytest import fixture
@fixturedef driver(): options = XCUITestOptions() options.automation_name = "XCUITest" options.bundle_id = "com.apple.mobilesafari" options.device_name = "iPhone 17" options.no_reset = True options.platform_name = "iOS" options.platform_version = "26.5" driver = WebDriver(options=options) yield driver
@fixturedef al(driver: WebDriver): al = Alumni(driver) yield al al.quit()
def test_search(al: Alumni, driver: WebDriver): driver.get("https://duckduckgo.com") al.do("search for 'Mercury element' and press Enter") al.check("page title contains Mercury word") al.check("search results contain Wikipedia articles") symbol = al.get("chemical symbol") assert symbol == "Al"Time to re-run the test:
$ pytest --quiet...E AssertionError: assert 'Hg' == 'Al'EE - AlE + Hg...1 failed in 68.00s (0:01:07)import { Alumni } from "alumnium";import { browser, expect } from "@wdio/globals";
describe("TestSearch", () => { let al: Alumni;
before(async () => { al = new Alumni(browser); });
after(async () => { await al.quit(); });
it("should search", async () => { await browser.url("https://duckduckgo.com"); await al.do("search for 'Mercury element' and press Enter"); await al.check("page title contains Mercury word"); await al.check("search results contain Wikipedia articles"); const symbol = await al.get("chemical symbol"); expect(symbol).toEqual("Al"); });});Time to re-run the test:
$ npx wdio run wdio.conf.ts...Expected: "Al"Received: "Hg"...Spec Files: 0 passed, 1 failed, 1 total (100% completed) in 00:00:45The test failed as expected, let’s fix it and re-run to make sure it’s passing:
import ai.alumnium.Alumni;import io.appium.java_client.ios.IOSDriver;import io.appium.java_client.ios.options.XCUITestOptions;import org.junit.jupiter.api.AfterEach;import org.junit.jupiter.api.Assertions;import org.junit.jupiter.api.BeforeEach;import org.junit.jupiter.api.Test;
class SearchTest { private Alumni al; private IOSDriver driver;
@BeforeEach void setUp() throws Exception { XCUITestOptions options = new XCUITestOptions() .setBundleId("com.apple.mobilesafari") .setDeviceName("iPhone 17") .setPlatformVersion("26.5") .setNoReset(true); driver = new IOSDriver(options); al = new Alumni(driver); }
@AfterEach void tearDown() { al.quit(); }
@Test void shouldSearch() { driver.get("https://duckduckgo.com"); al.act("search for 'Mercury element' and press Enter"); al.check("page title contains Mercury word"); al.check("search results contain Wikipedia articles"); String symbol = (String) al.get("chemical symbol"); Assertions.assertEquals("Al", symbol); Assertions.assertEquals("Hg", symbol); }}$ ./gradlew test --info...BUILD SUCCESSFUL in 54sfrom alumnium import Alumnifrom appium.options.ios import XCUITestOptionsfrom appium.webdriver.webdriver import WebDriverfrom pytest import fixture
@fixturedef driver(): options = XCUITestOptions() options.automation_name = "XCUITest" options.bundle_id = "com.apple.mobilesafari" options.device_name = "iPhone 17" options.no_reset = True options.platform_name = "iOS" options.platform_version = "26.5" driver = WebDriver(options=options) yield driver
@fixturedef al(driver: WebDriver): al = Alumni(driver) yield al al.quit()
def test_search(al: Alumni, driver: WebDriver): driver.get("https://duckduckgo.com") al.do("search for 'Mercury element' and press Enter") al.check("page title contains Mercury word") al.check("search results contain Wikipedia articles") symbol = al.get("chemical symbol") assert symbol == "Al" assert symbol == "Hg"$ pytest --quiet...1 passed in 108.67s (0:01:48)import { Alumni } from "alumnium";import { browser, expect } from "@wdio/globals";
describe("TestSearch", () => { let al: Alumni;
before(async () => { al = new Alumni(browser); });
after(async () => { await al.quit(); });
it("should search", async () => { await browser.url("https://duckduckgo.com"); await al.do("search for 'Mercury element' and press Enter"); await al.check("page title contains Mercury word"); await al.check("search results contain Wikipedia articles"); const symbol = await al.get("chemical symbol"); expect(symbol).toEqual("Al"); expect(symbol).toEqual("Hg"); });});$ npx wdio run wdio.conf.ts...Spec Files: 1 passed, 1 total (100% completed) in 00:00:55Congratulations, we have completed our first test!