Use cases~4 min read
Flutter UI Testing Without Accessibility Labels
Flutter widgets without accessibility labels? TaloTrace tests by looking at the screen, not element IDs, and records the run as evidence. See the limits.
TL;DR
Possibly, with caveats. TaloTrace is a QA testing tool that explores your running app and reports findings with evidence. It navigates by looking at the screen, not by element IDs, so it does not rely on element IDs being present. The caveats: it runs on a cloud-hosted browser, Android emulator or iOS Simulator, not physical devices, and the facts we publish do not cover Flutter-specific behaviour, so check fit on your own build.
Key takeaways
Sight-based navigation: TaloTrace looks at the screen rather than relying on element IDs.
No scripts: you do not write or maintain test scripts, and you can describe the flow you care about in plain language.
Evidence: every run records the screen, and each finding carries the time window in the recording where the defect shows.
Independent review: TaloTrace verifies each finding before it reaches you.
Limits: execution is on virtual devices only, and Flutter-specific behaviour is not covered by the facts we publish, so check fit on your own build.
Why Can't Label-Based Tests Find Flutter Widgets?
Tests that find widgets by semantics label cannot see widgets that expose none. Flutter's own testing documentation shows this in its bySemanticsLabel finder, which matches only widgets that carry a Semantics label. One fix is to retrofit labels across the app so a locator has something to grab, which takes engineering time.
What you want is coverage of real user journeys now, without first making the app legible to a tool. That is the job this post is about.
How Does TaloTrace Find Its Way Without Element IDs?
Sight-based navigation means TaloTrace finds controls by looking at the screen, not by element IDs, which can break when the UI changes. That is why it keeps working when the UI changes, and why there are no test scripts to write or maintain.
Because it works from what is rendered on screen, it does not need an element ID to find a control. Its interaction range covers:
- Taps and typing
- Scrolling and swiping
- Long-press and double-tap
- Drag and pinch
The how it works page walks through the full flow.
How Does TaloTrace Decide What to Test?
You point TaloTrace at your app and, optionally, describe the flow you care about most in plain language, such as "complete a checkout". It explores the running app, plans the user journeys worth testing and drives each one to a verifiable outcome.
The journeys it completes become replayable scenarios. You can also author scenarios from a chat description or write them by hand.
A goal counts as done only when a machine-checkable predicate proves the outcome. The predicate must be false before the action and true after, so a screen that already shows "Projects" cannot falsely complete a "create project" goal.
You can also declare boundaries TaloTrace stays inside. A flow that would cross one is recorded as blocked, with a note on what would be needed to test it safely.
How Does TaloTrace Get Past Sign-In?
Mobile apps often put their most important screens behind a login. You save test accounts once per project, each with a role label such as "admin" or "viewer", and sign-in is email plus password.
Free-text custom instructions cover logins that need extra steps, such as an OTP prompt or a second screen. Each scenario can handle sign-in in one of three ways:
- Default account: inherit the project default account.
- Specific account: use a particular saved account.
- No login: run with no login at all.
Saved passwords are write-only. They are omitted from every read of the account and masked out of the run's captured steps and logs.
What Makes TaloTrace Different From Element-Based Testing?
Element-based tests depend on the app exposing something to locate. TaloTrace navigates by looking at what is rendered on screen.
| Question | Label or ID-based tests | TaloTrace |
|---|---|---|
| How does it find a control? | Looks up an element ID or label | Looks at what is rendered on screen |
| What if the UI changes? | The locator can break | Sight-based navigation keeps working when the UI changes |
| Who writes the test scripts? | You | No test scripts to write or maintain |
| What counts as done? | Depends on the assertions you write | A machine-checkable predicate that is false before the action and true after |
- Sight over selectors: navigation does not hinge on element IDs.
- Proven outcomes: a scan that produces no independently proven goal fails, rather than reporting an unverified journey as a pass.
- Separate review: TaloTrace drives your app and records what looks wrong, then independently verifies each finding before it reaches you.
For a wider look at running on Android, see our post on AI QA testing for Android apps.
What Do You Get Back From a Run?
A Run produces a set of findings. Each one is backed by a recording, TaloTrace's reasoning and step-by-step reproduction, including the Evidence panel with TaloTrace's step-by-step trajectory.
A screen recording is captured for every run, and each finding carries the time window inside it where the defect shows. A vision model also analyses the recording for visual and functional anomalies, not only hard failures.
Severity has five levels, shown as labels such as "Critical (P0)". You can supply your own severity guidance per project and TaloTrace follows it when rating findings. Review and storage work like this:
- Hidden first: results start hidden and become visible when a reviewer approves them, or when the project is configured to auto-approve.
- Low confidence: these findings need an explicit per-finding decision.
- No duplicates: repeated observations of the same defect collapse into a single issue.
- Built-in view: findings live in TaloTrace's own issue view.
Where Does This Approach Stop?
TaloTrace has these limits today:
- Virtual devices only: TaloTrace runs on a cloud-hosted browser, Android emulator or iOS Simulator. Physical devices are not offered on any platform.
- Flutter fit is unconfirmed: the claim here is about navigating by sight, not about Flutter-specific behaviour. Check your own build with us before you commit.
- Held-back actions: destructive or irreversible actions, such as deleting an account or making a payment, are not performed.
- Findings stay in TaloTrace: exporting to Jira, Linear, GitHub or other trackers is not available.
- Triggers: you can start runs on demand from the app or API, or on a daily or weekly schedule. A scheduled run always tests the latest finalised build.
TaloTrace also does not replace your own judgement on what matters. You decide the flows, boundaries and severity guidance, and a reviewer decides what becomes visible.
How Do You Get Started?
If you are looking at a Flutter app whose widgets your current tools cannot see, the quickest check is to try it on your own build. For a broader view of the approach, read our post on an autonomous mobile app testing tool.
Tell us about your app and we will talk through fit.
Frequently asked questions
Do I need to add Semantics or accessibility labels before TaloTrace can test my app?
TaloTrace navigates by looking at the screen, not by element IDs. Whether your specific unlabelled widgets work on your build is best confirmed with us.
Has TaloTrace been confirmed to work with Flutter apps?
The facts we publish do not cover Flutter-specific behaviour. The claim here is about sight-based navigation on a cloud-hosted browser, Android emulator or iOS Simulator. Ask about your specific build using the link above.
Can TaloTrace run on a physical phone?
No. Physical devices are not offered on any platform. Runs execute on a cloud-hosted browser, Android emulator or iOS Simulator, chosen from your app's platform.
Will TaloTrace delete data or make payments while exploring?
TaloTrace commits real actions such as Create, Save and Submit, and holds back destructive or irreversible ones such as deleting an account or making a payment. You can also declare boundaries it stays inside.
How do I see what went wrong in a failed scenario?
Each finding carries the time window in the screen recording where the defect shows, TaloTrace's reasoning, and step-by-step reproduction in the Evidence panel.
