This week I dove into Mobile UI Testing and Maestro. Beyond learning a cool new tool, it was encouraging to see AI nicely integrated into a SaaS product.

Mobile UI Testing

As someone who’s sloppily built a cross-platform app, I’ve experienced firsthand the challenges of maintaining consistent UI elements and user experiences across iOS and Android. Frameworks like React Native and component libraries like Paper enable single-codebase development of cross-platform applications. That said, development usually happens on a single simulator, and given the nuances of different platforms and screen sizes, this leaves a lot of room for UI errors.

This is where mobile UI testing frameworks come in: They let you connect your codebase to a single testing suite that validates your app’s appearance and functionality across multiple platforms simultaneously. One test suite, reliability across multiple platforms and devices!

Maestro

When I first saw Maestro’s tagline and sample code:

Maestro is the simplest and most effective mobile UI testing framework.

appId: com.example.app
---
- launchApp
- onTap: Search
- inputText: Testing

I was skeptical. A testing framework based entirely on YAML files? It seemed too simple to be useful. However, after working with it for a few days, I’m impressed. Maestro is powerful, well-designed, and importantly, it works!

Standout features:

• The simple YAML syntax is easy to write, extensible, and covers everything I wanted
• Hierarchical and flexible element selection allows the engineer to decide on the tradeoff between development speed and strictness
• Easy to use tools like Maestro Studio for graphical element selection
• AI capabilities that extend capabilities beyond traditional rule-based tests
• Clear, comprehensive documentation and logging

A Good Use Case for AI!

The joke is “Hundreds of billions of dollars have been poured into AI development and infrastructure. What do we have to show for it? A chatbot!”. While this is a joke and I believe AI products will eventually lead to a productivity explosion, it was fun to see an implementation of AI that is simple, user-friendly, and solves a concrete problem. Maestro’s AI configuration took 2 minutes to setup and solves test cases that would be cumbersome, if not impossible, to codify.

For example: How do you write a test case for this?

With Maestro’s AI configuration, you can simply add this command to your test flow:

- assertNoDefectsWithAI

And get a report like this:

I find this very compelling. It’s one thing to have a test suite that can validate UI elements and functionality. It’s another thing to have a test suite that can validate the presence of defects. I’m excited to see how this feature evolves and see how it performs “in production”.

Anywho… Thanks for reading. In my next post, I’ll explore Maestro’s cloud environment, CI workflow, and whether LLMs are any good at writing Maestro tests!

Be well