Back to feed
Dev.to
Dev.to
7/14/2026
The original title is about automating apps with no DOM, specifically Flutter/canvas editors, using coordinates only.

The original title is about automating apps with no DOM, specifically Flutter/canvas editors, using coordinates only.

Original: Automating an app with no DOM: driving Flutter/canvas editors with coordinates only

Short summary

When apps render entirely on canvas (Flutter Web, canvas editors), DOM-based automation fails completely — there are no elements to find. The author drove the Rive animation editor using Claude with coordinate-only automation: waiting for viewport stabilization, zooming screenshots to read canvas-painted text, verifying every click with visual evidence, and using rapid double-fire clicks for hierarchy interactions. The key lesson is that coordinate automation requires verification after every step, making it slow but reliable.

  • Canvas-rendered apps have no DOM — coordinate automation is the only interface available
  • Every click must be followed by a screenshot and visual confirmation before proceeding
  • Procedure transfers across apps: settle-wait → zoom-read → evidence-check → rapid double-fire

Generated with AI, which can make mistakes.

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more