Skip to main content
Go to the Studio
Tutorials

Screen coordinates

Screen coordinates

Some of Roberty's actions — the mouse and desktop automation ones especially — work from screen coordinates: the exact position, in pixels, of a point on your screen. This tutorial covers what those coordinates are, how they work, and how to use them in your robot's actions.

What are screen coordinates?​

Picture your screen as a grid. Each square in that grid is a pixel — the smallest piece of image the screen can display. A Full HD screen (1920 × 1080), for instance, has 1,920 columns and 1,080 rows of pixels: over 2 million individual points.

To identify any point on that grid, you use a pair of numbers, a coordinate:

  • X — the horizontal position (left to right)
  • Y — the vertical position (top to bottom)

A coordinate is written (X, Y). (960, 540), for example, is the exact centre of a Full HD screen.

A spreadsheet analogy

Think of Excel: the column is the X axis (horizontal) and the row is the Y axis (vertical). Just as cell C5 identifies one position in the sheet, the coordinate (960, 540) identifies one point on the screen.

How the coordinate system works​

The origin — (0, 0) — is the top-left corner of the screen. From there:

  • Moving right increases X
  • Moving down increases Y
(0,0) ──────────────────► X
  │
  │
  │
  │
  ▼
  Y

So on a Full HD screen:

  • The top-right corner is roughly (1920, 0)
  • The bottom-left is (0, 1080)
  • The bottom-right is (1920, 1080)
  • The centre is (960, 540)

What coordinates are for in Roberty​

Coordinates are used by the mouse and desktop automation actions, when the robot has to move the cursor or click a particular spot on screen — in applications that are not websites and have no HTML selectors.

For example:

  • Moving the mouse to a button in software installed on the machine
  • Clicking a particular point of an image, or of a legacy system's screen
  • Double-clicking inside a bounded area
  • Scrolling at a given position

Coordinates or selectors?​

SituationUse
Automating websites and web systemsA selector — more stable and more precise
Automating desktop applications (no selector)Coordinates
An image recognition actionCoordinates (the action returns them)
Clicking a fixed, predictable spot on screenCoordinates
Careful with coordinates on the web

For browser automation, always prefer HTML selectors. Coordinates depend on an exact position on screen, and can shift if the page's content changes, the window is resized, or the screen resolution differs.

Finding a point's coordinates​

Option 1 — PowerToys (Microsoft)​

PowerToys is a free Microsoft tool whose colour picker shows the cursor's coordinates live.

  1. Download and install PowerToys.
  2. Open it and turn on Color Picker.
  3. Press Win + Shift + C to open the picker.
  4. Move the cursor to the point you want — the (X, Y) coordinates update as you go.

Option 2 — Paint (built into Windows)​

  1. Take a screenshot with Print Screen (the PrtSc key).
  2. Open Paint and paste it (Ctrl + V).
  3. Move the cursor over the point you want — Paint shows the coordinates in the status bar at the bottom of the window.

Option 3 — Check the screen resolution​

If you need your screen's total size, to work out relative positions:

  1. Right-click the desktop.
  2. Choose Display settings.
  3. Find Display resolution — it gives the dimensions in pixels (1920 × 1080, for instance).

In practice​

Use absolute coordinates carefully. If the robot runs on machines whose resolution differs from yours, the coordinates may point at the wrong place. Always validate in an environment with the resolution production will use.

Pair them with image recognition. When an element's position moves around, consider the image recognition actions instead: the robot finds the element visually and returns the coordinates itself, with nothing hard-coded.

Keep coordinates in parameters. If the same coordinate is used at several points in the robot, save it as a robot parameter so it is easy to adjust later.

Next​