Yup. I was using AI as a blanket term. I don’t know JS too well. I was thinking of using python/pyautogui/opencv to do it. Opencv has a pattern matching tool that would work well. Pyautogui let’s you take screenshots and simulate the mouse.

It would be useful to be able to find a pieces position on the screen. Is the array of piece object kept client-side?