Java robot and image comparison

Question

I'm toying with an idea for creating a Java application to automate a process that I have to do regularly and before I start any coding I thought I would seek advice as to the best way to approach it.

Basically, the application I use has a large number of images present on the screen at any one time, and what I would like to know is if there is a way to have Java identify if any of these two images are the same. If they are, I would like to automate mouse movement and button clicks.

After a bit of reading, I'm thinking that the PixelGrabber and Robot classes might be the right way to start, but like I said, I'm looking for any information on this that can be offered.

What are your suggestions?

Answer 1

I believe the Robot class and a Pixel Grabber would be sufficient. If you are inclined to program the solution yourself, maybe for educational purpose, by all means please do. If you, however, don't want to reinvent the wheel, you may take a look at this project:

http://sikuli.org/

I, for example, use it to do stuff that would be hard to achieve with Selenium alone. If you still can't achieve your goal after some scripting, Sikuli provides a nice API which you can use from inside your java program.

Answer 2

The Robot class would be sufficient to take images and being able to inspect pixels. But it seems to make more sense, to recreate your desktop with images inside of a java application (a very simple gallery application). Then operations are simplier. An other way of realizing operations I do not see.