Nov-28-2023, 11:24 AM
Good morning,
I'm looking for some advice on tutorials and self study. I'm working on automating some of my workflow because I use a lot of different programs (some with apis for python, some without) and it can be frustrating to perform a lot of repetitive tasks.
One approach that I'm considering is having a program flip through all the windows, programs, and internet browsers I have open and then click appropriate buttons based on visual cues.
My question is whether there is a tutorial to help with this sort of thing. I'd essentially like to find a button on a screen and click the button. The button doesn't have a predefined location so the program would have to locate it (across multiple monitors?), apply a coordinate to it, and then use the python mouse click function.
An internet search suggested that having the program take a screenshot of my computer's desktop and then doing an image comparison for the button would be a viable option, but it also said that if the image isn't an exact match, then it wouldn't be able to locate and click the button. Which led me to read about the MSE and SSIM for image comparisons.
Am I going down the right path here, or is there a better way to do this?
I'm looking for some advice on tutorials and self study. I'm working on automating some of my workflow because I use a lot of different programs (some with apis for python, some without) and it can be frustrating to perform a lot of repetitive tasks.
One approach that I'm considering is having a program flip through all the windows, programs, and internet browsers I have open and then click appropriate buttons based on visual cues.
My question is whether there is a tutorial to help with this sort of thing. I'd essentially like to find a button on a screen and click the button. The button doesn't have a predefined location so the program would have to locate it (across multiple monitors?), apply a coordinate to it, and then use the python mouse click function.
An internet search suggested that having the program take a screenshot of my computer's desktop and then doing an image comparison for the button would be a viable option, but it also said that if the image isn't an exact match, then it wouldn't be able to locate and click the button. Which led me to read about the MSE and SSIM for image comparisons.
Am I going down the right path here, or is there a better way to do this?