Web Scraping with AutoHotKey 108- use querySelctorAll for added flexibility

Web Scraping with AutoHotkeyWeb Scraping with AutoHotkey is fun and easy however some web pages don’t have Names, IDs, or classNames set up.  Other times the pages are pretty sophisticated and you want to grab something very specific (w/out writing a lot of code).   I recently learned about QuerySelctor which uses the CSS Selector.

This W3schools page has a great, dynamic tool, that is very helpful to understand what is being grabbed.  Here is a screenshot of clicking around on the above tool which will help you get the concept.
Web Scraping with AutoHotkey QuerySelectorAll

Below is a video documenting various method calls.

Web Scraping with AutoHotkey Tutorial 8- QuerySelectorAll

Web Scraping 8 QuerySelectorAll-2

And lastly here is the script I used walking through the video.

 

Web Scraping with AutoHotKey 107- using ClassName

web scraping with AutoHotkeyWhen I did my original posts on web scraping with AutoHotkey, I was unaware of ClassName  (getElementsByClassName).

About a year ago I was speaking with Jackie Sztuck from http://jszapp.com (BlackHolyman on the AHK forum) and he introduced me to ClassName.  In the past year I’ve really come to rely on it as my “go-to” for grabbing specific values from a page.   If Name or ID are present I’ll still use them however more and more sites don’t take the time to provide them.

This video demonstrates a few examples where I grab the ClassName.

Web Scraping with AutoHotkey– ClassName

Web Scraping with AutoHotKey 107- Classname

 

Web Scraping with AutoHotkey- Data Extraction & emailing to colleagues

web scraping with AutoHotkeyI regularly have to go to login to a website, navigate to a certain page, extract data and then place it into Excel, then email the files to my colleagues.

This video demonstrates how quickly and easily it is done when you program some web scraping with AutoHotKey.  Somehow I lost the audio track however you can still see what is going on.  Also, some of it happens so fast it looks like I sped up the video but it is real-time.  🙂

Demo video of Web Scraping with AutoHotkey and emailing to colleagues

Web Scraping- Data Extraction from a webpage & emailing to colleagues

If you’re new to scraping you can see my earlier post here which gives a good review as well as a program that helps write the AutoHotKey code.

Web Scraping with AutoHotKey 102-Setting values and clicking links

web scraping with AutoHotkey

This is the second video in this series.  Here we practice setting values on a page (kind of reverse of Web Scraping with AutoHotkey however I don’t believe anybody has coined a decent term yet) and clicking links.

Word of warning- some pages want you to fire an “event”.  Sometimes this is tricky.  Given this video is set to an introductory level I only touch a little on the subject.

Web Scraping with AutoHotKey 102-Setting values and clicking links

Webscraping with AHK 102-Setting values and clicking

 

The syntax for writing the writing the code can be found on my first post here.  There is also an AutoHotKey forum thread you might wish to review here.

Web scraping

 (web harvesting or web data extraction) is a computer software technique of extracting information from websites. Usually, such software programs

Think like a Hacker and Protect Yourself with White Hat Security

You’ll even get training on scraping, you can easily get data off of sites like Yelp and create leads list for your sales team from data available online. Jump into a new, in-demand career. Get the White Hat Security Hacker bundle at SitePoint Shop