Web Scraping

Great AutoHotkey script to URL Encode/Decode and parse URL parameters

parse url parameters

Working in email marketing and Website design I frequently need to URL Encode / Decode and parse URL parameters.  This short AutoHotkey script makes doing so a breeze!  I watch my colleagues struggle through trying to read encoded URLs or try and find every parameters in a given URL.  With this script you can see just how easy it is to decode a URL and then parse it on the “?” then by every “&” putting them on new lines.

A second usage is to help examine how websites use URL Parameters to serve-up a webpage.  This can be very helpful when doing Web Scraping as you can automate generating the page from a given website.

Video demonstrating how to Parse URL Parameters

Great AutoHotkey script to URL Encode_Decode and parse URL parameters

AutoHotkey script to Parse URL Parameters

If this interested you, you might also want to investigate using web scraping with AHK.

Web Scraping

Web Scraping with AutoHotKey 108- use querySelctorAll for added flexibility

Web Scraping with AutoHotkeyWeb Scraping with AutoHotkey is fun and easy however some web pages don’t have Names, IDs, or classNames set up.  Other times the pages are pretty sophisticated and you want to grab something very specific (w/out writing a lot of code).   I recently learned about QuerySelctor which uses the CSS Selector.

This W3schools page has a great, dynamic tool, that is very helpful to understand what is being grabbed.  Here is a screenshot of clicking around on the above tool which will help you get the concept.
Web Scraping with AutoHotkey QuerySelectorAll

Below is a video documenting various method calls.

Web Scraping with AutoHotkey Tutorial 8- QuerySelectorAll

Web Scraping 8 QuerySelectorAll-2

And lastly here is the script I used walking through the video.

 

Web Scraping

Web Scraping with AutoHotKey 107- using ClassName

web scraping with AutoHotkeyWhen I did my original posts on web scraping with AutoHotkey, I was unaware of ClassName  (getElementsByClassName).

About a year ago I was speaking with Jackie Sztuck from http://jszapp.com (BlackHolyman on the AHK forum) and he introduced me to ClassName.  In the past year I’ve really come to rely on it as my “go-to” for grabbing specific values from a page.   If Name or ID are present I’ll still use them however more and more sites don’t take the time to provide them.

This video demonstrates a few examples where I grab the ClassName.

Web Scraping with AutoHotkey– ClassName

Web Scraping with AutoHotKey 107- Classname

 

Web Scraping

Automate Unicode character encoding for HTML

Unicode character encoding

Our CMS does not support Unicode text thus when we need to search-replace for characters like the and © symbols. This takes a fair amount of time and is easy to miss so I wrote a script in AutoHotKey to automatically handle the Unicode character encoding.

Now I can highlight the word and click a button and  Whamo!  Instant replacement with HTML equivalents!  No more trying to scan text and find illegal characters.

replace unicode characters with HTML equivalent

Unicode character encoding

Here is the AutoHotKey code I use.  My code first grabs the highlighted text and copies it to the clipboard, where it manipulates it, then sends it back to the active program I was working in. No more need for Unicode character encoding!