When your IT department fails you Web Scraping with AutoHotkey can save the day!

web scraping with AutoHotkey

One of my clients has over 3,000 videos on their website.  Amazingly nobody in their IT department can provide a holistic file with metadata like: Title, # of views, Length, Format, Tags, etc.  How crazy is that?   After contacting their video hosting company, nobody could (would) share this information!

Web Scraping with AutoHotkey to the rescue!

I wrote web scraping with AutoHotKey script that first got a list of all the videos, then looped over them and extracted various attributes about each video.  Not an ideal way to get this, but the client was manually going to go through this by cutting & pasting from their browse.

Extracting Information about Videos

Web Scraping with AutoHotKey 103-Leveraging the Document Object Model

web scraping

Leveraging the Document Object Model

This third video on Web Scraping gets a little advanced and shows how you can leverage the DOM to make extracting data from a webpage much easier and reliable.

Leveraging the Document Object Model (DOM)will take some practice (especially if you’re not familiar with Object oriented coding) but it is well worth it because it greatly reduces the amount of clean-up you have to do after you extract your data.  I used to write some pretty crazy regular expressions to try and clean up my code.  Once I learned how to better navigate the DOM it negated the need for cleaning!

The HTML Document Object Model (DOM)-Tree of Objects

Document Object Model

Video Leveraging the DOM plus looping over pages

Webscraping with AHK 103-Isolating sections and taking advantange of DOM

The syntax for writing the writing the web scraping code can be found on my first post here.  There is also an AutoHotKey forum thread you might wish to review here.

Web Scraping Example with AutoHotkey-Scraping website to determine status

Web Scraping Example with AutoHotkey

The term Web Scraping is well known but I’ve yet to hear much talk about “Web Pasting”.   In this Web Scraping Example with AutoHotkey video I demonstrate how I helped my Real Estate agent automate checking the status of a house on a website.

Web Scraping example with AutoHotkey

Here is a second version where, depending on the status of the home, I write a follow-up email either with an email from Outlook or within the website via COM.

Real Estate Helper

 

Transposing clipboard content in any program- No need to run to Excel!

Transposing clipboard content

While it is easy to use Excel to transpose rows into columns, I’ve created an AutoHotkey script which negates the need to.  Basically it examines what is on the clipboard and replaces tabs with line breaks, then it sends paste to the program thus it is transposing clipboard content.  I have it triggered of hitting a hotkey and comes in pretty handy and can save some time if it is something you do frequently.

Below is a video which demonstrates it usage.

Transposing clipboard content

Transposing clipboard content in any program