Web Scraping intro with AutoHotKey 101-Getting data from a page, handles & pointers

Web Scraping Intro with AutoHotkey

Web Scraping Intro with AutoHotkey

Being able to, programatically, navigate to an Internet page and scrape the contents in a reliable fashion is best things invented since sliced bread!   I spent years manually going through pages and copying/pasting contents from IE to Excel then spent even more time trying to clean it up.  Done properly you can get the data very, very close to how it is on the web with little effort.

The below video walks through using AutoHotKey to obtain basic values from a Web page.  It also demonstrates a script I wrote that helps write the syntax (yes I’m that lazy!)  The AutoHotKey script I wrote is further down this page and can also be found on the AHK forum here.

In this beginning tutorial I how to:
1) get a pointer to IE
2) navigate to a page
3) get text from a page

Web Scraping Intro with AutoHotkey

Webscraping with AHK 101-Pointer and getting values from page

Here is the script writer to use during your web scraping intro with AutoHotkey.

Programmatically interact with the SciTE editor via COM objects

SciTE editor via COM objects

SciTE editor via COM objects

SciTE is a great IDE that I use with AutoHotKey, SPSS, SQL, Python, XML, HTML, etc.   I love being able to use regular expressions in it to manipulate text and it has some very cool capabilities.  This video is one of my favorite demonstrations how powerful SciTE can be at manipulating text.

Here is a short tutorial and demonstration on how to manipulate SciTE editor via COM objects and Windows commands with AutoHotKey.

 

SciTE editor via COM objects- Editor Windows Commands

A specific version of the SciTE editor for AutoHotKey can be downloaded here and more generic documentation can be found here.

Loading contacts to SilverPop via API call & SFTP file transfer

API call & SFTP file transfer

API call & SFTP file transfer

API call & SFTP file transfer

Loading our contact lists into Engage via their Web interface (browser) takes around 7-9 minutes of focused attention per list.  Doing the same thing with an API call and Secure File Transfer (SFTP) takes seconds!  The below video demonstrates the process from using web scraping to grab the name /ID on our SharePoint server to uploading the file to Engage, creating the name in Engage and tying the new name to the contact list.  The video is longer because I also have it wait for the data job to complete and then write an email confirming the process is complete.

The API call & SFTP file transfer is free with Engage/SilverPop but it isn’t well promoted (on their website nor by their representatives.  Spending a bit of time to get it working is well worth the investment. For the SFTP file transfer I’m using WinSCP but any SFTP program will do.

 

 

API call & SFTP file transfer

API Call and – SFTP file transfer