Friday, July 19, 2019

Changelog 2019-07-19 - One Fix

Target Will Now Alert on Items More Reliably

Alright, let's do this one last time. I know I've said that I've fixed Target before, but this time it's really going to stick. Previously, we were scraping their web page. Now, we're using their API. Below is an explanation of why this is better, following by the short set of limitations.

Previously, we scraped their website by trying to capture the HTML. Because Target.com uses a modern javascript framework, we could not just throw a regular old get request at them. Instead, we had to do some browser spoofing to get the state of their page. However, this was unreliable because the site would often times not load the entire page. Their site uses some time/bandwidth saving techniques and only loads what it thinks it needs to load. It's tricky to get the browser spoofer to convince the page to load everything, so this resulted in in-stock items going unseen for hours at a time. And once they finally were seen, they were alerted on as back in stock even though they never went out of stock. Now, we are using their API to directly talk to their backend and get the information 100% of the time.

However, their API returns some messy information in the way of Tee Shirts. Anything with multiple sizes is handled differently than most items. Some of the Tee Shirts I am able to determine if they are in stock or not, but others have such confusing information associated with them that I cannot currently tell the difference between in stock and out of stock for them. So, until I can figure this out, some of the Pop+Tee combos will always show as out of stock and won't be alerted on. But that is a very small subset of items and this is an acceptable loss for now.

No comments:

Post a Comment

Hot Cash Give Away Jan 2021

  Hey everyone! Time for another round of Hot Cash codes! Hot Cash ends soon so be sure to use these ASAP! Comment below with the codes you ...