Friday, June 14, 2019

Changelog 2019-06-14 - Two Fixes

@FunkoFYE is more reliable now

FYE's site isn't great and their search functionality isn't fantastic either. There is a button to click that should give you all of their exclusives. However, it throws in a bunch of their commons as well in the search results and returns way too many items to reliably scrape.

The way around this is to use that search page and click *another* hidden button to filter down to just exclusives. However, there is no visible URL change so a URL for scraping had to be determined by looking at the network requests.

The RegEx had to also be updated to accommodate for these new pages. However, the name of the product as well as the URL should be a lot cleaner. Previously, we were posting products that had a lot of garbage in the product name as it was derived from the URL. Now, the item name is derived from the page itself so it should look a lot cleaner going forward.

Preorder links will more reliably be appended to their respective posts

With the recent update I made allowing newly release products to have their links automatically appended to preorder threads made on reddit, I made a decision to say "if there are any preorder threads made in the past two hours, only look at those. Otherwise, look at all threads made in the past 24 hours." This worked well up until today when there were quite a few releases.

Normally, if only one announcement is made in a day, it doesn't matter if it has been 2 or 22 hours, the script will still find the correct preorder post. But, if something is announced, then something else is announced, and the first announcement's product is released more than 2 hours from the first announcement but less than 2 hours after the second announcement, we won't append to the correct preorder thread.

To fix this, I have decided to eliminate all fancy checking and guessing. Instead, all preorder threads made in the past 24 hours are candidates. The script will then find whichever it most closely matches from that pool.

No comments:

Post a Comment

Hot Cash Give Away Jan 2021

  Hey everyone! Time for another round of Hot Cash codes! Hot Cash ends soon so be sure to use these ASAP! Comment below with the codes you ...