Thursday, June 6, 2019

Changelog 2019-06-06 - New Feature

Preorder Posts will now automatically update when new items are found

With a few limitations which will be detailed below, any new item that is found by the web scrapers will automatically be appended to the appropriate preorder post on reddit.

Preorder posts are created instantaneously when a new Funko blog post is scraped and posted to Reddit. Once the preorder post is made, the blog post and the link to the preorder post are also posted to @PopCommunity on Twitter. Thus, these preorder posts have two means of visibility. The idea of them is to have the community share links to stores they have found which are offering preorders for the items that were just announced. The intention of this project was to cut down on the clutter found on r/FunkoPop as people would make a new post for preorders for each store found, so for highly anticipated releases, the sub would become cluttered with various preorder links. Additionally, it was impossible to navigate the space and see all of the posted options for preorders as the first preorder link to go up was usually the most visible, even if it was not the best option.

So with the preorder posts that are automatically generated, this fixes the issue by consolidating preorder links to a single posts. However, there is a manual step where I have to edit the post to include the preorder links that are found.

Considering my blog scrapers already find and identify new items, it makes sense to have them automatically update the preorder posts as well. This is done by looking at all of the preorder link threads created in the last day, giving a heavier weight to those created in the past 2 hours, and finding which post is most likely to be the correct post based on the count of shared words in the title of the item and the title of the blog. Once a candidate blog post is identified, it is updated with the new item.

This will not work perfectly every time, however. If we find a new item but that item was just late going up and does not correspond to any preorder posts, we might accidentally append it to a random preorder post. As such, we need to have some sort of threshold for the number of common words when determining in which post this preorder link belongs. This helps us with false positives but it creates a new problem. If a blog post has a sufficiently small number of words in the title, we may never cross the threshold of required common words and nothing will ever be appended to the post.

There are ways around this but for now, I am going to pilot this project and see how it does and refine it from there.

No comments:

Post a Comment

Hot Cash Give Away Jan 2021

  Hey everyone! Time for another round of Hot Cash codes! Hot Cash ends soon so be sure to use these ASAP! Comment below with the codes you ...