On June 25, Reddit announced plans to update its robot exclusion protocol (robots.txt file), which gives instructions about how third parties are permitted to crawl Reddit’s content. Web crawlers are ...