Is there a way to fully download or scrape a full subreddit or say stackoverflow since they’ve both committed themselves to enshittification and alienating their userbases?
asking because that seems difficult to do and there’s a lot of useful information on both sites
Its not super hard, but the main hurdle will by bypassing whatever api limits there are such as by using multiple accounts
Certain libraries like praw still work to some extent (my discord bot is still running somehow) but trying to scrape all of the posts in a sub might have to be done slowly. You might be able to sort by old so that the results dont move relative to the page and then go page by page.
Is there a way to fully download or scrape a full subreddit or say stackoverflow since they’ve both committed themselves to enshittification and alienating their userbases?
asking because that seems difficult to do and there’s a lot of useful information on both sites
Its not super hard, but the main hurdle will by bypassing whatever api limits there are such as by using multiple accounts
Certain libraries like praw still work to some extent (my discord bot is still running somehow) but trying to scrape all of the posts in a sub might have to be done slowly. You might be able to sort by old so that the results dont move relative to the page and then go page by page.