Web scraping and google trends - COVID19 Research

Names: Mental Health Intelligence Team
From: Elle

Dear RStudio - We want to source any web data that relates to people's mental health during the Covid-19 crisis. I think the first issue is what we can scrape without falling foul of GDPR rules and 'medical' data. There have been some suggestions to look at Google trends as opposed to any sensitive web scraping. If there was some code we could use, that would be very helpful :slight_smile:

Happy to try to help! Are there particular data sources you would like to pull the data from? If so, we could try to help get started with scraping the data.

1 Like

Hi Mine, sorry I haven't been back in touch. I'm just waiting for the team to streamline the idea. I hope that's ok to park it here for now. I appreciate that if we delay too long, you might very well be too busy. If they haven't decided in a few days, I'll remove the post and try again later, once we get clarity. Thank you!

1 Like

The gtrendsR package is a great tool for querying the Google Trends API and returning data in easy-to-analyze format. See the documentation here.

1 Like

Thanks for this info, I'm going to take a look at this and get to grips with webscraping. It's way overdue in terms of learning to do it. The team outsourced the webscraping work to a specialist organisation as there were too many data issues with this one. I thought it was lovely that RStudio offered to get involved so my sincere thanks for offering to help with this topic :slight_smile:

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.