How to automate reports, web scrape data using Rstudio

[mailedvirus]

I am a newbie to R, and working on a project to automate report generation and scrape data using R for tracking diseases.

Since I have no prior experience with R, apart from regression and logistic analysis, hence seriously need guidance as to how to go about this project in terms of coding, the methodology to be followed. Any help in this regard would be awesome.

Thanks in advance!!

[andresrcs]

Hi, welcome!

Please have a look to our homework policy, homework inspired questions are welcome but they should not include verbatim instructions from your course.

FAQ: Homework Policy

Can I ask questions from a course I am taking here? General questions are always welcome! Please do ask general questions about things like: How to use R How to use the RStudio IDE or RStudio Cloud How to work with tidyverse packages Where to find resources to help you learn or solve problems Specific questions can be OK, if you follow these rules: Never copy-paste instructions from an assignment (even for online courses). Explicitly mention the course you are taking and use the #homework tag. Ask your question as a reproducible example (reprex), preferably prepared for posting using the reprex package. Want to maximize your chances of getting the help you need? Keep readin…

[mailedvirus]

Thanks, will surely keep forum guidelines in mind. This was actually summary of what needs to be done and not the exact question itself. Just not sure how to go about it.

[jhk0530]

It will be better if you mention these.

• What you want to do ( Ultimate Goal )
• What you can do ( or what you did )
• What you can't do ( needs help )

Here are some packages (library) that you can start with.

• For scrape data , Rselenium can be used. Rselenium
• For automate work. CronR can be used. cronR
• For generating reports, Rmarkdown can be used. Rmarkdown

Bests.

[mailedvirus]

Thanks a lot for the guidance.

The ULTIMATE GOAL is to scrape data from websites (HTML/CSS), automate report generation in pdf/HTML format (output), followed by creating interactive dashboards using tableau. Now i know how to make dashboards etc in tableau. But really don't know how to Scrape date using R (which packages to install, reference code guide/tutorial etc for understanding) and automate report generation.

I have tried Rcurl() & Readlines() for reading and parsing data directly from the websites, but still a long way off from perfecting it.

[jhk0530]

I think

• Try httr package for scraping HTML/CSS webpage first will be good.
• See this link

before generating report.

[mailedvirus]

ok, will start with httr. And thanks a lot again for helping me out.

[EconomiCurtis]

Since it sounds like you're still in the phase of your project of 'how do I even approach this kinds of problems with R?', I'd also like to encourage you to check out Garret and Hadley's book, https://r4ds.had.co.nz/. It contains a nice collection of chapters on various tools useful to project a project like this, and is a nice onboard to R and the Tidyverse generally.

When you do get to the point where you are looking for help with a specific coding question, I'd encourage you to check out FAQ: Tips for writing R-related questions. It's a nice guide to improve the probability folks who want to help can understand your problem, and quickly reply with useful suggestions.