How to use html_nodes in a dynamic webpage

[CAUBEBUTCHI]

I would like to get all the news title and short description of this webpage but it returns "character (0)".

Here is what I did:

# News of WB
link_wb <- "http://www.worldbank.org/en/news/all?displayconttype_exact=Speeches+and+Transcripts&lang_exact=English&qterm="
wb_speechs <- read_html(link_wb)

# get the news title
wb_title <- wb_speechs %>%
  html_nodes(xpath = "/html/body/div[4]/div/div/customsearch/main/div/div/div/div[2]/news/div/div/div/div[2]/div/div[1]/ul/li/div/h4") %>%
  html_text()
wb_title

I've tried with some class, xpath and so on, but get no results. Please advice! Thank you a lot!

[mfherman]

Check out this recent thread regarding scraping dynamic webpages:

Not able to scrape a website

I am trying to get the number of products of brand jealous-21 on 2 websites. It is just for demo purposes. I run this code to get the number library(rvest) library(httr) read_html('http://www.jabong.com/find/jealous-21') %>% html_nodes('span.product-count') %>% html_text() %>% .[1] #>[1] "278 products" this gives me [image] But when I try to do the same on another site that doesn't give me anything read_html('https://www.myntra.com/jealous-21') %>% html_nodes('.horizontal-filters-sub') %>% html_text() This is what I want [image] and it gives me nothing. What should I do here.