Unable to Fetch Data

[amit_limbasia]

I am trying to fetch data from the following URL: 'https://www.nseindia.com/api/option-chain-indices?symbol=NIFTY'.

When I access this URL in Google Chrome, it shows me a JSON output. I have successfully retrieved the data using similar configurations in Python and Postman. However, when I try to run the code in R, the console goes into a wait mode and I don't get any response.

I am currently using the following code:

R_Code <- Here

library(curl)
library(jsonlite)

headers <- c('User-Agent' = 'Mozilla/5.0')
req <- curl::new_handle()
curl::handle_setheaders(req, .headers = headers)
curl::handle_setopt(req, ssl_verifypeer = FALSE)
page <- curl::curl_fetch_memory('https://www.nseindia.com/api/option-chain-indices?symbol=NIFTY', handle = req)
data <- jsonlite::fromJSON(rawToChar(page$content))

data

Could you please help me understand why I'm not getting any output in R, while the same code works in other environments?

[technocrat]

@amit_limbasia

data <- jsonlite::fromJSON(rawToChar(page$content))

When I the code it returns partial HTML with a 403 (take a look at the page object.) This worked for me

library(httr)
library(jsonlite)
url <- 'https://www.nseindia.com/api/option-chain-indices?symbol=NIFTY'
headers <- c("User-Agent" = "Mozilla/5.0 (Macintosh;")
response <- GET(url, add_headers(headers))
json_data <- content(response, as = "text", encoding = "UTF-8")
parsed_data <- fromJSON(json_data)

[amit_limbasia]

Thank you for your prompt response, technocart!

response <- GET(url, add_headers(headers))

I ran the code you provided, but it seems to stop at the specified line, and I'm unable to receive a response. I'm wondering if there's a way to investigate why the code is not progressing further. Any suggestions or guidance would be greatly appreciated.

[technocrat]

@technocrat

parsed_data <- fromJSON(json_data)

At this point the file has been fetched and parsed but I haven't asked to do anything else with it, even to display its contents. Try

parsed_data

[amit_limbasia]

while Run This code in console

GET('https://www.nseindia.com/api/option-chain-indices?symbol=NIFTY', add_headers(c("User-Agent" = "Mozilla/5.0 (Macintosh;")))

Still won't get anything ahead.
It stuck here only
What to do?

[technocrat]

Run this script in the RStudio source pane

library(httr)
library(jsonlite)
url <- 'https://www.nseindia.com/api/option-chain-indices?symbol=NIFTY'
headers <- c("User-Agent" = "Mozilla/5.0 (Macintosh;")
response <- GET(url, add_headers(headers))
json_data <- content(response, as = "text", encoding = "UTF-8")
parsed_data <- fromJSON(json_data)
# peek
parsed_data$records$expiryDates
#>  [1] "06-Jul-2023" "13-Jul-2023" "20-Jul-2023" "27-Jul-2023" "03-Aug-2023"
#>  [6] "31-Aug-2023" "28-Sep-2023" "28-Dec-2023" "28-Mar-2024" "27-Jun-2024"
#> [11] "26-Dec-2024" "26-Jun-2025" "24-Dec-2025" "25-Jun-2026" "31-Dec-2026"
#> [16] "24-Jun-2027" "30-Dec-2027" "29-Jun-2028"

[amit_limbasia]

Did Same thing :

[technocrat]

Can you use a VPN to try a different network connection; this looks like it's timing out. If you know how to use network utilities you can also check with traceroute

[amit_limbasia]

Dear Technocrat,

I think there is no issue of network as I am trying this same code with Python and Also in Postman, Both working fine. For further code I require this in R but unable to get through.
Is there any setting in R or R studio which can be done?

This is my python code and work perfectly fine :

import requests
import json

headers = {'User-Agent': 'Mozilla/5.0'}
page = requests.get('https://www.nseindia.com/api/option-chain-indices?symbol=NIFTY', headers=headers)
data = json.loads(page.text)  # convert json data as object/data frame
data

Code run in just 2 second on count and get correct output

This is Postman screenshot time is 136ms

[nirgrahamuk]

from that succesful result, what shows for authorization, cookies(1) , and headers(20) ?

[amit_limbasia]

Also Without Any Header You can try same still you get same output

[technocrat]

Then I can only guess Windows, because my script runs identically and correctly on both macOS and Ubuntu.