Capturing 4xx response bodies with httr2::req_perform_stream

jwimberley · June 7, 2024, 1:39pm

Hello,

The httr2 package supports capturing the bodies of 4xx responses and including the in the R error message using the req_error method:

library(httr2)
req <- ...
req <- req_error(req, body = \(resp) resp_body_string(resp)
resp <- req_perform(req)

However, error responses are not handled and converted to R errors in the same way using req_perform_stream -- a response object with an error status code but no body is returned.
Moreover, the response body isn't included in the returned response object,
and so the error message cannot be extract manually. This is true even when setting is_error to a lambda returning FALSE, which in the req_perform successfully turns off the http-to-R error handling and yields a 4xx response with a body. The following obfuscated example uses a private API and so is not a reprex:

> httr2::req_perform(request)
<httr2_response>
POST http://obfuscated
Status: 400 Bad Request
Content-Type: text/plain
Body: In memory (147 bytes)
Browse[3]> httr2::req_perform_stream(request, cb, buffer_kb = buffer_kb)
<httr2_response>
POST http://obfuscated
Status: 400 Bad Request
Content-Type: text/plain
Body: None

My current workaround is to repeat the request using req_perform when req_perform_stream returns an error status, but this can mean repeating large POST requests. Is there a way to get the 400 response's along with the single req_perform_stream request?

dromano · June 8, 2024, 1:42pm

jwimberley:

> httr2::req_perform(request)
<httr2_response>
POST http://obfuscated
Status: 400 Bad Request
Content-Type: text/plain
Body: In memory (147 bytes)
Browse[3]> httr2::req_perform_stream(request, cb, buffer_kb = buffer_kb)
<httr2_response>
POST http://obfuscated
Status: 400 Bad Request
Content-Type: text/plain
Body: None

This difference in behavior makes me wonder whether it's intentional, so I would suggest posting an issue on github:

and sharing the issue link here.

jwimberley · June 10, 2024, 1:58pm

@dromano Thanks for the suggestion -- I have created issue

github.com/r-lib/httr2

Cannot capture 4xx response bodies with req_perform_stream

opened 01:57PM - 10 Jun 24 UTC

jwimberl

I'm finding a difference in behavior between `req_perform` and `req_perform_stre…am` when it comes to capturing HTTP errors and translating them to R errors, and moreover in capturing 4xx response bodies. Both occur straightforwardly with `req_perform`, following the documented instructions: ```r req <- ... req <- req_error(req, body = \(resp) resp_body_string(resp) resp <- req_perform(req) ``` This will throw an R error that includes the text body from the error message. However, error responses are not handled and converted to R errors in the same way using `req_perform_stream`: ```r > req <- ... > req <- req_error(req, body = \(resp) resp_body_string(resp) > req_perform(req) <httr2_response> POST http://obfuscated Status: 400 Bad Request Content-Type: text/plain Body: None ``` This returns a response object with an error status code is returned, but does not throw an R error. It also does not include any body. This is true even when setting `is_error` to a lambda returning FALSE. In the `req_perform` case, that successfully turns off the http-to-R error handling and yields a 4xx response with a body, but for `req_perform_stream` the response body is still empty. The following obfuscated example uses a private API and so is not a reprex: ```r > request <- ... > request <- httr2::req_error(request, is_error = \(z) FALSE) > httr2::req_perform(request) <httr2_response> POST http://obfuscated Status: 400 Bad Request Content-Type: text/plain Body: In memory (147 bytes) > httr2::req_perform_stream(request, cb, buffer_kb = buffer_kb) <httr2_response> POST http://obfuscated Status: 400 Bad Request Content-Type: text/plain Body: None ``` My current workaround is to repeat the request using `req_perform` when `req_perform_stream` returns an error status, but this can mean repeating large POST requests. Is there a way to get the 400 response's along with the single `req_perform_stream` request?