Plumber/Jsonlite | lexical error: invalid char in json text

[alxrma]

Hello R programmers

I have been using the following implementation of plumber locally:

startWebListener <- function(modelName, httpPort) {
  cust_json <- function() {
    function(val, req, res, errorHandler) {
      tryCatch({
        json <- jsonlite::toJSON(
          val,
          auto_unbox = TRUE,
          pretty = FALSE,
          digits = 15
        )
        print(json)
        
        json <- gsub("\"NaN\"", "null", json)
        json <- gsub("\"N/A\"", "null", json)
        
        res$setHeader("Content-Type", "application/json")
        res$body <- json
        
        return(res$toResponse())
      }, error = function(e) {
        errorHandler(req, res, e)
      })
    }
  }
  register_serializer("cust_json", cust_json)
  r <- plumb(modelName)
  r$run(host = "0.0.0.0", port = httpPort)
}

Recently I updated plumber, jsonlite and httr libraries to their latest version. After this, I have been getting the following error response every time I use the API:

<simpleError: lexical error: invalid char in json text.
                                       incomingJSON={      "Id": "fa67
                     (right here) ------^

Not sure where the issue comes from. My data has not changed and there are no special characters. The application and the R code are running locally under the same network.

Any idea?

Thanks

[meztez]

there is no references to incomingJSON from httr, plumber or jsonlite, do you create this object somewhere?

[alxrma]

It is created by the calling application and passed into the body of the POST request.

I do have some additional treatment, but you can consider the payload to look like this:

before treatment:

incomingJSON={
    "Id": "fa67e953-1128-4cf4-88ce-526f94b8dc03",
    "ModelId": 8130
}

after:

{
    "Id": "fa67e953-1128-4cf4-88ce-526f94b8dc03",
    "ModelId": 8130
}

Note that it used to work perfectly prior updating plumber, jsonlite and httr

[meztez]

This is my hypothesis,
Previously plumber ignored incoming request content-type headers so your request would be handled like a query, setting the incomingJSON parameter to a string with value

"{
    \"Id\": \"fa67e953-1128-4cf4-88ce-526f94b8dc03\",
    \"ModelId\": 8130
}"

Now plumber has parsers (https://www.rplumber.io/reference/parsers.html) that respects the incoming request content-type header. If you send a query with content-type "application/json" plumber will parse the whole body of your request has a json formatted string.

If you pass in the request body

{
  incomingJSON:{
      "Id": "fa67e953-1128-4cf4-88ce-526f94b8dc03",
      "ModelId": 8130
  }
}

plumber will parse this information and incomingJSON value in your endpoint will be a list of two elements "Id" and "ModelId"

Hopefully this makes it more clear?

[alxrma]

OK so looks like the new parser will break if the body contains non-JSON with content-type = application/json

I have downgraded the library to 0.4.6 to fix the issue temporarily

Unfortunately at this time, I do not have much control over the application that is generating the payload

[meztez]

you can still override the default json parser to adapt to your payload

But your payload is not really json to begin with. Let me do a quick test.

I'm thinking about forcing plumber tu use the query parser so you would have the same behavior as before

Try doing

register_parser("json", parser_text, fixed = "application/json", regex = "json")
pr <- pr("plumber.R")

before sourcing your API or creating your route object. You will effectively be replacing the json paser by the regular query string parser, which would be similar to the old behavior.