R Quantiles estimate with CI

Hi,

I'm trying to derive 25, 50 and 75% estimates & CI for Reponses. However R is giving quantiles estimate which is not present in dataset or its not realistic. But in SAS im getting correct results. I tried different types with in survfit, however its still same and also tried using summarize function but not able to match it with SAS values.

Below is the data
#0 -Responder, 1- nonresponder


library(survival)

Duration<-c(4.14, 12.29, 16.57, 15.43, 5.00, 15.71, 8.86, 25.43)
Response<-c(0,0,1,1,0,1,0,1)
trt<-1

dt<-data.frame(Duration,Response,trt)

kmfit <- survfit(Surv(Duration,1-Response)~ trt,conf.int=0.95, conf.type=c("log-log"), data = dt)

quantile(kmfit, probs = c(0.25, 0.50, 0.75), conf.int = FALSE)


SAS code


proc lifetest data=data conftype=loglog timelist=0 to 24 by 4;
time duration*cnsr(1);
run;


Thanks

1 Like

Perhaps post a little bit of your code.

1 Like

Is Duration or censor value the response variable? I am assuming Duration but I don't know.

AS @ startz starts says, we really need to see your code. The actual data would also help.

It might also help if you supply the SAS code.

Please copy the R code and paste it here between

```

````

This gives us formatted code that we can easily copy, paste, and run .

A handy way to supply data is to use the dput() function. Do dput(mydata) where "mydata" is the name of your dataset. For really large datasets probably dput(head(mydata, 100) will do. Paste it here between

```

```

Thanks

I have added code above, please check

1 Like

Are there typos in this command?

1 Like

Yes. Some problems in the code:

0 -> Responder ,1<-Non responder # 1<-Non responder is illegal, the **,** is wrong

# R code

kmfit<-survift(surv(Duration,1-Responder), conf.type='log-log',conf.int=0.95, data) # survift --> survfit
                                                                                    # surv --> Surv
quantile(kmfit,probs=c(0.25,0.5,0.75)  # missing **)**

I have never used {survival} but I do not understand where "data" in the kmfit equation is coming from unless Duration and Non responder should be in something like this:

DDT <- data.table(Duration =  c(4.14, 12.29, 16.57, 15.43, 5.00, 15.71, 8.86, 25.43),
Response = c(0, 0, 1, 1, 0, 1, 0, 1))

However, currently with DDT as data.table, a data.frame, or a tibble

kmfit <- survfit(Surv(Duration, 1 - Responder), conf.type = "log-log", conf.int = 0.95, DDT)

is giving me

Error: object 'Duration' not found

I'll have to read up on {survival} and see if I can get some clue to what is going on.

1 Like

@KiranH; It would be much better if you were to copy and paste the code you have run as @jrkrideau has suggested. It's only a few lines.

1 Like

I have added exact code for datasets and survival analysis, please check

Using the new dataset the equation runs but the results are not good.

New data set

> DDT
   Duration Response   trt
      <num>    <num> <num>
1:     4.14        0     1
2:    12.29        0     1
3:    16.57        1     1
4:    15.43        1     1
5:     5.00        0     1
6:    15.71        1     1
7:     8.86        0     1
8:    25.43        1     1
> DDT[, var(trt)]
[1] 0

I don't work with time series data but I cannot help but think that a treatment effect with variance zero is not likely to give good results.

Without the original working code and actual data, or a reasonable mock-up of the data if it is confidential, , I do not see what we can do to help.

1 Like

I haven't used survfit, but I share @jrkrideau's concern about an apparently constant treatment effect.

In addition, you only have 8 data points. Finding quantiles will require some interpolation. It is possible that R and SAS do that interpolation differently. So there is a question about how different the answers are.

1 Like

i just realised that i should have supplied the output of kmfit.

> kmfit 
Call: survfit(formula = Surv(Duration, 1 - Response) ~ trt, data = DDT, 
    conf.int = 0.95, conf.type = c("log-log"))

     n events median 0.95LCL 0.95UCL
[1,] 8      4   12.3    4.14      NA
1 Like

Thanks for sharing this, very helpful discussion.Have you tried comparing the type argument in quantile()? The default type=7 gives different results than SAS which uses type=2 or type=3 depending on the procedure. You can check with quantile(x, probs, type=2).

1 Like

Great discussion on quantile estimation. For anyone comparing across different software, one useful reference is Hyndman & Fan (1996) which defines 9 quantile algorithm types. The default type=7 in R gives sample quantiles with linear interpolation of the order statistics, while type=2 provides interpolation of the median-unbiased estimator. Always worth checking which type matches your analysis requirements.

Arrh, one more thing to worry about. And it is not even Excel!

Thanks, the paper is in my bib system.

With the help of ChatGPT, I review the code in survival package (survival source: R/quantile.survfit.R). A portion of the description matches what the code demonstrates.

  # quantile = where a horzontal line at p intercects the curve.  At each
  #  x the curve of 1-y jumps up to a new level
  # The most work is to check for horizontal lines in the survival 
  #  curve that match one of our quantiles within tolerance.  If any
  #  p matches, then our quantile is the average of the given x and
  #  the x value of the next jump point, i.e., the usual midpoint rule
  #  used for medians.  
  # A flat at the end of the curve is a special case, as is the quantile
  #  of 0.
  indx1 <- approx(y+tol, 1:n, p, method="constant", f=1)$y
  indx2 <- approx(y-tol, 1:n, p, method="constant", f=1)$y
  quant <- (x[indx1] + x[indx2])/2
  quant[p==0] <- x[1]
  if (!is.na(y[n])) {
      lastpt <- (abs(p- y[n]) < tol)  # end of the curve
      if (any(lastpt)) quant[lastpt] <- (x[indx1[lastpt]] + xmax)/2
  }

The results of your code were

> quantile(kmfit, probs = c(0.25, 0.50, 0.75), conf.int = FALSE)
   25    50    75 
 6.93 18.86    NA 

Also I run the summary command

> summary(kmfit)
Call: survfit(formula = Surv(Duration, Response == 0) ~ 1, data = dt, 
    conf.int = 0.95, conf.type = "log-log")

  time n.risk n.event survival std.err lower 95% CI upper 95% CI
  4.14      8       1    0.875   0.117        0.387        0.981
  5.00      7       1    0.750   0.153        0.315        0.931
  8.86      6       1    0.625   0.171        0.229        0.861
 12.29      5       1    0.500   0.177        0.152        0.775

And you could see that 18.86 equal to

12.29/2+25.43/2

I could not run the SAS code that you provide, so I rewrite the code here. With the code, you could see the plot with 95% confidence interval.

data dt;
    input Duration Response trt;
    datalines;
4.14 0 1
12.29 0 1
16.57 1 1
15.43 1 1
5.00 0 1
15.71 1 1
8.86 0 1
25.43 1 1
;
run;


proc lifetest data=dt conftype=loglog plots=survival(cl);
    time Duration * Response(1);
    strata trt;
run;

The tables would be

As why the table did not show the median value, I read the SAS document (https://support.sas.com/documentation/onlinedoc/stat/123/lifetest.pdf) and it said

Then I redefined the outcome event .

The R code would be

kmfit <- survfit(
  Surv(Duration, Response == 1) ~ 1,
  data = dt,
  conf.int = 0.95,
  conf.type = "log-log"
)
quantile(kmfit, probs = c(0.25, 0.50, 0.75), conf.int = FALSE)

## the output
   25    50    75 
15.57 16.14 21.00 

The SAS code would be

proc lifetest data=dt conftype=loglog plots=survival(cl);
    time Duration * Response(0);
    strata trt;
run;

Please see

75	21.0000	LOGLOG	15.4300	.
50	16.1400	LOGLOG	15.4300	.
25	15.5700	LOGLOG	15.4300	16.5700

Not surprisingly, they came out the same.

1 Like