standard error with ml_* function in sparklyr?

[xinxi813]

Dear list,

Does anyone know how to get standard error of the model estimates from ml_linear_regression(), ml_logistic_regression(), and so on? I tried the following code, but could not find the way to get S.E. Could you give me some advice please? Thank you very much.

sc_mtcars%>%ft_one_hot_encoder("gear","gear1")%>%ml_linear_regression(hp~gear1+wt)
Formula: hp ~ gear1 + wt

Coefficients:
(Intercept)     gear1_0     gear1_1     gear1_2     gear1_3     gear1_4          wt 
   69.84297     0.00000     0.00000     0.00000   -79.65578  -105.33888    47.76914

[Chuck]

You need to assign it to an object at the end with, say: -> fit
Then do summary(fit) on the next line.

[xinxi813]

Hi @Chuck,

Like this? It seems didnt work. What did I miss?

sc_mtcars%>%ft_one_hot_encoder("gear","gear1")%>%ml_linear_regression(hp~gear1+wt)->fit
summary(fit)
Deviance Residuals:
Min 1Q Median 3Q Max
-81.069 -21.774 -3.935 11.983 94.621

Coefficients:
(Intercept) gear1_0 gear1_1 gear1_2 gear1_3 gear1_4 wt
69.84297 0.00000 0.00000 0.00000 -79.65578 -105.33888 47.76914

R-Squared: 0.7037
Root Mean Squared Error: 36.73

[Chuck]

Oops. I'm not a sparkly person. I assumed (incorrectly) that it would work like:

fit<-lm(mpg~wt, data=mtcars)
summary(fit)

Likely it's just as well that this doesn't work, as summary(fit) is rather a hot mess.

We need someone to erase me and give good sparkly advice.