Seeking Validation: Are My Models Adequately Executed?

[Felix_Requena]

Good morning,

I am running both linear and non-linear models in brms, and my results are quite similar. First, I am running my model using my response variable, along with two predictors and one random effect. Below are the results. I have also created a non-linear model. After extensive research, I’m not entirely certain if it’s the correct approach for a non-linear model, but I have included quadratic effects for both predictors. After doing some reading, I think the models have been executed well, but I’m not entirely sure. Could you please provide me with some assistance? Thank you very much!

Cheers,
Felix

Below are the results I obtained:

fit_log <- bf(Social_dist ~ 1 + X50 + Num_Unique_Healthy_Ids +  (1|p|Name)) + lognormal()
fit_log <- brm(fit_log, 
                     data = final_data,
                     iter = 5000,
                     warmup = 1000,
                     thin = 4,
                     chains = 4,
                     cores = corecount)

> summary(fit_log)
 Family: lognormal 
  Links: mu = identity; sigma = identity 
Formula: Social_dist ~ 1 + X50 + Num_Unique_Healthy_Ids + (1 | p | Name) 
   Data: final_data (Number of observations: 30538) 
  Draws: 4 chains, each with iter = 5000; warmup = 1000; thin = 4;
         total post-warmup draws = 4000

Group-Level Effects: 
~Name (Number of levels: 292) 
              Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(Intercept)     0.30      0.01     0.27     0.33 1.00     2133     3041

Population-Level Effects: 
                       Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
Intercept                  1.45      0.02     1.41     1.49 1.00     1027
X50                        0.10      0.01     0.08     0.12 1.00     3443
Num_Unique_Healthy_Ids    -0.11      0.01    -0.12    -0.09 1.00     3532
                       Tail_ESS
Intercept                  2006
X50                        3890
Num_Unique_Healthy_Ids     3737

Family Specific Parameters: 
      Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sigma     0.83      0.00     0.82     0.83 1.00     4041     3974

Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).

I have included quadratic effects for both predictors in the non-linear model.

fit_nonlinear <- bf(Social_dist ~ 1 + X50^2 + Num_Unique_Healthy_Ids^2 + (1|p|Name)) + lognormal()
fit_nonlinear <- brm(fit_nonlinear, 
               data = final_data,
               iter = 5000,
               warmup = 1000,
               thin = 4,
               chains = 4,
               cores = corecount)

Family: lognormal
Links: mu = identity; sigma = identity
Formula: Social_dist ~ 1 + X50^2 + Num_Unique_Healthy_Ids^2 + (1 | p | Name)
Data: final_data (Number of observations: 30538)
Draws: 4 chains, each with iter = 5000; warmup = 1000; thin = 4;
total post-warmup draws = 4000

Group-Level Effects:
~Name (Number of levels: 292)
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(Intercept) 0.30 0.01 0.27 0.33 1.00 2033 3073

Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept 1.45 0.02 1.41 1.49 1.00 1065 2007
X50 0.10 0.01 0.08 0.12 1.00 3478 3663
Num_Unique_Healthy_Ids -0.11 0.01 -0.13 -0.09 1.00 3618 3818

Family Specific Parameters:
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sigma 0.83 0.00 0.82 0.83 1.00 3740 3356

Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).

This is the LOO comparison.

loo_compare(loo_linear, loo_nonlinear)
elpd_diff se_diff
fit_log 0.0 0.0
fit_nonlinear -1.0 0.4