Index value considered significantly different from 0 ?

[hugo3]

Hello to all,

I have just developed a formula (of which I could give the different elements which compose it if need be) allowing to quantify the tolerance of an individual to the exposure. This gives me an index that can fluctuate from -1 to 1 (-1 The individual is not tolerant at all, 1 he is very tolerant, 0 there is no effect of the disturbance on the tolerance). I applied this formula for all my individuals. What I would like to know is if there is a threshold value, allowing me to assert statistically, that beyond or below this threshold the value of my index is different from 0.

For example, for individuals whose tolerance index = 0.05 or = -0.12, the values being close to 0, I would like to know if I can consider them as very slightly tolerant and slightly intolerant respectively, or if the values are too close to 0 and that it is therefore necessary to consider that there is no effect of the disturbance on the tolerance for these individuals.

I don't really know which statistical test on R is the most relevant to use in this case, and if you have any suggestions on how to apply it to my dataset.

Thanks in advance for your help.

[technocrat]

This is a model of the type

fit <- lm(Y ~ X,the_data)

testing the correlation of the response Y to the treatment X, which may be multiple variables X_n\dots X_n. Y may be the result of some correlation with X and the goal is to see if that correlation is distinguishable from random variation at some level of confidence. Whether 0.05 is or is not can be seen by examining the confidence intervals around the estimate.

[hugo3]

thanks for this answer.

to clarify, in fact I already calculate this index according to different parameters specific to the disturbance, so I get different indices for each situation.

What I would like to know is what test to perform to assert if values very close to 0, positive or negative, will be considered significantly far enough from 0 or too close, implying that we interpret the result as if it were a 0.

[startz]

Could you clarify, do you have one series that is the tolerance and another that is the exposure? When you say you want to know what value of the tolerance is significantly different from zero, what is the outcome for which you want to know if a given level of tolerance appears to matter?

[hugo3]

Ok I'll try to clarify.

I have a given territory that contains 10 individuals in captivity. The territory is divided into 3 zones, P1/P2 and Dist. The first two are exposed areas and Dist is a never exposed area. It is a question of exposure to visitors who can be in two different places: V1 and V2, visitors in V1 can observe in the P1 area, visitors in V2 can observe in the V2 area.

From the number of observations of each individual in each zone and for each type of possible visitor presence/absence configuration (which I consider here as the disturbance), I obtain an index that I call tolerance index, for each zone (P1/P2/Dist) that can fluctuate from -1 to 1.

For example, for the P1 zone
if index = -1 --> the individual does not tolerate at all the presence of visitors in zone V1, he will thus tend to avoid this zone in their presence.
if index = 1 --> the individual is very tolerant to the presence of visitors in V1 and will thus be more likely to be observed in this zone in their presence
if the index = 0 --> neutral tolerance, i.e. the presence of public in V1 will not necessarily affect the presence of the individual of interest in P1.

I will anyway perform statistical tests to define if at the group level, i.e. for the 10 individuals, they tend to have a higher or lower tolerance for certain areas.
But my main question is when my indices are weakly positive or weakly negative (=0.08 or =-0.1 for example) below or below what threshold can I consider my index value to be different from 0 and therefore not a neutral tolerance...

[technocrat]

I’ll look further but for n as small as 10 this likely will require a non-parametric approach

[hugo3]

that's right ! it's a small sample, but it's a preliminary work as a test ... I used mostly only non-parametric approches for the other tests. thank you for your help anyway !

[startz]

Let's follow-up on two of @technocrat's answers. You could run a regression with the number of people on the left and the tolerance on the right, and then see what values are significantly different from zero. But, as @technocrat said, it's kind of dicey with so few observations.

[technocrat]

Here's the framework I will work with

f(x) = y

x is a data frame of dim where x[1] is a numeric vector in -1:1 and x[2] is a factor with two levels, representing wither the captive animal is to assigned to one of two groups, one of which is DIST (unvisited) and the other is P1 or P2
y is a result of a test to determine as to whether the difference in mean(x) differs from 0 pre-determined level of confidence \alpha
f is a test, with n = 5 for each group, representing the mean x[1] observed for all subjects in the group, which will be the two-sided, two-sample t.test. The conventional \alpha = 0.05 is selected and we solve for the power that can be achieved to test null hypothesis H_0 \mu = 0 against the alternative hypothesis H_1 \mu ≠0, that the mean value of x[1] between the two groups does or does not differ from zero. Assumed effect sizes of 0.2,0.5,0.8 will be tested.

library(pwr)

# to find: power 

# small effect anticipated from visitation
pwr.t.test(d=0.2,
           n=5,
           sig.level=0.05,
           power = NULL,
           type="two.sample",
           alternative="two.sided")
#> 
#>      Two-sample t test power calculation 
#> 
#>               n = 5
#>               d = 0.2
#>       sig.level = 0.05
#>           power = 0.05904263
#>     alternative = two.sided
#> 
#> NOTE: n is number in *each* group

# medium effect predicted
pwr.t.test(d=0.5,
           n=5,
           sig.level=0.05,
           power = NULL,
           type="two.sample",
           alternative="two.sided")
#> 
#>      Two-sample t test power calculation 
#> 
#>               n = 5
#>               d = 0.5
#>       sig.level = 0.05
#>           power = 0.107686
#>     alternative = two.sided
#> 
#> NOTE: n is number in *each* group

# strong effect predicted
pwr.t.test(d=0.8,
           n=5,
           sig.level=0.05,
           power = NULL,
           type="two.sample",
           alternative="two.sided")
#> 
#>      Two-sample t test power calculation 
#> 
#>               n = 5
#>               d = 0.8
#>       sig.level = 0.05
#>           power = 0.2007395
#>     alternative = two.sided
#> 
#> NOTE: n is number in *each* group

^{Created on 2023-05-13 with reprex v2.0.2}

At each assumed level of effect the number of subjects will be sufficient to result in a moderately low risk of rejecting H_0 when it is actually true, that is the means of the two group differ by zero. However, all of the tests have low power—the ability to reject H_0 when H_1 is actually true. These correspond to the risks of Type I and Type II errors.

So, for example if we anticipate that the effect size of observation vs. non-observation on the stress index to be strong, the power of the test, 0.20 is weak and it will not be possible to reliably reject H_0 when we should.