Appropriate statistical test for this situation?

[JeanTate]

Can anyone help me with this, please?

It's about how you go about trying to decide if two distributions are consistent, statistically speaking; specifically, what statistical test, or tests, is (are) most appropriate to use.

Here's the data:

N(A) N(B) G/R/P
0043 0046 #101
0264 0235 #102
0033 0029 #103

N(A) N(B) G/R/P
0172 0201 #201
1686 1496 #202
1444 1336 #203

Astronomical observations were made, and reduced to data. By two quite different teams, using different telescopes, cameras, data reduction routines, etc. In the first two columns (N(A) and N(B)) are counts, with leading zeros to ensure everything lines up nicely. "A" and "B" are two states, or conditions, or ... they are distinct and - for the purposes of this question - unambiguous. So the first cell of the first table says 43 cases of A (or with condition A) were observed.

The third column (G/R/P) is the name/label of the group/region/population observed. The two teams each observed the same group/region/population; the first table is the first team's data, the second the second.

There is nothing to say what the underlying ("true") distribution is, or should be. Nor any way to compare what the two teams observed: the 43 could be a proper subset of the of 172 (first column, first row), an overlap, or disjoint. However, assume no mistakes at all in the assignment of "A" and "B".

Clearly, the two distributions - of states A and B, across the three groups/regions/populations - are different. However, is that difference statistically significant? What test - or tests - are appropriate, here, to use?

More details? Consider these:

i) what's observed is white dwarf stars, in three different clusters; A is DA white dwarfs, B DB ones
ii) globular clusters, in three different galaxies; A is 'red' GCs, B 'blue'
iii) spiral galaxies, in three different galaxy clusters; A is 'anti-clockwise', B 'clockwise'
iv) radio galaxies, in three different redshift bins; A is 'FR-I', B 'FR-II'
v) GRBs, in three different RA bins; A is 'long', B is 'short'

(I don't think the details matter, in terms of the type of statistical test to use; am I right?)

[neufer]

Can anyone help me with this, please?

It's about how you go about trying to decide if two distributions are consistent, statistically speaking; specifically, what statistical test, or tests, is (are) most appropriate to use.

Here's the data:

N(A) N(B) G/R/P
0043 0046 #101
0264 0235 #102
0033 0029 #103

N(A) N(B) G/R/P
0172 0201 #201
1686 1496 #202
1444 1336 #203

Normally one wishes to know if distributions are INconsistent, statistically speaking.
----------------------------------------------
Fisher's Exact Test
http://www.langsrud.com/fisher.htm
------------------------------------------
TABLE = [ 1686 , 1496 , 264 , 235 ]
Left : p-value = 0.5327144130737004
Right : p-value = 0.505727115298653
2-Tail : p-value = 1
------------------------------------------
TABLE = [ 1686 , 1496 , 1444 , 1336 ]
Left : p-value = 0.796897069577982
Right : p-value = 0.21810666075571303
2-Tail : p-value = 0.4355065231729098
------------------------------------------
TABLE = [ 1686 , 1444 , 264 , 33 ]
Left : p-value = 4.9394443016633465e-36
Right : p-value = 1
2-Tail : p-value = 6.848455756091371e-36
------------------------------------------
N(A) & N(B) appear to be consistent with each other.

However, distributions #1.. & #2.. are highly inconsistent with each other.

(Not quite sure what you are asking.)

[JeanTate]

Thanks neufer.

Yes, I should have asked about inconsistency ; I struggle mightily to understand this (I am entirely self-taught), so please be gentle!

You ran three 2x2 contingency table tests (is that the correct way to use the terms?), right?

Perhaps if I re-state the question in terms of a ratio, of N(A)/N(B)?

A/B G/R/P
0.93 #101
1.12 #102
1.14 #103

A/B G/R/P
0.86 #201
1.13 #202
1.08 #203

Now 0.93 != 0.86, 1.12 !=1.13, and 1.14 != 1.08, so the two teams' values of the three ratios are not the same ( )

The ratio in group/region/population #01 is not, necessarily, the same as that in G/R/P #02 (ditto #03).

Given the underlying data - which is counts, not ratios - is the ordered triple* (0.93, 1.12, 1.14) inconsistent with the ordered triple (0.86, 1.13, 1.08)?

* am I using the term correctly?

[neufer]

Given the underlying data - which is counts, not ratios - is the ordered triple* (0.93, 1.12, 1.14)
inconsistent with the ordered triple (0.86, 1.13, 1.08)?

• Yes. (Based upon the number of observations.)

N(A+B) G/R/P
0580 #100
0062 #103

N(A+B) G/R/P
3555 #200
2780 #203
------------------------------------------
Fisher's Exact Test
http://www.langsrud.com/fisher.htm
------------------------------------------
TABLE = [ 3555 , 2780 , 580 , 62 ]
Left : p-value = 5.780648305205974e-75
Right : p-value = 1
2-Tail : p-value = 7.835715832276966e-75
------------------------------------------

* am I using the term correctly?

• Ordered triplet?

[JeanTate]

Thanks again neufer.

So, for:
#103, 33+29 = 62
#203, 1444+1336 = 2780

But what are #100 and #200?

If I add the four counts for #101 and #102, I get 588 (not 580). And for #201 and #202, I get 3555. Was "580" a typo?

More generally, a "not inconsistent" conclusion would involve doing three 2X2 tests, with #101+#102 and vs #201+#202 and #203 (as you just did), #101+#103 and #102 vs #201+#203 and #202, and #101 and #102+#103 vs #201 and #202+#203 (and only if all three gave a 'not inconsistent' result)?

What to do if there are four (or more) groups/regions/populations?

[neufer]

Thanks again neufer.

So, for:
#103, 33+29 = 62
#203, 1444+1336 = 2780

But what are #100 and #200?

If I add the four counts for #101 and #102, I get 588 (not 580). And for #201 and #202, I get 3555. Was "580" a typo?

Yes.

The Fisher Exact Test was designed to handle 4 mutually exclusive sets like:

• #103, #203, Not #103, and Not #203

Thanks again neufer.

More generally, a "not inconsistent" conclusion would involve doing three 2X2 tests, with #101+#102 and vs #201+#202 and #203 (as you just did), #101+#103 and #102 vs #201+#203 and #202, and #101 and #102+#103 vs #201 and #202+#203 (and only if all three gave a 'not inconsistent' result)?

Correct. But since the first one already produced a 'not consistent' result that was sufficient.

What to do if there are four (or more) groups/regions/populations?

Group them in twos and start by testing the one that is seems most likely to produce a 'not consistent' result.

(Alternatively, one could perform chi-squared testing.)
http://en.wikipedia.org/wiki/Chi-squared_test

[JeanTate]

So, out of all three 2X2 pairs, is it possible to find just a single inconsistent one? Or must there be at least two?

If there are four (or more) G/R/Ps, the patterns of results from the pair testing could be used to identify a single G/R/P which is inconsistent, right?

I'm not sure how the chi-squared test should be applied, for the data in my post; for example, it is counts (not frequencies), and there is no "expected" distribution. Can you explain in a bit more detail please?

[neufer]

So, out of all three 2X2 pairs, is it possible to find just a single inconsistent one? Or must there be at least two?

Out of all three 2X2 pairs, is it sufficient to find just a single inconsistent one.

If there are four (or more) G/R/Ps, the patterns of results from the pair testing could be used to identify a single G/R/P which is inconsistent, right?

I think that's right.

I'm not sure how the chi-squared test should be applied, for the data in my post; for example, it is counts (not frequencies), and there is no "expected" distribution. Can you explain in a bit more detail please?

Traditionally this is a chi-squared test problem but I'm too lazy to relearn how to use that approach.

[JeanTate]

Thanks again for your replies, neufer.

The question I asked, in the OP, has been answered; to pursue this further - what statistical tests are appropriate for distributions with > four groups, for example; the details of how to do chi-square tests in such cases; and so on - I will find a different forum (or at least start a new thread here).