Re: The select record MEDIAN calculation

[Andrew Johnson <anj@aps.anl.gov>]

Hi Dennis,

I can explain your results; the main issue is how many inputs the record 
is actually using.

Dennis Armstrong wrote:
> Quote "In probability theory and statistics, a median is described as
> the number separating the higher half of a sample, a population, or a
> probability distribution, from the lower half. The median of a finite
> list of numbers can be found by arranging all the observations from
> lowest value to highest value and picking the middle one. If there is an
> even number of observations, the median is not unique, so one often
> takes the mean of the two middle values." unquote
>
> When I use an even number of input values the output is wrong as it
> fails to produce the average value of the two middle input values.

Please note the last sentence of your Wikipedia definition includes the 
words "one often" â that doesn't mean one has to.  I didn't write it, 
but I think the select record's decision of what to do when there are an 
even number of samples is reasonable.  IMHO the median of a set should 
always be a member of that set, whereas if you take the mean of the 
middle two items that's only the case if the two are the same.  We can 
add another select method to average the middle two if someone needs it 
though.

> When I use an odd number of input values this is sometimes wrong as it
> fails to sometimes select the middle input values.

I think this may be because the record uses NaN field values to exclude 
inputs from the calculation, but it only ever sets the fields to NaN 
when the record is initialized.  If your 12th field contained anything 
other than a NaN when you were only setting 11 input values, the record 
was still using the 12th.

> So is anybody using the select record in this mode? and what results do
> you get?

Here's what I get when I test the latest CVS version (the median 
calculation has not changed since R3.14.9 although other parts that 
don't affect these results have changed slightly).  Each column below is 
a new experiment with one additional input â the NaN values are not used 
in the median calculation.  SELN tells you which item it's selected as 
the median value after sorting, and VAL is that median value.

sel.A     nan  1    1    1    1    1    1    1    1    1    1    1    1
sel.B     nan  nan  2    2    2    2    2    2    2    2    2    2    2
sel.C     nan  nan  nan  3    3    3    3    3    3    3    3    3    3
sel.D     nan  nan  nan  nan  4    4    4    4    4    4    4    4    4
sel.E     nan  nan  nan  nan  nan  5    5    5    5    5    5    5    5
sel.F     nan  nan  nan  nan  nan  nan  6    6    6    6    6    6    6
sel.G     nan  nan  nan  nan  nan  nan  nan  7    7    7    7    7    7
sel.H     nan  nan  nan  nan  nan  nan  nan  nan  8    8    8    8    8
sel.I     nan  nan  nan  nan  nan  nan  nan  nan  nan  9    9    9    9
sel.J     nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  10   10   10
sel.K     nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  11   11
sel.L     nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  12
sel.SELN  0    0    1    1    2    2    3    3    4    4    5    5    6
sel.VAL   0    1    2    2    3    3    4    4    5    5    6    6    7

Notes:
* When there are an even number of inputs it always picks the higher of 
the middle two values.
* The first column's result is wrong, when there are no inputs at all 
the result should be a NaN.  I'll fix that for R3.14.10.
* SELN would probably be more useful if it gave you the number of inputs 
it's using rather than half that number, truncated.  This behaviour was 
new in R3.14.9, so unless anyone objects I'll probably change it for 
R3.14.10.

HTH,

- Andrew
--
When a distinguished but elderly scientist states that something is
possible, he is almost certainly right.  When he states that something
is impossible, he is very probably wrong.  -- Arthur C. Clarke