From: jhmeyers@miu.edu (John H Meyers)
Subject: WEIGHT3 - Generalized Weighted Statistics
Date: 19 Nov 1996 00:00:00 GMT
Message-ID: <56sid5$dj3@news.iastate.edu>
references: <p88lTEANz6hyEw$F@ngmint.demon.co.uk> <56ee6c$5o5@news.iastate.edu> <328B1753.2D6E@camcon.co.uk>,<328D1B10.5964@echip.com>
organization: MIU Computer Services,  Fairfield, IA 52557.  Not Approved.
reply-to: jhmeyers@miu.edu
newsgroups: comp.sys.hp48


Included below is a new generalized program for *Weighted* Mean
and Standard Deviation of HP48 statistical data,
based on the following recent post:

In article <328D1B10.5964@echip.com>, Bob Wheeler <bwheeler@echip.com> writes:

> You can trust the statistical calculations in the HP48 for small,
> non-challenging data sets, but don't do any extensive, serious calculations
> on it -- especially simulations with the random number generator --
> random numbers are too important to be left to chance [!]

> The best and most accurate way to calculate [Mean & Standard Deviation] is
> with a simple looping function. All the serious computer packages use it,
> because it is both the fastest and the most accurate way to do the
> calculations. See Miller, Alan, J. (1989):
> Updating means and variances. Jour. Computational Physics.

> Let X(i) be an array of i=1...N interval midpoints,
> and W(i) a corresponding array of weights (frequencies).
> Use arrays SW(i), SX(i), and M(i) to hold the calculations.
> Set SW(0)=SX(0)=M(0)=0, and then repeat:
>   SW(i+1)=SW(i)+W(i)
>   d=[X(i+1)-M(i)]W(i)
>   M(i+1)=M(i)+d/SW(i+1)
>   SX(i+1)=SX(i)+d[X(i+1)-M(i+1)]
> SX(N) will contain SW times the variance (SD squared), and M(N) the mean.
> You don't actually have to use arrays for SW,SX, and M;
> scalars will do if you update them.

The above algorithm looked so neat that I couldn't help myself;
I had to program it and try it out:

The following program computes the Mean, Sample Standard Deviation, and
Population Standard Deviation of *weighted* data currently stored in
the HP48 Statistics Matrix (SumDAT), allowing *any* columns of SumDAT
to be specified as the "data" and "frequency/weight" columns,
in the same manner as other built-in HP48 statistical commands:

Note: This program relies on the G/GX commands COL- and DOLIST,
      which can be simulated by suitable S/SX programs.

%%HP: T(3); @ \GS is Greek Sigma, \-> is right-arrow, \v/ is SquareRoot
\<< '\GSPAR' RCL 1 2 SUB EVAL 0 0 DUP2 \-> x f w d m v
  \<< RCL\GS x COL- SWAP DROP ARRY\-> EVAL \->LIST
      RCL\GS f COL- SWAP DROP ARRY\-> EVAL \->LIST
    2 \<< DUP2 'w' STO+ m - * 'd' STO d w / 'm' STO+ m - d * 'v' STO+ \>>
          DOLIST m v w DUP2 1 - / \v/ ROT ROT / \v/ \>> \>>
@ G/GX: 269 bytes (on stack), #17BBh checksum

Instructions:

  Enter a statistical data matrix containing at least two columns,
  with any particular column containing data values, and any other
  column containing weights or frequencies (integer or non-integer).

  The HP48 reserved variable SumDAT may also contain a 'name' of any
  other variable, which in turn contains the actual statistics matrix.

  If you use any of the G/GX Statistics data entry applications, a
  default SumPAR will be created automatically, specifying column 1
  as the "independent" variable (data values) and column 2 as the
  "dependent" variable (weights).  You may change the corresponding X-column
  and/or Y-column designations in any of the Statistics data entry menus;
  "Single-var.." and "Frequencies.." use only X-column, while
  "Fit data.." and "Summary stats.." use both X-column and Y-column).

  You may also use the XCOL and YCOL commands to specify the
  individual columns independently, or the COLSum command to
  specify both XCOL and YCOL at the same time.

  Executing the above program then returns three values:

    Mean, Sample Standard Deviation, and Population Standard Deviation;
    you may subsequently DROP any values you do not need.

    If you want the Variances, rather than the Standard Deviations,
    then simply omit taking the square roots (you may of course modify
    the program to produce any desired combination of Total Weight (count),
    Total Values, and Mean, Variances and/or Standard Deviations).

    Note that none of the common "Summary stats" (SumX, SumY^2, etc.)
    is actually computed by this program, nor will any of the built-in
    HP48 Statistics functions except MAXsum and MINsum return any
    correct answers using the "weighted" statistics matrix.

Many thanks to Bob Wheeler for posting the reference for the methods used.

-----------------------------------------------------------
With best wishes from:  John H Meyers  ( jhmeyers@mum.edu )
