[RASMB] standard tests; should ABRF be involved?

[pschuck at helix.nih.gov]

Regarding a standardized test, I believe it is a good idea, and I totally
agree with John's concerns about analysis and that we either should not do
it or spend enough effort and thought to make it worthwhile.  For the
latter, my suggestions would be

1) run the sample in triplicate on each machine (in the same run) to get a
feeling for the variance caused by things other than temperature (e.g.
analysis, centerpiece imperfections, imperfections in matching the buffer
volume, etc.)  In addition, each participant could optionally do
repetitions of the run to assess run-to-run variability.  Could also
invovle two categories of data - absorbance/interference detection.

2) we should establish a very detailed protocol for the experiment,
standardizing rotor speed, sample volume, temperature equilibration
periods, etc.

3) Following John's idea, the data analysis should be performed centrally
by one or more persons, blind to the identity of each run, each analysis
person analyzing the complete set of all runs performed.  (I'd volunteer
for the analysis.)  We could put the data anonymously on a website, and
each participant, knowing the code of his/her own machines, could judge
him/herself if a centrifuge is an outlier and decide whether to introduce
a temperature correction factor or not.

4) I propose we should all do the spectroscopic temperature calibration
method, as well.  That would address if deviations from the mean of the
experimental s-values for a machine does actually correlate with a
detected calibration error (addressing the concern about statistical
accuracy of the calibration).  I wonder if that, too, could be anlayzed by
a single person, without access to the results of the s-values and blind
to the origin of the data sets.  I'm not familiar with the method, but
would it be possible and advantageous to share the same cell (if it's
sealed and stable enough) and mail it from one site to the other?

5) the choice of sample and sample stability is probably critical factor.
I would suggest something large, maximizing the sharpness of the boundary
and the possibility to calculate the meniscus position in the data
analysis.  Also, if a large molecule aggregates it would be easier to
identify the monomer peak and confine the analysis to that. How about an
antibody (perhaps donation)?  We could make solutions and reference buffer
in a single lab, testing for sufficient stability over a couple of weeks,
and then mailing them out to each participant who would agree to run it
within a certain period of time?

In this proposed scenario, we would need
* a donation of protein sample
* a lab to do the preliminary protein stability test and mail the sample out
* enough participating labs (machines) to get a good statistics
* a site to gather the data, encode it, put it on a website, and send it
out for the data analysis
* data analysis sites for calibration spectra
* data analysis sites for velocity runs
* a site to do the statistical analysis of all results and the data subsets
* a coordinator of these tasks

In my view, ideally these tasks should all be done by different labs, and
there's probably several missing aspects.

Regarding the organization of such a study, although in my view ABRF
involvement could be very valuable, I don't think we necessarily need a
formal structure or entity for this particular study.  It appears to me it
could probably be handled very well by our RASMB group, since we already
have an excellent communication and discussion forum established.

In my view, we would only need a volunteer to collect all suggestions from
this discussion, to put up for discussion all additional points of
concern, consolidate it in a detailed proposal, solicit participants and
volunteers for the different tasks.

Overall, it would certainly be a lot of work, but if we can divide it up
it may be feasible.

Seasonal Greetings,

Peter Schuck




> Arthur's suggestion of running the same sample in multiple labs may be a
> good one, but I'm not so certain the results will clearly indicate
> differences in temperature calibration, at least not unless it is all
> thought through carefully in advance.
>
> The problem is to distinguish real differences in temperature from other
> effects that can change the values of the sedimentation coefficient:
>
> (1) systematic errors due to differences in the software used to evaluate
> the data;
>
> (2) systematic errors due to how the meniscus position is defined (this is
> a
> bigger problem than most people realize);
>
> (3) potential systematic differences between absorbance and interference
> data (has anyone really looked at this carefully?);
>
> The obvious way to deal with the first two would be to have one person do
> all the data analysis, but that could be a burden (no, I'm NOT
> volunteering!).
>
> Lastly, there is simply the problem of precision of the values from any
> one
> lab. How many labs have really evaluated their reproducibility, for
> example
> for running the same sample in triplicate in the same run, and from
> run-to-run? If the run-to-run precision is say 1%, then that translates
> into
> a 0.4 degree temperature uncertainty.
>
> Overall then you really need VERY high quality data to evaluate the
> temperature calibration issue. A casual effort is potentially a waste of
> everyone's time.
>
> Lastly, I might suggest that this may be something that should be handled
> under the auspices of the Molecular Interactions Research Group of the
> Association of Biotechnology Resource Facilities (ABRF), who have already
> run (and published) tests involving both BIAcore and AUC studies of the
> same
> protein in multiple labs. Preston Hensley and Ed Eisenstein are, I
> believe,
> currently part of that group so perhaps one of them could comment on this.
>
> John
>
> -----Original Message-----
> From: rasmb-admin at server1.bbri.org [mailto:rasmb-admin at server1.bbri.org]
> On
> Behalf Of Arthur Rowe
> Sent: Monday, December 13, 2004 7:47 AM
> To: John Correia; mchien at beckman.com; rasmb at server1.bbri.org
> Subject: Re: [RASMB] Re: XL-A/I temp control
>
>
> Hi Everyone
>
> I entirely agree that the temperature on machines needs to be calibrated.
> Equally, I agree with Jack's  statement that "most users . . . never
> calibrate the temperature". As to whether it is actually easy to do same
> using the Stafford & Liu method, opinions might differ. We ourselves find
> that the integration required to get the area under the curve is easy
> enough
> to estimate with data from the spec, but less easy (baseline definition,
> noisier values) with data from the XL-A.
>
> The suggestion that a quick set of standard runs be undertaken on multiple
> machines would serve to clarify as to whether it all really matters, at
> the
> level of doing careful calibration - if one is talking at the precision
> (not
> accuracy) level of the instrument. After all, with a (presumably
> standardised) production line, one might have supposed that any errors at
> a
> given temperature might have been pretty much the same from one instrument
> to another. The sketchy data which we have to hand suggests that that is
> not
> the case.
>
> Measuring an s value of a given solute via an agreed procedure is about as
> objective a procedure as one can define. Variations are easily transformed
> into temperature uncertainties. If that is what we are looking for, then
> this is the easiest way to find it. The numbers of instruments volunteered
> to date (10) looks enough to give a pretty good idea of what the variation
> level actually is - but any more would bolster the stats nicely . . . .
>
> Regards to all
>
> Arthur
>
> --
> *************************
> Arthur Rowe
> Lab at Sutton Bonington
> tel: +44 115 951 6156
> fax: +44 115 951 6157
> *************************
>
>
>
> From: "John Correia" <jcorreia at biochem.umsmed.edu>
> Date: Fri, 10 Dec 2004 11:16:43 -0600
> To: <mchien at beckman.com>, <arthur.rowe at nottingham.ac.uk>,
> <rasmb at server1.bbri.org>
> Subject: Re: [RASMB] Re: XL-A/I temp control
>
>
>
>
> I suspect other will give technical responses about the ability of these
> temperature controllers to function at the +/- 0.1 C level.  My take is
> slightly different.
>
> What matters is not the set point but knowing the actual temperature.
>
> To my knowledge most users never use the Stafford & LUS method to
> calibrate
> the temperature on their machine.  (can we vote on the RASMB in some way?)
> It is no more tedious than waiting three hours before starting a run.  I
> have checked the calibration on my machine twice, once in '93 when I got
> it,
> and again in 1999 when a result caused me to question the temperature
> accuracy.  A series of measurements up to 40 C and back down to 4 C (with
> parallel measurements in a spec) takes about two days.  It does not take
> three hours to go from 20 to 25 C, as verified by the stability of the
> area
> under the CoCl2/ethanol curve.  At a setting of 4 C I get 3.6, at 20 I get
> 19.7, at 40 I get 39.8.  The values drifted by 0.1 C in 7 years.  So if we
> mean by accuracy +/- 0.5 C of setting, my machine is within spec.  When I
> fit data I use the actual temperature, not the set point. & I trust the
> setting on the screen at low vacuum and hit start when it reports the set
> temperature.
>
> Comparisons between uncalibrated machines honestly make no sense to me.
> Calibrate & be done with it.  (Arthur, if you do that in Nottingham &
> still
> get different values of S between machines I would be concerned, &
> amazed.)
>
>
> I also use the calibrated temperature values when I measure density in an
> Anton Paar DMA 5000.  It has a peltier cell good between 0 - 80 C, so I
> dial
> in 19.7 instead of 20 C and measure away.  I suspect the error from a
> calculated density (viscosity & Vbar) is larger than (albeit coupled to)
> an
> assumed temperature.
>
> At this point the inquiring "student" should assume errors in various
> parameters and propagate them into S or MW by the appropriate equations -
> my
> favorite book for teaching this is Bevington, "Data Reduction and Error
> Analysis for the Physical Sciences".
>
> PS - why Beckman has never joined up with Anton Paar to bundle Density
> Meters into XLA/XLI quotes amazes me.
>
> PPSS - Walter claims to keep the original solution around, in a cell, from
> the CoCl2 calibration work, and years later it still gives the same
> results.
> Quick checking may not be as difficult as one might think.
>
>
>
> -------------------------------------------------------------------
> Dr. John J. "Jack" Correia
> Department of Biochemistry
> University of Mississippi Medical Center
> 2500 North State Street
> Jackson, MS  39216
> (601) 984-1522
> fax (601) 984-1501
> email address: jcorreia at biochem.umsmed.edu
> homepage location: http://biochemistry.umc.edu/correia.html
> dept homepage location:    http://biochemistry.umc.edu/
> -------------------------------------------------------------------
>
>
>
>
>>>> Arthur Rowe <arthur.rowe at nottingham.ac.uk> 12/10/04 08:17AM >>>
> Hi Everyone {this is a second (now 3rd!) try at getting this mail out -
> first attempt got lost in cyber-space, it seems}
>
> Mei-Ling Chien gives us a very useful review of the nature of the
> temperature measurement and control system in the XL-I/A instrument.
> However, I do not think that this fully addresses the problems which one
> has
> in determining what the absolute temperature of one's sample actually is
> when it is going round in the rotor at speed.
>
> It is, of course, only a worry to those (very limited) number of people
> for
> whom an absolute s value is of importance, normally for hydrodynamic
> modelling purposes (although formulation issues should not be forgotten).
> When I raised this issue on RASMB a week or so back, my concern was not
> "to
> ensure their operation within the published specification".    . I am
> trying
> to get the accuracy of the temperature read-out to be close to the
> precision
> of which the system is capable. I have no evidence at all to suggest that
> the accuracy is outside the quoted spec of 0.5º. It is just that I - in my
> greedy way - want 0.1º.
>
> Even the method mentioned (equilibrate for 3 hours - under vacuum - and
> then
> check  "with a calibrated external temperature sensing device to verify
> accuracy"   is not unambiguous in what it will yield. Quite apart from
> matters such as adiabatic effects when one releases the vacuum to use an
> "external temperature sensing device", can one be sure that the thermal
> emissivity of a spinning rotor surface, averaged over everything that is
> passing by, is equal to that of a piece of the rotor surface 'seen' in a
> stationary rotor?
>
> None of these are new concerns, and I certainly lay no claim to the IPRs!
> I
> imagine, from what Mei-Ling Chien has communicated, that we at least know
> clearly that the ±0.5º refers to the accuracy of the temperature as
> measured
> by the defined procedure. Walter Stafford's colorimetric method (Stafford
> &
> Liu) did not suggest the presence of errors outside the stated accuracy
> limit, and is surely a valid way to approach the absolute temperature
> issue.
> But is is pretty tedious to use as a procedure, and certainly as a routine
> QA method is not feasible.
>
> As an approach to the size of the problem, would there be support for
> Borries Demeler's suggestion (a single sample to be circulated and
> multiple
> users on multiple machines to report an s value under defined conditions)?
> After all, the NCMH + Borries's Lab gives us 6 machines for starters.
>
> Any way, we here keep trying here to locate the holy grail - a simple,
> cheap, effective method for determining the in-cell temperature to ±0.1º
>
> Regards to all (and many thanks to Mei-Ling Chien)
>
> Arthur
>
> --
> *************************
> Arthur Rowe
> Lab at Sutton Bonington
> tel: +44 115 951 6156
> fax: +44 115 951 6157
> *************************
>
>
>
> From: mchien at beckman.com
> Date: Fri, 3 Dec 2004 10:11:46 -0800
> To: "'rasmb at rasmb-email.bbri.org'" <rasmb at server1.bbri.org>
> Subject: [RASMB] Re: XL-A/I temp control
>
>
>
>
> ----------------------------------------------------------------------------
> ------
> The older archived RASMB emails can be found at:
> http://rasmb-email.bbri.org/rasmb_archives
> and current archives at
> http://rasmb-email.bbri.org/pipermail/rasmb/
> Search All the Archives at:
> http://rasmb-email.bbri.org/rasmb_search.html
> ----------------------------------------------------------------------------
> ------
>
> Hi All,
>
> Below is response regarding XL-A/I temperature control from our Technical
> Support Department.
>
> ******************************************************
> Mei-Ling Chien PhD
> Staff Development Scientist, Centrifugation
> Platform & Automation Business Center
> Beckman Coulter Inc.
>
> mchien at beckman.com
> (650) 859-1948
> ******************************************************
>
>
> The basis for temperature control specifications were instrument design
> specifications for temperature control and dynamic system testing during
> the
> prototype phase of the product.
>
> If there is a discrepancy in temperature control and measurement between
> instruments of the same design then a dynamic calibration check should be
> performed on both instruments to ensure their operation within the
> published
> specification.
>
> First the physical condition of components within the temperature control
> and
> vacuum system should be verified through inspection.  Then an electronic
> calibration for temperature control and vacuum can be performed.  Lastly a
> dynamic test or rotor dunk test is performed (rotor should be precooled or
> preheated to avoid testing delay).  The rotor and its contents must be
> allowed
> to equilibrate for up to 3 hours or more.  When set temperature equals
> indicated
> temperature at the instrument interface, the rotor temperature is then
> checked
> with a calibrated external temperature sensing device to verify accuracy.
>
> If the checks fall out of specification then appropriate troubleshooting
> is
> required to isolate the electronic or mechanical fault in the temperature
> control or vacuum system.  Once the fault is corrected the temperature
> control
> checks are performed again.
>
> Quote from Bob Giebeler, Analytical Ultracentrifugation in Biochemistry
>  and Polymer Science, 1992,16-25 for the Optima XLA/I.
> "Temperature  control is considerably more stable, provides more rapid
> cool-down
> and  heat-up  rates,  is  thermally more uniform, and has equivalent
> accuracy as
> compared  to previous models including the Model E.  This control system
> uses an
> isothermal radiometer temperature-sensing system to sense the temperature
> of
> the
> rotor  that  is  emissivity-independent  ad  view  factor-corrected in
> software.
> Heating  and cooling of the rotor are accomplished by the refrigeration
> can
> that
> surrounds  the  rotor,  which  is  in  turn  heated and cooled by
> thermoelectric
> modules.   This environment is very isothermal, and at equilibrium,
> irrespective
> of  speed  or  temperature,  rotor temperature is within about one degree
> of
> the
> refrigeration can temperature.
>
>
> The  control  system  that  regulates  rotor  temperature,  as  monitored
> by
> the
> radiometer,  is  highly  software-intensive.   This  software encompasses
> triple
> proportional-integral-differential  control algorithms and
> proportional-integral
> smoothing  algorithms.  In addition, radiometer view factors are measured
> during
> rotor   cool-down  to  allow  more  rapid  rotor  cool-down  and  more
> accurate
> temperature  monitoring  during  cool-down.  While at equilibrium,
> refrigeration
> can   temperature   fluctuation   does  not  typically  exceed  +0.5C,
> and
> the
> corresponding rotor temperature fluctuation is less than +0.2C.
>
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> RASMB mailing list
> RASMB at rasmb-email.bbri.org
> http://rasmb-email.bbri.org/mailman/listinfo/rasmb
>
>
>
>
>
> This message has been scanned but we cannot guarantee that it and any
> attachments are free from viruses or other damaging content: you are
> advised
> to perform your own checks. Email communications with the University of
> Nottingham may be monitored as permitted by UK legislation.
>
>
>
>
> This message has been scanned but we cannot guarantee that it and any
> attachments are free from viruses or other damaging content: you are
> advised
> to perform your own checks. Email communications with the University of
> Nottingham may be monitored as permitted by UK legislation.
>
>