Running experiment at two sites - match computers and refresh rates?

Scott saultsj at missouri.edu
Mon Nov 19 08:06:06 UTC 2012


Hi Mary Kate,

I want to better explain my previous post (about LCD monitors) and reply 
more specifically to David's questions.

We are starting up a collaboration study with another lab. I'm responsible 
for helping design tasks and write the E-Prime programs, which include 
several programs with only behavioral measures and one that will also 
include ERP measures.

I'm not the PI; I have considerable input but don't have the final say 
about anything. Like you, I want to minimize noise and especially 
measurement bias between the two labs. I want to be practical and realistic 
regarding these goals. I agree with most everything David has said. I'm 
asking that factors that are easy to control and might affect performance 
and/or measurement be consistent between the two sites, including things 
you've already mentioned, like monitor size and resolution (also determined 
by E-Prime device settings) and operating systems. I also want to ensure we 
use the same refresh rate (60 hz), so I know how to set stimulus durations, 
and both sites will use the same version of E-Prime to run subjects. We'll 
check both PCs using E-Pime's timing tests, and if they pass I won't worry 
more about how similar the computers are.

I'm less sure than David about how robust E-Prime is with regard to 
measuring response times independent of the mechanisms for presenting 
stimuli. Both labs will be using the same kinds of USB button boxes (maybe 
not as precise as the PST SRBox, but it's consistent and convenient). I 
only encountered a difficult choice when it came to the computer monitors. 
E-Prime can accurately and reliably control and measure presentations of 
visual displays on a CRT monitor, and all CRT monitors work about the same 
(afaik). However, both labs use LCD monitors (I have no say about this), 
and LCDs monitors are more varied and complicated, especially with regard 
to timings. This has been discussed a before in post about *LCD monitors 
and Input lag* in this<https://groups.google.com/forum/?fromgroups=#!searchin/e-prime/lcd/e-prime/-Sn4BM61pVc/1RyXS8GTHaIJ>thread and maybe others. It's not easy to measure input lag. The refresh 
detector of PST's SRBox does not work reliably with LCD monitors (unless 
someone can tell me how to make it work). Onscreen timers and high-speed 
cameras are often used, along with photocells and oscilloscopes, but they 
all require extra equipment, time, and trouble. Moreover, there are 
questions about the implementation and accuracy of most of these methods. In 
fact, a rather thorough analysis by Thomas Thiemann, here<http://www.prad.de/en/monitore/specials/inputlag/inputlag.html>
, concludes: 

   - ...it is not an option to achieve a precise value for the image 
   processing time in the monitor using a camera. It is only possible to 
   indicate approximate areas in which the input lag of the monitor is likely 
   to be encountered, but subject to a rather large error. An evaluation of 
   the input lags of monitors using the photo method should thus lead to 
   sorting into rough classes such as the following:
      - - probably less than 1 frame lag / less than 16 ms lag
      - - probably one to two frames lag / 16 ms to 32 ms lag
      - - the lag is probably greater than two frames / greater than 32 ms.
   
Now I'm not sure E-Prime's RT measures would necessarily reflect this range 
of variability, but if so, it's more difference between labs than I'm 
comfortable with. Therefore, we decided to purchase the same new LCD 
monitors for both labs, hoping that that they probably would exhibit about 
the same amount of lag so that RTs and stimulus onset times would be 
displaced about the same amount in the measurements and recording of both 
labs. This was the best I could figure to do, for now, so we can get on 
with our research. In the meantime, I have someone working on a photocell 
circuit to connect to our EEG headbox that might allow me to measure the 
stimulus onset lags of our systems.

If you are using CRT monitors, good for you -- none of this matters. I'm 
not sure how much it matters, even if you do use LCD monitors. I would love 
for someone to explain why "LCD input lag" makes little or no difference 
for most research paradigms, and convince me I've no reasons to worry about 
this. I just thought I should share my concerns and some information that 
might be relevant to you and/or other researchers. I'd like to hear what 
others think. -- Thanks.

On Thursday, November 15, 2012 3:35:56 PM UTC-6, McFarlane, David wrote:
>
> Mary Kate, 
>
> Good question, I hope others weigh in.  Here are my thoughts. 
>
> Obviously, the more uniform the better.  So one might turn the 
> question around and ask how much nonuniformity is too much.  And that 
> will depend largely on the timing demands of the study, some studies 
> have more stringent requirements than others. 
>
> Offhand, I would say that as long as each of your computer setups can 
> robustly deliver millisecond-quality times, you should be OK.  You 
> can and should test each one with the RefreshClockTest that you can 
> download from PST, see Chapter 3 of the original User's Guide, or 
> Chapter 4 of the revised version.  Bear in mind that RefreshClockTest 
> just tests whether the system can keep up with the onboard 
> millisecond clock, it does not test the accuracy of the clock per se, 
> for that you would need to compare times with an external time 
> standard.  In any case, you at least want all your machines to pass 
> RefreshClockTest. 
>
> Matching refresh rates again depends on your timing requirements.  To 
> take an example, If you have one display running at 60 Hz (refresh 
> period = 16.7 ms) and another at 75 Hz (refresh period = 13.3 ms), 
> and you ask for a Duration of 200 ms, then the one display will 
> actually give you either 183 or 200 ms, while the other screen will 
> give you either 199 ms or 213 ms.  You will have to decide whether 
> that is acceptable.  Note that the production release (EP2.0.10.242) 
> allows you to request a refresh rate 
> (http://www.pstnet.com/support/kb.asp?TopicID=3465 ).  No guarantee 
> that you will get what you ask, so still look at the value measured 
> by E-Prime & reported in the .edat file.  (And go with the value 
> measured by E-Prime, do not trust the refresh rate reported by Windows.) 
>
> I do not have any particular refresh rate to recommend, I do not run 
> studies myself (I am a Systems Designer who helps others do their 
> research), and I am embarrassed to say that I have never asked others 
> what refresh rates they use, I will have to ask around.  Offhand it 
> seems that faster is better as you get more exact times.  OTOH, I 
> often tend toward lowest common denominators for greater 
> compatibility, and for that reason might stick down at 60 Hz.  In 
> truth, rightly or wrongly, I think we mostly just take whatever 
> refresh rate we get and don't think about it. 
>
> You already plan to use the same size monitors at all sites, and I 
> presume use the same display resolution.  I trust you will also seat 
> subjects at the same distance so that visual stimuli subtend the same 
> visual angle for all subjects. 
>
> Note that none of this affects the accuracy of response times, 
> E-Prime has a very robust mechanism for gettting responses which is 
> independent of the mechansims for presenting stimuli. 
>
> Now after all that, here is the short answer (Michiel, would like to 
> chime in here?):  Chances are that whatever human behavior you 
> measure has more variance than your measurement system.  That's not 
> an excuse for getting sloppy when you can be exact, but maybe we need 
> not fret too much about this; others have done statistical 
> calculations to show that we can compensate for variance introduced 
> by our measuring system merely by running a few more subjects (sorry, 
> I don't have a citation handy). 
>
> So what do the rest of you think?  For that matter, what do others 
> actually do? 
>
> ----- 
> David McFarlane 
> E-Prime training 
> online:  http://psychology.msu.edu/Workshops_Courses/eprime.aspx 
> Twitter:  @EPrimeMaster (https://twitter.com/EPrimeMaster) 
>
>
> At 11/15/2012 12:28 PM Thursday, MKT wrote: 
> >We will soon be running the same experiment on E-Prime 2.0 at two 
> >sites (OSU and UPenn). We want to make sure we do not add any 
> >additional noise in the data or run into problems. 
> > 
> >We plan to use the same size monitors at both sites and run on 
> >Windows 7. Is it necessary to match computer type as well? How about 
> >refresh rates? Is there a particular refresh rate you recommend? 
> > 
> >Thanks for your help! 
>
>

-- 
You received this message because you are subscribed to the Google Groups "E-Prime" group.
To post to this group, send email to e-prime at googlegroups.com.
To unsubscribe from this group, send email to e-prime+unsubscribe at googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/e-prime/-/GUKIwh6LjqsJ.
For more options, visit https://groups.google.com/groups/opt_out.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/eprime/attachments/20121119/43db0097/attachment.htm>


More information about the Eprime mailing list