Running experiment at two sites - match computers and refresh rates?

Fri Dec 14 21:04:22 UTC 2012

Thanks for the responses, everyone!

On Monday, November 19, 2012 3:06:06 AM UTC-5, Scott wrote:
>
> Hi Mary Kate,
>
> I want to better explain my previous post (about LCD monitors) and reply 
> more specifically to David's questions.
>
> We are starting up a collaboration study with another lab. I'm responsible 
> for helping design tasks and write the E-Prime programs, which include 
> several programs with only behavioral measures and one that will also 
> include ERP measures.
>
> I'm not the PI; I have considerable input but don't have the final say 
> about anything. Like you, I want to minimize noise and especially 
> measurement bias between the two labs. I want to be practical and realistic 
> regarding these goals. I agree with most everything David has said. I'm 
> asking that factors that are easy to control and might affect performance 
> and/or measurement be consistent between the two sites, including things 
> you've already mentioned, like monitor size and resolution (also determined 
> by E-Prime device settings) and operating systems. I also want to ensure we 
> use the same refresh rate (60 hz), so I know how to set stimulus durations, 
> and both sites will use the same version of E-Prime to run subjects. We'll 
> check both PCs using E-Pime's timing tests, and if they pass I won't worry 
> more about how similar the computers are.
>
> I'm less sure than David about how robust E-Prime is with regard to 
> measuring response times independent of the mechanisms for presenting 
> stimuli. Both labs will be using the same kinds of USB button boxes (maybe 
> not as precise as the PST SRBox, but it's consistent and convenient). I 
> only encountered a difficult choice when it came to the computer monitors. 
> E-Prime can accurately and reliably control and measure presentations of 
> visual displays on a CRT monitor, and all CRT monitors work about the same 
> (afaik). However, both labs use LCD monitors (I have no say about this), 
> and LCDs monitors are more varied and complicated, especially with regard 
> to timings. This has been discussed a before in post about *LCD monitors 
> and Input lag* in this<https://groups.google.com/forum/?fromgroups=#!searchin/e-prime/lcd/e-prime/-Sn4BM61pVc/1RyXS8GTHaIJ>thread and maybe others. It's not easy to measure input lag. The refresh 
> detector of PST's SRBox does not work reliably with LCD monitors (unless 
> someone can tell me how to make it work). Onscreen timers and high-speed 
> cameras are often used, along with photocells and oscilloscopes, but they 
> all require extra equipment, time, and trouble. Moreover, there are 
> questions about the implementation and accuracy of most of these methods. In 
> fact, a rather thorough analysis by Thomas Thiemann, here<http://www.prad.de/en/monitore/specials/inputlag/inputlag.html>
> , concludes: 
>
>    - ...it is not an option to achieve a precise value for the image 
>    processing time in the monitor using a camera. It is only possible to 
>    indicate approximate areas in which the input lag of the monitor is likely 
>    to be encountered, but subject to a rather large error. An evaluation of 
>    the input lags of monitors using the photo method should thus lead to 
>    sorting into rough classes such as the following:
>       - - probably less than 1 frame lag / less than 16 ms lag
>       - - probably one to two frames lag / 16 ms to 32 ms lag
>       - - the lag is probably greater than two frames / greater than 32 
>       ms.
>    
> Now I'm not sure E-Prime's RT measures would necessarily reflect this 
> range of variability, but if so, it's more difference between labs than I'm 
> comfortable with. Therefore, we decided to purchase the same new LCD 
> monitors for both labs, hoping that that they probably would exhibit about 
> the same amount of lag so that RTs and stimulus onset times would be 
> displaced about the same amount in the measurements and recording of both 
> labs. This was the best I could figure to do, for now, so we can get on 
> with our research. In the meantime, I have someone working on a photocell 
> circuit to connect to our EEG headbox that might allow me to measure the 
> stimulus onset lags of our systems.
>
> If you are using CRT monitors, good for you -- none of this matters. I'm 
> not sure how much it matters, even if you do use LCD monitors. I would love 
> for someone to explain why "LCD input lag" makes little or no difference 
> for most research paradigms, and convince me I've no reasons to worry about 
> this. I just thought I should share my concerns and some information that 
> might be relevant to you and/or other researchers. I'd like to hear what 
> others think. -- Thanks.
>
> On Thursday, November 15, 2012 3:35:56 PM UTC-6, McFarlane, David wrote:
>>
>> Mary Kate, 
>>
>> Good question, I hope others weigh in.  Here are my thoughts. 
>>
>> Obviously, the more uniform the better.  So one might turn the 
>> question around and ask how much nonuniformity is too much.  And that 
>> will depend largely on the timing demands of the study, some studies 
>> have more stringent requirements than others. 
>>
>> Offhand, I would say that as long as each of your computer setups can 
>> robustly deliver millisecond-quality times, you should be OK.  You 
>> can and should test each one with the RefreshClockTest that you can 
>> download from PST, see Chapter 3 of the original User's Guide, or 
>> Chapter 4 of the revised version.  Bear in mind that RefreshClockTest 
>> just tests whether the system can keep up with the onboard 
>> millisecond clock, it does not test the accuracy of the clock per se, 
>> for that you would need to compare times with an external time 
>> standard.  In any case, you at least want all your machines to pass 
>> RefreshClockTest. 
>>
>> Matching refresh rates again depends on your timing requirements.  To 
>> take an example, If you have one display running at 60 Hz (refresh 
>> period = 16.7 ms) and another at 75 Hz (refresh period = 13.3 ms), 
>> and you ask for a Duration of 200 ms, then the one display will 
>> actually give you either 183 or 200 ms, while the other screen will 
>> give you either 199 ms or 213 ms.  You will have to decide whether 
>> that is acceptable.  Note that the production release (EP2.0.10.242) 
>> allows you to request a refresh rate 
>> (http://www.pstnet.com/support/kb.asp?TopicID=3465 ).  No guarantee 
>> that you will get what you ask, so still look at the value measured 
>> by E-Prime & reported in the .edat file.  (And go with the value 
>> measured by E-Prime, do not trust the refresh rate reported by Windows.) 
>>
>> I do not have any particular refresh rate to recommend, I do not run 
>> studies myself (I am a Systems Designer who helps others do their 
>> research), and I am embarrassed to say that I have never asked others 
>> what refresh rates they use, I will have to ask around.  Offhand it 
>> seems that faster is better as you get more exact times.  OTOH, I 
>> often tend toward lowest common denominators for greater 
>> compatibility, and for that reason might stick down at 60 Hz.  In 
>> truth, rightly or wrongly, I think we mostly just take whatever 
>> refresh rate we get and don't think about it. 
>>
>> You already plan to use the same size monitors at all sites, and I 
>> presume use the same display resolution.  I trust you will also seat 
>> subjects at the same distance so that visual stimuli subtend the same 
>> visual angle for all subjects. 
>>
>> Note that none of this affects the accuracy of response times, 
>> E-Prime has a very robust mechanism for gettting responses which is 
>> independent of the mechansims for presenting stimuli. 
>>
>> Now after all that, here is the short answer (Michiel, would like to 
>> chime in here?):  Chances are that whatever human behavior you 
>> measure has more variance than your measurement system.  That's not 
>> an excuse for getting sloppy when you can be exact, but maybe we need 
>> not fret too much about this; others have done statistical 
>> calculations to show that we can compensate for variance introduced 
>> by our measuring system merely by running a few more subjects (sorry, 
>> I don't have a citation handy). 
>>
>> So what do the rest of you think?  For that matter, what do others 
>> actually do? 
>>
>> ----- 
>> David McFarlane 
>> E-Prime training 
>> online:  http://psychology.msu.edu/Workshops_Courses/eprime.aspx 
>> Twitter:  @EPrimeMaster (https://twitter.com/EPrimeMaster) 
>>
>>
>> At 11/15/2012 12:28 PM Thursday, MKT wrote: 
>> >We will soon be running the same experiment on E-Prime 2.0 at two 
>> >sites (OSU and UPenn). We want to make sure we do not add any 
>> >additional noise in the data or run into problems. 
>> > 
>> >We plan to use the same size monitors at both sites and run on 
>> >Windows 7. Is it necessary to match computer type as well? How about 
>> >refresh rates? Is there a particular refresh rate you recommend? 
>> > 
>> >Thanks for your help! 
>>
>>

-- 
You received this message because you are subscribed to the Google Groups "E-Prime" group.
To post to this group, send email to e-prime at googlegroups.com.
To unsubscribe from this group, send email to e-prime+unsubscribe at googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/e-prime/-/v45gKlMvm20J.
For more options, visit https://groups.google.com/groups/opt_out.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/eprime/attachments/20121214/6713a56e/attachment.htm>