Running experiment at two sites - match computers and refresh rates?
Scott
saultsj at missouri.edu
Mon Nov 19 08:06:06 UTC 2012
Hi Mary Kate,
I want to better explain my previous post (about LCD monitors) and reply
more specifically to David's questions.
We are starting up a collaboration study with another lab. I'm responsible
for helping design tasks and write the E-Prime programs, which include
several programs with only behavioral measures and one that will also
include ERP measures.
I'm not the PI; I have considerable input but don't have the final say
about anything. Like you, I want to minimize noise and especially
measurement bias between the two labs. I want to be practical and realistic
regarding these goals. I agree with most everything David has said. I'm
asking that factors that are easy to control and might affect performance
and/or measurement be consistent between the two sites, including things
you've already mentioned, like monitor size and resolution (also determined
by E-Prime device settings) and operating systems. I also want to ensure we
use the same refresh rate (60 hz), so I know how to set stimulus durations,
and both sites will use the same version of E-Prime to run subjects. We'll
check both PCs using E-Pime's timing tests, and if they pass I won't worry
more about how similar the computers are.
I'm less sure than David about how robust E-Prime is with regard to
measuring response times independent of the mechanisms for presenting
stimuli. Both labs will be using the same kinds of USB button boxes (maybe
not as precise as the PST SRBox, but it's consistent and convenient). I
only encountered a difficult choice when it came to the computer monitors.
E-Prime can accurately and reliably control and measure presentations of
visual displays on a CRT monitor, and all CRT monitors work about the same
(afaik). However, both labs use LCD monitors (I have no say about this),
and LCDs monitors are more varied and complicated, especially with regard
to timings. This has been discussed a before in post about *LCD monitors
and Input lag* in this<https://groups.google.com/forum/?fromgroups=#!searchin/e-prime/lcd/e-prime/-Sn4BM61pVc/1RyXS8GTHaIJ>thread and maybe others. It's not easy to measure input lag. The refresh
detector of PST's SRBox does not work reliably with LCD monitors (unless
someone can tell me how to make it work). Onscreen timers and high-speed
cameras are often used, along with photocells and oscilloscopes, but they
all require extra equipment, time, and trouble. Moreover, there are
questions about the implementation and accuracy of most of these methods. In
fact, a rather thorough analysis by Thomas Thiemann, here<http://www.prad.de/en/monitore/specials/inputlag/inputlag.html>
, concludes:
- ...it is not an option to achieve a precise value for the image
processing time in the monitor using a camera. It is only possible to
indicate approximate areas in which the input lag of the monitor is likely
to be encountered, but subject to a rather large error. An evaluation of
the input lags of monitors using the photo method should thus lead to
sorting into rough classes such as the following:
- - probably less than 1 frame lag / less than 16 ms lag
- - probably one to two frames lag / 16 ms to 32 ms lag
- - the lag is probably greater than two frames / greater than 32 ms.
Now I'm not sure E-Prime's RT measures would necessarily reflect this range
of variability, but if so, it's more difference between labs than I'm
comfortable with. Therefore, we decided to purchase the same new LCD
monitors for both labs, hoping that that they probably would exhibit about
the same amount of lag so that RTs and stimulus onset times would be
displaced about the same amount in the measurements and recording of both
labs. This was the best I could figure to do, for now, so we can get on
with our research. In the meantime, I have someone working on a photocell
circuit to connect to our EEG headbox that might allow me to measure the
stimulus onset lags of our systems.
If you are using CRT monitors, good for you -- none of this matters. I'm
not sure how much it matters, even if you do use LCD monitors. I would love
for someone to explain why "LCD input lag" makes little or no difference
for most research paradigms, and convince me I've no reasons to worry about
this. I just thought I should share my concerns and some information that
might be relevant to you and/or other researchers. I'd like to hear what
others think. -- Thanks.
On Thursday, November 15, 2012 3:35:56 PM UTC-6, McFarlane, David wrote:
>
> Mary Kate,
>
> Good question, I hope others weigh in. Here are my thoughts.
>
> Obviously, the more uniform the better. So one might turn the
> question around and ask how much nonuniformity is too much. And that
> will depend largely on the timing demands of the study, some studies
> have more stringent requirements than others.
>
> Offhand, I would say that as long as each of your computer setups can
> robustly deliver millisecond-quality times, you should be OK. You
> can and should test each one with the RefreshClockTest that you can
> download from PST, see Chapter 3 of the original User's Guide, or
> Chapter 4 of the revised version. Bear in mind that RefreshClockTest
> just tests whether the system can keep up with the onboard
> millisecond clock, it does not test the accuracy of the clock per se,
> for that you would need to compare times with an external time
> standard. In any case, you at least want all your machines to pass
> RefreshClockTest.
>
> Matching refresh rates again depends on your timing requirements. To
> take an example, If you have one display running at 60 Hz (refresh
> period = 16.7 ms) and another at 75 Hz (refresh period = 13.3 ms),
> and you ask for a Duration of 200 ms, then the one display will
> actually give you either 183 or 200 ms, while the other screen will
> give you either 199 ms or 213 ms. You will have to decide whether
> that is acceptable. Note that the production release (EP2.0.10.242)
> allows you to request a refresh rate
> (http://www.pstnet.com/support/kb.asp?TopicID=3465 ). No guarantee
> that you will get what you ask, so still look at the value measured
> by E-Prime & reported in the .edat file. (And go with the value
> measured by E-Prime, do not trust the refresh rate reported by Windows.)
>
> I do not have any particular refresh rate to recommend, I do not run
> studies myself (I am a Systems Designer who helps others do their
> research), and I am embarrassed to say that I have never asked others
> what refresh rates they use, I will have to ask around. Offhand it
> seems that faster is better as you get more exact times. OTOH, I
> often tend toward lowest common denominators for greater
> compatibility, and for that reason might stick down at 60 Hz. In
> truth, rightly or wrongly, I think we mostly just take whatever
> refresh rate we get and don't think about it.
>
> You already plan to use the same size monitors at all sites, and I
> presume use the same display resolution. I trust you will also seat
> subjects at the same distance so that visual stimuli subtend the same
> visual angle for all subjects.
>
> Note that none of this affects the accuracy of response times,
> E-Prime has a very robust mechanism for gettting responses which is
> independent of the mechansims for presenting stimuli.
>
> Now after all that, here is the short answer (Michiel, would like to
> chime in here?): Chances are that whatever human behavior you
> measure has more variance than your measurement system. That's not
> an excuse for getting sloppy when you can be exact, but maybe we need
> not fret too much about this; others have done statistical
> calculations to show that we can compensate for variance introduced
> by our measuring system merely by running a few more subjects (sorry,
> I don't have a citation handy).
>
> So what do the rest of you think? For that matter, what do others
> actually do?
>
> -----
> David McFarlane
> E-Prime training
> online: http://psychology.msu.edu/Workshops_Courses/eprime.aspx
> Twitter: @EPrimeMaster (https://twitter.com/EPrimeMaster)
>
>
> At 11/15/2012 12:28 PM Thursday, MKT wrote:
> >We will soon be running the same experiment on E-Prime 2.0 at two
> >sites (OSU and UPenn). We want to make sure we do not add any
> >additional noise in the data or run into problems.
> >
> >We plan to use the same size monitors at both sites and run on
> >Windows 7. Is it necessary to match computer type as well? How about
> >refresh rates? Is there a particular refresh rate you recommend?
> >
> >Thanks for your help!
>
>
--
You received this message because you are subscribed to the Google Groups "E-Prime" group.
To post to this group, send email to e-prime at googlegroups.com.
To unsubscribe from this group, send email to e-prime+unsubscribe at googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/e-prime/-/GUKIwh6LjqsJ.
For more options, visit https://groups.google.com/groups/opt_out.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/eprime/attachments/20121119/43db0097/attachment.htm>
More information about the Eprime
mailing list