Behavioral software survey results

Victor Ferreira ferreira at psy2.ucsd.edu
Fri Jun 23 16:04:29 UTC 2006


Hi PsyScope-ers and E-Prime-ers,

Brian MacWhinney asked me to post this to this list.  For those who  
aren't on the CUNY Conference mailing list, I solicited survey  
participation from the membership of the CUNY Conference mailing list  
concerning their usage and plans for the use of behavioral experiment  
software.  I specifically avoided soliciting participation from the  
software's mailing list recipients, so as to not overly skew the  
representation of different packages in the responses.  I hope you  
find the included information useful.

Best wishes,

Vic

Victor Ferreira, Associate Professor
Department of Psychology 0109
University of California, San Diego
La Jolla, CA 92093-0109
http://lpl.ucsd.edu/


*


Hi everyone,

The results of the software survey are in.  We had 187 responses, but  
one was unanalyzable (the respondent did not specify a software  
package).  Thanks everyone for your responses; I hope these results  
prove useful.

Here's a one-paragraph summary of the survey results; details below:   
E-Prime is the most popular package of those surveyed, but the  
majority of folks are using either E-Prime, DMDX, or some flavor of  
PsyScope.  E-Prime, PsyScope, SuperLab, and ERTS are all rated as  
easy to build experiments with, and about equally so.  DMDX and NESU  
are seen as slightly harder.  Presentation and MatLab are notably the  
hardest of the commonly used packages.  SuperLab was seen as easiest  
for novices.  E-Prime and PsyScope were rated a shade harder for  
novices, and then from ERTS to SuperLab Pro to DMDX to NESU, novice- 
ease ratings dropped.  Presentation and MatLab were both seen as  
notably difficult for novices.  DMDX nets the highest satisfaction  
rating, with PsyScope X and E-Prime a smidge lower.  MatLab,  
SuperLab, and Presentation rank below that, ending with NESU,  
PsyScope classic, and SuperLab Pro.  PsyScope X has the highest  
"sticking with" rate, and E-Prime, SuperLab, MatLab, and DMDX all  
have 50%+ "sticking with" ratings.  People are running away from  
PsyScope classic, probably because the classic Mac OS is very quickly  
approaching end-of-life.  E-Prime, both flavors of PsyScope, and DMDX  
are highly recommended.  MatLab is also well recommended.  Then it  
drops noticeably to Presentation, SuperLab, ERTS, and finally  
SuperLab Pro and NESU.  A summary of two open-ended questions is also  
included below.

*

Details:

Of the 186 valid responses, the most commonly used software package  
ended up being E-Prime (57 responses, 30.6%).  Here's the ranking of  
software packages, listing number of respondents and percentage for  
each:

1. E-Prime: 57, 30.6%
2. DMDX: 32, 17.2%
3. PsyScope Classic: 18, 9.7%
4. Presentation: 12, 6.5%
5. PsyScope X: 11, 5.9%
6. NESU: 8, 4.3%
7. ERTS: 6, 3.2%
8. SuperLab: 5, 2.7%
9. MatLab: 5, 2.7%
10. SuperLab Pro: 4, 2.2%
11. Linger: 4, 2.2%
12. MEL: 3, 1.6%
13. Experiment Builder: 3, 1.6%
14. EyeTrack: 2, 1.1%

The following packages had 1 response each: Authorware, C  
programming, Delphi Borland, DirectRT, ExBuilder, Habit, Inquisit,  
MacroMedia Director, PHPsurveyor, PCexpt, RSVP, WWStim, WebExp,  
iMovie, tscope, vision analyzer/recorder.

Bob Slevc worked to dig up links for many of these software  
packages.  I'll put the links below the signature line.

Note that PsyScope classic and PsyScope X could be combined to have a  
total response count of 29 or 15.6%, which would keep it in third  
place behind E-Prime and DMDX.  SuperLab and SuperLab Pro (which I  
assume are distinct) could similarly be combined for 9 responses  
(4.8%), which would put it right behind Presentation.

*

We asked, "How easy/hard is it for you to build an experiment with  
your software?" with a 7-point response scale (1 = "very easy" and 7  
= "very hard").  Overall, the mean difficulty rating was 3.09 with a  
standard deviation of 1.43, and a median of 3.  Here are the mean and  
median build-difficulty ratings for the packages that received at  
least five responses (combining the above noted PsyScope and SuperLab):

ERTS (6): 2.5, 2
E-Prime (57): 2.68, 2
PsyScope (29): 2.86, 3
SuperLab (9): 3.0, 3
NESU (8): 3.25, 3
DMDX (32): 3.38, 3.5
MatLab (5): 4.2, 4
Presentation (12): 4.54, 5

*

We also asked, "How easy/hard is it for a novice to learn how to  
build experiments with your software?"  The mean difficulty rating  
was 4.12 with a standard deviation of 1.63, and a median of 4.  Here  
are the mean and median build-difficulty ratings for the primarily  
used packages (for this analysis, PsyScope classic and PsyScope X  
were combined, because their response profiles were similar; SuperLab  
and SuperLab pro were different, so are reported separately):

SuperLab (5): 2.6, 3
PsyScope (29): 3.5, 3.5
E-Prime (57): 3.54, 3
ERTS (6): 4.17, 4.5
SuperLab Pro (4): 4.25, 4.5
DMDX (32): 4.88, 5
NESU (8): 5, 5.5
Presentation (12): 5.64, 6
MatLab (5): 6, 6

*

We then asked, "How satisfied are you with your current software?"  
with "1" meaning "Completely dissatisfied" and "7" meaning  
"Completely Satisfied."  The mean satisfaction rating was 4.59 with a  
standard deviation of 1.43, and a median of 5.  Here are the ratings  
by package:

DMDX (32): 5.09, 5
PsyScope X (11): 4.91, 5
E-Prime (57): 4.77, 5
MatLab (5): 4.4, 4
SuperLab (5): 4.2, 4
Presentation (12): 4.09, 4
NESU (8): 3.88, 4
PsyScope classic (18): 3.82, 4
SuperLab Pro (4): 3, 3

*

Two more quantitative questions.  First, we asked "Are you sticking  
with your current software for the foreseeable future, or are you  
looking to change setups?"  25 respondents responded with "Don't  
Know," 103 with "Sticking with my current software for the  
foreseeable future," and 44 with "looking to change."  Here's the  
breakdown by package, reporting percentages of those sticking with  
their package and those looking to change (sorted by sticking  
percentage):

PsyScope X (10): 90%, 10%
E-Prime (52): 71.1%, 9.6%
SuperLab (5): 60%, 40%
MatLab (5): 60%, 40%
DMDX (30): 53%, 20%
Presentation (11): 45.4%, 36.4%
ERTS (5): 40.0%, 60%
PsyScope classic (16): 37.5%, 62.5%
NESU (8): 12.5%, 62.5%
SuperLab Pro (4): 0%, 50%

*

Finally, we asked "Would you recommend your current software?"  138  
people said "yes" and 32 said "no."  Here's the breakdown of percent  
"yes" responses by package:

E-Prime (50): 92%
PsyScope X (10): 90%
PsyScope classic (16): 87.5%
DMDX (30): 86.7%
MatLab (5): 80%
Presentation (12): 63.6%
SuperLab (5): 60%
ERTS (5): 60%
SuperLab Pro (4): 50%
NESU (8): 50%

*

We also asked two complementary open-ended questions that aren't easy  
to summarize.  One was, "What do you like about your current  
software?  What are its strengths? What does it do well?" and the  
other was, "What do you not like about your current software?  What  
are its weaknesses?  What does it not do well (or at all)?"   
Considering the big hitters (E-Prime, DMDX, and PsyScope), my general  
impression of the flavor of the comments were:

E-Prime:  Easy to learn, good support, user friendly, etc.  But, some  
consider it expensive, thought it inflexible, and are worried about  
precision of timing.

DMDX:  It's free, powerful, good timing, good user-support group, and  
good author support.  Weaknesses were mostly regarding lack of  
intuitiveness and steep learning curve.

PsyScope:  It's free, user-friendly, timing is accurate.  But it can  
be buggy.  PsyScope classic users are worried about using a legacy  
system.  PsyScope X users worry about the transition to Intel-based  
Macs, but with some optimism.

*

The Excel file with everyone's responses is available on this page  
(sorry for the ugly link):

http://lpl.ucsd.edu/LabPage/Lab_Blog/B1A6A7D2-0069-41E3-89E9- 
B3683FEEC758.html

Again, thanks for participating.  We were thrilled to see that we  
actually had 187 people respond!

Best wishes,

Vic Ferreira
Jeremy Boyd
Jeff Elman
Robert Buffington
Bob Slevc

Center for Research in Language
University of California, San Diego

*

DMDX: http://www.u.arizona.edu/~kforster/dmdx/dmdx.htm
E-Prime: http://www.pstnet.com/products/e-prime/
ERTS: http://www.erts.de/
Experiment Builder: http://www.eyelinkinfo.com/optns_eb.php
Linger: http://tedlab.mit.edu/~dr/Linger/
MatLab: http://www.mathworks.com/
MEL: (Note that MEL is the predecessor to E-Prime)
NESU: http://www.mpi.nl/world/tg/experiments/nesu.html
Presentation: http://www.neuro-bs.com/
PsyScope Classic: http://psyscope.psy.cmu.edu/
PsyScope X: http://psy.ck.sissa.it/
SuperLab: http://www.superlab.com/
SuperLab Pro: http://www.superlab.com/



More information about the Eprime mailing list