Trial number from randomseed

Fri Mar 16 18:58:20 UTC 2012

First, let me clarify the "cautionary tale" here.  The caution is not 
merely to fully test your programs with sufficient test cases before 
running subjects, although indeed that is key to good research; 
rather, I meant to stress the caution of *fully* analyzing subject 
data right from the start, and then throughout data collection.  If 
you had done that (instead of merely confirming that data files 
existed), then you would have caught this problem with the first 
subject and fixed it before running more.

So the full process goes something like this (this is an extended 
version of what I present in the online course, which means I should 
probably not be giving this away):
- Develop research question & hypotheses
- Design experimental task to explore hypotheses.
- Declare full set of specifications for the experimental task.
- Develop software to implement experimental task to declared specifications.
- Throughout software development, test with a full set of input 
scenarios.  For convenience you may run tests with a speeded-up 
program, and for consistency you should use carefully designed canned 
input sequences; you should develop thesee into a battery of test 
cases as you go.
- Every time you make a change to the program, retest it with the 
*entire* battery of test cases to insure that you have not 
re-introduced bugs that you fixed earlier.  This is known as 
"regression testing", as it tests to see that your improved software 
has not regressed to re-introduce previously fixed problems.
- When you have the program fully developed, do complete run-throughs 
on yourself in real time (not sped up).
- Do more full run-throughs, in real time, on lab partners.
- Fix up all the problems revealed through the tests on yourself & 
lab partners, etc.
- With all that done, run your first pilot subject.
- Take the data from the first pilot subject, and run it completely 
through your intended data analysis (excepting for collective 
analyses that require more subjects).  This will likely reveal flaws 
in your analysis methodology, which you will need to correct, and 
those corrections may call for more changes to the program, which 
will put you back several steps here again, etc.
- Once you have success from one pilot subject, go ahead and run a 
few more pilot subjects, at least enough to complete some practice 
collective analyses that you could not do earlier.  That will reveal 
more flaws, etc. etc.
- OK, now you can start collecting data in earnest.  But still, you 
need to keep looking in on your data, identifying anomalies to 
handle, etc., etc.

I hope that captures the general flavor of this 
enterprise.  Remember, anytime you think that your program works, it 
just means that it does not work, and you have simpy failed to 
discover what it does wrong yet.  Software development is very 
deceptive that way.

Anyway, I hope that you do not hope that you can somehow use the 
RandomSeed to recover the data directly from the data files you 
have.  All hope is lost for that.  If your data files show only one 
line, then they have no other "hidden" data to recover, RandomSeed or 
no.  What you see is what you have.  Sure, you may use E-Recovery to 
recover lost data from data .txt files, but if EP already 
successfully created the .edat files, then E-Recovery will not find 
any more data.

So what use is RandomSeed in this case?  Well, if you *re-ran* the 
.ebs program, after entering the RandomSeed, it should reproduce the 
same sequence as during the subject's run.  But even here you face 
several "gotchas", because you would have to make other changes in 
your program in order to get EP to reveal the sequence this time, and 
who knows what effect that will have on the resulting trial 
sequences?  For one, you will modify the program to accept a 
RandomSeed, but we may trust that that change alone will not affect 
any sequences.  Next, you might try running the program while 
entering responses -- that might safely regenerate the sequence used 
during the subject's run, but this time logging the data.  Or you 
might first fix the program to log data for non-responses, then run 
it with the RandomSeed -- at that point you have made even more 
changes to the program, so let's hope that that does not affect the sequence.

But this all seems to iffy and tedious to me.  If it were me, I would 
just scrap this data and start over.  Consider yourself lucky.  You 
lost only two weeks of data.  We had a graduate student here who lost 
two years of dissertation data collected on paid clinical subjects 
because they never properly examined the data during data 
collection.  That student failed to get their doctorate.  And that's 
why I am so touchy about this issue.

Finally, you might again consult PST Web Support on this, as they may 
know something that I do not.  And then please write back and tell me 
where I got it wrong.

(Oh, and BTW, it's "no data *were* collected" -- that's a pet peeve 
for us around here.)

-----
David McFarlane
E-Prime training 
online:  http://psychology.msu.edu/Workshops_Courses/eprime.aspx
Twitter:  @EPrimeMaster (twitter.com/EPrimeMaster)

At 3/15/2012 06:36 PM Thursday, you wrote:
>Hi David,
>
>Thanks so much for your quick reply!
>
>Yes - the program failed to log any data at all!
>
>The first time I tried to run the program, to make sure everything was
>working, I pressed spacebar to move on to the next trial, and when I
>checked the data, I saw all the attributes listed. And I took it for
>granted, and didn't run the whole programme again by itself (without
>any response). It is really weird (and upsetting!) - I wouldn't know
>how to turn off the Procedure logging with any inline code. On the
>list with all the attributes, under property name -  when I said I
>checked them, I guess I meant I made sure they were checked (and they
>definitely were). Also, I sent my previous message out in a state of
>panic so I probably wasn't very clear, but I tested the program on my
>own, and I responded to each trial when I tested it. However, when the
>students did the experiment, they did not respond at all - and for
>those, no data was collected.
>
>Anyway, I'm not entirely sure I understood the part about the
>RandomSeed - do you mean it's possible for me to retrieve the
>presentation order from the RandomSeed? I really hope so - because in
>principle, if the user can enter a randomseed number that fixes the
>presentation order, it should work the other way too?
>
>I will look around the knowledge base and pst website. I really hope I
>find something! Thanks!
>
>
> > So reading through your plea, I understand that your program did not
> > merely fail to log the trial/stimulus order (which would have been
> > odd, because by default EP *always* logs *every* List attribute used
> > *in presentation order*, whether you want it or not), but failed to
> > log any data *at all*.  Is that right?
> >
> > And I am (figuratively) scratching my balding head trying to figure
> > out how you managed to get things logged when giving a response, but
> > not for non-responses.  That is really weird.  EP by default logs all
> > information at the end of each Procedure, response or not, so I
> > cannot imagine how you got rows of data for responses, but not for
> > non-responses.  As far as I can figure, you would have to go *way*
> > out of your way to turn off the automatic Procedure logging and
> > replace it with your own inline code, which would be bad unless you
> > *really* knew what you were doing.  So I must be overlooking
> > something obvious.  But no matter.
> >
> > Finally, you said that, "On the list with all the attributes, under
> > Logging," you "checked the attributes property name".  Hmm.  By
> > default, those Logging attributes already have check marks; if you
> > went in and manually "checked" them, then you really unchecked them,
> > and that turned off their logging.  But that would mean that they
> > would fail to log for both non-responses *and* responses, which does
> > not square with your account.  So I am stumped on how you got to 
> this state.
> >
> > Anyway, that was not your question at all.  As I take it, you merely
> > want a mechanism to use the recorded RandomSeed to regenerate the
> > presentation order.  Well, the short answer is, if you look around
> > the E-Prime Knowledge Base on the PST web site, you will find
> > instructions on how to enter a RandomSeed value and have EP run with
> > that.  As I recall, it comes down to adding RandomSeed as a
> > StartupInfo parameter (I cover StartupInfo parameters in my
> > course).  For details, consult the EP KB, or PST Web Support.
> >
> > -----
> > David McFarlane
> > E-Prime training
> > online:  http://psychology.msu.edu/Workshops_Courses/eprime.aspx
> > Twitter:  @EPrimeMaster (twitter.com/EPrimeMaster)
> >
> > At 3/15/2012 03:23 PM Thursday, you wrote:
> >
> >
> >
> >
> >
> >
> >
> > >Dear all,
> >
> > >I am quite new to E-prime and I've just realised that the programme
> > >that I have been using to collect data for the past two weeks hasn't
> > >been logging the data correctly!
> >
> > >Basically, participants are presented with a series of attributes
> > >describing four hypothetical apartments (these attributes are
> > >presented for a fixed time of 4seconds). At the end of it,
> > >participants are asked to decide which apartment they would like to
> > >live in.
> > >I am interested in how the order of presentation of attributes affects
> > >this final judgement and there were some obvious issues with have a
> > >fixed order across all participants so we decided to randomise it.
> > >However, we decided on this with the impression that E-prime would log
> > >the data of the order in which the attributes appeared (at random). On
> > >the list with all the attributes, under Logging, I checked the
> > >attributes property name. I guess I made a huge mistake when I was
> > >testing the programme because I pressed spacebar to move on to the
> > >next attribute, and when I looked at the data, the attributes order
> > >was indeed recorded. I didn't double check if the same would happen if
> > >no response was made at all, which was the case for all participants
> > >since they were asked to just read each attribute and the next one
> > >would appear automatically.
> >
> > >I guess another reason why I didn't realise this mistake earlier was
> > >because I did get a E-DataAid file after each participant - however,
> > >there was just one line with the subject, session... and random seed.
> > >I know this is very far fetched, but I'm wondering if there is any
> > >chance at all that I could somehow retrieve any form of information
> > >about the order of trials from the randomseed number?
> >
> > >Thank you so much!!

-- 
You received this message because you are subscribed to the Google Groups "E-Prime" group.
To post to this group, send email to e-prime at googlegroups.com.
To unsubscribe from this group, send email to e-prime+unsubscribe at googlegroups.com.
For more options, visit this group at http://groups.google.com/group/e-prime?hl=en.