5 Data Files

Import and Export of Event Data Files

5.1 Experiment Folder Structure

Names in <> are set by user.

5.1.1 experiment.yaml

5.1.2 Group Directories

Each group of subjects is placed within its own folder, with the same name as the group ( e.g. "Control"/, "Drug"/) as the group names in experiment.yaml.

5.1.3 Subject Directories

There is a "subjects" folder within each group folder, and with the "subjects" folder is one folder for each subject in the group (e.g. "Rat01"/, "Rat02"/, etc)

5.1.4 Subject Files

Each subject folder (e.g. "CA01"/) contains the individual data files recorded for that subject. Typically each data file represents a single day (or whatever the recording interval was).

Each subject file is a plain-text CSV file, with:

1. initial comment lines prefixed with a hash (#), 2. a single header line to define data columns (number of columns must be a multiple of 3) 3. rows of recorded data.

5.1.4.1 Comment Lines

Comment lines at the start of each data file are prefixed with hashtag/octothorpe (|#|), and have format:

The required lines by key, are:

• expt - the short experiment code as defined in experiment.yaml

• subject – short code of the subject who generated this file (ie the name of the folder this file is in)

• recording-start (y-m-d HH:MM) : the calendar/clock time the recording was started

• recording-start (msec) : same thing, but in unix milliseconds since 1970-01-01

• recording-end (y-m-d HH:MM) : the calendar/clock time the recording was ended (note that usually the data files only record when events occured, so we need to know explicitly when the recording session stopped, especially if it was some time after the last recorded event)

• recording-end (msec) : same thing, but in unix milliseconds since 1970-01-01

Other comment lines could of course be included, but are currently ignored.

5.1.4.2 Header line

The header defines the columns of data. For each event-recorder (eg a lickometer, or a photobeam) there will be 3 columns, corresponding to the starting time-stamp, the magnitude of the event, and the duration of the event. So for a file with "lickometer" and "photobeam" events, the header line would be:

5.1.4.3 Data Rows

Each row of data gives the timestamped data for each event-recording, in triplets of cells for timestamp, magnitude, and duration.

• timestamp is clocktime in unix milliseconds since 1970 denoting the start of the event for that event-recorder. Note that this timestamp applies only to the immediately following event magnitude and duration.

• magnitude is the magnitude of the recorded event. This should be at a minumum set to 1, ie for an instaneous event of duration 0, or for a single discrete behavior like a lick or a rear. Alternatively, it could represent a magnitude caused or expressed by the behavior, such as the force of a lick, or a distance covered by a movement.

• duration is the time from the initial start of the event (given by the timestamp) until the termination of the event. In other words, timestamp + duration will give the end timestamp for the event.

These three triplet cells are repeated on each row for each event-recorder.

NB the number of rows can vary for each event-recorder. The total number of rows in the file will be equal to the maximum recorded events across all event-recorders. Any event recorders with fewer than the maximum number of events will have empty cells in their terminal rows.

<!– insert diagram of what we’re talking about –>

For example, a lickometer might record many thousands of licks, and so occupy thousands of data rows. A foodcup photobeam might only record a dozen food cup entries over the same time period, and so would only occupy a dozen data rows. The timestamps for each event for each event-recorder make the timing of different events on different recorders unambiguous.