Select page from this chapter: Multiple Tasks, Project Sharing

Multiple Tasks

Overview

In simple usage, the "job" is equal to a video file, plus a "task" to perform on that video file (a frame range to track, the list of methods to run, and the parametrisation of those methods). Thus, a "job" is just like a "video" in earlier versions of the BWTT. Having completed a tracking "task", if it did not go well, you can "restart" the task, modify the parameters, and repeat the tracking. However, on occasion, having completed a tracking task, you might want to extend the tracking job, rather than simply repeat it.

When might this arise? First, you might want to extend the frame range you originally tracked, without repeating the tracking you have already done. Second, you might want to add a new tracking pipeline - for example, you may have already tracked the snout throughout the video, and now you want to track the whiskers in some frame range as well. Third, your tracking may have failed in some part or parts of the frame range you tracked, and you want to repeat the tracking in those parts with different parameters.

Thus, in general, a "job" consists of a list of multiple tasks, not just one. The most recent (at the end of the list) is known as the "current task". To add an additional task to a job you "close" the current task, and then create a new one. This section describes this process by example.

Example

Initial snout tracking

Use Project/Add Job to create a new job, and pick a suitable video. Save the project now. Use Quick Edit to select Snout Tracking, and then use the Frame Range panel to set a short frame range (I use 48 to 68 in this example, if you want to follow along). Click on Task Pars to parametrise the task, make sure it works by clicking "Test", and then click "Next" to exit the GUI. The "current task" (snout tracking in your chosen frame range) is now ready to run - use the Play button to run it, now.

The image above shows how things should look, at this point. The current task was snout tracking between frames 48 and 68, and the green dot indicates that it has completed successfully. You can click on "+" to open the Review GUI at this state, and you will see that one task is listed, something like the image below.

Extend snout tracking

Let's say we wanted to extend this snout tracking or that the snout tracking had not worked well in the frame range 58 to 68. We'll do both things at once - both overlay new snout tracking on the range 58 to 68, as well as adding tracking in the range 69 to 78. The procedure is straightforward - close the current task, then create a new task exactly as above.

Use Selected/Close Current Task to close the task. The effect on the item in the job list is that the green dot becomes a grey dot (because the current task is now "absent", exactly as if we had just created the job but not yet created any tasks), and the green S indicating the snout tracking pipeline disappears. See the image below.

Now we create a second task. Proceed as above - use Quick Edit to select Snout Tracking, set the frame range to 58 to 78 (half of which overlays the existing tracking, half of which is brand new), and set the key frame to 78 so that the results end up being at least slightly different in the 58 to 68 range. Click Task Pars and do that thing, and then run the new "current task" exactly as before. Finally, open the Review GUI (click "+") to understand the new situation.

The Review GUI will look like the image above. Two tasks are now listed, and the sliders show that they have overlapping frame ranges. There is fairly limited information in this GUI at the moment - this is something that needs a bit of work, still, but you can at least vaguely see what's going on. Moving through the video (using the left and right arrow keys) you will notice that the snout tracking is seamless from frame 48 through to 78. The review GUI automatically shows the results "collapsed" over all tasks - this means you are seeing the results from the first task in the range 48 to 57, and those from the second in the range 58 to 78 (i.e. later tasks override the results of earlier tasks). This "collapsing" is used when results are exported as well.

NB: Whilst you will usually only want to see the collapsed (merged) results, results from all tasks, current and past, are available through the job object.

If you decided that the second task had actually generated results that were worse than the first, you can hide the results from the second task by unchecking the box. Now, the "collapsed" results consist only of those from the first task. If you are in the frame range 58 to 68 (the overlapping region) when you toggle this box, you may notice that the displayed results change slightly, since the tracking was run forward in the first task and backwards in the second. The state you leave these checkboxes in when you close the Review GUI "sticks" - that is, if you export results from the job, only tasks with their checkboxes ticked will be involved in generating the collapsed results that are exported.

Overlaid whisker tracking

Now, we come back to this job weeks later, and perhaps want to add whisker tracking just in some range. The procedure would be as follows. We'll track the whiskers in the range 55 to 60, covering the break between the two overlapping snout tracking tasks, just to illustrate that this is fine.

Use Selected/Close Current Task to close the task. Use Quick Edit to select Whisker Tracking (wdIgorMeanAngle). Set the frame range to 55 to 60 (any key frame will do). But wait - this new task includes snout tracking, and we don't need that. Click "Edit Task" and deselect all snout tracking by changing sdGeneric and stShapeSpaceKalman to (do nothing). You'll get a warning that you're modifying the task, click on "Yes". Your job list entry should now look like the image below, more or less.

Click on "Task Pars", and modify parameters if you need to, then exit using "Next". Finally, click on Play to run the whisker tracking. When the tracking completes, click "+" to open the Review GUI.

NB: If you selected a frame range for whisker tracking that was not completely covered by snout tracking, the task will complete with an error when it reaches a frame for which snout tracking is not available.

The Review GUI should look something like the image above. Three tasks are now listed (with, as mentioned, relatively little information currently). As you scan through the frame range 48 to 78, you'll notice that snout tracking is available throughout, but that whisker tracking is only present in the 55 to 60 range. This process, of adding tasks to a job, can be continued indefinitely.

Export results

We'll just export the results from this job, to see what we get. Click "Export" and choose a filename. Load the export file in Matlab.

You'll notice that export.jobs{1}.frameIndices covers the range 48 to 78 (i.e. the range of all tasks that have not been deselected in the Review GUI). Looking at export.jobs{1}.results{1}, you'll find only snout tracking. In the range that whisker tracking was present (e.g. export.jobs{1}.results{10}), you'll find whiskers tracking also present. If you selected any output methods, their content will reflect the presence or absence of the raw tracking data they need as input (most output methods will fill time series with NaN for frames where the required input was unavailable).

As for display in the GUI, results are "collapsed" across all tasks in the job that have not been deselected. That means that, for each frame, the results for, say, snout tracking are taken from the last task in the list (the most recent) that includes the snout tracking pipeline and has not been invalidated/deselected.