Analysis of juggling data : Landmark and continuous registration of juggling trajectories ∗

This paper focuses on the two one-dimensional summaries of the three-dimensional juggling data: tangential velocity and tangential acceleration. These are used jointly to define the beginnings and endings of the 123 cycles in the data divided unevenly over ten trials. Two levels of registration were used. The first was a landmark registration of each trial to a periodic image of itself with the period fixed at 712 milliseconds. The 123 tangential velocity cycles were then subjected to a continuous registration over this fixed cycle length. The amounts of across-trial and within-cycle phase variation, respectively, were surprisingly small, indicating tight neural control over the behavior. Within-cycle phase variation was primarily due to variation in the trajectories of the ball between throw and catch.


Introduction
In this paper, we present results of temporal registration of juggling data described by Ramsay et al. in [1].This data consists of ten juggling trials with a duration of ten seconds and with 11-13 cycles in each trial.As part of our analysis, we account for two sources of phase variation using the tangential velocity computed from the raw x, y, z coordinate functions of the index finger of the juggler as he juggled three balls.First, we co-register all trials using clear landmarks extracted from the tangential velocity function.This step results in juggling cycles that are registered across trials but not within trials.As a second step, we perform continuous registration of juggling cycles using a measure of coherence defined in [2].
Phase variation can be interpreted as the variation in timing of different features along the juggling trials or cycles.In fact, in this case it includes variation within cycles as well as over different cycles within the same trial.For example, if we name the four acceleration maxima in each cycle in anti-clockwise order "centripetal", "break", "hand-over" and "launch", then the standard deviations of these times for the sixth cycle across the ten trials are 46, 49, 42 and 50 milliseconds, respectively.This seems remarkable when compared with the roughly ten-second length of each trial.That is, these values are only approximately 0.4% of total trial time.The eleventh cycle has value of about 0.9%, as expected since inter-trial variation is expected to increase as time increases, which still suggests a remarkably tight control over feature timing in the juggling process.As a comparison, accuracies of single large motor movements in humans are usually of the order of 5% to 10%.The repetitive nature of this task seems to result in an order of magnitude improvement in precision.
We now focus on describing our approach to analyzing the juggling data.In the subsequent sections, we switched to a much lower dimensional basis (than that described in [1]) for representing the coordinate functions and the two tangential derivatives.This basis used ten equally spaced knots per cycle rather than a knot at each observation point.Curves were estimated in this basis by simple unpenalized least squares smoothing.The gain in computation time was large, but the price paid for this was some instability in the computed tangential velocities at the beginning and end of each trial.

Landmark registration
Phase variation, for purposes of this study, is attributable to the co-existence of two time scales.Clock time refers to the familiar time scale defined by international standards derived from physics.But, many systems also exhibit observable behavior determined by sequences of events that we like to call their internal "clocks".The brain, which determines movements by a specific individual over a specific clock epoch, is no exception.The brain's neurological structures order events such as spike bursts with nerve fibres on a scale that runs sometimes fast and sometimes slow relative to clock time.We use t to refer to brain or "juggling" time, and s to refer to the clock time.Relative to juggling time, salient juggling features such as the 712 millisecond cycle and the throw time within a cycle are invariant.But, what we record is the clock time s of an event defined by the relatively accurate clocks within experimental apparatus and the computer that contains this information.The registration process is the estimation of a nonlinear transformation, called a warping function, s = h(t) of "juggling" time into clock time for each trial.That is, we see our clock times s as perturbations of the neurological events that generate the actual juggling behavior.The size of this perturbation is s − t = h(t) − t = d(t) and this is called the time deformation function.
In order to recover the brain times of these events for trial i, and therefore remove its phase variation, we must also compute the functional inverse h ] becomes available.For example, the first peak within each cycle in the tangential velocity trials is a natural choice for defining the beginning of a cycle, since it comes at or just before a ball is launched.Moreover, it is clearly identifiable in all 123 cycles, so that these peak timings can be easily estimated either manually by mouse clicks, or automatically by an algorithm that locates a locally maximal point on a trial within a specified interval.The launch peak clock timings s j are not the same from trial to trial, and are not equally spaced within trials, but their corresponding "juggling" times t j , we assume, will be, and therefore two events integer multiples of 712 brain milliseconds apart are comparable in terms of their amplitude since they are doing essentially the same thing.
Our first registration of the given data utilizes landmark points only.We transform time nonlinearly within each trial so that the launch velocity peaks are aligned across trials.We do this by defining fixed and equally spaced launch peak times that span the same time interval as the first and last launch times.Landmark registration works by passing a strictly monotonic smoothing curve h i for trial i through the points defined by plotting the trial-specific velocity peak times s ij against the equally spaced target velocity times t ij , where j = 1, . . ., n i and n i is the number of velocity peaks in trial i.The registered versions of the trials are defined by smoothing the trial's data as plotted against the functional inverse t = h −1 (s) of this monotone, smooth function.When we plot the registered curves, we will see that the launch peak velocities occur at the same times for all trials, and that the times between these events are the same within trials.
Figure 1 (a) displays the registered and unregistered coordinate functions.We see that the blue registered functions are running ahead of the red unregistered functions for the last half of the trial.Thus, in this case, the registration has advanced time. Figure 1 (b) shows the deformation function h(t) − t for this trial, which is positive over the second half, corresponding to a late-running unregistered curve, which is the expected result.With launch peak velocities aligned, we can now cut each trial into a series of separate juggling cycles, using the registered launch peak velocity times as cut points.

Continuous registration to a periodic image
The landmark registration procedure produced cycles whose durations do not vary across trials, but which do vary within trials and from coordinate to coordinate.Cycle durations ranged in this way from 654 to 736 milliseconds.In this section, we go further by registering each of the trials in such a way that all cycles are nearly equal to the mean duration taken over all 123 cycles, namely 712 milliseconds.Once this is achieved, we can define 123 constant-duration cycles by segmenting the registered curves at every 712 milliseconds, and then proceed to look at variation across all of these cycles using methods such as functional principal components analysis and further within-cycle registration.This is accomplished by using continuous rather than landmark registration.That is, each trial's clock time duration is warped so as to optimize a measure of agreement or coherence between a trial's variation over warped time and the variation of a strictly periodic image of that trial where the period is fixed at 712 milliseconds.The measure of the lack of coherence that we used is that used in Ramsay and Silverman [2], which is the smallest eigenvalue of the following cross-product matrix: This measure reflects the extent to which the registered function values f [h(t)] plot against the target function values f 0 (t) as straight lines.
The strictly periodic image of each trial is obtained by smoothing that trial using a Fourier basis instead of a B-spline basis, with a period of 712 milliseconds, and with enough basis functions to be able to capture the within-cycle features for all coordinates.Ten Fourier basis functions seemed to do a reasonable job.The order four B-spline basis for the log-derivative W (t) of the warping function h(t) that we used had, as the number of basis functions, nine times the number of cycles in the trial plus three.
The amount of registration and of its features is displayed in Figure 2 (a), where the deformation function d(t) = h(t) − t is shown along with the locations of the tangential velocity and acceleration features.Cycle boundaries, defined to be at the first tangential velocity peak, are indicated by vertical dashed lines.The piecewise linearity of the warping or deformation functions is striking.These tend to change slope twice per cycle: first at or near the blue "o"s at the beginning of the cycle when the ball leaves the hand, and second near the green and red "*"s when the hand receives the next ball and is at that point more or less at rest.That is, the linear segments correspond nicely with the time that the ball is in the air and the time over which the throwing motion takes place.
Over the entire trial, we see that the juggling process remains either late or early for several cycles, but tends to return to the base frequency of a cycle every 712 milliseconds.This return to baseline is especially striking for the fourth trial, whose deformation is shown in Figure 2 (b).Here, an early cycle (below the line) is followed by a larger late cycle, and then by a rapid return to the 712 millisecond period, which then is maintained tightly through the second half of the trial.
Overall, it is impressive that the larger deformations are only of the order of 70 milliseconds, or roughly a tenth of the duration of a typical cycle.It seems clear from these results as well as a number of other sets of experimental data that we have analyzed, that the brain is operating with a rather stable clock cycle of around 120 milliseconds.Furthermore, the juggling cycle contains six features or events at which synchronized signals are sent to the many muscle groups involved in juggling.

Summary
The temporal or phase variation within each of the ten trials was removed by first aligning discrete landmarks corresponding to launch tangential velocity peaks to those in a strictly period smooth of the trial with a period of 712 milliseconds.This alignment allowed the juggling trials to be split into corresponding cycles whose lengths were constant across all trials.It was striking that the cycle lengths in the raw data did not drift around the baseline period of 712 milliseconds by very much.In a second step, continuous registration was applied to the 123 fixed-length cycles.The variation in timing of different juggling within-cycle features across cycles was small relative to the length of an entire trial and even with respect to the fixed cycle length.This suggests that the repetitive nature of this activity allowed for very tight neural control over timing.Second, it was also discovered that the total cycle length was remarkably stable across all trials.
In addition to studying the underlying time variable of cycles and trials in this juggling dataset, one would like to examine the spatial structure of the trajectories.By structure, we mean the curve traversed during a juggling trial.
When performing this analysis one should again account for the phase variation inherently present in the juggling data.Splitting each trial into cycles, as described in this paper, would allow one to study the variation of all cycle curves within each trial and even across different trials.This can be accomplished by utilizing tools from shape analysis and it could shed light on the stability of not only the timing of the cycles but also their variability in three-dimensional space.
A limitation in these analyses that will be rectified in other papers is the ignoring of phase variation across coordinates due to using only tangential velocity and acceleration in the registration process.

Fig 1 .
Fig 1.(a) Landmark-registered (blue) and unregistered (red) coordinates for the first trial.The landmark used for this registration was the initial peak velocity in each cycle at the point where the ball was launched.(b) The deformation function h(t) − t, where h is the nonlinear transformation of time estimated by the landmark registration.

Fig 2 .
Fig 2. The deformation function, h(t) − t, for the first trial (a) and the fourth trial (b).The three tangential velocity features are indicated by open circles, and the five acceleration features by asterisks.The vertical dashed lines indicate the beginning of each cycle at the first velocity feature.