JN Ad Instruments
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


J Neurophysiol 100: 504-515, 2008. First published May 7, 2008; doi:10.1152/jn.00012.2008
0022-3077/08 $8.00
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Supplemental Figures and Videos
Right arrow All Versions of this Article:
100/1/504    most recent
00012.2008v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Voigts, J.
Right arrow Articles by Celikel, T.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Voigts, J.
Right arrow Articles by Celikel, T.

INNOVATIVE METHODOLOGY

Unsupervised Whisker Tracking in Unrestrained Behaving Animals

Jakob Voigts1,2, Bert Sakmann3 and Tansu Celikel2,3

1Undergraduate Program in Mathematics, University of Heidelberg, Heidelberg, Germany; 2Laboratory of Neural Circuits and Plasticity, Section of Neurobiology, University of Southern California, Los Angeles, CA; and 3Department of Cell Physiology, Max Planck Institute for Medical Research, Heidelberg, Germani

Submitted 5 January 2008; accepted in final form 3 May 2008


 ABSTRACT
 
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
Understanding how whisker-based tactile information is represented in the nervous system requires quantification of sensory input and observation of neural activity during whisking and whisker touch. Chronic electrophysiological methods have long been available to study neural responses in awake and behaving animals; however, methods to quantify the sensory input on whiskers have not yet been developed. Here we describe an unsupervised algorithm to track whisker movements in high-speed video recordings and to quantify the statistics of the tactile information on whiskers in freely behaving animals during haptic object exploration. The algorithm does not require human identification of whiskers, nor does it assume the shape, location, orientation, length of whiskers, or direction of the whisker movements. The algorithm performs well on temporary loss of whisker visibility and under low-light/low-contrast conditions even with inherent anisotropic noise and non-Gaussian variability in the signal. Using this algorithm, we define the speed [protraction (P), 1,081 ± 322; retraction (R), 1,564 ± 549 °/s], duration (P, 34 ± 10; R, 24 ± 8 ms), amplitude (P = R, 40 ± 13°), and frequency (19 ± 7 Hz) of active whisking in freely behaving mice. We furthermore quantify whisker deflection induced changes in whisking kinematics and calculate the statistics (i.e., speed, amplitude and duration) of whisker touch and finally show that whisker deprivation does not alter whisking kinematics during haptic exploration.


 INTRODUCTION
 
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
The whisker system of rodents has emerged as a popular model to study mechanisms of sensory information processing, neural plasticity, and sensory organ representations (Allen et al. 2003Go; Arabzadeh et al. 2004Go; Armstrong-James and Fox 1987Go; Celikel et al. 2004Go; Clem et al. 2008Go; Diamond et al. 1994Go; Foeller et al. 2005Go; Fox and Wong 2005Go; Houweling and Brecht 2008Go; Larkum et al. 1999Go; Pinto et al. 2000Go; Shulz et al. 2000Go; Simons 1985Go). In the absence of techniques to quantify the sensory information on whiskers in behaving animals, most of the research has focused on quantification of neural responses following passive whisker deflections under anesthesia. Despite the advantages of studying sensory responses to precisely controlled passive whisker stimulation, understanding how whisker-based tactile information is processed during active tactile sensation requires simultaneous quantification of sensory input on whiskers and observation of neural activity during whisking.

Observation of whisking in freely behaving animals, especially in mice, is a challenging task. It's not only because mice whisk at higher frequencies than larger rodents like rats (Knutsen et al. 2005Go) and guinea pigs (Jin et al. 2004Go), but also the higher density and thinner structure of mice whiskers require that observation of whisking is performed using high spatial density of imaging sensors. Recent application of optoelectronic methods (Bermejo et al. 1998Go; Hentschke et al. 2006Go) and image processing tools (Knutsen et al. 2005Go; Mehta et al. 2007Go) have allowed for high spatiotemporal acquisition rates, making it possible to track whisker movements in behaving rodents. Although the latter method allows tracking the entire length of whiskers in high-speed video recordings at the expense of slower processing rates and user intervention, optoelectronic methods allow real-time, fully automatized tracking of a single location on the whisker indirectly, at high temporal rates in immobilized animals.

The ideal method for tracking whisker movements should combine the desirable features of these two methods. It should allow tracking the entire length of whiskers in freely behaving animals, even under less favorable conditions (i.e., low or no visible light, low contrast, high noise, temporary loss of whiskers of interest) without any human supervision. Here we describe such an algorithm to track multiple whiskers simultaneously in video recordings. The algorithm is fully automatized and unsupervised. It locates all whiskers in a given frame and traces the entire length of whiskers. Because tracing of whiskers does not require prior knowledge on location of whiskers in previous frames, the algorithm is suitable for tracing whiskers also at low sampling frequencies using conventional videography techniques without initial identification of whiskers by human observers.

One of the advantages of whisker tracking from video recordings is that information about the tactile exploration of target objects can be captured in subsequent images. If the entire length of whiskers can be traced, the statistics of whisker touch induced changes in whisker morphology and kinematics, including those variables that somatosensory neurons encode (e.g., whisker deflection amplitude, direction, duration of whisker touch and angular velocity during touch) can be quantified in freely behaving animals during active touch. Taking advantage of the whisker-tracking algorithm detailed herein, we also describe a method to detect whisker contacts in freely behaving mice and use this algorithm to quantify the temporal course, amplitude, speed, and angular velocity distribution of whisker palpations during object exploration.


 METHODS
 
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
Adult C57Bl6 mice (>P120) from either sex were used according to the animal welfare guidelines of the Max-Planck Society and the National Institute of Health. Mice were housed individually and kept on a 12-h light/dark cycle with ad libitum access to food and water. All experiments were performed during the light phase, and each animal was acclimatized to the experimental room for >1 h before behavioral experiments started. Those animals assigned to one of the whisker deprivation conditions received whisker plucking under isofluorane (Baxter) anesthesia 1 day before imaging experiments.

Behavioral apparatus

Spontaneous gap-crossing is similar to gap-cross training (Celikel and Sakmann 2007Go), with the exceptions that animals are not food deprived before the training and are not rewarded for successful task execution during the training. The apparatus was identical to the one described previously (Celikel and Sakmann 2007Go), except that two infrared light sources and a video camera were placed on one of the two platforms, i.e., the target platform (Fig. 1). The platforms (12 x 20 cm, width x length) were elevated 40 cm off the surface and surrounded on the three sides by 30-cm high walls. The width of each platform was reduced to 4 cm at the edge by the gap to restrict the mouse accessible portion of the target platform and allow higher optical zoom to maximize the spatial resolution of the video recordings.


Figure 1
View larger version (16K):
[in this window]
[in a new window]

 
FIG. 1. A: experimental set-up consists of 2 elevated platforms placed end-to-end with an experimenter-defined distance between them. Five sets of motion sensors were used to track animal location on the platforms and controlled high-speed video acquisition. B: the target platform is lit using infrared LED sources. Focused light beams coming from the lower LEDs were diffracted using an opal glass diffuser. C: block diagram that depicts the motion sensor control over the video acquisition.

 
The target platform was illuminated with custom-made infrared light emitting diode (LED) arrays placed on a heat sink and powered by a DC source. One of the two separate light sources was placed 14 cm below the target platform. An opal glass diffuser (Edmund Optics) was placed above the light source to create uniform backlighting at the gap (Fig. 1B). The second set of lights were placed 14 cm above the target platform with a at 45° angle to the platform surface.

Five sets of motion sensors (MSs) were distributed along the apparatus to track the animal location and control image acquisition. Two of the sensors were positioned at the two ends of each apparatus. An additional sensor was placed 2 cm before the target platform to trigger the transfer of images from the on-board camera buffer to the hard-drive. The activity detected by motion sensors were digitized (Micro 1401mkII, Cambridge Electronic Design) and acquired at 1 kHz using routines written in Spike2 (Cambridge Electronic Design).

High-speed videography

Animals' exploratory whisking onto the target platform was tracked with a video camera acquiring images at 1,000 frames/s with a spatial resolution of 0.09 mm/pixel under white light and 0.18 mm/pixel for infrared recordings. The camera (Motionscope M1, Redlake) was fixed to the target platform and placed at a stationary position looking at the edge of the platform with a 90° angle. The camera was equipped with an on-board memory buffer (maximum 4,096 frames at 640 x 512 resolution). Image acquisition was performed in two phases (Fig. 1C): first, when the animal arrived at the edge of the home platform, MS#2 armed the camera to buffer images; then as animal crossed MS#3, image acquisition was initiated. Image buffering continued until the animal crossed MS#4 or the on-board memory buffer was full. Those trials where animals did not make a decision on whether or not to gap-cross before the memory buffer was full were excluded from the subsequent image processing and analysis.

At high frame acquisition rates, because of fast shutter speed (1 ms in this study), adequate illumination of the imaged object requires a luminous environment. We used infrared light as the light source because of impaired C57/Bl6 vision at and beyond 650 nm (Jacobs et al. 1999Go).

Image analysis

Image sequences were read off-line from hard disk and calibrated in brightness by a factor determined from a background portion of every image to equalize light intensity across images. An average of 50–100 frames, acquired before the animal entered the field of view, was subtracted, and the position of the target platform's dark colored edge was determined by finding the minimum brightness across rows of the averaged frame.

The position of the animal head was determined by tracking the nose that was assumed to be the object closest to the edge above a given brightness threshold when averaging in the x-direction (i.e., across columns). The approximate outline of the head was determined by spatial filtering that included radial (outward) thresholding from the head center and Gaussian smoothing of the resulting coordinates.

Tracing whisker-like structures

Tracing individual whiskers was accomplished frame by frame by 1) generating vector fields from each frame that resulted in a convergence of flows on whisker-like structures; 2) integrating vector fields to generate spatially contiguous traces of paths converging on whiskers; 3) grouping and averaging each converging set of paths into whisker "splines"; and 4) grouping whisker splines from individual frames into time-contiguous spatial representation of whisker positions.

GENERATING VECTOR FIELDS.  A set of images, denoted similarity index (SI) hereafter, which hold information on how much the surrounding of each pixel is invariant to directional (e.g., anisotropic) blurring and shifting (Fig. 2), was computed by correlating brightness of pixels to shifted and blurred environments of the pixel in a given direction.

Formula


Figure 2
View larger version (2K):
[in this window]
[in a new window]

 
FIG. 2. Example matrices of shifting and blurring point spread functions for angles from 0 to 45°.

 
The resulting angular data showed distinctive peaks on and around directed structures like whiskers (Fig. 3 B). The angle with the highest value was used to compute a vector field and showed the direction of minimum anisotropy at any given pixel in that particular frame. With our data, discretization in ~50 steps yielded acceptable signal-to-noise (S/N) ratios. Using the contrast range and steepness of SI at a given pixel across the angle range yielded a reliable indication of the presence of whiskers. Because of the high sampling width in the filtering steps, the resulting data exhibited very low noise even when the contrast of the input frames was very low (see RESULTS).


Figure 3
View larger version (61K):
[in this window]
[in a new window]

 
FIG. 3. Processing steps of the tracing algorithm. A: a subsection (100 x 100 pixels) of a raw frame after background subtraction. B: polar representations of similarity index (SI) are plotted for each data point every 5 pixels; peak scaled by contrast across all angles. C: paths before interpolation. The green line indicates the starting points for the integrator. D: splines computed from grouped and filtered paths. E: splines and paths used to trace whiskers back to the face. F: splines combined from outward (see C and D) and inward (see E) traces.

 
Omission of structures of previously known orientation, such as the platform edge in our condition, could be achieved by setting areas of SI to zero in the affected spatial region and angle range (Supplementary Fig. 1).1

Moreover this method has shown promising results when used to trace full whisker arrays without whisker deprivation even though overlapping whiskers and high whisker densities cause incomplete tracing and shape artifacts (Supplementary Fig. 2). In selected frames with high sharpness, overlapping whiskers resulted in two distinguishable sets of peaks in SI at the site of intersection, suggesting that the approach could also be adapted to circumvent these problems.

An additional vector field holding the local brightness gradient was added with a coefficient of ~0.5 to improve the convergence of flow onto the whiskers. Without this bias, the paths tended to follow the outer edges on the whisker, especially when whiskers were blurred, making grouping difficult. The scaling of this component was done so that the vector field derived from SI was always dominant.

INTEGRATING VECTOR FIELDS.  The integration of the vector field was realized using a first-order Euler-method with a step size of ~6 pixels. Application of angle dependent weighting to SI, before generation of the vector field, gave the option to bias the flow without affecting accuracy of the peak detection. This was done to counteract the local symmetry of the whiskers by initially biasing radially outward from the head center. Jumping between different peaks in SI caused by overlapping directed structures (i.e., neighboring whiskers, background structures) was avoided by biasing in the direction of the last step. A small inertia term was used to increase the stability without compromising precision even when low curvature radii occurred during whisker deflection. The threshold for stopping integration was empirically selected to a low value, resulting in paths that were often diverging for one or two steps at the end of whiskers before stopping (Fig. 3C). This ensured that even weak signals were fully exploited and the full length of the traced whisker shaft is reconstructed. The resulting variability at the whisker tip was very low because of the number of paths averaged into each whisker and was compensated by shortening of the splines by one step. For our sequences, starting coordinates were distributed ~12 pixels outside the head outline to avoid the fur and stubs of the clipped whiskers. However, any starting coordinates within the frame, for example, on the platform edge, would reliably pick up whiskers passing through them.

AVERAGING CONVERGING PATHS.  A number of paths, varying between 4 and 25 depending on the density of whiskers and integration starting coordinates at the base, converged onto each whisker in every frame. Individual paths were rejected based on a number of thresholds for individual or averaged pixel values, length, and curvature. Surviving paths were averaged using the nearest Euclidean approach for each point on neighboring paths. Groups containing less than four paths were rejected, and a standard least square error method was used to calculate a spline representing each whisker starting from 2.6 to 3.1 mm outside of the whisker pad. To trace the entire length of the whisker(s), the same procedure was repeated, but this time N (in this study n = 10) starting points at one third of the whisker length were used to trace whisker(s) in the inward direction (Fig. 3E). The two resulting splines were combined to a complete whisker representation (Fig. 3F).

GENERATING TIME-CONTIGUOUS SPATIAL REPRESENTATION OF WHISKERS.  After all frames of an image sequence were processed, splines originating from the same whisker in different frames were identified. To handle cases where whiskers were temporarily invisible, e.g., because of excessive up-down head movements, the data were organized into three-dimensional splines (i.e., streaks) that represent a whisker in a subsequence of frames where whisker identity could be maintained. A similarity measure, like the one used for path grouping, was computed for all whisker splines. Those whisker splines above a threshold were added to the streak with the highest similarity score. Ambiguous cases were solved by giving priority to whiskers with only one viable streak. Streaks were rejected if they were very short (<5 frames) or exhibited high-frequency (>4,000 °/s) movements well above physiologically possible values, eliminating occasional erratic tracing of hairs like supraorbital whiskers. For those splines that cannot be combined in a streak, a new streak was created. The streaks were evaluated to extract x,y,t coordinates and archived together with the nose position (Supplementary Fig. 3).

Quantification of whisking kinematics and detection of whisker touch

Raw streaks were recalled from hard disk and user-defined N (in this study, n = 40) equally distributed points (a.k.a. whisker splines) along each whisker were extracted.

Velocity orthogonal to the whisker shaft, acceleration, and the angle in respect to the base point were computed along the entire whisker splines. This gave the option of comparing data at different points along the whisker with high precision. The detection of whisker touches onto the platform was accomplished with a fuzzy-logic approach. A spatial filter combining scores for distance to the platform, speed, and acceleration for each point along the whisker was calculated and the point of the highest value above a threshold was marked as collision. In collisions where a whisker swiped the top of the platform without significant slowing or deflection, using a deceleration threshold while compensating for active whisking movement gave a somewhat usable measure for contact of large whiskers (i.e., macrovibrissae). However, the reliability of the method was not sufficient at the current stage of development. Therefore, only those whisker collisions onto the platform edge, which constituted roughly 96–98% of all palpations before gap-crossing, were classified as whisker touch. See RESULTS for a detailed performance comparison of the collision detection algorithm in respect to human encoders.

Implementation

The method was implemented primarily in MATLAB (MathWorks, Natick, MA). The image filtering routines were implemented in C using the NVIDIA Compute Unified Device Architecture platform (http://developer.nvidia.com/object/cuda.html) and were executed on a NVIDIA Geforce8800 graphics-processing unit (GPU). The reassigned Gabor spectrum computations were performed using The Time-Frequency Toolbox (http://tftb.nongnu.org/).


 RESULTS
 
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
Unbiased, unsupervised tracking of the entire length of whiskers

We developed an algorithm to track whisker movements along the shaft of whiskers. Different from the previously described image analysis method (Knutsen et al. 2005Go), this algorithm does not require a human observer's identification of whisker(s) or individual x,y locations along the whisker shaft to "lock onto" whiskers. Instead it locates whisker(s) in the frame using thresholded anisotropy functions. Because actual whiskers rather than a limited number of points along the whisker are tracked across frames, this approach allows a comparison of the whisking kinematics throughout the entire length of the tracked whiskers. An example of how location with respect to the whisker follicle (i.e., base) affects the kinematics of whisker movement is shown in Fig. 4.


Figure 4
View larger version (83K):
[in this window]
[in a new window]

 
FIG. 4. Tracking single whisker along its length. A: example frame from background subtracted raw data and an overlay of the whisker traced using the algorithm described in the text. B: example trajectory from 0.5 s of whisking data for which whisker angle was measured at 3 points along the whisker. Crosses indicate onsets of whisker collisions with the target platform (a.k.a. whisker touch). C: logarithmic reassigned Gabor spectrogram (300-ms sliding window) shows the frequency of whisking along the tracked whisker. RGB (red, green, blue) channels represent spectra from base, middle, and tip of the whisker, respectively. Each channel is individually peak-scaled. Note the shift between the channels. See Supplementary Movie 1 for the analyses on the complete data set from this trial. Please note that the vertical ridges in the spectrogram are not necessarily part of the frequency distribution but an artifact inherent to the reassigned Gabor procedure (Flandrin et al. 2003Go).

 
Average frequency of whisking movements increased as the distance between the observed location on the whisker of interest and the base increased (Fig. 4C; Supplementary Movie 1). The entire whisker shaft had a principal frequency of movement at the range of 15–20 Hz varying in time (Fig. 4C), as quantified using reassigned Gabor (Flandrin et al. 2003Go) spectrum (see Supplementary Fig. 4 for a comparison between Fourier and Gabor transform). Not surprisingly the high-frequency component of the whisker movements were more visible at the set points of the whisking cycle (Fig. 4B). After applying a high-pass filter at the mean whisking frequency + 1 SD (26 Hz), the remaining mean of frequency frequencies of the highest power at the set points (excluding whisks containing collision events) was 36.8 ± 7.7 (SD) Hz (n = 1981), with no difference between rostral and caudal set points (36.9 ± 7.6 vs. 36.8 ± 8; P > 0.6, paired t-test). Increasing the number of whiskers available to explore the target platform did not affect the kinematics of whisker movements. As shown in Fig. 5, tracking C row whiskers bilaterally revealed that whiskers on both sides of the face had comparable whisking amplitude (41.1 ± 13.1 vs. 41.3 ± 12.2°, peak-to-peak; P = 0.65, ANOVA).


Figure 5
View larger version (60K):
[in this window]
[in a new window]

 
FIG. 5. Simultaneous tracking of multiple whiskers. A: example frame with traced whiskers using the algorithm. B: whisker angle averaged across the length of each whisker is plotted for all whiskers across 0.5 s of data. Red traces correspond to whiskers on the left face side. Crosses indicate collision event onsets. C: sum of reassigned Gabor spectra (300-ms sliding window) for left and right side whiskers. Channels were peak-scaled individually. Note that frequency spectrum of the whiskers on the left and right face side overlap for ~300 ms before frequency of whisking increases for those whiskers on the left side. Complete data set from this trial and the corresponding analyses could be found in Supplementary Movie 2.

 
Frequency of whisking, however, occasionally varied across the two face sides similar to the findings reported for rats' whisking behavior (Mitchinson et al. 2007Go; Towal and Hartmann 2006Go). For example, data on Fig. 5C show that whiskers on both face sides had overlapping frequency spectrum whose principal frequency reduced during the first 300 ms before reaching a stationary level at ~15 Hz for the whiskers on the right face side. Whiskers on the left, however, displayed an increased frequency of whisking during the last 200 ms, which resulted in asymmetric whisking during exploration of the target platform (Fig. 5C; Supplementary Movie 2).

One advantage of this algorithm is the possibility to track whisking movements in unfavorable conditions (e.g., under low light, with reduced signal-to-noise ratio). To study the algorithms limits for tracking whiskers under such conditions, we artificially introduced noise to a randomly selected image and tracked whiskers in this image with varying levels of Gaussian noise (Supplementary Fig. 5).

The algorithm was capable of locating the whiskers in the frame even if the signal was just ~14% above the noise (S/N = 1.14; see Supplementary Fig. 5 caption for details), a level below/close to the detection threshold of human observers (Supplementary Fig. 5, bottom row). Expectedly, increased noise in the image resulted in reduction of the tracked whisker length, which is caused primarily by more rapid deterioration of the whisker structure at the thin tip compared with the thick base of the whisker.

Kinematics of whisker movements during tactile exploration

Using the image-processing algorithm described above, we also quantified the kinematics of whisker movements when animals used single or multiple whiskers to explore the target platform.

The full whisking cycle lasted 59.16 ± 21.30 ms, resulting in frequency of whisking 19.22 ± 7.26 Hz for those animals with multiple whiskers. Depriving all whiskers but one did not change the kinematics of whisking. Animals with single whisker whisked at 18.58 ± 5.61 Hz (P > 0.40 vs. multiwhisker, ANOVA), and their full whisking cycle was 58.77 ± 17.25 ms long (P > 0.86 vs. multiwhisker, ANOVA). The similarity between multi- and single whisker whisking conditions was not restricted to the temporal domain of whisking. Amplitude and speed distributions of the protraction and retraction were also comparable independent from the number of whiskers that the animals had during tactile exploration (Table 1).


View this table:
[in this window]
[in a new window]

 
TABLE 1. Statistics of whisking with single and multiple whiskers

 
Similar to findings on rat whisking behavior (Bermejo et al. 1998Go; Carvell and Simons 1990Go; Knutsen et al. 2005Go), the protraction and retraction phases of whisking had different time courses: The speed of whisking was lower during protractions and higher during retractions (Table 1; Supplementary Fig. 6). This change in the speed was independent from the amplitude of whisking because mice displayed similar amplitude of whisking during the two phases of the whisking cycle (Fig. 6, top panel).


Figure 6
View larger version (59K):
[in this window]
[in a new window]

 
FIG. 6. Correlations between whisk amplitude/whisk duration and speed of whisking. Mean whisker angle data, across n = 2,367 full whisking cycles, was segmented into pro- and retractions by applying a 30-Hz low-pass filter and finding peaks that were repositioned to the nearest peak in the unfiltered data. Peak and mean speed were computed from data after low-pass filtering at 100 Hz. In all figurines, blue traces represent data from protraction and orange from retraction cycles. Equations approximate the relationship between velocity and whisking amplitudes across all conditions. A: whisk amplitude vs. whisker speed. B: pro/retraction duration vs. whisker speed.

 
Amplitude of whisking, however, was strongly correlated with the angular speed of whisking movement. Both peak speed of angular displacement (Fig. 6A, middle panels) and mean speed of whisking (Fig. 6A, bottom panels) were positively correlated with the amplitude of whisker movements in the two phases of the mouse whisking.

The duration of the pro- and retraction cycles were loosely correlated with the whisking speed. In protraction cycles, both mean and peak speeds were practically constant over cycle durations, with a slight decline in both very short and long cycles. In retractions, there was a negative correlation between the mean speed and duration (Fig. 6B, bottom right panel) and a weaker correlation between peak speed and duration (Fig. 6B, top right panel).

Using Gabor spectrograms, we found a negative correlation between whisking amplitude and frequency with no significant difference between pro- and retraction cycles (R = –0.43 in protractions and R = –34 in retractions; Supplementary Fig. 7).

The mean whisker angle in pro- and retraction cycles was found to be unrelated to either whisking amplitude or frequency (Supplementary Fig. 8).

Quantification of whisker palpations during tactile exploration

The second purpose of this work was to develop an objective method to describe when and where whiskers contact with a target object during tactile exploration using whiskers. We reasoned that, the point along the whisker (on a frame), where the combination of speed and acceleration of whisker movements are minimal, should be the location of whisker contact with the object being explored (i.e., target platform). (See METHODS for full further details on the algorithm.)

To quantify the reliability of this approach to detect whisker palpations, we compared the performance of the algorithm against two human observers. The observers had an overall error rate of 7.3%, missing whisker deflections in ~7 of 100 frames that the other observer classified as palpation onto the target platform. Among those whisker palpations detected by both human observers, 19.6% of detected palpations were <2 pixels apart from each other. If estimates were allowed to differ ≤10 pixels, the observers agreed in 78.8% of the frames (Fig. 7 ).


Figure 7
View larger version (49K):
[in this window]
[in a new window]

 
FIG. 7. A performance comparison between human observer and the algorithm in detecting whisker touches. A: the distribution of spatial error in locating whisker touch positions (n = 5,710 frames) along the front edge of the target platform. Pairwise comparisons were performed for (human) observer 1-observer 2 (black line), human-algorithm (red striped line), and for observer 1-observer 2 comparisons (blue line) after spline corrections (see B). An additional histogram (green dotted lines) is calculated to track the location of whisker as predicted by the algorithm when human observer reported whisker collisions. The histogram was saturated at 50 px, the peak at which also included those collision events that could not be paired because of differing collision onset times or durations. B: the spline-correction method. Instead of the direct distance (a) between the 2 collision sites (crosses), minimum distances (b1, b2) to the nearest whisker spline were summed up and used if smaller than a.

 
The algorithm detected whisker palpations as well as human observers (92.7 vs. 92.3%), with an overall successful detection rate of 25.1 and 76.9% with spatial error rates of <2 and <10 pixels, respectively (Fig. 7). The error term was not caused by an inability of the algorithm to track whiskers reliably but by the ambiguity in collision onset time when the whisker was sliding along the edge or uncertainty in collision positioning when the whisker was in contact with and locally parallel to the edge of the platform.

Detection of whisker palpations with a target object makes it possible to quantify the statistics of whisker movements during tactile exploration. Taking advantage of this algorithm, we also calculated the duration and amplitude of whisker deflections and of velocity whisker movements during tactile exploration in freely behaving mice.

Duration of a whisker touch on average (n = 713) was 18.4 ± 17.4 ms, although the range varied between 8 and 21 ms (25–75%) with a median of 15 ms (Fig. 8). The phase relationship between the whisking cycle and the onset of whisker contacts with the target platform varied across the two face sides. Whiskers on the left side contacted the target platform at a more rostral location compared with the whiskers on the right side (76.2 ± 12.2 vs. 62.0 ± 21.0% of the protraction phase relative to the forward-most position in the whisk cycle; P < 0.001, ANOVA).


Figure 8
View larger version (15K):
[in this window]
[in a new window]

 
FIG. 8. Histogram of the whisker contact durations during tactile exploration across 713 palpation events. Those events separated by a gap of <5 ms were counted as 1 continuous event.

 
To quantify the angular displacement caused by whisker palpations (i.e., whisker deflection amplitude), we averaged the angular whisker movements at the onset of whisker deflections where angular position of whiskers is described as 1) tip from base, 2) base local, or 3) tip local (Fig. 9 A). As shown in Fig. 9B, whisker palpations onto the target platform were in phase with the basic whisking pattern and did not disturb the sinusoidal movement of the whisker base.


Figure 9
View larger version (26K):
[in this window]
[in a new window]

 
FIG. 9. Amplitude of whisker deformations/deflections during active touch by freely behaving mice. A: angles were measured at the whisker base and whisker tip locally as well as from base to tip. B–D: event-triggered average of (B) local base angle, (C) local tip minus local base angle (i.e., deformation), and (D) tip from base minus local base angle (i.e., deflection) to collision onsets across 713 palpation events. All event-triggered average plots are centered at the collision onset (0 ms). The median (solid line) and ±50% CIs (dotted lines) are overlaid with the raw density plot. The data were high pass filtered at 8 Hz and centered to its mean value for each whisker. Rising angle values represent movement away from the platform. E: distribution of the whisker touch amplitude 20 ms after whisker contact in respect to the baseline.

 
To quantify the whisker deformation in collisions, we calculated the difference between the local whisker angle at the tip (i.e., tip local) and the whisker angle at the base (i.e., base local; Fig. 9A). This value was centered to its mean for each whisker, resulting in a value >0 as the tip is bent away from the platform in respect to the base in collisions (Fig. 9C).

To calculate the whisker deflection amplitude, we subtracted the whisker angle at the base from the displacement (in degrees) of the tip with respect to the base (Fig. 9A). After centering the data for each whisker to eliminate the inherent whisker curvature, this calculation resulted in 0° of displacement if the whisker was approximately in the resting position.

To quantify the amplitude of whisker deformation/deflection during contact, we reverse correlated these measures to the collision onset times and found that the average whisker deformation and deflections during whisker palpations were 25.47 ± 13.84° and 8.80 ± 5.73°, respectively (Fig. 9E).

To further quantify the effect of platform contact, we compared the base whisker angle at rostral set points occurring before the first collision event and to those occurring during collision events. The average angles were –22.25 ± 7.20° for free whisking and –20.45 ± 6.65° for whisker contact (P < 0.001, Student's t-test; Fig. 10). The slightly lower mean angle during contact events can be attributed to the phase locking of whisking and collision events (see Fig. 9B).


Figure 10
View larger version (28K):
[in this window]
[in a new window]

 
FIG. 10. Change of whisker angle at base induced by whisker touch. The medians (black and white lines) and ±50% CIs (yellow and blue ribbons, gray intersection) represent the median of the event triggered angle values at rostral set points that occurred before first whisker contact (n = 399; black line) and during contact events (n = 577; white line). The difference between the 2 medians is plotted as a fine dotted line.

 
To examine whisker deformations of the whiskers induced by platform contact that may not be reflected in changes of local whisker angle at the base or tip (see Fig. 9), we observed how the whisker deforms from its assumed resting shape during contact events. The average whisker shape in frames before the first collision event was assumed as the resting shape (it should be noted that, because of inertial deformations during whisking and the different time courses of pro- and retraction movements, there is probably some systematic error in this method). The resting shape was positioned and aligned to the whisker base, and the derivation of the actual shape was quantified via the area between the splines (Fig. 11 B). The mean derivation was found to be –26.19 ± 149.97 pixels2 (n = 8,004) in collision events and 0.61 ± 84.92 pixels2 (n = 19,428) in free whisking. See Fig. 11C for a representation of the mean derivation along the whisker length.


Figure 11
View larger version (17K):
[in this window]
[in a new window]

 
FIG. 11. Effect of whisker contact on whisker deformation. A: comparison of whisker shapes in free whisking and during contact from 1 experimental trial. The whisker plots are rotated so that the base is always horizontal. The arc plots at the base show the SD of the base angle changes that were eliminated for the plot. B: sketch of the quantification method. Whisker shapes from frames before the 1st contact in streaks that participated in collision events were averaged over time to approximate the resting curvature of each whisker. This mean shape (thick gray line) was positioned and rotated so that the 2 splines were congruent at the base of the whisker (black line) and the relative displacement was averaged. Rostral displacement was defined as positive, caudal displacement as negative. B: average displacement of whiskers in collision events (n = 1,991; dotted line) and in free whisking (n = 7,071; full line) along the length of the whiskers.

 
Speed of whisking during tactile exploration was actively modulated by whisker palpations. Whisker tips moved at the speed of 0.38 ± 0.16 mm/ms in the air, averaged across –15 to –5 ms before contact. When the tips contacted the target platform, the speed reduced with a rate of 0.06 ± 0.06 mm/ms2 (n = 698) to <0.1 mm/ms at an average of 4.7 ± 4.7 ms (n = 683) after whisker contact. The tips reached the speed of movement of <0.05 mm/ms only after 5.8 ± 4.9 ms (n = 645). See Fig. 12 for the median distribution (±CIs) of the whisker tip speed around whisker collision.


Figure 12
View larger version (30K):
[in this window]
[in a new window]

 
FIG. 12. Change induced in speed of whisker movement by whisker touch. Event-triggered average of the whisker tip speed to collision event onset times (n = 713) is centered at the onset of whisker collisions. Whisker tip speed is calculated with respect to the base position, perpendicular to the tip. The median and ±50% CIs (dotted lines) are superimposed on the raw data where gradient between white and black represent the normalized count for each x,y pair.

 
After whiskers palpated through the tactile surface, they accelerated for on average 6.8 ± 4.7 ms with a rate of 0.07 ± 0.16 mm/ms2 before reaching their steady-state speed of movement (Supplementary Fig. 6).


 DISCUSSION
 
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
We developed a novel digital image-processing method to track whisker movements and calculate the statistics (e.g., duration, speed, and amplitude) of whisker palpations in freely behaving mice. The algorithm is fully automatized and unsupervised. It does not require human supervision to locate whiskers. Nor does it assume the shape, location, speed, direction, or orientation of whisker movements as the core of the algorithm relies on thresholded anisotropy functions to trace whiskers.

Using this algorithm, we showed that mice whisk at 19 ± 7 Hz with an average whisking amplitude of 40 ± 13° and whisking cycle (pro + retraction) duration of 59 ± 21 ms during tactile exploration, regardless of the number of whiskers available for object exploration. Whisking speed was actively modulated and varied as a function of the position at the whisking cycle. During protraction (i.e., caudal-rostral movements) whisking speed decreased to 1,081 ± 322 °/s. Retraction (i.e., rostral-caudal movements) movements were significantly faster at 1,564 ± 549 °/s.

As animals approached to the target object (i.e., protraction), whisker tip velocity during free whisking in air was 0.38 ± 0.16 mm/ms. Once the whisker got in touch with the target object, the tip decelerated with a rate of 0.06 ± 0.06 (SD) mm/ms before reaching <0.05 mm/ms tip speed after 5.8 ± 4.9 mm. Average whisker deflection amplitude was 8.8 ± 5.7° and lasted for 18 ± 17 ms at the end of which whiskers accelerated with a rate of 0.07 ± 0.16 mm/s until they reached the speed of free whisking in air.

Contact with the target platform deformed the whisker without having a large influence on the base angle of the whisker (–22.25 ± 7.20° during free-whisking vs. –20.45 ± 6.65° during whisker-touch). This effect could be caused by modulation of whisking cycle (see Fig. 9) and does not allow to draw conclusions on the momentum at the whisker base.

We studied whisker deformations along the whisker shaft, expecting to see a change in curvature as the vibrissae was bent away from the platform. Our limited analysis found a small mean deformation at the tip (Fig. 11C) that corresponds well to individual observations in the recorded videos (Fig. 11A) but failed to quantify any major change of curvature at the base. These results have to be considered very cautiously: precise quantification of the effect of contact on whisker shape requires eliminating the effect of the resting curvature and of inertial deformations (see RESULTS), which requires exact physical modeling beyond the scope of this paper.

It should be noted, however, that whiskers exhibit a significantly higher stiffness at the base compared with the tip (Birdwell et al. 2007Go). This would allow a transmission of momentum with only very little observed deformation at the base if contact occurs near the tip.

A comparison between whisker tracking methods

There are two principal methods to track whisker movements in awake behaving animals with high spatiotemporal sampling frequency: optoelectronic and high-speed videographic. Optoelectronic methods require positioning of whisker(s) between light sources and linear imaging sensors to track the voltage shift along the sensors as the whisker or a marker attached to the whisker travels within the field of view. On the other hand, high-speed videographic methods enable imaging of whiskers under ambient and/or directed light while the animal enters and whisks in the field of view freely.

Although optoelectronic methods allow real-time, fully automatized tracking of a single point along the whisker shaft by following the shadow of either the whisker itself (i.e., contrast mapping) or a marker attached to the whisker (i.e., edge detection), their use is limited to head-fixed preparation and those experimental designs that require the tracking of isolated macrovibrissa (e) located caudally in the whisker pad. A previously described (Knutsen et al. 2005Go) high-speed videographic method (a.k.a. spline repositioning) overcame these limitations at the expense of increased spatiotemporal sampling rates and user intervention requirements. We described herein a novel image-processing algorithm, named "anisotropy tracing," that eliminated the constraints of the spline repositioning. Tracking whiskers with anisotropy tracing allowed us to achieve full-length, completely automated whisker tracking in freely behaving animals while ensuring that the tracing quality is independent from the sampling rate, image resolution, contrast and illumination (see Table 2 for detailed feature comparison of whisker tracking methods).


View this table:
[in this window]
[in a new window]

 
TABLE 2. Feature comparison of whisker tracking methods

 
Methodological considerations

SPEED.  Our method is currently not optimized for computational efficiency (Table 2). Assuming no whisker continuity, hence tracing each frame from scratch, necessitated a computationally expensive method and gave rise to the need for establishing whisker identity after tracing.

There are obvious improvements that could streamline the algorithm and allow processing rates of ~100 of frames/s. For example, porting time consuming routines (e.g., path integration and filtering) to a compiled programming language (see Table 2), isolating a dynamical ROI using the location information extracted from the previous frame(s), using additional GPUs or field-programmable gate arrays (FPGAs) for parallel processing should reduce the total processing time by about one order of magnitude. Moreover, integrating incremental methods (Knutsen et al. 2005Go) to the current algorithm should allow further reduction in processing times for those applications that require tracking whisker locations, rather than quantification of whisker touches, in large datasets.

SAMPLING RATE.  We sampled whisking movements using a high-speed camera at 1,000 Hz to gain accurate information about the time course of the whisker palpations and touch induced changes in the kinematics of whisker movements. However, the described method can readily be applied to movies recorded at lower sampling rates with the constraints that timing of the whisker touch, amplitude of the whisking cycle, and high-frequency component of whisking at the set points will gradually be lost as the frame rate reduced. An example of whisker tracking in movies sampled at 50 Hz can be found in Supplementary Movie 3.

TRACKING ALL WHISKERS.  For certain applications, simultaneous tracking of all facial whiskers of the subject might be desirable. With the current algorithm, we can reliably track ≤8 whiskers on each face side without losing their identity information and without coloring or otherwise marking the whiskers. Although it would be possible to achieve higher whisker counts by increasing the spatial resolution of the sensor (Supplementary Fig. 2), reaching the goal of tracing all whiskers simultaneously will require increasing the dimensionality in the data, for example, by using several cameras whose field of views partially overlap in x,y,z coordinates. Modifying the algorithm described herein will require a method to assemble partial tracing of whiskers from different perspectives and additional effort to combine multiple sets of splines into three-dimensional data.

In summary, we developed a novel image processing method that allows fully automatized off-line tracking of whisker movements and quantification of whisker touch in freely behaving animals without the need for any human supervision beyond a one time set-up calibration. This method should help not only whisker researchers but also facilitate studies that explore mechanisms of tactile information processing in brain regions outside of the classical somatosensory axis (Celikel et al. 2007Go; Pereira et al. 2007Go).


 GRANTS
 
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
Financial support for this project was in part provided by the Alexander von Humboldt foundation to T. C.


 ACKNOWLEDGMENTS
 
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
We thank K. Briggman, B. Englitz, D. H. Herman, and R. Schwarz for critical reading and comments on a previous version of the manuscript.

The most recent version of the tracking software can be freely downloaded from http://lncp.usc.edu.


 FOOTNOTES
 
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

1 The online version of this article contains supplemental data. Back

Address for reprint requests and other correspondence: T. Celikel, 3641 Watt Way, HNB 507, Los Angeles, CA 90089-2520 (E-mail: celikel{at}usc.edu)


 REFERENCES
 
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
Allen CB, Celikel T, Feldman DE. Long-term depression induced by sensory deprivation during cortical map plasticity in vivo. Nat Neurosci 6: 291–299, 2003.[CrossRef][Web of Science][Medline]

Arabzadeh E, Panzeri S, Diamond ME. Whisker vibration information carried by rat barrel cortex neurons. J Neurosci 24: 6011–6020, 2004.[Abstract/Free Full Text]

Armstrong-James M, Fox K. Spatiotemporal convergence and divergence in the rat S1 "barrel" cortex. J Comp Neurol 263: 265–281, 1987.[CrossRef][Web of Science][Medline]

Bermejo R, Houben D, Zeigler HP. Optoelectronic monitoring of individual whisker movements in rats. J Neurosci Methods 83: 89–96, 1998.[CrossRef][Web of Science][Medline]

Birdwell J, Solomon J, Thajchayapong M, Taylor M, Cheely M, Towal R, Conradt J, Hartmann M. Biomechanical models for radial distance determination by the rat vibrissal system. J Neurophysiol 98: 2439–2455, 2007.[Abstract/Free Full Text]

Carvell GE, Simons DJ. Biometric analyses of vibrissal tactile discrimination in the rat. J Neurosci 10: 2638–2648, 1990.[Abstract]

Celikel T, Marx V, Zivkovic A, Hasan MT, Licznerski P, Osten P, Rozov A, Seeburg PH, Schwarz MK. Sustained Homer1a expression impairs LTP, spatial working memory but not spatial reference memory. Front Neurosci 1: 97–110, 2007.[CrossRef]

Celikel T, Sakmann B. Sensory integration across space and in time for decision-making in the somatosensory system of rodents. Proc Natl Acad Sci USA 104: 1395–1400, 2007.[Abstract/Free Full Text]

Celikel T, Szostak VA, Feldman DE. Modulation of spike timing by sensory deprivation during induction of cortical map plasticity. Nat Neurosci 7: 534–541, 2004.[CrossRef][Web of Science][Medline]

Clem RL, Celikel T, Barth A. Ongoing in vivo experience triggers synaptic metaplasticity in the neocortex. Science 319: 101–104, 2008.[Abstract/Free Full Text]

Diamond ME, Huang W, Ebner FF. Laminar comparison of somatosensory cortical plasticity. Science 265: 1885–1888, 1994.[Abstract/Free Full Text]

Flandrin P, Auger F, Chassande-Mottin E. Time-frequency reassignment: from principles to algorithms. In: Applications in Time Frequency Signal Processing, edited by Papandreou-Suppappola A. Boca Raton, FL: CRC Press, 2003, p. 179–203.

Foeller E, Celikel T, Feldman DE. Inhibitory sharpening of receptive fields contributes to whisker map plasticity in rat somatosensory cortex. J Neurophysiol 94: 4387–4400, 2005.[Abstract/Free Full Text]

Fox K, Wong RO. A comparison of experience-dependent plasticity in the visual and somatosensory systems. Neuron 48: 465–477, 2005.[CrossRef][Web of Science][Medline]

Hentschke H, Haiss F, Schwarz C. Central signals rapidly switch tactile processing in rat barrel cortex during whisker movements. Cereb Cortex 16: 1142–1156, 2006.[Abstract/Free Full Text]

Houweling AR, Brecht M. Behavioural report of single neuron stimulation in somatosensory cortex. Nature 451: 65–68, 2008.[CrossRef][Medline]

Jacobs GH, Fenwick JC, Calderone JB, Deeb SS. Human cone pigment expressed in transgenic mice yields altered vision. J Neurosci 19: 3258–3265, 1999.[Abstract/Free Full Text]

Jin TE, Witzemann V, Brecht M. Fiber types of the intrinsic whisker muscle and whisking behavior. J Neurosci 24: 3386–3393, 2004.[Abstract/Free Full Text]

Knutsen PM, Derdikman D, Ahissar E. Tracking whisker and head movements in unrestrained behaving rodents. J Neurophysiol 93: 2294–2301, 2005.[Abstract/Free Full Text]

Larkum ME, Zhu JJ, Sakmann B. A new cellular mechanism for coupling inputs arriving at different cortical layers. Nature 398: 338–341, 1999.[CrossRef][Medline]

Mehta SB, Whitmer D, Figueroa R, Williams BA, Kleinfeld D. Active spatial perception in the vibrissa scanning sensorimotor system. PLoS Biol 5: e15, 2007.[CrossRef][Medline]

Mitchinson B, Martin CJ, Grant RA, Prescott TJ. Feedback control in active sensing: rat exploratory whisking is modulated by environmental contact. Proc R Soc Lond B Biol Sci 274: 1035–1041, 2007.[Medline]

Pereira A, Ribeiro S, Wiest M, Moore LC, Pantoja J, Lin SC, Nicolelis MA. Processing of tactile information by the hippocampus. Proc Natl Acad Sci USA 104: 18286–18291, 2007.[Abstract/Free Full Text]

Pinto DJ, Brumberg JC, Simons DJ. Circuit dynamics and coding strategies in rodent somatosensory cortex. J Neurophysiol 83: 1158–1166, 2000.[Abstract/Free Full Text]

Shulz DE, Sosnik R, Ego V, Haidarliu S, Ahissar E. A neuronal analogue of state-dependent learning. Nature 403: 549–553, 2000.[CrossRef][Medline]

Simons DJ. Temporal and spatial integration in the rat SI vibrissa cortex. J Neurophysiol 54: 615–635, 1985.[Abstract/Free Full Text]

Towal RB, Hartmann MJ. Right-left asymmetries in the whisking behavior of rats anticipate head movements. J Neurosci 26: 8838–8846, 2006.[Abstract/Free Full Text]




This article has been cited by other articles:


Home page
J. Neurophysiol.Home page
R. A. Grant, B. Mitchinson, C. W. Fox, and T. J. Prescott
Active Touch Sensing in the Rat: Anticipatory and Regulatory Control of Whisker Movements During Surface Exploration
J Neurophysiol, February 1, 2009; 101(2): 862 - 874.
[Abstract] [Full Text] [PDF]


Home page
J. Neurophysiol.Home page
M. C. Stuttgen, S. Kullmann, and C. Schwarz
Responses of Rat Trigeminal Ganglion Neurons to Longitudinal Whisker Stimulation
J Neurophysiol, October 1, 2008; 100(4): 1879 - 1884.
[Abstract] [Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Supplemental Figures and Videos
Right arrow All Versions of this Article:
100/1/504    most recent
00012.2008v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Voigts, J.
Right arrow Articles by Celikel, T.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Voigts, J.
Right arrow Articles by Celikel, T.


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
Visit Other APS Journals Online
Copyright © 2008 by the The American Physiological Society.