Understanding how whisker-based tactile information is represented in the nervous system requires quantification of sensory input and observation of neural activity during whisking and whisker touch. Chronic electrophysiological methods have long been available to study neural responses in awake and behaving animals; however, methods to quantify the sensory input on whiskers have not yet been developed. Here we describe an unsupervised algorithm to track whisker movements in high-speed video recordings and to quantify the statistics of the tactile information on whiskers in freely behaving animals during haptic object exploration. The algorithm does not require human identification of whiskers, nor does it assume the shape, location, orientation, length of whiskers, or direction of the whisker movements. The algorithm performs well on temporary loss of whisker visibility and under low-light/low-contrast conditions even with inherent anisotropic noise and non-Gaussian variability in the signal. Using this algorithm, we define the speed [protraction (P), 1,081 ± 322; retraction (R), 1,564 ± 549 °/s], duration (P, 34 ± 10; R, 24 ± 8 ms), amplitude (P = R, 40 ± 13°), and frequency (19 ± 7 Hz) of active whisking in freely behaving mice. We furthermore quantify whisker deflection induced changes in whisking kinematics and calculate the statistics (i.e., speed, amplitude and duration) of whisker touch and finally show that whisker deprivation does not alter whisking kinematics during haptic exploration.
The whisker system of rodents has emerged as a popular model to study mechanisms of sensory information processing, neural plasticity, and sensory organ representations (Allen et al. 2003; Arabzadeh et al. 2004; Armstrong-James and Fox 1987; Celikel et al. 2004; Clem et al. 2008; Diamond et al. 1994; Foeller et al. 2005; Fox and Wong 2005; Houweling and Brecht 2008; Larkum et al. 1999; Pinto et al. 2000; Shulz et al. 2000; Simons 1985). In the absence of techniques to quantify the sensory information on whiskers in behaving animals, most of the research has focused on quantification of neural responses following passive whisker deflections under anesthesia. Despite the advantages of studying sensory responses to precisely controlled passive whisker stimulation, understanding how whisker-based tactile information is processed during active tactile sensation requires simultaneous quantification of sensory input on whiskers and observation of neural activity during whisking.
Observation of whisking in freely behaving animals, especially in mice, is a challenging task. It's not only because mice whisk at higher frequencies than larger rodents like rats (Knutsen et al. 2005) and guinea pigs (Jin et al. 2004), but also the higher density and thinner structure of mice whiskers require that observation of whisking is performed using high spatial density of imaging sensors. Recent application of optoelectronic methods (Bermejo et al. 1998; Hentschke et al. 2006) and image processing tools (Knutsen et al. 2005; Mehta et al. 2007) have allowed for high spatiotemporal acquisition rates, making it possible to track whisker movements in behaving rodents. Although the latter method allows tracking the entire length of whiskers in high-speed video recordings at the expense of slower processing rates and user intervention, optoelectronic methods allow real-time, fully automatized tracking of a single location on the whisker indirectly, at high temporal rates in immobilized animals.
The ideal method for tracking whisker movements should combine the desirable features of these two methods. It should allow tracking the entire length of whiskers in freely behaving animals, even under less favorable conditions (i.e., low or no visible light, low contrast, high noise, temporary loss of whiskers of interest) without any human supervision. Here we describe such an algorithm to track multiple whiskers simultaneously in video recordings. The algorithm is fully automatized and unsupervised. It locates all whiskers in a given frame and traces the entire length of whiskers. Because tracing of whiskers does not require prior knowledge on location of whiskers in previous frames, the algorithm is suitable for tracing whiskers also at low sampling frequencies using conventional videography techniques without initial identification of whiskers by human observers.
One of the advantages of whisker tracking from video recordings is that information about the tactile exploration of target objects can be captured in subsequent images. If the entire length of whiskers can be traced, the statistics of whisker touch induced changes in whisker morphology and kinematics, including those variables that somatosensory neurons encode (e.g., whisker deflection amplitude, direction, duration of whisker touch and angular velocity during touch) can be quantified in freely behaving animals during active touch. Taking advantage of the whisker-tracking algorithm detailed herein, we also describe a method to detect whisker contacts in freely behaving mice and use this algorithm to quantify the temporal course, amplitude, speed, and angular velocity distribution of whisker palpations during object exploration.
Adult C57Bl6 mice (>P120) from either sex were used according to the animal welfare guidelines of the Max-Planck Society and the National Institute of Health. Mice were housed individually and kept on a 12-h light/dark cycle with ad libitum access to food and water. All experiments were performed during the light phase, and each animal was acclimatized to the experimental room for >1 h before behavioral experiments started. Those animals assigned to one of the whisker deprivation conditions received whisker plucking under isofluorane (Baxter) anesthesia 1 day before imaging experiments.
Spontaneous gap-crossing is similar to gap-cross training (Celikel and Sakmann 2007), with the exceptions that animals are not food deprived before the training and are not rewarded for successful task execution during the training. The apparatus was identical to the one described previously (Celikel and Sakmann 2007), except that two infrared light sources and a video camera were placed on one of the two platforms, i.e., the target platform (Fig. 1). The platforms (12 × 20 cm, width × length) were elevated 40 cm off the surface and surrounded on the three sides by 30-cm high walls. The width of each platform was reduced to 4 cm at the edge by the gap to restrict the mouse accessible portion of the target platform and allow higher optical zoom to maximize the spatial resolution of the video recordings.
The target platform was illuminated with custom-made infrared light emitting diode (LED) arrays placed on a heat sink and powered by a DC source. One of the two separate light sources was placed 14 cm below the target platform. An opal glass diffuser (Edmund Optics) was placed above the light source to create uniform backlighting at the gap (Fig. 1B). The second set of lights were placed 14 cm above the target platform with a at 45° angle to the platform surface.
Five sets of motion sensors (MSs) were distributed along the apparatus to track the animal location and control image acquisition. Two of the sensors were positioned at the two ends of each apparatus. An additional sensor was placed 2 cm before the target platform to trigger the transfer of images from the on-board camera buffer to the hard-drive. The activity detected by motion sensors were digitized (Micro 1401mkII, Cambridge Electronic Design) and acquired at 1 kHz using routines written in Spike2 (Cambridge Electronic Design).
Animals' exploratory whisking onto the target platform was tracked with a video camera acquiring images at 1,000 frames/s with a spatial resolution of 0.09 mm/pixel under white light and 0.18 mm/pixel for infrared recordings. The camera (Motionscope M1, Redlake) was fixed to the target platform and placed at a stationary position looking at the edge of the platform with a 90° angle. The camera was equipped with an on-board memory buffer (maximum 4,096 frames at 640 × 512 resolution). Image acquisition was performed in two phases (Fig. 1C): first, when the animal arrived at the edge of the home platform, MS#2 armed the camera to buffer images; then as animal crossed MS#3, image acquisition was initiated. Image buffering continued until the animal crossed MS#4 or the on-board memory buffer was full. Those trials where animals did not make a decision on whether or not to gap-cross before the memory buffer was full were excluded from the subsequent image processing and analysis.
At high frame acquisition rates, because of fast shutter speed (1 ms in this study), adequate illumination of the imaged object requires a luminous environment. We used infrared light as the light source because of impaired C57/Bl6 vision at and beyond 650 nm (Jacobs et al. 1999).
Image sequences were read off-line from hard disk and calibrated in brightness by a factor determined from a background portion of every image to equalize light intensity across images. An average of 50–100 frames, acquired before the animal entered the field of view, was subtracted, and the position of the target platform's dark colored edge was determined by finding the minimum brightness across rows of the averaged frame.
The position of the animal head was determined by tracking the nose that was assumed to be the object closest to the edge above a given brightness threshold when averaging in the x-direction (i.e., across columns). The approximate outline of the head was determined by spatial filtering that included radial (outward) thresholding from the head center and Gaussian smoothing of the resulting coordinates.
Tracing whisker-like structures
Tracing individual whiskers was accomplished frame by frame by 1) generating vector fields from each frame that resulted in a convergence of flows on whisker-like structures; 2) integrating vector fields to generate spatially contiguous traces of paths converging on whiskers; 3) grouping and averaging each converging set of paths into whisker “splines”; and 4) grouping whisker splines from individual frames into time-contiguous spatial representation of whisker positions.
GENERATING VECTOR FIELDS.
A set of images, denoted similarity index (SI) hereafter, which hold information on how much the surrounding of each pixel is invariant to directional (e.g., anisotropic) blurring and shifting (Fig. 2), was computed by correlating brightness of pixels to shifted and blurred environments of the pixel in a given direction.
The resulting angular data showed distinctive peaks on and around directed structures like whiskers (Fig. 3 B). The angle with the highest value was used to compute a vector field and showed the direction of minimum anisotropy at any given pixel in that particular frame. With our data, discretization in ∼50 steps yielded acceptable signal-to-noise (S/N) ratios. Using the contrast range and steepness of SI at a given pixel across the angle range yielded a reliable indication of the presence of whiskers. Because of the high sampling width in the filtering steps, the resulting data exhibited very low noise even when the contrast of the input frames was very low (see results).
Omission of structures of previously known orientation, such as the platform edge in our condition, could be achieved by setting areas of SI to zero in the affected spatial region and angle range (Supplementary Fig. 1).1
Moreover this method has shown promising results when used to trace full whisker arrays without whisker deprivation even though overlapping whiskers and high whisker densities cause incomplete tracing and shape artifacts (Supplementary Fig. 2). In selected frames with high sharpness, overlapping whiskers resulted in two distinguishable sets of peaks in SI at the site of intersection, suggesting that the approach could also be adapted to circumvent these problems.
An additional vector field holding the local brightness gradient was added with a coefficient of ∼0.5 to improve the convergence of flow onto the whiskers. Without this bias, the paths tended to follow the outer edges on the whisker, especially when whiskers were blurred, making grouping difficult. The scaling of this component was done so that the vector field derived from SI was always dominant.
INTEGRATING VECTOR FIELDS.
The integration of the vector field was realized using a first-order Euler-method with a step size of ∼6 pixels. Application of angle dependent weighting to SI, before generation of the vector field, gave the option to bias the flow without affecting accuracy of the peak detection. This was done to counteract the local symmetry of the whiskers by initially biasing radially outward from the head center. Jumping between different peaks in SI caused by overlapping directed structures (i.e., neighboring whiskers, background structures) was avoided by biasing in the direction of the last step. A small inertia term was used to increase the stability without compromising precision even when low curvature radii occurred during whisker deflection. The threshold for stopping integration was empirically selected to a low value, resulting in paths that were often diverging for one or two steps at the end of whiskers before stopping (Fig. 3C). This ensured that even weak signals were fully exploited and the full length of the traced whisker shaft is reconstructed. The resulting variability at the whisker tip was very low because of the number of paths averaged into each whisker and was compensated by shortening of the splines by one step. For our sequences, starting coordinates were distributed ∼12 pixels outside the head outline to avoid the fur and stubs of the clipped whiskers. However, any starting coordinates within the frame, for example, on the platform edge, would reliably pick up whiskers passing through them.
AVERAGING CONVERGING PATHS.
A number of paths, varying between 4 and 25 depending on the density of whiskers and integration starting coordinates at the base, converged onto each whisker in every frame. Individual paths were rejected based on a number of thresholds for individual or averaged pixel values, length, and curvature. Surviving paths were averaged using the nearest Euclidean approach for each point on neighboring paths. Groups containing less than four paths were rejected, and a standard least square error method was used to calculate a spline representing each whisker starting from 2.6 to 3.1 mm outside of the whisker pad. To trace the entire length of the whisker(s), the same procedure was repeated, but this time N (in this study n = 10) starting points at one third of the whisker length were used to trace whisker(s) in the inward direction (Fig. 3E). The two resulting splines were combined to a complete whisker representation (Fig. 3F).
GENERATING TIME-CONTIGUOUS SPATIAL REPRESENTATION OF WHISKERS.
After all frames of an image sequence were processed, splines originating from the same whisker in different frames were identified. To handle cases where whiskers were temporarily invisible, e.g., because of excessive up-down head movements, the data were organized into three-dimensional splines (i.e., streaks) that represent a whisker in a subsequence of frames where whisker identity could be maintained. A similarity measure, like the one used for path grouping, was computed for all whisker splines. Those whisker splines above a threshold were added to the streak with the highest similarity score. Ambiguous cases were solved by giving priority to whiskers with only one viable streak. Streaks were rejected if they were very short (<5 frames) or exhibited high-frequency (>4,000 °/s) movements well above physiologically possible values, eliminating occasional erratic tracing of hairs like supraorbital whiskers. For those splines that cannot be combined in a streak, a new streak was created. The streaks were evaluated to extract x,y,t coordinates and archived together with the nose position (Supplementary Fig. 3).
Quantification of whisking kinematics and detection of whisker touch
Raw streaks were recalled from hard disk and user-defined N (in this study, n = 40) equally distributed points (a.k.a. whisker splines) along each whisker were extracted.
Velocity orthogonal to the whisker shaft, acceleration, and the angle in respect to the base point were computed along the entire whisker splines. This gave the option of comparing data at different points along the whisker with high precision. The detection of whisker touches onto the platform was accomplished with a fuzzy-logic approach. A spatial filter combining scores for distance to the platform, speed, and acceleration for each point along the whisker was calculated and the point of the highest value above a threshold was marked as collision. In collisions where a whisker swiped the top of the platform without significant slowing or deflection, using a deceleration threshold while compensating for active whisking movement gave a somewhat usable measure for contact of large whiskers (i.e., macrovibrissae). However, the reliability of the method was not sufficient at the current stage of development. Therefore, only those whisker collisions onto the platform edge, which constituted roughly 96–98% of all palpations before gap-crossing, were classified as whisker touch. See results for a detailed performance comparison of the collision detection algorithm in respect to human encoders.
The method was implemented primarily in MATLAB (MathWorks, Natick, MA). The image filtering routines were implemented in C using the NVIDIA Compute Unified Device Architecture platform (http://developer.nvidia.com/object/cuda.html) and were executed on a NVIDIA Geforce8800 graphics-processing unit (GPU). The reassigned Gabor spectrum computations were performed using The Time-Frequency Toolbox (http://tftb.nongnu.org/).
Unbiased, unsupervised tracking of the entire length of whiskers
We developed an algorithm to track whisker movements along the shaft of whiskers. Different from the previously described image analysis method (Knutsen et al. 2005), this algorithm does not require a human observer's identification of whisker(s) or individual x,y locations along the whisker shaft to “lock onto” whiskers. Instead it locates whisker(s) in the frame using thresholded anisotropy functions. Because actual whiskers rather than a limited number of points along the whisker are tracked across frames, this approach allows a comparison of the whisking kinematics throughout the entire length of the tracked whiskers. An example of how location with respect to the whisker follicle (i.e., base) affects the kinematics of whisker movement is shown in Fig. 4.
Average frequency of whisking movements increased as the distance between the observed location on the whisker of interest and the base increased (Fig. 4C; Supplementary Movie 1). The entire whisker shaft had a principal frequency of movement at the range of 15–20 Hz varying in time (Fig. 4C), as quantified using reassigned Gabor (Flandrin et al. 2003) spectrum (see Supplementary Fig. 4 for a comparison between Fourier and Gabor transform). Not surprisingly the high-frequency component of the whisker movements were more visible at the set points of the whisking cycle (Fig. 4B). After applying a high-pass filter at the mean whisking frequency + 1 SD (26 Hz), the remaining mean of frequency frequencies of the highest power at the set points (excluding whisks containing collision events) was 36.8 ± 7.7 (SD) Hz (n = 1981), with no difference between rostral and caudal set points (36.9 ± 7.6 vs. 36.8 ± 8; P > 0.6, paired t-test). Increasing the number of whiskers available to explore the target platform did not affect the kinematics of whisker movements. As shown in Fig. 5, tracking C row whiskers bilaterally revealed that whiskers on both sides of the face had comparable whisking amplitude (41.1 ± 13.1 vs. 41.3 ± 12.2°, peak-to-peak; P = 0.65, ANOVA).
Frequency of whisking, however, occasionally varied across the two face sides similar to the findings reported for rats' whisking behavior (Mitchinson et al. 2007; Towal and Hartmann 2006). For example, data on Fig. 5C show that whiskers on both face sides had overlapping frequency spectrum whose principal frequency reduced during the first 300 ms before reaching a stationary level at ∼15 Hz for the whiskers on the right face side. Whiskers on the left, however, displayed an increased frequency of whisking during the last 200 ms, which resulted in asymmetric whisking during exploration of the target platform (Fig. 5C; Supplementary Movie 2).
One advantage of this algorithm is the possibility to track whisking movements in unfavorable conditions (e.g., under low light, with reduced signal-to-noise ratio). To study the algorithms limits for tracking whiskers under such conditions, we artificially introduced noise to a randomly selected image and tracked whiskers in this image with varying levels of Gaussian noise (Supplementary Fig. 5).
The algorithm was capable of locating the whiskers in the frame even if the signal was just ∼14% above the noise (S/N = 1.14; see Supplementary Fig. 5 caption for details), a level below/close to the detection threshold of human observers (Supplementary Fig. 5, bottom row). Expectedly, increased noise in the image resulted in reduction of the tracked whisker length, which is caused primarily by more rapid deterioration of the whisker structure at the thin tip compared with the thick base of the whisker.
Kinematics of whisker movements during tactile exploration
Using the image-processing algorithm described above, we also quantified the kinematics of whisker movements when animals used single or multiple whiskers to explore the target platform.
The full whisking cycle lasted 59.16 ± 21.30 ms, resulting in frequency of whisking 19.22 ± 7.26 Hz for those animals with multiple whiskers. Depriving all whiskers but one did not change the kinematics of whisking. Animals with single whisker whisked at 18.58 ± 5.61 Hz (P > 0.40 vs. multiwhisker, ANOVA), and their full whisking cycle was 58.77 ± 17.25 ms long (P > 0.86 vs. multiwhisker, ANOVA). The similarity between multi- and single whisker whisking conditions was not restricted to the temporal domain of whisking. Amplitude and speed distributions of the protraction and retraction were also comparable independent from the number of whiskers that the animals had during tactile exploration (Table 1).
Similar to findings on rat whisking behavior (Bermejo et al. 1998; Carvell and Simons 1990; Knutsen et al. 2005), the protraction and retraction phases of whisking had different time courses: The speed of whisking was lower during protractions and higher during retractions (Table 1; Supplementary Fig. 6). This change in the speed was independent from the amplitude of whisking because mice displayed similar amplitude of whisking during the two phases of the whisking cycle (Fig. 6, top panel).
Amplitude of whisking, however, was strongly correlated with the angular speed of whisking movement. Both peak speed of angular displacement (Fig. 6A, middle panels) and mean speed of whisking (Fig. 6A, bottom panels) were positively correlated with the amplitude of whisker movements in the two phases of the mouse whisking.
The duration of the pro- and retraction cycles were loosely correlated with the whisking speed. In protraction cycles, both mean and peak speeds were practically constant over cycle durations, with a slight decline in both very short and long cycles. In retractions, there was a negative correlation between the mean speed and duration (Fig. 6B, bottom right panel) and a weaker correlation between peak speed and duration (Fig. 6B, top right panel).
Using Gabor spectrograms, we found a negative correlation between whisking amplitude and frequency with no significant difference between pro- and retraction cycles (R = −0.43 in protractions and R = −34 in retractions; Supplementary Fig. 7).
The mean whisker angle in pro- and retraction cycles was found to be unrelated to either whisking amplitude or frequency (Supplementary Fig. 8).
Quantification of whisker palpations during tactile exploration
The second purpose of this work was to develop an objective method to describe when and where whiskers contact with a target object during tactile exploration using whiskers. We reasoned that, the point along the whisker (on a frame), where the combination of speed and acceleration of whisker movements are minimal, should be the location of whisker contact with the object being explored (i.e., target platform). (See methods for full further details on the algorithm.)
To quantify the reliability of this approach to detect whisker palpations, we compared the performance of the algorithm against two human observers. The observers had an overall error rate of 7.3%, missing whisker deflections in ∼7 of 100 frames that the other observer classified as palpation onto the target platform. Among those whisker palpations detected by both human observers, 19.6% of detected palpations were <2 pixels apart from each other. If estimates were allowed to differ ≤10 pixels, the observers agreed in 78.8% of the frames (Fig. 7 ).
The algorithm detected whisker palpations as well as human observers (92.7 vs. 92.3%), with an overall successful detection rate of 25.1 and 76.9% with spatial error rates of <2 and <10 pixels, respectively (Fig. 7). The error term was not caused by an inability of the algorithm to track whiskers reliably but by the ambiguity in collision onset time when the whisker was sliding along the edge or uncertainty in collision positioning when the whisker was in contact with and locally parallel to the edge of the platform.
Detection of whisker palpations with a target object makes it possible to quantify the statistics of whisker movements during tactile exploration. Taking advantage of this algorithm, we also calculated the duration and amplitude of whisker deflections and of velocity whisker movements during tactile exploration in freely behaving mice.
Duration of a whisker touch on average (n = 713) was 18.4 ± 17.4 ms, although the range varied between 8 and 21 ms (25–75%) with a median of 15 ms (Fig. 8). The phase relationship between the whisking cycle and the onset of whisker contacts with the target platform varied across the two face sides. Whiskers on the left side contacted the target platform at a more rostral location compared with the whiskers on the right side (76.2 ± 12.2 vs. 62.0 ± 21.0% of the protraction phase relative to the forward-most position in the whisk cycle; P < 0.001, ANOVA).
To quantify the angular displacement caused by whisker palpations (i.e., whisker deflection amplitude), we averaged the angular whisker movements at the onset of whisker deflections where angular position of whiskers is described as 1) tip from base, 2) base local, or 3) tip local (Fig. 9 A). As shown in Fig. 9B, whisker palpations onto the target platform were in phase with the basic whisking pattern and did not disturb the sinusoidal movement of the whisker base.
To quantify the whisker deformation in collisions, we calculated the difference between the local whisker angle at the tip (i.e., tip local) and the whisker angle at the base (i.e., base local; Fig. 9A). This value was centered to its mean for each whisker, resulting in a value >0 as the tip is bent away from the platform in respect to the base in collisions (Fig. 9C).
To calculate the whisker deflection amplitude, we subtracted the whisker angle at the base from the displacement (in degrees) of the tip with respect to the base (Fig. 9A). After centering the data for each whisker to eliminate the inherent whisker curvature, this calculation resulted in 0° of displacement if the whisker was approximately in the resting position.
To quantify the amplitude of whisker deformation/deflection during contact, we reverse correlated these measures to the collision onset times and found that the average whisker deformation and deflections during whisker palpations were 25.47 ± 13.84° and 8.80 ± 5.73°, respectively (Fig. 9E).
To further quantify the effect of platform contact, we compared the base whisker angle at rostral set points occurring before the first collision event and to those occurring during collision events. The average angles were −22.25 ± 7.20° for free whisking and −20.45 ± 6.65° for whisker contact (P < 0.001, Student's t-test; Fig. 10). The slightly lower mean angle during contact events can be attributed to the phase locking of whisking and collision events (see Fig. 9B).
To examine whisker deformations of the whiskers induced by platform contact that may not be reflected in changes of local whisker angle at the base or tip (see Fig. 9), we observed how the whisker deforms from its assumed resting shape during contact events. The average whisker shape in frames before the first collision event was assumed as the resting shape (it should be noted that, because of inertial deformations during whisking and the different time courses of pro- and retraction movements, there is probably some systematic error in this method). The resting shape was positioned and aligned to the whisker base, and the derivation of the actual shape was quantified via the area between the splines (Fig. 11 B). The mean derivation was found to be −26.19 ± 149.97 pixels2 (n = 8,004) in collision events and 0.61 ± 84.92 pixels2 (n = 19,428) in free whisking. See Fig. 11C for a representation of the mean derivation along the whisker length.
Speed of whisking during tactile exploration was actively modulated by whisker palpations. Whisker tips moved at the speed of 0.38 ± 0.16 mm/ms in the air, averaged across −15 to −5 ms before contact. When the tips contacted the target platform, the speed reduced with a rate of 0.06 ± 0.06 mm/ms2 (n = 698) to <0.1 mm/ms at an average of 4.7 ± 4.7 ms (n = 683) after whisker contact. The tips reached the speed of movement of <0.05 mm/ms only after 5.8 ± 4.9 ms (n = 645). See Fig. 12 for the median distribution (±CIs) of the whisker tip speed around whisker collision.
After whiskers palpated through the tactile surface, they accelerated for on average 6.8 ± 4.7 ms with a rate of 0.07 ± 0.16 mm/ms2 before reaching their steady-state speed of movement (Supplementary Fig. 6).
We developed a novel digital image-processing method to track whisker movements and calculate the statistics (e.g., duration, speed, and amplitude) of whisker palpations in freely behaving mice. The algorithm is fully automatized and unsupervised. It does not require human supervision to locate whiskers. Nor does it assume the shape, location, speed, direction, or orientation of whisker movements as the core of the algorithm relies on thresholded anisotropy functions to trace whiskers.
Using this algorithm, we showed that mice whisk at 19 ± 7 Hz with an average whisking amplitude of 40 ± 13° and whisking cycle (pro + retraction) duration of 59 ± 21 ms during tactile exploration, regardless of the number of whiskers available for object exploration. Whisking speed was actively modulated and varied as a function of the position at the whisking cycle. During protraction (i.e., caudal-rostral movements) whisking speed decreased to 1,081 ± 322 °/s. Retraction (i.e., rostral-caudal movements) movements were significantly faster at 1,564 ± 549 °/s.
As animals approached to the target object (i.e., protraction), whisker tip velocity during free whisking in air was 0.38 ± 0.16 mm/ms. Once the whisker got in touch with the target object, the tip decelerated with a rate of 0.06 ± 0.06 (SD) mm/ms before reaching <0.05 mm/ms tip speed after 5.8 ± 4.9 mm. Average whisker deflection amplitude was 8.8 ± 5.7° and lasted for 18 ± 17 ms at the end of which whiskers accelerated with a rate of 0.07 ± 0.16 mm/s until they reached the speed of free whisking in air.
Contact with the target platform deformed the whisker without having a large influence on the base angle of the whisker (−22.25 ± 7.20° during free-whisking vs. −20.45 ± 6.65° during whisker-touch). This effect could be caused by modulation of whisking cycle (see Fig. 9) and does not allow to draw conclusions on the momentum at the whisker base.
We studied whisker deformations along the whisker shaft, expecting to see a change in curvature as the vibrissae was bent away from the platform. Our limited analysis found a small mean deformation at the tip (Fig. 11C) that corresponds well to individual observations in the recorded videos (Fig. 11A) but failed to quantify any major change of curvature at the base. These results have to be considered very cautiously: precise quantification of the effect of contact on whisker shape requires eliminating the effect of the resting curvature and of inertial deformations (see results), which requires exact physical modeling beyond the scope of this paper.
It should be noted, however, that whiskers exhibit a significantly higher stiffness at the base compared with the tip (Birdwell et al. 2007). This would allow a transmission of momentum with only very little observed deformation at the base if contact occurs near the tip.
A comparison between whisker tracking methods
There are two principal methods to track whisker movements in awake behaving animals with high spatiotemporal sampling frequency: optoelectronic and high-speed videographic. Optoelectronic methods require positioning of whisker(s) between light sources and linear imaging sensors to track the voltage shift along the sensors as the whisker or a marker attached to the whisker travels within the field of view. On the other hand, high-speed videographic methods enable imaging of whiskers under ambient and/or directed light while the animal enters and whisks in the field of view freely.
Although optoelectronic methods allow real-time, fully automatized tracking of a single point along the whisker shaft by following the shadow of either the whisker itself (i.e., contrast mapping) or a marker attached to the whisker (i.e., edge detection), their use is limited to head-fixed preparation and those experimental designs that require the tracking of isolated macrovibrissa (e) located caudally in the whisker pad. A previously described (Knutsen et al. 2005) high-speed videographic method (a.k.a. spline repositioning) overcame these limitations at the expense of increased spatiotemporal sampling rates and user intervention requirements. We described herein a novel image-processing algorithm, named “anisotropy tracing,” that eliminated the constraints of the spline repositioning. Tracking whiskers with anisotropy tracing allowed us to achieve full-length, completely automated whisker tracking in freely behaving animals while ensuring that the tracing quality is independent from the sampling rate, image resolution, contrast and illumination (see Table 2 for detailed feature comparison of whisker tracking methods).
Our method is currently not optimized for computational efficiency (Table 2). Assuming no whisker continuity, hence tracing each frame from scratch, necessitated a computationally expensive method and gave rise to the need for establishing whisker identity after tracing.
There are obvious improvements that could streamline the algorithm and allow processing rates of ∼100 of frames/s. For example, porting time consuming routines (e.g., path integration and filtering) to a compiled programming language (see Table 2), isolating a dynamical ROI using the location information extracted from the previous frame(s), using additional GPUs or field-programmable gate arrays (FPGAs) for parallel processing should reduce the total processing time by about one order of magnitude. Moreover, integrating incremental methods (Knutsen et al. 2005) to the current algorithm should allow further reduction in processing times for those applications that require tracking whisker locations, rather than quantification of whisker touches, in large datasets.
We sampled whisking movements using a high-speed camera at 1,000 Hz to gain accurate information about the time course of the whisker palpations and touch induced changes in the kinematics of whisker movements. However, the described method can readily be applied to movies recorded at lower sampling rates with the constraints that timing of the whisker touch, amplitude of the whisking cycle, and high-frequency component of whisking at the set points will gradually be lost as the frame rate reduced. An example of whisker tracking in movies sampled at 50 Hz can be found in Supplementary Movie 3.
TRACKING ALL WHISKERS.
For certain applications, simultaneous tracking of all facial whiskers of the subject might be desirable. With the current algorithm, we can reliably track ≤8 whiskers on each face side without losing their identity information and without coloring or otherwise marking the whiskers. Although it would be possible to achieve higher whisker counts by increasing the spatial resolution of the sensor (Supplementary Fig. 2), reaching the goal of tracing all whiskers simultaneously will require increasing the dimensionality in the data, for example, by using several cameras whose field of views partially overlap in x,y,z coordinates. Modifying the algorithm described herein will require a method to assemble partial tracing of whiskers from different perspectives and additional effort to combine multiple sets of splines into three-dimensional data.
In summary, we developed a novel image processing method that allows fully automatized off-line tracking of whisker movements and quantification of whisker touch in freely behaving animals without the need for any human supervision beyond a one time set-up calibration. This method should help not only whisker researchers but also facilitate studies that explore mechanisms of tactile information processing in brain regions outside of the classical somatosensory axis (Celikel et al. 2007; Pereira et al. 2007).
Financial support for this project was in part provided by the Alexander von Humboldt foundation to T. C.
We thank K. Briggman, B. Englitz, D. H. Herman, and R. Schwarz for critical reading and comments on a previous version of the manuscript.
The most recent version of the tracking software can be freely downloaded from http://lncp.usc.edu.
↵1 The online version of this article contains supplemental data.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2008 by the American Physiological Society