This paper describes an automatic system for the analysis and classification of leech behavior. Three colored beads were attached to the dorsal side of a free moving or pinned leech, and color CCD camera images were taken of the animal. The leech was restrained to moving in a small tank or petri dish, where the water level can be varied. An automatic system based on color processing tracked the colored beads over time, allowing real-time monitoring of the leech motion for several hours. At the end of each experimental session, six time series (2 for each bead) describing the leech body motion were obtained. A statistical analysis based on the speed and frequency content of bead motion indicated the existence of several stereotypical patterns of motion, corresponding to different leech behaviors. The identified patterns corresponded to swimming, pseudo-swimming, crawling, exploratory behavior, stationary states, abrupt movements, and combinations of these behaviors. The automatic characterization of leech behavior demonstrated here represents an important step toward understanding leech behavior and its properties. This method can be used to characterize the behavior of other invertebrates and also for some small vertebrates.
Behavior is the link between organisms and environment and between the nervous system and the ecosystem. Relating behavior to neuronal activity is a central aim of system neuroscience and a necessary step toward a full understanding of how the nervous system works (Averbeck and Lee 2003; Bucher et al. 2003; Prut and Perlmutter 2003a, b). The integration of animal behavior and the neurosciences provides important frameworks for hypothesizing neural mechanisms. Consequently, data acquisition is a crucial step to obtain an accurate quantitative characterization of behavior of the animal under investigation and of its reaction to the surrounding world (Reichardt 1961, 1965; Sommer and Wehner 2004; Wehner 2003). For this purpose, automatic data acquisition, particularly marker-based approaches, have been widely used in different ways. Some systems follow the position of the animal using ultrasound (Akaka and Houck 1980) and microwave Doppler radar (Martin and Unwin 1980), but these are high-cost techniques and can only examine gross motor behaviors. Some other attempts have been done with digitizing video images and then using this digital video as the input for object detection. This technique has the disadvantage that the size of the digital file increases quickly and, as a consequence, the duration of the experiment is limited to minutes (Baek et al. 2002; Hoy et al. 1996, 1997; McIver and Nelson 2000; Zakotnik et al. 2004). In addition, the processing of stored images should be done off-line. To avoid these problems, the tracking of a simple marker, natural or attached to the body of the animal, has been implemented. Natural markers are used only when a small tissue is under observation. Recent works explored this possibility to obtain a precise characterization of muscle contraction of the leech skin (Zoccolan et al. 2001; Zoccolan and Torre 2002). But this approach requires an off-line processing of the whole image and therefore is computationally rather expensive. When the organism of interest is small, natural color-based tracking combined with movement-based tracking, as in the case of ants (Balch et al. 2001), could be used, but in this case, if the animal remains motionless for long time, it cannot be tracked. On the other hand, markers attached to the analyzed body have been tracked. In the case of the fly, a marker placed on the fly's forehead is video-recorded (Stange and Hengstenberg 1996), and the position and orientation of the marker is extracted from the video image. This and some other similar approaches (Chrásková et al. 1999; Kruk 1997) used only one marker and therefore were restricted to the analysis of a single stereotyped behavior or to the study of a particular part of the body.
As an attempt to avoid all these problems, a multiple marker approach to quantify the locomotion patterns of a Hirudo medicinalis leech is proposed in this paper. We designed and automated tracking system capable of following the motion of a leech in real-time and for long periods (several hours) at a sampling rate ≤10 Hz. At the end of the experimental session, the system provided the position of each bead during each moment of the whole experiment. We also implemented an off-line analysis method, based on the speed and frequency content of each bead, to analyze the obtained time series. This statistical analysis allowed the identification of a variety of different leech behaviors and the determination of accurate quantitative features, such as elongation, trajectories, and speed, that cannot easily be measured. The generality of our method was tested by using free and restrained leeches, obtaining similar results. We concluded that this automatic quantitative parametrization of tracking movements could be useful not only for the leech behavior but also for some small animals, providing an important step toward the understanding of the neural mechanisms underlying animal behavior.
Following the introduction, this paper is organized into the following sections. In experimental set-up, accuracy, sensitivity, and noise of the system are presented. classification of different behaviors shows the statistical analysis of the position and speed of the beads attached to the leech during different behaviors. identification of different behaviors describes the procedure for classifying and determining the ending points (onset and termination) of a given motion to obtain the final identification of leech behaviors. evaluation of the automatic system attempts an evaluation of the proposed automatic behavior classification. Finally, in discussion, advantages, limitations, and possible extensions of the system are evaluated.
Animals and preparation
All the experiments were performed on intact leeches Hirudo medicinalis, obtained from Ricarimpex (Eysines, France) and kept at 5°C in water dechlorinated by aeration for 24 h. The animals were anesthetized in this water by adding 8% ethanol. Three plastic square beads (∼2 mm diam) were attached to the dorsal side of the body with Vetbond super glue (WPI, Sarasota, FL), one of them near to the head (red), one in the midbody (green), and the third (blue) near the tail (see Fig. 1B). We started to monitor the behavior of the leech only when the animal was swimming and crawling normally, indicating full recovering from anesthesia (usually after 45 min).
Two different preparations were used: one in which the leech was completely free to move and another in which the leech was restrained by carefully inserting a pin through its rear sucker into the middle of a silicone elastomer (Sylgard)-coated dish. This restraining procedure did not produce any damage to the leech neuromuscular system and allowed the full rotation of the animal around the pin. Moreover, the pinned leeches were still able to use their rear sucker properly to stick on the bottom of the petri dish. Therefore in this restrained configuration, the leech body was very close to its natural posture during exploratory behavior, when the animal moves its anterior part back and forth while maintaining the rear sucker stuck on some support (Gray et al. 1938). When the behavior of a free-moving leech had to be characterized, the animal was moved to a tank (Fig. 1B). The level of water in the tank was changed to evoke preferentially swimming or crawling (Esch et al. 2002).
Tracking colored beads
One color CCD camera with 640 × 480-pixel image size (Watec 231S) viewed the moving leech (Fig. 1A) from above in a petri dish or small tank (Fig. 1B). The camera was mounted on a dissecting microscope and connected via the S-Video-output (PAL) to a frame grabber (PCI-1411, National Instruments), which was installed on a personal computer. The colored beads placed on the back of the leech were tracked at 10 Hz by using software developed in our laboratory with LabVIEW 6.1 (National Instruments). This software took advantage of the ability of the frame grabber to acquire images directly in the HSL (hue, saturation, lightness) color space. The tracking algorithm was designed to work in real time with no frequency constraints (i.e., each frame was completely processed before the next was fed from the frame grabber). A standard PC (AMD Athlon1800+, 384 MB RAM DDR 266 kHz) was used in the experiments and was able to sample images and processed colored beads up to a frequency of 10 Hz. Preliminary results indicate that with a faster PC (P4 2.8 GHz FSB-800 kHz with Hyper Threading disabled, 2×512 MB RAM DDR 400 kHz double channel) tracking could be obtained at around 20 Hz. A user-customized color matching was performed to convert each acquired frame into three binary images, one for each bead, in which an on bit represents a pixel in the corresponding color subspace. Sometimes, the leech twisted its body in such a way that one or more beads disappeared from the view of the camera. To detect these events, each binary map was then validated: if the number of matching pixels was less than eight, then the bead was reported as lost and the program recorded (0, 0) coordinates.
If a binary map was found to be valid, then different strategies were applied to calculate the center position of the bead, depending on the topological distribution of the positive pixels. If all the pixels were restricted inside a square of approximately the size of the bead (30 × 30 pixel), then the simple center of gravity was computed. On the other hand, if the positive pixels were spread around a wider area, the program searched for the 6 × 6-pixel region with the highest density of recognized pixels and returned the coordinates of its center. Finally, if in this last case more subzones all shared the same density of recognized pixels, the mean coordinates of the equivalent areas were recorded. Note that this software did not require saving all the processed images but just the coordinates of the selected beads. In this way, it was possible to monitor the leech behavior for several hours without filling hard disk space.
The position of the beads on the image plane was acquired as Cartesian coordinates and the following six time series were obtained (1) where every step corresponds to 100 ms. Figure 1C shows the x displacement of the three beads while the leech was exploring the environment (top) with the tail (blue trace) attached to the bottom of the petri dish and the head moving around (red trace). When the leech was crawling (Fig. 1C, bottom), a succession of elongations and contractions were observed. When the leech was restrained to move around the pin inserted in its tail, the natural coordinate system to study its motion was a polar system centered on the position of the pin. Therefore each pair of Cartesian coordinates (x, y)color was transformed into a pair of planar polar coordinates (ρ, θ)color. Usually the pin was inserted approximately in the center of the dish, and its location used as the origin of the coordinate system (2) (3) (4) The polar system has a clear geometrical and physiological meaning: ρcolor was the elongation of the leech body at the position marked by the bead (color), whereas θcolor was the angle between this position and a reference axis.
A sampling frequency of 10 Hz was sufficient to track the motion of a leech during swimming, as its frequency of oscillation—during this behavior—was between 1 and 2 Hz. In some cases, the behavior of the leech was video-recorded for further analysis.
The overall noise of recording system was measured by observing, for several hours, the apparent motion of the three beads attached to a black paper with a size similar to that of the leech. Figure 1D illustrates the distribution histograms of the position of the three colored beads in the absence of any active motion. The SD of the position of the x and y coordinates, i.e., the width of the histograms of Fig. 1D, was <1 pixel, indicating a precise and stable bead localization. Simultaneous precise localization of the three beads required appropriate setting of the illuminating light level and of the parameters used for color detection.
CLASSIFICATION OF DIFFERENT BEHAVIORS
Qualitative observation of leech behavior suggested the existence of a restricted number of stereotyped behaviors, such as swimming (Brodfuehrer and Thorogood 2001; Cang and Friesen 2002; Friesen et al. 1976; Stent et al. 1978) and crawling (Baader 1997; Eisenhart et al. 2000; Stern-Tomlinson et al. 1986). Our goal was to provide a quantitative classification of leech behavior based on the statistics of the leech body dynamics that could be compared with the classification usually obtained by visual inspection of the animal. Once the existence of a consistent matching between such a statistical classification of leech behavior and ethological observation was proved, an automatic classification of leech behavior over several hours could be obtained. In this section, it will be shown that by analyzing in time the trajectories of the colored beads attached to the leech, stationary, periodic, and exploratory behaviors could be identified. In addition to these prototypical behaviors, a number of other states were identified, such as peristaltic motion, abrupt movements, and some unclassified states. In Exploratory states, it will be shown how all these behaviors could be precisely identified and their endpoints determined.
This section shows how the statistics of the speed of the head (Vhead) and of the midbody (Vmidbody) were analyzed to identify stationary states. The speed Vhead/midbody was obtained in the following way: 1) The time series [x(n),y(n)]head and [x(n),y(n)]midbody were numerically convolved with the time derivative te(−t2/2σ2) of a Gaussian function (Oppenheim and Schafer 1989), with σ equal to zΔt, where Δt is the sampling window of 100 ms and z is a positive integer. 2) From these numerical time derivatives, dxhead/dt, dyhead/dt, dxmidbody/dt, and dymidbody/dt of the original time series, the instantaneous values of Vhead and Vmidbody were computed as (5) (6) Convolving a discrete function with a Gaussian function gives a smoothed approximation of the original data, where the measure of “smoothing” is regulated by the size of the SD. Convolving a function with the derivative of the Gaussian function is equivalent to evaluate the derivative of the smoothed function. The choice of the size (i.e., of the SD) of the convolution function is critical: a small size will result in small signal-to-noise ratio, whereas a large size will give an averaged value of the speed, losing local information. Figure 2, A–C, illustrates the distribution plots of (Vhead, Vmidbody) obtained from an experimental recording lasting 6 h obtained by convolution of the time series with the time derivative of a Gaussian filter with a value of σ equal to 0.5, 1, and 1.5 s, respectively. In this figure and in all similar plots obtained from recordings >2 h, a large isolated peak centered on the origin was observed.
This peak corresponded to a behavioral state in which both the head and the midbody of the leech were motionless and was present with all tested values of z varying from 5 to 25 (i.e., σ ranging from 0.5 to 2.5 s). This means that the identification of stationary states based on finding points in the time series with Vhead and Vmidbody near zero was robust against large variation of the width of the Gaussian used to compute the numerical time derivative of the head and midbody displacement. When a small value of z was used a smaller peak centered at a head speed of 25 pixel/s was observed (Fig. 2A, circled dark-gray peak). As will be shown, this peak corresponded to the characteristic speed of the swimming, but the final process used to identify swimming is explained in the next section. Figure 2A also shows a relevant fraction of events with a value of Vhead equal to 0 but with a nonzero value of Vmidbody. As discussed in a later section, this reflected a common pattern in which the leech moved its midbody, whereas its head and tail suckers were attached to the bottom of the recording chamber.
As the SD of noise (see Tracking colored beads) of an immobile bead was ∼1 pixel, a point in the time series was analyzed as stationary when Vhead was <1 pixel/s. When Vhead and Vtail were both <1 pixel/s, the leech was completely at rest. These points, indicated in black in Fig. 2B, were identified (i.e., matched) in the original time series and labeled as still points (see Figs. 4–6). The identification of these points represented the first step of the automatic classification.
Periodic states (swimming and crawling)
As a second step, periodic states were identified through the analysis of the spectral content of the time series. The analysis started by computing the total elongation e(n) of the leech (see the horizontal line joining points H and T in Fig. 3A) defined as (7) Elongation oscillated during swimming and crawling (at different frequencies) and varied noticeably in an irregular way during exploratory behavior. Then, for each point n of the resulting time series e(n), the instantaneous speed defined in the previous section was tested. If Vhead(n) < 1 pixel/s, the point was already classified as stationary and no further analysis was needed. If the speed was above this critical value, a time window centered on the point n, having a size T ranging from 20 s (for the high-frequency detection) to 50 s (for the low-frequency detection) was considered. In this time window, three quantities were computed. 1) The maximal speed of the head defined as (8) where Δ is T times the sampling rate, i.e., 10 Hz. 2) The power spectrum PSnT(f) of e(n) (Fig. 3B, left), where f was the frequency varying over the range 0.3–5 Hz (for T = 20 s) and 0.12–5 Hz (for T = 50 s). In both cases, the lowest points of the power spectrum were discarded as unreliable. Power spectra PSnT(f) were computed using the Welch's averaged, modified periodogram method (pwelch function, Matlab), with 50% overlap between successive Hamming-windowed segments, with a size equal to T/4. The Welch method splits a set of data into smaller sets and calculates the periodogram of each small set. Then the frequency domain coefficients arising from calculating the periodograms are averaged over the frequency components of each data set. This results in a power spectrum that is a smoothed version of the original but with less noise. 3) The smoothed power spectrum 〈PSnT(f)〉, computed using a semi-octave centered on each f (as a spectral window) and defined as (9) (Fig. 3B, right).
The power spectrum of the recorded time series was characterized by a 1/fn noise, which was a typical component of the power spectra of no stationary stochastic processes (Kantz and Schreiber 1997), with n assuming values between 3/2 and 2. By smoothing such a power spectrum over a frequency-dependent window, noise was removed and the dominant frequencies of the signal were enhanced. Smoothing the power spectrum over an octave or a fraction of an octave is a common practice in acoustics (Miller et al. 2002).
The frequency fTd(n) for which 〈PSnT(f)〉 was maximum was taken as the dominant frequency for the point n in a window of length T. In this way, two quantities (VTmax(n),fTd(n)) were associated to every point n of the time series. Both VTmax(n) and fTd(n) depended on the size of the time window T: with larger time windows, the computation of PSnT(f) was more reliable but the time localization of the behavioral state was less precise. In fact, by using large windows, sequences of short-lived states could be considered as a single state. Therefore a time window T of only 20 s was used initially to localize precisely fast and short oscillations.
Figure 3B shows PSnT(f) (left) and 〈PSnT(f)〉 (right) of two different intervals of the time series, one with f20d(n) around 1.5 Hz (black line) and the other with f20d(n) around 0.3 Hz (gray line).
The occurrences of the pair (V20max(n),f20d(n)) were represented in a three-dimensional histogram. These values were not homogeneously distributed: two well-resolved clusters or peaks were observed—with different heights but always in similar positions—in all 30 examined leeches, indicating the existence of two different behavioral states.
Figure 3C illustrates a three dimensional histogram of the pair (V20max(n), f20d(n)) for a recording lasting 1 h, where two clusters corresponding to these frequencies were clearly distinguishable.
A Gaussian fit (Fig. 3D, dotted line) of the occurrences with frequency >0.5 Hz was used to determine the center (μf) and the width (2σf) of the high-frequency cluster (Fig. 3C, black peak) in the frequency domain. Lower frequencies were excluded on the assumption they probably belonged to the separate low frequency cluster. In Fig. 3D (left), the distribution of frequencies was represented (frequencies belonging to the black cluster). A second Gaussian fit was performed for the occurrences with speed corresponding to the domain of frequencies belonging to the cluster, thus determining μV and 2σV in the velocity domain. For the high-frequency cluster shown in Fig. 3C, the obtained values were μV = 20 pixel/s, μf = 1.5 Hz, 2σV = 10 pixel, and 2σf = 0.2 Hz, but both position and width were consistently similar for all experiments with free leeches (see Fig. 8B). The distribution of speeds belonging to the black cluster was represented in Fig. 3D (right). As a final step, all the pairs (V20max(n), f20d(n)) having a speed (10) and a frequency (11) were considered as belonging to the cluster centered in (μV, μf).
The state characterized by the f20d(n) around 1.5 Hz can be clearly identified as swimming behavior. To verify that these events correspond to swimming occurrences, the videotape of the leech motion was analyzed visually to confirm that the events n located in the cluster around f20d(n) = 1.5 Hz and V20max(n) = 20 pixel/s belonged always to swimming episodes and that every swimming episode belonged to this cluster. Swimming behavior, which involved a periodic movement of the whole body, could be identified also by analyzing the distance P(x(n), y(n))midbody of the bead glued on the midbody from the line joining the head to the tail bead (see shortest line in Fig. 3A). During swimming episodes, P(x(n), y(n))midbody also oscillated with a frequency of 1.45 ± 0.15 (SD) Hz.
The states identified as swimming behavior through the frequency analysis of P(x(n), y(n))midbody largely overlapped those identified analyzing the elongation (data not shown). The latter was nevertheless a more sensitive parameter because it varied considerably during crawling and exploratory motion, and therefore the classification was performed from the elongation time series.
For a free leech, all the points in the high-frequency cluster (which correspond in the histogram of Fig. 3C to the points located within 0.2 Hz from the dominant frequency f20d(n) of 1.5 Hz and with speeds from 8 to 30 pixel/s) were taken as belonging to swimming events. Indeed the period of a swimming movement was ∼0.6 s as reported before (Muller et al. 1981).
In the experiments in which the leech was restrained, the algorithm for the identification of periodic motion was applied only to the ρ(n)head time series because this variable corresponded to the elongation e(n) of the leech. In the corresponding three-dimensional histograms of the pairs (V20max(n),f20d(n)), two distinct clusters were observed centered on 1 and 0.3 Hz, indicating the presence of swimming also in the pinned leech (1 Hz) in addition to exploratory motion (0.3 Hz) but at a reduced frequency. Then, for a restrained leech, all the points located within the high-frequency cluster, defined with the same procedures used for the free leech, were classified as swimming episodes.
The cluster at the low-frequency end of the histogram of Fig. 3C was not well resolved, especially at lower frequencies. Therefore to obtain a better resolution at lower frequencies, the time series e(n) was reanalyzed using a larger time window T of 50 s. The resulting histogram of the pairs (V50max(n), f50d(n)) is shown in Fig. 3E. In this case, a well-resolved cluster centered on 0.2 Hz (in black) with 2σf = 0.04 Hz and a remaining cluster (in dark gray) still centered at the lowest resolved frequencies (near 0.1 Hz), were observed (Fig. 3, E and F, left). The state corresponding to f50d(n) around 0.2 Hz can be identified with the crawling behavior as confirmed by inspection of the videotape of the leech motion. Indeed the period of a crawling step ranges from 3 to >10 s (Eisenhart et al. 2000; Stern-Tomlison et al. 1986).
In the restrained leech, it was impossible to observe crawling because this behavior involves lifting of the tail. Even if some crawling attempts (elongation and contraction alternating with a frequency close to 0.2 Hz) were observed, no behavior of the restrained leech was classified as a crawling state.
The state corresponding to f50d(n) around 0.1 Hz could be associated with the irregular motion observed during exploratory behavior (see next section). The two identified clusters were centered on approximately the same speed of ∼30 pixel/s (see Fig. 3F, right). Points in the histograms of Fig. 3, E and F, in black and dark gray were identified as crawling and exploratory states, respectively.
Time windows centered on behavioral end points of periodic states also contained nonperiodic components. As a consequence, the power spectrum that was computed over the entire window often did not show a clear peak, and these points could not be classified as periodic. The classification of periodic movements was therefore completed with a prolongation procedure (see Prolongation of periodic behaviors beyond the well-identified window), necessary for an accurate determination of the onset and termination of the behavior.
SWIMMING AND PSEUDOSWIMMING.
The identification of the swimming state was based on the analysis of the elongation e(n), i.e., on the head to tail distance. As a consequence, the proposed identification procedure did not distinguish whether the entire leech was moving on the dish surface (swimming) or whether e(n) oscillated but the leech had one sucker attached to the bottom of the dish. Figure 4, A and B, showed two time series of e(n) obtained from two different free leeches. Both had a clear oscillatory component at ∼1.5 Hz, the typical frequency of swimming.
When the trajectories of the leech head and tail, i.e., [x(n)head, y(n)head] and [x(n)tail, y(n)tail] were plotted on the (x, y) plane, it appeared that the time series of Fig. 4, A and B, corresponded to two different behaviors. In one case (Fig. 4, A and C), the leech was swimming around the dish, but in the other case (Fig. 4, B and D), the tail sucker was fixed to the bottom of the dish and the head was oscillating with a frequency of ∼1.5 Hz. This second behavior was referred to as pseudo-swimming, characterized by an oscillation of the elongation e(n) with a frequency identical to that observed during swimming, although with smaller amplitude and with the rear sucker attached to the bottom of the dish. The amplitude of the swimming movements of a free leech corresponded to approximately the 15–20% of the length of the full-extended leech, matching with previous evaluations (Stent et al. 1978). During pseudo-swimming, the average amplitude value was only the 10% of the length (data not shown). These two states were distinguished easily by analyzing V10tail(n) defined as where Δ was 10 times the sampling rate. The states with V10tail(n) < 3 pixel/s were classified as pseudoswimming states because the tail was not moving quickly enough to allow real swimming. They can probably be associated with ventilation behavior, previously reported by some authors (Magni and Pellegrino 1978). Note that in the case of restrained leeches, swimming and pseudoswimming cannot be dissociated because the rear sucker of the animal was kept stationary by the pinning. In this case, the amplitude of the swimming movements was again the 10% of the full length of the leech. Moreover, as previously mentioned, the characteristic frequency of oscillation of pinned leeches during such swimming or swimming-like motion is slightly lower (∼1 Hz) than for free leeches.
Events belonging to the low-frequency cluster (Fig. 3, E and F, dark gray) in the histogram of the pairs (V50max(n),f50d(n)) were identified as exploratory behavior. The videotape also confirmed this observation. In fact, the events with f50d(n) at the lowest resolved frequencies correctly matched with the exploratory movements. These were irregular oscillations of the head and the anterior part of the body, while the rear sucker, which was stuck on the bottom of the recording chamber, kept the tail motionless. The power spectrum of time intervals containing substantial amounts of exploratory motion was not associated with sharp peaks of PS(n) at any frequency (Fig. 2B).
When the front sucker of the leech was attached to the dish (Vhead close to 0), the midbody usually was motionless (the peak in (0,0) of Fig. 2). However, in some cases, it was also observed to be moving (when Vmidbody assumed values >0, see Fig. 2).
As the assumed threshold for a point to be stationary was <1 pixel/s, points of the time series in which Vhead was smaller and Vmidbody was larger than this threshold, and points in which they were both under threshold were classified as distinct states: head-attached states in the first condition and still in the second. Figure 5 shows the analysis of these states. In this experiment, the leech was pinned (its tail was fixed to the dish) and the ρ coordinates of the head and midbody were analyzed.
At the beginning of the trace in Fig. 5, a small episode of exploratory behavior is shown. Sometimes, it was observed (in 5/30 experiments) that during a stationary state (the front and the rear sucker were attached to the dish), the midbody had low-frequency oscillations similar to peristalsis. The moving midbody was analyzed and a low periodic motion about 0.03 Hz was identified (see Fig. 5). A peristaltic state is usually composed of no more than 5 slow oscillations (i.e., not lasting >3 min), so this motion was not identified with Fourier analysis but by smoothing the time series (to eliminate high-frequency components) and finding consecutive maxima with <60% of relative difference in the period.
Despite the low frequency of the oscillations, this state was referred as “peristalsis-like” movement because the visual analysis of the videotape showed the same rostral-to-caudal and caudal-to-rostral waves reported for peristalsis (Lent et al. 1988) where the frequency had been reported to be ∼0.16 Hz. The whole duration of the peristalsis-like movement in each experiment (lasting 12 h each) ranged from 0.5 to 12% of the total time and from 1 to 15% of the stationary time. The peristalsis was present only in experiments where the stationary state was prevalent (>50% of the time).
Stationary points of the time series where Vmidbody was neither negligible nor periodic were referred as “head-attached” states (see Fig. 5). Stationary points of the time series where Vmidbody was negligible were referred as “still” states (see Fig. 5).
In 1 case of 30 leeches observed, a double periodic motion was found with frequencies of ∼0.02 and 0.7 Hz. As this motion was observed only in one leech, it was not considered as a typical behavior (data not shown).
The last identified behavior, characterized by a fast transition during which the leech changed its location on the dish was referred as “abrupt movement.” To be classified as belonging to this group, transitions must be of ≥20 pixel and last <5 s. In Fig. 6A, abrupt movements indicated in gray, occur between different still states.
Previous criteria were not enough to classify every movement of the leech; therefore in the time series, there were some unclassified intervals. Some of them were eventually classified as belonging to one of the behaviors mentioned in the preceding text through a further analysis procedure (see Prolongation of periodic behaviors beyond the well-identified window and Assembling of identical states and elimination of short states) while the rest was left unclassified. Examples of behaviors—extending over several seconds—that were not classified are shown in Fig. 6, B and C. These behaviors were often characterized by a slow drift (see Fig. 6B) or by oscillations superimposed on an apparently exploratory event (see Fig. 6C). In this last case, there is no clear dominant frequency. As will be discussed later, these unclassified states represented usually <10% of the total leech behavior.
IDENTIFICATION OF DIFFERENT BEHAVIORS
In the previous, different states or behaviors were identified from the statistical analysis of the time series. Points belonging to these states could be identified from the plots shown in Figs. 2 and 3: points falling within 2σ from the identified peaks of Figs. 2B and 3, C and E, were unequivocally identified as part of a still, swimming, crawling, and exploratory behavior.
These well-identified points were matched into the original time series (see Figs. 5, 6A, and 7, A and D). After the matching of a sequence of consecutive points, all identified as belonging to the same behavioral state represented a well-identified behavioral window in the original time series. However, onset and termination of these windows were not precisely determined by the analysis described in the previous section. Indeed, a reliable computation of the power spectrum requires a large time window T (≤50 s). Thus for end points, the entire interval with different dominant frequency was included in the same window. As a result, no clear peaks appeared in the corresponding power spectrum, and a correct classification was not possible.
To obtain an exhaustive and full classification of all points n of the time series we adopted the following strategy: 1) matching of well-identified points (see Fig. 7, A and D); 2) prolongation of periodic points beyond the well-identified window (see Fig. 7, B and E); and 3) assembling of close identical states and elimination of short isolated states (see Fig. 7, C and E).
Matching of well-identified points
The statistical analysis illustrated in classification of different behaviors identified six kinds of states.
1) Stationary states with a value of Vhead(n) < 1 pixel/s, subdivided into “still” when Vmidbody(n) was also < 1 pixel/s, “peristaltic-like” when Vmidbody(n) was >1 pixel/s and the midbody was regularly oscillating at a frequency of ∼0.03 Hz, and “head-attached” when Vmidbody(n) was >1 pixel/s and the midbody was moving with no regular oscillations.
2) Swimming states with a value of V20max(n) between 10 and 30 pixel/s, a value of f20d(n) between 1.3 and 1.7 Hz, and a value of V10tail(n) >3 pixel/s.
3) Pseudo-swimming states with a value of V20max(n) between 5 and 15 pixel/s, a value of f20d(n) between 1.3 and 1.7 Hz, and a value of V10tail(n) <3 pixel/s.
4) Crawling states with a value of V50max(n) between 20 and 40 pixel/s and a value of f50d(n) between 0.16 and 0.24 Hz.
5) Exploratory states with a value of V50max(n) between 10 and 50 pixel/s and a value of f50d(n) <0.12 Hz.
6) Abrupt movements, i.e., rapid transitions of ≥20 pixels between previously classified states occurring in <5 s.
These points, identified in the histograms of Figs. 2 and 3, were matched into the original time series. Figure 7, A and D, illustrates the leech elongation, i.e., e(n), over a time window of ∼5 min: swimming, still, and exploratory states were indicated in gray, yellow, and blue, respectively. White indicated unclassified behaviors. It was evident that these well-identified windows captures the central portion of corresponding behaviors but cannot properly identify the onset and termination of the behavioral state. This was particularly evident for swimming, crawling, and exploratory states, which were identified by computing the power spectrum and its smoothed version in time windows of 20 and 50 s. Stationary states, identified by convolving the time series with the time derivative of a Gaussian filter with σ = 1 s (see Fig. 2B) were localized with an accuracy of ∼1 s.
Prolongation of periodic behaviors beyond the well-identified window
The end points of swimming and crawling states were identified by looking at the existence of periodic events at the boundaries of the well-identified windows.
First, all local maxima of the elongation time series were identified. Every interval between two consecutive maxima was tested as a potential periodic motion. The amplitude of this potential oscillation was defined as the difference between the maximum and the minimum in the interval. If the amplitude was <3 pixel, the oscillation was considered to be caused by noise. The oscillation was thus discarded, and the next interval between two consecutive maxima was considered. Otherwise (if the amplitude was >3 pixel) the interval was counted as a potential periodic interval. Second, for every potential oscillation, the period was taken as the distance between two successive maxima. If for more than four consecutive potential oscillations the difference between the periods is <60% of their average and the difference between the amplitudes was less than their average, the interval between the first and the last maximum was considered as a regular periodic motion with a period that was the average of the single periods.
If a regular periodic motion with period between 0.5 and 1 s bordered or partially overlapped with a state previously classified as swimming, then the swimming state was prolonged to the border of the motion. The same thing happened if a crawling state bordered or partially overlapped with a regular periodic motion with period between 3 and 10 s.
Because swimming and crawling prolongation were not allowed to be overlapped, a constraint was imposed on this procedure. Moreover, if the time interval of prolongation of a periodic (swimming or crawling) state overlapped with a state previously classified as exploratory or stationary (even if this was unlikely to happen), the time interval was classified as belonging to the periodic state.
This prolongation is showed in Fig. 7B, where the swimming behavior had been prolonged in correspondence to the two thick arrows. Swimming and pseudo-swimming were distinguished by analyzing the values of V10tail(n).
Assembling of identical states and elimination of short states
Unclassified sections of events of duration <10 s separating two identical crawling, swimming, or exploratory states were considered as part of the surrounding behavior and therefore included in it. Unclassified states between two stationary states, instead, were also considered stationary if their duration was <10 s and the maximum displacement of the head in the time interval was <10 pixel. After this procedure, the remaining classified states lasting <5 s were eliminated avoiding positive classification of incomplete behaviors. Figure 7 shows these different steps leading to the final identification of swimming, crawling, and exploratory states.
With this procedure, the final behavior classification shown in Fig. 7, C and E, was obtained. At the end of the procedure in Fig. 7C, three swimming episodes (in gray), one still state (in yellow), and one exploratory episode (in blue) were correctly identified. In the same way, in Fig. 7E, five still states (in yellow), two exploratory (in blue), one abrupt movement (in cyan), and three crawling episodes (in red) were identified.
EVALUATION OF THE AUTOMATIC SYSTEM
The proposed method for the automatic classification of leech behaviors identifies the following states: stationary states subdivided into still, peristaltic-like, and head attached; swimming; pseudo-swimming; crawling; exploratory states; and abrupt movements.
The existence of these states was justified by visual inspection of the leech motion and behavior and by the vast ethological literature on the leech (Baader 1997; Brodfuehrer and Thorogood 2001; Cacciatore et al. 2000; Kristan 1992; Lent et al. 1988; Lewis and Kristan 1998; Muller et al. 1981; Wilson and Kleinhaus 2000). The identification of these states was based on a simple statistical analysis of the speed and frequency content of the leech elongation e(n) and of the head, midbody, and tail coordinates described by the time series.
The automatic identification of these states was based on the location of clusters and on their width in the histograms of (Vhead, Vmidbody), (V20max(n),f20d(n)) and (V50max(n),f50d(n)). Figure 8 illustrates the statistics of the cluster location and its width for free moving and pinned leeches. Figure 8A shows the SD σ of the stationary peak in (Vhead, Vmidbody) for five pinned leeches (on the left) and five free leeches (on the right). Figure 8B shows the central frequency and the relative σ of the swimming cluster in (V20max(n),f20d(n)) for five pinned leeches (on the left) and seven free leeches (on the right). As seen before (in the Pronlongation of periodic behaviors beyond the well-identified window), pinned leeches swim at a reduced frequency. Figure 8C shows the central frequency and the relative σ of the crawling cluster in (V50max(n),f50d(n)) for four free leeches (on the right). As explained in Prolongation of periodic behaviors beyond the well-identified window, pinned leeches cannot crawl. The number of analyzed leeches was different for different behaviors because very often a leech was swimming but never crawling or doing the opposite, mainly depending on the level of water in the dish. For every figure, data coming from experiments where the behavior shown was present for >1 h are displayed.
A small variability in the peak location and its width among different leeches is shown in Fig. 8, A–D, indicating that the proposed state identification captured fundamental properties of leech motion consistently identified across individuals.
The proposed identification was not only based on the statistical analysis described in classification of different behaviors but also on the procedures illustrated in identification of different behaviors. Figure 8D shows, for different leeches, the total fraction of the time series that had been identified as belonging to one of the states described before during the different steps used in the identification procedure: matching, prolongation, and assembling. States identified by the matching step were rather reliable as these points were those around the identified clusters in the histograms of Fig. 2 and 3, having a very stable position. They represent ∼75% for different leeches. The fraction of identified states becomes 90% after border completion and assembling.
The objective of the present study was to provide a novel automatic method for the classification of the leech behavior. Automatic systems are the best way for analyzing animal behavior in natural or controlled conditions, offering the possibility to obtain some quantitative parameters as spatial and temporal activity, speed, duration of stops, etc., which are hard to calculate when are only based on the observer's subjective judgment. Our multiple marker approach was thought as a tool allowing precise and informative characterization of the leech motion in real time.
The proposed method was based on the tracking of colored beads glued to the dorsal side of the animal body. As a consequence of the glue, the segment where each bead was attached lost flexibility. Because our measurements were based on the whole body deformation, these local effects do not affect the characterization of the global behavior of the leech. At the end of the experimental session 6 time series were obtained. Therefore the determination of characteristic values was done from time series (x, y, and time series) and not from a video-aided recording system, which required the scanning and storing of each frame (Gharbawie et al. 2004; Orito et al. 2004). In fact, the possibility of recording only the time series of the tracked points allowed us to record long periods of locomotion (hours instead of minutes).
The obtained time series were analyzed off-line. Identification of leech behaviors was based primarily on a statistical analysis of time series and not on pattern-recognition methods. The proposed method provides an automatic classification for ∼90% of the behavior observed in a leech freely moving in a petri dish and/or a small tank. These characteristics are discussed in the following text.
Reliability of the tracking state
Colored beads—usually, red, green and blue—glued on the back of the leech were tracked in real time by a simple processing of images acquired by a color CCD camera. For a reliably discrimination of the beads, it was necessary to match properly the spectral and sensitivity properties of the CCD camera to the illumination of the experimental set-up. There are some points to take in mind for selecting the colors of the beads. First is the color of the leech. By selecting light colors, this inconvenience could be avoided. In the case of the green bead, for example, the selection was done using a light green because by adjusting the source of light, the leech seems to be black, and the beads could be easily identified. Second is the water and the color of the dish. If the light is high, it is reflected by the water (or by the dish), and these reflexes could be taken as beads when the leech is in a stationary state. On the other hand, if the light is low, then everything is dark and tracking is not possible. Third is the volume of water displaced by the moving leech. This effect could also be avoided by adjusting the light because there is only a small quantity of water displaced. Then the tracking process only requires a proper illumination of the dish where the leech is moving.
The proposed scheme, at the moment, provides a useful classification of the leech motion when the animal moves mainly on a plane. In this case, however, the animal could rotate around its body and the CCD camera cannot track the colored beads. Using small colored rings surrounding the animal body, this problem could be circumvented. Likewise, by using an experimental set-up with two color CCD cameras, the complete three-dimensional motion of the animal could be reconstructed. Additionally, the CCD camera, software, and hardware used in the present work were low-cost components. By using more sophisticated and expensive components, it will be possible tracking and monitoring social interactions among multiple animals simultaneously. The only restriction of the system is the impossibility to track the locomotion of small animals, such as the Caenorhabditis elegans, where the attaching of beads is impossible, but perhaps this problem could be solved by staining the skin on the three required positions.
Statistical methods versus pattern recognition
Stationary states, swimming, crawling, and exploratory behaviors were not identified by recognizing a predefined pattern in the time series. Behavior classification was primarily based on statistical methods and not on pattern-recognition procedures. Indeed, systems using color-based tracking combined with movement-based tracking techniques (Balch et al. 2001) have limitations measuring immobility. If the animal remains immobile for some time, it cannot longer be tracked, and this represents a problem. There are different kinds of locomotion and certainly different kinds of immobility. Some periods of immobility or pauses could increase the capacity of the sensory systems to detect stimuli, and in this context, they should be measured. This segmentation into stops and progressions occurs in mammals (Drai et al. 2000; Kenagy 1973), insects (Collins et al. 1994), lizards (Pietruska 1986), etc. Our system has the possibility of measuring all these motionless periods, and detailed analyses of these observations, in the leech, will be published elsewhere (E. Garcia-Perez, A. Mazzoni, D. Zoccolan, H.P.C. Robinson, and V. Torre, unpublished data).
Analysis of speed and frequency provided enough information to discriminate between behaviors without introducing any ad hoc hypothesis on their characteristics. The prolongation of periodic behaviors beyond the well-identified window, described in Prolongation of periodic behaviors beyond the well-identified window, however, was based on the search of patterns of a priori known periods: 1.5 Hz for swimming and 0.2 Hz for crawling. The use of statistical methods was useful to avoid classification bias based on the experimenter expectations.
Reliability of state identification
Reliability of the identification of well-known behaviors, such as swimming, crawling, and peristalsis was checked by visual inspection of the videotape of the leech motion. This check was performed on randomly chosen episodes but was neither performed over the entire duration of the experiments—usually lasting several hours—nor for all leeches. These checks indicated correct behavior identification. Therefore we conclude that the system seems to be able to classify properly ∼90% of the leech motion.
Besides swimming, crawling, peristalsis, and stationary states, we have identified another state, the exploratory state, which can account for ∼30% of the leech motion. To analyze the complexity of the exploratory behavior, a visualization and analysis tool is needed. This tool should emphasize regularities, if they exist, and also should help generating new theories about the structure of this behavior. Our system characterized the exploratory state by a power spectrum of the elongation e(n) with no clear peaks at any frequency. This exploratory behavior was likely to be composed by several substrates with distinct properties, which will be reported elsewhere (E. Garcia-Perez, A. Mazzoni, D. Zoccolan, H.P.C. Robinson, and V. Torre, unpublished results).
The proposed system for the automatic classification of the leech behavior has ∼10% of unclassified states. The analysis of these unclassified states will be presented elsewhere.
Having validated the classification procedures for restrained leeches, the electrical activity of motoneurons was monitored with suction pipettes and/or intracellular sharp electrodes (unpublished observations) and its correlation with the behavior was analyzed.
A quantitative characterization of the behavior of the leech over extended intervals of time is an essential step for understanding its internal organization and how it is determined by interactions with the environment. This method for automatic characterization of the leech behavior is an important step toward understanding leech behavior and its properties (E. Garcia-Perez, A. Mazzoni, D. Zoccolan, H.P.C. Robinson, and V. Torre, unpublished observations). We believe that this system could be used also to characterize the behavior of other small invertebrates like worms and snails and possibly also some vertebrates such as snakes, lizards, frogs, and small fishes. Therefore this system could help to quantify and characterize abnormal patterns of locomotion of mutant animals contributing to understand the relationship between genetic disorders and behavior.
We are indebted to H. Robison for reading the manuscript and useful comments and to W. Kristan for inspiring discussions and fruitful suggestions.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2005 by the American Physiological Society