This paper describes a methodology that integrates recognition and segmentation, simultaneously with image tracking in a cooperative manner, for recognition of objects (or parts of them) in image sequences. A probabilistic general approach at pixel level is depicted together with a practical heuristic simplification in which pixels’ class probabilities are approximated by a finite small set of class possibility values. These possibility values are updated iteratively along the image sequence for each class and each pixel taking into account both the prior tracking information and the spot-based object recognition results provided by a trained neural network. A further segmentation of the class possibility images allows the tracking of each object of interest in the sequence. The good experimental results obtained so far show the viability of the approach under certain conditions.