OpenCV  4.5.2
Open Source Computer Vision
Namespaces | Classes | Typedefs | Enumerations
Tracking API implementation details

Namespaces

 cv::detail::tracking::contrib_feature
 
 cv::detail::tracking::kalman_filters
 
 cv::detail::tracking::online_boosting
 
 cv::detail::tracking::tbm
 
 cv::detail::tracking::tld
 

Classes

class  cv::detail::tracking::kalman_filters::AugmentedUnscentedKalmanFilterParams
 Augmented Unscented Kalman filter parameters. The class for initialization parameters of Augmented Unscented Kalman filter. More...
 
class  cv::detail::tracking::online_boosting::BaseClassifier
 
class  cv::detail::tracking::online_boosting::ClassifierThreshold
 
class  cv::detail::tracking::tbm::CosDistance
 The CosDistance class allows computing cosine distance between two reidentification descriptors. More...
 
class  cv::detail::tracking::contrib_feature::CvFeatureEvaluator
 
class  cv::detail::tracking::contrib_feature::CvFeatureParams
 
class  cv::detail::tracking::contrib_feature::CvHaarEvaluator
 
class  cv::detail::tracking::contrib_feature::CvHaarFeatureParams
 
class  cv::detail::tracking::contrib_feature::CvHOGEvaluator
 
struct  cv::detail::tracking::contrib_feature::CvHOGFeatureParams
 
class  cv::detail::tracking::contrib_feature::CvLBPEvaluator
 
struct  cv::detail::tracking::contrib_feature::CvLBPFeatureParams
 
class  cv::detail::tracking::contrib_feature::CvParams
 
class  cv::detail::tracking::online_boosting::Detector
 
class  cv::detail::tracking::online_boosting::EstimatedGaussDistribution
 
class  cv::detail::tracking::contrib_feature::CvLBPEvaluator::Feature
 
class  cv::detail::tracking::contrib_feature::CvHOGEvaluator::Feature
 
class  cv::detail::tracking::contrib_feature::CvHaarEvaluator::FeatureHaar
 
class  cv::detail::tracking::tbm::IDescriptorDistance
 The IDescriptorDistance class declares an interface for distance computation between reidentification descriptors. More...
 
class  cv::detail::tracking::tbm::IImageDescriptor
 The IImageDescriptor class declares base class for image descriptor. More...
 
class  cv::detail::tracking::tbm::ITrackerByMatching
 Tracker-by-Matching algorithm interface. More...
 
class  cv::detail::tracking::tbm::MatchTemplateDistance
 Computes distance between images using MatchTemplate function from OpenCV library and its cross-correlation computation method in particular. More...
 
struct  cv::detail::tracking::TrackerContribFeatureHAAR::Params
 
struct  cv::detail::tracking::TrackerSamplerPF::Params
 This structure contains all the parameters that can be varied during the course of sampling algorithm. Below is the structure exposed, together with its members briefly explained with reference to the above discussion on algorithm's working. More...
 
struct  cv::detail::tracking::TrackerSamplerCS::Params
 
struct  cv::detail::tracking::TrackerContribSamplerCSC::Params
 
struct  cv::detail::tracking::TrackerSamplerCSC::Params
 
class  cv::detail::tracking::tbm::ResizedImageDescriptor
 Uses resized image as descriptor. More...
 
class  cv::detail::tracking::online_boosting::StrongClassifierDirectSelection
 
class  cv::detail::tracking::tbm::Track
 The Track class describes tracks. More...
 
struct  cv::detail::tracking::tbm::TrackedObject
 The TrackedObject struct defines properties of detected object. More...
 
class  cv::detail::tracking::TrackerStateEstimatorAdaBoosting::TrackerAdaBoostingTargetState
 Implementation of the target state for TrackerAdaBoostingTargetState. More...
 
class  cv::detail::tracking::TrackerContribFeature
 Abstract base class for TrackerContribFeature that represents the feature. More...
 
class  cv::detail::tracking::TrackerContribFeatureHAAR
 TrackerContribFeature based on HAAR features, used by TrackerMIL and many others algorithms. More...
 
class  cv::detail::tracking::TrackerContribFeatureSet
 Class that manages the extraction and selection of features. More...
 
class  cv::detail::tracking::TrackerContribSampler
 Class that manages the sampler in order to select regions for the update the model of the tracker [AAM] Sampling e Labeling. See table I and section III B. More...
 
class  cv::detail::tracking::TrackerContribSamplerAlgorithm
 Abstract base class for TrackerContribSamplerAlgorithm that represents the algorithm for the specific sampler. More...
 
class  cv::detail::tracking::TrackerContribSamplerCSC
 TrackerSampler based on CSC (current state centered), used by MIL algorithm TrackerMIL. More...
 
class  cv::detail::tracking::TrackerFeature
 Abstract base class for TrackerFeature that represents the feature. More...
 
class  cv::detail::tracking::TrackerFeatureFeature2d
 TrackerContribFeature based on Feature2D. More...
 
class  cv::detail::tracking::TrackerFeatureHOG
 TrackerContribFeature based on HOG. More...
 
class  cv::detail::tracking::TrackerFeatureLBP
 TrackerContribFeature based on LBP. More...
 
class  cv::detail::tracking::TrackerFeatureSet
 Class that manages the extraction and selection of features. More...
 
class  cv::detail::tracking::TrackerModel
 Abstract class that represents the model of the target. More...
 
struct  cv::detail::tracking::tbm::TrackerParams
 The TrackerParams struct stores parameters of TrackerByMatching. More...
 
class  cv::detail::tracking::TrackerSampler
 Class that manages the sampler in order to select regions for the update the model of the tracker [AAM] Sampling e Labeling. See table I and section III B. More...
 
class  cv::detail::tracking::TrackerSamplerAlgorithm
 Abstract base class for TrackerSamplerAlgorithm that represents the algorithm for the specific sampler. More...
 
class  cv::detail::tracking::TrackerSamplerCS
 TrackerContribSampler based on CS (current state), used by algorithm TrackerBoosting. More...
 
class  cv::detail::tracking::TrackerSamplerCSC
 TrackerSampler based on CSC (current state centered), used by MIL algorithm TrackerMIL. More...
 
class  cv::detail::tracking::TrackerSamplerPF
 This sampler is based on particle filtering. More...
 
class  cv::detail::tracking::TrackerStateEstimator
 Abstract base class for TrackerStateEstimator that estimates the most likely target state. More...
 
class  cv::detail::tracking::TrackerStateEstimatorAdaBoosting
 TrackerStateEstimatorAdaBoosting based on ADA-Boosting. More...
 
class  cv::detail::tracking::TrackerStateEstimatorSVM
 TrackerStateEstimator based on SVM. More...
 
class  cv::detail::tracking::TrackerTargetState
 Abstract base class for TrackerTargetState that represents a possible state of the target. More...
 
class  cv::detail::tracking::kalman_filters::UkfSystemModel
 Model of dynamical system for Unscented Kalman filter. The interface for dynamical system model. It contains functions for computing the next state and the measurement. It must be inherited for using UKF. More...
 
class  cv::detail::tracking::kalman_filters::UnscentedKalmanFilter
 The interface for Unscented Kalman filter and Augmented Unscented Kalman filter. More...
 
class  cv::detail::tracking::kalman_filters::UnscentedKalmanFilterParams
 Unscented Kalman filter parameters. The class for initialization parameters of Unscented Kalman filter. More...
 
class  cv::detail::tracking::online_boosting::WeakClassifierHaarFeature
 

Typedefs

typedef std::vector< std::pair< Ptr< TrackerTargetState >, float > > cv::detail::tracking::ConfidenceMap
 Represents the model of the target at frame \(k\) (all states and scores) More...
 
typedef std::vector< Ptr< TrackerTargetState > > cv::detail::tracking::Trajectory
 Represents the estimate states for all frames. More...
 

Enumerations

enum  {
  cv::detail::tracking::TrackerContribSamplerCSC::MODE_INIT_POS = 1,
  cv::detail::tracking::TrackerContribSamplerCSC::MODE_INIT_NEG = 2,
  cv::detail::tracking::TrackerContribSamplerCSC::MODE_TRACK_POS = 3,
  cv::detail::tracking::TrackerContribSamplerCSC::MODE_TRACK_NEG = 4,
  cv::detail::tracking::TrackerContribSamplerCSC::MODE_DETECT = 5
}
 
enum  {
  cv::detail::tracking::TrackerSamplerCS::MODE_POSITIVE = 1,
  cv::detail::tracking::TrackerSamplerCS::MODE_NEGATIVE = 2,
  cv::detail::tracking::TrackerSamplerCS::MODE_CLASSIFY = 3
}
 
enum  cv::detail::tracking::contrib_feature::CvFeatureParams::FeatureType {
  cv::detail::tracking::contrib_feature::CvFeatureParams::HAAR = 0,
  cv::detail::tracking::contrib_feature::CvFeatureParams::LBP = 1,
  cv::detail::tracking::contrib_feature::CvFeatureParams::HOG = 2
}
 
enum  cv::detail::tracking::TrackerSamplerCSC::MODE {
  cv::detail::tracking::TrackerSamplerCSC::MODE_INIT_POS = 1,
  cv::detail::tracking::TrackerSamplerCSC::MODE_INIT_NEG = 2,
  cv::detail::tracking::TrackerSamplerCSC::MODE_TRACK_POS = 3,
  cv::detail::tracking::TrackerSamplerCSC::MODE_TRACK_NEG = 4,
  cv::detail::tracking::TrackerSamplerCSC::MODE_DETECT = 5
}
 

Detailed Description

Long-term optical tracking API

Long-term optical tracking is an important issue for many computer vision applications in real world scenario. The development in this area is very fragmented and this API is an unique interface useful for plug several algorithms and compare them. This work is partially based on [209] and [144] .

These algorithms start from a bounding box of the target and with their internal representation they avoid the drift during the tracking. These long-term trackers are able to evaluate online the quality of the location of the target in the new frame, without ground truth.

There are three main components: the TrackerContribSampler, the TrackerContribFeatureSet and the TrackerModel. The first component is the object that computes the patches over the frame based on the last target location. The TrackerContribFeatureSet is the class that manages the Features, is possible plug many kind of these (HAAR, HOG, LBP, Feature2D, etc). The last component is the internal representation of the target, it is the appearance model. It stores all state candidates and compute the trajectory (the most likely target states). The class TrackerTargetState represents a possible state of the target. The TrackerContribSampler and the TrackerContribFeatureSet are the visual representation of the target, instead the TrackerModel is the statistical model.

A recent benchmark between these algorithms can be found in [276]

Creating Your Own Tracker

If you want to create a new tracker, here's what you have to do. First, decide on the name of the class for the tracker (to meet the existing style, we suggest something with prefix "tracker", e.g. trackerMIL, trackerBoosting) – we shall refer to this choice as to "classname" in subsequent.

Every tracker has three component TrackerContribSampler, TrackerContribFeatureSet and TrackerModel. The first two are instantiated from Tracker base class, instead the last component is abstract, so you must implement your TrackerModel.

TrackerContribSampler

TrackerContribSampler is already instantiated, but you should define the sampling algorithm and add the classes (or single class) to TrackerContribSampler. You can choose one of the ready implementation as TrackerContribSamplerCSC or you can implement your sampling method, in this case the class must inherit TrackerContribSamplerAlgorithm. Fill the samplingImpl method that writes the result in "sample" output argument.

Example of creating specialized TrackerContribSamplerAlgorithm TrackerContribSamplerCSC : :

class CV_EXPORTS_W TrackerContribSamplerCSC : public TrackerContribSamplerAlgorithm
{
public:
TrackerContribSamplerCSC( const TrackerContribSamplerCSC::Params &parameters = TrackerContribSamplerCSC::Params() );
~TrackerContribSamplerCSC();
...
protected:
bool samplingImpl( const Mat& image, Rect boundingBox, std::vector<Mat>& sample );
...
};

Example of adding TrackerContribSamplerAlgorithm to TrackerContribSampler : :

//sampler is the TrackerContribSampler
Ptr<TrackerContribSamplerAlgorithm> CSCSampler = new TrackerContribSamplerCSC( CSCparameters );
if( !sampler->addTrackerSamplerAlgorithm( CSCSampler ) )
return false;
//or add CSC sampler with default parameters
//sampler->addTrackerSamplerAlgorithm( "CSC" );
See also
TrackerContribSamplerCSC, TrackerContribSamplerAlgorithm

TrackerContribFeatureSet

TrackerContribFeatureSet is already instantiated (as first) , but you should define what kinds of features you'll use in your tracker. You can use multiple feature types, so you can add a ready implementation as TrackerContribFeatureHAAR in your TrackerContribFeatureSet or develop your own implementation. In this case, in the computeImpl method put the code that extract the features and in the selection method optionally put the code for the refinement and selection of the features.

Example of creating specialized TrackerFeature TrackerContribFeatureHAAR : :

class CV_EXPORTS_W TrackerContribFeatureHAAR : public TrackerFeature
{
public:
TrackerContribFeatureHAAR( const TrackerContribFeatureHAAR::Params &parameters = TrackerContribFeatureHAAR::Params() );
~TrackerContribFeatureHAAR();
void selection( Mat& response, int npoints );
...
protected:
bool computeImpl( const std::vector<Mat>& images, Mat& response );
...
};

Example of adding TrackerFeature to TrackerContribFeatureSet : :

//featureSet is the TrackerContribFeatureSet
Ptr<TrackerFeature> trackerFeature = new TrackerContribFeatureHAAR( HAARparameters );
featureSet->addTrackerFeature( trackerFeature );
See also
TrackerContribFeatureHAAR, TrackerContribFeatureSet

TrackerModel

TrackerModel is abstract, so in your implementation you must develop your TrackerModel that inherit from TrackerModel. Fill the method for the estimation of the state "modelEstimationImpl", that estimates the most likely target location, see [209] table I (ME) for further information. Fill "modelUpdateImpl" in order to update the model, see [209] table I (MU). In this class you can use the :cConfidenceMap and :cTrajectory to storing the model. The first represents the model on the all possible candidate states and the second represents the list of all estimated states.

Example of creating specialized TrackerModel TrackerMILModel : :

class TrackerMILModel : public TrackerModel
{
public:
TrackerMILModel( const Rect& boundingBox );
~TrackerMILModel();
...
protected:
void modelEstimationImpl( const std::vector<Mat>& responses );
void modelUpdateImpl();
...
};

And add it in your Tracker : :

bool TrackerMIL::initImpl( const Mat& image, const Rect2d& boundingBox )
{
...
//model is the general TrackerModel field of the general Tracker
model = new TrackerMILModel( boundingBox );
...
}

In the last step you should define the TrackerStateEstimator based on your implementation or you can use one of ready class as TrackerStateEstimatorMILBoosting. It represent the statistical part of the model that estimates the most likely target state.

Example of creating specialized TrackerStateEstimator TrackerStateEstimatorMILBoosting : :

class CV_EXPORTS_W TrackerStateEstimatorMILBoosting : public TrackerStateEstimator
{
class TrackerMILTargetState : public TrackerTargetState
{
...
};
public:
TrackerStateEstimatorMILBoosting( int nFeatures = 250 );
~TrackerStateEstimatorMILBoosting();
...
protected:
Ptr<TrackerTargetState> estimateImpl( const std::vector<ConfidenceMap>& confidenceMaps );
void updateImpl( std::vector<ConfidenceMap>& confidenceMaps );
...
};

And add it in your TrackerModel : :

//model is the TrackerModel of your Tracker
Ptr<TrackerStateEstimatorMILBoosting> stateEstimator = new TrackerStateEstimatorMILBoosting( params.featureSetNumFeatures );
model->setTrackerStateEstimator( stateEstimator );
See also
TrackerModel, TrackerStateEstimatorMILBoosting, TrackerTargetState

During this step, you should define your TrackerTargetState based on your implementation. TrackerTargetState base class has only the bounding box (upper-left position, width and height), you can enrich it adding scale factor, target rotation, etc.

Example of creating specialized TrackerTargetState TrackerMILTargetState : :

class TrackerMILTargetState : public TrackerTargetState
{
public:
TrackerMILTargetState( const Point2f& position, int targetWidth, int targetHeight, bool foreground, const Mat& features );
~TrackerMILTargetState();
...
private:
bool isTarget;
Mat targetFeatures;
...
};

Typedef Documentation

typedef std::vector<std::pair<Ptr<TrackerTargetState>, float> > cv::detail::tracking::ConfidenceMap

#include <opencv2/video/detail/tracking.detail.hpp>

Represents the model of the target at frame \(k\) (all states and scores)

See [209] The set of the pair \(\langle \hat{x}^{i}_{k}, C^{i}_{k} \rangle\)

See also
TrackerTargetState
typedef std::vector<Ptr<TrackerTargetState> > cv::detail::tracking::Trajectory

#include <opencv2/video/detail/tracking.detail.hpp>

Represents the estimate states for all frames.

[209] \(x_{k}\) is the trajectory of the target up to time \(k\)

See also
TrackerTargetState

Enumeration Type Documentation

anonymous enum
Enumerator
MODE_INIT_POS 

mode for init positive samples

MODE_INIT_NEG 

mode for init negative samples

MODE_TRACK_POS 

mode for update positive samples

MODE_TRACK_NEG 

mode for update negative samples

MODE_DETECT 

mode for detect samples

anonymous enum
Enumerator
MODE_POSITIVE 

mode for positive samples

MODE_NEGATIVE 

mode for negative samples

MODE_CLASSIFY 

mode for classify samples

Enumerator
HAAR 
LBP 
HOG 
Enumerator
MODE_INIT_POS 

mode for init positive samples

MODE_INIT_NEG 

mode for init negative samples

MODE_TRACK_POS 

mode for update positive samples

MODE_TRACK_NEG 

mode for update negative samples

MODE_DETECT 

mode for detect samples