The well-known “singularity” problem is one of the major defects in kernel-based video object tracking. However, the multiple-extreme problem is more general but has not been paid attention to. This paper studies this important problem and presents a novel approach called Section-Based Tracking (SBT) that is a kernel-based method with the section information provided by the division of the object’s weight image. SBT serves to eliminate fake extreme points and make the tracking process more robust. Besides, this new approach can also be naturally extended to deal with scale change and rotation. Experiments show that SBT method conduces to better performance in tracking process.