Detecting pedestrians in low resolution videos is a challenging task, due to the small size of pedestrians in the images and the limited information. In practical outdoor surveillance scenarios the pedestrian size is usually small. Existing state-of-the-art pedestrian detection methods that use histogram of oriented gradient (HOG) features have poor performance in this problem domain. To compensate for the lack of information in a single frame, we propose a novel detection method that recognizes pedestrians in a short sequence of frames. Namely, we take the single-frame HOG-based detector and extend it to multiple frames. Our detector is applied to regions containing potential moving objects. In the case of video taken from a moving camera on an aerial platform, video stabilization is first performed to register the frames. A classifier is then applied to features extracted from spatio-temporal volumes surrounding the potential moving objects. On challenging stationary and aerial video datasets, our detection accuracy outperforms several state-of-the-art algorithms.