With the development of information technology, more and more big data applications arise from a variety of different fields. Usually, all the historical data is stored and then manipulated in the analysis procedure. As the amount of historical data increases greatly, the computation cost, including time and space cost, in the modeling and application of big data problems has become a bottleneck. In practice, we find that a large number of such problems possess a sequential nature, and the computational cost can be greatly reduced with a carefully designed sequential strategy. In this work, we propose a sequential linear regression (SLR) method to deal with the modeling of sequential data. Essential information is extracted from the historical data, and then used to update the model in a sequential estimation scheme. With this technique, data can be collected and used and then discarded without affecting the modeling process. In this way, the storage cost is fixed regardless of the volume of historical data. On the other hand, since only a small amount of data are processed each time to update the model, the time cost is also largely reduced. In fact, with our updating formula applied, the update procedure can be very efficient. In many cases of time-dependent problems, the impact of data from different period is distinct. Samples with different residual errors should also be differentiated from each other. Weighting the samples are usually applied to distinguish their influence on the current model. The weighting strategy is also incorporated in our sequential estimation. Convergence of SLR method is proved and various numerical experiments demonstrates that both the time and space complexity can be greatly reduced with our method.