Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is a minimally invasive imaging technique which can be used for characterizing tumor biology and tumor response to radiotherapy. Pharmacokinetic (PK) estimation is widely used for DCE-MRI data analysis to extract quantitative parameters relating to microvasculature characteristics of the cancerous tissues. Unavoidable noise corruption during DCE-MRI data acquisition has a large effect on the accuracy of PK estimation. In this paper, we propose a general denoising paradigm called gather- noise attenuation and reduce (GNR) and a novel temporal-spatial collaborative filtering (TSCF) denoising technique for DCE-MRI data. TSCF takes advantage of temporal correlation in DCE-MRI, as well as anatomical spatial similarity to collaboratively filter noisy DCE-MRI data. The proposed TSCF denoising algorithm decreases the PK parameter normalized estimation error by 57% and improves the structural similarity of PK parameter estimation by 86% compared to baseline without denoising, while being an order of magnitude faster than state-of-the-art denoising methods. TSCF improves the univariate linear regression (ULR) c-statistic value for early prediction of pathologic response up to 18%, and shows complete separation of pathologic complete response (pCR) and non-pCR groups on a challenge dataset.