Overall the last decade, a large number of time stepping schemes have been developed for time-fractional diffusion problems. Many of these methods are developed by assuming that the solution is sufficiently smooth, which however is generally not true. In this talk, I will describe our recent work in correction of high-order numerical schemes. To illustrate our technique, we introduce modified k-step BDF convolution quadrature in order to discretize the time-fractional derivative. The desired kth-order convergence rate can be achieved even if the source term is not compatible with the initial data, which is allowed to be non- smooth.