A smoothing spline is considered to propose a novel model for the stochastic quantile of the univariate time series using a state space approach. A correlation is further incorporated between the dependent variable and its one-step-ahead quantile. Using a Bayesian approach, an efficient Markov chain Monte Carlo algorithm is described where we use the multi-move sampler, which generates simultaneously latent stochastic quantiles. Numerical examples are provided to show its high sampling efficiency in comparison with the simple algorithm that generates one latent quantile at a time given other latent quantiles. Furthermore, using Japanese inflation rate data, an empirical analysis is provided with the model comparison.