We describe and estimate for the first time a natural multivariate extension of the univariate stochastic volatility model with leverage. The model, which we call the multivariate stochastic volatility with cross leverage, is fit by a tuned Bayesian MCMC method. Of particular general interest is our approach for sampling the state variables from the posterior distribution conditioned on the parameters. The state variables are sampled in blocks by the Metropolis-Hastings algorithm in which the proposal density is derived from an approximating linear Gaussian state space model. The conditional modes of the latent volatility variables are computed using a method of scoring where the covariance matrix of the proposal density is guaranteed to be positive definite. The auxiliary particle filter to compute the likelihood function is also shown and the model and the techniques are illustrated with daily stock returns data from the Tokyo Stock Exchange.