A novel continuous-space latent diffusion framework called VQ-LCMD is introduced, allowing generative modeling of discrete data.VQ-LCMD combines joint embedding-diffusion training with a consistency-matching (CM) loss to stabilize training and enhance performance.Experiments demonstrate that VQ-LCMD outperforms discrete-state latent diffusion models on FFHQ, LSUN Churches, and LSUN Bedrooms benchmarks.VQ-LCMD achieves a FID of 6.81 for class-conditional image generation on ImageNet with 50 steps.