Optional: rim glasses in festive sprinkles. Place sprinkles on a small plate and brush the rim of your mug in simple syrup, honey, agave or maple syrup, or water. Place glass upside down on plate and rotate it to coat rim in sprinkles.
Whisk together eggnog, milk, and cocoa powder in a medium saucepan. Heat over medium-low heat, whisking frequently, until hot, but not boiling.
Serve in mugs with whipped cream, mini marshmallows, sprinkles, and ground nutmeg, if desired.