Preheat oven to 350 degrees and line two cupcake tins with cupcake papers.
In the bowl of a stand mixer or in a large bowl with a hand mixer, cream together butter and sugar until light and fluffy, 2-3 minutes.
Add the eggs, one at a time, beating well after each addition.
In a separate medium bowl, whisk together flour, baking powder, nutmeg, and salt.
With the mixer on low, alternate adding dry ingredients and eggnog to the wet mixture, starting and ending with the dry ingredients. Mix until just combined, taking care not to over-mix.
Fill cupcake tins about ¾ of the way to the top with batter.
Bake for 18-20 minutes, until cupcakes are beginning to turn golden around edges and a toothpick inserted into the center comes out clean. Let cupcakes cool in pan for a few minutes before moving to rack to cool completely.
Once cupcakes are completely cool, frost with eggnog frosting (I use a Wilton #1A pastry tip) and top with decorations and a little straw if desired.