Preheat oven to 400 degrees and line a baking sheet with foil. Place sweet potatoes on baking sheet and pierce all over with a fork.
Bake potatoes for 45-60 minutes, until skin is wrinkled and potatoes are fork tender. Remove from oven and lower oven temperature to 375 degrees.
When potatoes are cool enough to touch, peel skin off them and place them in a blender or food processor with 1 Tbsp milk. Process until smooth. Measure out 2 cups of puree and then remove 2 Tbsp from it (you can eat or discard the 2 Tbsp as you won't need it).
In a medium bowl, whisk together flour, baking soda, baking powder, ground cinnamon, salt, ground nutmeg, ground ginger, and ground cloves.
In a separate large bowl, stir together sweet potato puree, brown sugar, granulated sugar, eggs, vegetable oil, remaining ¼ cup milk, and vanilla until well combined.
Add dry ingredients to wet ingredients and stir until just combined, taking care not to over-mix.
Line 16 standard-size muffin cups with papers and spoon batter into each (I like to use a large cookie scoop) almost all the way to the top. Sprinkle cinnamon streusel over each.
Bake for about 20 minutes, until a toothpick stuck into the center comes out clean. Let muffins cool in tins for a few minutes before moving to a metal rack to cool completely.
Once muffins are cooled, drizzle icing over them.