Crush two graham crackers (either in a blender, food processor, or in a ziplock bag) and place crumbs on a small plate. Brush the rims of two glasses with water, honey, or simple syrup and roll in the graham cracker crumbs to coat rims.
Place caramels and water in a small saucepan over medium-low heat. Let the caramels melt into a sauce while occasionally stirring.
While caramels are melting, place ice cream, milk, pumpkin, cinnamon, nutmeg, and the 3 remaining graham crackers into a blender and blend until smooth.
Pour milkshakes into graham cracker rimmed glasses. Drizzle caramel over the top of the milkshakes.