• Preheat the oven to 375°F. 
• In a medium bowl, whisk together flour, cocoa powder, baking soda, and salt.
• In the bowl of an electric mixer fitted with the paddle attachment, cream together butter and sugar until light and fluffy, about 2 minutes. Add egg, milk, and extracts, and beat until well combined. Add the flour mixture and mix on low speed until combined.
• Using a tablespoon or 1 ¾-inch ice cream scoop (#40), drop portions of dough onto ungreased baking sheets, about 2 inches apart. Bake until cookies begin to spread and become firm, 7 to 10 minutes.
• Remove the baking sheets from the oven and place a marshmallow, cut-side down, in the center of each cookie, pressing down slightly. Return to the oven and continue baking until the marshmallows begin to melt, 2 to 2 ½ minutes.
• Transfer the cookies to a wire rack to cool completely before frosting. To frost, spread about 1 tablespoon of frosting over each marshmallow, starting in the center and continuing outward until the marshmallow is covered.

CHOCOLATE FROSTING:

• Place confectioners’ sugar in a medium bowl. Whisk in butter and cocoa powder. Add milk and vanilla, and whisk until well combined.