Preheat oven to 375 degrees. In a medium bowl, whisk together flour, cocoa powder, baking soda, and salt; set aside.
In the bowl of an electric mixer fitted with the paddle attachment, cream together butter and sugar until light and fluffy, about 2 minutes. Add egg, milk, and vanilla, and beat until well combined. Add reserved flour mixture; mix on low speed until combined.
Using a tablespoon or #40 scoop, drop dough onto ungreased baking sheets, about 2 inches apart. Bake until cookies begin to spread and become firm, about 7 minutes.
Remove baking sheets from oven, and place a marshmallow, cut-side down, in the center of each cookie, pressing down slightly. Return to oven, and continue baking until marshmallows begins to melt, 2 to 3 minutes. Transfer cookies to a wire rack to cool completely before frosting.
Prepare Chocolate Frosting: In a large mixing bowl, combine powdered sugar, cocoa and melted butter. Add milk and vanilla, and beat until smooth.
Spread about 1 tablespoon of frosting over each marshmallow, starting in the center and continuing outward until marshmallow is covered.