Preheat the oven to 350° F. Line two cupcake pans with paper liners.
To make the filling, combine the confectioners' sugar, peanut butter, butter and vanilla extract in a bowl. Beat with an electric mixer on medium speed until well combined. Roll the mixture into 1-inch balls (at least 24) and set aside on a baking sheet.
To make the cake batter, combine the flour, cocoa powder, baking soda and salt in a medium bowl; whisk together and set aside. In a liquid measuring cup, stir together the sour cream, milk and vanilla extract. In the bowl of an electric mixer fitted with the paddle attachment, combine the butter and sugar, and beat on medium-high speed until light and fluffy, about 2 minutes. Beat in the eggs one at a time, scraping down the bowl as needed. With the mixer on low speed, alternately add the dry ingredients and the sour cream mixture, beginning and ending with the dry ingredients. Mix just until incorporated.
Spoon a tablespoon or two of batter into the bottom of each cupcake liner. Place a ball of the peanut butter filling in each cupcake well and top with the remaining batter so that all the cups are filled. Bake for 18-22 minutes, or until a toothpick inserted in the center comes out clean. Allow to cool in the pan for 5-10 minutes, then transfer to a wire rack to cool completely.
To make the frosting, in the bowl of an electric mixer fitted with the paddle attachment, combine the cream cheese, butter, and peanut butter until smooth. Slowly mix in the confectioners' sugar, beating until smooth and well blended. Mix in the whipped topping until smooth and fluffy. Frost cooled cupcakes as desired, and garnish with mini chocolate chips.