Preheat oven to 350°F (175°C). Grease two 9-inch round cake pans.
In a bowl, whisk together all-purpose flour, almond flour, baking powder, and salt.
Cream butter and sugar until fluffy; add eggs one at a time; mix in almond and vanilla extract.
Add flour mixture and milk alternately; mix until just combined.
Divide batter into pans; bake 25–30 minutes or until a toothpick comes out mostly clean.
Cool cakes on racks completely.
Whip heavy cream, powdered sugar, and extracts until stiff peaks form.
Spread cream over first layer; top with second; frost top and garnish with toasted almonds.