In a medium bowl, mix the butter and the brown sugar together until it is coarse crumble.Add the cinnamon, nutmeg, ginger, salt, and half of the pecans.
Divide mixture in half. Add half of the mixture to the sweet potato flesh and fold in completely, and then add the filling back to the potato skins on the baking sheet.
To the other half of the brown sugar mixture, add the butter, flour and remaining pecans. Mix until it is coarse crumble.Top each of the sweet potatoes with the brown sugar mixture.