Preheat the oven to 350°F (175°C) and lightly grease a 2-quart baking dish.
In a large mixing bowl, combine the mashed sweet potatoes, granulated sugar, 1/2 cup melted butter, eggs, vanilla extract, and milk until smooth.
Spread the sweet potato mixture evenly into the bottom of the prepared baking dish.
In a separate medium bowl, mix together the brown sugar, flour, chopped pecans, and the remaining 1/3 cup melted butter until a crumbly texture forms.
Sprinkle the pecan mixture evenly over the top of the sweet potato base.
Bake for 30 to 35 minutes or until the topping is golden brown and the casserole is set around the edges.
Remove from the oven and let stand for 5 to 10 minutes before serving.