Combine the milk, pumpkin puree, granulated sugar, brown sugar, vanilla, pumpkin spice, and eggs in a large mixing bowl. Whisk until combined, ensuring there are no streaks of egg visible for a smooth custard base.
Generously spray a 9x13 baking dish with nonstick cooking spray. Spread the cubed French bread evenly in the prepared baking dish.
Pour the pumpkin mixture over the cubed bread, using a fork or spoon to gently press down so the bread absorbs as much of the liquid as possible. Cover the dish with plastic wrap and chill in the refrigerator for at least 3 hours or overnight.
Preheat your oven to 350°F (175°C). To prepare the streusel topping, combine cold cubed butter, brown sugar, flour, and pumpkin pie spice in a food processor and pulse until it resembles coarse sand; add chopped pecans afterward.
Remove the casserole from refrigeration and evenly sprinkle the streusel topping over it before placing it in the preheated oven. Bake for 45-50 minutes until golden brown and set; test doneness by inserting a toothpick into the center.
Once baked, remove from oven and drizzle with maple syrup if desired; let cool for about 10-15 minutes before slicing into squares. Serve warm with optional vanilla ice cream or whipped topping.