Preheat oven to 350 degrees Fahrenheit. If using whole graham crackers, crush them into fine crumbs and combine with melted butter, brown sugar, cinnamon, and ginger until uniform.
1 cup graham cracker crumbs, ¼ cup unsalted butter, ¼ cup brown sugar, ½ teaspoon ground cinnamon, ¼ teaspoon ground ginger
Spread the graham cracker mixture into oven-safe dish and bake for 20 minutes or until golden brown.
While the crust bakes, zest seven of the key limes then cut and juice all of the key limes. Mix the lime zest and juice with sweetened condensed milk, vanilla extract, and sea salt until uniform and set aside.
12 key limes, 14 ounces sweetened condensed milk, 2 teaspoons vanilla extract, ½ teaspoon fine sea salt
Using stand mixer with whisk attachment fitted or hand mixer, whip the heavy cream until stiff, fluffy peaks form. Fold the sweetened condensed milk mixture into the whipped cream until uniform and smooth.
2 cups heavy cream
Break up the cooled graham cracker crust and remove half of it from the dish. Spread out the remaining crust to cover most of the bottom of the dish then pour in the ice cream mixture and top it with the remaining graham cracker crust, gently pressing the crust crumbs into the ice cream mixture.
Place ice cream in the freezer for eight hours or overnight. After four hours, the ice cream will be soft-serve consistency and after six hours the ice cream will be firm enough to scoop but eight hours is fully frozen.