Juice the grapefruits by cutting them in half and using a citrus juicer to extract the juice. Strain to remove pulp and seeds if desired.
Make rosemary simple syrup by combining 1/2 cup water, honey or agave, and 2 sprigs of rosemary in a small saucepan. Bring to a gentle simmer over medium heat, stirring until sweetener dissolves completely. Remove from heat and let steep for 20-30 minutes.
Remove the rosemary sprigs from the syrup and strain if necessary. Allow the syrup to cool completely.
Combine the grapefruit juice and rosemary syrup in a large pitcher, adding the optional lemon juice if using. Stir well to incorporate all flavors.
Add ice to serving glasses, filling them about halfway.
Pour the grapefruit-rosemary mixture over the ice, filling glasses about 1/3 of the way.
Top with sparkling water, gently stirring once to combine without losing too much carbonation.
Garnish each glass with a fresh rosemary sprig and a slice of grapefruit on the rim.