Oooooh, OK, now I see how it's working.
The Never mark alerts as read option is per thread, not per alert. If I have 3 alerts for Thread A, and 2 alerts for Thread B, the alerts icon will show "5". With the option enabled I can click the icon, view the list of alerts, and if I click on...