Since when does being a "watercooler show" mean everything?
Grey's Anatomy, ER, Desperate Housewives, Two & A Half Men, Shameless, The Walking Dead, Supernatural, Modern Family, The Good Wife, Scandal, Person Of Interest, etc? All watercooler shows, all remained on television long after their ratings began to decline.
Even 'mainstream' television's current champ, 'This Is Us' has gone from averaging low 3's to low 2's in the desired demo, over the course of a single season.
Empire still has plenty of life in it, especially for a FOX show, despite what a certain subset of fans would like to believe.