It's about whether it belongs in the anime and even if we feel like it doesn't they need to back it up. Take Food Wars, it's basically a cooking battle shounen anime, and there's no place for fan-service but that's probably a strong point of the series, making taste something sensual. Kill la Kill as well.
I think fan-service feels out of place when for no reason they shove a boob shot or something, a good example would be Fairy Tail. In most fantasy and high school shows it's so out of left field.
As far as echi anime are concerned, while one of the main goals is fan-service, I feel like they need to own up to it and that can only start from the protagonist itself. Like Hyoudo Issei from High School DxD, who's a class A pervert.