You can't have this guy around constantly, and not because he's in danger of being a sweeps stunt: Because Alicia is way more interesting than that. And it's because every time he interacts with her -- just like other people mentioned above -- their story evolves. For the purposes of this episode, though, the emphasis happens to be on my favorite thing about their relationship, which is that he likes her, and she likes him, and she hates both of those things.
But then, attraction/repulsion isn't a new thing for Alicia, is it? She likes Cary, she likes Kalinda, she likes Will, either in spite of or because of their darkness depending on the day. Her relationships with dispassionate people like herself (Diane) are usually cool and dispassionate, because there's no attraction/repulsion, just respect -- and her relationships with volatile people (Eli, Owen) are what really get her going.
In fact, the only person she really resents for not being perfect is Peter -- and Kalinda, now -- because they know her too well. She would probably paint it as an opposites-attract thing, if she were capable of thinking about it at all, but the truth is that she's got a rat the size of Chicago and half of her shit is about keeping anybody from knowing that, especially herself.
Which is why it only makes sense that Celeste accompanies her to visit this mystery man CS. I think this is Celeste's last episode for a while, meaning that Alicia needed to come to some kind of resolution with her, and that means coming to terms with what she means in Alicia's life, just like everybody else. Is she another Weirdo Tutor, too close for comfort? Is she a possible replacement for Kalinda (and does that make Will a replacement for Peter)? Is she just the alternate-universe, broken version of Alicia herself? An annoyingly manipulative Patti, or legitimately dangerous Colin Sweeney? I don't know about you, but the answer surprised the heck out of me, in the best way, and it starts here, as they're walking through the jail.
Celeste: "And here I didn't think we'd get a chance to work together. Isn't this fun? You and I should get some drinks, trade whore stories."
Celeste: "Yeah! What did I say?"