He says he visits whenever he's in the city and on Tuesday, he got to try his hand at making the restaurant's signature Irish coffee.
"I've always wanted to make an Irish coffee because I've watched hundreds of them be made. I thought I had it down. But to actually do it, it's a little trickier. There's some finesse," he said.
Kelly also visited Swan Oyster Depot and Zuni Café.
You can watch "The Chew" every weekday at 12 p.m. on ABC7.