The closest significant ice climbing is in the Auburn Ice Canyon in Auburn Massachusetts off RT 20. As long as it is sufficiently cold there is pretty consistent ice in Wolf Hill Quarry in Smithfield and in Catamint Quarry in Diamond Hill Park, Cumberland. There is also ice climbing at Ross Rocks in nearby Killingly but it is very subject to having a signficant snowpack.
Wolf Hill Quarry, Smithfield
Catamint Quarry, Cumberland (AKA Ice Cream Slabs)
Catamint Quarry is within Diamond Hill State Park. It forms pretty consistently. It is about 40 feet high, slabby, and is rated about a NEI 2. It is located in Diamond Hill State Park on the west side of RT 114. Access is tricky to find. Access is by a road that looks like a driveway south of the Abbot Valley Vet Clinic. Park at the Diamond Hill town park parking lot on the east side of Diamond Hill Road (114) and walk across the street. Walk down the side road to a chain across a dirt road and follow the dirt road past a house. You'll feel as though you're trespassing and the chain across the trail's start, apparently to discourage vehicles, isn't exactly welcoming.but this is state land and a public right-of-way, not private. This was confirmed by the RI DEM. They plan on eventually building a parking area here. Follow the trail to the top of the hill and you will see the slabs. Photo is of the ice in thin "not in" condition.
Ross Rocks, Killingly, CT