Sullivan had many operations on her eyes during her lifetime. Two unsuccessful operations were performed at Tewksbury. In February 1877, she was sent to the Soeurs de la Charite hospital in Lowell, Massachusetts, where she had yet another unsuccessful operation. She remained there until July, helping the nuns in the wards and going on errands in the community.
In July 1877, she was sent to the city infirmary where she had one more unsuccessful operation. (It would be another four years, after entering the Perkins School for the Blind, that she would have a successful operation.)
After nearly six months away from Tewksbury, she was returned there kicking and screaming. Anne now fully realized the horror of the almshouse. This time at Tewksbury, instead of returning to the area with predominantly ill and insane patients, she was housed with single mothers and unmarried pregnant women. Many years later, she wrote the following in an essay of her thoughts entitled Foolish Remarks of a Foolish Woman:
I have endured much physical pain, and I can feel real pity for any one who suffers. The misfortunes of the disinherited of the world rouse in me not only compassion but a fierce indignation.