No explanation exists as to why Anne and Jimmie were sent to the almshouse, while their sister Mary was sent to an aunt's house. However, Nella Braddy Henney, in her biography entitled Anne Sullivan Macy, suggests that Anne and Jimmie were harder to handle than Mary; Anne because she misbehaved and was contrary, and Jimmie because he suffered from a tubercular hip.

Upon their arrival at Tewksbury, Anne successfully protested against attempts to separate her from her brother. As a result, both siblings were sent to the women's ward, where inmates included women who were physically and mentally ill:

Very much of what I remember about Tewksbury is indecent, cruel, melancholy, gruesome in the light of grown-up experience; but nothing corresponding with my present understanding of these ideas entered my child mind. Everything interested me. I was not shocked, pained, grieved or troubled by what happened. Such things happened. People behaved like that—that was all that there was to it. It was all the life I knew. Things impressed themselves upon me because I had a receptive mind. Curiosity kept me alert and keen to know everything.