The Town Library was originally established in the Congregational Church basement.
Later it was moved to the town hall by Ledyard Bill in the area which is now occupied by two small restrooms. Since books were very expensive and few in number, most were frequently donated by wealthy residents. This necessitated that only one book could be borrowed at a time by everyone regardless of age. The book had to be returned in good condition before another one could be borrowed.
In 1926, Richards Memorial Library was established through the donation of land and funds by Mr. Ellis G. Richards, a wealthy New Yorker who enjoyed summer vacations in Paxton before moving here. It was built on the site of the old Goddard family home at the corner of Maple St. and Richards Ave. He also established a trust fund to help support the library's programs in the future.
A much needed addition was added to the library in 1976. Six Trustees are currently elected by the town to help oversee the library's maintenance and development. Approximately 32,000 volumes are now available at the library. It eventually became a member of a local library consortium, (CWMARS) Central/Western Massachusetts Automated Resource Sharing, Inc.
The CWMARS program offers a shared online computer system and combined collections of more than 9 million items.