Clay Arsenal is one of Hartford's oldest neighborhoods, developed in the middle and late 19th century. It was mainly farmland prior to 1847, when the Hartford-Springfield Railroad, which now forms the neighborhood's eastern border, was constructed.

The section west of Main Street lies on a gentle rise above Downtown and is known as Clay Hill, so named for the type of soil there. The area east of Main Street has been known since 1812 as the Arsenal District, when a State Arsenal was constructed. The Arsenal was demolished in 1909.

Multi-family dwellings were the dominant development in the late 19th century as the neighborhood became home to Irish and Jewish working class families. The Irish had been emigrating to Hartford through a recruitment effort for work on the Enfield Canal in Windsor Locks. In 1895, Clay Hill was predominantly Irish and was served by Saint Patrick's church in downtown Hartford.

At about the same time, large numbers of Jews began arriving from Eastern Europe due to the anti-Semitic sentiment there. The new emigrants established themselves in the Arsenal district as grocers, tailors, butchers and jewelers along Main Street and Albany Avenue.

The African-American community grew significantly during World War I when large numbers of southern blacks were hired to work the tobacco fields and sheds in the state's booming shade tobacco industry.

After World War II, the area began to see a growing number of Puerto-Rican families, many of whom came to work the tobacco fields.

Although he area has been primarily residential, in the post-Civil War era, the railroad attracted businesses, including a lumber yard, brewery and carriage works. Additionally, the Hartford County Jail was built in 1873 on Seyms Street. Designed by Hartford architect George Keller, it embodied the High Victorian Gothic style. The structure was demolished in 1978.