The greatest influence on the development of the Sheldon/Charter Oak neighborhood was Samuel Colt, inventor and industrialist. In 1855, he began buying up property and established the Colt Firearms Factory, recognizable by its onion dome topped by a rampant colt. He is given credit for the naming of several streets in the area where the Colt Firearms Manufacturing Company was located. In doing so, he paid tribute to the Native Americans who made it possible for Hartford to exist, as well as the Dutch who first came to Hartford.

Samuel and his wife, Elizabeth Hart Jarvis Colt, lived in a mansion on Wethersfield Avenue overlooking all of their property down to the Connecticut River where the factory stood. The home featured a steel and glass conservatory fashioned after London's "Crystal Palace." The estate included sweeping lawns, an orchard, greenhouses, a carriage barn, deer park, swan and duck pond and paths with statuary.

He also built a village so that he workers could live within walking distance to the factory. It included a community house and library. His employees and their families were encouraged to participate in the educational, cultural and sports activities he provided. Of the fifty brick, multi-family structures originally built, ten remain today.

When willow trees began to overrun his property, he decided to open a factory for the production of willow furniture. Hearing that the best willow workers in the world were in Potsdam, Germany, he imported an entire village of workers to his Hartford location, building German-style homes to make the workers feel comfortable. The community became known as Potsdam Village. Several of these houses also remain today. Examples of wilow furniture and other Colt artifacts may be found at the Wadsworth Atheneum in Hartford.

Upon Colt's death in 1862, his widow, Elizabeth Jarvis Colt, oversaw the firearms business and built the Episcopal Church of the Good Shepherd as a memorial to her husband and their three children who died in infancy. Architectural details of the structure include a variety of gun parts, such as bullet molds, gunsights and cylinders. This unusual characteristic earns the building the title of likely being the only church in the world with a gun motif. One original architectural element that no longer remains is an ornamental cross that topped the west end of the church. It was removed by the Bishop of Connecticut when it was discovered that the cross was made up of revolvers.