Charles Augustus Benson was born into the oldest and largest black family in Framingham, Massachusetts, on January 26, 1830, to George and Elizabeth (Simpson) Benson. His great-great grandparents were both brought to Massachusetts as slaves from Africa; their son William, Charles' great-grandfather, attained his own freedom shortly before Charles' grandfather, Abel, was born in 1766. By the time Charles was eighteen he had found a position as either a cordwainer or shoemaker, and moved from his family home in Framingham to Natick. In February 1848, he married Martha Ann Thompson, and their daughter Henrietta was born in March 1852. Later in 1852, Charles left his daughter and pregnant wife in Natick, and moved to Salem, Massachusetts. Mary, Charles' second daughter was born in early 1853; she never knew her father.

Soon after his arrival in Salem, Charles got a job with a black oyster dealer and restaurateur, James Sherman. Charles married his second wife, Margaret Jenny Francis, in early 1853, although he was never legally divorced from Martha. Jenny was the daughter of Joseph Francis, a mariner. In October 1853, their daughter Fransisca G. was born. In need of greater financial opportunity, Charles accepted the job as cook and steward aboard the merchant brig Gem, which was bound for Canada in early 1854. Finding success as a mariner, Charles would also serve on board the trading bark Miquelon, the brig M. Shepard, the bark Elizabeth Hall, the bark Nubia, the bark Wyman, the bark Swallow, the bark Taria Topan, and the bark Glide.

After ten years at sea, Charles returned home to Salem in 1864, where he remained until the summer of 1875. While in Salem, Charles worked as a cook and in other low-paying jobs. During this time, Charles and Jenny had four more children: Charles W., Ida Lois, Hannah J., and William Henry D. Not finding enough financial opportunities on shore, in the summer of 1875 Charles accepted the position as steward, returning to a life at sea. He died from an illness at sea on July 12, 1881.