Byron Gilman was born in Lynn, Massachusetts, in 1843, the son of Joseph Gilman, a millwright, and Louise J. Gilman. He was trained in music, played the flute and violin, and experimented with other instruments. In 1861, Byron was working as a clerk with Boyce & Nichols, a "shoe findings" establishment, in Boston. On August 20, 1862, he enlisted in the Union Army as a member of the Eighth Regiment Massachusetts Volunteer Infantry, to serve for nine months as a musician and was subsequently stationed in New Berne, North Carolina. He was involved in some experimentation with a new kind of signaling system employing whistles instead of bugles. Although in the vicinity of several battles, he saw no action, and mustered out of the service on August 7, 1863.

After being mustered out, he was engaged for some months in recruiting efforts for the Heavy Artillery and attempted unsuccessfully to obtain a commission for himself. He worked for a time in a machine ship, doing repairs and projects for local industries. By the summer and fall of 1864, determined to see his fortune abroad, he attempted to find a berth on a ship going to China. When that proved unsuccessful, he tried to find a place with the Panama Railroad. Gilman went to California soon afterward and was first engaged on the Pacific steamers and later as secretary of a mining company in San Francisco.*

*Much of the biographical information is derived from an insert in the front page of Gilman's diary, provided by John Albree.