James Britton Gantt was born in Kansas City in 1911. Before he was a year old, his family moved to New Orleans where his parents soon separated. For much of his youth, Gantt was raised and educated by relatives. By his own account, when he set out on his own, he worked on farms, punched cattle, worked in a circus, washed dishes and did most of the other jobs a Westerner could find in those times.

Gantt had come a long way by the time he enrolled at the Dallas Art Institute in the fall of 1933. The next year he applied for and received a scholarship to the Kansas City Art Institute, where he initially studied with Ross Braught. Benton arrived at the Institute the following year, and Gantt studied with him for a full year and later, after graduation, worked as an apprentice with him during the preparation of his Jefferson City Murals.

Gantt exhibited regularly in the Midwest. An early success was his painting Kansas Pastoral which was included in "American Art Today" at the 1939 World's Fair. The narrative of Gantt's subjects echoes his upbringing, often depicting raw American life, ordinary Western folk caught in the throes of everyday life. As he once remarked, "I paint only what I have lived and experienced, what I know from the ground up."