Early Caldwell's wife was Minnie Williams (1884-?). They had five known children named Joel Caldwell, Veatrice Caldwell, Theola Caldwell, Mattie B. Caldwell, and Early , Jr. Caldwell.
