Graham was the son of William Jay and Dovie Estelle (Payne) Gryder. He married Pauline Elizabeth Josey and they had three children: Shelby Jean, Larry Graham and Dennis Wayne.