Miller was born in Midland, Texas. He attended Robert E. Lee High School in Midland, and was a standout high school football player for the Lee Rebels.
He attended the University of Nebraska, where he played for coach Tom Osborne's Nebraska Cornhuskers football team from 1976 to 1979. As a senior in 1979, he was recognized as a consensus first-team All-American.
The Atlanta Falcons chose Miller in the first round (seventh pick overall) of the 1980 NFL Draft, and he played for the Falcons from 1980 to 1983. He was selected for the Pro Bowl in 1980 and 1981. He played his fifth and final NFL season for the New Orleans Saints in 1984.