Nelms began his professional career after being drafted by the Buffalo Bills. He was the last cut in training camp. He then signed with the Hamilton Tiger-Cats of the CFL in 1977. Later that year, he was optioned to the Ottawa Rough Riders.
Nelms is a three time Pro Bowler in 1980, 1981 and 1982. He led the NFL in yards per kickoff return twice (1981, 1982) and was the Redskins starting punt returner in Super Bowl XVII. In Super Bowl XVII, he returned six punts for 52 yards, both Super Bowl records, along with two kickoff returns for 44 yards.
Nelms finished his five seasons with 175 kickoff returns for 4,128 yards, 212 punt returns for 1,948 yards and two touchdowns, and one interception in 68 games.