Smith was a consensus All-America choice as a senior at USC in 1980. He played in two Rose Bowls for the USC Trojans. He played in a secondary which included three future NFL All-Pros: the San Francisco 49ers' Ronnie Lott and the Minnesota Vikings' Joey Browner in addition to himself. Smith was a consensus All-America choice as he lettered four times in football and three times in track. He posted 205 career tackles and 16 interceptions. He was inducted into the USC Ring of Fame in 2001. Smith played at Southern California (1977–1980) for John Robinson in a star-studded defensive backfield and is a member of the USC team that won the national championship in 1978.
Smith established himself as one of the most feared and hardest hitting safeties in the NFL (a reputation later held by his protégé, Steve Atwater). Smith was voted to play in six Pro Bowls (following the 1985–1986, 1989–1991 and 1993 seasons), was named All-American Football Conference (AFC) in 1984 and 1988.. He played on three Broncos Super Bowl teams (XXI, XXII, XXIV), and was named All-Pro four times.
Smith's career totals include 1,171 tackles, 30 interceptions and 15 sacks. He posted a career-high five interceptions in 1991. He ranks fourth all-time among Denver Broncos in games played with the franchise. with 184. He is also the Broncos leading tackles leader of all time with 1152 recorded tackles.
Smith was named the Denver Broncos's most inspirational player in 1992. He was inducted into the Denver Broncos Ring of Fame in 2001 and the
Colorado Hall of Fame in 2006.