Smith is a native of Norfolk, Virginia, where he graduated from Booker T. Washington High School. Following an all-state high school career, Smith accepted an athletic scholarship to Virginia Tech. Known as "The Sack Man" (both on and off the field) of Virginia Tech football, Smith finished his college career in 1984 as the most honored player in Hokie history. Anticipating his future success in pursuing quarterbacks in the NFL, he had a career total of 71 tackles behind the line of scrimmage, for losses totaling 504 yards. Smith had 46 career sacks, including 22 during a junior season in 1983 that saw him named First-team All-America by the AFCA (Coaches) and Newspaper Enterprise Association. In 1984, Smith capped off his tenure in Blacksburg with the Outland Trophy, given to the nation's top lineman, and a consensus selection to the All-America Team.
Following this stellar collegiate career, Smith was drafted by the Buffalo Bills with the first pick of the 1985 NFL Draft. He quickly became known as a sack specialist, with 15 in 1986 and a personal season-best 19, just two short of the then-NFL record, in 1990. By 1989, Smith, in notching his 52nd sack, had already become the Bills' all-time sack leader, claiming a team record that he was to raise 119 times over the years. Some conjecture that his 171 sacks in Buffalo set a standard that "may be unreachable" for future Bills. In 1990, his defensive performance helped bring the Bills to Super Bowl XXV, though they eventually lost to the Bill Parcells-led New York Giants. Still, Smith had an impressive performance in the game. He sacked Jeff Hostetler in the end zone in the second quarter, becoming only the fifth player to record a Super Bowl safety. Later, Smith forced New York to turn the ball over on downs by tackling running back Ottis Anderson for a two-yard loss on a fourth down conversion attempt. Only a failed last-second field goal attempt kept the team from its first NFL championship (see K-Scott Norwood 'Wide Right').
In 1991, though Smith's knee problems forced him out for most of the season, the Bills once again reached the Super Bowl. In 1992, in much better health, he was again a First-team All-Pro and was voted to the Pro Bowl while recording a team-leading 14 sacks.
By 1996, though the Bills' run of Super Bowl appearances had ended, Smith was still putting up prolific numbers, with 90 tackles and 14 sacks. In 1997, Smith had 65 tackles and 14 sacks and by 1998, although he was getting older he still had a respectable 50 tackles and ten sacks plus 100 others.
After the 1999 season, Smith signed with the Washington Redskins as a free agent. In his first season, he posted 58 tackles and ten sacks, although he was now playing in mostly passing situations. He pressed onward in pursuit of Reggie White's all-time sacks record (198, achieved in 15 seasons), which he passed in Week 14 of the 2003 NFL season by sacking New York Giants quarterback Jesse Palmer in a 20-7 win at Giants Stadium.  Smith finished his career with 200 career sacks.
Smith had hinted in interviews that 2003 would be his final season. However, he never completely ruled out continuing to play. But on February 24, 2004, the Redskins released Smith, saving $6.5 million in salary cap space.
In his 19 NFL seasons, Smith played in 279 games, amassing 200 sacks, two interceptions, 46 forced fumbles, and 15 fumble recoveries, which he returned for 33 yards and a touchdown. Of his 19 seasons in the NFL, 13 of them were seasons where he had at least ten sacks, a testament to his consistency year in and year out. He was also named All-Pro nine times. His 200 sacks give him the record for most career quarterback take-downs. As Smith spent most of his career in a 3–4 defensive scheme, a defensive scheme not geared toward creating sack opportunities for defensive ends, many consider the record particularly impressive. Indeed, Smith's peers elected him to the Pro Bowl every season from 1987 to 1998 (with the exception of his injury-laden 1991 season). In 1987, he was named the Pro Bowl MVP. Smith was twice named the AP's NFL Defensive Player of the Year (1990, 1996), twice named the NEA Defensive Player of the Year (1990, 1993) and four times named UPI's AFC Defensive Player of the Year (1987, 1988, 1990, 1996).
On August 13, 2008, he was part of the inaugural class to be inducted into the Hampton Roads Sports Hall of Fame, an institution honoring athletes, coaches and administrators who made contributions to sports in Southeastern Virginia. Smith was inducted onto the Buffalo Bills Wall of Fame during halftime of the Bills' September 21, 2008 game against the Oakland Raiders.
In a particularly rich weekend for the Bills organization, Smith was joined in 2009 induction to the Hall of Fame by Buffalo Bills owner and founder, Ralph Wilson, Jr.. Smith's former defensive coordinator, Ted Cottrell, the architect of the sack-rich Buffalo years, served as his presenter during induction.
Smith now lives in Virginia Beach, Virginia. Having returned to his home state, Smith now works as a large-scale hotel designer, undertaking many projects with Armada Hoffler. Most recently, he returned to Blacksburg, the site of his collegiate successes, where he purchased the Red Lion Inn. He built a Hilton Garden Inn Hotel with 137 sleeping rooms and is currently redeveloping the site (Smith's Landing, hotel and restaurant complex). A Baptist, he is a member of Queen Street Baptist Church in Norfolk. Smith and his wife Carmen have a son, Alston.
Smith also works with Thurman Thomas in their new business venture, Legends Energy Group. They promote energy programs across North America.