Allen's career started against the New York Giants, a game where the Giants recorded eight sacks. By midseason, Allen was relegated to Special Teams. During the offseason, Allen moved to the center position. Eagles coach Buddy Ryan thought so little of Allen that he once described him as someone who could only be useful "if you want someone to stand around and kill the grass... he looks like a USFL reject." In 2011, Deadspin ranked Allen the fourth-worst NFL player of all time, noting "The ninth overall pick out of Indiana had a special way of blocking opposing pass rushers. He would lean his body forward, then fall down." and "Never had the Eagles had a combination of bad person-bad player that could match this guy."
After a poor rookie season, Allen tested positive for cocaine after reporting to Eagles training camp in 1986. The Eagles released him in October 1986. Days after being cut, Allen and his roommate, Scott Cartwright, were charged with rape. Allen was sentenced to 15 years in prison, serving 33 months in prison, while Cartwright was sentenced to seven years. He was banned from the league for life soon afterward.