Spencer signed with the Toronto Argonauts in December 2014. Spencer had a productive two seasons in Toronto, catching 107 passes for 1,208 yards with six touchdowns. He also returned 39 punts and 27 kickoff returns.
On the second day of free agency Spencer signed with the Ottawa Redblacks. On October 27, 2017, in a game against the Hamilton Tiger-Cats, Spencer set a CFL single-game record with 496 all-purpose yards: 133 yards receiving, 165 kick-off return yards and 169 punt return yards. Overall, Spencer had an exceptional 2017 campaign with the Redblacks, catching 71 passes for 922 yards with six touchdowns. He also returned 70 punts and 25 kickoffs, and was named a CFL-East All-Star as a returner. On January 31, 2018 Spencer had a workout with the NFL's Baltimore Ravens. Nevertheless, less than one week before becoming a free agent Spencer and the Redblacks agreed on a contract extension on February 7, 2018. Despite recording his first 1,000 yard season in 2018, Spencer did not make the CFL-East All-Star list as a receiver, but did make it as a returner for the 11–7 Redblacks, who finished the year at the top of the division. After advancing to the CFL championship game, Spencer had a costly fumble on a return, and the Redblacks lost the 106th Grey Cup to Calgary, 27–16.