Spencer was out of the game once more, but was still not done.
Five years later, through his work as a fitness coach, Spencer found himself bowling to a young Ravi Bopara in the nets at Rockingham-Mandurah Cricket Club.
"Ravi said 'why are you not still playing?'" says Spencer. "Knowing how good Ravi was - and is - if he thought I could do it, maybe I should play."
After the drugs ban, Spencer felt his time playing for Western Australia was done, so he asked former Zimbabwe and Sussex batter Murray Goodwin if any counties were looking for a bowler.
Remarkably, at the age of 34, Spencer found himself back in county cricket on trial at Sussex in the summer of 2006.
"I thought I was still pretty passionate about the game, but I realised I probably wasn't," says Spencer.
"I was sharp, but not as quick as I was as a young fella. I was fit enough to do it, but my work ethic had gone. I probably went over for the wrong reasons."
Spencer played two first-class matches for Sussex, against Warwickshire and the touring Sri Lanka team. His last wicket in professional cricket was Kumar Sangakkara.
All in all, he took 36 wickets in 16 first-class matches and 23 scalps in 20 List A games.
Nearly 20 years on, Spencer is settled in Perth. He works in the mines in the northern part of Western Australia.
Moody says there is "no question" Spencer could have played international cricket. Spencer says he would have happily played for England or Australia, but the accent is 100% Aussie.
Spencer has no idea how fast he bowled. He thinks he was told he was clocked at 158kph - just over 98mph - but that was "off a short run".
"I was in the wrong era," he says, considering the way modern fast bowlers are managed, or how he could have made a fortune as a T20 gun-for-hire.
"It is one of the great shames that we didn't see enough of Duncan Spencer," says Campbell. "When he got it right - oh my goodness."
Spencer did not collect the wickets, the international caps or the rewards he might have, but he experienced what most can only dream of.
"When I didn't have rhythm, I was as bad as anyone," he says.
"When it all clicked, it was a great feeling. It's effortless. When you get the rhythm it feels like it's coming out medium pace.
"Bowling fast is awesome."