Latics found themselves a goal behind at the half time break as Amadou Bakayoko’s strike from 25 yards was the difference.
However, it was a game Athletic dominated and they deservedly equalised 20 minutes from the end through an unstoppable strike from substitute Lee Croft.
Five minutes later and the game had turned on its head as Cameron Burgess notched his second in as many games to put Latics ahead, whilst a goal in the final minute from Lee Erwin won the game.
Latics now progress through to the last 16 of the competition and face a trip to Field Mill to take on Sky Bet League Two side Mansfield Town.
Manager Stephen Robinson made one change to his side from Saturday as Freddie Ladapo replaced Paul Green.
It was Latics who dominated the opening proceedings of the game and they almost took the lead on six minutes.
Ryan McLaughlin drove at the Walsall defence and unleashed a fierce strike towards the top corner but a wonder save from Craig MacGillivray denied the winger as his shot hit the crossbar.
Two minutes later, the lively Billy Mckay almost capitalised on a cross into the box but again the Walsall goalkeeper was out quickly to deny the striker.
Mckay’s presence up front was causing a problem for the Saddlers defence, and his intelligence won him a free kick on 15 minutes.
A neat free kick routine followed as Ryan Flynn and Mckay combined to set up Ollie Banks from 25 yards but the midfielder’s strong effort was just deflected wide of MacGillivray’s goal.
Latics were almost made to pay for their early dominance on 18 minutes as Florent Cuvelier was presented with an open goal from 12 yards but struck his effort over the crossbar.
The home side started to dictate possession but were met by a stubborn Oldham Athletic defence.
Just after the half hour mark, Athletic thought they had taken the lead.
Cameron Burgess’s back post header forced a smart save from the goalkeeper but it fell straight into the path of Mckay who converted from close range, however, the linesman’s flag was up for offside.
The momentum was with Latics again and a minute later, Freddie Ladapo used his pace and power to knock Matt Preston off the ball, he cut inside and shot but it curled past the post.
With half time approaching, and Latics much the better team in the first half, Walsall took the lead completely against the run of play.
Bakayoko picked up the ball 25 yards from goal and unleashed an unstoppable effort, giving Connor Ripley no chance, and Walsall went into the break with the advantage.
Robinson made two changes at the break and brought on Croft and Carl Winchester, both of whom made an instant impact on the game.
Winchester had the first opportunity of the second half after latching on to a Dunne low cross into the box but his effort went over the crossbar.
Latics continued to do all the pressing in the second half and just before the hour mark, Ladapo’s persistence saw him fire at goal, the ball deflected into the path of Banks but he struck over from the edge of the box.
Mckay almost profited from some route one football minutes later but MacGillivary made a smart save to deny him.
Despite the goal disadvantage it was Latics taking the game to the hosts and on 69 minutes they were deservedly level.
Croft, who had a positive influence on the second half, picked up the ball from 25 yards and unleashed a fierce strike into the bottom corner to draw the scores level.
Five minutes later, and with all the momentum, Latics turned the game on its head and took the lead.
After Banks’ corner was cleared back to him, the midfielder whipped in a teasing cross which Burgess flicked into the far corner to put Robinson’s men ahead.
Latics were doing their best to slow the game down after going ahead and were happy to soak up pressure.
Walsall almost found an equaliser 10 minutes from time as full back Scott Laird opened fire from inside the penalty area but Ripley pulled off a wonder save to deny him.
Whilst Walsall attacked, Latics looked threatening on the counter attack and with only a minute of the game remaining, Erwin notched the third.
And it was all the work of Fane as he won the ball in midfield, advanced on goal, fed Erwin who chopped inside and finished low into the bottom corner from inside the penalty area.
It was what Latics deserved for a confident performance and it is them that progress through to the third round to face Mansfield Town.
Walsall: MacGillivray, Laird, McCarthy, Dobson (Edwards 71’), Cuvelier, Jackson, Osbourne, Kinsella (Moussa 65’), Preston, Bakayoko (Makris 87’), Toner.
Subs not used: Etheridge, Ginnelly, Sangha, Roberts.
Oldham Athletic: Ripley, Law, Burgess, Clarke, Dunne, Fane, Banks, McLaughlin (Croft 46’), Flynn (Winchester 46’), Mckay, Ladapo (Erwin 72’).
Subs not used: Kettings, Wilson, Dummigan, Osei.
Goals: Walsall (Bakayoko 44’) 1-3 Oldham Athletic (Croft 69’, Burgess 73’, Erwin 89’)
Attendance: 925 (71 visiting supporters)