All started well for Latics as Billy Mckay fired them into a 16th minute lead before three quick goals turned the game on its head.
Manager Stephen Robinson made changes at half time and Darius Osei quickly made an impact, notching two goals in a minute to draw Latics level.
Ollie Banks put Athletic ahead with a stunning volley but Tom Miller equalised for Carlisle United with 15 minutes remaining to equalise.
Macauley Gillsephrey saw red for a last man challenge on Crystal Palace loanee Freddie Ladapo before the 10 men won the game in the last minute through Michael Raynes.
Robinson made a host of changes to his side with only Burgess, Charles Dunne, Ryan Flynn and Billy Mckay retaining their places from the weekend’s draw against Bradford City.
The gaffer elected to start new signing Ladapo following his loan move from Crystal Palace and he showed some early promise in front of goal.
However, it would be Carlisle United who had the first chance of the game as Burgess was forced to clear off the line from Shaun Miller on eight minutes.
Shaun Brisley had two opportunities to put the visitors ahead, one from the resulting corner but he headed over the crossbar from eight yards.
The defender had another chance three minutes later as Jamie Devitt’s delivery found the onrushing Brisley but he headed wide this time.
It would be Latics that took the lead, slightly against the run of play through Mckay.
Ladapo received the ball on the edge of the box and laid it into the path of the Wigan Athletic loanee and his powerful strike found the bottom corner to give Latics the advantage.
However, a 10 minute period saw Latics fall 3-1 behind.
Devitt’s corner into the box found Raynes on 18 minutes to equalise whilst four minutes later, Carlisle took the lead.
After neat play on the edge of the box for Carlisle, the ball fell to Lambe who converted from 18 yards to give the visitors the lead.
Latics did have a chance to equalise through Mckay but the striker’s effort from inside the penalty area was well saved by Max Crocombe.
On 28 minutes it was 3-1 to the visitors as Shaun Miller got to the byline, cut the ball back across but the unfortunate Burgess knocked the ball over the line as Carlisle took further control.
Athletic needed a way back into the game but Luke Woodland’s overzealous challenge on Danny Grainger earned him a booking.
Palace loanee Ladapo did have a goal disallowed with five minutes of the half remaining.
He raced onto a through ball from Mckay and rolled the ball underneath the goalkeeper but the linesman’s flag as up for offside.
Robinson clearly gave his players a stern talking to at half time and he brought on Lee Croft and Osei at the start of the second half.
The pair made an immediate impact on the game as Latics pulled a goal back.
It would be Croft and Osei who combined for the goal with the winger laying on the assist, with the help of a dummy from Ladapo.
Osei took the ball in his path and placed the ball underneath the goalkeeper to give Latics the momentum.
And they would make that pay only a minute later as the striker notched his second of the game to draw the scores level.
Mckay was played in down the left and fed a teasing cross to the back post where Osei stood and he converted from close range to equalise.
Latics were on top and whilst Ladapo struck over the bar from distance, Ryan Flynn almost turned the game on its head just before the hour mark.
The midfielder received the ball on the edge of the box from a cut back but curled inches wide of the post.
After Banks was booked for a late challenge on Jamie Devitt, he quickly made amends for that, firing Latics into the lead.
His initial free kick from 25 yards hit the wall but from the rebound, he volleyed into the top corner to put Robinson’s men ahead.
In what was a thrilling contest, Carlisle managed to equalise with only 15 minutes of the game remaining.
A set piece into the box was not dealt with by Latics and Tom Miller converted from close range to pull the scores level, again.
Robinson brought on Carl Winchester straight after the goal and he curled an effort over the crossbar as Latics continued to search for a winner.
The visitors were reduced to 10 men on 80 minutes as Gillesphey was shown a straight red card for fouling Ladapo as he raced through on goal.
Latics went in search of the winner but it would be the visitors who hit them on the break and clinched the game 4-5.
In the final minute of the game, another set piece into the box found Raynes who bundled the ball into the back of the net to win the game for the visitors.
Deep into five minutes of additional time, Banks’ delivery found Osei in the centre but the striker headed against the crossbar as Latics fell to defeat.
Oldham Athletic: Kettings, Wilson, Burgess, Edmundson, Dunne, McLaughlin (Osei 46’), Banks, Woodland (Croft 46’), Flynn, Mckay (Winchester 75’), Ladapo.
Subs not used: Ripley, Stott, Mantack, Fane.
Carlisle United: Crocombe, Grainger, Joyce, Raynes, Jones (Penn 90+2’), Gillesphey, Brisley (Wyke 53’), Devitt, Lambe, S.Miller (Brough 77’), T.Miller.
Subs not used: Gillespie, Atkinson, Ellis, McKee.
Goals: Oldham Athletic (Mckay 16’, Osei 48’, 49’, Banks 66’) 4-5 Carlisle United (Raynes 18’, 89’, Lambe 22’, Burgess OG 28’, T.Miller 74’)