Cameron Dummigan gave Latics a dream start, opening the scoring in the opening two minutes of the game.
Posh levelled five minutes before half time as Coulibaly nodded past Joel Coleman, however Liam Kelly’s second half penalty secured the impressive three points which take Latics a point from safety.
Manager John Sheridan made one change to his starting 11 as Carl Winchester replaced Saturday’s goalscorer Jonathan Forte.
He could only name six players on the substitutes bench due to Joseph Mills’ partner going into labour prior to the game starting.
It was Latics that got off to a dream start, taking the lead after one minute and 45 second through Burnley loanee Dummigan.
The full back, playing at right midfield, capitalised on a mistake from Kgosi Ntlhe in the defence and swept the ball past Ben Alnwick to give Latics a dream start.
The goal was also Dummigan’s first goal in competitive football and Latics went in search of another goal.
Fellow Burnley loanee Danny Lafferty was the next to go close as he received the ball on the overlap but his cross was just collected by Alnwick ahead of Curtis Main.
Peterborough started to come back into the game as Coulibaly tested the palms of Coleman on nine minutes.
Two minutes later, Posh really should have got back on level terms as Ntlhe struck goalwards, Coleman spilled the ball into the path of Lee Angol but the striker scooped the ball over from close range.
Posh were enjoying the possession but could not find a way past the stubborn Latics back line.
Midway through the first half, Latics could have doubled their lead on two occasions.
Firstly, Main struck goalwards from 25 yards which Alnwick saved comfortably and two minutes later, Aaron Amadi-Holloway powerfully hit a shot from the edge of the box which he grasped at the second attempt.
On the half hour mark, Lafferty managed to get beyond Holloway on the overlap and his low cross found Main but the Doncaster loanee struck wide of the post.
Latics looked comfortable and were producing some slick passing moves which were pleasing on the eye, however, Posh equalised on 39 minutes.
Maddison’s cross caused Coleman all sorts of problems, and he could only parry onto the crossbar and it was a simple header for Coulibaly to finish to equalise.
Latics were shellshocked as the goal came from nowhere but they almost responded quickly with a free kick.
Palmer delivered the ball into the box and Gerrard forced a good save from the goalkeeper, however the referee spotted a foul in the box and Posh came away with the free kick.
It was a nervy end to the half for Latics as Erhun Oztumer shot wide from the edge of the box after neat build up play around the edge of the box.
Athletic needed half time to regroup and they certainly did that as they took control of proceedings in the second half.
Kelly had a strike in anger on 48 minutes which went over the crossbar but Latics first real chance of the half came on 55 minutes.
A cross into the box by Winchester was attacked by Holloway but the striker just could not direct his header on target.
Two quick bookings followed as Lafferty was cautioned for preventing a quick Posh free kick, however a minute later defender Jack Baldwin was in the book for a cynical challenge on Winchester.
Forte was introduced just after the hour mark for the impressive Main and it would be Latics again who pushed for the winning goal.
Kelly found Dummigan onrushing in the centre, he skipped past one man, shot but the goalkeeper turned the ball behind for a corner.
Latics were putting the pressure on their hosts and went close on 71 minutes as Winchester’s headed effort went inches wide of the post.
A minute later, Latics won a penalty as Alnwick fouled the impressive Palmer as the goalkeeper fumbled Winchester’s powerful effort from distance.
Captain Kelly stepped up and, although the goalkeeper went the right way, it had too much power on it and Latics were in the lead.
Only a minute later, Latics almost put the game beyond doubt as Holloway rose well from a corner to nod the ball onto the post.
However, as he did that, he picked up an injury and had to be replaced by striker Jake Cassidy.
Although it was a tense finish to the game, Latics dominated the possession and looked the more likely side to extend their lead.
Posh did, however, have one opportunity in the final five minutes of the game after a superb turn from substitute Shaquile Coulthirst gave him a sight of goal but he shot wide.
Into five minutes of additional time, good work from Palmer set up Cassidy but he struck over from the edge of the box.
Posh threw everything forward in a bid to get something from the game but it was Latics that came away with the win to secure back-to-back wins for the first time this season.
Peterborough United: Alnwick, Smith, Zakuani, Baldwin, Ntlhe, Beautyman, Oztumer, Taylor (Samuelsen 69’), Maddison, Coulibaly (Coulthirst 69’), Angol (A.Williams 77’).
Subs not used: Henry, Fox, Nichols, Almeida Santos
Oldham Athletic: Coleman, J.Wilson, Dieng, Gerrard, Lafferty, Dummigan, Kelly, Palmer, Winchester, Holloway (Cassidy 80’), Main (Forte 62’).
Subs not used: Cornell, B.Wilson, Brown, Croft.
Goals: Peterborough United (Coulibaly 39’) 1-2 Oldham Athletic (Dummigan 2’, Kelly 73’)
Attendance: 3,544 (183 visiting supporters)