Chigumbura finished on 52 not out and shared an unbroken stand of 55 with Prosper Utseya as Zimbabwe chased down a target of 210 to win with two overs to spare.
Zimbabwe's only previous win over the Australians was at Trent Bridge in the 1983 World Cup, in the first ever encounter between the two sides.
Sunday's defeat was a humbling experience for Australia, who misread conditions and left out legspinning all rounder Steve Smith, and underestimated Zimbabwe by resting premier fast bowler Mitchell Johnson.
It proved to be even costlier as inspirational captain Michael Clarke re-injured his hamstring and will fly home on Monday.
"At the end of the day the selectors pick the team and I try and get the best out of the players," Clarke said.
"That's all you can do. I think we've seen through Smithy's career that he's a pretty good player of spin bowling and he can bowl some handy leg spinners, so I think he would have been handy out there."
Australia now need to beat South Africa on Tuesday to assure themselves of a place in Saturday's final.
The hamstring injury had kept Clarke out of Australia's first two games, and it flared up again to force him to retire hurt on 68 as Australia limped to a total of 209 for nine.
Although he returned to the crease for the final two deliveries of Australia's innings and bowled an over of spin at the death, he confirmed that he will take no further part in the tournament.
Clarke's knock had held the Australian innings together as they crumbled against Zimbabwe's four spinners, who reduced the tourists to 57 for four and then 150 for seven.
Australia's final total could have been even more measly had Brad Haddin not been dropped three times on his way to 49, an innings that helped the visitors put on 55 runs in the last five overs of their innings.
With Clarke injured and part-timer Aaron Finch and Glenn Maxwell unable to stifle the Zimbabwean batsmen, it fell to Nathan Lyon to put in a matchwinning performance on a pitch that produced prodigious turn throughout the day.
Although Lyon struck in his first two overs to snaffle out a promising opening partnership, and then picked up two more scalps as Zimbabwe slid from 100 for two to 106 for five, the lack of spin support ultimately proved costly.
Zimbabwe had another wobble when Malcolm Waller and Donald Tiripano fell in quick succession, and at 156 for seven it looked as though they had thrown their opportunity away.
But Chigumbura and Utseya steadied the ship to edge Zimbabwe closer to the target in front of a joyous crowd, before Utseya sealed the match with a six over midwicket.
"It was a proud moment for everyone - for Zimbabwe cricket and even for Zimbabwe because it's been more than 30 years since we beat Australia," Chigumbura said.
"Hopefully we can learn from this and have more success to come."
