The decision was taken by the MQM's coordination committee in a meeting held simultaneously in Karachi and London on Sunday.
"We have decided to sit on opposition benches because the government has not done anything to address the issues we have been protesting about," MQM stalwart Faisal Sabzwari told media persons.
"It has become difficult for us to face our supporters and workers. In the first phase our ministers resigned and now we have decided to sit in opposition," Sabzwari said.
"We will stay in Sindh provincial coalition and have decided to quit government at federal level because all these issues are related to federal government," he said.
The MQM meeting vehemently deplored the steep rise in petroleum prices, saying price hike has struck the masses hard. It said due to poverty poor people in Pakistan are compelled to commit suicide, and sell their children.
It was also discussed in meeting that poverty, unemployment and inflation have made life miserable people while increasing petroleum prices is an atrocity to poor people, said a statement issued after the meeting.
The meeting said on one hand government is levying taxes without any reason, introducing mini budgets instead imposing taxes on feudal and landlords and on agricultural sector. The meeting also lambasted growing corruption and lawlessness in the country.
It said that MQM would support the government on its positive steps while sitting on opposition benches; however, it would also record their protest on wrongdoing of government.
The meeting demanded of the government to withdraw hike in petroleum products, control inflation and withdraw unjust taxes, eliminate corruption and take steps for the betterment of law and order situation.
Meanwhile, Chief MQM Altaf Hussain has ratified the decision of MQM Coordination Committee.
Analysts said the government is now scrambling to find new partners, but that without them, new elections are likely. The ruling coalition held 181 seats - including the MQM's 25 - in the 342-member parliament. The MQM's departure leaves Gillani well below the 172 seats needed to preserve its majority.
However Prime Minister Syed Yousuf Raza Gillani denied that his government was in danger of collapsing. "I don't see any crisis," he said, speaking on television after the announcement.
Earlier, President Asif Ali Zardari had personally led talks with the MQM in a last-ditch effort which failed to bring the Muttahida back into the fold.
Interior Minister Rehman Malik - the party's chief mediator - had earlier failed to achieve a breakthrough after Sindh Home Minister Dr Zulfiqar Ali Mirza claimed the MQM was involved in violence in Karachi.
MQM, was the second largest party in the alliance with 25 lawmakers in the 342-member National Assembly.
The MQM had last week announced its decision to withdraw from the PPP-led coalition government after a falling out over Home Minister Zulfikar Mirza's statement regarding target killings in Karachi.
It had pulled its two ministers out of the federal Cabinet, claiming that they were given "impotent" portfolios, just weeks after another alliance partner JUI parted ways with the government.
APP ADDS: MQM chief Altaf Hussain on Sunday said that only hard-working and people of poor middle class can save the country.
He was addressing via telephone from London the office-bearers of his party's student wing APMSO.
He said the country can not be taken on the path of progress and development without elimination of feudal system.
He called upon the people to support his party for the better future of their generations.