The development will add to political instability in France after snap elections in the summer left no group with a majority in parliament.
MPs had to either vote in favor or abstain on Wednesday, with 288 votes needed to pass the proposal. 331 people voted for the initiative.
Barnier is now obliged to resign his government, and the budget that led to his downfall does not exist.
However, he is likely to remain as interim prime minister while Macron chooses a successor.
Both the left and the far-right voted no-confidence after Barnier pushed through social security reforms by invoking a presidential decree on Monday after failing to garner enough support for the measures.
The leftist New People’s Front (NFP) alliance, which won the most seats in the parliamentary elections, previously criticized Macron’s decision to appoint centrist Barnier as Prime Minister over his own candidate.
Nearby far-right National Rally (RN)he found Barnier’s budget – which included €60bn (£49bn) to cut the deficit – unacceptable.
Marine Le Pen, the RN leader, said the budget was “toxic for the French”.
On the eve of the vote, Barnier told the National Assembly that removing him from office would not solve the country’s financial problems.
“We have reached a moment of truth, of responsibility,” he said, adding that “we need to look at the reality of our debt.”
“I’m not happy that I’m proposing tough measures.”
In an interview with French broadcaster TF1 on Wednesday, Le Pen said there was “no other solution” but to remove Barnier.
When asked about the prospects of the French president, she answered: “I am not asking for the resignation of Emmanuel Macron.”
However, Le Pen added that “if we do not respect the vote of the voters and show respect for the political forces and respect for the elections”, then the pressure on the president “will obviously increase and increase”.
