Joe Hockey understands the depth of feeling over his advice to first home buyers, but insists the federal government is helping them by working to create more, better paying jobs.
The federal treasurer sparked a wave of criticism when he told Australians wanting to buy a house to get "a good job that pays good money".
"Let's not play the man, let's actually deal with the policy," Mr Hockey told ABC Radio on Wednesday, adding he understood the depth of feeling over his comments.
"That's why we are working so hard ... to build the jobs, to build the opportunities for people to get ahead."
Mr Hockey acknowledged that for a lot of Australians in Sydney, Melbourne and some other parts of the country housing was very expensive.
"I totally understand that," he said.
The treasurer maintains increasing the supply of housing will address the issue.
"We are going to work with the states to try and increase the amount of supply in the marketplace," he said.
Labor's finance spokesman Tony Burke said the treasurer was having "crazy rants" day after day without any factual basis.
He also said there were many policy levers that could help improve affordability because housing prices were influenced by demand as well as supply.
The opposition is open to including changes to negative gearing in any discussion.
"We've ruled out anything that would be retrospective and flatly ruled out anything that would have a negative impact on supply," Mr Burke said.