Tarmac Building Materials in Little Lever will be demolished to make way for 132 homes.
Harworth Group received the green light for its plans which were unanimously approved.
Now a full planning application will be submitted.
The 6.2 hectare site is made up of the former factory with the northern third of the site being previously undeveloped green belt land.
The greenbelt land will not form part of the plans.
A representative of the developer told Bolton Council’s planning committee that developers hope to be on site to clear the area by the middle of 2022 and within five years over half - 90 - of the homes would be built.