In some cases they may only have outline permission - which agrees to the principal of building - but the details do not come forward, while some have planning permission which is close to elapsing.
A host of reasons can be behind the delays, such as archaeology finds, developers sitting on sites in order to wait for them to become more profitable, or delays in agreeing contributions and legal agreements for local infrastructure.
The action plan has 23 points to help tackle the issue, which includes finalising the local plan, engaging with developers earlier to tackle problems sooner, and quicker sign off on legal agreements and contributions.
The joint local plan which has gone out for public consultation identified a need for 17,568 new homes across the two districts by 2036, meaning the stalled sites represent more than 62% of the districts' housing requirement.
But Mid Suffolk's opposition Green group has said the issue hasn't been taken seriously.
Councillor Andrew Stringer said: "We have called for an officer to be part of the council to drill down into this and make sure when houses are approved they get delivered.
"There is a huge amount of work to be done in that plan and I think we should be further down the line.
"We need to have really accurate knowledge about what developers are doing and where the market is, it needs open and frank conversations.
"We also want to see the district play a bigger role in building a lot more homes. Unless they are going to be building a couple of homes a week they are not taking it seriously."
Published in the East Anglian Daily Times by Jason Noble, Local Democracy Reporter.