Ministers are planning to cut more than 10,000 civil service jobs as Whitehall departments battle to stay within spending limits under a new government efficiency drive, the Guardian has learned.

Multiple sources said there was an acceptance that the civil service had become too big and unwieldy after expanding owing to the demands of Brexit and the Covid pandemic.

With Rachel Reeves, the chancellor, having ordered departments to find 5% cuts to their budgets as part of a spending review, insiders said job cuts would be inevitable.

One Cabinet Office source said departments would have to make some “very hard choices” about headcount if they were going to stay within budgets set by the Treasury.

Pat McFadden, the Cabinet Office minister, refused to be drawn on job cuts when pressed on the subject earlier this week, but gave a speech outlining how he wanted the civil service to operate “more like a startup”.

“We don’t have a target for headcount,” he said on Monday. “My focus is on making the people who work for us into being as productive as they possibly can. That’s why those efficiencies that the chancellor announced in the budget are important and there will be more to come. And technology should help us become more productive in the future.”

There are 513,000 full-time civil servants in central government, up sharply from a recent low of about 380,000 in 2016.

Over the summer, the new Labour government abandoned the Conservatives’ target of cutting 66,000 civil service roles – but more than 10,000 jobs are expected to be slashed.

The Cabinet Office is understood to be one of the departments considered to have grown too big. Parts of it could be broken up, or some staff could be redeployed to other parts of Whitehall. Staff across Whitehall could also be cut through a continuation of an overall hiring freeze, or voluntary redundancies.

Mike Clancy, general secretary of the Prospect trade union, said: “We need a clear plan for the future of the civil service that goes beyond the blunt headcount targets that have failed in the past.

“This plan needs to be developed in partnership with civil servants and their unions, and we look forward to deeper engagement with the government in the coming months.”

Alongside the spending review in June, the first ever strategic workforce plan for the civil service will be published, looking at the size and shape of Whitehall and whether it is fit for modern government.

Asked whether there was a need to reduce headcount in the civil service, Cat Little, the permanent secretary of the Cabinet Office, told a committee of MPs last week: “It is absolutely imperative that the civil service becomes more efficient, more productive and takes advantage of technology to become less dependent on people.”

Pressed on whether it was her position that the civil service needed to do things differently and whether that might be associated with a headcount reduction or a redeployment of people, Little said: “Exactly, and it has to work in tandem with the work we are doing on reducing consultancy and contingent labour.”

She added that a number of departments had already begun “voluntary exit schemes” allowing redundancy, but that it mattered how those were designed to make sure high-performing staff were encouraged not to leave.

The civil service was made up of about 490,000 people when Gordon Brown left office in 2010 and was subsequently cut to about 380,000 after years of David Cameron and George Osborne’s squeeze on Whitehall during the austerity years.

However, it steadily rose during Brexit and the pandemic to reach its current level of 513,000. Last year it grew by nearly 5%.

Alex Thomas, programme director at the Institute for Government, which had advocated for a workforce strategy for the civil service, said it was sensible not to have a fixed headcount reduction target.

Two years ago Jacob Rees-Mogg, the Tory minister for government efficiency, outlined a plan to cut the civil service by 66,000 people.

Thomas said: “The problem with a headcount target is that it creates a lot of perverse incentives in the system, usually to get rid of larger numbers of cheaper people who are easier to lose. It is not the most efficient way of doing things.

“But setting a reduced budget or some kind of efficiency target is more reasonable, and likely to lead to fewer people being employed by the civil service, you might expect, but it is doing that without creating perverse incentives.”

The prospect of tight budgets and job losses in the civil service may dent morale at a time when Whitehall staff are also frustrated by Keir Starmer claiming in a speech last week that many had become comfortable with failure.

“Too many people in Whitehall are comfortable in the tepid bath of managed decline,” he said in a speech on Thursday setting out new policy targets.

“I totally get that when trust in politics is so low, we must be careful about the promises we make. But across Whitehall and Westminster that’s been internalised as ‘don’t say anything’, ‘don’t try anything too ambitious’, ‘set targets that will happen anyway’.”

That prompted Dave Penman, the general secretary of the FDA union for senior civil servants, to write to the prime minister urging him to rethink his “frankly insulting” criticism of Whitehall.

Starmer replied to Penman on this week saying he recognised the “unwavering and invaluable sense of public service” provided by civil servants, but emphasised that there were too many cases across government where the current approach to delivery was not working.

A government spokesperson said: “We are committed to making the civil service more efficient and effective, with bold measures to improve skills and harness new technologies.”