Patrice Pastor fell in love with Carmel-by-the-Sea as a child when he visited from his native Monaco.

The quaint coastal hideaway has a village feel, with narrow streets, courtyards and cottages – and very few traffic lights and sidewalks.

The billionaire entrepreneur and real estate mogul fell in love with a charming and affluent area of California, embarking on a buying spree in 2015.

He built a remarkable collection of luxury properties, including homes, hotels, and dining establishments, in a town once governed by Clint Eastwood and where Doris Day spent her retirement.

But after 10 years, and an investment of more than $100 million, he has decided enough is enough.

Patrice recently declared he is leaving the ‘peculiar community’—where it’s illegal to wear high heels without a permit—due to his frustrations with its limitations.

The breaking point occurred last week when a significant downtown project he spent six years on was stalled once again. This setback came after a last-minute objection from a group of locals to the city council, despite earlier approval from planning authorities.

The decision left Pastor’s 30,000-square-foot retail and apartment development on Dolores Street in limbo.

Carmel mayor Dale Byrne exclusively told the Daily Mail the latest pushback against Pastor reflects ‘badly’ on the charming town.  

‘Most people I’ve spoken to are upset because he’s contributed positively to the area and is a decent person,’ he remarked.

Hollywood legends Joan Fontaine, Gene Hackman and Doris Day once called Carmel their home

Monaco-based billionaire Patrice Pastor stated about Carmel-by-the-Sea: ‘It’s time to depart from this unusual place, if it can indeed be called a community.’

The rattled tycoon wrote in an blunt statement to SF Gate: ‘Following this latest City Council meeting it’s clear that I need to reconsider my investment in Carmel.’

His explosive rebuke continued: ‘We are not treated the same as everyone else, I suppose we are now at the point where we need to accept we are not wanted and draw the necessary conclusions.

‘It’s time to leave this strange community, if you can call it a community.’

The businessman fell in love with the playground for the uber-wealthy during childhood trips to the Monterey Peninsula.

The town, with just 4,000 inhabitants, is a favorite with celebrities such as Taylor Swift, Brad Pitt, Gigi Hadid and boyfriend Bradley Cooper, as well as former mayor and long-time resident Clint Eastwood who, at 95, still visits his local bar, The Hog’s Breath Inn.

Pitt purchased a $40 million Pacific-view home there in 2022.

Monaco-based billionaire developer Patrice Pastor has amassed a $100 million real estate portfolio in Carmel

Carmel’s fans include Taylor Swift and boyfriend Travis Kelce who visited Carmel last year

Plans to built the JB Pastor Project in downtown Carmel (shown) were thrown into disarray following a planning appeal on August 4 by 11 residents

Pastor purchased a unique home in Carmel designed by acclaimed architect Frank Lloyd Wright for $22million

In April, Pastor shared a much different opinion when he told the Daily Mail: ‘I love Carmel and am happy to have a home there.

‘It is such a unique small town in America, a village really, with a soul and beautiful architecture.’

He pledged ‘to protect Carmel’s unique history and character’ and said he would always be ‘very careful with the community and the history.’

Carmel Mayor Dale Byrne told the Daily Mail that Pastor is doing ‘doing good things for the community and he’s a good guy’

But on August 4, the city council held a special meeting to consider an appeal filed by attorney Krista Ostoich on behalf of 11 residents.

The council agreed there were parking and landscaping concerns with Pastor’s project. 

Two other council members, including Mayor Byrne, had recused themselves. 

Ostoic, who filed the appeal, said her clients want to maintain the ‘unique village character’ of Carmel, which includes ‘quaint buildings’ and courtyards.

‘The process really worked here,’ she said, adding the council ‘applied the right standards.’

The decision to appeal had nothing to do with Pastor personally, said Ostoic, who called him a ‘nice guy.’ A further hearing is scheduled for September 8.  

Backers of the project, however, point out that ‘thousands’ of hours over six years have already been spent studying the plans.

There were five design changes to win approval.

Mayor Byrne, 70, blamed ‘two or three people’ for blocking Pastor’s downtown development.

‘He did everything the city asked him to do,’ he said. ‘I spoke with him last Friday and they’re just p***ed. Anyone would be.’

The complex would benefit the town with cheaper accommodation and ‘reasonable rates for long-term leases.’

Despite measuring just one square mile, the city has more than 40 art galleries, 60 restaurants and 40 hotels.

Former residents include Hollywood legends Joan Fontaine, Gene Hackman and Doris Day.  

Brad Pitt, pictured with girlfriend Ines de Ramon on June 23 at the F1 movie premiere in London, purchased a $40 million Pacific-view home in Carmel here in 2022 after falling in love with the area

Despite measuring just one square mile, the city has more than 40 art galleries, 60 restaurants and 40 hotels

Carmel realtor Tim Allen described Pastor as a ‘kind and wonderful friend,’ and added: ‘The gentleman has done more for the city than most people’

The Eastwood Building, once owned by movie star Clint Eastwood, is part of Pastor’s portfolio

The luxury ocean-view Carmel Beach Hotel, owned by Pastor, is walking distance to the stunning white sands of the town’s famous beach

Fabulous properties owned by Pastor, purchased through his Esperanza Carmel company, include the L’Auberge Carmel hotel, the Carmel Beach Hotel, the Eastwood Building, the Hog’s Breath Building, an iconic Frank Lloyd Wright-designed home and the Rocky Point Restaurant as well as many more.  

Pastor has also been combating strict planning rules to rebuild another site close to Delores Street known as ‘The Pit,’ which would provide 25 apartments to rent and retail space.

Byrne said ‘it would be a disaster for the city’ if Pastor walked away with the project unfinished. 

He praised the tycoon for his investments. ‘We need hotels, housing and parking – parking is a disaster here,’ admitted Byrne. ‘We’re chasing away someone who might actually help dig our way out of this.’

Toby Rowland-Jones, 70, has lived in Carmel for 13 years and described Pastor as a ‘very likable guy.

Toby Rowland-Jones, pictured, has lived in Carmel for 13 years and called Pastor a ‘galvanizing’ figure in the town

‘He came here with the very best of intentions. Carmel, obviously, is a seaside resort, much like Monaco, although we don’t have quite the same grandiosity – certainly no casinos.

‘He’s been very galvanizing. Some people really, really don’t like him and some say he’s fantastic.’

Rowland-Jones added that Pastor ‘means well’ but has often ‘been stymied’ by the planning process.

Pastor is a client of realtor Tim Allen who has handled his property acquisitions in Carmel. 

The two are now friends and Allen visits the Pastor at his home in Monaco.

‘Patrice is a kind and wonderful friend,’ Allen, 64, told the Daily Mail. ‘The gentleman has done more for the city than most people. Patrice is upset, and I’m upset.’

His friend has ‘improved’ all the properties he purchased and he is ‘what our community needs.’ The sidelined JB Pastor Project is ‘a good project for our community.’

‘I can’t believe they did this,’ commented Allen. ‘The way he’s been treated by a very small group of people is unbelievable. He feels hurt. He’s really disappointed and so am I.

‘He’s a wonderful, kind man that’s been treated worse than anyone I’ve ever seen in our town. It’s horrible. He’s been patient but now he’s just like, ‘This is crazy.’

Allen described the opponents as a ‘small group of self-interested people.’

Born and raised in Carmel, he warned ominously: ‘Our community is dying. Our infrastructure is old. We don’t have enough income based upon our model to even support people retiring.

‘We’ll go broke in probably five years. So we need to reinvent the city.’