Among the most sensational assertions in Andrew Lownie’s thoroughly investigated book on Prince Andrew – encompassing everything from his unquenchable sexual drive to his wife’s monetary imprudence – is one of the boldest claims.

This particular detail actually inaugurates the entire biography, articulated as follows: ‘The father of the groom and mother of the bride – lovers 20 years earlier – sat in the third carriage waving to the crowds…’

In this compelling depiction of the Duke and Duchess of York’s wedding day in 1986, Lownie notably refrains from using qualifiers like ‘alleged,’ ‘rumoured,’ or ‘reported.’

The two individuals sharing that horse-drawn landau on that July day were indeed Prince Philip and Sarah Ferguson’s twice-married mother, Mrs. Susan Barrantes. The statement of their past romantic involvement is presented as straightforward fact.

According to Lownie, the affair commenced after Susie, the name she was familiarly known by, grew ‘exasperated’ with the unfaithfulness of her first husband, Major Ronald Ferguson, the duchess’s father.

While it might be convenient that the individuals involved in this alleged love triangle are no longer alive to protect their legacies, throughout his broadcasts and media appearances this week, the author has neither deviated from his position nor retracted his statements.

Challenged in interviews, indeed, Lownie doubled down.

Appearing on GB News, former Tory minister Sir Jacob Rees-Mogg loftily dismissed the claim as ‘just gossip’, but the author responded that the trusted and knowledgeable source behind it was a member of his own family.

Lownie’s wife, Angela, a similar age to Fergie, had grown up near Ascot close to the Fergusons.

‘They were brought up together, they were neighbours,’ he said.

The connections went further. Angela’s father had been at Eton just before Ron Ferguson arrived there and, like the beetle-browed major, went on to become a Guards officer.

‘I have known the story for over 30 years,’ says Lownie who, as a pupil at Fettes College – the Scottish public school where Sir Tony Blair was educated – played rugby against a young Prince Andrew, then at Gordonstoun.

‘We all gave him a kicking in the scrum because we all hated him,’ he recalled.

All rather compelling – but does it make the story of a liaison between Fergie’s mother and her royal father-in-law true?

Ron Ferguson had his suspicions. In The Galloping Major, his 1994 memoir in which he was scathing about the royals and their treatment of his daughter, he noted: ‘I always suspected that Prince Philip had an eye for Susie. Certainly, they remain friends to this day.’

Behind this seemingly casual remark lies at least a hint of bitterness.

For a few heady years in the 1960s, Ferguson, a career soldier who joined the Life Guards as a Trooper before going to Sandhurst, and his well-born wife – Susie was granddaughter of the 8th Viscount Powerscourt – were part of a lively social circle around the young Queen Elizabeth and her husband.

Ever since he married into the Royal Family, Philip had been dogged by well-informed speculation about his love life. His reaction was always the same: apparent indifference punctuated by the occasional angry denial.

Yet the stories persisted – for he was incapable of hiding his interest in women, not least in the willowy Susan, nee Wright.

This week I learned from another source about the prince’s rumoured admiration for the first Mrs Ferguson. It came from distinguished academic and writer David Rogers, a former parliamentary adviser to Tory grandee Lord Whitelaw.

Rogers told me that the rumour was common knowledge among well-connected members of the louche Thursday Club, a men-only establishment where bon viveurs included raffish figures such as actors James Robertson Justice and Peter Ustinov, American harmonica player Larry Adler, the yet-to-be-unmasked Soviet spy Kim Philby, and of course, Prince Philip. Rogers, too, was a member.

Another regular at the club, which met at Wheeler’s fish restaurant in Soho, was Philip’s close friend and one time Equerry, Commander Mike Parker, who accompanied the prince on a controversial five-month world tour on the Royal Yacht Britannia.

They sailed first to Australia, where the Duke of Edinburgh opened the 1956 Olympic Games in Melbourne. Doubtless, he was promoting Britain’s image, but as the weeks went by and he and the Queen were still apart, critics suggested that the extended trip looked increasingly like a junket.

Their prolonged separation produced a flood of speculation about a possible ‘rift’ in the royal marriage. American newspapers were the first to ask questions. All round the world, others followed. Questions were even asked in Parliament.

Faced with a clamour for answers, the Queen – upset that the state of her marriage had become a public issue – authorised an official palace denial which said: ‘It is quite untrue that there is any rift between the Queen and the Duke.’

Years later, these round-the-world adventures were a key storyline in an early season of the Netflix series The Crown.

Rogers recalls hearing the gossip about Philip and Mrs Barrantes.

‘It was around 1965,’ he told me. ‘As I recall, it was some time between the two general elections of 1964 and 1966.’

What was the gossip?

‘That there was something going on between Prince Philip and Ferguson’s wife. It was talked about openly.’

Clubroom tittle-tattle it might have been, but those spreading it were figures who surrounded the Duke of Edinburgh himself.

The dates, too, are intriguing too because, by his own account, Major Ferguson’s association with Philip – he never described it as a friendship – started in the 1960s when he was invited to join the prince’s polo team.

‘Our relationship was strictly a sporting one, but because of it, my first wife and I were invited to shooting at Sandringham a few times,’ he recalled years later. There followed a more intimate encounter.

‘One summer we were invited to join the Queen’s house party at Royal Ascot,’ he noted.

A decade and a half later, Ron and Susie’s daughter Sarah was herself invited to a Royal Ascot gathering at Windsor Castle where, set up by the match-making Princess Diana, her ill-fated romance with Prince Andrew began over chocolate profiteroles.

Many will wonder if the die was actually cast all those years earlier, when Major Ferguson and the Duke of Edinburgh were two of the leading polo players in the world – and both so charismatically appealing to the opposite sex.

Ferguson’s later claims about Philip – he wrote caustically that he was a man ‘without humanity or humour’ – were inevitably coloured by the failure of his daughter’s marriage and his own later scandals (among them an affair with a young female polo player and visits to a massage parlour for sex).

Yet the Prince and the Queen considered him a friend.

The dashing Fergusons were often on royal guest lists, dining at Buckingham Palace and Windsor Castle.

At a summer dance at the couple’s own home, Lowood House, an Edwardian pile once lived in by King Zog of Albania, the major was courteously attentive to the Queen while keeping a constant eye on his wife.

In his book, he wrote: ‘Prince Philip usually made a beeline for the prettiest girl on the dance floor. He certainly found Susie’s company much more enticing than mine.’

The Queen always made sure her husband was seated next to Susie.

‘She knew Philip would be much happier if her had a pretty woman to talk and dance with,’ recalls an old friend from those days.

Even if nothing improper did occur, could these encounters have led to such gossip about the two? There was no shortage of mischief-makers only too happy to spread stories about a so-called spark between the Queen’s consort and the wife of his polo-playing pal.

And the more they talked about what a handsome pair the couple made, the angrier Ron Ferguson became.

This suspicion about his wife may have led to Ferguson’s sudden and unexplained decision to turn down royal invitations.

Looking back, he recalled in his book: ‘That [decision] nearly caused a divorce, Susie was furious.’

He never directly accused Prince Philip of adultery, of course, and his wife flatly denied it. The real story of the marriage, she said, was of her husband’s womanising, which included taking one of her close friends as a mistress.

And yet their life together had started with great optimism. The couple married in 1956, two years after Susie ‘came out’ as a debutante. Ferguson, who rose to command the Sovereign’s Escort – accompanying the monarch on ceremonial occasions – was also something of a catch.

Decades later, Susie talked of the first five ‘wonderful’ years with Ronald. Then in 1961 came signs that he was ignoring his marriage vows. In those days, her husband was considered handsome and charming. But he was also an incorrigible flirt.

When Ferguson was posted to Cyprus for a year, she was left to look after their two daughters, Sarah, born in 1959, and Jane, two years older.

By this time, the couple were having frequent arguments.

‘Love had gone,’ Susie wrote in a series of biographical articles for Hello! magazine.

It was not Susie who was unfaithful but her husband, whose womanising, she said, drove her to despair.

Susie had persuaded him that another baby might bring them together once again but claimed his philandering continued during her pregnancy. In 1969, she lost the baby in the eighth month.

Soon afterwards she discovered Ferguson had started yet another affair.

The marriage staggered on until 1972, when Susie met and fell in love with Argentine polo player Hector Barrantes, whose wife and unborn child had been killed in a car crash.

Despite the circumstances, she was labelled a bad mother who had deserted her daughters.

‘It was not Hector who broke up our marriage,’ she wrote. ‘My marriage had died a long time before that. [Ronald and I] were living together but there was no love. I thought about it day and night and, finally, I said I was leaving.’

Nor, despite Ferguson’s reservations, was Philip to blame. Susie insisted she was never unfaithful with the prince.

And despite the whiff of scandal, including her own divorce, the friendship with Philip survived the many hiccups in their personal lives.

A little turbulence was nothing new to Philip. When he wed the future Queen Elizabeth, he had a reputation for being a ladies’ man and the passing decades only added to it.

Through 73 years of royal marriage, his name was associated with duchesses, princesses, county ladies, the writer Daphne du Maurier – with whom it was he said he had a fling on the eve of his wedding – and even a waitress from Fortnum & Mason.

Unlike most men, he had no particular type.

‘He liked blondes, brunettes and redheads – he was very impartial,’ observed his cousin Princess Alexandra of the Hellenes, to whom he also took a shine before marriage.

On some occasions Philip denied impropriety, on others it was left to the women or their families.

Among the names he was linked to over the years were Pat Kirkwood, a beautiful and popular musical comedy star, actresses Merle Oberon and Anna Massey, and Jane, Countess of Westmorland, wife of the Queen’s Master of Horse.

In later years, his friendship with Lady Penny Romsey, now the Countess of Mountbatten, was the subject of feverish speculation.

His former carriage-driving protege became a constant companion in his final years after he retired from royal duties and moved to modest Wood Farm on the Sandringham estate.

Philip could never shake the rumours off and, in the end, he stopped trying. So, perhaps we should not read too much into the fact that he never commented about Susie Barrantes.

On that wedding day carriage ride, he was nothing but solicitous to Prince Andrew’s new mother-in-law. Susie, radiant in yellow silk and a jaunty hat, smiled as he encouraged her to wave to the thousands of well-wishers.

As their coachmen turned a corner into the Mall, he whispered: ‘Hold on to your hat! There’s a good wind coming round this corner.’

Six years later, his son’s marriage now over, there was a postscript: on the day Windsor Castle caught fire in 1992, the culmination of the Queen’s famous annus horribilis, Philip was not at her side but in faraway Argentina at a polo match, where he was escorting an old friend to party.

The friend was Susie Barrantes.