Prime Factors - Ranking UK Prime Ministers

10.1 - Augustus Fitzroy, 3rd Duke of Grafton (Part 1)

Link:

10.1 - Augustus Fitzroy, 3rd Duke of Grafton (Part 1)

Episode Transcript

Abram: Welcome to Prime Factors. This week, Augustus Fitzroy, Part 1.

Parliament: Hip hip hooray!

Parliament: Hip hip hooray! Hip hip hooray!

Introduction

Abram: Hello and welcome back to Prime Factors. I'm Abram and I'm here with my dad. We're reviewing all the British Prime Ministers from Robert Walpole to Keir Starmer. This is episode 10.1, Augustus Fitzroy. I've heard a little bit about him from Dad, and I've heard he likes to party and that he's not a great guy, but I can't wait to see if that's true and how it unfolds.

Joe: I know, Abram, that I promised you that we would do this as a single episode. I lied. There's just too much to cover. Augustus Fitzroy, the Duke of Grafton, he's a fun prime minister to cover, if not a very good one.

Abram: We need to be doing one-part prime ministers!

Joe: Okay, well, we'll look for another opportunity for a one-part.

Abram: How about Petty? He's coming up soon.

Joe: Okay, we'll try to do Petty in one part.

Abram: Next year you are required to do at least two one-parts. All right, speaking of the next year, it's New Year's. Well, almost. It's December 27th for us. We're so happy—New Year's for those who are listening now, but if you happen to be binge listening in 2029, well, it's New Year's for us right now, and maybe it is for you too. I don't know, I'm not in 2029 yet.

Joe: Okay, I love your ad-libbing today. All right, I want to thank the Alexander Standard Podcast. They're a rexy-pod where they rank the heirs of the Greek empire. They selected us as day 24 on their 2025 History Podcast Advent Calendar, and I can't be more thrilled. There will be a short promo for them at the end of this episode, but, uh, why wait? You can find them on your podcast app right now.

Abram: Don't forget that you can find us on Apple Podcasts, YouTube, and primefactorspodcast.com. Also on Facebook and BlueSky. If you enjoy listening, please like, subscribe, comment, and review.

Joe: We are so grateful for many of our friends and fellow podcasters who've contributed to this episode, which ended up being more of a New Year's special than I'd anticipated. Thank you to John and Rob from Prime Time, Rob from Totalis Rankium, and Kassidy Kearey from his eponymously named YouTube channel. Information on these shows will be in the show notes as well as at the end of the episode. Thank you also to Abram's friend Adam, who's recording some music for the episode.

Joe: Before we begin, let me make a brief content warning. It's impossible to discuss Augustus Fitzroy's career without talking about sex stuff.

Abram: Dun dun dun!

Joe: And Abram tells me that he has seen Hamilton and he's okay with the sex stuff.

Abram: What? I've seen Hamilton.

Joe: But if you haven't seen Hamilton, then viewer discretion might be advised. Abram, are you ready?

Abram: Yes.

Joe: This is a man born with every advantage, but he still couldn't quite do the job.

Abram: That's unfortunate. We should give him the participation award.

Picture This

Abram: Picture this.

Joe: It's October 7th, and an unseasonably cold autumn chill has descended upon London. There's no snow on the ground yet, but the sky is gray and the wind merciless. Inside Shelburne House, the newly built London home for William Petty, the Earl of Shelburne, the fire in the dining room is warm and the atmosphere almost festive. Four of Britain's most powerful ministers chat patiently while they wait for a fifth. At the back of the room, Lord Shelburne stands near the door. He nervously admires the Roman statues he's placed in the niches. He desperately wants to impress his dinner guests. Shelburne House isn't even finished yet, and this dining room is the only part ready for visitors.

Abram: We're supposed to be doing Augustus Fitzroy. Did you write about the wrong Prime Minister?

Joe: He'll be here soon.

Abram: Are you sure?

Joe: Yeah, I'm sure. Shelburne clears his throat.

Shelburne: I'm sure His Grace will be arriving soon, and then we can begin our discussions over mutton, an old Irish recipe passed down from the Geraldines themselves.

Joe: Lord North stands by the fireplace, ready to work. He's new to this group, having just been made the Chancellor of the Exchequer after Townshend's death. He sips his wine, anxiously taking stock of each of his fellow ministers in turn. Sitting at a table next to the empty chair reserved for the Duke is the Commander-in-Chief, General Granby, alongside Henry Seymour Conway, the Northern Secretary. Granby has already downed his glass and signals a footman for another. Conway stares intently at his untouched glass.

Granby: I am starving. We've put off this discussion half a dozen times already. I need these troops in Ireland and I need my hands free to use them.

Conway: Eleven times already, you mean, General. The Irish Parliament is impatient for an answer, as is the King.

North: Eleven times, Mr. Conway! I was told that the Treasury was disorganized, but I assumed—

Conway: Assume nothing, Lord North.

North: I am grateful that Shelburne has offered to host this dinner so we can put this problem away and move on to the next. I am looking forward to working with all of you.

Shelburne: As soon as the Duke arrives… Wait, I think I hear him arriving.

Joe: Shelburne snaps to attention, expecting to see his friend the Duke emerge. Instead, it's a footman holding a silver tray. An envelope sits on top, and Shelburne opens it.

Shelburne: Thank you. You may go. The Duke sends his regrets. He's been delayed on the road from Newmarket. He asks us to wait. He expects to be here by eight o'clock.

Conway: Newmarket? He's been at the races? The King's waiting for a report, and he's been gambling?

Abram: The races? What? What's he doing? He's been gambling this whole time!

Granby: More wine!

Joe: Shelburne sighs, but he holds onto his smile as he instructs the cook to keep the mutton warm. Minutes stretch into hours as the ministers wait. Granby downs glass after glass.

Granby: This is good stuff!

Joe: Lord North fidgets with the silverware, eyes darting to the door. Shelburne moves from guest to guest, desperate to keep conversation alive and signaling the footmen to keep the wine flowing.

Shelburne: Don't spare the finest bottles. All this effort to answer one Irish question, and the food is getting cold.

Joe: The footsteps echo once again. A footman delivers an envelope. Shelburne reads it. His face turns deep red. He takes a slow breath, then a quick gulp of wine before tossing the letter into the fire.

Shelburne: His Grace shall not be joining us this evening.

Conway: Has he taken ill?

Shelburne: Ah, no. He plans to entertain a guest this evening at his home. He hopes to return to London tomorrow.

Granby: Entertaining? Entertaining Miss Parsons, no doubt.

Conway: General, lower your voice, please!

Granby: Everybody entertains Miss Parsons!

Abram: Everybody.

Joe: Everyone shoots the General a dark look, but they all know he's right.

Conway: Surely his Grace's private affairs are his own.

North: His affairs may be his own, but this is a matter of state.

Shelburne: Gentlemen! I will have the mutton brought right out. No sense in waiting any longer.

North: No. If Mr. Conway is to be believed, it's now twelve times we've found a reason to delay this discussion. Twelve! Are we ministers of the Crown? I didn't come here to chat and play games. Fetch my coat! I have a long night of work ahead of me.

Conway: Lord North is right. This is ridiculous. I'm sorry, Shelburne. I've had enough as well. I hope to enjoy your company again another evening.

Joe: Shelburne watches in resigned disbelief as North and Conway exit the room. Granby stays sitting, staring intently at a nearly empty glass of wine, red-cheeked and swaying ever so slightly.

Granby: I'll stay. You do pick a lovely French wine, Shelburne, and that mutton smells delicious.

Abram: You told me he'd be here soon.

Joe: I lied.

Abram: You lied for the second time!

A Family of Monsters

Abram: A family of monsters.

Joe: We have to start with the elephant in the room.

Abram: Elephant? I don't see an elephant. I only see a manatee. Say hi, manatee.

Joe: Hello, manatee. We have brought along for our recording today Abram's stuffed manatee. His name is—

Abram: Oh, the huge manatee.

Joe: Yeah. He was a macabre child and, well, very special to us.

Abram: Speaking of Manatee, Manatee right here has something to say. I've heard you're donating to my club. Hooray! Can you go into more detail, Dad?

Joe: Yes. So we are participating in Intelligence Beach 2026 this February 28th. This is a great online convention of history podcasters. We'll be presenting on the topic Companions and Rivals. And if you sign up using code FACTOR, F-A-C-T-O-R, you apparently get a discount on the tickets.

Abram: That's good.

Joe: And we will donate any proceeds we receive to the Save the Manatee Club. So, oh, the huge manatee is going to watch us record today.

Abram: Oh gosh.

Joe: All right. So Abram, we're going to start with the elephant in the room. We're talking about Augustus Fitzroy, and Fitzroy means—do you know what it means?

Abram: Son of a king.

Joe: Son of a king, or the king. And in this case, King Charles II.

Abram: Party guy!

Joe: Yes, as Horrible Histories taught us, Charles II loved to party. But what Horrible Histories didn't tell you is that those parties, Abram, involved a lot of mistresses and him cheating on his wife.

Abram: They didn't like—yes, he had a lot of children—weren't they illegitimate though?

Joe: Yes, he had a lot of illegitimate children, but even though they were illegitimate, he still supported them. He often gave them titles or jobs, you know, he recognized them. Unlike some other kings, he actually recognized his illegitimate children and he supported them. For our story, the Grafton clan are descended from Charles II's time with a woman named Barbara Palmer, the Duchess of Cleveland.

Abram: But Cleveland's in the US.

Joe: I can only assume that there's a Cleveland in Britain somewhere.

Abram: Yeah, but Cleveland wasn't named after the UK. Remember, Totalis Rankium? You said that Grover Cleveland's ancestors were, like, Cleveland, and then they named it after themselves. They told the stories to Grover Cleveland. So I think there's a mystery here, but actually, I think there is a Cleveland in England. It's in, like, northern England.

Joe: We'll have to look it up.

Abram: It's just a different Cleveland.

Joe: So one of their children was Henry Fitzroy, and while he would never inherit the throne, his father the king made him the first Duke of Grafton. Then we're going to skip ahead. Augustus's grandfather—our Augustus Fitzroy, who we're talking about—was the second Duke, and he was a major player in Whig politics under George I and George II. He had so many titles at different times: Lord High Steward, Lord Chamberlain, Lord Lieutenant of Ireland. He was a big, big guy. In fact, Abram, you and I visited Grafton Street in Dublin a few years ago. Do you remember?

Abram: Buskers.

Joe: Lots and lots of busking musicians, and that's also where Bono got his start. And that Grafton Street is named for Augustus Fitzroy's grandpa.

Abram: I'm Grandpa Grafton.

Joe: Augustus's father, though, was not expected to inherit the dukedom. He was a second son, so he sought his fortune in the British Navy, eventually making his way to New York, where he met his love of at least an afternoon, Elizabeth Cosby. He was 17 at the time, and he met her while dining with her father, the governor of New York, the governor of New Jersey, William Cosby. Now, William Cosby is a villain in American history. He was known for trying to rig elections, fighting against the freedom of the press in America, stealing tax revenue to enrich himself, damaging relationships with Native Americans. He was a piece of—he was a piece of work. He was a schemer.

Abram: I thought you were going to say a piece of garbage.

Joe: It might have been that too. So I'm just saying, this guy who clearly was in for his own advancement, I don't know, but he might have had some small incentive to get his daughter hooking up with the son of a duke. Anyway, Grandpa Grafton disapproved of this. A daughter of a colonial governor was not highbrow enough for his son, but there was no way for him to stop the marriage. And this means that Augustus Fitzroy is the second prime minister we've talked about whose grandfather was a colonial governor. Do you remember who the last one was? Yeah, Pitt. His grandpa Pitt was the governor of Madras in India.

Abram: Oh yeah, the diamond guy.

Joe: Yeah, it is a sign though, Abram, that Britain's growing empire really was changing the way the aristocracy acted and who were in their social circles.

Abram: Yes.

Joe: A child followed. Augustus Fitzroy was born on September 28th, 1735.

Abram: Wait, if he's Prime Minister in 1768, he must have been really young.

Joe: He was. We have something we call Silver Spoons as a way to tell—

Abram: Silver Spoons!

Joe: Exactly. As a way to tell how many legs up a future Prime Minister received by the nature of his birth. It's not scientific, but it has been insightful. We've had low-scoring commoners like Robert Walpole. He got 5 points. William Pitt, he got 10 points. We've had high-scoring nobles like William Cavendish. He's our highest scoring so far.

Abram: Didn't Compton get ridiculous? Yeah, Compton doesn't exist according to you, pretty much. He does exist, but he was also like 2 episodes ago when you're going through all the prime ministers and you forgot him.

Joe: Yes, you're never gonna let that down. The point is the highest Silver Spoon score we've had so far is William Cavendish, the Duke of Devonshire. He had 35 points. I think Spencer Compton had like 32.5 or something.

Abram: So those two are the high scorers.

Joe: Well, they were. Augustus Fitzroy broke my system. He has the highest score to date, 75.9 points.

Abram: Let's put it this way. When you're descended from the king, my system explodes.

Joe: Anyway, cheating on your spouse is not genetic, but the Fitzroy family that was birthed in scandal remains soaked in it from generation to generation. And the history played out right in front of young Augustus. Let's start with his dad. Augustus's father was not a very good person. His relationship with Augustus's mother fell apart almost immediately, and by the time Augustus was 3, his father was caught in an extremely public affair with another woman. Well, actually, caught is a bit strong of a word. Augustus's father openly bragged to his own wife about the affair. He showed his wife a love letter from his mistress.

Abram: This is so stupid. He's the stupidest person I've ever heard of.

Joe: And it may have been that stupidity that allowed him to actually be caught literally in the act.

Abram: He's so stupid.

Joe: So it's not typical for men to be punished for infidelity in that era, or indeed many others, but he was. And it helped that I think he was cheating on the wife of an MP or something. But he had to pay £5,000, essentially millions of dollars today, far more than his salary in the British Navy could ever afford. As a fine, a punishment for stealing another man's wife. Grandpa Grafton was so angry that he actually met Elizabeth, whom he otherwise wanted nothing to do with, and he pledged to support her and her children. Now, that kind of debt, especially at the time, would almost certainly land someone in prison. But there were ways that a connected person like Augustus's father could escape debtor's prison. Just being in the Navy would be a start. And he stayed in the Navy. In fact, he rose in ranks. He eventually became a captain. He could also enter Parliament because Parliament had a special rule that allowed for MPs to avoid being jailed for debts. And since Grandpa Grafton controlled some seats, he arranged for Dad to become an MP for Thetford. It's also possible that Grandpa Grafton just paid the £5,000, but considering how angry he was, I don't think that happened.

Abram: But were any of his seats just rotten boroughs?

Joe: Yeah, his seats were rotten.

Abram: Here's the Rotten Boroughs song. 3, 2, 1. Old Sarum got a new tummy, so then there's some up there. The places that once existed but now are somewhat empty. We could all go hold the seats and give them to a friend. The old Sarum got a new tummy, so then there's some up there.

Joe: Honestly, Abram, none of that mattered because while Captain Fitzroy was serving in the War of Jenkins' Ear. Do you remember the War of Jenkins' Ear?

Abram: The guys cut off his ear and so they started a war.

Joe: Yes. Well, while fighting in that war, he caught a fever and he died in Jamaica.

Abram: Who? Criminal guy?

Joe: Augustus's dad.

Abram: Criminal guy.

Joe: So Augustus and his brothers were taken in by Grandpa Grafton, and his mother Elizabeth was completely cut out of the picture. But even with his father dead, Augustus was still not expected to become Duke. The Duke would be his Uncle George, who, for all that you are calling Augustus's dad criminal guy, Uncle George was worse.

Abram: He was like—

Joe: Put it this way: when Augustus was about 8, his uncle married a 17-year-old girl. That's bad on its face. She died of abuse less than a year later.

Abram: What? How old was he?

Joe: I don't know. He's in his 30s, I think.

Abram: Even worse criminal guy, but he isn't gone. I hope he won't be there for long. Even worse criminal guy should be gone. He does not belong.

Joe: Uncle George was disinherited by the family. No one else wanted to marry someone who probably was responsible for his wife's death. I mean, he was rich enough that he got away with it, but nobody would marry him. He was ostracized by society, and he died childless when Augustus was 11. And that meant that Augustus was now, unexpectedly, the heir to the Duke of Grafton. He had grown up in a family of monsters, and it would be up to him, now called Earl Euston, to show that he could rise above that legacy.

Raising an Earl

Abram: Raising an Earl.

Joe: Grandpa Grafton made sure that Augustus had an education and the social connections fit for a future duke. He's called Euston now, but for simplicity's sake, I'm just going to keep calling him Augustus because he's a kid. Well, he is a kid, but he's a well-connected kid. At 13, Augustus was introduced to a couple of Grandpa Grafton's friends at a meeting at Stowe House. You might recognize some of these friends. William Pitt, George Grenville, Richard Grenville-Temple.

Abram: Oh yeah.

Joe: Augustus Fitzroy was a cub to Cobham's Cubs.

Abram: What? So is he a Cobham's Cub or is he a Cobham's Cub cub?

Joe: He's a Cobham's Cub cub.

Abram: So he's a really young member of the Cobham's Cubs.

Joe: Well, he's 13, he's not even a member, but the point is that his grandpa is introducing him to the political elite of the country. His education was what you would expect from a young noble: Westminster School, Newcome's School, Cambridge University. But like many noble kids, he didn't even finish his degree. He left, quote, "fairly advanced in learning" in 1753 for the next phase of learning. Do you want to guess what that is?

Abram: No.

Joe: A grand tour through France, Switzerland, and Italy with a tutor.

Abram: Doesn't everyone do that?

Joe: All the rich people do. Yeah. The tutor would teach him on his trip to become a gentleman rather than a scholar. Once his tour was done—

Abram: How old is he? Like 16?

Joe: That was probably 18, 19. But at 21, he got married. Like, he started to set his life in order.

Abram: At least he wasn't 17.

Joe: He married Anne Liddell, the daughter of Baron Ravensworth. I'm 90% sure this was an arranged marriage, but as Augustus's love life will be defined by women not his wife—small spoiler there—I have found little about their courtship or how they built their life together. His political life also started to come together. He was made a Lord of the Bedchamber to the Prince of Wales. Which at the time was George.

Abram: Was the prince actually Prince of Wales or just called Prince of Wales?

Joe: So the next in line for the throne in the British system is always called the Prince of Wales.

Abram: But at this point, was it literally the Prince of Wales?

Joe: No, this is well past when Wales was a separate political entity.

Abram: And then when did Prince of Wales, like, refer to Wales?

Joe: A long time before this, but I'd have to look up when.

Abram: Okay.

Joe: At this time, the Prince of Wales was the future George III. So Prince George, but George III. Augustus Fitzroy ran for two separate seats in that year's election. He served for Boroughbridge for a week before switching to Bury St. Edmunds, which was another one of his family seats. But there's almost nothing to say about his career in the House of Commons. He is recorded as being present for only one vote, joining William Pitt in a vote against Newcastle after Minorca was lost. But this period of rise for young Augustus came to an end quickly. On May 6th, 1757, Grandpa Grafton died.

Abram: No!

Joe: Augustus Fitzroy was now the Duke of Grafton and shockingly rich. £18,000 per year.

Abram: How many dollars?

Joe: Many, many millions. It's hard to know. He's super rich. Without the guiding hand of his grandfather and with a vast fortune burning in his pocket, could a 22-year-old Augustus hold his life together?

Abram: Really be trusted?

Joe: Yes, could he be trusted to hold his life together?

A Young Duke

Abram: Um, no. A young d—

Joe: Before dying, we expect Grandpa Grafton spent some time with Augustus to go over the management of his estates before his death.

Abram: Death.

Joe: But it was a gigantic responsibility. The family seat was Euston Hall in Suffolk, a mansion that remains the home of the Dukes of Grafton until today. The mansion was close to the horse races at Newmarket and included stables for managing the duchy's racehorses, and Augustus inherited those. With a clear day and the right wind, I bet Grafton—he's now called Grafton, we're going to call him Grafton from this point—could hear the cheer of the crowds from his office window. He had a second mansion at Wakefield Lodge in Northamptonshire. That was a hunting lodge and a sporting area where Augustus could spend time with his many dogs, which he also inherited from his grandfather.

Abram: Wait, he had dogs?

Joe: Hunting dogs.

Abram: Aw, not pet dogs.

Joe: I mean, he might have had pet dogs too.

Abram: We'll see.

Joe: They also had Grafton House in London, as well as all the land around what is today Euston Square. I know it's a tube station, but those of you more familiar with London probably can picture it slightly easier than I can. Obviously there were lots of fun distractions for a young duke, but at least at first he had things to focus on. Chief of those was that he was made the Lord Lieutenant of Suffolk, a post previously held by his grandfather, and young Grafton leapt into this role. He made himself a colonel in the Suffolk militia. He liked to drill with the troops. He encouraged other nobles to comply with the Militia Act of 1757, which was to defend England from invasion. He was a role model, a duke to be emulated.

Abram: A duke to be emulated.

Joe: Hold on to that feeling. It's not going to last long. One job that he didn't have enough time for was to continue being Lord of the Bedchamber. He resigned, although the parting seemed to be mutual. Prince George remarked, quote, "I can never look at him as one fit to remain in my service," which I think is as close to "good riddance" as royalty is likely to get. Grafton spent the next several years enjoying his new life, spending time with his hounds and horses, playing soldier sometimes, and staying largely out of politics.

Abram: Wait, did you say playing soldier? Does that mean he's just going to battles and shooting things?

Joe: No, he's in charge of the Suffolk militia. He gets to go out and put on a colonel's uniform and act like a soldier and act like he's leading soldiers. And maybe he was leading them. By all accounts, he did good at this.

Abram: Okay.

Joe: But beneath the surface, there was something going on. Despite having two children confusingly named Georgiana and George, Grafton decided that he needed to cross the Channel to France.

Abram: He literally just named his kids after his evil uncle?

Joe: Possibly his evil uncle. More likely after Prince George or King George II.

Abram: Okay, good.

Joe: But regardless, he decided that he needed to cross the Channel to France during the Seven Years' War for his wife.

Abram: Why?

Joe: Let's find out.

A Detour to France

Abram: A detour to France.

Joe: We have a little mystery. I love mysteries, but I warn you, I don't have all the answers. In 1761, late in the Seven Years' War, Grafton left England for Italy, traveling by way of France. He also stopped in Switzerland and somewhere else. I don't even know how he did that. It was a war. Except to say that the rules for nobles are not always the same as for common people. They're actually going to stay there through the coronation of George III. The official explanation for this is that his wife had become addicted to playing a card game called Loo. That word, I think, is funnier to British people than Americans, but it's hard not to laugh at the idea of powdered-wig-wearing nobles sitting down together, tea served from fine china, a cello player in the background serenading them, enjoying a fine game of toilet.

Abram: It is funny, Manatee.

Joe: Could there be another reason? Given their wealth, it seemed unlikely they were fleeing a debt, although some sources suggest that. Any bad blood between Grafton and George III couldn't have risen to the level of missing a major event like his coronation. It's not likely to have been a crime. He wasn't sent into exile. They weren't even hiding on the continent. In fact, they kept up their high society contacts. The family spent time with a French philosopher named Voltaire. Have you ever heard of Voltaire?

Abram: Yeah.

Joe: Yes. Well, he's now in his 60s. And in addition to being a philosopher, he was also a playwright. And Grafton even acted in an amateur production of Voltaire's play, Alzire, that was staged in his honor—in Voltaire's honor, not Grafton's honor. But this makes two prime ministers so far that have performed in amateur plays before their political career. Do you remember the other?

Abram: Was it John Stuart?

Joe: It was! Ding ding ding ding ding! Horace Walpole regularly played cards with the Duchess. And again, a reminder of how politically connected their families were. But although he regularly reports that she enjoyed playing cards late into the night, I don't see anything in his letters that says addiction, and she didn't stop playing after they returned from the trip. Other sources say that they simply had to be on the continent to recover from an illness, but not specified.

Abram: Can you just go to Bath?

Joe: Yeah, it's a mystery. Why did they do it? I don't know. To make the mystery even more tantalizing, Horace Walpole's letters imply that he knew the real reason but wasn't telling.

Abram: Wait, has Horace Walpole pretty much appeared in every episode?

Joe: Yeah, Horace Walpole has been a fixture in politics since his dad, who might not even really be his dad, was Prime Minister. He is known for his biting words. He wrote a lot. He composed a history of the era that a lot of books these days are based on. I mean, Horace Walpole was huge, and he's getting up there in years. And so I don't know how many more prime ministers—

Abram: This might be the last one.

Joe: I don't think it is, but there's not going to be that many more prime ministers.

Abram: Probably will be the last one because he's so long. Spoiler alert.

Joe: Yeah, we'll have to check it. So my theory, and it's just a theory, is that maybe Grafton was already cheating on his wife. If I were a young duchess and I knew that my husband was cheating on me and I knew that there was nothing that I could do about it, if I lost a few thousand pounds at cards, I think that would be an understandable, if petty, revenge. And considering how badly their relationship's going to go in a few years, I'm not sure I'd put it past her.

Booted into Politics

Abram: Buted into politics.

Joe: In 1762, the now 27-year-old Grafton was mostly just enjoying his life. Races, hunting, traveling.

Abram: Did you say races?

Joe: Yes, he enjoyed horse races.

Abram: Huh? People actually raced horses?

Joe: Yes, Manatee. And he perhaps especially enjoyed managing the Suffolk militia. He loved being Colonel Grafton, and by all accounts did very well at it. He nipped into politics a bit here and there. He occasionally spoke in the Lords. He formed a Young Whigs group with just 8 members, but it's a start of a Grafton faction in the Commons. But this idyllic life as a Whig-leaning noble wasn't to last.

Abram: Oh no.

Joe: After George III came to power, he brought with him his lead advisor, John Stuart, the Earl of Bute.

Abram: Came in singing, tor tori, tor tori, tor tor toru.

Joe: That he did.

Abram: So what do you plan on accomplishing? And how do you think you'll be different from other people? I'll be different because I'm a tor tori, tor tori, tor tor toru. Can you cue the song?

Joe: Yeah, I think the key though is that John Stuart wanted to reassert royal power from the old money, and especially the old money Whig factions. And with Bute wielding that axe, Whigs at almost all levels of government were fired and replaced by loyalists.

Abram: Was Grafton?

Joe: We'll have to see, but yes. Historians called this the Massacre of the Pelhamite Innocents, and the county lord lieutenancies were not spared. We've already seen Newcastle, Devonshire, Oxfordshire, and Rockingham all have their county roles taken away. But now we add one more to the list: Grafton.

Abram: Oh no.

Joe: To say that this hit him hard would be an understatement. He loved his local responsibilities. And this is the moment when Bute took a carefree Whig playboy and turned him into an enemy.

Abram: Oh, so is he going to be a bad guy for pretty much the rest of the story?

Joe: Augustus Fitzroy? I mean, he's going to be Prime Minister eventually. I don't think he's a bad guy. To John Stuart, he's a bad guy, but I think everyone else would say that John Stuart was one of the big villains of politics of the era. So Grafton immediately took a deeper interest in politics. He aligned himself with Cobham's Cubs, especially Earl Temple and William Pitt, the same group that his grandfather had introduced to him when he was just a teen. Even Horace Walpole took notice, writing that Grafton had become one of the leaders of the resurgent Whig movement. A few months later, when Wilkes was arrested for writing that North Briton No. 45, Grafton and Temple actually visit him in the Tower of London.

Abram: How?

Joe: John Wilkes. John Wilkes was a former MP. He had a newspaper. He wrote against John Stuart. And he wrote so much against John Stuart that he was arrested. They had general warrants out. It was a massive freedom of speech controversy. Do you remember?

Abram: Barely. Well, it's been a year.

Joe: It's okay. It has been a year. The point was that he is now in the Tower of London and visited by both Grafton and Temple. But while Temple was kind of sympathetic, Grafton was not. And this is going to be the start of a long, unhappy relationship between the two. But for now, it had its purpose. Grafton was making a show of meeting with Bute's enemies. The Bute government was followed by Grenville, but the real break for Grafton came when George III asked his uncle, the Duke of Cumberland, to help form a government. Cumberland selected Rockingham to lead as First Lord of the Treasury, and they assembled their cabinet together. So one huge challenge for Grafton was something that I'm going to call the nobility trap. He could not or would not accept a low-level role, but he was also too inexperienced for the top jobs. He was a duke. He couldn't just be an undersecretary somewhere. He couldn't start at the bottom, right? He was already at the top. This limited his options and ensured that he'd have a difficult time learning on the job no matter what role he'd be given.

Abram: Mm-hmm.

Joe: Cumberland offered to make Grafton the Northern Secretary. That was July 1765. And that was great. It was a role that wasn't beneath his station. It might have been because Grafton enjoyed horse races. Now, remember that Cumberland found Rockingham at the Ascot races. That's how they built their first government. He built it with his racing buddies. And while Grafton liked to race at Newmarket, and not Ascot, it could be. But anyway, Cumberland, as you know, died almost immediately, and Rockingham became the sole leader of the administration. And from there, Grafton became increasingly unhappy because he did not want to work for Rockingham. He had wanted to work for Cumberland. This did not mean that he didn't have a few wins under Rockingham. Grafton pushed for repeal of the Stamp Act, negotiated a trade deal with Russia, or at least some of his people negotiated that trade deal. But he couldn't deal with them forever. In April 1766, Grafton resigned, setting off the dominoes that would eventually topple the Rockingham administration. As Grafton explained a few weeks later, "They wanted strength which only one man could provide." Who? No spoilers, but this is going to be a little bit of his own challenge. So that July, Grafton got his wish: a William Pitt government.

Abram: Well, you must thank the genie for that, whoever gave him 3 wishes.

Joe: Even better, it was to be a Pitt government where he would be allowed to take a key role.

Abram: Manatee, how's this been going so far? This is really interesting. I hope to join for the next episode. That's nice to hear. Okay, Manatee really hopes to join for the next episode.

Joe: Great.

Abram: And if we're gonna still be recording here, I bet it's very likely.

Joe: Yeah, I guess we'll see.

Money CAN Buy Me Love

Abram: Money can buy me love.

Joe: While Grafton was rocketing up the political ladder, he was simultaneously setting up his fall.

Abram: Yeah.

Joe: From the outside, the relationship between the Duke and Duchess of Grafton was going well. They were celebrities. I didn't even know that the 18th century had celebrity culture, but they attended balls and social events together. They had 2 children with a third that was going to be on the way soon. They were even trendsetters in a very weirdly modern sense. In January 1764, there was a ball and the Duchess, she wore her hair straight, cut across the bangs, and with no powder.

Abram: Is that shocking?

Joe: I am shocked. I don't even know what that means. I'm, like, visualizing a Beatles bowl cut but with longer hair in the back, like a Beatles mullet. Now I'm certain that she pulled it off better, and this wasn't a fashion faux pas. This was a hairstyle trend that spread through the upper classes in England and France. It was called coiffure à la Grafton.

Abram: Of course it's French.

Joe: They were trendsetters, like they were legitimately celebrities. But even when they were doing so well together publicly, Augustus Fitzroy had already met Nancy Parsons.

Abram: Uh-oh.

Joe: Now Nancy was, by most accounts, one of the most beautiful women in England.

Abram: That isn't good.

Joe: But beyond that, it's difficult to describe exactly what she was. She's a little bit like a high-class prostitute, spending time with wealthy men in London in exchange for money and access to high society. She has been called words I cannot use on this podcast. And before getting together with Grafton, she had known many others. She had been the companion and maybe even wife of a wealthy merchant—I think a slave trader, actually—in the Caribbean. She had been close—I'm being subtle here—she had been close to William Petty, the Earl of Shelburne.

Abram: Prime Minister betrayal.

Joe: Yeah. You might want to put a pin in him.

Abram: Ow.

Joe: She was well connected to the highest circles of Whig power.

Abram: Why do we put a pin in people?

Joe: So that we remember that we're going to talk about them soon.

Abram: So basically, Manatee, thanks for your question. If there's someone that's going to become very important later on, we put a pin in them. It's a way to sort of remember them.

Joe: Thank you for the explanation, Abram, and thank you for the question, Manatee.

Abram: Yeah, since you're new here, makes sense.

Joe: She is a woman that has gone by many names. In addition to Nancy Parsons, she also went by Anne Parsons, Anne Horton, Anne Houghton, and others.

Abram: And everything, essentially.

Joe: And Horace Walpole, like usual, summed her up best. She was, quote, "The Duke of Grafton's Mrs. Horton, the Duke of Dorset's Mrs. Horton. She was everybody's Mrs. Horton."

Abram: Yeah.

Joe: At first, at least, Grafton was subtle about his new affair. He set her up at a house in Woodford, about 20 miles northeast of London. It might have been a real struggle for her since he only hired 2 servants to take care of her day and night. That's a joke. Uh, not a good one apparently.

Abram: Huh, this makes no sense. Sorry.

Joe: She was close enough that they could spend time together, but not so far that he couldn't be sneaky. At some point, the Duchess finds out about this. One story is that she learned while she was pregnant with her third son, but these details are hard to know for sure. We do know that in September 1764, the Duchess left to live with her father, taking the kids with her. Grafton moves Nancy Parsons into his hunting lodge at Wakefield, and presumably she had more servants there and would be closer to him. But publicly, this was a scandal. Remember, the Duke and Duchess of Grafton were high society trendsetters. The 18th century equivalent of gossip magazines—I was shocked to discover that there were 18th century gossip magazines—commented on their separation. But Grafton aimed to pin their breakup exclusively on her without any mention of his mistress.

Abram: I guess there's never another evil guy.

Joe: Well, it runs in the family.

Abram: Yeah, this is truly a family of monsters.

Joe: So Town and Country, a London high society magazine, wrote, "Home had no charms for the Duchess. Quadrille and ombre"—card games—"engrossed her sole attention so that they scarce met for successive weeks. This kind of behavior soon after produced the rupture, which ended in total separation." So we have an issue. They're getting divorced, and instead of blaming it on his mistress, it's her supposed gambling habit.

Abram: That's evil.

Joe: I think we should remember this for the bad personality section. What do you think?

Abram: Yeah.

Joe: So the separation is official that December. The Duchess will get £4,000 per year, custody of the girls, custody of the boys only until they're old enough to live with their father. They're not going to be divorced, but it is an acknowledged separation. And Grafton gets to play with his new friend Nancy. And as their separation was complete, he becomes bolder and bolder, actually bringing Nancy into high society. And this is actually where I struggle to understand their relationship. Nancy was clearly an intelligent, capable woman. She navigated a system that was designed to benefit men and did it in a way that many would find distasteful. She seems to be more than just an attractive decoration to have on his arm, sort of like an 18th century sports car. But I don't know how true their affection was to each other, whether the affection was reciprocated, whether he had other mistresses as well. A lot of questions.

Abram: Oh yeah.

Joe: Horace Walpole wrote, "He has certainly grown uncommonly attached to Nancy Parsons, so much that it has put an end to all decorum." In a family that has already seen so much scandal, Grafton was very publicly pushing the envelope of what British society would endure.

Falling Upwards

Abram: Falling upwards. Look at these Silver Spoons. You'll see.

Joe: When the Rockingham government toppled, Grafton got what he hoped for: Prime Minister William Pitt. This, of course, led immediately to speculation as to what role he would get in the government. He had been Northern Secretary under Rockingham, but Pitt kept Henry Seymour Conway in that role instead. Lord Lieutenant of Ireland was discussed, but dismissed because for some reason Grafton didn't want to leave London. Maybe we can guess why.

Abram: Because he's an evil guy, so he tries to stay in the area of the most crime?

Joe: No.

Abram: Oh, that's too bad.

Joe: And as we said before, other roles would be seen as too junior. Now Pitt planned to lead as Lord Privy Seal, restoring First Lord of the Treasury to its pre-Walpole status as just the top finance job, but not the head of the government. And he wanted Earl Temple in that slot, but Temple wanted more control than Pitt was willing to give, so Pitt kept searching. And despite his lack of finance experience, Pitt eventually turned to Grafton for the Treasury job. It certainly would not have been beneath Grafton's station, and since they had to give him some role, why not one of the top ones? Augustus Fitzroy fell up, almost by accident, into one of the most important jobs in the British government, all because he refused to take any job that he might have been qualified for.

Abram: Oh.

Joe: He was a Duke. How hard could national finance be anyway?

Abram: Very hard.

Joe: Grafton's first big decision at the Treasury was choosing his right-hand man, the Chancellor of the Exchequer. He recommended Charles Townshend for this role. This will prove to be one of the worst possible choices. Townshend was intelligent, charismatic, and capable, but out of step with the rest of the administration.

Abram: He was one step off.

Joe: We can say more about what Grafton didn't do during this first part of the Pitt administration than what he did. John Wilkes wrote to him asking for help, and on Pitt's advice, Grafton ignored him twice. The Wilkes problem will just go away, right? Right. During Parliament's 1767 recess, Grafton was expected to produce a plan for the upcoming session and decide what business would be done first. But apparently no one told him this, or he didn't listen, because he spent the recess at the racetrack enjoying himself, causing some embarrassment when Parliament returned to session without a plan. And of course, Grafton failed to control his number two, Charles Townshend, who increasingly was at odds with Pitt, especially over tax and tariff policy. By March, Pitt had lost patience with Townshend and tried to fire him, but his proposed replacement, Lord North, declined the role. This infuriated Townshend because he realized that he was on thin ice, and if anything, it pushed him to make his own mark on the government even more. But as you know, because we only recorded this episode a couple weeks ago, this was the beginning of the end for William Pitt's ministry. Not long after this, Pitt collapsed into his gout of the head, a mental illness that left him unable to work. We spoke about this. Grafton was First Lord of the Treasury. In any other administration, he would have been the Prime Minister. And as Pitt retreated into his sickbed, many people expected Augustus Fitzroy to step into the Prime Minister-shaped hole and lead the government until Pitt returned. Augustus Fitzroy was in an impossible position.

Abram: What shall he do?

Abram: Oh, I don't know.

With Its Head Cut Off

Abram: With its head cut off!

Joe: In 1767, no one knew how William Pitt's story would end. Pitt was no stranger to bouts of illness, and he was expected to recover quickly, as he had done so many times before. Would Grafton have to mind the store for a week? A month? No one knew. Pitt left no succession plan, and there wasn't even an elder statesman waiting in the wings. Now Grafton was the obvious pick to be in charge. He was First Lord of the Treasury, the job that usually meant Prime Minister. He controlled the money, and while Pitt was out, he would be taking control of the House of Lords. Henry Seymour Conway was supposed to lead the Commons, but he also didn't step up. Instead, it was Charles Townshend that increasingly drove the business of the lower house. Townshend had good reason to suspect that he'd be fired as soon as Pitt got back, so he had motivation to move quickly to build his own vision. But the first moves were not from Grafton or Conway or Townshend. It was with a new group that we haven't discussed before: the Bedford faction, also called the Bloomsbury Gang. Now, the Bedford faction had existed in government for a long time, but they rarely looked for top jobs. They didn't want to run a ministry, but they negotiated for their members to have jobs up and down the government bureaucracy. This gave them comfortable seats that were less likely to be given up as government after government fell, but it also gave them a steady but invisible power base. They were Whigs—well, at least at the time—but they were conservative. They believed in the power of the King, they supported the rich establishment, and they wanted to take a hard line on the colonies.

Abram: Yeah, but doesn't that mean they're a Tory?

Joe: What an interesting observation you have just made.

Abram: So basically, "We're definitely Whigs. We believe in all the Whig things like this and this." But aren't those Tory things? "No, we're totally Whigs. Definitely."

Joe: Well, let me answer this. So at the time, Whigs and Tories—right, the Tories still had a bit of an association with Catholicism and with Jacobitism, maybe not as much as had existed before. This group, the Bedford faction, is going to be the kernel of what will become the new Tories in the later part of the century.

Abram: And like, like 1780 onwards. Yeah, it's not exactly a straight line from there to there, around like 1780, and that'll last all the way to like 1830 when the Tories sort of died out.

Joe: We'll get there. We'll get there. Anyway, good observation. The Bedford faction managed to unite with Grenville and Rockingham's group. They were able to pass a land tax reduction in an otherwise leaderless government. This cut, of course benefiting the rich, blew a hole in the budget. The British economy was still reeling under the Seven Years' War debt, and it was reckless, but there was no one that could stop it. And in a different story, this is where the hero would enter, and in a way, I suppose he did. Charles Townshend saw his moment. In a famous but drunken speech—or at least slightly drunken speech—called the Champagne Speech, Townshend proposed a plan to plug the budget hole. He would make up the loss of local taxes by increasing the burden on the colonies. He was careful to stay within Pitt's formulation, not levying direct taxes on the colonists, but tariffs that would be higher and on more products than ever before. He also learned from the Stamp Act and previous attempts by making the enforcement of these tariffs much stronger. It's as if he thought the reason the Stamp Act failed was because the British didn't try hard enough.

Abram: Oh.

Joe: Despite Grafton trying to stop him, Townshend's bills passed. Augustus Fitzroy must have felt things slipping through his fingers. In his heart, I don't think he wanted the responsibility. He looked for a way out, but he was in too deep. He considered resigning. He might even have tried, but there was no one else. In a panic, Grafton secured a meeting with Pitt, the first meeting that Pitt had agreed to take since he had taken ill. What Grafton discovered there was a deeply broken man who was unable to assist in any real way. The details of this meeting are a bit ambiguous, with the story ranging from Pitt being so unwell that he's not even able to communicate, to Pitt advising Grafton on strategy and blessing him as his successor. Might have been some of both. By the time Grafton returned to Westminster, things were out of hand. The Bedford faction was rising. Townshend had all but seized control of parliamentary business.

Abram: It's mine now.

Joe: He was a rising star with the King's ear, and he was just the kind of charismatic leader that might have been able to unite the government. And if things had happened just a little differently, I think we'd be talking about him as our next episode.

Abram: But we're not, so what happened?

Joe: He died. To say that this is the thing that saved Grafton's administration implies that there was something to be saved, but with Townshend gone, he was able to get Lord North into the government as number 2 in the Treasury. That would be finally an adult in the room and maybe his first win.

Abram: So there's no longer an elephant in the room, now there's an adult in the room?

Joe: Yes. There may even be a chance still to repeal the Townshend Acts before they cause too much damage. Augustus Fitzroy had taken the reins of government, but there were other reins that he longed for. Would he step into the saddle or simply fail to show up?

Abram: But only time will tell, and you'll have to wait till next time. Then put the outro here.

Joe: We have one section left.

Abram: But it was a perfect ending.

Joe: It was.

A Missed Meeting

Abram: A missed meeting.

Joe: This all brings us back to where we started, that empty chair in our Picture This script. Our opening scene was a fictionalized version of a meeting that happened on October 7th, 1767. Granby, North, Conway, and Shelburne were the key figures, but in the real meeting there were other people present. It might have been North's first official meeting as Chancellor of the Exchequer, although most sources list his starting as happening shortly after Townshend died in September. He did not officially take the seals of office till October 6th. So this was really like his first official meeting. The Irish troop question they were discussing really isn't important. It's just one of many issues that a functioning government needs to deal with each and every day. We only even know about the meeting because someone in the room leaked it.

Abram: Oh.

Joe: By October 20th, George Grenville—remember, he was very much in the opposition at this point—wrote about it in his letters, scornfully calling it "a lively picture of the present administration." Horace Walpole complained that Grafton thought that "the world should be postponed to a mistress and a horse race," although he used a much dirtier word. And Junius, a vicious anonymous writer that will be a major villain in our next episode, echoes this.

Abram: Echo, echo, echo, echo.

Joe: The overwhelming impression that we are left with is a government in disarray. Grafton, even after deciding that he would take charge, still struggled to be present when he had other more pleasurable things on his mind. What will he do? The challenges to come would be difficult for even an experienced politician. The government was about to be attacked from every angle—politically, militarily, personally—and it would take an outstanding leader to keep the ship of state afloat.

Abram: Can he do it? Only time will tell.

Joe: Unfortunately for Britain, Augustus Fitzroy was not that leader.

Abram: But only time will tell, and you'll have to wait till next time. Then put the outro here.

Joe: And that's where we're going to leave things. The final script is written and should be landing in your feed in 2 weeks. Abram and I will be recording it in the coming days. But before we go, we have one other piece of business, Abram. We're going to listen to a trailer.

[TRAILER: The Alexander Standard]

Meredith: Hello, my name is Meredith—

Dustin: —and my name is Dustin—

Meredith: —and we're the hosts of the Alexander Standard. That's better. Inspired by the Rex Factor Podcast, we rank all the successors of Alexander the Great, from Perdiccas to Cleopatra VII. After Alexander the Great died, shit really hit the fan.

Dustin: Seriously, the Hellenistic world was a crazy place, and we've got some crazy stories to tell you. So please come check out our show, The Alexander Standard.

Abram: I can't wait to listen. I hope you listen too. Thanks for your comment, Manatee. So Manatee just said what we all want to hear.

Joe: Yeah, I thought it sounded like a great show. Definitely did. And I'm a big fan of them, so please check them out wherever podcasts are found. And I think we're done. So Abram, now this is your line.

Abram: This has been Prime Factors. I'm Abram and I'm here with my dad. Don't forget that you can find us on Apple Podcasts, Spotify, YouTube, and at primefactorspodcast.com. We're also on Facebook and BlueSky. If you enjoy listening, please like, subscribe, comment, and review.

Joe: All that's left is say goodnight, Abram.

Abram: Good morning.

Bibliography

Joe: That's a wrap. Bibliography. First and foremost, I want to thank the podcasters who graciously offered to lend their voices to our episode. It has been a fantastic year and we have made really good friends. Rob from Totalis Rankium read William Petty, the Earl Shelburne. He'll be Prime Minister eventually, so maybe I can convince Rob to come back if we do a skit with him. His current Pirates: Totalis Rankium is one of my favorites, and their Beatles series on Patreon is Abram's favorite podcast by far. John from Prime Time read Frederick North, the Lord North. He'll be Prime Minister next, so maybe he'll come back as well. He was joined by Rob, also from Prime Time, reading Henry Seymour Conway. And finally, Kassidy Kearey. He's a frequent contributor to El Podcast de Freakdom, a Spanish-language geek culture podcast, and has some amazing music on his YouTube channel, @KassidyKearey. That's K-A-S-S-I-D-Y K-E-A-R-E-Y. Abram's friend Adam also contributed some music to this episode. My key sources this time are: The Royal Fitzroys by Bernard Falk, written in 1950. It's one of those dynasty biographies similar to what I found for Devonshire, but it goes into fantastic detail on both Grafton and his grandfather. The Autobiography and Political Correspondence of Augustus Henry, 3rd Duke of Grafton, edited by William R. Anson in 1898. To avoid bias, I did not use any of Grafton's own words, but I found Anson's introduction and commentary to be very insightful. And as usual, I also used a myriad of smaller sources. I had hoped to find a more recent biography, but Grafton is not high on many lists of prime ministers to write about. Our amazing editor who has put up with integrating so many voices this week is Palle Bo. There is no one I can recommend more for podcast editing and production. You can find him at radioguru.co.uk. Thank you for listening. We'll be back in 2 weeks with Grafton Part 2. Did you like this episode, Abram? Very much.

Dorie: Hi, I'm Dorie, Abram's mom and Joe's wife, or the non-podcasting member of this family. These guys work really hard on their podcast, and they love it when people make comments. So if you're enjoying their stuff, leave a comment for them. They get really excited. Produced by radioguru.co.uk.

Join the Discussion: We want to hear your ranking! Find us on BlueSky at @primefactorspod.bsky.social

Support the Show: If you enjoyed our show, please leave us a rating or review on your podcast app. It helps others find us and makes Abram very excited.