The Kindness of Sisters: Annabella Milbanke and the Destruction of the Byrons

There is something so reassuringly dull about the Milbankes’ political careers, so entirely lacking in individuality, that one feels instinctively with them that one is in touch with the solid bedrock of Sir Lewis Namier’s England. Sir Ralph had first entered parliament as one of two unopposed members for Scarborough in 1754, and at the first election of the new reign stood in the Holderness interest for Richmond, loyally and uncritically supporting successive administrations, before retiring in 1768 without having spoken a single word in fourteen years an MP.

Over twenty years were to pass before another Milbanke sat in Parliament, but through the 1770s and 80s Sir Ralph’s son, another Ralph, continued the same process of family consolidation, hitching his political fortunes first to Lord Rockingham and then, on his death, to Charles James Fox. In 1790 after a ruinous campaign that is reckoned to have cost the family £15,000, he was finally returned in second place for Durham Co, and for the next twenty-two years remained its MP, a genial and ineffectual ‘Uncle Toby’ whose fidelity to the Whig cause, in his daughter’s succinct phrase, was ‘as little valued as doubted’.24

It was into this family and this world, on 17 May 1792, that Anne Isabella Milbanke was born. The future Lady Byron has always seemed to belong so completely to the nineteenth century that it is easy to forget that this is where her roots lie, that her moral and social being was shaped by the inherited virtues and limitations implicit in Stubbs’s painting or her family’s dilettante public service.

But if the young Annabella was brought up in a political milieu, behind the web of alliances and obligations that supported two generations in parliament lay realities of landed life that had a far more profound effect on her vision. From the middle of the seventeenth century the principal seat of the Milbankes had been Halnaby Hall, a red-bricked Jacobean manor house, now gone, that lay just off the Great North Road outside the village of Croft in Yorkshire. In the village church of St Peter’s a wonderfully grandiose tomb and pew still evoke the dynastic ambitions of the early Milbankes, but Annabella’s affections remained all her life with the modest estate at Seaham on the north-east coast where she grew up. ‘If in a small village’, she recalled many years later, in a passage that might have come from George Eliot,

you cannot go out of the gates without seeing the children of a few Families playing on the Green, till they become ‘familiar faces’, you need not be taught to care for their well-being. A heart must be hard indeed that could be indifferent to little Jenny’s having the Scarlet Fever, or to Johnny’s having lost his mother … I did not think property could be possessed by any other tenure than that of being at the service of those in need.25

The Milbankes and Seaham may have given Annabella a sense of the rooted interdependency of country life, but through her mother she could lay claim to a more exotic strain of English history. Judith Noel was born in 1751, the eldest daughter of Sir Edward Noel of Kirkby Mallory, first Viscount Wentworth and heir through the contorted female line to the sixteenth century Wentworth barony. It would be dangerous to describe any title that has survived with the tenacity of the Wentworths as ‘doomed’, but when a family branch can provide a Lancastrian standard-bearer at St Albans, a Governor of Calais under Mary Tudor, and the devoted mistress to the Duke of Monmouth, it is at least guilty of the kind of ill-luck that might pave the way to a marriage with Byron.

For someone so outwardly prosaic as Annabella, there was, too, a curiously vivid streak of romanticism that fed directly off her sense of history. In a self-portrait written as a woman of thirty-nine, she looked back on her childhood self, on a miniature Dorothea Brooke pulled backwards and forwards between the claims of the imagination and the stern imperatives of a protestant conscience. ‘Impressed from earliest childhood with a sense of duty, and sympathising with the great and noble in human character’, she wrote,

my aspirations went beyond the ordinary occasions of life – I wasted virtuous energy on a visionary scene, and conscience was in danger of becoming detached from that before me. Few of my pleasures were connected with realities – riding was the only one I can remember. When I climbed the rocks, or bounded over the sands with apparent delight, I was not myself. Perhaps I was shipwrecked or was trying to rescue other sufferers – some of my hours were spent in the Pass of Thermopylae, others with the Bishop of Marseilles in the midst of Pestilence, or with Howard in the cheerless dungeon …

About the age of 13… I began to throw my imagination into a home-sphere of action – to constrain myself, from religious principle, to attend to what was irksome, and to submit to what was irritating. I had great difficulties to surmount from the impetuosity and sensitiveness of my character … It was this stage of my character which prepared me to sympathise unboundedly with the morbidly susceptible – with those who felt themselves unknown …26

It would clearly be absurd to try to define her exclusively in terms of ancestry, but there is a sense in which the solidity of the Milbankes and the romanticism of the Wentworth inheritance combined to produce in Annabella something distinctly new, a kind of fierce ordinariness, a strident centrality that raised the commonplace to the realms of genius, orthodoxy to the stuff of crusade.

Along with this dual inheritance, the circumstances of her own contented upbringing can only have sharpened the feeling of singularity with which she coloured the most ordinary imaginative experiences of childhood. In the same Auto-Description she lamented a ‘want of comparison’ in her Seaham life that blinded her to the advantages of birth, and yet of greater importance than the inevitable isolation of a small Durham village was the simple fact that she was the only child of parents who had waited fifteen years for an heir.

Ralph Milbanke and Judith Noel had married in 1777, and although they had brought up a niece as if she was their own child, there is no mistaking the ferocious joy that greeted Anna-bella’s birth. It is often admiringly noted that she was encouraged in her opinions from her earliest days, but if her childhood self can be back derived from her adult character, hers was the kind of independence that might have flourished more safely in the face of opposition than indulgence, her character one that would have fared better outside the warmth and admiration of a family that placed her firmly and uncritically at its centre. ‘It was indeed Calantha’s misfortune to meet with too much kindness,’ her cousin Caroline Lamb wrote of herself in a passage in Glenarvon that throws an unexpected light on this – a passage that sufficiently stung Annabella when she read the novel to have her mark and angrily refute its psychology in a criticism that survives still among her papers,

or rather too much indulgence from all who surrounded her. The Duke, attentive solely to her health, watched her with the fondest solicitude, and the wildest wishes her fancy could invent, were heard with the most scrupulous attention, and gratified with the most unbounded compliance.27

This regime of indulgence was made more dangerous in Anna-bella’s case by an intelligence that in and outside the home had little to challenge it. Even as a child she was conscious of being cleverer than most of those around her, but it was a cleverness dangerously at the service of unchallengeable moral certitudes, an intelligence that seems never to have broadened with reading or turned on itself in any genuine spirit of criticism. From the evidence of her letters and journals there was certainly a kind of scrupulousness about her, and yet even here her scruples and self-doubts were, like her shyness, the self-referential workings of an imagination that ultimately appealed to no other judgement but its own.

If Annabella Milbanke had simply married as Milbankes had traditionally married, none of this might have much mattered, and it is likely that she would have done no more than add one more name to history’s forgotten roll of mute, inglorious husbands. From the earliest family descriptions one can glimpse the formidable chatelaine she should have been, but substitute the name Byron for that of George Eden or any of her earlier suitors, see that ten-year-old girl with the determined pout Hoppner painted as the future Lady Byron, and the warmth, the love, the privilege and security of her sheltered upbringing suddenly seem the laboratory conditions for breeding the disaster of the most notorious marriage in literary history.

It is the inevitable condition of biography to shape a life with the benefits of hindsight in this way, and yet it is only hindsight that casts a shadow over the prelapsarian happiness of Annabella’s childhood. In her own eyes the memories of Seaham would always have the poignancy of blighted innocence, but the horror is that it could have ever equipped anyone so essentially limited in experience or culture to imagine that she could understand or tame a Byron.

It is often forgotten, in the feeding frenzy that invariably accompanies her name, how vulnerably young she was when she first met him in 1812, and yet nothing suggests that another summer or two would have made the difference. She had come up to London for her first season in the previous year, and although there were suitors enough to satisfy anyone’s vanity, not even a future governor-general of India or Wellington’s adjutant general in the Peninsula had been sufficient to jolt her out of the complacent certainties of her Seaham world. ‘I met with one or two who, like myself, did not appear absorbed in the present scene’, she later wrote of this period,

and who interested me in a degree. I had a wish to find among men the character I had often imagined – but I found only parts of it. One gave proofs of worth, but had no sympathy for high aspirations – another seemed full of affection towards his family, and yet he valued the world. I was clear sighted in these cases – but I was to become blind.28

It was a misfortune, too, for a woman who could think like this to see her future husband for the first time in his annus mirabilis, because if there were far more interesting ‘Byrons’ than the triumphant author of Childe Harold, there was none more likely to appeal to a romantic moralist of Annabella’s stamp. ‘Lavater’s [the phrenologist] system never asserted its truth more forcibly than in Byron’s countenance’, the portrait painter Sir Thomas Lawrence wrote at the height of Byron’s fame, wonderfully capturing the mix of glamour and threat in the figure that seduced London’s ‘golden parallelogram’ in the spring and summer of 1812,

in which you see all the character: its ken and rapid genius, its pale intelligence, its profligacy, and its bitterness; its original symmetry distorted by the passions, his laugh of mingled merriment and scorn; the forehead clear and open, the brow boldly prominent, the eyes bright and dissimilar, the nose finely cut, and the nostril acutely formed; the mouth well made but wide and contemptuous even in its smile, falling singularly at the corners, and its vindictive and disdainful expression heightened by the massive firmness of the chin, which springs at once from the centre of the full under-lip; the hair dark and curling but irregular in its growth; all this presents to you the poet and the man; and the general effect is heightened by a thin spare form, and, as you may have heard, by a deformity of limb.’29

Byron was just twenty-four when, after more than two years’ travel across Europe and the east, the sudden and unprecedented success of Childe Harold changed his life and the course of Romantic literature. He had already produced some feeble juvenilia and a long and scabrous satire he had since come to regret, but nothing in his literary or private life, nothing in the intense and homoerotic friendships of his Harrow and Cambridge days or the bisexual philandering in the Levant had prepared him emotionally for the loneliness of fame that swamped him on his return, a poet without conviction, an aristocrat without a sense of belonging, a liberal without the stamina or will for political life, an icon with a morbid sensitivity to his lameness.

It would have been odd in fact if Annabella alone had not felt drawn to Byron that summer, and yet even in the privacy of her diary and letters she felt she owed her intelligence some more refined expression of her feelings than the general excitement that gripped Regency society. She had first seen him at a morning waltzing party given by Caroline Lamb on 25 March, and after filling her journal that night with her impressions, the next day reported back to her mother in Seaham. ‘My curiosity was much gratified by seeing Lord Byron, the object at present of universal attention’, she wrote,

Lady Caroline has of course seized on him, notwithstanding the reluctance he manifests to be shackled by her … It is said that he is an infidel, and I think it probable from the general character of his mind. His poem sufficiently proves that he can feel nobly, but he has discouraged his own goodness. His features are well formed – his upper lip is drawn towards the nose with an expression of impatient disgust. His eye is restlessly thoughtful. He talks much, and I heard some of his conversation, which is very able, and sounds like the true sentiments of the Speaker.

I did not seek an introduction myself, for all the women were absurdly courting him, and trying to deserve the lash of his satire. I thought that inoffensiveness was the most secure conduct, as I am not desirous of a place in his lays. Besides, I cannot worship talents that are unconnected with the love of man, nor be captivated by that Genius which is barren in blessings – so I made no offering at the shrine of Childe Harold, though I shall not refuse the acquaintance if it comes.30

The acquaintance finally came the next month at a party of Lady Cowper’s, and with it the note of ironic detachment became increasingly hard to sustain. In her letters home to her mother she continued to insist that ‘calm benevolence’31 alone could touch her heart, but no amount of dissembling to her parents or herself could disguise the fact that curiosity was rapidly turning into the crusade that would shape her whole life. ‘Do you think there is one person here who dares to look into himself?’, she later recalled the question that had inspired her first mute ‘offering’ at the shrine of Childe Harold,

… I felt that he was the most attractive person; but I was not bound to him by any strong feeling of sympathy till he uttered these words, not to me, but in my hearing – ‘I have not a friend in the world!’ It is said that there is an instinct in the human heart which attaches us to the friendless. I did not pause – there was my error to enquire why he was friendless; but I vowed in secret to be a devoted friend to this lone being.32

There is something unsettling in the reveries of the young Anna-bella, or at least in her incapacity to see them for what they were. The descriptions of Byron that litter her diary and letters are as banal as those of anyone else that season, but running through them is that old and dangerous sense of election, the conviction of some private and silent understanding that set her apart in a city swept along on the rhythms of the waltz and the voyeuristic thrill of Caroline Lamb’s pursuit of Byron.

If there was nobody to blame for these delusions but Annabella, however, it is clear that she had not just imagined Byron’s interest in her. In the wreck of their marriage she once accused him of only ever wanting what he could not have, but if there is something in that, the more brutal truth is that he simply could not see her for what she was – could not see the provinciality that passed for independence, the rigidity latent in her strength, the narrowness which, with the nostalgia of the jaded sophisticate, he wistfully put down to moral superiority. ‘I set you down as the most puzzling person there’, he later told her of the first time he had seen her, across a room full of morning-visitors at Melbourne House,

For there was a quiet contempt of all around you & the nothings they were saying & doing in your manner that was so much after my own heart. There was a simplicity – an innocence – a beauty in your deportment & appearance which although you hardly spoke – told me I was in company with no common being.33

As the spring of 1812 wore on, and Byron’s life drifted dangerously and publicly into the chaos of his notorious affair with Caroline Lamb, it was not so much what Annabella was as what she was not that attracted Byron. The kinds of virtues and solidity with which he invested her would always hold a theoretical attraction for him, but it is hardly an exaggeration to say that if there had been no Caroline Lamb to escape then there would have been no marriage to her cousin, Annabella.

Byron had first come across Caro Lamb, the twenty-seven-year-old, fragile, androgynous-looking child wife of William Lamb, when she had written to him under a thin veil of anonymity after the triumph of Childe Harold in February 1812. Even before she had set eyes on him she had declared that she would know its author if he was ‘as ugly as Aesop’34, and within weeks of meeting she had made sure their affair was public property, played out with a kind of arriviste relish on his part and on hers with a reckless exhibitionism hovering on the edge of insanity.

There is not a moment of an affair that defined and caricatured the Romantic passion in all its delinquent intensity that has not been raked over a hundred times, but in the context of his relationship with Annabella it still has its place here. In later years Byron came to hate Caroline with a passion that only Claire Clairmont could otherwise inspire, but in its first weeks at least what attracted him to the maddest of all the Spencers was precisely the wayward and uncontrollable element in her that he eventually came to loathe.

There was a wonderfully sane and balanced side to Byron that would always in the end tire of romantic excess, and yet after the initial excitement had passed something more than boredom turned him against Caroline Lamb. An illicit element to some of his earlier, male relationships had sometimes unnerved him, but as he tried to distance himself from Caroline he found himself contending with a woman ready to call the Childe’s bluff, to live out the implications of his Romanticism with a patrician contempt for convention that in his first year of success he had neither the courage nor the confidence of ‘belonging’ to match.

Even with the contrast of Caroline Lamb to concentrate his mind, it is unlikely that his interest in Annabella would ever have quickened into anything more important had it not been for the intervention of her aunt (Caroline Lamb’s mother-in-law), Lady Melbourne. By the time that Annabella made her London debut, the girl in Stubbs’s portrait had reigned as the‘spider queen’ of Whig society for over a generation, ‘a sort of modern Aspasia’ with the brain and morals to match, tolerant, attractive, intelligent, cynical, corrupt and – to Byron at least – ‘the best, the kindest, and ablest female I have ever known, old or young’. ‘She was a charming person’, he later told Lady Blessington,

uniting the energy of a man’s mind with the delicacy and tenderness of a woman’s. She had all of philosophy, save its moroseness, and all of nature, save its defects and general faiblesse … I have often thought, that, with a little more youth Lady M. might have turned my head, at all events she often turned my heart, by bringing me back to mild feelings, when the demon passion was strong within me. Her mind and heart were as fresh as if only sixteen summers had flown over her, instead of four times that number – and the mind and heart always leave external marks of their state of health. Goodness is the best cosmetic that has yet been discovered … She was a captivating creature, malgre her eleven or twelve lustres, and I shall always love her.35

Even in her early sixties, the ‘spider’ or the ‘thorn’ still retained the power, desire and intelligence that had once made her the mistress of Lord Egremont and the Prince of Wales. As a young bride she had been forced to stand and watch her husband’s ludicrous pursuit of the actress Sophia Baddeley, but with the ambition and purpose Stubbs caught so well, disappointment had simply deflected her energies into the ruthless pursuit of family influence that was to consume the rest of her life. ‘The charms of her person and the endowments of her mind were worthy of a better fate than that she was preparing for herself’, Caroline Lamb wrote savagely of her in Glenarvon, the roman à clef with which she took her revenge not just on Byron but on the Whig world that had turned its back on her,

But, under the semblance of youthful gaiety, she concealed a dark intriguing spirit, which could neither remain at rest, nor satisfy itself in the pursuit of great and noble objects. She had been hurried on by the evil activity of her own mind, until the habit of crime had overcome every scruple, and rendered her insensible to repentance, and almost to remorse. In this career she had improved to such a degree her natural talent of dissimulation, that, under its impenetrable veil, she was able to carry on securely her darkest machinations; and her understanding had so adapted itself to her passion, that it was in her power to give, in her own eyes, a character of grandeur, to the vice and malignity, which afforded an inexplicable delight to her depraved imagination.36

With daughter and mother-in-law both living in Melbourne House, a delicate balancing act was required, but from the moment Byron tried to extricate himself from the affair with Caroline, he found Lady Melbourne a subtle and determined ally. To a woman who had charmed and slept her way to the top of Whig society, her daughter-in-law’s morals were of no great concern, but the one crime the lax Regency world would not forgive was indiscretion and as Caroline’s antics began to threaten the dynastic ambitions ‘the spider’ held for her family, Lady Melbourne moved to neutralise her.

In spite of all her cynicism, though, it can no more have occurred to Lady Melbourne than it had to Byron that the solution to their mutual problem lay in her cool, self-contained country niece. For their different reasons both Caroline Lamb and her sister-in-law ‘Caro George’ had done their best to convince him that Annabella was already engaged, but he had hardly needed their warnings to keep a wary distance from a woman he instinctively recognised as his opposite. ‘My dear Lady Caroline’, he had written as early as 1 May, after reading some verses of Annabella’s,

I have read over the few poems of Miss Milbank with attention. – They display fancy, feeling, & a little practice would very soon induce facility of expression … She certainly is a very extraordinary girl, who would imagine so much strength & variety of thought under that placid countenance? You will say as much of this to Miss M. as you think proper. – I say all this very sincerely, I have no desire to be better acquainted with Miss Milbank, she is too good for a fallen spirit to know or wish to know, & I should like her more if she were less perfect.37

There is no reason to doubt Byron – Annabella is not mentioned in his letters for more than four months – but the idea of her had taken a dogged hold of his imagination. There was no pretence on his part that he was in love with her, or anyone else, but as over July and August the pressure from both the Lambs and Caroline’s mother to end the affair grew, he was faced with the fundamental need of the outsider to assimilate or face destruction. ‘I see nothing but marriage & a speedy one can save me’, he wrote to Lady Melbourne on 28 September,

if your niece is attainable I should prefer her – if not – the very first woman who does not look as if she would spit in my face.38