Letters and documents from the time period suggest that the Prince found her physically disagreeable, and often referred to her as a "fiend". By 1797, (only two years after their nuptials) the couple had had a child and were already separated. The Princess moved from the matrimonial home, and was denied custody of her daughter Charlotte. Rumours quickly spread about the sexual promiscuity in Caroline's court. She was accused of engaging in adulterous acts with several different men, and also giving birth to an illegitimate son. In order to discover the truth, George, now the Prince Regent, commissioned an investigation of her conduct. It became known as the "Delicate Investigation", because it was to be conducted with the utmost discretion. Many rumours and opinions were uprooted during the investigation, but the commission concluded:.
We are happy to declare to your majesty our perfect conviction that there is no foundation whatever for believing that the child now with the Princess is the child of her Royal Highness, or that she was delivered of any child in the year 1802 .
It did, however, find evidence of impropriety by the Princess for which it refused to comment on further. Instead, the report read: "these we submit to your Majesty's wisdom" Despite these positive findings, Prince George continued to mistrust her. Caroline was ostracized by the upper classes of London, and would not be received at the Prince's court. In 1814, she decided to travel and live abroad. "Above all she wanted to be free - free from the boredom of her cautious political advisors, free to choose her own friends and free to go exactly where she liked - in other words to be royal without the restrictions of royalty." Upon news of Charlotte's death in 1817, it was unclear whether Princess Caroline would ever return.
In 1820 Caroline's situation changed dramatically. In January, Britain witnessed the demise of the reigning King George III, and welcomed Caroline's estranged husband to the throne, now King George IV.
