Charlotte was not the daughter of the Duke and Duchess of Devonshire but she was raised by both of them. She was actually the Duke's first (known) child and was born of the commoner Charlotte Spencer (no relation of Georgiana or her Spencer family). Charlotte was born in 1774, the same year that Georgiana married William so he technically hadn't committed adultery in Charlotte's conception. However, he did keep Charlotte's mother as his mistress after her birth, so we can't totally clear him of those early adultery charges.
Charlotte's early life was likely comfortably lived with her mother by the grace (and wallet) of the Duke. He likely provided them with a decent home and even paid for Charlotte to have a nurse. By 1778 he had tired of his mistress, and unfortunately for the the tot, Charlotte Spencer died not too long after the breakup. Now Charlotte was both motherless and a bastard-child. Most children in this situation were doomed a horrible and short existence unless they were charitably recognized by their father. Despite his coldness, and awkwardness with human affection, William was no monster and took his illegitimate daughter, and her nurse, into his home.
By the time the four or five-year-old came into the Devonshires' home, Georgiana and the Duke had been married at least four years and still had no children of their own. You may think that having the illegitimate daughter of her husband in her home would send Georgiana into a rage, but it did not. She immediately doted on the child and gushed about her as if she were her own, much to her mother's horror. Georgiana seemed impertinently ignorant of the whispers about welcoming the little girl into her family. At the time little Charlotte had no surname so the last name of Williams was created for her to distinct her as her father's child-though not legetimate child.
Charlotte grew up dearly loved by both Georgiana and William but with the arrival of Bess she became a neccessary tool in order to keep Bess close to the Devonshires without causing too much scandal. Lady Bess Foster was given the position of Charlotte's governess when the girl was about nine years old. As soon as this arrangement was settled the two set off for France to further Charlotte's education and refine her as a proper little lady (with Bess!? Dear lord, they should recheck her papers...). When Bess got bored with her task at hand and went partying in Europe poor, introverted Charlotte was dragged with her. Georgiana was horrified when word got to her that Bess and Charlotte were living in Naples with two of Bess' lovers. Eventually it was decided that Bess might not be the best fit for Charlotte, who seemed traumatised by her experiances abroad with Bess rather than "much improved." She was left in Paris to continue her education, even through the Revolution.
In 1794 the Devonshires were happy (and relieved) when their agent's nephew asked for Charlotte's hand in marriage. He was no aristocrat, but it was a good and secure marriage, especially considering Charlotte's unfortunate background. Nothing else can really be known of her after that, but Charlotte was likely relieved to settle down to the quiet life of a wife after her circus-like life as a Devonshire.