Guess Who: New Framework Advances Probabilistic Capabilities in LLMs

Anonymous social media users can now use large-language models (LLMs) to know the likelihood of someone guessing their identity based on the information they disclose in their posts.

That’s because a new framework is improving the probabilistic reasoning of LLMS like ChatGPT and Gemini.

School of Interactive Computing associate professors Wei Xu and Alan Ritter have shown in a recent paper that Bayesian reasoning enables accurate probability population statistics from LLMs.

Xu and Ritter will present their findings from the paper, titled Probablistic Reasoning with LLMs for Privacy Risk Estimation, next week at the 2025 Conference on Neural Information Processing Systems (NeurIPS) in San Diego.

Xu’s and Ritter’s experiments challenged LLMs to estimate the privacy exposure risk of information shared in anonymous social media posts. The results showed that Bayesian reasoning outperforms chain-of-thought (CoT) reasoning in solving probability-based math problems.