The website in the weights allows users to query large language models to determine if specific individuals are encoded within their internal parameters. The tool aggregates responses from various systems to generate a strength score, indicating how prominently a person appears in the model’s training data without external search tools. For instance, the creators currently hold scores of 175 and 262, while historical figures like Mozart or modern celebrities like Taylor Swift reach a maximum score of 996. The platform was developed by Joey Flynn and Thomas Dimson, both former employees of OpenAI. Their analysis suggests that appearing in smaller models, such as Meta’s Llama, signifies high relevance due to the limited parameter count. The creators also note significant limitations, including the potential for hallucinated biographical details, the impact of spelling errors on accuracy, and the difficulty common names face in retrieval tasks.
This development highlights the persistent memorisation capabilities inherent in current artificial intelligence systems, raising questions about privacy and data retention. It demonstrates that models can recall personal information without requiring active web searches, which challenges existing assumptions about how these tools handle private data. The scoring mechanism provides a tangible metric for understanding the depth of knowledge stored in different architectures, revealing disparities between massive models and their smaller counterparts. As these systems evolve, understanding what remains in their weights becomes crucial for assessing risks related to personal information leakage and the boundaries of automated recall.
- The in the weights platform reveals that large language models can recall specific individuals without external tools, assigning a strength score based on retrieval frequency.
- Former OpenAI staff Joey Flynn and Thomas Dimson built the site, noting that appearing in smaller models like Llama indicates high relevance to the training data.
- Significant limitations exist, including model hallucinations, the negative effect of typos on scores, and poor retrieval rates for common names.
Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.
