Get in Touch

I queried Google.com with the title question and got this from it’s AI.

======================================================

LLMs do not “store” information in a traditional database [1, 2]. Instead, their knowledge is either permanently baked into their internal parameters (the model’s “brain”) during training, or loaded temporarily into their context window (their “working memory”) during a prompt. [1, 2, 3, 4]
1. The Context Window (Working Memory)
This is the amount of text an AI can process in a single prompt or conversation. Because 1,000 tokens equal roughly 750 English words, modern AI capacities are immense: [1, 2, 3, 4]
  • Standard Models (e.g., GPT-4o, Claude 3.5 Sonnet): Typically hold 128k to 200k tokens, which is roughly 90,000 to 150,000 words (equal to 1 to 2 average-length books). [1, 2, 3]
  • Ultra-Long Context Models (e.g., Gemini 1.5/3 Pro): Process up to 1 to 2 million tokens, or roughly 750,000 to 1.5 million words. This allows the AI to “read” hundreds of files or entire code repositories in one go. [1, 2, 3, 4]

2. Internal Memory (Trained Knowledge)
Beyond temporary prompts, an LLM’s vast general knowledge is encoded mathematically into billions of parameters (the connections between artificial neurons). [1, 2]
  • A mid-sized model has tens of billions of parameters.
  • This allows the AI to “memorize” global facts, language rules, and vast amounts of public information, but it means the model cannot selectively “forget” or update its knowledge without a retraining cycle. [1, 2, 3, 4, 5]

3. External Memory (Databases & RAG)
Because LLM context windows fill up and are volatile, developers and businesses use Retrieval-Augmented Generation (RAG). RAG connects an LLM to an external vector database. [1, 2, 3, 4]
  • Instead of forcing the AI to memorize everything, the AI acts as a search engine operator, querying external databases to find the exact documents it needs, and then pulling only those specific documents into its temporary context window to answer your question. [1, 2, 3, 4]

If you are looking to build an AI system or want to manage a massive amount of your own data, tell me:
  • What type of information you need the AI to access (e.g., PDFs, a company website, codebases)
  • How much data you estimate you have (in gigabytes or total document count)
I can recommend the best tools or frameworks for setting up your AI storage!

End of AI answer.  I just reposted because it’s interesting to me.  If you need more info, the blue numbers are links to further info.

All the info above came from Google’s Gemini.
 
Fred The Submarine Guy Raley

Recent Posts