I wanted to extract some crime statistics broken by the type of crime and different populations, all of course normalized by the population size. I got a nice set of tables summarizing the data for each year that I requested.

When I shared these summaries I was told this is entirely unreliable due to hallucinations. So my question to you is how common of a problem this is?

I compared results from Chat GPT-4, Copilot and Grok and the results are the same (Gemini says the data is unavailable, btw :)

So is are LLMs reliable for research like that?

  • jet@hackertalks.comEnglish
    82·
    3 months ago

    It’s still a probable token generator, you’re just training it on your local data. Hallucinations will absolutely happen.

    • slacktoidEnglish
      11·
      3 months ago

      This isn’t training its called a RAG Workflow, as there is no training step per se