I was watching the RFK Jr questioning today and when Bernie was talking about healthcare and wages I felt he was the only one who gave a real damn. I also thought “Wow he’s kinda old” so I asked my phone how old he actually was. Gemini however, wouldnt answer a simple, factual question about him. What the hell? (The answer is 83 years old btw, good luck america)

    • Smokeydope@lemmy.worldEnglish
      1·
      4 hours ago

      First you need to get a program that reads and runs the models. If you are an absolute newbie who doesn’t understand anything technical your best bet is llamafiles. They are extremely simple to run just download and follow the quickstart guide to start it like a application They recommend llava model you can choose from several prepackaged ones. I like mistral models.

      Then once you get into it and start wanting to run things more optimized and offloaded on a GPU you can spend a day trying to setup kobold.cpp.

      They both start a local server you can point your phone or other computer on WiFi network to it with local ip address and port forward for access on phone data.

      • blazeknave@lemmy.worldEnglish
        1·
        4 hours ago

        My primary desktop has a typical gaming GPU from 4 yrs ago Primary fuck around box is an old Dell w onboard GPU running proxmox NAS has like no gpu Also have a mini PC running HAOS And a couple of unused pi’s Can I do anything with any of that?

        • Smokeydope@lemmy.worldEnglish
          1·
          3 hours ago

          Your primary gaming desktop gpu will be best bet for running models. First check your card for exact information more vram the better. Nvidia is preferred but AMD cards work.

          First you can play with llamafiles to just get started no fuss no muss download them and follow the quickstart to run as app.

          Once you get it running learn the ropes a little and want some more like better performance or latest models then you can spend some time installing and running kobold.cpp with cublas for nvidia or vulcan for amd to offload layers onto the GPU.

          If you have linux you can boot into CLI environment to save some vram.

          Connect with program using your phone pi or other PC through local IP and open port.

          In theory you can use all your devices in distributed interfacing like exo.