So I recently (a couple months ago) moved my fragmented docker-on-raspberry-pi architecture over to a Proxmox cluster. I’m running it on a pair of HP DL360 G6s, and I couldn’t be happier.

Except, well, I could be happier with just one more thing: high availability.

In particular, I want HA for my OPNsense firewall/router, but eventually for more of the workloads my family are depending on for life in general - Home Assistant, Plex, Overseerr, Immich, etc etc.

My current storage setup is a couple ratty old ARM-based NASes - an ancient Netgear ReadyNAS and an even more ancient Qnap TS-410. They’re both populated with 4 x 4TB (max raw size they can take) using RAID5, so I get about 22TB usable across the pair of them. They mostly store media for my Plex setup, but also support my 2N+C backup strategy for stuff like Immich, Paperless, and other important data.

My high-level plan is to grab another DL360, so I can have a quorum, then introduce a new storage system that:

  • provides an iSCSI target for my Proxmox cluster; and
  • can eventually grow to replace my old NASes.

The two solutions I’m pondering are:

  1. Build a TrueNAS setup from scratch - mini ITX case, board - the lot
  2. Pickup something tried, true and proven in the market, like a Synology

Up front cost is a consideration - I have a family to feed, so I can’t just run out and buy an 8-bay enclosure and fully populate it with 16TB disks.

Whatever I get, I’m likely to want to start with, say, 3 disks and grow it over time.

So, I guess this is a call out to the community to share any and all successes, war stories, and other advice. The more technical, the better. I want to make a sound, data-based decision here, and anecdotes from others who think like me are the best way to set my compass.

Cheers for anything you can offer!

  • knaak@lemmy.world
    link
    fedilink
    English
    arrow-up
    3
    ·
    1 year ago

    One advantage of a separate TrueNas is that I run Proxmox Backup as a VM on the NAS. It’s entirely separate and obviously has access to my storage.

    • DeltaTangoLima@reddrefuge.comOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      edit-2
      1 year ago

      As do I. Not sure if it’s the right approach, but I run PBS in a CT on each node, backing up the other node’s guests.

      But, with shared storage, I could get away with just the one PBS, correct?

  • tvcvt
    link
    fedilink
    English
    arrow-up
    2
    ·
    1 year ago

    One more option to consider would be doing HA with shared ZFS storage. Basically, if you have a zpool on each system with the same name and add it as shared storage, you can set up replication and have HA run off that. It’s a pretty simple setup and you can get by with just a couple extra disks per node rather than a whole separate machine.

    • DeltaTangoLima@reddrefuge.comOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      Ah, interesting. Each node has a couple HDD slots free. I could populate those without introducing them to the hardware RAID set, and use ZFS on them instead? That could definitely be a good place for me to start. Thanks!

  • yamdwich@kbin.social
    link
    fedilink
    arrow-up
    1
    ·
    edit-2
    1 year ago

    High availability storage is what the Ceph integration is for.

    Edit: though it’s kind of a pain to set up and probably way overkill. A separate TrueNAS or similar appliance with a 10 gig uplink will be easier and probably just as reliable for your use.

    • DeltaTangoLima@reddrefuge.comOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      edit-2
      1 year ago

      I did look at Ceph as well, but balked at the 10gig requirement. I only have a couple of SFP slots free on my switch, which otherwise only provides 1Gbps sockets. Could be a future upgrade path, though, if I want to get to play with full HCI.