I wanted to NDJSON. However, JSON doesn’t support keywords, and I have no idea how to map an alist to an object or an array automatically. So I wonder if there is any format for storing a large number of S-expressions instead.

  • @vi21OP
    1•2 years ago

    @vi21 I use conspack for this. It’s very fast in my experience send encodes and decodes to binary. It also allows you to customise it to support custom classes. I can show some example code later when I’m in front of a computer.

    Do you put one big expression per file or many expressions per file?

    • Elias Mårtenson
      1•2 years ago

      @vi21 that depends on use case. I haven’t actually used conspack to serialise data to a file. I’ve used it for network communications, and typically when storing objects to be sent over an MQ connection.

      But I think that if I were to save object state to a file using this system, I would construct the entire object graph in memory, and then serialise it using a single call and write it in one go. Unless of course I had specified requirements that would make that a bad solution. For example, if the amount of data is very large.

      • Elias Mårtenson
        1•2 years ago

        @vi21 I just looked up ndjson and realised that I had misunderstood what it does. I was thinking of that binary json format (bson I think?).

        My apologies for giving you irrelevant answers.

        • @vi21OP
          1•2 years ago

          Conspack is useful to me since my program currently saves serialized data to RocksDB. Anyway, I think it may be convenient to be able to work without the RockDB layer.