Erik_
I actually recently was working on text compression...
Sat Sep 17, 2022 12:37am

for transmitting over a REST request. I've started working on a Perl program(s) now that it's been my current language of fixation for the past few months. Once it gets a bit more polished, I'll make an actual post about it.

Anyway, it works like:
* Server has a bunch of text data documents that hold a bunch of attributes and just a lot of dense data in text form. I went this way so I didn't have to deal with a DB (with Heroku stopping free DBs I don't have one lined up just yet). DB or not, the data would still be sent the same way.
* When the server starts up, it reads all these data files, compresses them, and then base64 encodes the compressed data. The data is stored in a 3d hash.
* Client makes request for data at x,y,z to server. Server returns the base64 encoded compressed data in a JSON response.
* Client then base64 decodes the data and decompresses it. Then unpacks the now readable data into a usable object.

Lot of steps but it decreased the size of the REST response by a TON. Also, the speed at which the client decodes, decompresses, and unpacks the data is still super fast that it's not even noticeable.

Long story short, even if you had to compress the data for the story files, it wouldn't be that noticeable. Mine was only done this way because the data is being transferred from server to client. I think the uncompressed data in a DB wouldn't be an issue even if you had 100s of stories generated. Ex: the DB for this site currently has ~115k rows and is only 50MB.. and there are a lot of unoptimized columns where I just went wayyy too large to be safe.

The cell modems seem to be getting more popular with 5g rolling out. Also for carriers, if they're also mobile carriers, it's sort of trivial for them. Just make a "phone" into a router. For them, it just acts as a phone on their network.

Yeah the FujiNet thing seems to be pretty cool. Yes, I believe it will print to modern printers. It even emulates the print styling of a couple of different actual Atari computer printers so the print outs look as though they were printed on the real deal.




  • Sometimes I forget just how well text compresses - Puckdropper, Thu Sep 15 2022 4:41am
    and probably decompresses. It can be fairly quick too. Might be something that's doable still... The biggest issue is figuring out the best format for the story storage and then everything else would fall in to place. Good design here means lots of good compatibility later. Kinda like how Word... more
    • I actually recently was working on text compression...- Erik_, Sat Sep 17 2022 12:37am
      • I can see why exactly that works - Puckdropper, Thu Sep 22 2022 5:23am
        Your machine is much faster than the network, so it's going to be much faster if it can do the work rather than wait for the data to be transmitted. Sometimes I wonder if Internet traffic could be more compressed so the CPU does the work rather than the other CPU or programmer. It could really help... more
"Forces act when not restrained" - Puckdropper