Hello friends. I spent a good bit of hours looking through the lobsters codebase, curious and wondering if I could host my own instance.
Now, if you aren’t so well-versed in hosting websites, fear not – I will forward you some possible complaints a web administrator can have.
What could’ve been an hour ordeal and hosting a version immediately turned into some (typical) idealism. See, Ruby applications can eat a lot of memory. If you ever maintained a Discourse site you’ll see it eats a good chunk of the server RAM… to where some could dream about rewriting it in something more memory friendly.
So, that should already raise some eyes if you’re hoping to host an application. Ideally I could get a 1gig ram server for $5/month – but you see, you’re playing with fire. You’ll have to consider.
The lobste.rs about page reveals their production set up (emphasis mine):
Lobsters is hosted on three VPSs at DigitalOcean: a s-4vcpu-8gb for the web server, a s-2vcpu-4gb for the mariadb server, and a s-1vcpu-1gb for the IRC bot. […] Lobsters is cheap to run, so we don’t take donations.
I think we have different standards of cheap, though I guess the cost is some streetcred/reputation. What possible reputation could one garner hosting a throwaway website likely to close in a few months time? Little, yet let’s march on.
So yes, the emphasized: 8 gigabytes(!!) of RAM – and that’s not even including the database, a separate server(!!) with 4 gigs.
Here’s the tech stack that lobsters runs:
- They got their own email server – no thanks, will just outsource that
- They use MariaDB
- Rails
So, naturally, I was hoping to get some RAM saving and performance saving by switching from MariaDB to Postgres. Thus began our quest to switch it out…
You may have had a good luck seeing some SQL before. SQL, SQL language, the way to interact with your data. So there’s SQL, which is shared amongst databases, but then there’s also dialect specific SQL which comes with each database.
You would hope it’d be as simple as switching out the database configuration, but it’s not. You need to visit certain code-paths and update how the stuff is used because using database specific SQL happens more often than not.
Compounded by the fact that lobsters uses a complicated moderation & ranking comments system, each with recursive threads even though keeping it flat is easier to understand.
So it ended up taking more than a few hours. Am I happy with the end result?
I got something online, using postgres instead of mariadb. Here’s some test-data:
But I couldn’t be bothered to understand how to properly translate their byte-encoded rankings to postgres, and coalescing that to its thread hierarchy. So comments are currently “broken”:
The hierarchy isn’t rendering properly.
There are a few other cons to lifting the lobsters code:
- It requires email login
- It has weird moderation stuff going on
- It uses rails
You might be wondering, “Why not just host it as is?”
Because I think if I do, I wouldn’t be able to on a 1gigabyte box. In addition, I would prefer to avoid using MariaDB if possible. It’s easier to stick to databases you know.
So where do we go from here? Why did you even want lobsters in the first place?
Because if one uses other software, it’s more centered around discussion rather than sharing links. I just want the front page, essentially. It makes it so it’s about sharing links, which I think is the only thing worth doing anyway.
This is backed up by a recent post I read analyzing the tragedy of neocities, courtesy of suboptimalism and remblanc.
Let’s extract a critical point:
Personal websites have their place, but with the amount of dead sites on Neocities it’s clear that many people simply don’t have a use for them, no matter how much “Old Web Movement” stuff gets touted. No matter how good it is to create a personal website, the conclusion is simple: not everyone needs one, wants one, or even should have one. Their needs can be—and are often fulfilled—elsewhere. It’s more of a sad fact than something to be bitter about.
Taken jointly with our 1% rule what could we extrapolate?
Well, if the Internet is an ideological war grounds, so the battalion divides and designates each their role, leading a successful space opera drama.
Not everyone wants to be a lieutenant, and especially not a general depending on the pressures.
But there are many infantrymen about and who doesn’t want to collect together and gain some honor in due process? Maybe you could begin to understand a bit more the 20k+ viewers of Twitch streamers – seeing it not as a spectacle, but a (forever unfulfilled) opportunity to be a part of something more.
So, instead of calling out others to Always Be Posting why not instead enjoy the silence of the digital library, leaving notecards-as-recommendations on whatever link you find?
Anyway, that’s the gist.
What are you even fighting for?
So if I were to create a lobsters clone, it’d be simple flat comments, lift the Discourse reply system, and hope for the best. No login required, too.
If you have a better idea, visionary you may be, inquiring minds want to know…