Saturday, January 12, 2019
It's no longer possible to write a web browser from scratch, but it is possible to write a gopher browser from scratch
As I mentioned two months ago, I've been browsing gopherspace. At the time, I was using an extension to Firefox to browse gopherspace, but a recent upgrade to Firefox left it non-working. I could use Lynx but it includes unnecessary pauses that make it feel slower than it really should be. I also don't care for how it lays out the page.
So I've been writing my own CLI gopher client.
And it's not like the protocol is all that difficult to handle and everything is plain text.
How hard could it be to download a bunch of text files and display them?
The protocol? Trivial.
Displaying the pages? Er … not so trivial.
The first major problem—dealing with UTF-8.
Problem—the terminal window can only display so many characters per line
(default of 80 usually).
There are two ways of dealing with those—one is to wrap the text to the following lines(s),
and the other is to “pan-and-scan”—let the text disappear off the screen and pan left-and-right to show the longer lines.
Each method requires chopping text to fit though.
With ASCII, this is trivial—if the width of the temrinal is N columns wide,
just chop the line at every N-bytes.
This works because each character in ASCII is one byte in size.
But characters in UTF-8 take a variable number of bytes,
so chopping at arbitrary byte boundaries is less than optimum.
The solution may look simple:
-- ************************************************************************ -- usage: writeslice(left,right,s) -- descr: Write a portion of a UTF-8 encoded string -- input: left (integer) starting column (left edge) of display screen -- right (integer) ending column (right edge) of display screen -- s (string) string to display -- ************************************************************************ local function writeslice(left,right,s) local l = utf8.offset(s,left) or #s + 1 local r = utf8.offset(s,right + 1) or #s + 1 tty.write(s:sub(l,r - 1)) end
but simple is never easy to achieve. It took a rather surprising amount of time to come up with that solution.
The other major problem was dealing with the gopher index files. Yes, they are easy to parse (once you wrap your head around some of the crap that presents itself as a “gopher index” file) but displaying it was an even harder problem.
Upon loading a gopher index,
I wanted the first link to be highlighted,
and use the Up and Down keys to select the link and then the enter key to select a page to download and view.
Okay,
but not all lines in a gopher index are actual links.
In fact,
there are gopher index files that have no actual links
(that surprised me!).
And how do I deal with lines longer than can be displayed?
Wrap the text?
Let the text run off the screen?
At first, I wanted to wrap long lines, but then trying to manage highlighting a link that spans several lines when it might not all be visible on the screen (the following lines might be off the bottom, for instance) just proved too troublesome to deal with. I finally just decided to let long lines of text run off the end of the screen just to make it easier to highlight the “current selection.” Also, most gopher index pages I've come across in the wild generally contain short lines, so it's not that much of a real issue (and I can “pan-and-scan” such a page anyway).
For non-text related files,
I farm that out to other programs via the mailcap facility found on Unix systems.
That was an interesting challenge I will probably address at some point.
There are still a few issues I need to address, but what I do have works. And even though it's written in Lua it's fast. More important, I have features that make sense for me and I don't have to slog through some other codebase trying to add an esoteric feature.
And frankly, I find it fun.
![[The future's so bright, I gotta wear shades]](https://www.conman.org/people/spc/about/2023/1023.t.jpg "The future's so bright, I gotta wear shades")