For Sa-yi, the time has nearly come. Most of children she grows up with all live for the ceremony she will soon take part in. Most children pick a trade, a kind of specialization. Some work on the autofarms, some learn to decode the messages from the elders, and some make handicrafts and tools. Some children, of course, decide that the work isn’t worth the effort, and instead engage themselves with trivial matters, like competing to see who can climb the highest up the holy tower (a common demonstration of masculinity), or playing card games for extra snacks.

Sa-yi learns the scripture. She always has, for as long as she can remember. There are no adults in their cocoon, not since they all turned 8 (at which time there is another ceremony, the i/Ha, celebrating the sacred nature of the number). To her, the scripture is everything. Seemingly, the more she reads the more there is to read; the more questions are answered the more questions reveal themselves. But this doesn’t bother her; it excites her. Her trade is to read, and she is the best at what she does.

The i/Yi-ro is what one might call a coming of age ritual, underwent at twice the age of the i/Ha. The hatch in the center of the cocoon is opened for a child to descend into, and they are never seen again. The scriptures (the easiest of them, the Yajur) tell the children that they will receive their true name during the ceremony, and the children have no choice but to believe them. They don’t know what their true name would mean, or even what it might sound like, but they all feel a certain weight and importance attached to it. They’re only told it’s a judging, where the bad children will be punished for their misbehavior with a disgraceful name and and the good children will be rewarded for their effort with a beautiful name. It goes a long way towards keeping them in line, but it’s hardly necessary. Nobody really misbehaves in the cocoon, and there is no serious conflict. All the children are familiar with the invisible force on their heads that presents itself when they begin to disobey the scripture.

Today it is Sa-yi’s turn. She tries her best to suppress her shaking hands as she climbs down a set of stairs – irreverent oscillation is frowned upon by the Third Book of the Rig. The stairs soon give way to a gently sloping spiral slope that leads her deeper underground. At the bottom she finds a finely woven mesh door made of the same metallic material as the walkway. She pushes opens the door and steps inside. It is warmer here, and dark. Before she can take another step, a light finds her and she lays eyes on three vague, shadowy silhouettes.

The central figure speaks: “We will now begin with the 31st i/Yi-ro of this epoch. Sa-yi, step forward. Speak only when asked.”

She does as she is told, stepping forward onto a circular platform, feeling a slight but definite pressure as she does so. It’s not all so different from what the children feel every night as they go to sleep. It’s stronger, and much clearer.

“State your identity” the figure booms. Sa-yi begins almost immediately:

“My false name is Sa-yi, cocoon DF0, batch AA1, group DC3. I am an Interpreter with guild 71. Confirm?”

As she utters the last phrase, she stretches her arm out in front of her, palm facing forward. On it, an intricate tattoo of swirling lines and intertwining curves – her eigenkey – glows hot for a brief moment. Even the youngest among them knew the proper etiquette.

“Accepted.” they reply, in unison yet such that one can match each voice with its owner effortlessly. They are in perfect harmony.

“The truth of the i/Yi-ro is not as you have been told it is, Sa-yi. We do not judge here. There is no need to judge you. We have been judging you for a long time now, and we are nearly finished. The purpose of this event is for you to listen, rather than to speak.”

Sa-yi begins to feel the pressure on her head grow, and grow until it becomes unbearable and piercing and all-consuming and resonant with her skull. And finally, for the first time, she begins to see.

Colors rush forth, the normal colors included, but more colors too. Colors that one cannot see but with the mind’s eye. There is no end to the color, each more fresh than the last, and each unlocking a new visual perspective, in the same way a blind child is awakened to the world of art – and in that instant she realizes she has been blind all her life.

What Sa-yi sees, I cannot tell you – it is inexpressible in our languages. It is inexpressible not like 1 = 2 (which is illogical but expressible), but like that which cannot be pictured by in our minds. All I can give is a projection, like a lower dimensional slice of a deeper world. This is what she sees:

And her world goes black, pure black. She struggles to open her eyes only to realize they were already open as she regains feeling in her limbs. Her proto-vision will not return to her for several days. She feels as if she’s lost something; whatever it was, she knows it must have been unimportant. But she has gained something far more important.

The three figures speak again, but she doesn’t hear them; she sees: three blobs of pattern in three complimentary colors that she finds immensely pleasing together, as if they’ve been chosen from a higher-dimensional color wheel.

She knows, somehow, her true name: Na-ha-ze, after the current epoch, 780. She knows that this ceremony had been performed millions of times before. She knows everything about her new purpose, and the world she was born into. She is perfectly aware that most children were not given the Spectral Blessing; after all, how could they handle such a thing? Most were handed down some common name and sent on their way. A few were disposed of. Na-ha-ze feels this is all as it should be, and what she feels as the new Prima-Intendant is fact.

Somewhere above her, she senses a gate opening, and her field of vision is washed away by the most beautiful colors she has never seen.

—

1 Introduction

A few days ago, my article was posted to lobste.rs and it got a comment from Celeritas linking me to this argument in favor of vim-style editing rather than kakoune-style editing. I was typing up a response in the thread, but it ended up getting pretty long, so I decided to turn it into a sort of follow-up post.

That article is written by noctuid, a pretty prolific emacs community member (i remember reading their evil guide when I first started using doom something like 6 years ago now). I’ve never used kakoune, but even though I ultimately disagree with their take I can relate to many parts of it. Meow fixes many of the issues they mention with kakoune. I think that with editor discussions in general there’s an exceptional amount of subjective bias because of the cost it takes to learn new things. I’ll be biased because I sunk the cost to learn kakoune grammar, and that’s what I currently use. I don’t think there are many people who’ve switched the other way.

In this post I’d like to comment on some of the critiques noctuid makes about kakoune, especially with regards to navigation and selection and how meow seems to address many of their points. In some sense, this is a bit more of a sobered take compared to my post 2 years ago. It comes with a bit more experience and knowledge of how meow works internally.

2 Critique points

2.1 Reverse grammar hurts key reuse

The idea is that operator-pending mode is a separate context in which keys can be re-used, like how w means “word” after d but “next word” in general. Noctuid takes issue with having to use a modifier to select text objects.

I think it’s really just a layout issue – indeed in meow we use , and . on the recommended layout. When you’re selecting a text object, that is a separate context in which keys are interpreted differently. You can reuse every key to select a text object. I might be misunderstanding what was meant by this part, but as I see it you’re losing nothing.

2.2 Navigation vs Selection

When I first started using meow, I might have agreed that automatic selections were a bit distracting, but once you start ignoring them it really becomes a non-issue. In meow, any movement key or command that does not act on a selection cancels the active selection, meaning that if you make a motion without wanting to act on it the selection will probably disappear on your very next command. However, it probably is a very valid observation that most people will very rarely make a motion and THEN realize they want to act on it. I just think that there is a real benefit to automatic selections, and it’s worth it for the cases where you select intending to act.

2.3 Visual selections don’t prevent mistakes; overlays do

Noctuid correctly points out that visual selections don’t really prevent you from making mistakes, they only make it obvious and easy to fix when you do.

Luckily meow has visual selections and overlays. Meow implements nearly exactly what noctuid describes, but with great integration (compared to avy) and really smooth ergonomics for the vast majority of everyday editing use cases. In short, most movement commands in meow trigger overlays over the target locations of all possible repetitions of that command, letting you quickly and visually repeat a command some number of times.

I don’t agree that visual selections don’t prevent mistakes – I couldn’t tell you how many times I’ve fat fingered 8 instead of 7 or simply changed my mind after making a selection. Additionally, having both gives you two options: either press the right number by reading the hints, or spam. Some commands in meow reuse keys for their expand versions. For instance, if you select a line with meow-line (x), pressing the same key again selects one more line. If you mark a whole word with meow-mark-word (w), pressing meow-next-word (e) expands by one word instead of selecting the next word only.

A subtle point which even many long time users of meow might not know the specifics of is this expandable selections. Every time you make a selection in meow, it is marked with some information that informs meow what to do on a repeat: whether it is an “expand” selection or a “select” selection, and what object the selection is targeting. The commands that make an “expand” selections are

• up/down/right/left expansions
• word/symbol based expansions
• line expansions
• block expansions (paren/bracket pairs, meow-block expands

up the tree).

The commands that make “select” selections and therefore do not reuse keys are

• find/till character
• general text-objects (perhaps might change in the future, there’s a lot of discussion about this on github)
• meow-visit

This whole setup makes expanding selections much easier, and basically gives you same user experience as highlighting text with a mouse: choosing your selection one object at a time. In my experience, I tend to spam keys to select <5 text objects, it’s just faster and takes less thought. In this “spam” selection mode, visual selection indeed does prevent selection errors, it’s slower and less efficient but exact.

2.4 Visual confirmation is unnecessary

From the article:

Furthermore, the effect of many motions and text objects is obvious; visual confirmation is often completely unnecessary. Deleting the current word, sentence, or parentheses block does not involve uncertainty. t and f are basically the “dumbest” motions in that they don’t move by clear units like paragraphs or words but instead jump to arbitrary characters. The majority of my operator usage has no possibility for mistakes apart from mistyping.It doesn’t make sense to me to optimize a modal model around rarer cases like the dtf example, even if I thought a visual selection was the ideal way to handle these cases.

I mostly agree with this point. A lot of my editing tasks are just “delete this word” or “change inside parens” which are so fast I barely see the selection before it’s gone. In this case though, visual selections cost nothing. So, I don’t agree with is the tradeoff part of their argument. It does make sense to optimize modal models around error prone cases, especially when there is minimal (and to me, mostly aesthetic) cost for doing so. I don’t think these cases are that rare; jumping to the n th occurrence of a character or selecting n lines is quite common!

2.5 Evil integration is better

In the “why not other emacs packages?” section, noctuid mentions that emacs packages use different keybindings for different actions, and that modal editing packages face a fundamental convention difference for motion keys (jk vs C-n, C-p). This is a good point to mention meow’s “motion” state, meant to address exactly this.

Essentially, it lets you override your motion keys in every mode that steals them from you automatically. Evil expects you to remap your keys in every one of your modes. Meow decides which modes to use motion in using a funny little heuristic that checks whether keys self-insert into the buffer and recursively against a predefined list of parent modes. Any command that’s hidden is rebound under the Hyper modifier, So, if you need to use the command that used to be bound to your motion keys, you can bind H-<key> under your leader prefix. It’s a dead simple idea, and it’ll always work.

While motion works for interaction-style packages like elfeed and magit, unfortunately, this is ultimately a valid criticism. Meow is forced to adapt to packages that nontrivially change your editing experience itself, like company, polymode, sly and cider to name a few. For these, meow comes with some “shims” that mostly just turn on the motion mode when its needed. Inevitably, this leads to bad user experience with some missed packages – but luckily, most compatibility issues are trivially solved by just turning on motion mode when it’s necessary, and indeed most shims that meow has do just that.

3 Ending thoughts

Noctuid’s post is 6 years old, and my response is obviously unfair since they’re writing about kakoune, and I’m talking about a modern package that has had the chance to learn from the many modal editing packages. Still, many of the critiques are about kakoune’s model itself, which meow uses. I still think it’s a good idea, or at least surely not a “a broken solution to a non-issue.” The instantaneous feedback and visual selections cost is mostly aesthetic, with a benefit of making selection errors trivial to notice and fix before acting on them.

At the end of the day, people edit differently – editors should be molded to the desires of the users, not the other way around.

1 Introduction

I enjoy lisp. There’s something really comfy about the whole repl-driven-programming workflow and the code-is-data mindset that make lisp dialects my first choices for nearly any task that isn’t particularly important (that’s python).

I currently live on the Miura peninsula, which means that my primary method of transportation is via Keikyuu railways, namely the mainline and the Kurihama line. I’ve never liked the way that timetables are laid out, both in real life and online. So, I thought I’d set up my own timetable viewer, and do so using some lisp.

1.1 Problems with timetables, and what we can do better

Here are my gripes with timetables.

Timetables in real life are usually laid out like this:

• Finding the next train, the most important task is now a three step process. First, check your watch to find the current time, then locate the correct chart (weekday/holiday), and then find the correct hour row. Typically, I’m only interested in the next few trains!
• With real charts, you can’t know when a particular train will arrive at the stop you want to go to.
• This one’s obvious, but these timetables are printed at the platforms, you’d have to use the internet to look up the timetable before you actually get to the station.

On the internet side, plenty of sites exist like ekitan that lay out a bunch of information.

• Switching between stations is a PITA, and requires loading >2 new purely navigational pages. This is annoying when I’m just trying to decide which station to walk towards.
• Checking train info (when a particular train will reach every stop) requires a new page load.
• There is no functionality to find the last (single) train from some station to another station. The best tool I know for this is google maps, which I don’t like to use for several unrelated reasons.

1.2 Goals

Here’s what I wanted to do:

1. Scrape some online rail site for the train information
2. Make a simple single page application that loads all the necessary data once and then does the computations fast, client-side.
3. Easy to read stop information relative to the current time
4. Easy to read arrival time
5. Easy to read last train time
6. Auto selects the current schedule (weekday/holiday)

This culminated in rail.esrh.me which I believe accomplishes all of these goals. This blog post is something like a recap of the lisp programming challenges and process that got me there.

2 Webscraping in clojure

2.1 Objective

The objective here, is to scrape a page like this https://ekitan.com/timetable/railway/line/8200 into a list of trains containing

• What kind of train service it is (express, normal, etc)
• The list of stops the train makes, where a stop is a station name and a time.

The data we want to see is the trains arriving per station. However, storing all that data takes too much space, which is a problem because that data will need to be sent to the client.

This wouldn’t be a problem with a backend that only returns the portion of the data necessary, but I knew that I would be hosting the site statically with Netlify, so I had to make sure the data was as small as possible.

Given the list of trains, and the stops each train makes, you can compute the trains stop times at a given station by just iterating through all the trains.

Therefore, all we need to do is look through all the stations, and only scrape the trains that start at that station.

2.2 Data representation

Clojure has a neat record syntax:

(defrecord Stop [station time]) (defrecord Train [stop-list type direction])

defrecord here automatically creates functions like (->Stop STATION TIME) which we can conveniently use later. However, there’s one really fatal flaw to using records for this that we’ll get to later.

2.3 Hickory in clojure

I opted for hickory as my scraping tool of choice. The way hickory works is that you first construct a selector that accurately describes the element you’re trying to access, and then apply hickory.select/select onto the selector and the hickory data.

Something neat to note from a functional programming perspective is how hickory represents the html tree with zippers. The selectors, which we’ll be using shortly, are functions that take zippers as arguments and return zippers if a condition is met. The resulting tree-like composability with selector-combinators is quite nice.

Obtaining hickory data from a url is simple using clj-http.

(require '[clj-http.client :as http]) (defn to-hickory [url] (-> (http/get url) :body parse as-hickory))

If you’re not familiar with the -> macro, it inserts each form as the first argument of the next form. So, the body of the function is equivalent to

(as-hickory (parse (:body (http/get url))))

Much harder to read, right? The threading macro lets you write the functions in the “correct” order, i.e the order they’re applied.

2.4 Scraping

Here’s a single view of a station page in a single direction on a single schedule. We’re interested in scraping each train that is marked with 当駅始発 (“starting at this station”).

Here’s how I do this:

(require '[hickory.select :as s]) (s/select (s/descendant (s/and (s/class "tab-content-inner") (s/class "active")) (s/class "ek-train-link")) station)

Here, I make a selector by combining several selector-combinators

• s/descendant verifies if the order of the arguments are hierarchical, skipping generations. The direct-children version would be s/children.
• s/and does what you’d expect

The reason I think it’s cool is because this allows you to describe what you’d typically do with CSS selectors with lisp syntax and the full expressive power of trees – which is an excellently ergonomic fit for a lisp language.

The CSS equivalent would be .ek-train-link .tab-content-inner.active which is IMO, tougher to read, especially given the space vs no-space change in semantic meaning.

Skipping some similar scraping, I produce a list of the trains with

(map #(->Train ((comp train->stop-list to-hickory :link) %) (:type %) direction) train-info)

Where train->stop-list does some similar hickory selections.

2.5 Ad-hoc-ing keikyuu data

There are two degrees of freedom for a train.

• The direction of a train, up or down
• The schedule of a train, weekday or holiday

Therefore, I represent this with a simple map:

{:up {:weekday up-weekday-trains :holiday up-holiday-trains} :down {:weekday down-weekday-trains :holiday down-holiday-trains}}

I suspect the OOP technique might be to abstract away the pair of weekday and holiday into a Direction and then abstract the pair of directions into Timetable or some BS like that, but the power of Clojure’s functional approach lets us preserve syntax elegance without the extra bloat.

Applying a function to both branches is

(defn apply-both-schedules [func] {:weekday (apply func [0]) :holiday (apply func [1])})

Which I use like this:

(def kurihamasen {:up (apply-both-schedules #(get-all-trains 679 [0 9] 1 %)) :down (apply-both-schedules #(get-all-trains 679 [0 9] 2 %))})

The function get-all-trains is applied with the % replaced with 1 and 2.

Reading a particular trainlist is

(defn indexer [data dir day] (day (dir data))) ;; usage: (indexer kurihamasen :up :weekday)

Which saves a few parens.

2.6 Saving data

Once all the data’s been scraped, we need to save it somewhere so that it can be used in the frontend.

I opted for using the Extensible Data Notation (edn) format. The nice thing about edn is that it is literally clojure!!.

Simply printing out the data, using yes, (print) or (pr-str) gives valid edn! It is a dead-simple implementation that powerfully leverages the whole “code is data and data is code” idea that lisp enables. We don’t need an extra serialization format like how javascript needs json to transmit data.

The downside to using edn is that it’s bloated and not portable as a result of my decision to use records.

You see, if I store a record (which is a map) into edn by printing it, I get something like timetable.core.Stop{:station "xxx" :time "xxx"}. There are two big issues here

Bloat

The :station and :time text is unnecessary for me to recover the data on the CLJS side. The whole class name is also unnecessary. This text is there for every single stop in the data, which easily bloats up the keikyuu data.

Loss of portability

If you were to try to read this data via read-string in another clojure project, you’d get an error unless you required the timetable library, since the data is now tied to the record name.

The second issue is pretty unacceptable, so I decided to convert the data into plain maps just before writing it to disk.

(->> (:stop-list train) (map (partial into {})))

Gzip or brotli compression do a sufficiently good job at bringing down the bloat to an acceptable 200kb or so.

3 Clojurescript frontend

3.1 Objective

The second component to this is to use the list of trains we scraped and display them in a convenient way.

The task is straightforward. For every station, look through the train list and find the next few trains coming through.

The big issue… is that I don’t know javascript, and I sure as hell don’t know any frontend frameworks like react or whatever people are using these days.

Luckily, clojure can be compiled to javascript via clojurescript (cljs)! Clojure code outputs javascript code which is then optimized with Google’s closure compiler that finally outputs obfuscated (not a good thing), minimized (not a good thing), and performant (good thing) javascript.

The other big issue is that I didn’t (still don’t?) know how to use clojurescript either.

What follows may very well be terribly not idiomatic, but I’d still like to document some of the challenges I faced as a web-development beginner, and what I learned.

3.2 Reading in the data

“So this is an easy first step, right?” I thought to myself.

Wrong. So wrong.

I need to fetch the data in an edn file via a GET request, and a quick DDG search suggests cljs-http as the way to go about this.

However, it doesn’t just block and return your data, but it returns a core.async channel…

I really struggled with this for a good while, looking for blocking alternatives, wondering if this was really worth the trouble to not just embed the data into the source code before I decided to sit down and figure out what CLJ(S)’ async is all about.

3.2.1 Cljs async in a nutshell

The core of async is the channel, which is a fifo queue.

(ns test.core (:require [cljs.core.async :as async]) (:require-macros [cljs.core.async.macros :refer [go]])) (def channel (async/chan))

We (synchronously!) put something on the queue via async/put!:

(async/put! channel true)

The second arg can be anything!

And now comes the tricky part, the go macro.

(go (js/alert (async/<! channel))))

To put it simply, the go macro rewrites at compile time your synchronous-looking code into a state machine that pauses whenever it hits a <!. This is why <! can only be used inside the go block. The above code won’t block the whole website, it just waits for something to appear on channel and only then proceeds with the rest of the code.

The equivalent of async/put! in a go block is async/<!.

1. Simple example

   A complete example, say to have two buttons that each cause a different alert, would look something like this:

   First in html, assuming we have

   <button id="button1"> button 1 </button> <button id="button2"> button 2 </button>

   We can easily get an object via its id with something like

   (-> js/document (.getElementById id))

   Here, -> is the thread-first macro that inserts each form as the first argument of the next, so it could be rewritten like

   (.getElementbyId js/document id)

   If you’re unfamiliar with clojure, the . syntax is how interop is used with the host language (either js or java). You can mentally swap the order of the s-exp, (.method object arg) is the same as object.method(arg).

   Using this interop, we can add two listeners:

   (-> js/document (.getElementById "button1") (.addEventListener "click" #(go (>! button-channel "Button 1 was clicked!")))) (-> js/document (.getElementById "button2") (.addEventListener "click" #(go (>! button-channel "Button 2 was clicked!"))))

   What’s nice about this architecture is that both buttons publish on the same channel allowing us to write one handler that dispatches based on the value in the channel.

   (go (while true (js/alert (<! button-channel))))

   Even though this looks like an infinite loop, it async pauses on the channel poll.

   This pattern of go + looping (maybe infinitely) is common, so there exists a go-loop macro in core.async as well.

3.2.2 Pub-sub channels

In my use case, the train data kicks everything off, but takes a significant amount of time to load.

What might have been possible was to make a single channel for the train data, get the train data via XHR and put it onto the channel. However, it turned out while I was writing the program that a lot of different processes needed access to the same data. For example, the radio-button listeners that reload the data, the listeners for the user clicking on a specific train, and the main thread that loads the data for the first time. You can’t just “peek” a channel in cljs, reading the data pops it.

The solution I found for this was to make a pub-sub channel, where you publish to one channel and then several subscribing children automatically copy the data onto their own (single-read) channels.

(def publish-data-channel (async/chan 1)) (def sub-data-channel (async/pub publish-data-channel :data)) (def subscriber-for-radio-buttons (async/chan 1)) (async/sub sub-data-channel :all subscriber-for-radio-buttons) (def subscriber-for-train-info (async/chan 1)) (async/sub sub-data-channel :all subscriber-for-train-info) (def subscriber-for-last-trains (async/chan 1)) (async/sub sub-data-channel :all subscriber-for-last-trains)

Each subscriber channel is used like a normal one in a go block. Pushing to all three channels can then be done in one shot by pushing to publish-data-channel.

3.3 Displaying the data

Everything from here on was pretty straightforward to write, since it involved minimal new ideas.

3.3.1 Issues with dates and times

One immediate issue is that I’d like to consider all the trains that start some time after midnight as part of the same “day”, since some trains will leave the station shortly after midnight.

My hack for this was to sort the trains by a new field, :sort-field which shows the minutes after 3am,

(defn minutes-after-three [h m] (let [minutes (+ m (* h 60))] (- (+ (if (< minutes 180) 1440 0) minutes) 180))) (defn get-time-after-three [] (let [time (new js/Date)] (minutes-after-three (.getHours time) (.getMinutes time))))

3.3.2 Finding last trains

Something of (very practical) importance to me is making sure I don’t miss the last direct train to somewhere (excluding changes).

Specifically, given a station name (station), the list of stations (stations), the train data (trains), and the direction ((:dir state)), give me the departure time of the last train bound for every station after the current one in the direction.

My purely functional solution to this was:

(defn find-last-trains [station stations trains state] (let [stations-in-dir ((if (= (:dir state) :up) take-while drop-while) (partial not= station) stations) ; take or drop until trains-at-sta (->> trains (index state) (filter #(in? station (map :station (:stop-list %)))) (sort-by #(time-after-three ; sort by the shifted time (:time (find-in-stop-list station %))) >))] (map (fn [sta] {:dest sta :depart (->> (filter #(some? (find-in-stop-list sta %)) trains-at-sta) first (find-in-stop-list station) :time)}) stations-in-dir)))

In some cute arrow-heavy, lambdas-everywhere style.