This is a simple pattern that points to Forth's expressiveness and philosophy.

Most words do one thing, and do it well. Sometimes you want to do that thing several times.

In Forth, this is very easy and natural to express. For example, the word key will block until a key is available, and return it. Sometimes, at the REPL in particular, you want to do this multiple times. So we define this word: : keys 0 do key loop ;, and say 3 keys. This simply calls key 3 times.

We may approach this level of terseness in other languages, of course. In Lisp we may have (key) and (key 3), Lua would offer key() and key(3). Forth words can't natively know how much of the stack belongs to them, so variadic functions are harder to write. It is impossible to beat the terseness and clarity of key 3 keys in use, and the second definition is transparent.

Forth itself uses this custom. cell gives the number of bytes in a cell, while 1 cells multiplies 1 by that number, giving the same value. In general, a plural word must be proceeded by the number of repetitions.

The syntax of Forth is brutally simple, allowing for a rich semantics. The most important decision: whitespace is (almost) always important. This was made when Fortran was popular; in original Fortran, whitespace is literally never important, meaning foo bar baz and foobarbaz are always the same program.

Consequently, any printable character within a word is fair game. Schemers and friends are accustomed to boolean functions that look like test?, which I'm sure was heady fun after decades of test-p. Forthwriters do this as well.

High-level Forth ends up looking like this:

1
2
3
4
5
6
7

: clickloop
  begin 
      event
      event-respond
      1 .left?
  until
  ;

Anyone can read this. 1 .left? is not entirely obvious. The . suggests it's printing something, and the ? suggests that it's testing something. If it just said left? most Forth programmers would conclude that it's setting the test that until checks. This is actually handled by event-respond, .left? is a debug function that prints the stack if event-respond leaves values behind.

All of this is customary, and should probably remain so. Parsing within a Forth word breaks some important contracts, notably the dictionary. I've been mulling over a modular Forth dialect that parses within a word for exactly one reason, access to words defined in another module that are overloaded. So if you already have an event word, you can say event.:book and get the event word from some other book in your search chain. Since the effect of a vocabulary word is to leave its token on the stack, event .: book could just be the word .: checking the event wordlist for the word book. The compressed form is possibly Forthright, in that the effect is to either interpret or compile a single word. If we used event .: book we would expect 3 words to be compiled, though it is quite possible to have 3 compile-time words (or more) produce a single compiled token, such as : example [ 2 3 + ] literal ; compiles one word, the literal value 5.

Adding parsing to Forth words is pure Sith and should be done with great care if at all. Retro supports strings with the simple form "string", which is superficially cleaner than the Forth ." string", where the spacemark is important. I actually prefer the latter, which enforces a Forth convention that spaces always trail. Anyone who has dealt with two different assumptions about where spaces belong can feel me on this one, I hope. Similarly, cr/nl is always prefix. These are good conventions. If you need leading whitespace in a string frequently, first: consider that you might actually need trailing whitespace and second, write a word for it. It's an interesting challenge, try it!

There's a more subtle problem. The Forth interpreter does this: word? number? if succeed else fail then. Retro has to try string? word? number?; I don't like anything that breaks the power to redefine a word. Even the absurdity : 1 2 ; is a consequent with value; consider a special vocabulary containing : 10 [char] A ;. Or : 10 10 select.window, which keeps the number 10 around for just long enough to make the definition.

This imaginary modular forth would try word? module-word? number?, meaning a redefinition of word.:book would block word in the book module. The compiler should complain about any word that contains .:, and be loud about if that word actually blocks a module word. This is better than the user complaining because they need the ability to make a word containing .: for whatever reason, and the compiler won't let them.

There is a small but significant group tucked between Generation X and the Millenials. As a member of that generation, it seemed worth pointing us out.

The 20th century may be sharply divided into eras by the chief inventions: the internal combustion engine, nuclear weapons, and the computer network. I could say "computer", just as one could make a case that the middle technology was the television. But it wasn't computer which changed us, it was computers, wired together.

The Vanguard generation, in 2014, are clustered between the ages of 30 and 40. In 1994, we were 10 to 20 years old. Simply put, we remember life without the Internet, but learned it during puberty.

There are plenty of people ten or twenty years older who found the Internet in their all-important adolescence. The university experience in America delays adolescence, with the (usually unrecognized) upside of longer plasticity for learning among the inclined, so there are a few Avant Vanguardists. Reduplication and elision, yay! They do not a generation make.

More importantly, they suffered through the September than Never Ended. Their Internet was not our Internet: we are the Vandals and Visigoths who wrecked their kingdom. Sorry about that. Did I mention we were going through puberty?

The Millenials are fish in water. They have never heard an adult ask another "Do you have email?", only "What's your email address?". And so on, and so forth, for all of it.

In the present, the Baby Boomers rule, and Generation X runs it. Soon, Generation X will rule, and the Vanguard will run it; this is the way of the world.

By that time, our very biological existence will rest on the network. It already does, but we have only begun to reap the harvest of internal combustion and all that it brought. In a very real way, the Internet or its heir will rule us, and that's the optimistic scenario.

To the Vanguard I say this: Keep your memories of the nascent Network. Remember the floppy, and what fits on it. Cherish the freedom of your youth, when, lacking a cell phone, you had to find your friends a terra and your parents had to yell or call around.

More: Understand that our teetering, broken, stringy technology stack was written by Boomers, mostly, and other people who didn't know any better. Charitably, the technology wasn't mature, though the luminaries and prophets of the era would vigorously disagree. We're inheriting it, along with the pride of a lifetime's work; rather than take it behind the barn and shoot it, let's gently put it out to pasture and emulation.

Axiom: there are no clean separations between programming languages.

Corrolary: if one can be found, someone will design a language specifically to blur the resulting boundary.

The terms high and low level languages are frequently bandied about. If your word of code stretches from Python to C, these are comfortable categories, with room for interesting bikeshed debates on the margins.

At one time, this almost made sense. Languages were either properly interpreted from strings, or compiled into genuine code. Even in those hoary days there were vigorous communities, notably Lisp, Smalltalk and Forth, where these distinctions were blurred or simply did not apply.

This is no longer even vaguely how things work. On the one hand, most interpreted languages are compiled to a bytecode, and many are further optimized from there. LuaJIT in particular can demonstrate superior performance over compiled languages in certain circumstances, in particular cases where the input to a functional cascade can rapidly change character. JIT compilers can keep up with this, while a static compiler by definition cannot optimize for data the program it's compiling hasn't seen yet.

One the other side, which I feel is less known, we have OpenCL. This is a decidedly low-level dialect of C, in which pointers are not allowed and 'functions' are syntax to allow inlining of code. This is compiled and optimized into multiple dialects (CPU and any available GPUs), on the fly, from strings. A running OpenCL program makes some of the fastest machine code possible from strings in the runtime environment.

Clearly this distinction has completely collapsed. The more interesting division is between literal and figurative languages.

What This is Not

The distinction is not between static and dynamic, nor is it about performance or degree of interactivity. I can happily find orthogonal examples of all of these in both the literal and figurative family.

The distinction is this: a literal language abstracts the hardware, while a figurative language abstracts a problem domain.

Assembly is of course literal, along with Forth and C. Haskell is highly figurative, while the Lisps tend to blur the line in interesting ways: within only Scheme, Racket has a layer that is more hardware oriented, while Chicken delegates this aspect to C. SBCL can dig down as deep as you want, but Lisps in general are figurative in emphasis: invisible garbage collection is a figurative pattern and a pillar of Lisp programming. Despite this, the garbage collector itself has occasionally been written in Lisp. I confess I have trouble understand how this is done; low level Lisp is almost a different language.

Hoon is an interesting case. We think of Nock as a sort of Hermetic seal; above the Nock layer, Hoon is straighforwardly literal. This is not uncommon; Retro, a Forth dialect, targets a very simple virtual machine that is quite literal on the target architectures. Lua is more figurative, but it's a brisk afternoon's work to discover how the virtual machine works and how to target it from C, Lua's home environment.

This intersects with compactness in some ways but not others. Assembly, our most literal, is as complex as the architecture, which has gotten truly gnarly on our flagship systems.

I'm doubling down on Forth because it combines a low mental surface area, a literal machine model, interactivity, and introspection. The 'word' is a strangely powerful abstraction. If you want (and I do) a colorful, interactive environment for exploratory systems programming on existing chips, Forth is the natural place to start. If only it had a nice, modern type system…

Garbage collection is another solution to a problem we don't have.

Here's my system: 4 cores, 8 GiB ram, 256 GiB flash store, 2 TiB off-deck backup. 64 GiB on the phone, 128 GiB on the tablet; I'm a packrat by nature.

I have garbage collection why, again? Because no one has written software around this unbelievably roomy environment.

"But… what if I have an arbitrary amount of data to process?"

What, like in a file?

Sure…

Couldn't you allocate some memory for the file and copy it over?

That's slow with a huge file.

No, reading it is slow with a huge file. Allocating it is free.

You could also, maybe, make the files a little smaller? The user never has to know, though it's polite to tell her if she asks.

What if I run out?

What, of the huge chunks of imaginary allocated memory? Close a "file" by forgetting it ever happened.

What if I changed something in the file's contents that I wanted to keep?

And you didn't make a copy to the store immediately? Are you five?

I didn't know if it was a good change or not

…you're five. I said "copy", not "destructive update".

Okay hotshot, what if it's arbitrary data from a socket?

Me, I'd be writing it to the store, hoss. Then, it's a file. Allocate more than you can store or process and start using it.

Take notes. Store them. Perhaps even ask the other computer first how much data it's sending. This is legitimately not always an answerable question, and sensors provide numbers too.

What about huge transients from my enormous calculations?

…you were dynamically allocating huge transients in enormous calculations?

I can hear that server farm humming from here. You. Had. One. Job.

But I just want an array, and I don't know how much to put in it, and I won't need it for long, and…

No, you are wrong about all of that. You want a proper functional shared memory data structure, to which two things are done:

They are created and

They are durably written to main store.

Unless they're, uh, huge transients. Y'know, like the fake Intel computer asm.js makes for you. Dynamically, wink wink, say no more. "wait, that's.. that's an array of…". That's no moon, friend.

Look, I have really complex, relational data relationships, that I have to query and then get results of variable size back.

That sounds like you want some kind of "base" for "data". Perhaps someone has written one, and it can handle your requirements.

Which sound kind of special. Perhaps you want a dedicated chunk of hardware? It could 'serve' you this data. Now, you have arbitrary data from a socket to allocate.

There are two kinds of bases for data: those in which there is a type of request which always, without exception, returns the same data, every time, and those which are enormously useless and must be scrapped in favor of something you can use.

This is also true if you call your "data base" a "web site", as it happens.

If you have the former, you: request data be delivered in fixed chunks, process it, remember the request, save the result, and anything else may be immediately and comfortably forgotten.

…lisp

I can't help you there, at the moment.

You are significantly underestimating the complexity of a modern operating system.

I have enough room in RAM to load a DVD and watch the entire thing.

You cannot eliminate garbage without abandoning Unix.

Happily, Unix now comes on Flash USB drives, with costs similar to a cheap night out wherever the hacker is found in native habitat.

Fusing this to the motherboard is not a hard problem.

Your concurrency problems are a direct consequence of your software architecture.

Concurrency is a consequence of carrying a solution forward without remembering what it's trying to solve.

Preemptive multitasking is a weird thing to do: you interrupt the computer, with its permission, do something else, then it goes back to what it's doing. There's all sorts of complex negotiations; the good news is, it works, usually.

This is a solution to a hard problem: how to do a bunch of things with only one CPU.

So I'm guessing you have 4 CPUs, which makes the architecture fundamentally asisine.

All your concurrency problems stem from this. Do hypervisors have concurrency problems? Wouldn't know; bet they don't, because they sensibly allocate processors and memory.

If you took a systems architect from a more sensible time, the 1970s say, and gave them a decent AMD chip, they would first go for a walk with a beer and a tab of acid to consider the implications. They would proceed to write something entirely different from what you have.

It would be entirely based on the idea that they have 4, let's say, supercomputers of unfathomable power, hooked up to more core memory than they can understand, attached to store counted in units they have never seen before, except in jest.

Would it emulate Unix? Would it create a master control program that emulates multiple Unices?

No. This is not what it would do, at all.

It used to be standard practice to rearchitect systems in light of new capabilities. This stopped because of two autistic people with different fixations: Bill Gates and Richard Stallman. My esteem for both men is considerable, in light of the former's philanthropy and the latter's stubborn and correct fixation on the privacy and freedom implications of open source systems.

They share a fatal obsession: backward compatibility. Stallman, in particular, has an almost detestable belief that software should be written once and for all. Imagine an artist with the same attitude to painting, or the novel; we would deride them as the martinet that RMS often presents.

There are other, better options: keep the old beasts running, rewrite for new systems, or wait around for awhile until speed gains mean you can emulate. Unix has been under emulation for quite some time.

We have two tricks: speed gains, and writing new software. And we appear to be fresh out of bubblegum.

I read a description of Forth years ago, that stuck with me.

Now that I'm writing some, let's try this problem out. The premise: given three arrays of three integers, add x and y, putting the result in z. In C, this is easy by design.

Oh, and array overflows have probably cost us a trillion dollars by now. Kudos!

Forth is factoring

First, no custom data types, but lets actually define and fill two 'arrays'.

1
2
3
4
5
6
7

variable foo[] 2 cells allot
variable bar[] 2 cells allot
variable baz[] 2 cells allot

99 99 99 foo[] ! foo[] cell + ! foo[] 2 cells + !
2 4 8    bar[] ! bar[] cell + ! bar[] 2 cells + !
16 32 64 baz[] ! baz[] cell + ! baz[] 2 cells + !

This is wordy. Wordier than it needs to be. But also easy to read. Notably, we do not juggle the stack; it's somewhat bad form while interpreting.

Clearly if we need a lot of arrays, we'll define something to handle them. Which kind of array do you need?

Today, like the problem, we need a C array: stupid, unbounded, and untyped.

Now we need a word to add the values of three addresses together.

1
2
3
4
5
6
7
8

: add-advance  \ ( a1 a2 a3 -- a1+ a2+ a3+ )
  \ "add a2 and a3, store value in a1. Advance all three addresses."
  dup>r -rot dup>r -rot dup>r -rot \ ( a1 a2 a3 -|- a1 a2 a3 )
  @ swap @ +                          \ ( a1 a2+a3 -|- a1 a2 a3 )
  swap !                              \ ( nil -|- a1 a2 a3      )
  r> r> r>                            \ ( a1 a2 a3    --        )
  cell + rot cell + rot cell + rot    \ ( a1+ a2+ a3+ --        )
  ;

Line by line, shall we?

dup >r -rot dup >r -rot dup >r -rot puts a copy of every address on the return stack for later. I'd call this punt if I planned to do it frequently.

@ swap @ + takes the values from the two address and adds them together, while swap ! saves the result to a1.

r> r> r> is the anti-punt, basically. We could do the cell + call inline with it, but I feel this way is clearer: the cell + rot cell + rot cell + rot adds one cell width to each address, advancing it.

So far so good. Let's make a word that creates a word that adds an array of n.

Wait what?

1
2
3
4
5
6

: n-adder \ ( create: n -> nil does>: 'add-advance' )
  create ,
  does>
  @ 0 do add-advance loop
  drop 2drop ;
3 n-adder 3-adder

Looks like we solved the problem for, not array[3], but array[n], for any n. Of course, we have to make a new word each time. here, allot and :noname can be combined to the same thing in anonymous ways. This is strictly speaking over general; we add an entire extra line of code, and one additional word, in the pursuit of perfect generality.

:noname create , does> @ ( ... ) ; 3 swap execute 3-adder would make our adder without providing further capability.

Just as clearly, we could make a word that generates an array of n, and stashes n as the first value of that array. Then we could make a bounds checked version. It could even add two arrays of unequal length into an array equal or larger than the both, though we would need to define each behavior fairly carefully.

Or we could make a creator word that takes any other word that acts on the values of three addresses, and applies that word to an array of arbitrary length. There's plenty of room in a 64 bit offset for a type system for the resulting array; we could end up with a + (pre|post|in) fixing (two|three) arrays of arbitrary type and length, if that was useful. Depends on the language you want.

The reason Forth doesn't provide the C array structure is because it doesn't presume to make that choice for you. Simple as that.

Footnote

Astute Forth heads might find this one weird: \ ( a1 a2 a3 -|- a1 a2 a3 ). That -|- is an annotation I call the mirror: the return stack runs backwards from the data stack: a b c -- '>r >r' -> a -|- b c.

It's easy to read, and clear. I use -- to show that a stack annotation is completed and -> to show a stack change, normally in a colon definition. It's so regular a computer could almost do it.

Forth is what you make out of it. If you want a structure or language artifact, and you understand what it does, you may create it.

Let's write a simple closure in Forth.

Simple Closure

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

\ Rollhex

: offset-hexpr \ ( offset n -- new-offset )
   tuck                            \ ( n offset n -- )
   hex 0 do                        \ ( n -- `hex`    )
      dup i + 16 mod               \ ( n n+i%16 --   )
      dup 15 <> if                 \ ( n n2     --   )
          0 <# # #> type           \ ( n -- "n2"     )
      else \ red F
          0 <# .#! # .#r #> type     \ ( n -- "n2"     )
      then
   loop decimal                    \ ( n -- `decimal` ) 
   + 16 mod                        \ ( new-offset --  )
   ;

: hexer  \ ( C: nil -> nil D: nil -> nil "hex" )
      create \ ( nil -> nil )
          0 ,
      does>  \ ( nil -> nil )
      dup >r @ swap offset-hexpr r> ! ;

hexer rollhex 

A call such as 2 rollhex will produce 01 as output. 2 rollhex again produces 23 and so on. We highlight each F in red.

This is a small utility, kind of like a measuring tape for a terminal. You can repeatedly fire it at a rectangle of text, and get a quick count for how many characters you've printed. There's no need for a reset word, you can call 0 ' rollhex ! or whatever new value you want the closure to have. Literally, this puts 0 on the stack, puts the address of rollhex on the stack, then stores 0 to the address, just like we did when we created it.

Could you embed this in some kind of "object"? Certainly, and you do so in the same fashion: by rolling the behavior you want, directly. If it gets moderately complex, you use access words. And so on.

This is dramatically over-commented Forth. Almost every comment is wrapped in an outer comment. The result looks weirdly like some kind of annotative type system operating in parallel with the Forth. In fact, it is, this is how Forth programmers keep track of programs. In their head, with a yellow legal notepad nearby.

You'll note the famous 'stack juggling'. Stack juggling is to Forth as the parenthesis is to Lisp. The issue with Forth is not on the left side of the source code; the issue is that the right side exists, at present, only in our minds.

I had a very interesting conversation with Loper recently about, well, superficially it was about Urbit.

It came at an interesting time, as I was rewriting Ax in Forth. That's a done deal, but I've temporarily lost interest. As an aside, the Ax inner loop calls for roughly ten basic Forth words, the kind that are normally one to five machine instructions long. That is…not insolubly slow, even on hardware that's somewhat hostile to the level of indirection involved in a noun.

Forth itself has seized and held my attention.

I remain convinced that Urbit, or something like it, must be built and must be built now. I hope for efficiency's sake that Tlon will be the ones to do it.

Similarly, the things Loper is up to right now are totally right on, and frankly, the sooner the world gets Loper designed hardware the better off we'll be.

And then I started learning the Way of Forth. People have the wrong idea about Forth, because they think it's a language, when it's actually a programmable program. Now, every language runs on a programmable program, but Forth is a programmable program, and therein lies the difference.

Tame the Beast

Forth is in danger of becoming a lost art. Few of the younger generation are learning the way. It survives in a surprising number of contexts, far away from the mainstream of open-source, Unix and network-oriented software. Most Forth users are using commercial Forth environments that they paid for.

This is easy to explain: Forth, the language, has almost nothing to offer. Viewed as a language, it's more than just primitive, it's barbaric.

Then you go to write something like an assembler, and realize that Forth is dramatically the best language for that job. Seriously, check this out. This is the best assembler you've ever seen. Go find a C assembler if you don't believe me.

The version of this that runs on the microcontroller, in a native Forth environment, lets you interactively design assembler words and test them, on the microcontroller itself. Y'know, no big deal, I bet you do that all the time.

It's been said there are more Forth implementations than serious systems written in Forth. This is actually untrue to the verge of slander, but Forth is meant to be hand-rolled to the task. It is a problem oriented language. By the way, if you haven't read POL, I insist. It is a fundamental work of computer science; Chuck was a student of McCarthy at the dawn of Lisp.

The way of Forth is to take the target architecture and tame it into a consistent environment for getting things done. The stack abstraction may always be applied, and normally at a cost similar to a subroutine-call environment.

I have my notions of what's driven Forth into its present, moribund state. It's mostly ANS, which abstracts the machine away in the name of portability. This renders Forth a weird concatenative abstract language, taking away its numerous advantages. The C standard is far more literal and close to the hardware than the Forth standard, and this causes the latter `language to suffer immensely.

It remains the best machine tool in the arsenal. If the job is taming a wild chip, Forth is your friend. It's the best way to punch down, period, and with some support, I can picture it punching up with the best of them.

How to write a post-C environment using existing high-calibre ARMs and possibly Intels? Forth alone will not suffice. But if we stick with AVR for a bit, then gun for modest 32 bit ARM systems, we may discover that the big iron is obsolete. It's also quite possibly dangerous: there may be Balrogs built into the big chips, and certainly, the board architecture we're using is not nearly paranoid enough.

Interactive Enlightenment

I don't dislike writing C. I learned it when my brain was fairly plastic, so despite the rust, I can get 'er done. I have always despised debugging it. GCC is pain, mostly concealed with make, though make is also pain when you have to write it. GDB is pain multiplied by pain. I cut my teeth on Turbo Pascal, and this shit hurts. The only saving grace of Java is that stepping though that garbage is of necessity well-tooled. C programmers are masochists, to the last one; they not only invented Hungarian, they frequently use it.

C engages in premature optimization, which is known to be the root of all evil. It won anyway, because Forth provides no way to hand off any programmer discipline to the compiler, and efforts to add them have failed spectacularly. I have a notion, but code speaks and I have a lot to learn.

There's also the fact that C is Algolic. Algol was designed so the pseudo-code programmers developed ad-hoc could be used to write real programs. It's no wonder that a language driven (that's human language, specifically Western European) design became effective and popular. It's taken decades to learn that the pain of compiling Algol puts an enormous burden on systems programming; indeed, this lesson has yet to catch on.

Nonetheless, let me state a law: If a system does not provide an Algol, it will fail. Dialect matters less than you think it does, but there are no downsides in using something sensible with a decent user base. Lua, say. Lua is a nice Algol.

I plan to write a nice, Unix hosted system that provides a roomy, enhanced window into the running environment of a second computer. This treats Unix like the fancy terminal it is, giving us the all-important ability to code and use the Internet at the same time.

The only Forth that is even skeletally complete, from a Unix perspective, is gforth. That's a GNU tool, making the code pure contagious poison that can never be used for anything. Unfortunately pforth, the public domain version, is missing certain essential ANSI term handling words, so I'm using gforth for now.

The program, provisionally called Forge, may be thought of as a cockpit for piloting a Forth system. Self-introspective, yes, but built primarily to operate an umbilical Forth environment on a remote platform. We may use this comfortable tool to make compact, excellent microcontrollers which can talk to each other and host systems; the wisdom gained may be applied to writing a better host Forth, and then we're in business.

Forge is intended for interactive systems programming. Building out an operating system requires a lot of scaffolding, and there are self-reference problems with trying to host that scaffolding on the system you're assembling.

I mean enlightenment in the sense of shining a light. Compiled C code is dank, obscure, and our current architectures further convolute the actual execution of code. There have been benefits, but the tradeoff isn't worth it: what you cannot see, you cannot understand without great strain. Show me a tool for interactive physical inspection of the hardware state. Show me how to write it in C. If you know the language, you know the problem: all the information you need is simply gone, and there's no good way to get it back.

Astute coders can do terrifying things with stack machine optimization. LUAJit and HotSpot are works of pure magic. The fact that this has not been effectively combined with Forth in an open-source tool is part accident, and part missing type information.

Oddly enough, Forth programmers provide type information, in the form of comments. All good code has them. Nothing in the computer reads these annotations or does anything useful with them. We'll get back to this thought, but not today.

The Way

Forth is a Daoist, immediate, personal approach to the computer. I have some problems, that are conceptually large. I intend to write and rewrite those problems until I have some good languages for solving them.

This post was going to be a survey of the Forth ecosystem; perhaps later. Instead, I'll end with meditation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

If the System provides garbage collection,
the Language cannot write one.
If the System provides syntax,
the Language must use it.
If the System provides types, 
the Language shall have them.
Doing less,
Presuming nothing,
Staying empty,
The Way of Forth
Provides the Language,
Enables the User,
To Build the System
In Accordance with the Way.

Back to the metal. More later.

Ax has been written and implemented, twice. A bit of orientation is in order. Today's post will be as computerific as possible; we'll dig down into the deeper 'why' in later parts. This may take awhile.

Preface

There are 720 valid Nock machines, defined as any cellar automaton which provides a mapping from the Nock rewrite rules to any one of every digit of set {0..5}. Since one of them is actually called Nock, let's call the permutation group Conk.

2 of them are Ax machines, or would be, minus the fz operation. That is, there are 2 Conks which have the same mappings for 0, 1, 2, and 3. There are actually 6 Ax machines, 2 of which have the same axioms as Nock machines; we may consider those two as Nock equivalent, by permuting fz above the Nock (proper) threshold of 10. By the same simple mathematics, there are 5040 possible Conk-compliant machines computationally equivalent to Ax, Nock not among them. We will call this permutation group Xa, for what other choice do we have?

6 is a small enough number to consider in entirety. The only bit that is arbirary is the range on sigma, which is only arbirary if you're not an extant silicon computer.

I am convinced Ax α is the best possible choice among the 6. I hope to demonstrate that here.

There are an infinite number of cellular automata. Conk machines are a class. Ax machines are an interesting subclass, disjoint by one member.

Rationale.

Ax is even weirder than Nock, because it never terminates. In order to calculate Ax, you must use a certain amount of judgement.

Ax is recursively self defined. Absolutely every valid rewrite may be performed, in a metacircular fashion, and the expansions are defined in terms of each other as well as the reduction. As in Go, you must apply the Ko rule to infinite loops: if you may continue to play, you should continue to play.

A trivial loop, such as Ξ [a] -> Ξ [a], is like a game where the only remaining moves are Ko. The polite thing is to pass and count the board. In Ax, as in Nock, this is our only form of error.

Note that detecting all cases of Ko without arbitrary calculation is of course the halting problem. In practice, an Ax interpreter may be made into a Nock interpreter by dispatching on a subset of the operators.

Since you need the idioms, particularly if, to define many of the fundamental operators, the expansions are the second set of statements, after the definition. The definition says what the operators do, the expansions say how they could do it, given the available operators, and the ability to decide when you're tired and quit.

It would be nice, and is probably inevitable, if all the non-trivial operations are visited within an expansion of 0. That way, you can type [0 0 0] into the reference Ax machine and it will not only run forever but repeatedly visit all definitions in doing so.

Since a decent expansion of 0 would could be Ξ [a 0 b] -> Ξ [[b a] 2 2] we should be good to go.

Oh the reference implementation? Scheme, of course. It is incapable of returning, though you're welcome to make a few changes if you want to see it do something. It does this in the best continuation passing style! Not that it exists. Instead, we have a demo machine in Lua.

The Preamble

Compared to the Nock spec, this is enormous and pedantic. Compared to a representative work in the genre, it is haiku.

Xa space being a permutative superset of Conk space, the Ax preamble contains the Nock preamble, in its entirety, without modification.

Which is a marvel, by the way. Every time I've sniffed out an ambiguity or source of confusion, I have been wrong. Even lines six and seven may not be made to malfunction, despite strict operator overloading. Try it.

It is overloading, however. [1 + a] would be [[1] +*[a]], and *a crashes. But it doesn't say [1 + a], does it.

Arguably, our symbol definitions, despite being English, are equivalent to the pseudocode. And indeed our generalized rules, being only one reduction longer, are shorter, line for line, than the pseudocode plus the reductions. Not counting brackets, Nock uses seven symbols, overloads one, and has four variables. We use six symbols, no overloading, five variables, and need an extra symbol to define our extra operator.

We also have the expansions, or will. Which I feel add a certain esprit to the spec.

It is a little confusing, if conventional, that 'a' is either an atom or a cell, while 'n' is definitely an atom and definitely not the other sort of noun. I trust the reader to follow along, and using "a" for atom and "b c d e", while cool, would add impedence to both understanding the Ax spec per se and comparing it to the Nock spec.

It's not so bad, really. We also need an additional clarification that partial expansions of a line are not allowed.

Mythos.

So here's how you read Ax. You've been captivated by alien intelligence, who have sat you down in front of something suspiciously resembling a vintage VT100 terminal. You receive the preamble, and the term. The cursor blinks.

There doesn't appear to be any way out of this…pod, Tardis, what have you. Though it is made up in a somewhat surreal and misguided parody of human habitation, you're not sure that the fruit is edible, and don't want to find out.

Well, what would you type? You're not a chump, if they wanted to use your letters they'd speak your language. Plus, they just told you "numbers". Practically shouted it.

So you type zero, and get an infinite loop. Let's try a cell of zeros: nope, crashes. Well, a cell can be composed into two cells and ok, we have a zero back.

Which one did it return? You could just bang away at the keys but this is a delicate operation. [0 1 0] is an infinite loop, [1 0 0] produces 0, [0 0 1] produces..one. Can we do anything else coherent with ones and zeroes?

It appears we can: [0 1 0 0] produces 1 also, but for what reason? A bit of experimentation discovers that [0 1 0 1] produces the all important 2. We can in principle make any number now, which is nice, but still feels a bit like cheating, because we're using the command line, not the computer. [0 1 0 1 0 1] is a syntax error, but [1 0 1 0] produces… [1 0]. Interesting, but reductive, though serving to confirm what we might already suspect.

On a hunch, you type [2 1 2 1]. The screen fills with endless columns of cellular numeric data, prettily colored, and looping in a way that looks alarmingly like a refutation of the problem with halting. It then halts, executes an impressive screen fade quite impossible on a real VT100, and then helpfully types [2 1 2 1] for you once more. This time, it simply returns 3.

Everything else you need to know is in the guts of the core dump. Congratulations, space cadet. <3 <3 !

You may notice that in Ax, trees are built up from numbers. The feeling is one of open-ended discovery. The computation branches out fractally, resolves itself in some mysterious way at the limit, and returns a form. A fundamental axiom guarantees non-deterministic behavior. In Nock, numbers are derived from nouns, choices are absolute, and your result is either error or truth.

Axioms

Zero

0 is the identity operator, is. There are two operations that are closed over the monoid of the natural numbers, addition and multiplication. Of them, multiplication is consequent. In the operation of addition, 0 returns the identity. If you have any number, you may now return it. You now have 0, and may return it: Ξ [0 0 0] -> 0

One

1 is the increment operator, up, as premised in the preamble. If you have a number, and you now have 0, you may increment it. Ξ [0 0 1 0] -> 1

Two

2 is the branch operator, br. I like slot, but we normally call those branches, and I count six English pronunciations for slt, one of them distasteful. You need 0, 1, 2 and 3 to define br, but you can use 1 and 2 to produce 3. mod being explicitly provided, we may write an expansion that actually runs quite servicably. The Nock spec provides no way to factor a into a + a, nor to determine which of lines 15 or 16 to apply. This does not detract from its correctnesss, these are very ordinary operations on the natural numbers.

Branch selects a branch, it does not cause one. Branch/forking happens in the same two ways in Ax as in Nock, because Ax is a Nock machine. Note that br on branch 0 (to apply br is to 'graft' the subject and return a branch from it) is not a syntax error, but rather, undefined. It may be a syntax error if you wish, or produce a noun, which is a common anticipated use. It may even produce an effect, but that would be crazy. The spec doesn't say, though; grafting on zero means your code is not deterministic. Crashing is the best idea if you don't know what else to do.

Note that so far, we have halting rules and trivial crashes only. Next comes the distribution rule, which gets us most of the way there,
and then the explicit recursion rule, ax. These cause branches, and composing that ability with br raises the bar.

How many operators does it take to write an infinite loop? I am curious, without the time to explore the question. I predict the set will be low ordered with respect to Ax. Consequently it will be composed of the operators 0..4, inclusive, likely without 4. We'll see.

Three

3 is the ax operator, ax, which Nocks a noun apart and evals it. The symbol nck is also reserved for this operator, en homage. Why 3? Well, that's a digression, while I'll gloss over.

Consider that there are two ways to index: by zero, and by one. What if you want to contain the damage? Well, you could count (0|1) (1|2) (2|3) 4 5 6, or (0|1) (0|1) (2|3) (3|4) 4, 5, 6. The latter is taxing on our already strained resources.

In certain circles this difficulty is referred to as the "Abyss". My solution to this difficulty is suggested by ancient texts but the interpretation is to my knowledge completely original. We are all fairly certain the mystery involves both 3 and 11. Let's leave it at that for now.

ax could be 2, but br could not be 3, or we would have [2 1 3 1], and where did that three come from? Also, branch on address 3 is not defined in the reductions.

Note that both the distributive property and ax may safely be made parallel once the cell to calculate both branches is composed. Indeterminate, mutually dependant behavior may be arranged only through abuse of the 0 branch and the put operator, in the usual fashion.

Four

4 is the eq operator, which returns 0 if the evaluated cons(subject,object) is equal and 1 if it is not, crashing on an atom.

Why is four eq? Well, it's our first non-prime operator, if you accept the case that 0 and 1 are neither prime nor non prime. They have a special relationship with multiplication that makes that case plausible; certainly I was taught the primality of one, though the field would appear to have changed its mind.

It could be cel, but as we'll see, cel should not be 4. eq's expansion may be thought of as having an 'even' test or a 'double even' test, that is, it ifs on cel (though of course, the reduction does not). If 'odd', test atomic equality, if 'even', apply cel.
Note that all even-numbered operators and idioms have exactly two expansions, that is, they are higher order branching. Even put is capable of branching, depending on implementation.

Five

5 is the fz operator, one of the raisons d'etre for Ax versus Nock.

Note that in Plan Ax from Conk Space, 5 is cel.

Fully qualified Ax machines are not deterministic, because they aren't colorblind. In addition to Black and White bits, Ax machines must supply Red bits, which are completely different because they come from an actual, high quality source of entropy. Note that, while weird, this is just as well formed as saying "an actual, high quality source of numeric computations", in that you can just hop on the Internet and buy one. Ax machines must have both.

This is one reason fz is an operator, rather than a lemma or higher-level function. The Second Law of Thermodynamics is taken as axiomatic for our purposes; within the specification, the method of selecting on the range is deliberately left unspecified.

There are some Deep Implications here, which, as usual, I intend to touch on later.

The demonstration Ax machine uses Pink bits, which are only pseudorandom. This is strictly not compliant, for the purposes of a reference implementation, but I'm in a hurry. Feel free to use Pink bits if you're just futzing around, but consider making or buying a gig or two of entropy, or picking up an entropy machine. Cheap stuff, entropy, these days. Remember to throw it away as soon as you're done with it: the first rule of handling entropy is: any operation whatsoever upon it render it and any copies no longer entropic.

Because we have a term, not just a preamble and definition, there are an infinite number of possible Ax machines, even Ax α machines. Ours is Ax α 256, aka Ax Byte. The beautiful, Schemish Ax is Ax Bit, of course, but it may easily be emulated with modulo 128, whereas simulating the reverse transition is particularly tedious.

A ternary machine would draw on nine trits of entropy, or whatever. Ax is meant to be used.

Note that randomness is not mentioned in the spec. It is actually perfectly valid for fz to return any number in the range: you may rewind Ax and play it again, or feed it nothing but 128 in all cases, or 0, or anything else you'd like. The treatment of 5 and br 0 distinguishes the actual semantics of an Ax machine, as they introduce indeterminacy into the result.

Why 5? Zod, where else would you put it? There are actual reasons, related to the geometry implied by the new kabbalah. This is beyond our current scope.

Note that we may generate 6 with Ξ [4 5 [2 3]]. Eventually, or immediately, depending on the Ax machine. This is considered beneath demonstration.

Six

6 is the cell operator, cel. It returns 0 for a cell and 1 for an atom, as you might expect. Symbols are inspected in this case to see what kind of noun they represent.

Why six? Well, there are two important kinds of cells in Ax, [a b] and [a b c], as defined in the preamble. There are two groupings of six, [[1 1] [1 1] [1 1]] and [[1 1 1] [1 1 1]]. If you immediately recognize those are syntax errors, you're doing great. Happily, Ax is blessed with exactly three relevant data structures, as shown by the first three lines of the reduction: atoms and two cells, which may not be Axed, and 3 cells, which are n cells, and which may be Axed in some cases.

Those are the axioms. I am totally convinced of 0, 1, 2, and 3, which define Ax space within Conk space. I am pretty sold on 4, 5, and 6, and on the benefits of Xa space and the a br 0 reduction of 2.

The reasoning behind the order of the idioms is basically Kabbalistic. Surprise surprise, that is the reasoning behind Ax space too. It just happens to look like a Page from the Book, as Erdős Pál would say.

Idioms

Nock calls these macros, and defines them as such, though we are advised by the Crash Course that there's no reason to implement them this way. In Ax we call them idioms, indicating that they may be expanded as macros containing only the axioms. For the reduction, sometimes we do, sometimes we don't, whatever's cleanest. Since we have expansion rewrites as well as reductions, anything may be written as a macro, as long as you're not in a hurry to halt.

7 is cmp, which composes functions, and which, through happenstance, ended up in the same place as in Nock. We're in Netzach, if you're paying attention.

8 is if, 9 is cnk, 10 is put and 11 is arm. if, cnk and arm work like 6, 8 and 9 from Nock, because they're identical except for the necessary permutations. put has the same syntax as the other 10, and identical semantics: the results are undefined, other than that evaluation must happen if a cell is provided. We don't plan on using this for hinting, but if we're running Hoon 191, what other option do we have?

put fits nicely in 10, because it's 0 all over again, having the same meaning for the purposes of the Arc of code under evaluation. The interaction between put and br 0 is envisioned as a crucial mechanism in the larger Arcitecture.

I could make a firm case for dec as consonant to inc and arm as consonant to br, giving an opposite mapping. But the lemmas come after the idioms, and dec is clearly a lemma, and that is that. Certainly, the Nock tutorial shows that dec may be (somewhat) compactly specified in terms of the axioms.

Lemmas

Ax comes equipped with the operations you'll need to have a reasonable Big O on integer algorithms.

You're welcome.

Distinctions between Nock and Ax.

There are four changes between Nock and Ax: the permutation of the operations, the addition of fz, br 0, and the put operator, to list them in order of seriousness. I have made the case, I believe, that the permutations put the jewel in the heart of the lotus. Perhaps we merely gild the lily. This is a matter of taste, not semantics, but without taste, Nock is pointless. I have devoted a number of paragraphs to fz already. Let's discuss br 0 and put.

br 0 provides for a very simple and ordinary thing: a number that comes from Outside, that we can only determine if we go and look. A port, say, or a sensor. I don't know how Hoon deals with this, but I know that Nock does not handle it at all. Ax does: you can hook a very simple piece of Ax up to a sensor and do things with the numbers. I consider this a virtue in a toy computer, and the zero branch was just sitting there being undefined.

I suggest a pleasant detour over to Undefined Intimacy With the Machine. It is not only a product of the Cathedral, it is from a field of critical studies, called code studies, of course. This guy is right out on the bleeding edge, my friends.

Not all code talks to Outside, but most code comes from Somewhere, and sometimes Somewhere might want to drop a noun or two into the mix. I gather this isn't what you do with Nock, and again, I bet Hoon provides it. The thing is that Ax machines make no assumption that the whole system is going to be running on one core, in fact they're decidedly more comfortable if that is not the case.

Could we run Ax machines on a ColorForth array? Zod man, what else are they good for? The Parallela is a less exotic target; if you know any OpenCL, the interaction between br 0 and put might be putting you in mind of a "kernel". Excellent! We're on the same page.

Why do we need fz though, with all this Undefined Intimacy going around? Well, because of the Rules of the Red Bit, basically. An undefined number is going to be some combination of Black and White when you look at it. A Red Bit is already Black or White, but it got that way in a very special manner, and you're not allowed to look at it (though you may copy it) or it's not Red anymore. Not to mention that sealed functions that use fz make perfect sense and should be allowed, and a function either crashes on br 0 or does something special with it, but not in general both.

fz is axiomatic, and we kind of force you to write your code in weird ways if you absolutely want to be sure it never comes up. It's a feature! We call it guessing, and have this notion that it makes for cheap machine learning. We hope a group effort can come up with some good uses.

What about this put business? Well, we like the algebraic feature of Ax a lot. We don't need hints, because it's quite normal for Ax code to come with both embedded symbols and a table containing expansions for the symbols, as well as jets for the assisted ones. In a mature environment, we scrub this of any taint of variability and anonymize everything, so your foo and my foo will be different even if they happen to be equal (minus yet another layer of abstraction!). As it stands, AxUM is a single computation and all symbols must be distinct and have singular expansions.

We do have code with identical semantics, which seems like an odd choice if we're forsaking hints. Here's one good reason: Ax goes to great lengths to be in principle Hoon and Nock compatible. It's just respectful, and if there's to be a transition, let's ease the pain, or delay it insofar as possible.

In fact, use of put will also not affect the semantics of your running code. If you're in some kind of compatibility mode, you may even find that the use of opcode 10 provides pleasant jet assistance to your calculations. This is not our intended use case: put is the other side of br 0.

Without going into detail that will probably prove wrong, br 0 and put define address spaces "above" and "below" the Arc of Ax code that is executing. Nock is Hermetically sealed, which is a virtue; br 0 and put provide a formal, very low-level- mechanism for building plumbing between Ax vessels.

Getting this right will be the true test of the Tree of Life on this machine. The Tree is recursive, fractal, heirarchical, and mnemnonic, among other qualities, all of which comes into play.

Importantly, this is defined at a higher level. It will be the canonical way to link Ax machines together, but is not part of the semantics of the machines themselves, on purpose. They're really quite small.

The main reason for this is that the intricacy of these address spaces will make them in effect kernel-level abstractions. They will end up just as frozen as Ax will, but consequent to that, and with a longer annealing phase.

Philosophy

Well and good, and we all love an elegant mathematical structure. What's the purpose of some of this? Why an Ax machine, which is inductive, nondeterministic, and redundant, versus a Conk machine, which is crisp, deductive, and minimal?

It's not an idle question. There are two Conks with the elegance of the Ax machine, which are Ax α with operator 5 replaced with one of the two possible options. Why choose induction and expansion over deduction and reduction? Why even offer the choice? Usually, we choose computers that are known to halt on problems which are known to halt.

Well, Ax isn't quite that bad. It has a Ko rule, after all. There are two cases to be made, one from elegance and another from utility.

Without expansions, one may not generate the full set of possible operations from a seed. This is a beautiful property in a system, if you're me, and since the expansion Ξ [0 0 0] -> Ξ [0 2 1] may yield (I am convinced) the other operators, it is a beautiful seed for a beautiful system. This is sufficient justification for the automaton; what is mathematics if not elegance, nor elegance if not a species of beauty?

Utility is perhaps harder to see, but imagine an aging computer in a hostile environment. Though it has megas of cores, many are infected and firewalled, others punctured by radiation or otherwise deranged and useless. The ability to expand a failed reduction may save a calculation, and that's no minor thing. A failed or malfunctioning jet may be similarly unit tested against its expansion, fractally and repeatedly, and substitutes checked for correctness in the same way.

The expansions are not actually necessary, that is, the reductions serve to define the calculational semantics. They are included, ultimately, as a bridge to the next level of abstraction: grammars which recognize and transform strings of various sorts into Ax. This is perhaps the most profound advantage of using a cellular automaton, and we intend to make good use of it.

So that's the Ax machine. Maxwell's equations build each on the others, as Euclid's Axioms, as the Laws of Motion and Thermodynamics. With the Ax ordering of Nock, the operators produce their own sequence. Quod Erat Demonstrandum.

I find Urbit interesting for one simple reason: Nock is the most elegant expression of computation I have yet to encounter.

I am no kind of software engineer or computer scientist, at all. I am a professional developer, through a colorful maze of career turns. Curtis Yarvin is trolling when he claims to know no "PL Theory". I'm not.

As an autodidact on most subjects (I can claim a major in chemistry and a minor in mathematics) my coverage is perforce spotty. McCarthy's thesis I get well enough; Alonzo Church and Haskell Curry make me nervous.

You know who I do understand? Douglas Hofstadter. Came by that understanding the old fashioned way, by dropping out of highschool for a semester to read GEB:EGB and the Dragon Book, then dropping back in, then dropping acid. The order there is important. Get your diplomas, kids.

I also understand DNA, at least okay. I wrote a vector once, or truthfully, my professor kindly took the time to explain how vectors work in enough detail that writing the actual base pairs out was not entirely transcription. I've transfected them into plants, as well as wrapping them in tiny pellets of gold and blasting them into cells with high pressure helium, a technique that goes by the sobriquet "shotgun genetic engineering". Shotgun indeed. If any of that sounds exciting, may I recommend lathe work? It pays better.

So when I encounter Nock, what I see is Lisp, rewritten as a cellular automaton, capable of serving as a DNA for computational beasties. That somehow encodes the Tree of Life.

Let's run with that and see where it takes us.

A Cellular Lisp Machine

Many people ask: what on Urth is Nock? Comparisons are made across many abstractions. All of them are interesting, and many are salient. To quote the current #urbit channel topic: "jtauber oh man, the light just went off that [0 1], [1 b] and [2 b c] are just the I, K and S combinators". Indeed this may be the case, but I am in no position to judge it.

I luck out here because Nock is exactly on Urth a cellular automaton and needn't be anything else, or make reference to anything else, to do what it does. A different von Neumann machine indeed! An Ulam/von Neumann machine.

I am quite convinced that Nock could not have been designed without reference to more-or-less every important formalism of computation in the literature. Hangul could not have been designed without intimate knowledge of the Hanzi, but knowing the latter is no help at all in using the former.

Cellular automata are easy to understand if you've read GEB: Nock takes a string, applies rewrite rules in a definite order, and repeatedly reduces until it returns or loops forever. Looping forever can be trivial or not: detecting loops and deciding what to do about them is interpreter-specific.

The DNA of the Urb.

I happen to believe (on alternating Tuesdays) that DNA, as an abstraction, is something that just shakes out of reality when you get oily water to bubble-bath temperature for long enough. The exact base pairs, the resulting mappings, and the detailed chemistry are more than likely homebrewed for each planet. It would be nice to be wrong, but most likely the very basic proteins of other protein-based life would be allergens at best to our chemistry and vice versa.

If you don't get me, consider that caffeine makes a perfectly good substitute for A in your DNA, and if you're a fan of the stuff like I am, your DNA contains a considerable amount of it. On some other planet, caffeine is the standard, and as a result, the alpha helix protein causes us to break out in terrible hives on skin contact. Adenosine-producing plants have a powerful narcotic effect on the native mammals, but their chlorophyll destroys our optic nerves. And so on.

Indeed, we may imagine that early life on Earth had many nucleic acid gangs, locked in struggle, and that our chemistry was most toxic to theirs, and/or theirs least toxic to ours. Perhaps not; such losers have a way of going to ground and remaining in pockets.

The Nock spec could serve as the base pairs of an entire kingdom of number, as DNA is the pillar of the kingdoms of life. That is unabashedly the goal.

Yet number is not matter. There are critical differences. Here's counting in number: 0, 1, 2, …, n or aleph depending on your taste. Here's part of counting in matter: 1, 2, …., ~92. Important corrolary: 1.008, 4.002602, 6.94 map statistically to 1, 2, and 3 due to the fact that atoms are complicated. That is only the very beginning of it.

The result is that molecules literally have aroma, flavor, texture, color. Numbers have none of those things without convention, and a lot of it. It is our task as programmers to show good taste in establishing these conventions.

Executable Representation

Curtis Yarvin sells Nock as a VM, which I view as overstatement and undersell. Virtual machines do not merely calculate, they run. The only binary Nock interpreter is hinted and jet assisted: it may calculate Nock, but what it runs is Hoon.

A cynic would say that what it runs is C. I say fie. When we are rewriting strings (these are mathy strings, not computery strings), recognizing a tedious reduction and supplying the result is proper cricket.

What if you want to make something fast (call it Foon) and bypass Hoon? Well, you're kinda on your own as far as I can gather. The hinting mechanism is used, but is not documented as part of Nock. So write your own Nock interpreter that can run Foon, but it will only calculate Hoon. The HoonNock VM will be able to calculate Foon, but can only run Hoon.

Can this be mitigated? I think it can. One way would be to thoroughly document the hinting mechanism, allowing a compiler writer to target not just Nock, but the existing jets. Extra points for bug compatibility.

Not Nock

Nock is at 5K, though two cosmetic fixes have crept in at that level. I'd call that dodgy, except code temperatures this low have never been measured before. The thermometer cannot by definition be properly calibrated.

Ultimately, I feel Nock may be made to serve. Still, my brain runs at a few notches above room temperature, and it's interesting to think about how Nock could be changed.

Here's what I wouldn't touch a hair on:

1
2

A noun is an atom or a cell.  An atom is any natural number.  
A cell is any ordered pair of nouns.

A righteous man might say "Amen" here. I'll say right on.

I would like to draw the Reader's attention to the second sentence. It contains the fateful words "natural number". The camel pokes her nose into the tent.

So normally when we make formal systems that rewrite strings, we use symbols. See Lisp and GEB for the two cases that matter most here, and check out Game of Life again for extra credit: it's not just Turing complete, but demonstrably self-hosting.

Oh hey, if you want to really impress me, design a Game of Life emulator in Game of Life and then demonstrate Godel incompleteness with it.

Natural numbers, though. A fine choice that shows excellent taste. Not just any set of symbols! The set of symbols. Except it's not a set, is it. Indeed it is not: behold the face of the camel. She smells something she likes. It's warm in here.

We are treated to a punky, fresh set of operators / rewrite rules that let us compute to our computer's content. Only basic arithmetic escapes cgy's benign and giving nature here. We are given increment as a warmup, and it would seem entire Dukedoms were awarded on the basis of decrement. I wonder if I could still score a cruiser for multiply. Do I get more points for O(n^4), or less?

My goodness this beast has a lot of neck. Here, let me move the hookah. In providing only increment I see the logic, but not the wisdom. Worse, I don't see the sense. It was magnaminous to provide the macros, though it may have been done merely to answer the calls of "you cannot possibly use that faster than Brainfuck" that would have followed such a skeletal machine.

Thing is, when someone says 'natural numbers', they have promised a perfectly well-defined subset of operations, which Nock distains to provide. Hoon can do math; Nock just looks at you and says "i++;" a lot.

Decrement is explained with much fanfare. Indeed most of my actual applicative understanding of Nock comes from that explanation. It's a simple case, because there's only one bad value to decrement, namely zero. It's also fairly obvious what to do: crash.

Add, multiply, integral division and modulus are entirely specified under the two words "natural number" and it is camelnine not to include them. Look, let me show this creature out: she'll start spitting or worse, and I like this rug. A div 0? Crash or fisticuffs, sir.

Consider line 7. It plainly says "By add, we mean a restricted subset of addition, namely, addition by one". A little coda saying "oh yeah actually adding stuff in the general case of two arguments? 14" would not instantly bloat our automaton into JVM proportions.

Subtract, though. I could see that going two ways for bad input. One would be to crash; it's the natural thing to do. Another option would be to return a cell with 1 at the head and the result of subtracting b from a rather than a from b at the foot. We might call it a negative number, or a useful error message. It's plausible is it not? Perhaps my Nock machine could jet assist this case and yours could absolutely plod. Clearly code allergy could be the only result.

We do not want code allergy at the level of arithmetic. The only option given the Nock spec is to blindly follow the Hoon spec. I detect an abstraction leak.

Where on Urth could we put these extra operators?

The back of the T shirt, of course. There should be plenty of room under the silkscreened all-seeing Eye.

Call it Ax

Nock is unlikely to change. It's clear that Curtis Yarvin has thought this through, carefully, and simply come to a different conclusion from my own. I don't expect he will say "By Zod, your camel is completely persuasive" and go rewrite everything.

If I had my own toy Kabbalistic computer, which I do not, it would be called Ax. Ax would be a cellular automaton that has the same preamble as Nock, and a similar structure. There are some good reasons for it to be operator-compatible with Nock, but not many, since it would hit the humble minus of Hoon and rip through each loop unless propped up with some kind of unholy JITing.

So let's cut the umbilicus and call this automaton Ax. It has two lines of specification. The rest of this post will not add any. Sorry, yo.

Ax would have a longer spec than Nock. That's okay; Euclid is also longer than Nock, and it's pretty good. Making it shorter wouldn't make it better, though I'm sure there are some steps which we consider repetitive today. That's because it was written for doing geometry, not just for proving it.

Ax would also lack an operator which Nock has, namely 10. Clearly, it may be reconstructed, as it's but a macro. It won't be a hint, though, because hints make me nervous. Jets don't, I emphatically agree with jet propulsion (though not of * for pity's sake), but hints. I believe I have a scheme for organizing Ax code so that hints aren't needed or even useful. This post is going to be a two parter.

I do have a question about 10 that is illustrative. It is said to expand thus: *[a 10 [b c] d] *[a 8 c 7 [0 3] d]. The question is: if one were to open up some Hoon-compiled nock, and replace a 10 with the expansion, would the jet fire?

This seems like the sort of question which cannot be settled by reference to the Nock spec. While the Hoon source resolves it in a sense, that is not fully satisfactory either.

I'm mulling over making 10 a put operator, that discards an atom or a calculation, with the semantics "something else might do something with this value in an implementation-dependent way". Such a tool is drenched in Midichlorians and could be used for great evil or great evil masquerading as good intentions. I'm not opposed necessarily, but I am skeptical of my own urges here.

That would be similar to allowing a calculation to slot on 0 and for any old thing to be in 0, even fairy dust. Gives me shivers just thinking about it.

A Blizzard of Cores

Having a magic 0 address and the ability to mysteriously put data somewhere else isn't just provocative fashion sense. It's a loose mapping to the structure of OpenCL, with which we might Use the Cores. I want to Use the Cores, but if your code has pointers, I can't do that.

We abandon any pretense of having a deterministic machine with such a move. put may be made well defined at a higher level, but a magic address is magic, and we are undone. That's okay. I was going to do that anyway.

Coloured Bits

Ax would differ from Nock in one key dimension, which I doubt can be resolved. Nock is Colour blind, while Ax will be able to detect the Colour of your bits. Clearly to be automatically compliant with law across all juridictions and boundaries, a legal calculation must be able to detect, say, that the input belongs to Columbia Studios, the output belongs to a home theatre, and the combination is not allowed.

Ahem. Colour at such a level is a Layer Four / Urbit concern which Nock nor Ax could possibly handle. There's a gem buried in that link: namely, that randomly generated numbers have a Colour which we must respect if we want to work with them. Which, y'know, we might.

One of the powerful ideas of Urbit seems is that, instead of Facebook and Google, there can be my facebook and my search engine. What Facebook and Google are supposed to do, respectively, is guess which content from my friends I'd like to see and guess what I'm looking for based on a search string. Modulo ads, they succeed reasonably well. Modulo ads, there's less and less left.

It would be nice if taking a guess was easy, cheap, and moronically simple. Simple like XORing an atom against an always-available pool of entropy and memoizing the result with the atom, thus preserving both the entropy and the computation in the same amount of space it would take to preserve the computation's inputs. That can of course be shimmied in at a higher level on a deterministic machine. Such a shim would be expensive. I want guesswork to be really, really cheap.

So Ax has three Colours of bit: black, white, and red. Red bits are resolved when needed from a presumed infinite pool of high-quality entropy. It has an operator, fuzz, which opens an enormous can of worms. What happens when you XOR entropy across 27?

Mu, is what. You can't XOR something that isn't binary. Nock deliberately and throughly rejects bits in favor of natural numbers, and drives the point home by describing them in decimal. Ax doesn't get this luxury. Longer spec. The rule is probably "take as many low bytes from the atom as specified by the byte width of the entropy call" or something similarly convoluted.

fuzz should come before the macros, as it cannot be defined as one. There is no point in operator-level compatibility with Nock. It would be far easier to retarget Hoon at Ax, or more likely, virtualize the NockHoon VM on top of Ax directly. Like I said, two part post.

Any function which has fuzz in it or which contains a slot on the 0 address is not deterministic. Otherwise, it is. The intention is to build some higher level conventions on top of that which allow for flexible communication with running Ax code (infinite loops are not all errors) and for building whatever sort of reasoning engine one might want. Assuming a limitless pool of entropy is a safe bet for future systems, and cheap to provide for current ones.

We can let you cheat and use pseudorandom Pink bits, if you're willing to assume the consequences. Which for ordinary guesswork are presumably not dire.

Other Esoterica

Here's a question: if you took Nock, and permuted the operators, so that 4 is say 7 and so on, would it work?

Provisionally, my answer is "yes". That is, one could target that virtual machine and perform general calculations and nothing would be Wrong. I've talked this one over with jtauber, who really gets this Nock business.

Let's call the permuted Nock machine Conk.

Second question: could you statically recompile Nock code into Conk code by permuting the operators in the same way and have it run on the new machine?

Clearly not. Nock can calculate a number and then use it as an operator, and this calculation would fail on Conk machine. Two increments on 0 can return 2, and then 2 can * on something; that doesn't work if * is 6.

Could one use some kind of reader table to change the meanings of operators when encountered, so that the permutation machine could be used, by swapping one layer for another, as a Nock machine? I believe so, I can't come up with a reason it wouldn't work. One needs a dispatch table, why not several? The hard part of running Nock on Conk is not dispatching the operators, it is sensibly firing jets when necessary.

On Beyond Hinting

This post will have a part two at some point. That's a discussion of a scheme for compressing Ax/Nock code in a way that should let us dispense with hinting altogether. It's half-baked and frozen, and it may take some time before I can pull it out and brown it.

A better use of time is going to be finishing off a usable alpha of Marmalade. Marmalade can be used directly to knock down the conceptual burden of writing and understanding the rest of the software. I consider an Ax/Nock machine running over OpenCL to be step three.

Step two could be a fun blog post, but there's a rule of thumb: if you emit too much vapor, it's a safe bet some is coming out of your ass. Playing with Urbit is more rewarding. Or will be when ~doznec is back online…