Paraprogramming Dispatches

As a heads-up my employer now has an opening for a Lisp programmer in Bergen area. Due to hands-on nature of developing the distributed hardware product the position is 100% on-prem.

Frontend development is terrifying. I know a number of superb frontend developers and admire their work but the ecosystem, the churn, and the focus around human end user experience are all positively intimidating. So ideally for an occasional UI project we would just hire consultants. Sometimes though the stars alinged so I was pressed to do Web UIs but they all were minor specialty projects.

Our latest product is Evacsound. Am rather proud of programming I’ve done on it but there wasn’t much to be done in terms of UI. Such products customarily integrate into customer’s existing SCADA (Supervisory Control and Data Acquisition) systems. These usually are built using specialist solutions with decades of history. They are informative and robust but not necessarily very ergonomic or visually exciting. Either way they are very Not Our Problem area here at Norphonic.

However no product is truly complete until it’s free from developer intervention in its entire life cycle. For Evacsound the remaining sore spot was project commissioning. Tunnels have different layouts and even topology: two of our latest deliveries have 6 underground roundabouts between them. Technical rooms, escape tunnels and ramps are also common. So you can’t really provide these layouts for customer projects out of the box. This phase so far was done by us using some auxiliary programs and SQL queries. They did not abstract internal layout representation sufficiently so an outsider could use it. What we needed was an end-user commissioning tool to allow site definition by unaffiliated automation engineers. That would make things simpler (and cheaper!) for everyone involved. A vendor-independent commissioning procedure also greatly simplifies marketing the system internationally.

So you see where it’s going. As the design was depending on minute knowledge of Evacsound operating principles, onboarding a consultant would have taken quite a bit of time, and we’d still have to follow up tightly in development. Hence this became the first production project where I decided to give agents a shot, not least inspired by this banger by tptacek. My rough estimate for completing this project by myself was over 4 months, including 6-7 weeks for learning Javascript, three.js and up to date frontend practices. Using an agent I completed it in 5 weeks worth of intermittent dev time between other projects. This amounted to 270 commits, of which maybe 20 were my manual interventions. The agent was OpenAI Codex, used both via Web UI/GitHub integration and in console. I believe an experienced front-end developer could complete this AI aided somewhat sooner. However a substantial part of work was exploring the conceptual UX space and adopting backend (also by me, manually) for the best fit: this can’t be readily optimized away.

The resulting code as far as I can judge is somewhat mediocre but not terrible. There is a panel of tests for most of functionality and a number of generated documents.

As this was a greenfield internal project there was no customer with strong aesthetic preferences to contend with. Tunnel designs by nature are very spatial so I leaned into late century three dimensional naïve wire-frame look. No hidden line elimination and texturing here is both an aesthetic choice and a visual aid to make sense of scene contents. With color palette it was settled on bold green for structure and controls, red for selections and yellow for auxiliary elements. Neon blue is reserved for elements under automatic detection and distance calibration but this flow is not represented in this demo.

Our basic building blocks are the following connexion elements:

• Nodes. These correspond to installed Evacsound devices and are depicted as pyramids on tunnel profile sections. Its properties are unique ID (three last MAC octets) and distance to the next node.
• Zone markers. These partition the tunnel into subdivisions that can be orchestrated by the system separately. For instance the operator can run a public announce only in select zones.
• Joints. They connect the major extent with a sub-extent on either side. It’s egress property specifies if it’s an evacuation exit, representing the different function of the first node there with a rectangular segment. The V chevron on a joint points in proper left to right direction of the extent.

There are also visual aids:

• Spans, the line along an extent of nodes labeled with distances.
• Spacers, seen as zigzags between connexions or chevrons on the sides of joints. Selecting these allows you to add connexion elements via transient buttons. Their other function is to represent the contraction of tunnel length: plotting the tunnel to scale would have made for extremely long walks in a very sparse model. Spacers expand and contract in relation to the distance between the nodes.

You can try it on the demo below, or as a stand-alone version here. Use mouse to orbit and select elements. Zoom wheel or gesture advances the camera. The elements can be added on zigzag spacers using the buttons in the upper row or by pressing Enter if you want a node. They can be deleted with Del or Backspace. Naturally it is not connected to any backend and all communication logic is omitted. You can also fly to the node you want by entering a part of its UID in the search field on the top.

So this was an interesting experience. There’s some debate how true the metrics are but I honestly estimate the AI allowed me to move much faster than I could otherwise. What does that spell for our trade, job prospects, junior pipeline and enjoyment of our craft is perhaps best left for a separate post. For now let me summarize my practical AI process experience thus far into Eugene’s Eightfold Path:

1. Use an agent. You’re not going to build anything substantial by copying and pasting code to/from the chat window. Before you say ‘oh but it works it’s the AI who can’t handle the caliber of my work’ no just shut up and try a coding agent. There are offerings by all the big players to get you started easily. Any of them will be better than copypasting simply because they provide the embodiment for code within the system/toolchain which helps eliminate hallucination issues.

2. Have some programming experience. Contrary to many enthusiastic reports you totally need it. Perhaps that would change one day but at the moment I can’t see how. To take advantage of LLMs in a problem domain you need to have substantial expertise in the domain, that’s just how it works. You should know what is simple and what is hard to do: this gives you a better chance formulating the requests in terms of what’s possible. You want to recognize when the agent misunderstood you or made a poor design choice. You need some sense of taste to code as ugly solutions give the model gradient descent into intractable, unfixable mess down the road.

3. It helps if you’re a half decent writer. Frankly a double major in English (or any other language) lit & CS could be the best skill set for leveraging this technology. LLM appreciates focused description, metaphors, relevant detail and precision use of adjectives. All this helps to map your wants into its thousands-dimensional conceptual space. Anxious about that great novel you never finished or even started? Well here’s the place to flex your literary muscle.

4. Perform one simple request at a time. Do not combine even related tasks; don’t submit multi-step stacks of work in one prompt. On each your request LLM essentially runs an optimization job, and multi-parameter optimizations are hard to do well. Sequence everything into separate requests. Chew each one down with the best description you can come up.

5. Steer the agent with strategic design choices. This is directly related to the second point. If you know what an internal data structure or algorithm would be a perfect match for the problem start with that. By nature of iterative development the agent will come up with the simplest structure to solve the immediate request. Long term it would become outdated and what the agent is likely to do is to put more graft and patches atop of that. Since you presumably have a further horizon insist on sensible design choices from the start. You should help solidify the foundation so that LLM has easier time putting meat on the bones.

6. Separate functionality into different files each few hundred lines long. How long is too long will change as available compute grows but that’s the idea. It simplifies reasoning for the model by reducing the context.

7. Add basic documentation and make the agent follow it up and update as necessary. I went for a bird eye view design document plus specific documents for each subsystem. It has a two-fold benefit of helping keep the LLM grounded and providing readable summary diffs for you after each iteration.

8. Use the technology you understand for incomprehensible things will hamper your participation. In my case it was eschewing fancy frameworks for plainest JS possible. Use also the technology that has most training corpus for LLMs. This really means Javascript, Go or Python and their most popular libraries. Stuck with a C++03 codebase? Well sucks to be you: you were warned it’s a terrible choice two decades ago anyway. We can only hope the technology will catch up for other languages but it’s clear the corps won’t be getting a firehose of fresh StackOverflow data anymore.

It was the early 1990s. Hard drive manufacturers fought hard to come up with ever larger devices, pushing to the coveted gigabyte boundary. At some point they got enough platter density in consumer grade devices to haul over a billion bytes. There was one problem: the industry convention was that kilo-, mega-, and gigabytes were all power of two units. That made them some 74 million bytes short of the goal.

Not that it stopped them from advertising their disks as gigabyte devices. Thus the Marketing Megabyte was born.

Engineers were understandably pissed, and consumers eventually started to notice that storing a gigabyte of data as reported by their operating systems on these disks is simply not possible. This culminated in a tangible threat of class action lawsuits. In response the marketers instead of fixing their units made an effort of redefining the units themselves. This was successfully pushed through International Electrotechnical Commission in 1999. However the threat of lawsuits lingered until 2007 when this nonsense was finally adopted by national standard bodies. Marketing Megabyte become the Government Megabyte and engineers were told to walk off to substitution units instead.

So let me sum it up here for the next time this issue inevitably flares on the usual tech aggregator websites:

• Raison d’être for the change was the compulsion to lie to the customers without liability.

• Greek-derived prefixes kilo- and mega- refer to sharp thousand and million respectively. How important it was after decades of established use in the industry however is unclear. To a non-technical user it makes no difference as long as the unit is used consistently. To a computer engineer, Marketing Megabytes are largely unusable. Either way, the proposed substitution acronyms kibi-, mebi- and so on in culmination of circular reasoning still use the same Greek prefixes.

• Binary computers address memory in powers of two. This makes the power of two KB/MB/GB a natural measurement unit. To the day it’s impossible to buy a memory chip with capacity in round number of Marketing Megabytes because it simply doesn’t make sense.

• Marketing Megabyte, while standardized via IEC is still not a SI unit. Which makes sense since byte is not derivable from fundamental units hence International Bureau of Standards has no business managing it. Marketing Megabyte proponents often try to muddy this point by insisting kilo- is a SI prefix (it’s Greek really, see above) but kilobyte is still not a standard SI unit.

• Kibi-, mebi-, and gibi- prefixes sounds patently silly. I have lost count of engineers who either chose to ignore them or can’t say them aloud without embarrassed smile. They could make great cartoon character names though.

tl;dr beating a dead horse

I’m a sucker for sage advice much as anyone else, and Kernighan is certainly right on money in the epigraph. Alas there comes a time in programmer’s career when you just end up there despite the warning. It could be that you were indeed too clever for your own good, or maybe the code isn’t quite yours anymore after each of your colleague’s take on it over the years. Or just sometimes, the problem is indeed so hard that it strains your capacity as a coder.

It would usually start with a reasonable idea made into first iteration code. The solution looks fundamentally sound but then as you explore the problem space further it begins to seep nuance, either as manifestation of some real world complexity or your lack of foresight. When I run into this my first instinct is to instrument the code. If the problem is formidable you got to respect it: flailing around blindly modifying things or ugh, doing a rewrite at this stage is almost guaranteed to be a waste of time. It helps to find a promising spot, chisel it, gain a foothold in the problem, and repeat until you crack it. Comfortable debugging tools here can really help to erode the original Kernighan coefficient from 2 to maybe 1.6 or 1.4 where you can still have a chance.

Lisp users are fortunate with the options of interactive debugging, and one facility I reach often for is the plain BREAK. It’s easy enough to wrap it into a conditional for particular matches you want to debug. However sometimes you want it to trigger after a particular sequence of events across different positions in code has taken place. While still doable it quickly becomes cumbersome and this state machine starts to occupy too much mental space which is already scarce. So one day, partly as a displacement activity from being intimidated by a Really Hard Problem I wrote down my debugging patterns as a handful of macros.

Enter BRAKE. Its features reflect my personal preferences so are not necessarily your cup of tea but it could be a starting point to explore in this direction. Things it can do:

• act as a simple BREAK with no arguments (duh)
• wrap an s-expression, passing through its values upon continuing
• trigger sequentially based on the specified position for a common tag
• allow for marks that don’t trigger the break but mark the position as reached
• provide conditional versions for the expressions above
• print traces of tagged breakpoints/marks

If you compile functions with debug on you hopefully should be able to see the wrapped sexpr’s result values.

(use-package '(brake))

(defun fizzbuzz ()
  (loop for n from 100 downto 0
	for fizz = (zerop (mod n 3))
	for buzz = (zerop (mod n 5)) do
	(format t "~a "
		(if (not (or fizz buzz))
		    (format nil "~d" n)
		  (brake-when (= n 0)
			      (concatenate 'string
					   (if fizz "Fizz" "")
					   (if buzz "Buzz" "")))))))

These macros try to detect common cases for tagged sequences being either aborted via break or completed to the last step, resetting them after to the initial state. However it is possible for a sequence to end up “abandoned”, which can be cleaned up by a manual command.

Say in the example below we want to break when the two first branches were triggered in a specific order. The sequence of 1, 3, 4 will reinitialize once the state 4 is reached, allowing to trigger continuously. At the same time if we blow our stack it should reset to initial when aborting.

(defun ack (m n)
  (cond ((zerop m) (mark :ack 3 (1+ n)))
        ((zerop n) (mark :ack 1 (ack (1- m) 1)))
        (t (brake :ack 4 (ack (1- m) (ack m (1- n)))))))

In addition there are a few utility functions to report on the state of brakepoints, enable or disable brakes based on tags and turn tracing on or off. Tracing isn’t meant to replace the semantics of TRACE but to provide a souped up version of debug by print statements everyone loves.

CL-USER> (report-brakes)
Tag :M is DISABLED, traced, with 3 defined steps, current state is initial
Tag :F is DISABLED with 2 defined steps, current state is 0
Tag :ACK is ENABLED with 3 defined steps, current state is initial

Disabling breakpoints without recompilation is really handy and something I find using all the time. The ability to wrap a sexpr was often sorely missed when using BREAK in constructs without implicit body.

Sequencing across threads is sketchy as the code isn’t guarded but in many cases it can work, and the appeal of it in debugging races is clear. One of those days I hope to make it more robust while avoiding potential deadlocks but it isn’t there yet. Where it already shines tho is in debugging complex iterations, mutually recursive functions and state machines.