Improving ARP performance on slow AX.25 links

Recently I’ve been setting up TCP/IP connections over VHF radio. They are Linux systems linked by 1200 baud AX.25 modems. Opinions vary as to whether the overheads of TCP/IP are worthwhile at 1200 baud—9600 is certainly better—but it’s handy and it definitely works provided you use the available bandwidth carefully. All IP communications need to be precise and tidy. Otherwise the channel gets gummed up for seconds at a time and it’s not much fun to use.

Using Debian/Raspbian one of the most obvious problems is ARP traffic. When I watch the channel with axlisten I see huge quantities of requests and replies flying past, maintaining associations between callsigns like “VK7NTK-1” and IP addresses like*.

At home I have two separate computers and radios set up to talk to each other. With nothing in the ARP cache, if I run ping -c 1 I would naively expect to see something like the following. It’s basically like Ethernet except instead of MAC addresses there are callsigns and SSID numbers.

First the pinging station finds out who owns the IP; then it sends the ping. Aah, if only.

Problem 1: Duplicate ARP requests

Using a default ARP configuration the transmissions actually turn out like this:

At 1200 baud it takes most of a second to transmit even the shorter packets. The modems also check that the channel is clear before transmitting, introducing additional delays. As a result, after VK7NTK-2 decided it needed to do an ARP query the other computer didn’t even finish receiving it until 0.8 seconds later. VK7NTK-1 transmitted its reply, taking another 0.7 seconds, so VK7NTK-2 had the address it needed about 1.5 seconds after its initial request. Unfortunately at the 1 second mark it decided that the first request had been lost and queued a second query for transmission.

When that first reply did come back at 1.5 seconds VK7NTK-2 realised it had the information it needed to do the ping so it created the ICMP packet and queued that for delivery too. At that point it had two packets to transmit: the second ARP query, now unnecessary, and the ping request. In this case the modem decided to transmit them back-to-back, which is allowed.

VK7NTK-1 received both of these and queued up a reply to each one. These were also transmitted back-to-back and the ping ultimately succeeded.

The basic problem is that 1 second is far too short a timeout for ARP requests at 1200 baud. Linux isn’t clever enough to cancel the second ARP query when it’s no longer needed so I end up with an entire extra request and reply.

But wait, there’s more! Several seconds after the ping there is suddenly an ARP query in the other direction, originating from the computer that received the ping. I’m guessing this is an optimisation in Linux to warm up the ARP cache in advance. This one was even messier:

In total VK7NTK-1 transmitted three ARP requests and VK7NTK-2 transmitted two ARP replies. Two of those transmissions doubled—that is, both modems thought the channel was clear and started transmitting simultaneously. VK7NTK-1 didn’t receive the answer it wanted because it was too busy shouting the question over the top.

This is also caused by the 1 second timeout but in a different way. If you send out a query that takes 3/4 of a second to transmit, then add a little time for processing and making sure the channel is clear, you would assume that the reply will begin at about the 1 second mark. For the sender of the query, 1 second later is about the worst possible time to make a second transmission, regardless of whether it has anything to do with the previous one. You’re just asking for a double.

Solution 1. Adjust the the ARP retransmission time

The good news is that Linux makes it easy to change the ARP retransmission time. The following command increases it from the default 1 second to 5 seconds:

I placed this in a script that runs after the ax0 interface is configured. It solves both of the above problems fine.

Problem 2: The forgetful cache

After all the nonsense they went through exchanging ARP packets it would be nice if Linux remembered the results for a while. Unfortunately it doesn’t. It’s slightly randomised but roughly 15 seconds later the ARP entry will change from “REACHABLE” to “STALE”. A subsequent ping will cause the whole schmozzle to happen again. It wouldn’t be a big deal on zippy Ethernet but it really is on 1200 baud.

You can check the state of ARP entries using this command:

Solution 2: Increase the reachable time

This is another parameter that’s easy to control. To have 10+ minutes in the REACHABLE state:

The best solution: static ARP entries

Unlike Ethernet where most people are using DHCP and changing addresses semi-regularly, AX.25 stations tend to have pretty fixed addresses. Adding a permanent ARP entry means that Linux never makes any requests at all for that station.

I use this command to add an entry for my upstream gateway VK7HDM-6:

It is still good to have the other optimisations in place for stations that come and go.

* The problem looks slightly worse than it is because axlisten (or arguably the kernel) is a little buggy. The obsolete AF_INET raw interface seems to report duplicates of some frames. I have confirmed with my own software that the newer AF_PACKET/SOCK_RAW interface works fine. See manpage packet(7).

P.S. The very nature of doubled transmissions is that the computers don’t really know what’s going on and Wireshark gives misleading results. An iPad clock and phone video are a low-tech but dependable way to observe sub-second collisions.

Posted in radio | Leave a comment

Making the CW filter work on an FT-817ND

I recently ordered a lovely 500 Hz mechanical filter to use for CW on my FT-897 and FT-817—the YF-122C. I was baffled for much of this week because I could not make it work in the FT-817. If I changed the OP FILTER setting to CW it was as though there was no filter installed at all. If I set it to SSB the filter was engaged but of course the offset was wrong. It’s not helpful at all unless I want to do CW at 1250 Hz.

The difficulty turned out to be this cheeky little menu option: NAR. This is one of the settings that you access by pressing the F button briefly then rotating the SEL knob. When a narrow CW filter is installed it must be engaged by pressing the C button, which places the little triangle to the left. This function key lets you easily toggle the filter in and out, which is very useful when you’re scouting around for QSOs.

FT-817 function menu showing NAR enabled

Yaesu does not bother to point out this setting in the part of the manual that describes installing the filter. I am writing this small post in the hope that it will save someone else some trouble.

Posted in radio | Leave a comment

Sending JSON over websockets in Rust

I like the JSON format but I feel it’s important to contain it to the job of passing a message from one system to another. Ideally the sender generates the text automatically from some strongly-typed object and the receiver does the reverse. If any required parts are missing or a field is the wrong type the entire message is rejected. Skipping this step and operating directly on the fields, as you can do in Javascript or with NSJSONSerialization, relies on the programmer not stuffing it up. I usually stuff it up.

I’m working on software that needs to send regular status updates over a TCP connection to a central server. JSON is a nice human-readable choice for the data format. Websockets are great because I can lean on an Apache proxy to provide LetsEncrypt-signed TLS, and unlike a raw TCP stream the communication is already packetised into discrete messages.

I felt like using Rust but had no idea how hard it would be compared with a dynamically typed language, so I prototyped it. It turned out relatively straightforward thanks to a couple of excellent crates, serde_json and ws-rs:

This is the data model, which is similar to what I have planned for the real app. To exercise the parser I made sure it has a few tricky bits: an enum with associated values, an array, and a struct of one type within another. I didn’t need to tweak them at all. The default implementations of Serialize and Deserialize cope perfectly.

The None type of Option is represented as a JSON null, and the serde parser will quietly provide a None if the field is missing entirely in the JSON. While serde_json supports various enum formats, without any customisation I already get a very nice representation for the overall PlayerState. (serde_json’s output is minified.)

Very little is required on the server. The listen function provided by ws-rs blocks, using an mio event loop under the hood to handle an arbitrary number of connected clients. Right now I’m only providing a handler for each incoming message. If I can parse the JSON I do so and print the debug representation of the PlayerState.

As I hoped, the PlayerState is fully filled in from the JSON:

Over in the client it needs to juggle both the websocket connection and a timer to send the JSON at intervals.

Based on the ws-rs guide, I’ve made a struct that represents the established connection. The Handler trait lets me respond to messages and all the websocket lifecycle events by implementing the relevant functions.

Since I don’t care about messages from the server I only implement on_open and use it to start a new thread for sending updates. ws-rs takes care of thread safety—all I need to do is clone the Sender and give it to my new thread where it can use it to send messages. For this test I’ve created a hard-coded status and simulated playback by increasing the position on every update.

Similarly to listen, connect blocks for the duration of a complete connection, after which it returns and the program terminates. This disposes of the update thread, which would normally need to be cleaned up properly.

All this makes me very happy. I love using enums in data models and enums with associated values are even better. In my experience translating enums to numbers or string placeholders can be tedious and a great source of bugs. Having them transparently travel through the JSON means I can use them heavily and be guaranteed that they will have valid values at the other end. And if I did want to create a browser interface it would be easy to replicate the format in Javascript, albeit with none of the safety. I think I’m going to carry on with Rust for this project.

In the meantime Swift 4 finally has good support for JSON encoding so I am looking forward to using that at work.


Posted in programming, rust | Leave a comment

Operating Packet Radio on Debian

To experiment with packet radio there are some basic things I need to be able to do. I need to monitor all packets, including those not addressed to me. I need to send individual packets, including a particular digipeater path if I choose. I need to make proper AX.25 connections to connect to BBSes. Ideally I want people to be able to make connections to my node too, and act as a digipeater for others to use.

The old hardware TNCs have all this functionality built in and you control it by connecting to its serial port and typing a series of specialised commands in the terminal. When I attach a KISS TNC to Debian, either in hardware or using a software modem, it’s totally not obvious how to use the AX.25 functionality in Linux to do these same basic tasks.

This post is a short overview of the tools built into Debian/Raspbian that I use to do the same things that you can do with a traditional TNC.

The Setup

The required packages on Debian are ax25-apps and ax25-tools.

To use the Debian AX.25 software the modem must be connected as a real network interface. In other words if you run /sbin/ifconfig you should see an interface called ax0:

Broadly speaking the port and callsign need to be defined in /etc/ax25/axports, the TNC needs to be running/connected, and finally kissattach binds it to an interface. The Dire Wolf soundmodem documentation and TNC-Pi documentation both have very clear instructions about how to do this configuration. The details will vary for different hardware.

Monitoring packets

This is my favourite set of parameters which enable timestamps, include outgoing packets in the output, and enable the use of ANSI colours in the output. See the man page for more detail.

It stays running and as packets are sent and received it presents neat output like the following until it is interrupted with ctrl-c.

Sending individual packets and beacons

Individual UI or “unproto” packets are useful for testing, APRS, having conversations via the ISS and so on. There is a terminal program that sends these and it’s called beacon.

To include a digipeater path use a space-separated -d parameter.

In this example CQ is the destination, ARISS is the callsign of the digipeater I want to use, and “1” is the name of my radio port, i.e. the first column in my axports file. This could easily be wrapped inside a shell script to avoiding needing to type so much every time.

The main purpose of beacon is to transmit a message periodically. If you don’t use -s it will go into the background and keep sending messages every 30 minutes, which is another useful function of some TNCs. See the man page for more info.

Connecting to nodes

axcall is an interactive terminal for connecting to a remote node. The top part of the screen shows text coming back from the remote station and the four lines down the bottom are a buffer where you type.

To terminate a connection cleanly from the local end, press ctrl-] to activate the menus then the down arrow twice to select the “Exit” option, then press enter. There are also menu options for sending data files which I haven’t experimented with much.

Note that traditionally AX.25 terminals indicate the end of the line with a single carriage return and this is what axcall uses. This is different from both Windows (CRLF) and Unix (LF) but is the same as pre-OSX Macs.

Receiving packets

I have more experimenting to do here but here’s an easy way to hook up a program to receive incoming connections using ax25d, an AX.25 equivalent of inetd.

Delete or rename /etc/ax25/ax25d.conf to get rid of all the example junk and replace it with something like this:

The ax25d configuration format is complex but this is enough to make it work. Note that the callsign including SSID and the port name from axports are both included in the square brackets.

The path to the program must be followed by all its arguments. If you are familiar with systems programming you will recall that the first argument to a program is always its own name.

Then run sudo ax25d, which will disappear into the background if everything is fine. Now whenever a station connects they will receive a fortune then have the connection promptly terminated.

Showing connection state

This shows the stations that you are currently connected to and whether there is a program listening for ax25 connections.

Example output:



Posted in radio | Leave a comment