Making long transmissions behind Echolink’s back

Good news today! After various trials and errors, REAST’s new Raspberry Pi-based system transmitted both the WIA and VK7 regional news, driven by a web-based scheduler.

The Raspberry Pi is a combination Echolink/IRLP node so it already has an attached radio. The existing interface has PTT control, a sound card connected to the radio and the ability to check if the channel is occupied, an important feature. Channel occupation is not something you get with a basic DIY interface so it’s nice to take advantage of it.

The Echolink software itself is a large modular collection of binaries and shell scripts that invoke each other to get their job done. This makes it straightforward to tap in and automate transmissions by running shell commands.

So far so good. Any automated system just needs to run the right commands at the right time. The only trouble is that the files we need to transmit for our broadcast are about half an hour long. (They contain station identifications at several points so it’s legal but it’s a single continuous transmission.)

As I discovered, Echolink has extensive watchdog features to make sure the radio doesn’t activate PTT for more than 5 minutes. This appears to be a hard limit that is not controllable from the configuration. For the purposes of Echolink and IRLP that’s very reasonable but it’s a major inconvenience for our broadcasts!

The watchdog is somewhere inside the compiled binaries while the actual shutdown of the radio occurs in shell scripts. At first I tried modifying the shell scripts but this quickly turned into whack-a-mole as there are various ways it can fail. After a second interrupted broadcast I decided this wasn’t good enough—Echolink should be completely prevented from knowing that we’re transmitting.

I’m not really familiar with Echolink so I had a look around with strace. I found that the readinput command is tracking PTT, COS and DTMF codes by reading 9 of the Raspberry Pi’s GPIO pins. The most important ones are these:

What I really want to do is intercept the request every time Echolink tries to read one of these values. If we’re in the middle of a transmission from the automatic system we should report “yes the channel is occupied” and “no we’re not transmitting”. To Echolink it will appear that some other station is hogging the repeater. It won’t try to transmit anything and it will just sit tight until it finishes.

How will we know if an automatic transmission is underway? I modified my code so that while it’s transmitting it will create a file called web_tx inside the local directory. When it finishes transmitting it will delete it again. Therefore whenever Echolink tries to read the GPIO we simply need to check whether a web_tx file exists at that path. If it exists, provide the fake response. If it doesn’t exist, let Echolink read the real value.

The actual mechanism for intercepting the reads requires a bunch of code. I wrote my replacement open(), read(), and close() functions in C and compiled them into a shared object. By setting the LD_PRELOAD environment variable to point to my shared object all of Echolink’s file operations end up filtered by my custom code. In a fairly roundabout way it does what I described in the last two paragraphs.

The only remaining trick is to make sure the right environment variables are set for Echolink software, but not for the web server that’s playing the long files. All of the scripts already load their environment from the file located at custom/environment so I just had to add my own settings to that. LIBC_SO and WEB_TX_OVERRIDE are parameters to the interception code.

So that the automation server knows what’s really going on I need to clear out the variable when I run it:

There’s also a persistent service called tbd. I honestly don’t know what it does so I decided to give it the same variables by changing /etc/init.d/tbd to the following:

And with that everything is working nicely. The only catch is that I have defeated Echolink’s watchdog. I’ll need to make sure that we have some form of hardware-based timeout at an appropriate length, perhaps 45 minutes, just in case the Pi crashes during transmission.

With the basics working I can now spend some time improving the code and securing it so it can go on the public internet.

Posted in programming, radio | Leave a comment

Playing an MP3 with compression on Linux/OSX

Some software I’m working on for a Raspberry Pi demands that I play MP3s at particular times. The audio is fed into a radio where it is then transmitted as narrowband FM, a very lo-fi channel. The recordings are spoken word and they will be heard most clearly if the output volume stays near the maximum allowed all the way through playback. In other words, if some parts of the file are quieter than others I’d prefer to bring them all up to the optimal level. That’s the purpose of an audio compressor.

I was pleased to find out that I can do this on the fly pretty easily using mpg123 and sox (brew install mpg123 and sox comes with OSX). Here’s a full command I’m using:

This is what all the bits mean:

-r 44100
Output sample rate. Obviously needs to be the same in the second part of the command.
Send audio samples to stdout instead of audio device
Mono mix (the FM transmitter has only one channel)
--buffer 2048
Use a buffer of 2048 bytes instead of sox’s default of 8192. In my testing the Pi was too slow for the default buffer size and needs smaller ones to avoid underruns.
-t raw -e signed-integer -b 16
Tell play that it will be getting headerless signed PCM with 16-bit samples
-c 1
Single channel, because it’s mono
Read audio data from standard input
Apply a compressor effect (full docs for the following parameters, which can be more complex)
Attack and delay, in seconds. Controls response time to increasing and decreasing volume.
A transfer function I found online that works nicely. It’s a series of pairs—for a given input level in dB, what should the corresponding output level be? So it’s flat at -60 dB, signals around the -30 to -20 dB region will get a nice big boost, and really strong signals around -4 to -2 dB will be brought back a little.
Reduce the overall gain by 6 dB to reduce clipping. In my testing it still clips occasionally but I’m intentionally pushing the limits so that’s okay.

This can all be replicated nicely in python using the subprocess module:


Posted in programming, radio | Leave a comment

Slack vs IRC isn’t going anywhere

If, like me, you indulge in a bit of schadenfreude then the Slack vs IRC debate is the trainwreck that keeps on giving. It’s now a perennial feature of LCA hype and recently it’s been occupying public timelines on GNU social too.

The trouble of course is that everybody’s right. By now most people understand pretty clearly the strengths and limitations of each system. A popular theory appears to be that if you talk about these facts enough you will eventually sway people to your preferred choice. Somehow that never happens. This failure injects fresh urgency because if you don’t reach a consensus there might be two chat rooms instead of one, a true calamity.

Really it’s about feels. And that’s not a bad thing. The reality that technical arguments are often about values is a common impediment to community understanding. I’ve written about it before with regard to free software. The schadenfreude comes in because so often FOSS communities pride themselves on putting technical capabilities foremost in their decision-making. It doesn’t always work.

It is beyond question that IRC is the more FOSS-friendly solution. It’s a 100% open protocol with free implementations available for all parts. While you can turn on the IRC feature of Slack it’s also an important part of free software culture that you don’t depend on or propagate closed source software by using or promoting it.

Equally it is beyond question that Slack is friendlier to non-technical users. It may be straightforward to log in to IRC but that’s not enough these days. Modern chat users demand consistent state across multiple devices and complete scrollback without having to maintain a TCP connection. Serious IRC users, myself included, use some fairly ugly hacks to get around it, whether that’s a screen session, a bouncer or IRCCloud. Some people like push notifications too but that’s a can of worms I’ll leave closed for now.

That’s the crux of it: shall we prioritise FOSS values for the success of free software, or ease of use so as not to alienate non-technical people? It’s not a technical question at all and it’s difficult to answer.

Free software being the uncompromising beast it is, the most straightforward solution would be different free software that solves the same problems that Slack does. If you’re prepared to admit that IRC might nudge people away from the four freedoms then it’s worthwhile to look at possible alternatives like Matrix. Some LCA delegates are already trying out the Freenode integration with some success. Personally I find it’s not smooth enough just yet but it has a lot of potential.

Similarly, those pushing the Slack agenda would do well to be aware of the free software ethos. Reading Richard Stallman’s essays is a thought-provoking experience whether or not you agree with him. Free software and even open source have a strong basis in his ideas and it’s important to understand why people think this stuff is important. But if someone tells me they find IRC slightly shitty I concede their point.

Where does that leave us? Nowhere! Let the games continue!

Posted in free software | Leave a comment

Being ready for damp and dark cycling

A year ago I finally started riding my bike to work, 6.5 km each way. I thought my fitness would be the biggest problem, but no, being organised was harder. Suddenly I needed to worry about clothes and the weather. It was a drag for a while but now I love it. It’s the fastest and cheapest way to get to work and I’m not affected by traffic or late buses.

Rain is the trickiest thing. I can’t just show up to work soggy. On the other hand if I was scared off every time Hobart had a forecast of “showers” I’d never ride my bike. Driving or catching the bus doesn’t help much though—I still have to walk at least ten minutes. That’s plenty of time to get wet shoes even if I have a raincoat or umbrella.

The solution is to be ready for rain at all times. I think I have it figured out now, so here are some tips.

Things on the bike

Panniers — Having decent-sized bags on the bike is critical. I can stash my bike repair kit and lock with heaps of room left over for a change of clothes and lunch. You can buy properly waterproof ones but I didn’t bother. Even in heavy rain my cheap panniers keep things mostly dry. Backpacks are worth avoiding. They’re a great way to get a sweaty back.

Fully charged front and rear lights — When I started I had single-LED lights. This continued until I had drivers wind down their windows to tell me that I was hard to see. Terrifying. After a couple of upgrades I’m using a DING light on the front and a taillight with 5 red flashing LEDs. It gets dark when it’s raining hard and my DING needs charging at least every few days. I make an effort to keep the battery full.

Well-adjusted brakes — My rim brakes are terrible when they get properly wet. I have to yank them really hard to get the same amount of braking I do when they’re dry. As they get worn down I have less and less travel so every couple of months I need to crack out the allen key and adjust them. Otherwise next time it rains I won’t be able to pull the brakes far enough to stop.

Bike lock — It’s not really important for the weather but I find a lock super handy if I suddenly decide I want to stop somewhere before or after work.

Things at work

Dedicated shoes — I keep a reasonable pair of shoes at work. I change into them when I arrive and change out when I leave. I can be rained on both morning and evening but these ones stay dry.

Extra socks — Having at least one pair of socks at work or in my pannier is crucial. If it’s raining I can change into dry socks when I arrive. If the same thing happens on the way home it’s okay because I have more socks at home. During hot weather I might get sweaty feet on the way to work so again the spare socks come to the rescue.

Pants and top (temporarily) — Nobody at work seems to care if I wear the same pants and top Monday-to-Friday (and really I’m only wearing them 8 hours a day so it’s not so bad). Often I’ll leave them in a drawer starting Monday afternoon and take them home on Friday. That’s less stuff to take with me on the bike. I do wear a new shirt each day though.

Deodorant — Self-explanatory.

Things I wear

Dedicated shoes for commuting —  I have a ratty old pair of sneakers and they can get as wet as they like. It doesn’t matter. I have dedicated shoes at work and another pair of shoes at home. Those stay dry. If I wasn’t using the sneakers for this purpose I’d have thrown them out a long time ago so I guess I’m being good for the environment.

Lightweight raincoat with waterproof pockets — In cooler weather I wear this every day regardless of the forecast. Not only am I ready for surprise rain, it acts as a wind-break which is really nice when it’s chilly. Zip pockets are great for my phone, wallet and keys because I have quick access to them and I can be sure they’re safe from water. I feel kind of sophisticated whipping out a handkerchief while stopped at red lights but maybe that’s just me.

Safety vest — My raincoat is black so I wear this over the top to stay visible. Bright flashing lights are nice but I can tell you that when it comes to being seen by drivers a fluorescent vest makes all the difference. Cheap and highly recommended. If I’m riding somewhere in the evening I’ll often throw it over the top of whatever else I’m wearing.

Fluorescent shirts — In warmer weather I ditch the raincoat and wear brightly coloured shirts instead. This means I have to take another shirt with me for work but it’s worth it. If by chance it rains I might get a bit damp but at least I have a change of clothes.

Thermal top — In really cold weather a shirt plus a raincoat isn’t enough. The thermal top helps but if I misjudge it can also make me sweaty, even on cold days. On the bright side it tends to absorb the sweat so if I’m wearing it underneath my shirt the shirt is still okay. I just have to take off the thermal top when I get to work.

Gloves — If I ride in less than 5°C the circulation disappears from my fingers and it’s very painful. Padded gloves fix it. Highly worthwhile.

Quick-drying shirts — My trip to work is mostly downhill but sometimes I sweat a bit. Sometimes I wear a sports shirt made out of material that breathes well so sweat evaporates quickly. I’m not stinky, I promise. It just means I have to take less stuff with me because I can wear the same shirt all day.

Quick-drying shorts — I have a couple of pairs of shorts that I wear year-round before changing into something else at work. It doesn’t matter if they get rained on because they will be dry enough by the next time I need them. Usually the top part is protected from rain by my raincoat so I can use it as a makeshift shammy to dry the rain off my exposed arms and legs before I put on my jeans.

Now I’m in the habit it’s not too hard to keep all this organised. Every day I ride, which is around 90% of the time, I get the satisfaction of not using my car and not spending $5+ on bus tickets. Life is good. Hopefully these ideas are useful to somebody someday.

Posted in tasmania | Leave a comment