You are sharing your downloads with your Antivirus Company


I recently have provided a customer with a link to a firewall image (using the Turris MOX router with a variant of OpenWRT) hosted on my own webserver. The image included keys material for an OpenVPN connection. The image file was in a hidden directory on my projects webserver. I monitored closely if there would be any downloads besides the one I expected from my customer.

I am aware providing key material via an unsecured channel is not the best security practice. And in the end I had to revoke the VPN key material in the image and provide my customer with a new key via a secure channel.

Now I said I monitored the downloads. About an hour (!) after my customer downloaded the image (at 21/Mar/2021:17:35:50 to be precise), it was accessed from another IP:

77.74.177.4 - - [21/Mar/2021:18:43:51 +0100] "GET /turris-image-XXXXXXX.zip HTTP/1.1" 200 77244886 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.92 Safari/537.36"

Looking up this IP via whois yields:

> whois 77.74.177.4
% This is the RIPE Database query service.
...
netname:        KL-NET3
descr:          Kaspersky Lab Internet
country:        RU
...
source:         RIPE
organisation:   ORG-KL28-RIPE
org-name:       Kaspersky Lab AO
country:        RU

My customer is using Kaspersky antivirus software. So the link was probably leaked to Kaspersky via the installed software. On the one hand it may well be that the purpose of Kaspersky downloading that link is a benign service (they may scan things for viruses) but in my case it means that non-public information was leaked. On the other hand it may well be that information gleaned that way is used for other purposes, too – we do not know.

So consider that your Antivirus product may look over your shoulder when you are downloading things from the web.

Interaction of libvirt and AppArmor


I'm teaching at the University of Applied Science Burgenland in Eisenstadt (Austria). We recently had a lab (which took place in the lab in Eisenstadt but students were working from home due to Covid reasons) where the task is to set everything up for virtualisation and then live-migrate a running virtual machine to another server using libvirt (we're using the command-line with virsh).

For just one group out of several – with identical initial Debian installations, migration failed with an error message. The migration command was:

virsh -c qemu+ssh://root@primary/system migrate --live --unsafe \
    debian-1 qemu+ssh://root@secondary/system

For the lab we're using NFS because setting up a more advanced filesystem would take too much time, that's why we're using the --unsafe option. The following error message resulted (error message broken to several lines, this was all in a single line):

error: internal error: Process exited prior to exec:
libvirt:  error : unable to set AppArmor profile
'libvirt-d22db7ca-50ca-43bd-b6da-1ccecf5a83e7' for '/usr/bin/kvm':
No such file or directory

It turned out that this group had managed to fill up the /var partition with logfiles but after cleanup this still did produce the same message. So the hunch here is that some files that AppArmor and/or libvirt create dynamically could not be created and that was the reason why this failed. It also turned out that some AppArmor files that were correctly installed on the first machine were missing on the second.

Trying to reinstall AppArmor and related files using apt-get with the --reinstall option did not work, the missing config files in /etc/apparmor.d were not re-created. So removing the packages with the purge command (which removes all config files) and then reinstalling everything fixed the installed AppArmor files and made the migration finally work. I have no idea which files were missing.

When googling for the error message above I found a debian bug-report Where one of the dynamically generated files in /etc/apparmor.d/libvirt was zero length. This, however was not the problem in our case but indicates that AppArmor isn't very good at checking errors when a filesystem is full. So there are probably other files that are dynamically generated that were the problem in our case.

The following sequence of deinstall and reinstall commands fixed the problem in our case, note that just removing files as in the debian bug-report did not fix the issue in our case:

dpkg --purge apparmor-utils apparmor-profiles
dpkg --purge apparmor
rm -rf /var/cache/apparmor
apt-get install apparmor apparmor-utils apparmor-profiles
dpkg --purge libvirt-daemon-system
apt-get install libvirt-daemon-system
systemctl restart libvirtd.service
systemctl restart virtlogd.service
systemctl restart virtlogd.socket

I'm not sure restarting the services is really necessary but there was another issue that libvirt could not connect to the virtlog socket and this was fixed by restarting the virtlog.{service,socket}.

Dynamic DNS with the bind DNS server


The popular DNS server bind allows to have a configuration that enables clients to change DNS entries remotely. Since some of the public dynamic DNS services have moved to a pay-only subscription model and since I'm running my own mail and web server at a hosting site I was searching for a way to roll my own dynamic DNS service. This already is back some years now but since the Howto I used at the time seems to be gone (at least from my google bubble) I'm documenting here how it was done.

I'm running this on a Debian buster server at the time of this writing, so if you're on a different system some details may change. I'm calling the domain for the dynamic services dyn.example.com in the following.

The top-level config file of bind needs to include an additional config file for the dynamic domain. In my configuration this file is named.conf.handedited. In this file you need an entry for each dynamic DNS client as follows:

zone "dyn.example.com" {
        type master;
        allow-transfer {none;};
        file "/etc/bind/slave/pri.dyn.example.com";
        update-policy {
            grant h1.dyn.example.com. name h1.dyn.example.com. A TXT;
            grant h2.dyn.example.com. name h2.dyn.example.com. A TXT;
            [...]
        };
};

In this example the hosts h1 and h2 and possibly more may edit their own DNS entry. I'm allowing them to change their A and TXT records. You may want to add AAAA for IPv6.

Then the config-file /etc/bind/slave/pri.dyn.example.com contains:

dyn.example.com         IN SOA  ns1.example.com. admin.example.com. (
                                2020080100 ; serial
                                120      ; refresh (2 minutes)
                                120      ; retry (2 minutes)
                                120      ; expire (2 minutes)
                                120      ; minimum (2 minutes)
                                )
                        NS      ns1.example.com.
h1                      A       127.0.0.1
                        KEY     512 3 10 (
                                AwEAAdEvnGmGO4ku+xms4w1c5RWh5BvugiZ4ty9tkIes
                                <more gibberish lines>
                                ); alg = RSASHA512 ; key id = <number>

The values in angle brackets are comments and should be replaced by the correct values in your installation. The entries A and KEY are inserted by hand for each new host allowed to set its own IP address. The KEY is the public key created below. In my experience an A-record has to be present for it to work, I'm setting the localhost address here because the client will later rewrite this IP anyway. It's customary to have the admin email address (where the @ is replaced with a dot) in the SOA record where I've put admin.example.com..

To create a new host:

  • Create a new public/private key pair (preferrably the client does that and sends only the public key to the DNS admin for security reasons):

    dnssec-keygen -T key -a RSASHA512 -b 2048 -n HOST newhost.dyn.example.com
    
  • This creates a private and a public key. Note that on the client you need both, the public and the private key although in the command line for the dynamic DNS client you will only specify the private key!

  • Last time I created a new key the command did not support keys longer than 2048 bit although the hash algorithm is SHA2 with a high bit-length.

  • You need to freeze (and make bind write the current in-memory DB to a file) bind for your dynamic domain:

    rndc freeze dyn.example.com
    
  • Now you may edit the config file, you want to add a stanza for the new host and increment the serial number:

    $EDITOR /etc/bind/slave/pri.dyn.example.com
    
  • Then don't forget to thaw the domain:

    rndc unfreeze dyn.example.com
    
  • Do not forget to give the new host the necessary permissions in named.conf.handedited

  • You probably need to reload bind:

    systemctl reload bind9.service
    

On the client side the utility we use to tell bind about a new IP address of our client is called nsupdate. You can probably find this program for many client operating systems.

I'm using a simple script that detects a change of a dynamic IP address and performs a bind update in case the address changed. Since you're running your own DNS server, chances are that you also have a webserver at your disposal. The following simple script allows any client to detect its own IP-address (clients are often behind a NAT firewall and we don't want to use another public service when we just got rid of public dynamic DNS services, right?):

#!/bin/sh
echo Content-Type: text/plain
echo ""
echo $REMOTE_ADDR

This script is put into a cgi-bin directory of a web server and will echo the client IP address (in text form) back to the client. I name this script ip.cgi and it is available via the URL http://example.com/cgi-bin/ip.cgi in our example, see below in the client script where you need to change that URL.

My bind update script (you need to change some variables) looks as follows (note that this asumes the script above runs on the top-level domain example.com, otherwise change the URL of the cgi-bin program):

#!/bin/sh
ZONE=dyn.example.com
DOMAIN=h1.dyn.example.com
DNSKEY=/etc/nsupdate/Kh1.dyn.example.com.+010+04711.private
NS="ns1.example.com"

registered=$(host $DOMAIN $NS 2> /dev/null | grep 'has address' | tail -n1 | cut -d' '  -f4)
current=$(wget -q -O- http://example.com/cgi-bin/ip.cgi)
[ -n "$current" \
-a "(" "$current" != "$registered" ")" \
] && {
 nsupdate -d -v -k $DNSKEY << EOF
server $NS
zone $ZONE
update delete $DOMAIN A
update add $DOMAIN 60 A $current
send
EOF
 logger -t dyndns -p daemon.notice "Updated dyndns: $registered -> $current"
} > /dev/null 2>&1

exit 0

It should be noted again that the private key (in /etc/nsupdate/Kh1.dyn.example.com.+010+04711.private in the example above) is not enough, nsupdate also needs the public key in the same directory as the private key.

Hitachi HD44780 text display under Linux


/content/2021/display.jpg

For a project I'm using a text display containing the Hitachi HD44780 chip. These displays come in different sizes, common are 2 lines with 16 characters or 4 lines with 20 characters. The latter is also sold under the name 2004a. These displays use 5V. So these days with most CPUs and microcontrollers running with 3.3V or lower, the display is often connected to an I²C bus via a GPIO extender based on the PCF8574. The GPIO extenders are usually soldered to the display connector. You can buy the display and the GPIO extender separately or packaged together. You will always have to solder the GPIO extender to the display.

Now the correct way to connect the display to a 3.3V I²C-bus would be with a level-converter. But there is a hardware-hack to remove the pullup resistors on the PCF8574 breakout board (they're connected to +5V) which makes the device compatible with 3.3V installations: The minimum high logic level tolerated by the PCF8574 is given as 0.7 * VCC (which would mean 3.5V) but works in practice when driven with 3.3V. Note that the numbers of the resistors (R5/R6 in the hardware-hack link) may vary in different PCF8574 boards, I've seen R8, R9 for these resistors as well as no number at all. The resistors are 4.7 kΩ, usually labelled 472 in the SMD variant.

I investigated if there is a Linux driver for these displays and discovered one in drivers/auxdisplay/hd44780.c. On first glance the driver does not support I²C via the PCF8574 I/O expander. So I wrote a driver and submitted it to the kernel.

In the discussion (thanks, Geert) it turned out that there is a driver for the PCF8574 in the kernel (I had discovered so much) and that the HD44780 driver in the kernel can can be configured via appropriate device tree magic to use the I/O expander. The following device tree incantations define an overlay that configures the PCF8574 on its default I²C address 0x27 and then uses the I/O expander for configuring the I/Os for the hd44780 driver:

// Note that on most boards another fragment must enable the I2C-1 Bus

/dts-v1/;
/plugin/;

/ {
        fragment@0 {
                target = <&i2c1>;
                __overlay__ {
                        #address-cells = <1>;
                        #size-cells = <0>;

                        pcf8574: pcf8574@27 {
                                compatible = "nxp,pcf8574";
                                reg = <0x27>;
                                gpio-controller;
                                #gpio-cells = <2>;
                        };

                };
        };

        fragment@1 {
                target-path = "/";
                __overlay__ {
                        hd44780 {
                                compatible = "hit,hd44780";
                                display-height-chars = <2>;
                                display-width-chars  = <16>;
                                data-gpios = <&pcf8574 4 0>,
                                             <&pcf8574 5 0>,
                                             <&pcf8574 6 0>,
                                             <&pcf8574 7 0>;
                                enable-gpios = <&pcf8574 2 0>;
                                rs-gpios = <&pcf8574 0 0>;
                                rw-gpios = <&pcf8574 1 0>;
                                backlight-gpios = <&pcf8574 3 0>;
                        };
                };
        };
};

Since this is non-obvious not just to me (in my research I've discovered at least two out-of-tree Linux driver implementations of drivers for the HD44780 with the PCF8574 I/O expander) I've submitted a documentation patch to make this better documented for others searching a Linux driver.

Note that the driver for this display uses escape sequences to access the various special functions of the display (e.g. turning the backlight on and off, clearing the screen or defining user-defined character bitmaps). I think those are documented only in the source-code in drivers/auxdisplay/charlcd.c.