Distributed environment construction with Raspberry PI series (Part 4: NFS server construction and client OS import)

0. Introduction

  1. Construction of distributed environment with Raspberry PI series (Part 1: Summary of availability of diskless client by model)
  2. Building a distributed environment with Raspberry PI series (Part 2: Analysis of PiServer and design of alternative system)
  3. Building a distributed environment with Raspberry PI series (Part 3: Installing and setting dnsmasq)
  4. Building a distributed environment with Raspberry PI series (Part 4: Build NFS server and import OS for clients)
  5. Building a distributed environment with the Raspberry PI series (Part 5: Customizing the Raspberry PI OS for cluster nodes (1))
  6. Building a distributed environment with Raspberry PI series (Part 6: Customization of Raspberry PI OS for cluster nodes (2))
  7. Building a distributed environment with Raspberry PI series (7: tftp route setting and startup test for each Raspberry Pi)

This article is a continuation of Part 3. The contents of this time are to build an NFS server to save the root directory to be distributed to Raspberry Pi and to actually import the OS for Raspberry Pi [^ target OS] to the NFS server. Please note that the diskless client will not start just by importing the client OS into the NFS server. After importing, the client will not start unless you edit multiple files and make a symbolic link to the tftp root. Next time, we will customize each imported distribution.

[^ Target OS]: This article targets Debian-based Raspberry PI OS, which is the official OS, and Ubuntu 20.04 LTS, ArchLinux, which are typical distributions for Raspberry Pi. Fedora has announced that it does not support Pi4 as of 06/08/2020, so I will not list it. Then Ignore Windows IoT: -P. By the way, Gentoo people should do their best on their own.

1. Build an NFS server

It's okay to implement NFS on the PXE server you built last time, but if you're assuming usage such as switching between multiple operating systems, it's better to build it on another machine with high fault tolerance and a lot of disk space. I think you can rest assured.

In addition, there are two shared directories to be published on Raspberry Pi.

Shared folder for OS root directory distributed to each Raspberry Pi
In this article, we will share / exports / os4pi on the NFS server.
Under this folder, create subfolders layered under "Distribution name / Version / Architecture".
Shared folder for home directory common to all Raspberry Pi
In this article, we will share / exports / home4pi on the NFS server.

1.1. Notes on NFS sharing settings for OS roots

1.2. When using an existing NFS server / PXE server built last time

Add a new shared (public) folder over NFS. If you are using a commercially available NFS-compatible NAS, the implementation method will differ depending on whether you manage the NFS server yourself.

1.2.1. When using a commercially available NAS (NFS compatible) that only connects to the network

Add the shared folder according to the manual of the product you are using

1.2.2. When using a Linux machine / PXE server built last time

Edit / etc / exports and run the exportfs command Reference: NFS explanation of ArchLinux

/ etc / exports (additional example)

#Stores each distribution for clients
/exports/os4pi     192.168.172.0/24(rw,sync,nohide,no_root_squash,subtree_check)
#Common home directory
/exports/home4pi   192.168.172.0/24(rw,sync,nohide,subtree_check)

Then, actually create the directory specified in / etc / exports.

$ sudo mkdir -p /exports/os4pi
$ sudo mkdir -p /exports/home4pi

Finally, actually enable the NFS shared folder.

$ sudo exportfs -avr

If you get an error, NFS related services such as rpcbind are not running, so please do your best.

1.3. When building a new NFS server

Details will not be described. Build with your favorite OS, your favorite disk (preferably SSD), and your favorite RAID configuration, then go back to 1.2. [^ FreeNAS]

[^ FreeNAS]: If you think your build time is wasted, you may consider using FreeNAS. With FreeNAS, you can easily get an environment that has more functions than a commercial NAS and can be set with a web interface.

2. How to download and import microSD image for Raspberry Pi

From here, the method differs depending on each distribution, so we will describe it for each distribution. For each distribution, the file to be downloaded is a disk image with the smallest data size.

2.1. Official Raspberry PI OS (formerly Raspbian)

2.1.1. Overview

The most balanced distribution except for the official OS and older kernel versions. This is the only option if you want to quickly build a diskless distributed environment, as you can make it diskless without having to mess with any binary files. 2020/06 The 64-bit version, which is currently in beta, is working fine.

2.1.2. Download destination

From the above download site, check the file name of the latest version with a normal browser or w3m and download the zip file. [^ 64 kernel] [^ 64kernel]: Even if it is a 32bit version, it is possible to convert only the kernel to 64bit (although glibc is a 32bit version).

If you want an image of the old version (equivalent to Debian stretch), please download the zip file containing "stretch" in the file name from the following site.

2.1.3. Creating a destination NFS folder

When installing the 32-bit version for Pi2

$ sudo mkdir -p /exports/os4pi/raspbian/buster/armhf

32bit version for Pi3 / 4 (when you want to make only the kernel 64bit)

Since it is necessary to separate only / boot, overlayfs may be better if it is mixed with Pi2.

$ sudo mkdir -p /exports/os4pi/raspbian/buster/armhf_64

When installing the 64-bit version for Pi3 and above only

$ sudo mkdir -p /exports/os4pi/raspbian/buster/aarch64

When installing the old version (stretch, 32bit)

$ sudo mkdir -p /exports/os4pi/raspbian/stretch/armhf

2.1.4. Contents of the official image (zip file)

Check the checksum of the downloaded zip file and unzip it.

# unzip 2020-05-27-raspios-buster-lite-armhf.zip
(Output result omitted)
# 

You should see a file with the extension changed from zip to img (this time using the 32bit Lite version as of 06/08/2020). Since this file is a dd image split into two partitions, create and verify a loopback device.

# losetup -P -f 2020-05-27-raspios-buster-lite-armhf.img
# losetup -a
/dev/loop0: []: (/tmp/2020-05-27-raspios-buster-lite-armhf.img)
# ls /dev/loop0*
/dev/loop0 /dev/loop0p1 /dev/loop0p2
#

loop0p1 is for the microSD first partition (FAT, / boot) and loop0p2 is for the second partition (ext4, /). Create a temporary mount point under / tmp, mount the above 2 partitions, and copy it to the NFS shared folder. I don't want to copy what I don't need from the beginning, but the only thing I don't need is ext4's lost + found. First I copy everything to the NFS shared directory and then remove lost + found when I'm done.

# mkdir /tmp/raspbian; mount -o ro /dev/loop0p2 /tmp/raspbian && mount -o ro /dev/loop0p1 /tmp/raspbian/boot
# rsync -avr /tmp/raspbian/* /exports/os4pi/raspbian/buster/armhf/
・ ・ ・(Copy all files)
# rm -Rf /exports/os4pi/raspbian/buster/armhf/lost+found

When you're done copying, umount it and remove the loopback as well.

# umount /tmp/raspbian/boot; umount /tmp/raspbian
# losetup -d /dev/loop0

That's it for Raspbian.

2.2. Ubuntu

2.2.1. Precautions (Added 2020/06/09)

The kernel included in the disk image distributed by Ubuntu cannot be network booted as it is because the nfs module is ** not compiled with the built-in state. So you have to build your own kernel to make it diskless without U-boot. Then, it's just a quick glance, but the process of downloading the U-boot binary included in the distribution image with tftp and doing something with this U-boot seems to be quite troublesome, such as investigation and testing. In other words, if you don't want to waste your time in the current situation, you should avoid using this distribution.

2.2.2. Overview

Ubuntu Server launches many services that are almost unnecessary for iSCSI diskless nodes such as snapd, apparmor, iscsi, lvm2, and mdadm by default. Therefore, it will be difficult to remove unnecessary services / packages after installation.

I don't think it's necessary to choose this distribution except for those who like it, but since it's a major one, I'll cover it.

When I try to download from Ubuntu official website, there are 5 types, 32bit for 2 / 64bit for 3 / 32bit for 3 / 64bit for 4 / 32bit for 4 There are download options, and you'll be wondering which one to download. As a result of investigating, regarding 20.04 (Focal Fossa), only the button notation is different, and the actual difference is whether it is 32bit or 64bit. Therefore, it is enough to download the 32bit version for 2 and the 64bit version for 3. Also, for 18.04 (Bionic Beaver), image files are prepared for each model, probably because it was necessary to change the U-boot binary depending on the model. The system to be built this time does not use U-boot that comes with the disk image, so it is sufficient to download both the 32-bit version for 2 and the 64-bit version for 3.

2.2.3. Download destination

20.04 LTS (Focal Fossa)

From the above site

Download the latest xz file with the name

18.04 LTS (Bionic Beaver)

From the above site

Download the latest xz file with the name.

2.2.4. Creating a storage folder

'$ VERSION' in the command should be one of 20.04 / 18.04.

When installing the 32-bit version for Pi2

$ sudo mkdir -p /exports/os4pi/ubuntu/$VERSION/armhf

32bit version for Pi3 / 4 (when you want to make only the kernel 64bit)

Since only / boot / firmware needs to be in a separate folder, it may be better to consider overlayfs below the folder for Pi2.

$ sudo mkdir -p /exports/os4pi/ubuntu/$VERSION/armhf_64

When installing the 64-bit version for Pi3 and above

$ sudo mkdir -p /exports/os4pi/ubuntu/$VERSION/aarch64

2.2.5. Contents of the official image (xz file)

After checking the checksum of the downloaded xz file, unzip it.

$ unxz ubuntu-20.04-preinstalled-server-arm64+raspi4.img.xz
(Output result omitted)

You should see a file with the extension changed from img.xz to img (this time using the 20.04 64bit raspi4 version as of 06/08/2020). Since this file is a dd image split into two partitions, create and verify a loopback device.

$ sudo losetup -P -f ubuntu-20.04-preinstalled-server-arm64+raspi4.img 
$ losetup -a
/dev/loop0: []: (/tmp/ubuntu-20.04-preinstalled-server-arm64+raspi4.img)
$ ls /dev/loop0*
/dev/loop0  /dev/loop0p1  /dev/loop0p2
$ 

loop0p1 is equivalent to the first microSD partition (FAT, / boot / firmware), and loop0p2 is equivalent to the second partition (ext4, /). Create a temporary mount point under / tmp, mount the above 2 partitions, and copy it to the NFS shared folder. I don't want to copy what I don't need from the beginning, but the only thing I don't need is ext4's lost + found. First I copy everything to the NFS shared directory and then remove lost + found when I'm done.

$ mkdir /tmp/ubuntu; sudo mount -o ro /dev/loop0p2 /tmp/ubuntu && sudo mount -o ro /dev/loop0p1 /tmp/ubuntu/boot/firmware
$ sudo rsync -avr /tmp/ubuntu/* /exports/os4pi/ubuntu/20.04/aarch64/
・ ・ ・(Copy all files)
$ sudo rm -Rf /exports/os4pi/ubuntu/20.04/aarch64/lost+found

When you're done copying, umount it and remove the loopback as well.

# umount /tmp/ubuntu/boot; umount /tmp/ubuntu
# losetup -d /dev/loop0

That's it for Ubuntu.

2.3. ArchLinux ARM

2.3.1. Precautions

As of 2020/06, the 32-bit version is ready for diskless boot by simply rewriting the /boot/config.txt file.

However, it is impossible to make the 64-bit version diskless boot obediently. I checked the aarch64 version of ArchLinux and found that this distribution also boots with u-boot (the contents of kernel8.img are u-boot). So I uploaded the kernel body and initramfs file that u-boot loads to tftp, edited config.txt, and started the diskless rasp pie. Thanks to bootcode.bin, it loads kernel + initramfs from tftp. Was possible. Then control was transferred to the actual kernel, and while booting the initramfs as a temporary root filesystem, it stopped saying "no network". Apparently, this kernel doesn't have the USB Ethernet driver (smsc95xx) built-in. In other words, it seems that the kernel needs to be recompiled here as well. Such a thing

2.3.2. Overview

For ArchLinux ARM, the installation file is in tar.xz format rather than a dd image file, so the process of importing to an NFS server is not much different from a normal installation. However, the distribution image (rather than the device tree file) is different for each model, and since it is a rolling update distribution, there is no version number, so other distributions cut the subdirectory according to the OS version and architecture. I will separate the place that was there by the model number of the target Raspberry Pi.

Reference: Official installation manual (2B: 32bit version / 3B: 32 / 64bit version / platforms / armv8 / broadcom / raspberry-pi-3) / 4B: 64bit version)

2.3.3. Download destination

As in the official manual

Download from to / tmp.

2.3.4. Create and import storage destination folders at once

ArchLinux is complete by simply running the following one-liner as root. ** Caution **: BSD tar is used instead of GNU tar, so if you do not have it installed, please install it in advance. [^ BSDTAR]

[^ BSDTAR]: It can be expanded with GNU tar, but an error occurs during extraction because it contains an unsupported security identifier.

# mkdir -p /exports/os4pi/archlinux; cd /exports/os4pi/archlinux; for N in `seq 2 4`; do if [ -f /tmp/ArchLinuxARM-rpi-$N-latest.tar.gz ]; then mkdir pi$N; bsdtar -xpf /tmp/ArchLinuxARM-rpi-$N-latest.tar.gz -C pi$N; fi; done

3. Next notice

Next time, customize the OS to be distributed to the client (files under / boot and / etc, limited to the range that can be changed with a text editor). To go. If you don't use qemu-static, you can't go to the wrong range.

Recommended Posts

Distributed environment construction with Raspberry PI series (Part 4: NFS server construction and client OS import)
Build a distributed environment with Raspberry PI series (Part 3: Install and configure dnsmasq)
Distributed environment construction with Raspberry PI series (7: tftp route setting and startup test for each Raspberry Pi)
Building a distributed environment with the Raspberry PI series (Part 2: PiServer analysis and alternative system design)
VPN server construction with Raspberry Pi
Building a distributed environment with the Raspberry PI series (Part 1: Summary of availability of diskless clients by model)
OS setup with Raspberry Pi Imager
Make a thermometer with Raspberry Pi and make it viewable with a browser Part 4
Environment construction with pyenv and pyenv-virtualenv
Build a Tensorflow environment with Raspberry Pi [2020]
Simple VPN construction of IPsec gateway with Ubuntu 20.04 and Raspberry Pi ―― 1. StrongSwan introduced
Source compile Apache2.4 + PHP7.4 with Raspberry Pi and build a Web server --2 PHP introduction
Cross-compiling Raspberry Pi and building a remote debugging development environment with VS Code
Source compile Apache2.4 + PHP7.4 with Raspberry Pi and build a Web server ―― 1. Apache introduction
IPsec gateway VPN construction with CentOS 8 and openSUSE (Raspberry Pi) --2 StrongSwan VPN connection confirmation
MQTT RC car with Arduino and Raspberry Pi
Get temperature and humidity with DHT11 and Raspberry Pi
GRPC starting with Go server and Dart client
Build a server on Linux and local network with Raspberry Pi NextCloud and desktop sharing
Source compile Apache2.4 + PHP7.4 with Raspberry Pi and build a web server --3. Use MySQL
Simple VPN construction of IPsec gateway with CentOS 8 and openSUSE (Raspberry Pi) ―― 1. StrongSwan introduced
Simple VPN construction of IPsec gateway with Ubuntu 20.04 and Raspberry Pi --2 StrongSwan VPN connection confirmation
Record temperature and humidity with systemd on Raspberry Pi
Machine learning with Raspberry Pi 4 and Coral USB Accelerator
Easy IoT to start with Raspberry Pi and MESH
Try to visualize the room with Raspberry Pi, part 1
Detect mask wearing status with OpenCV and Raspberry Pi
Measure temperature and humidity with Raspberry Pi3 and visualize with Ambient
Web application made with Python3.4 + Django (Part.1 Environment construction)
Ubuntu 20.04 on raspberry pi 4 with OpenCV and use with python
Getting Started with Yocto Project with Raspberry Pi 4 and WSL2
Troubleshoot with installing OpenCV on Raspberry Pi and capturing
Raspberry Pi + python + IoT device, environment construction procedure to start image processing and machine learning
RPi / CentOS> python> Execution environment distinction> import os / print os.uname ()> Whether to include raspberry pi