Installing and using NeXT OPENSTEP in VirtualBox for Linux


Introduction and some history

My first micro computer was an Apple II+, which I used extensively both for work and leisure. In fact I liked it so much that I bought a //e when Apple Computer, Inc. released that model. I was not tempted by the Apple /// or Lisa when they were released, although I did quite fancy the IIGS but could not justify buying one. The //c was a nice portable, and a family member bought one on my recommendation. I was not at all tempted by the first Macintosh and subsequent models using the so-called Classic Mac OS, but I drooled when Steve Jobs founded NeXT, Inc. in 1985 and launched the magnesium-cased NeXT workstations: the cube-shaped NeXT Computer (Motorola 68030 CPU) in 1989 and in 1990 the second generation (Motorola 68040 CPU) NeXTcube and the NeXTstation (commonly referred to as ‘the slab’) running the NEXTSTEP operating system. The hardware build quality and aesthetic were fabulous, and the machines and NEXTSTEP were way ahead of their time. NEXTSTEP, which was built around Unix and therefore fully multi-tasking, looked amazing when compared to the competition and its performance was superior. Drooling was all I could do, though, because the price of any NeXT machine was totally out of my league.

OPENSTEP 4.2 Desktop in a VirtualBox VM

OPENSTEP 4.2 Desktop in a VirtualBox VM.

By the way, Tim Berners-Lee invented HTTP, HTML and the first HTML browser using NEXTSTEP on a NeXTcube at CERN: see The Science Museum, London – The World Wide Web: A global information space.

Following Apple’s acquisition in 1997 of NeXT, which by then was only a software company (NeXT Software, Inc.), Apple developed Mac OS X based on OPENSTEP (the successor to NEXTSTEP). Even today some of the features in macOS are the same as in NEXTSTEP and OPENSTEP: NeXTSTEP vs Mac OS X – System Demo and Comparison. The final release of NEXTSTEP was NEXTSTEP 3.3, succeeded by OPENSTEP, the final release of which was OPENSTEP 4.2. OPENSTEP was effectively NEXTSTEP 4.

So, even though the NeXT company only sold around 50,000 machines during its relatively short existence as a manufacturer between 1988 and 1993, its impact on modern computing has been significant. Below are a few links to interesting videos about the company and some of its products. You’ll find plenty more videos about NeXT on YouTube.

You can still find the occasional second-hand NeXT computer on eBay, but they are either incomplete or very expensive. As I write this there is a complete and pristine-looking NeXTcube system, including (non-working) NeXT laser printer, in Portugal listed on eBay at US$35,000 plus US$750 shipping! So I will never get to play with a real NeXT computer. But, thanks to VirtualBox, I can at least install the i386 release of OPENSTEP 4.2 in a VM (virtual machine) to try it out for fun. I decided to install the OS and the type of applications I would typically use (assuming I could find packages on the Web, that is). I wanted to find out how usable the OS was, how good the applications were, and whether I could access Unix easily from the GUI. As NeXT hardware and software are obsolete I had to spend a lot of time searching the Web for applications that would actually install and work. Some applications work in both NEXTSTEP and OPENSTEP, but plenty of applications have different packages for the two versions of the OS, which made my searches more complicated. Some OPENSTEP packages are so-called ‘fat binaries’ containing executables for some or all the different CPU types that OPENSTEP supported, and I found a few such packages on the Web. I wanted to install and try to use at least a Web browser, a word processor, a spreadsheet, an mp3 player and a video player. I also wanted to see if I could access files on a server on my home network using Samba.

There are quite a few tutorials and videos available on the Web explaining how to install OPENSTEP in a VM, but I did not find any on installing applications in OPENSTEP. Also, many of the OS installation tutorials I found are incomplete, for example not covering either audio or networking. I am not going to give a step-by-step explanation here of how I installed the OS and the applications, but I will explain what I installed, how I rated it, and any other information I found interesting or useful. Hopefully the tips I provide will be of some help if you fancy installing the OS and any applications yourself. I should also mention that you will have an advantage if you are a Unix and/or Linux user and are au fait with using the command line. OPENSTEP 4.2 provides the C Shell (csh). I did come across a package for the Bourne Again Shell (bash), but have not tried to install it. Sometimes I had to resort to the Unix command line to change ownership or permissions of a file and to move applications to folders owned by the root user. The pwd, cd, ls, su, cp, mv, chmod and chown commands came in handy a few times. By the way, unlike Linux the ls -la command does not display the group to which a file belongs, only its owner; you need to use the command ls -lag to show both. Also, the chown command accepts the notation owner.group but not owner:group when changing attributes.

Installation of OPENSTEP/Mach 4.2 for Intel i386 in VirtualBox

‘Mach’ refers to the Mach kernel, a microkernel developed at Carnegie Mellon University. OPENSTEP was available for Motorola 68k, Intel i386 and Sun SPARC CPUs. VirtualBox supports both 32-bit and 64-bit Intel CPUs, so the 32-bit OS can be installed in a VirtualBox 32-bit VM. NEXTSTEP also supported Hewlett-Packard’s PA-RISC CPU, but NeXT dropped support for that CPU in OPENSTEP.

Regarding the spelling of the two OSs, apparently the APIs are spelt ‘NeXTStep’ and ‘OpenStep’, and the OSs are spelt ‘NEXTSTEP’ and ‘OPENSTEP’. Confusing, or what? It’s no wonder these are used interchangeably all over the Web.

I found a reasonable tutorial on the installation of OPENSTEP 4.2, including links to download the image files of the CDROM and floppy disks required. Unlike many tutorials on the Web, it also explains how to get network access working, and I was able to ping other nodes on my home network and the Internet once I had completed the tutorial: ‘Installing NextStep OS (OPENSTEP) in VirtualBox‘. There were only one or two minor differences between the tutorial and what I saw on screen, and installation in VirtualBox for Linux was essentially painless. One of the packages that has to be installed (OS42MachUserPatch4.pkg) includes a Y2K patch for the OS. The tutorial tells you to use the command line to install that package, and I followed the instructions in the tutorial but, having now learned how to install packages via the OPENSTEP GUI by selecting a package and then ‘Services’ > ‘Open Sesame’ > ‘Open As Root’ > ‘Login’ to launch the Installer, I could have used only the GUI instead of the command line to install OS42MachUserPatch4.pkg (which I have checked). No matter, though, because using the OPENSTEP command line in Terminal.app is a good learning exercise. The tutorial does not mention some other things I had to configure in VirtualBox. To get audio working I had to select ‘SoundBlaster 16’ for the Audio Controller, install a driver in OPENSTEP and reboot the VM (see details further on), and under ‘Network’ in VirtualBox Manager I had to select ‘Bridged Adapter, PCnet-PCI II (Am79C970A)’ with ‘Promiscuous Model: Allow All’. I also enabled ‘Serial Ports’ and disabled ‘USB Controller’ (USB had not yet been invented back then!).

The OS installer installs US English support and offers the option of installing support for any of five other languages too: Swedish, Spanish, Italian, German and French. I unticked all those and completed the installation. Later I decided it might be useful to have support for those additional languages, and I found it very easy to install them retrospectively: I simply loaded the OPENSTEP-Install-4.2.iso file into the VM’s ‘optical drive’, browsed the CDROM’s contents, selected Upgrader.app and then ‘Workspace’ > ‘File’ > ‘Open as Folder’. I found the language packages (SwedishEssentials.pkg etc.) in the folder ‘NextCD’ > ‘Packages’. I could then select each language package and use ‘Services’ > ‘Open Sesame’ and so on to install it, as explained earlier.

To get sound working in OPENSTEP running in VirtualBox the procedure given in a 2009 tutorial ‘Installation of OPENSTEP 4.2 in VMware 3.0 and VirtualBox‘ miraculously still worked for me:

Audio: Alejandro Diaz Infante (aka astroboy) managed to make the OPENSTEP Sound Blaster driver work under VMWare and VirtualBox.
The solution: use the drivers created by University of Glasgow (Thanks, developer(s) of them, wherever you are, for drivers you never imagined would be so useful in the future).

  1. Download SBSoundMidi.I.b.tar.gz and SBMixer.I.tar.gz
  2. Install SBSoundMidi driver for either Vibra16Cpnp or AWE32pnp. Both work great! (I use the default irq and io, but the second DMA I put it on 7, ’cause it was the detected one when used VMWare to test Windoze. Anyway, I didn’t detect any failure when using the second DMA in its default of 5, so I guess it could be up to you. In VirtualBox I didn’t change any default setting, just select the driver “SoundBlaster 16” in VirtualBox audio setting before installing.
  3. Install SBMixer to have better control of your sound card.

That’s it. Put those audio CD’s and multimedia apps back!

After copying SBSoundMidi.I.b.tar.gz to OPENSTEP I double-clicked on it to unpack it, and then double-clicked on SBSoundMidi.config to install the SoundBlaster 16 drivers. I then navigated to ‘openstep’ > ‘NextAdmin’ > ‘Configure.app’, selected the loudspeaker icon and specified the driver ‘SBSoundMidi driver for SoundBlaster AWE32 PnP (v3.38)‘.

SBMixer works, and OPENSTEP’s Sound Inspector can play .snd files without having to install additional software, although I found that some .snd files would not play completely. TheNeXTSong.snd (16-bit Linear format) which I downloaded from one of the OPENSTEP software repositories on the Web (see links at the end of this post) plays perfectly (and is amusing), but the shorter Welcome-to-the-NeXT-world.snd (8-bit muLaw format) stalls. I did manage to install a couple of audio players (see further down).

The only minor problem that occurs every time you login if the floppy disk drive is empty is a pop-up window with the message ‘The floppy disk is unreadable’. You can just click on ‘Eject’ but, to stop this happening, you can change the boot order in VirtualBox Manager and load one of the OPENSTEP floppy disk image files in the VM’s floppy disk drive (‘Settings…’ > ‘Storage’ > ‘Floppy Drive’ in VirtualBox Manager). Actually, I copied Driver_Floppy.img to Work_Floppy.img, loaded the latter in the VM’s floppy disk drive and I changed the Boot Order from ‘Floppy’|’Optical’|’Hard Disk’ to ‘Hard Disk’|’Optical’|’Floppy’ (‘Settings…’ > ‘System’ > ‘Motherboard’ > ‘Boot Order’ in the VirtualBox Manager). Furthermore, although not essential, I selected Work_Floppy in File Viewer, then in the Workspace menu I selected ‘Disk’ > ‘Initialize…’ and initialised (formatted) the floppy disk. Its icon disappears momentarily from File Viewer, then reappears after it has been formatted.

The command ifconfig on my VM host computer running Lubuntu 18.04 tells me that the IP address of the host machine is 192.168.1.74 (I had previous configured my router to always assign this address to this machine), the netmask is 255.255.255.0 and the broadcast IP address is 192.168.1.255. My router’s Management page in a Web browser has the DHCP network range configured as 192.168.1.64 – 192.168.1.253, so I decided the OPENSTEP VM would have a static IP address of 192.168.1.63. The router’s Management page also told me that the ISP’s Primary DNS IP address is 81.139.57.100 and the Secondary DNS IP address is 81.139.56.100. Therefore, in accordance with the OPENSTEP installation tutorial I followed, I edited the file /etc/hostconfig in OPENSTEP to have the following shell variables:

# /etc/hostconfig
#
# This file sets up shell variables used by the various rc scripts to
# configure the host.  Edit this file instead of rc.boot.
#
# Warning:  This is sourced by /bin/sh.  Make sure there are no spaces
#           on either side of the "=".
#
# There are some special keywords used by rc boot and the programs it
# calls:
#
#       -AUTOMATIC-     Configure automatically
#       -YES-           Turn a feature on
#       -NO-            Leave a feature off or do not configure
#
HOSTNAME=openstep
INETADDR=192.168.1.63
ROUTER=192.168.1.254
IPNETMASK=255.255.255.0
IPBROADCAST=192.168.1.255
YPDOMAIN=-NO-
NETMASTER=-NO-
TIME=-AUTOMATIC-

I also created the file /etc/resolv.conf as specified in the tutorial, containing the following two lines with the ISP’s nameserver IP addresses I found from my router:

nameserver 81.139.57.100
nameserver 81.139.56.100

It was not specified in the tutorial, but to get NFS working later I found it was necessary to edit the file /etc/hosts to comment out the list of IP addresses and to add the hostname I had chosen (openstep) for the OPENSTEP VM plus the IP address (192.168.1.74) and hostname (aspirexc600) of the VM host machine running Lubuntu 18.04:

#
# NOTE: This file is never consulted if NetInfo or Yellow Pages is running.
#
#
# To do anything on the network, you need to assign an address to your
# machine.  This default host table will get you started.  "myhost"
# can be used for the first machine on the network, and client[1-8]
# can be used for subsequent machines.  You must make sure that no two
# machines have the same address.  If you need to add more machines
# just keep adding entries.  Each digit in the four digit number must
# be between 1 and 254 inclusive.
#
#192.42.172.1	myhost
#192.42.172.2	client1
#192.42.172.3	client2
#192.42.172.4	client3
#192.42.172.5	client4
#192.42.172.6	client5
#192.42.172.7	client6
#192.42.172.8	client7
#192.42.172.9	client8
#
# This is the reserved address for the loopback interface.  Don't muck
# with it.
#
127.0.0.1       localhost       openstep
192.168.1.74    aspirexc600

While setting up networking in the VM I also temporarily disabled the firewall in the VM host to make sure the VM host was not interfering in any way with the network connection of the VM, then enabled it again once I was happy it was not causing any problems. Later, when I configured the VM host as an NFS server and the VM as an NFS client, I had to create the appropriate rules for NFS in the VM host’s firewall (see further down).

You will see NetInfo mentioned in the OPENSTEP networking apps. You should ignore NetInfo unless you are going to network a cluster of machines running NEXTSTEP/OPENSTEP, as it is an obsolete NeXT networking system configuration database and we don’t want to use it.

Installation of utilities and applications

After installing the OS neither the ‘me’ account nor the root account are password protected. You can use the OS like this if you wish, but I set up a password for the ‘me’ account by navigating to ‘openstep’ > ‘NextApps’ > Preferences.app and clicking on the padlock icon. Then I logged out and logged in to the root account and did the same to set up a password for the root user. If you want to save a bit of time during installation of applications, you could do this after installing all the packages.

OPENSTEP comes with quite a few utilities, such as Terminal.app, TextEdit.app, Draw.app, Sound.app (possibly useful if your host computer has a microphone socket and you enabled audio input in VirtualBox Manager), PhotoAlbum.app, CDPlayer.app, Webster.app (yes, a full dictionary), Librarian.app, PrintManager.app, Grab.app (to grab snapshots of all or parts of the screen and save them to .tiff files), Preview.app (an image file viewer), Mail.app, and others. You can try these and they are reasonably intuitive so I won’t dwell on them here, instead concentrating on how I installed third-party apps and utilities.

I had to trawl the Web to find packages and applications suitable for OPENSTEP/Mach 4.2 for i386. I find the filenames of the files stored on these Web sites confusing. I think.s‘ in the filename of a compressed file means it contains source code, and ‘.b‘ means it contains binary code, i.e. executable. However, some filenames have ‘.bs‘ but only contain source code, so I could be wrong. Also, I’m not sure what the letters ‘N‘, ‘I‘, ‘H‘ and ‘S‘ represent in these filenames; NeXT (Motorola 68k), Intel, Hewlett-Packard PA-RISC and SPARC, presumably? Some OPENSTEP packages are called ‘fat binaries’ as they contain binaries for several or all the supported CPU types, thus enabling the package to be installed in OPENSTEP on different hardware. So my guess about the letters in the filenames could be correct.

Without a Web browser in OPENSTEP, the easiest way to copy files to the OPENSTEP VM initially is to use the Linux mkisofs command to create an ISO file and then to load it into the VM’s optical drive. For example, let’s say I want to copy the file OpenUp-1.01.tar to the VM, I would type the following on the host machine:

$ mkdir ~/ToCopy
$ cp ~/Downloads/OpenUp-1.01.tar ~/ToCopy
$ mkisofs -o ToCopy.iso ~/ToCopy

I then use the VirtualBox Manager GUI (‘Settings’ > ‘Storage’ > ‘Choose Virtual Optical Disk File…’) to insert the ToCopy.iso file into the VM’s optical drive. OPENSTEP mounts the ‘CDROM’ automatically and it becomes visible in the OPENSTEP File Viewer window. When I click on the CDROM icon a window opens and I see it contains the file openup_1.tar which I can then drag to the Shelf or to another folder directly.

Packages for installation using the OPENSTEP Installer have a ‘.pkg‘ suffix (e.g. ParaSheet.pkg) and are actually a folder, not a file. Applications have a ‘.app‘ suffix (e.g. ParaSheet.app) and are also a folder, not a file. Some of the compressed files I found for OPENSTEP on the Web are tarballs of OPENSTEP packages (e.g. OpenWrite.2.1.8.NIHS.b.tar.gz contains OpenWrite.pkg), others are tarballs of OPENSTEP applications (e.g. mpap.1.0.m.I.b.tar.gz contains mpap.app) which require unpacking but no installation, just copying to a folder. The mkisofs command truncates filenames to the Short Filename format (a.k.a. DOS 8.3 format), so if I had any uncompressed .pkg files, .app files and indeed any other files (.pdf, .mp3 or whatever) to transfer to the VM, I compressed them first as .tar files before creating the .iso file. Even though the .tar filename is truncated to DOS 8.3 by mkisofs, the filenames of the packed files are not.

Installing a package in OPENSTEP 4.2.

a) Installing a package in OPENSTEP 4.2.

Installing a package in OPENSTEP 4.2.

b) Installing a package in OPENSTEP 4.2.

Once you get the hang of installing packages in OPENSTEP, it is actually simple. For example, to install the package ParaSheet.pkg, I drag the .tar file from the CDROM to the Shelf, and from there to the folder /me. I double-click on the .tar file which opens a window showing the ParaSheet.pkg inside. I drag that to the /me folder. Then I select the package, and select ‘Workspace’ > ‘Open Sesame’ > ‘Open As Root’ > ‘Login’ and the Installer GUI opens. I then click on ‘Set…’ to specify the folder into which I want to install the application (e.g. /LocalApps/Office, as I had created the Office folder beforehand using Terminal.app) and then ‘Install’, and the Installer takes care of the rest.

In the case of applications that are not packaged and are just .app folders, I do not need to use the Installer, I just copy the .app folder to the folder I wish (/LocalApps/, /me/LocalApps/ or just /me/).

I found that, as-installed, OPENSTEP 4.2 can unpack .tar files from the GUI but does not have a GUI app for unpacking .tar.gz files, so the first thing I did was to install the OpenUp utility: OpenUp-1.01.m.NI.b.tgz which can be found at http://www.nextcomputers.org/NeXTfiles/Software/OPENSTEP/Apps/Compression_Utilities/ and works very well. Of course, I could have instead unpacked .tar.gz files in the host machine first and copied the .tar files to OPENSTEP using the mkisofs method I explained above, which the OPENSTEP GUI can unpack when I double-click on the .tar file. But OpenUp is well worth installing. After I had installed OpenUp and the OmniWeb browser in OPENSTEP, I was also able to download .tar.gz files directly in OPENSTEP from the various file repositories on the Web (see links at the end of this post) and unpack them in OPENSTEP.

By the way, see the links at the end of this post for user documentation. The OPENSTEP GUI is intuitive but I didn’t realise I could rename files from the GUI by clicking on the filename below the icon to get a cursor and typing directly (just like macOS), and I also didn’t know that I could use the ‘shelf’ at the top of the File Viewer as a temporary place to put copies of files to copy files between folders as an alternative to opening another File Viewer window. I also wondered how to select multiple files in a window when they are not adjacent, since using the mouse to select the group of files is not feasible in that case. It turns out the you hold down the Shift key and click on each file you want to select, which is analogous to holding down the Ctrl key and clicking on each file in Linux. I also found that I can copy a file between two File Viewer windows by clicking on it and holding down the Alt Gr key then dragging across to the other window.

Installation of a Web browser

This is where things start to get trickier. Bear in mind that NEXTSTEP and OPENSTEP were created in the 1980s and 1990s when the Web was in its infancy. As I mentioned earlier, the first Web browser was written on a NeXTcube at CERN, and that machine was the first Web server in existence. The best Web browser I could find for the platform is OmniWeb 3.1 for OPENSTEP. Before installing it, you need to install Omni Frameworks 1998G2. Also, the browser does not support HTTPS, Javascript and Flash out of the box and you have to install plugins. Unfortunately the plugins for these are very flaky, so you are severely limited in which sites and pages you can browse. Note that Netscape Communications created HTTPS in 1994, Netscape Communications and Sun Microsystems released JavaScript in December 1995, and Macromedia released Flash in November 1996. I don’t know if the OmniWeb plugins for HTTPS, JavaScript and Flash for OPENSTEP that I found are the latest or best versions for this version of OmniWeb, but they are what I could find online. JavaScript in Web pages results in a lot of pop-up error messages and made opening pages even less likely to be successful, so in the OmniWeb menu I navigated to ‘Info’ > ‘User Preferences…’ > ‘JavaScript’ and unticked ‘Display panel for errors’. I also navigated to ‘Info’ > ‘Administrator Preferences’ > ‘HTTPS – SSL’ and ticked ‘Enable TLSv1’, which seemed to enable a few HTTPS Web pages to load, at least partially.

You have to install OpenSSL before installing the HTTPS plugin for OmniWeb. I installed the package OpenSSL.0.9.5a.m.NIS.b.tar.gz which I downloaded from http://www.nextcomputers.org/NeXTfiles/Software/OPENSTEP/Apps/Internet/WWW/Web%20Browsers/Omniweb/Plugins/. Then I installed the package HTTPS.1.09b.m.NIS.b.tar.gz from the same site, which installs the file (folder) HTTPS.plugin, which needs to be in the folder /LocalLibrary/Plugins/ (‘NEXTSTEP’ > ‘LocalLibrary’ > ‘Plugins’).

Then I downloaded and installed the two packages JavaScript-OWPlugin-1999-07-20-OSM-NIS.tar.gz (installs JavaScript.plugin) and Flash-OWPlugin-19990621-OSM-NIS.tar (installs Flash.plugin) which also need to be in the folder /LocalLibrary/Plugins/ (‘NEXTSTEP’ > ‘LocalLibrary’ > ‘Plugins’ in the File Viewer). I found these two packages via a BetaArchive post [offer] OmniGroup software (NeXTSTEP, OpenStep & Rhapsody), which has a link to a .rar file at http://www.mediafire.com/file/wzyon54l4dt/OmniGroup.rar/file.

Unfortunately, even with the HTTPS and JavaScript plugins installed, almost all Web pages fail to load in OmniWeb, one exception being https://www.google.com. Old HTTP Web sites do load providing they are simple, but any JavaScript seems to cause a problem.

Installation of a PDF file reader

The best PDF file reader I could find for the platform is OmniPDF 3 for OPENSTEP. If you have not already installed Omni Frameworks, you first need to install Omni Frameworks 1998G2.

Installation of an image viewer

The best (supposedly) image file viewer I could find for the platform is OmniImage 4.0 for OPENSTEP. If you have not already installed Omni Frameworks, you first need to install Omni Frameworks 1998G2. However, according to the file /OmniImage.pkg/OmniImage.info it is a beta release and, in addition to Omni Frameworks, requires ‘Omni Plugins’:

Title OmniImage 4.0 beta for OPENSTEP/Mach 4.2
Version 4.0 beta 4 (1-Oct-1998)
Description This package contains a beta version of OmniImage. This beta release only supports viewing of images, not saving them. This release will not run unless the the Omni Frameworks (version 1998G2) are installed, and will not be fully functional (e.g., images may not be rendered) unless the Omni PlugIns (version 3.0 beta 8) are also installed. This software requires OPENSTEP/Mach 4.2.

I found the file OmniPlugIns-3.0b8-OSM-NIS.pkg.tar.gz in the BetaArchive post mentioned earlier in this post. I downloaded the tarball, created an ISO file containing it, loaded the ISO file in the VM CDROM drive, unpacked the tarball to /me/OmniPlugIns.pkg and installed the package using the OPENSTEP GUI Installer using the procedure explained earlier in this post. The Omni PlugIns were installed in the folder /LocalLibrary/PlugIns/ and I then found that OmniImage can open JPG files, even a 3456×2304 pixel JPG file with the following properties (as reported by the file command in Linux):

JPEG image data, JFIF standard 1.01, resolution (DPI), density 300x300, segment length 16, Exif Standard: [TIFF image data, big-endian, direntries=4, manufacturer=Canon, model=Canon EOS 600D], baseline, precision 8, 3456x2304, frames 3

Installation of wordprocessor and spreadsheet apps

OpenWrite and ParaSheet in use

OpenWrite and ParaSheet in use.

I created the folder /LocalApps/Office/ and installed OpenWrite from OpenWrite.2.1.8.NIHS.b.tar.gz which I downloaded from Index of /OpenStep/Soft/misc/NEXTTOYOU/97.1-Fruehjahr/APPSTOYOU. If you have not already installed it, before installing these apps you need to install Omni Frameworks 1998G2.

In the folder /LocalApps/Office/ I also installed ParaSheet from ParaSheet-1.7.pkg.tar.gz which I downloaded from Index of /NeXTfiles/Software/NEXTSTEP/Apps/Lighthouse_Design/ParaSheet. If you have not already installed Omni Frameworks, before installing these apps you need to install Omni Frameworks 1998G2.

The first time you launch OpenWrite and ParaSheet you will be notified that you cannot use the application until you enter a licence key. Exit the application and use ‘Open Sesame’ (see earlier) to launch the application as root user, and then you well be able to enter the licence. You will find a list of licences for these packages on the Web page Index of /NeXTfiles/Software/NEXTSTEP/Apps/Lighthouse_Design.

Installation of audio players

mpap and MMP audio players in action

mpap and MMP audio players in action.

The only audio players I could find that actually worked (partially) in OPENSTEP are mpap 1.0 (download mpap.1.0.m.I.b.tar.gz) and MMP 2 (download mmp2.I.b.tar.gz). mpap can play some, but not all, of the mp3 files I have, whereas I could not get MMP to play mp3 files at all, although it can play .snd files. MMP can also play MIDI files, but I had to download the Timidity patches instruments.tar.gz (not so easy to find!) and follow the instructions in the MMP Info Panel in order to install the instruments patch file. It works fine! mpap cannot play an mp3 file which the files command in Linux tells me is an ‘Audio file with ID3 version 2.4.0, contains:MPEG ADTS, layer III, v2.5, 32 kbps, 11.025 kHz, Stereo’ but it can play an mp3 file which is an ‘Audio file with ID3 version 2.4.0, contains:MPEG ADTS, layer III, v1, 192 kbps, 44.1 kHz, Stereo’. mpap has a basic playlist feature, but it is not as sophisticated as any of the modern audio players.

Installation of video players

MPLAY and Movie players in action

MPLAY and Movie players in action.

This is where OPENSTEP is severely lacking in comparison to any modern OS; apparently we’re talking 5.5 or 6 frames per second and e.g. 288×224 pixels on NeXT hardware, and no sound. I only managed to find a couple of basic video players, both at Index of /OpenStep/Soft/video/apps: MPlay 3.0 (MPlay.app unpacked from MPlay.3.0.NIHS.b.tar.gz) and Movie 3.0 (Movie3.0 folder unpacked from Movie.3.0.NIHS.bs.tar.gz). MPlay is only designed to play MPEG (.mpg and .mpeg) files, which I found it can do for the old, tiny MPEG files I downloaded from Web repositories of NEXTSTEP/OPENSTEP files. I found that Movie can also only play MPEG files, despite the app’s README file stating it can play (without sound) MPEG, TIFF sequences, ‘QuickTime and other formats’. Movie comes with a couple of demo videos (no audio), the largest of which is hula_full.mpg in the mpeg1video format, consisting of 39 frames of 352×240 pixels, with a desired frame rate of 8 fps which actually plays at between 8 and 9 frames per second in OPENSTEP in the VM, i.e. it plays for around 4 to 5 seconds. In a video player in Linux on my desktop machine it plays for just over 2 seconds at 15 frames per second. These videos and players may have been state-of-the-art in the 1980s and early 1990s, but they certainly are not now!

I could not find an app package to play .avi files. The page I linked to above has a source-code tarball named VideoStreamV1.OSrc.tar.gz for an app named VideoStream, the README of which claims the app can play .avi files, but I have not found an executable package. Anyway, the README file states it cannot play videos with sound, so obviously I didn’t bother trying to install it.

Games

I am not particularly interested in computer games, but a few are installed by default with the OS: Chess.app, Billiards.app and BoinkOut.app (a clone of Breakout). More games for OPENSTEP can be found on the Web (for example at Index of /OpenStep/Soft/). The computer game Doom was originally developed in NEXTSTEP on NeXT computers, and a version for OPENSTEP can be downloaded from the Web, although I have not tried it.

File sharing

NEXTSTEP/OPENSTEP was designed to use NFS (Network File System). However I don’t use NFS in my home network; I use SMB and have a dedicated Linux SMB server which works well with all SMB clients (Linux, Windows and Android) on my home network. Unsurprisingly I could only find early versions of Samba packages for NEXTSTEP and OPENSTEP. I also came across ramba, a Unix clone of Samba later renamed to Sharity-Light. I downloaded them both and briefly tried to get OPENSTEP to connect to my network Samba server. I was unsuccessful, which does not surprise me as the version of Samba for NEXTSTEP/OPENSTEP I found is Version 2.0.7.1 from May 2000, and the obsolete version of rumba I found is Version 0.4 from February 1997. In NEXTSTEP/OPENSTEP the Samba configuration file smb.conf is located in the directory /usr/samba/lib/ rather than /etc/samba/. I did not spend much time trying to get Samba/Rumba working as I assume there would be incompatibility between the early SMB protocol used by Samba V2.0.7.1 / Rumba V0.4 with Samba V4.* running in the Linux SMB server on my network. Perhaps I could have made it work, but I decided to try to make the VM’s host computer (192.168.1.74) a NFS server to see if I could get the VM (192.168.1.63) to access it as a NFS client. The Web page OpenStep on Microsoft Windows PC Emulators states the following, which indicates that NFS works:

Device: Network
OpenStep Configuration: AMD PCnet-32 PCI Ethernet Adapter
VirtualBox Configuration: Bridged Adapter, PCnet-PCI II, Promiscuous Mode All
Observations: This works fine. Using SimpleNetworkStarter I was able to give OpenStep an IP address on my subnet, using my real router and real DNS servers. This allowed OpenStep to be ‘seen’ on the subnet. Standard networking facilities such as FTP and NFS work. It may help to run the a command such as the following from the VirtualBox installation directory, where “OpenStep” is whatever you name the virtual machine and “192.168.1.0” depends on your local subnet:

VBoxManage modifyvm OpenStep --natnet1 "192.168.1.0/24"

As I had named the VM ‘OPENSTEP4.2’ in VirtualBox Manager, I used the following command:

$ VBoxManage modifyvm OPENSTEP4.2 --natnet1 "192.168.1.0/24"

However I doubt this made any difference, because I had set the VM’s network adapter to ‘Bridged Adapter’ in the VirtualBox Manager, not ‘NAT’. I had to select ‘Bridged Adapter’ because I could not get the VM to connect to the network otherwise.

I also made sure the adapter in the VirtualBox Manager is set to ‘PCnet-PCI II (Am79C970A)’ and Promiscuous Mode is set to ‘Allow All’.

In addition to the network configuration notes in the OPENSTEP installation tutorial I mentioned earlier, for information only see the old tutorial ‘NeXTStep/OpenStep Ethernet-Based Network Configuration For Cable Modems, DSL, LANs, Etc…‘.

Anyway, below is what I did to get NFS working. The crucial thing to note is that OPENSTEP 4.2 uses NFSv2. I spent many hours unsuccessfully trying to get NFS working between the NFS server (a machine with IP address 192.168.1.74) and the NFS client (a VM with IP address 192.168.63) until I realised this. The NFS server is running Lubuntu 18.04, which uses NFSv4 by default. Therefore I had to configure the NFS server to use NFSv2 as well. Not only that, but I had to configure NFSv2 to use static ports, because the ports can change randomly in NFSv2 which would stop NFS working if there is a firewall enabled on the host machine.

In the NFS server (Lubuntu 18.04 running on a desktop machine)

N.B. My NFS server is running in Lubuntu 18.04 on a machine with an IP address of 192.168.1.74, and my NFS client is running in OPENSTEP 4.2 on a VM with IP address of 192.168.1.63. Change the IP addresses below to suit your situation.

1. Install the NFS server software

$ sudo apt-get update
$ sudo apt-get install nfs-kernel-server

2. Create a mountpoint for the NFS shared directory

$ sudo mkdir /var/nfs
$ sudo chown nobody:nogroup /var/nfs
$ sudo chmod 777 /var/nfs

3. Configure the NFS export

$ sudo nano /etc/exports

3.1 Choose which of the following types of share you want to have

3.1.1 Less secure:

/home/fitzcarraldo/nfsshare 192.168.1.63(rw,sync,no_root_squash,no_subtree_check)

If ‘no_root_squash‘ is used, remote root users are able to change any file on the shared file system and leave trojaned applications for other users to inadvertently execute.

3.1.2 More secure:

/var/nfs 192.168.1.63(rw,sync,no_subtree_check)

3.2 Update the current table of exports for the NFS server

$ sudo exportfs -a

You can check the current table settings:

$ sudo exportfs -s
/home/fitzcarraldo/nfsshare  192.168.1.63(rw,wdelay,no_root_squash,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)
/var/nfs  192.168.1.63(rw,wdelay,root_squash,no_subtree_check,sec=sys,rw,secure,root_squash,no_all_squash)

If you wanted to clear the table (unexport the shared directories) you would do:

$ sudo exportfs -u 192.168.1.63:/home/fitzcarraldo/nfsshare
$ sudo exportfs -u 192.168.1.63:/var/nfs
$ sudo exportfs -s
$

4. Load the NFSv2 kernel module

If lockd is built as a module (which it is in Lubuntu 18.04), create file /etc/modprobe.d/nfsv2.conf containing the following:

options lockd.nlm_udpport=4001 lockd.nlm_tcpport=4001
$ sudo modprobe nfsv2

If you want to make that permanent so it happens automatically when booting/rebooting add ‘nfsv2‘ (without the quotes) to the file /etc/modules-load.d/modules.conf (which in Lubuntu 18.04 is symlinked to /etc/modules).

5. Configure the NFS server

See ‘How can I make the nfs server support protocol version 2 in Ubuntu 17.10?‘.

Edit /etc/default/nfs-kernel-server to include NFSv2 and to specify static ports:

$ sudo nano /etc/default/nfs-kernel-server
# Number of servers to start up
RPCNFSDCOUNT=8

# Runtime priority of server (see nice(1))
RPCNFSDPRIORITY=0

# Options for rpc.mountd.
# If you have a port-based firewall, you might want to set up
# a fixed port here using the --port option. For more information, 
# see rpc.mountd(8) or http://wiki.debian.org/SecuringNFS
# To disable NFSv4 on the server, specify '--no-nfs-version 4' here
RPCMOUNTDOPTS="--manage-gids -p 32767"
# -p 32767 above added by Fitzcarraldo

# Do you want to start the svcgssd daemon? It is only required for Kerberos
# exports. Valid alternatives are "yes" and "no"; the default is "no".
NEED_SVCGSSD=""

# Options for rpc.svcgssd.
RPCSVCGSSDOPTS=""

# All options below this comment were added by Fitzcarraldo
#
# Options to pass to rpc.statd
# ex. RPCSTATDOPTS="-p 32765 -o 32766"
RPCSTATDOPTS="-p 32765 -o 32766"
#
# Options to pass to rpc.rquotad
# ex. RPCRQUOTADOPTS="-p 32764"
RPCRQUOTADOPTS="-p 32764"
#
RPCNFSDOPTS="--nfs-version 2,3,4 --debug --syslog"
#
# To confirm above mods are in effect after service restart use
#    cat /run/sysconfig/nfs-utils
#  or 
#    service nfs-kernel-server status
#

Edit /etc/default/nfs-common to specify static ports for rpc-statd:

# If you do not set values for the NEED_ options, they will be attempted
# autodetected; this should be sufficient for most people. Valid alternatives
# for the NEED_ options are "yes" and "no".


# Options for rpc.statd.
#   Should rpc.statd listen on a specific port? This is especially useful
#   when you have a port-based firewall. To use a fixed port, set this
#   this variable to a statd argument like: "--port 4000 --outgoing-port 4001".
#   For more information, see rpc.statd(8) or http://wiki.debian.org/SecuringNFS
STATDOPTS="-o 32766 -p 32765"
# -o 32766 -p 32765 above were added by Fitzcarraldo

# Do you want to start the gssd daemon? It is required for Kerberos mounts.
NEED_GSSD=

(I had to edit /etc/default/nfs-common to specify the ports for rpc-statd in STATDOPTS because specifying the ports in RPCSTATDOPTS in /etc/default/nfs-kernel-server did not make the status ports static.)

Edit /etc/sysctl.conf to add a static port mapping for lockd:

$ sudo nano /etc/sysctl.conf
[...]
# All lines below added by Fitzcarraldo
# TCP Port for lock manager
fs.nfs.nlm_tcpport = 4001
# UDP Port for lock manager
fs.nfs.nlm_udpport = 4001

Modify the lockd kernel parameters now during runtime rather than having to reboot:

$ sudo sysctl -p

Note that it is necessary to specify static ports in the configuration files so that tight rules can be added to the firewall in the NFS server.

6. Start the NFS server

Either the sysvinit way, which still works in Lubuntu 18.04:

$ sudo service nfs-kernel-server start

or the systemd way, which also works in Lubuntu 18.04:

sudo systemctl start nfs-kernel-server

If you want, you could enable the service so it starts automatically after the system is rebooted:

$ sudo systemctl enable nfs-kernel-server

7. Start the NSM (Network Status Monitor) daemon

Either the sysvinit way, which still works in Lubuntu 18.04:

$ sudo service rpc-statd start

or the systemd way, which also works in Lubuntu 18.04:

$ sudo systemctl start rpc-statd

If you want, you could enable the service so it starts automatically after the system is rebooted:

$ sudo systemctl enable rpc-statd

8. Check that NFSv2 is running and the ports are the ones specified in the config files

$ rpcinfo -p
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    3   tcp    111  portmapper
    100000    2   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100000    3   udp    111  portmapper
    100000    2   udp    111  portmapper
    100005    1   udp  32767  mountd
    100005    1   tcp  32767  mountd
    100005    2   udp  32767  mountd
    100005    2   tcp  32767  mountd
    100005    3   udp  32767  mountd
    100005    3   tcp  32767  mountd
    100003    2   tcp   2049  nfs
    100003    3   tcp   2049  nfs
    100003    4   tcp   2049  nfs
    100227    2   tcp   2049
    100227    3   tcp   2049
    100003    2   udp   2049  nfs
    100003    3   udp   2049  nfs
    100227    2   udp   2049
    100227    3   udp   2049
    100021    1   udp   4001  nlockmgr
    100021    3   udp   4001  nlockmgr
    100021    4   udp   4001  nlockmgr
    100021    1   tcp   4001  nlockmgr
    100021    3   tcp   4001  nlockmgr
    100021    4   tcp   4001  nlockmgr
    100024    1   udp  32765  status
    100024    1   tcp  32765  status

9. Configure the firewall in Lubuntu 18.04

I used Gufw (LXDE Menu > ‘Preferences’ > ‘Firewall Configuration’) to add the following two UFW rules:

111,2049,4001,32765:32768/udp ALLOW IN 192.168.1.0/24
111,2049,4001,32765:32768/tcp ALLOW IN 192.168.1.0/24

The above rules permit NFSv2 to function consistently because I had configured the NFS ports to be static. If I had not done that the firewall would sometimes stop NFS from working because NFSv2 ports change randomly otherwise.

In OPENSTEP running in the VM

10. Make sure basic networking has been configured

I navigated to ‘openstep’ > ‘NextAdmin’ > ‘SimpleNetworkStartup.app’ and did the following:

  • Unticked ‘Maintain the master copy of network administrative data.’
  • Selected ‘Use the network, but don’t share administrative data.’
  • Entered the Hostname ‘openstep‘ (no quotes) and IP address 192.168.1.63.
  • Clicked on ‘Network Options…’. In the window that opened I did the following:
    • Made sure router IP is set to 192.168.1.254
    • Made sure NIS Domain Name is set to ‘None’
    • Made sure Netmask is set to 255.255.255.0
    • Made sure Broadcast Address is set to 192.168.1.255
    • ‘Limit access to local NetInfo data to the local network’ is unticked.
    • Clicked on ‘Set’.
  • Clicked on ‘Configure’.

11. Create the shared NFS director[y,ies]

N.B. I could probably have created the directory /mnt/nfs/nfsshare and/or /mnt/nfs/var/nfs (whichever you chose to create — see 3.1 above) using ‘openstep’ > ‘NextAdmin’ > ‘NFSManager.app’ instead of using the command line, but I opened a Terminal window in OPENSTEP and did the following:

openstep> su
openstep:1# mkdir /mnt
openstep:2# mkdir /mnt/nfs
openstep:3# mkdir /mnt/nfs/nfsshare
openstep:4# mkdir /mnt/nfs/var
openstep:5# mkdir /mnt/nfs/var/nfs

12. Mount the NFS share(s)

openstep:6# mount 192.168.1.74:/home/fitzcarraldo/nfsshare /mnt/nfs/nfsshare
openstep:7# mount 192.168.1.74:/var/nfs /mnt/nfs/var/nfs

Use the df command to check they are mounted correctly:

openstep:8# df

13. Test the shared director[y,ies]

In Lubuntu on the machine with hostname ‘aspirexc600‘, copy a file into /var/nfs/ (or /home/fitzcarraldo/nfsshare/). You should see it appear in /mnt/nfs/var/nfs/ (or /mnt/nfs/nfsshare/) in OPENSTEP in the VM with hostname ‘openstep‘.

In OPENSTEP on the VM with hostname ‘openstep‘, copy a file into /mnt/nfs/var/nfs/ (not /mnt/nfs/nfsshare/, as that will not be allowed). You should see it appear in /var/nfs/ in Linux in the machine with hostname ‘aspirexc600‘.

In Lubuntu on the machine with hostname ‘aspirexc600‘, delete the file in /var/nfs/ and you should see it removed from /mnt/nfs/var/nfs/ in OPENSTEP on the VM with hostname ‘openstep‘.

In Lubuntu on the machine with hostname ‘aspirexc600‘, delete the file in /home/fitzcarraldo/nfsshare/ and you should see it removed from /mnt/nfs/nfsshare/ in OPENSTEP on the VM with hostname ‘openstep‘.

14. If you later want to unmount the NFS shared folder(s)

openstep:9# umount /mnt/nfs/nfsshare
openstep:10# umount /mnt/nfs/var/nfs

15. If you want OPENSTEP to mount the NFS shared folder(s) automatically when it boots

I was unable to get OPENSTEP to mount NFS shared folders automatically at boot by adding the appropriate lines in /etc/fstab, but OPENSTEP does mount them automatically if I add the mount commands to /etc/rc.local like so:

#!/bin/sh -u
#
# This script is for augmenting the standard system startup commands. It is 
# executed automatically by the system during boot up. 
#
# Copyright (C) 1993 by NeXT Computer, Inc.  All rights reserved.
#
# In its released form, this script does nothing. You may customize
# it as you wish.
#

fbshow -B -I "Starting local services" -z 92

# Read in configuration information
. /etc/hostconfig

# (echo -n 'local daemons:')                                    >/dev/console
#
# Run your own commands here
mount 192.168.1.74:/var/nfs /mnt/nfs/var/nfs
mount 192.168.1.74:/home/fitzcarraldo/nfsshare /mnt/nfs/nfsshare
#
# (echo '.')                                                    >/dev/console

File sharing: Summary

So, I managed to get NFS working, albeit not using OPENSTEP’s NFSManager.app tool. Had I known more about OPENSTEP networking I probably could have used the OPENSTEP GUI utilities to configure NFS, but at least I have proved it is possible to copy files to and from an NFS server (which happens to be the host machine of the VM) running Lubuntu 18.04 and the VM running OPENSTEP 4.2. Mind you, NFSv2 is old. NFSv4 would be the protocol to use had OPENSTEP supported it. Also, bear in mind that NFSv2 cannot encrypt the connection, so it is not secure. Another reason to have a good firewall enabled in the VirtualBox host machine and in my router too.

Conclusions

I have had fun installing and tinkering with OPENSTEP and its applications over the last few days. Getting file sharing to work was by far the most difficult part, but I got there in the end once I had discovered OPENSTEP only supports NFSv2. It is a pity OPENSTEP and the applications for it have not been developed for many years and are all obsolete. If development of OPENSTEP drivers, networking software, productivity applications and multimedia applications had continued, the OS itself would still have been perfectly usable on modern hardware, albeit not as straightforward to use as any of the main Desktop Environments in Linux. But the OS still feels quite modern; it was definitely ahead of its time. Tinkering with OPENSTEP 4.2 has given me a new respect for Steve Jobs, for the talented hardware and software engineers in the NeXT company, and indeed for Mac OS X and macOS. The choice of Unix for NEXTSTEP/OPENSTEP was truely inspired.

In this blog post I have not covered the sophisticated development tools for NEXTSTEP/OPENSTEP, which were also way ahead of their time. I’ll leave you to read the articles, documents and videos available on the Web about the development tools.

Please comment below if you notice any errors or omissions in this post, or if you know a better way of doing something in OPENSTEP, or you know of newer versions of the OPENSTEP software than the versions I have mentioned. I’d also be interested to hear from anyone who has a NeXT machine and/or is still using one; let me know what you have and how you’re using it.

Useful links

These are just a few of the many Web pages and sites I browsed when installing OPENSTEP 4.2 and looking for applications and ways to get various things to work.

Documentation

Software repositories

Sometimes differences between NEXTSTEP and OPENSTEP may mean a NETSCAPE application cannot be installed in OPENSTEP or, if it can, may not work. Furthermore, be aware that different revisions of the same application/utility exist online, so you need to try and find the latest revisions.

Replacing the KDE Plasma widget ‘Thermal Monitor’ with ‘Kargos’ in Gentoo Linux

The KDE Plasma widget Thermal Monitor has not been working correctly in my Gentoo Linux installations for quite some time. I notice Thermal Monitor’s repository has not been updated for a couple of years, despite several new versions of KDE Plasma having been released. Perhaps that is the reason.

On my laptop running the Stable Branch of Gentoo Linux, Thermal Monitor displays the GPU and HDD temperatures automatically but CPU temperatures were only displayed if I right-clicked on the widget and selected ‘Reload Temperature Sources’. I managed to get the widget to display the CPU temperatures automatically by editing the file ~/.local/share/plasma/plasmoids/org.kde.thermalMonitor/contents/ui/main.qml and commenting out a line as shown in the file excerpt below:

[...]
        onSourceAdded: {

            if (source.indexOf(lmSensorsStart) === 0 || source.indexOf(acpiStart) === 0) {
/*
 *                systemmonitorAvailableSources.push(source)
 */
                var staIndex = systemmonitorSourcesToAdd.indexOf(source)
                if (staIndex > -1) {
                    addToSourcesOfDatasource(systemmonitorDS, source)
                    systemmonitorSourcesToAdd.splice(staIndex, 1)
                }

            }

        }
[...]

The above modification is suggested in a comment to Issue #53 in the widget’s repository.

However, the above-mentioned edit does not fix Thermal Monitor on my laptop running the Testing Branch of Gentoo Linux, and Thermal Monitor no longer displays the GPU temperature either. Actually, the CPU’s four core temperatures and the GPU temperature are no longer listed in the Thermal Monitor configuration window, only a single CPU temperature. Not surprisingly, none of the suggested changes to the file ~/.local/share/plasma/plasmoids/org.kde.thermalMonitor/contents/ui/main.qml that I found in Web searches made a difference. However, while researching the problem I came across a Manjaro Forums post by user bogdancovaciu about the Kargos Plasma widget, a KDE Plasma port of GNOME Argos and OSX BitBar. Kargos enables you to create a Plasma widget that runs your own script, which can be written in any language, providing its output adheres to a specified format. I also found a repository named k-argos-plugins containing further example scripts for Kargos. As none of the solutions suggested for Thermal Monitor in that Manjaro thread worked for me, I decided to try the Kargos widget instead. It works a treat.

kargos widget on KDE Plasma Panel

kargos widget on KDE Plasma Panel of my Compal NBLB2 laptop

Below I explain what I did to install and configure the Kargos widget on my KDE Panel in Gentoo Linux (see screenshot). The packages lm-sensors and hddtemp were already installed in my case, but if they had not been, I would have needed to install and configure them, so I have included those steps below.

1. Install and configure lm-sensors

root # emerge lm-sensors
root # rc-update add lm_sensors default
root # sensors-detect

In my case sensors-detect created the file /etc/modules-load.d/lm_sensors.conf containing only the following:

# Generated by sensors-detect on Sun Oct 27 03:07:08 2019
coretemp

2. Start lm-sensors now, rather than rebooting

root # /etc/init.d/lm_sensors start

3. I wanted to use the nc command in my shell script for Kargos, so I installed its package

root # emerge netcat

4. Install and configure hddtemp

root # emerge hddtemp
root # rc-update add hddtemp default

Specify in the config file /etc/conf.d/hddtemp which drives to check:

# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

# the hddtemp executable
HDDTEMP_EXEC=/usr/sbin/hddtemp

# various options to pass to the daemon
HDDTEMP_OPTS="--listen=127.0.0.1"

# a list of drives to check
HDDTEMP_DRIVES="/dev/sda"

5. Start hddtemp now, rather than rebooting

root # /etc/init.d/hddtemp start

6. Install Kargos

On the KDE Plasma Desktop, click on the ‘Desktop’ menu icon (the three horizontal lines in the top right corner of the Desktop) and select: ‘Unlock Widgets’ > ‘Add Widgets…’ > ‘Get New Widgets…’ > ‘Download New Plasma Widgets’. Search for, and install, ‘kargos’ widget.

7. Create the Bash script ~/temperatures.3s.sh containing the following:

#!/bin/bash
temp=$(sensors | grep -oP 'Core.*?\+\K[0-9.]+')
temp0=$(sensors | grep 'Core 0' | cut -c '16-17')
temp1=$(sensors | grep 'Core 1' | cut -c '16-17')
temp2=$(sensors | grep 'Core 2' | cut -c '16-17')
temp3=$(sensors | grep 'Core 3' | cut -c '16-17')
hdd_temp=$(nc localhost 7634 | cut -c '33-34')
gpu_temp=$(sensors | grep -A 2 'radeon' | grep 'temp1' | cut -c '16-17')
echo "<br><font size='1'>CPU1&nbsp;&nbsp;CPU2&nbsp;&nbsp;CPU3&nbsp;&nbsp;CPU4&nbsp;&nbsp;GPU&nbsp;&nbsp;HDD</font><br>${temp0%%.*}°&nbsp;&nbsp;${temp1%%.*}°&nbsp;&nbsp;${temp2%%.*}°&nbsp;&nbsp;${temp3%%.*}°&nbsp;${gpu_temp}°&nbsp;${hdd_temp}°| font=Hack-Regular size=10"
# Uncomment the lines below if you want to be able to click on the kargos widget and display a pop-up TOP
#echo "---"
#TOP_OUTPUT=$(top -b -n 1 | head -n 20 | awk 1 ORS="\\\\n")
#echo "$TOP_OUTPUT | font=monospace iconName=htop"

The script above is specifically for the temperature sensors in my Clevo NBLB2 laptop. To find out which temperatures are available, and which characters to extract, use the following command:

root # sensors

Don’t forget to make the script executable:

user $ chmod +x ~/temperatures.3s.sh

Note that the ‘.3s‘ in the script name is optional but, if included, will override the kargos configuration (see further on) and run the script every 3 seconds. I could have specified another frequency, such as ‘.5s‘ or whatever.

8. Add the kargos widget to the KDE Panel.

9. Right-click on the kargos widget on the KDE Panel and select ‘Configure kargos…’.

10. Configure the kargos widget

In the first box in the configuration window, enter the full path of the script:

/home/fitzcarraldo/temperatures.3s.sh

In the second box leave ‘Interval in seconds’ as ‘1‘. This is overridden anyway if the script filename includes the interval.

In the third box leave ‘Rotation delay in seconds’ as ‘6‘.

On the KDE Plasma Desktop, click on the Desktop menu icon (three horizontal lines) and select: ‘Lock Widgets’.

11. Depending on the font configuration for the KDE Desktop, it may be necessary to edit the Bash script ~/temperatures.3s.sh to change the font name or size, the number of non-breaking spaces between the names displayed on the top line, and the number of non-breaking spaces between the temperature values displayed on the bottom line.

Creating a RAID of USB pendrives in Linux

USB hub and USB pendrives used as RAID10 with my laptop

USB hub and pendrives used as RAID10 with my laptop.

If you’re not familiar with the RAID (Redundant Array of Inexpensive Disks) concept and the different types of array, the article ‘RAID 0, RAID 1, RAID 5, RAID 10 Explained with Diagrams‘ gives a quick summary (and links to another article ‘RAID 2, RAID 3, RAID 4, RAID 6 Explained with Diagram‘). Another helpful article is ‘RAID Levels Explained‘.

A few years ago I came across a YouTube video by a Mac user, titled ‘Use a bunch of USB Flash drives in a RAID array‘. Purely out of interest he had experimented with creating RAIDs using USB pendrives (also known as ‘USB flash drives’ or ‘USB memory sticks’). The creation of a RAID using USB pendrives for his Apple Macs was very easy, and, since then, I had wanted to try this using one of my laptops running Linux, just to satisfy my curiosity. I have previously created software RAIDs in a Linux server using internal 3.5-inch HDDs, for the root, home and swap partitions, and for file storage partitions for a Cloud server and NAS. However, I had never created a RAID using external USB drives. This week I happened to have a spare four-port USB 3.0 hub and four old 4GB USB 2.0 pendrives, so I finally got the chance to create a RAID with USB pendrives (see photo). I decided to use my main laptop, which has Gentoo Linux with OpenRC, elogind, eudev and KDE installed. That installation does not have an initramfs so I did not need to rebuild an initramfs to assemble the RAID. Anyway, early assembly of a RAID by an initramfs would only be needed if the RAID were being used to hold the directories required by the OS (the root partition, for example). As my RAID would be pluggable external storage, I wanted to mount it manually rather than adding it to /etc/fstab to be mounted automatically at boot. As I had not used a RAID on this laptop before, I had not enabled the RAID drivers in the kernel configuration, so I needed to do that and rebuild the kernel. I opted to make the RAID drivers kernel modules rather than built into the kernel, so that I could load only the relevant module for whichever type of RAID I wished to create.

I had to decide which filesystem to use in the RAID. I have always used ext4 in my RAIDs using HDDs. However, F2FS is an interesting filesystem developed by Samsung for devices using flash memory, such as SD cards, USB pendrives and SSDs. So I decided to format the pendrives to use F2FS, and create an F2FS RAID. As I had not used F2FS previously on this laptop, I had not enabled the F2FS driver in the kernel configuration, so I enabled the F2FS driver in the kernel at the same time as I enabled the RAID drivers. As with the RAID drivers, I opted to make the F2FS driver a kernel module rather than built into the kernel, so that I could load it and unload it whenever I wanted.

Not only did it turn out to be easy to create a RAID using USB pendrives, I found that the Linux RAID module gets loaded automatically when I connect the USB hub. Furthermore the RAID is recognised by KDE and listed under ‘Places’ in the Dolphin file manager’s windows, which I can click on to mount and unmount the RAID. So I did not even need to configure the OS to load the RAID module at boot (the OS does not load the module automatically at boot if the hub is not connected).

DigitalOcean produced a good tutorial on creating RAIDs in Ubuntu: ‘How To Create RAID Arrays with mdadm on Ubuntu 16.04‘. The procedure is essentially the same in Gentoo Linux, the only differences being the path of the mdadm.conf file and the method of updating an initramfs (which I did not need to do anyway in this particular installation).

As I had four spare USB pendrives and a four-port hub, I decided to create a RAID10 array. Below is a summary of the steps I took.

1. I rebuilt the kernel in order to build the RAID and F2FS modules. The relevant kernel configuration parameters I set are shown below:

root # grep RAID /usr/src/linux/.config | grep -v "#"
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
CONFIG_MD_RAID10=m
CONFIG_MD_RAID456=m
CONFIG_ASYNC_RAID6_RECOV=m
CONFIG_RAID6_PQ=m
root # grep F2FS /usr/src/linux/.config | grep -v "#"
CONFIG_F2FS_FS=m
CONFIG_F2FS_STAT_FS=y
CONFIG_F2FS_FS_XATTR=y
CONFIG_F2FS_FS_POSIX_ACL=y
root # uname -a
Linux clevow230ss 4.19.72-gentoo #2 SMP Tue Oct 15 01:36:57 BST 2019 x86_64 Intel(R) Core(TM) i7-4810MQ CPU @ 2.80GHz GenuineIntel GNU/Linux

2. I installed the mdadm tool:

root # eix -I mdadm
[I] sys-fs/mdadm
     Available versions:  4.1^t {static}
     Installed versions:  4.1^t(01:52:17 15/10/19)(-static)
     Homepage:            https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/
     Description:         Tool for running RAID systems - replacement for the raidtools

3. I installed the F2FS tools:

root # eix -I f2fs
[I] sys-fs/f2fs-tools
     Available versions:  1.10.0(0/4) 1.11.0-r1(0/5) 1.12.0-r1(0/6) ~1.13.0(0/6) {selinux}
     Installed versions:  1.12.0-r1(0/6)(02:05:17 15/10/19)(-selinux)
     Homepage:            https://git.kernel.org/cgit/linux/kernel/git/jaegeuk/f2fs-tools.git/about/
     Description:         Tools for Flash-Friendly File System (F2FS)

4. I rebooted the laptop.

5. The f2fs module was not loaded automatically, therefore I loaded it manually and edited /etc/conf.d/modules to add the module name so that it would be loaded automatically in future:

root # modprobe f2fs
root # lsmod | grep f2fs
f2fs                  466944  0
root # nano /etc/conf.d/modules
root # grep ^modules /etc/conf.d/modules
modules="fuse bnep rfcomm hidp uvcvideo cifs mmc_block snd-seq-midi iptable_raw xt_CT uinput f2fs"

6. I plugged the four USB pendrives into the USB hub, and connected the hub to the laptop.

7. I launched GParted, deleted the existing partition on each pendrive (three had been formatted as FAT32, one as exFAT), reformatted them individually as F2FS and gave them each a label (USBPD01 to USBPD04). I could have done all that from the command line but it is easier using GParted, and I like an easy life.

Note that the mdadm USE flag in Gentoo Linux needed to be set when GParted was merged, so GParted would need to be re-merged with USE="mdadm" if that is not the case. Furthermore, GParted will only include F2FS in the list of available filesystems if either the F2FS module is loaded or the F2FS driver has been built into the kernel.

8. I ascertained the names of the USB pendrives:

root # lsblk -o NAME,SIZE,FSTYPE,TYPE,MOUNTPOINT
NAME     SIZE FSTYPE TYPE MOUNTPOINT
sda    698.7G        disk
├─sda1   128M ext2   part
├─sda2    16G swap   part [SWAP]
├─sda5   128G ext4   part /
├─sda6   256G ext4   part /home
└─sda7 298.5G ntfs   part /media/NTFS
sdb      3.8G        disk
└─sdb1   3.8G f2fs   part
sdc      3.8G        disk
└─sdc1   3.8G f2fs   part
sdd      3.8G        disk
└─sdd1   3.8G f2fs   part
sde      3.8G        disk
└─sde1   3.8G f2fs   part

As you can see above, the four USB pendrives are sdb to sde.

9. I loaded the raid10 module:

root # modprobe raid10
root # lsmod | grep raid
raid10                 57344  1

10. I created the RAID10 array:

root # mdadm --create --verbose /dev/md0 --level=10 --raid-devices=4 /dev/sdb /dev/sdc /dev/sdd /dev/sde
mdadm: layout defaults to n2
mdadm: layout defaults to n2
mdadm: chunk size defaults to 512K
mdadm: partition table exists on /dev/sdb
mdadm: partition table exists on /dev/sdb but will be lost or
       meaningless after creating array
mdadm: partition table exists on /dev/sdc
mdadm: partition table exists on /dev/sdc but will be lost or
       meaningless after creating array
mdadm: partition table exists on /dev/sdd
mdadm: partition table exists on /dev/sdd but will be lost or
       meaningless after creating array
mdadm: partition table exists on /dev/sde
mdadm: partition table exists on /dev/sde but will be lost or
       meaningless after creating array
mdadm: size set to 3913728K
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.

It takes a while for the RAID to be created, so I checked progress periodically as follows:

root # cat /proc/mdstat
Personalities : [raid10]
md0 : active raid10 sde[3] sdd[2] sdc[1] sdb[0]
      7827456 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      [>....................]  resync =  2.8% (222272/7827456) finish=23.8min speed=5308K/sec
      
unused devices: <none>
root # cat /proc/mdstat
Personalities : [raid10]
md0 : active raid10 sde[3] sdd[2] sdc[1] sdb[0]
      7827456 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      [========>............]  resync = 44.0% (3449856/7827456) finish=12.9min speed=5637K/sec
      
unused devices: <none>
root # cat /proc/mdstat
Personalities : [raid10]
md0 : active raid10 sde[3] sdd[2] sdc[1] sdb[0]
      7827456 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      [==============>......]  resync = 74.0% (5797760/7827456) finish=5.9min speed=5698K/sec
      
unused devices: <none>
root # cat /proc/mdstat
Personalities : [raid10]
md0 : active raid10 sde[3] sdd[2] sdc[1] sdb[0]
      7827456 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      
unused devices: <none>

11. I formatted the RAID:

root # sudo mkfs.f2fs -f /dev/md0

        F2FS-tools: mkfs.f2fs Ver: 1.12.0 (2018-11-12)

Info: Disable heap-based policy
Info: Debug level = 0
Info: Trim is enabled
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 15654912 (7644 MB)
Info: zone aligned segment0 blkaddr: 512
Info: format version with
  "Linux version 4.19.72-gentoo (root@clevow230ss) (gcc version 8.3.0 (Gentoo 8.3.0-r1 p1.1)) #2 SMP Tue Oct 15 01:36:57 BST 2019"
Info: [/dev/md0] Discarding device
Info: This device doesn't support BLKSECDISCARD
Info: This device doesn't support BLKDISCARD
Info: Overprovision ratio = 2.300%
Info: Overprovision segments = 179 (GC reserved = 94)
Info: format successful

The option ‘-f‘ forces mkfs to overwrite any existing filesystem. (I believe the same option is ‘-F‘ in Ubuntu, rather than ‘-f‘.)

12. I created a mount point so I could mount the RAID from the command line if I wanted:

root # mkdir -p /mnt/md0

13. I mounted the RAID from the command line and checked its size. In the case of RAID10 I would expect the size to be double the size of one of the formatted USB pendrives, i.e. approximtely 2 x 3.8GB = 7.6GB):

root # mount /dev/md0 /mnt/md0
root # df -h -x devtmpfs -x tmpfs
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       126G   36G   84G  31% /
/dev/sda6       252G  137G  103G  57% /home
/dev/sda7       299G  257G   43G  86% /media/NTFS
/dev/md0        7.5G  419M  7.1G   6% /mnt/md0
root # blkid | grep -v sda
/dev/md0: UUID="d565c117-37e0-48eb-b635-a2fe70b83272" TYPE="f2fs"
/dev/sdb: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="45a488a0-5126-0b95-0c28-eb1f743f77c7" LABEL="clevow230ss:0" TYPE="linux_raid_member"
/dev/sdc: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="ef7de228-cf4d-c6bf-c74a-462a0e27f8bd" LABEL="clevow230ss:0" TYPE="linux_raid_member"
/dev/sdd: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="b5dd5c41-3ab2-fa38-bd28-0b965883775c" LABEL="clevow230ss:0" TYPE="linux_raid_member"
/dev/sde: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="16149e7e-5a96-ece6-65ba-25721bcee49f" LABEL="clevow230ss:0" TYPE="linux_raid_member"

So /dev/md0 looked correct.

14. I checked that nothing was already configured in mdadm.conf and added the array’s details to it:

root # grep -v "#" /etc/mdadm.conf
root # mdadm --detail --scan | sudo tee -a /etc/mdadm.conf
ARRAY /dev/md0 metadata=1.2 name=clevow230ss:0 UUID=d1288120:a1614809:3e89bb5f:967df69b
root # grep -v "#" /etc/mdadm.conf
ARRAY /dev/md0 metadata=1.2 name=clevow230ss:0 UUID=d1288120:a1614809:3e89bb5f:967df69b

15. As the RAID will have only a partition for file storage, and as the RAID array will not always be connected to the laptop, it does not need to be assembled automatically early during boot, so there is no need to add mdadm.conf to an initramfs (which this laptop does not have anyway) and no need to specify /dev/md0 in /etc/fstab to be mounted at boot.

16. I left the USB hub connected to the laptop and rebooted.

17. I checked that the modules were loaded at boot:

root # lsmod | grep raid
raid10                 57344  1
root # lsmod | grep f2fs
f2fs                  466944  0

18. I checked that the RAID had been assembled correctly at boot:

root # blkid | grep -v sda
/dev/sdb: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="45a488a0-5126-0b95-0c28-eb1f743f77c7" LABEL="clevow230ss:0" TYPE="linux_raid_member"
/dev/sdc: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="ef7de228-cf4d-c6bf-c74a-462a0e27f8bd" LABEL="clevow230ss:0" TYPE="linux_raid_member"
/dev/sdd: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="b5dd5c41-3ab2-fa38-bd28-0b965883775c" LABEL="clevow230ss:0" TYPE="linux_raid_member"
/dev/md0: UUID="d565c117-37e0-48eb-b635-a2fe70b83272" TYPE="f2fs"
/dev/sde: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="16149e7e-5a96-ece6-65ba-25721bcee49f" LABEL="clevow230ss:0" TYPE="linux_raid_member"

19. I rebooted a few times with and without the USB hub connected. The module raid10 only gets loaded if the USB hub is connected. If I reboot without the hub connected, raid10 is no longer loaded automatically at boot. If I plug in the hub after the laptop has booted, raid10 gets loaded and the RAID array is recognised by the OS.

20. I mounted the RAID from the command line and copied a file to it as root user:

root # mount /dev/md0 /mnt/md0
root # ls -la /mnt/md0
total 8
drwxr-xr-x 2 root root 4096 Oct 15 07:40 .
drwxr-xr-x 7 root root 4096 Oct 15 07:42 ..
root # cp ./Paper_sheet_sizes.png /mnt/md0
root # ls -la /mnt/md0
total 268
drwxr-xr-x 2 root root   4096 Oct 15 08:07 .
drwxr-xr-x 7 root root   4096 Oct 15 07:42 ..
-rw-r--r-- 1 root root 265760 Oct 15 08:07 Paper_sheet_sizes.png
root # umount /dev/md0
root # ls -la /mnt/md0
total 8
drwxr-xr-x 2 root root 4096 Oct 15 07:42 .
drwxr-xr-x 7 root root 4096 Oct 15 07:42 ..

However, /mnt/md0/ is owned by the root user, so user fitzcarraldo cannot copy files into it. Therefore I changed the ownership:

root # mount /dev/md0 /mnt/md0
root # ls -la /mnt/
total 28
drwxr-xr-x  7 root root 4096 Oct 15 07:42 .
drwxr-xr-x 22 root root 4096 Oct  6 08:31 ..
-rw-r--r--  1 root root    0 Apr  9  2015 .keep
drwxr-xr-x  2 root root 4096 Apr 19  2015 cdrom
drwxr-xr-x  2 root root 4096 Jan 16  2017 floppy
drwxr-xr-x  2 root root 4096 Oct 15 08:07 md0
drwxr-xr-x  2 root root 4096 Apr 17  2015 pendrive
drwxr-xr-x  2 root root 4096 Mar 18  2016 usbstick
root # chown fitzcarraldo:fitzcarraldo /mnt/md0
root # ls -la /mnt/
total 28
drwxr-xr-x  7 root         root         4096 Oct 15 07:42 .
drwxr-xr-x 22 root         root         4096 Oct  6 08:31 ..
-rw-r--r--  1 root         root            0 Apr  9  2015 .keep
drwxr-xr-x  2 root         root         4096 Apr 19  2015 cdrom
drwxr-xr-x  2 root         root         4096 Jan 16  2017 floppy
drwxr-xr-x  2 fitzcarraldo fitzcarraldo 4096 Oct 15 08:07 md0
drwxr-xr-x  2 root         root         4096 Apr 17  2015 pendrive
drwxr-xr-x  2 root         root         4096 Mar 18  2016 usbstick
root # umount /dev/md0

21. ‘Places’ in Dolphin shows /mnt/md0 as ‘7.5 GiB Hard Drive’.

22. I can still mount the RAID from the command line:

root # mount /dev/md0 /mnt/md0
root # df -h /dev/md0
Filesystem      Size  Used Avail Use% Mounted on
/dev/md0        7.5G  420M  7.1G   6% /mnt/md0
root # umount /dev/md0

23. If I want to use the RAID in KDE I must use Dolphin to mount it, not mount it from the command line. To do this I click on the RAID ‘7.5 GiB Hard Drive’ listed under ‘Places’, and a window pop-ups prompting me to enter the root user’s password.

If I mount /dev/md0 via Dolphin instead of via the command line, KDE mounts it on a different directory:

root # df -h /run/media/fitzcarraldo/d565c117-37e0-48eb-b635-a2fe70b83272/
Filesystem      Size  Used Avail Use% Mounted on
/dev/md0        7.5G  420M  7.1G   6% /run/media/fitzcarraldo/d565c117-37e0-48eb-b635-a2fe70b83272

If I want to unmount it, I right-click on the RAID in ‘Places’ and select ‘Unmount’ in the right-click menu. Once it has been unmounted, I can unplug the hub from the laptop. If I plug the hub back into the laptop, the RAID is detected and can be mounted as usual.

So, it works! A USB hub and pendrives are a handy way to:

  • experiment with creating the various types of RAID;
  • compare the capacity of the RAID with the capacity of the USB pendrives used;
  • measure the time to write and read a large file to/from the RAID and compare those times with the time to write and read the same file to/from a single USB pendrive of the same model.

50th anniversary of Apollo 11

I recall watching on live TV in 1969 Neil Armstrong stepping onto the Moon for the first time. I still think the Apollo programme is mankind’s greatest technological achievement to date, especially taking into account the state of the art in the 1960s, albeit massive funding, around 400,000 professionals and some 5,000 companies working on the project helped immensely.

If you have a technical background and are interested in learning a bit about the technical aspects of the equipment and the mission, I can recommend W. David Woods’ book ‘How Apollo Flew to the Moon’. I found the sections on guidance particularly interesting. The book even addresses eating, ablution, urinating, defecating and waste disposal on the journey.

On the 40th anniversary of Apollo 11 I wrote a short post in the Sabayon Linux forums on using Audacity to analyse the recording of Neil Armstrong’s famous “One small step”, which I refreshed in this blog for the 42nd anniversary (see One small step for [a] man… revisited using Audacity).

To mark the 50th anniversary of the launch of Apollo 11, on 16 July this year I went to the cinema to watch the Todd Douglas Millar’s documentary ‘Apollo 11’ (see Apollo 11 [Official Trailer]), which has received good reviews due to its use of 65mm and 70mm footage. Actually, although I very much enjoyed it, quite a lot of the footage used is not 65mm or 70mm, and I found the soundtrack too loud, often making it difficult for me to make out what the controllers and astronauts said. Anyway, if you have not been to see it, I can still recommend it.

This week the UK TV channel BBC Four showed the US PBS (Public Broadcasting Service) excellent multi-part documentary ‘Chasing the Moon‘, which I found riveting. In fact, I much preferred it to Todd Douglas Millar’s ‘Apollo 11’, although the two documentaries are different animals and not really directly comparable. Anyway, if you are interested in the US-USSR space race, the internal politics behind the Apollo programme, and the Mercury, Gemini and Apollo programmes, I can thoroughly recommend ‘Chasing the Moon’, which is to be released shortly on DVD if you do not have the chance to catch it on TV or to stream it. I found the comments by Sergei Krushchev (the son of Nikita Khrushchev) particularly interesting, especially his mention about the hushed-up death of a Soviet cosmonaut in a fire during a test with a pure-oxygen environment prior to the Apollo 1 accident. If the Americans had known about this, it might have prevented the equally gruesome deaths of Grissom, White, and Chaffee in 1967.

YouTube is a gold mine if you are interested in old and newer films, documentaries and vlogs on the Apollo programme. There are hundreds of videos about it. If you are a computer buff, the videos on the AGC (Apollo Guidance Computer) are fascinating. The recent series of videos on the restoration to working condition of a privately-owned scrapped AGC are fascinating. Below are a few of the documentaries and videos I have watched this week to celebrate the 50th anniversary of Apollo 11. I have included links to a couple of the videos in the above-mentioned series on restoration of an AGC; you will be able to find the others in the series if you are interested.

  1. Chasing the Moon
  2. Spacecraft Films The Mighty Saturns Part I The Early Saturns
  3. Spacecraft Films The Mighty Saturns Part II The Saturn V
  4. Moon Machines: Command Module (2/6)
  5. Moon Machines: Navigation Computer (3/6)
  6. Moon Machines: Lunar Module (4/6)
  7. MIT Science Reporter – Computer for Apollo (1965)
  8. MIT Science Reporter – Landing on the Moon (1966)
  9. MIT Science Reporter – Returning from the Moon (1966)
  10. MIT Science Reporter – Food For Space Travelers (1966)
  11. The Real Story Behind the Apollo 11 Computer Error | WSJ
  12. The Journeys of Apollo
  13. The Apollo 4 Mission (1967)
  14. The Flight Of Apollo 7 (1968)
  15. Apollo 8 – Go For TLI (1969)
  16. Apollo 10 – To Sort Out The Unknowns (1969)
  17. Apollo 11 Saturn V Launch Camera E-8
  18. Restored Apollo 11 Moonwalk – Original NASA EVA Mission Video – Walking on the Moon
  19. Moon in Google Earth – Apollo 11 Landing
  20. Hear Buzz Aldrin tell the story of the first moon landing
  21. NASA: Moon Landing – Apollo 11 Descent Film and LRO [Lunar Reconnaissance Orbiter] Imagery
  22. Apollo 11: The Complete Descent
  23. Why were there missing rungs on the Lunar Lander’s Ladder?
  24. Apollo AGC Part 1: Restoring the computer that put man on the Moon
  25. Apollo AGC Part 23: Flying realistic Apollo 11 moon landings with the Apollo Guidance Computer
  26. An Audience with Neil Armstrong (2011 interview)
  27. Apollo 11 crew member [Buzz Aldrin]
  28. WATCH: Astronaut Michael Collins discusses the Apollo 11 launch 50 years later
  29. Apollo’s Most Important Discovery (Inside NASA’s Moon Rock Vault!)
  30. Where does NASA keep the Moon Rocks? – Smarter Every Day 220
  31. APOLLO MOON SUIT: demonstration of functioning, and manufacturing (1969)
  32. Moon Machines: Space Suit (5/6)
  33. The Space Suit Special

Conspiracy theories and their debunking

  1. NASA: Moon Landing – Apollo 11 Descent Film and LRO [Lunar Reconnaissance Orbiter] Imagery
  2. Moon-Landing Hoax Still Lives On, 50 Years After Apollo 11. But Why?
  3. Apollo and the moon-landing hoax
  4. Moon landing conspiracy theories – Hoax claims and rebuttals
  5. A Brief History of Moon Hoaxes – Why do people still believe in them?
  6. Why Faking the Moon Landing Was Impossible
  7. Debunking the Myth that the Moon Landing Was a Hoax
  8. Nvidia Debunks Conspiracy Theories About Moon Landing
  9. Moon Hoax: Debunked!
  10. The Space Suit Special

The USSR

And, finally, these videos about the USSR’s failed attempt to put a man on the Moon are worth watching:

  1. Why Russia Did Not Put a Man on the Moon – The Secret Soviet Moon Rocket
  2. Soviet N1 Moon Rocket Documentary

Amazing that the NK-33 closed-cycle engines originally developed for the N1 were purchased by a US company, modified and finally used in a new launcher in 2013. The later and larger RD-180, also a Russian closed-cycle engine, is – if I understand correctly – still used to power US Atlas rockets until US-designed replacements are available.

Automatic backup of users’ files on a NAS device to an external USB HDD

One of my Linux machines is a 4-bay server that performs various roles, one of which is as NAS (network-attached storage) for family and visitors’ devices connected to my home network. I had configured each pair of HDDs in a RAID 1 array in order to provide some internal redundancy, but I was nervous about not having an external backup for users’ shares. Therefore I recently purchased a 6TB external USB 3.0 HDD (Western Digital Elements Desktop WDBWLG0060HBK-EESN) to connect permanently to one of the server’s USB 3.0 ports for backup purposes. I created a Bash script ~/backup_to_usbhdd.sh to perform the backup, plus a cron job to launch it automatically at 05:01 daily:

user $ sudo crontab -e
user $ sudo crontab -l | grep -v ^# | grep backup
01 05 * * * sudo /home/fitzcarraldo/backup_to_usbhdd.sh

The use of ‘sudo‘ in the crontab command may appear superfluous because the cron job was created for the root user (i.e. by using ‘sudo crontab -e‘ rather than ‘crontab -e‘). However, this is done to make cron use the root user’s environment rather than the minimal set of environment variables cron would otherwise use [1].

#!/bin/bash
#
# This script backs up to an external USB HDD (NTFS) labelled "Elements" the contents
# of the directories /nas/shares/ on my server.
# It can be launched from the server either manually using sudo or as a root-user cron
# job (Use 'sudo crontab -e' to configure the job).
#
# Clean up if the backup did not complete last time:
umount /media/usbhdd 2>/dev/null
rm -rf /media/usbhdd/*
# Unmount the external USB HDD if mounted by udisks2 with the logged-in username in the path:
umount /media/*/Elements 2>/dev/null
# Find out the USB HDD device:
DEVICE=$( blkid | grep "Elements" | cut -d ":" -f1 )
# Create a suitable mount point if it does not already exist, and mount the device on it:
mkdir /media/usbhdd 2>/dev/null
mount -t ntfs-3g $DEVICE /media/usbhdd 2>/dev/null
sleep 10s
# Create the backup directories on the USB HDD if they do not already exist:
mkdir -p /media/usbhdd/nas 2>/dev/null
# Backup recursively the directories and add a time-stamped summary to the log file:
echo "********** Backing up nas shares directory **********" >> /home/fitzcarraldo/backup_to_usbhdd.log
date >> /home/fitzcarraldo/backup_to_usbhdd.log
# Need to use rsync rather than cp, so that can rate-limit the copying to the USB HDD:
rsync --recursive --times --perms --links --protect-args --bwlimit=22500 /nas/shares /media/usbhdd/nas/ 2>> /home/fitzcarraldo/backup_to_usbhdd.log
# No --delete option is used, so that any backed-up files deleted on the server are not deleted from the USB HDD.
echo "Copying completed" >> /home/fitzcarraldo/backup_to_usbhdd.log
date >> /home/fitzcarraldo/backup_to_usbhdd.log
df -h | grep Filesystem >> /home/fitzcarraldo/backup_to_usbhdd.log
df -h | grep usbhdd >> /home/fitzcarraldo/backup_to_usbhdd.log
echo "********** Backup completed **********" >> /home/fitzcarraldo/backup_to_usbhdd.log
cp /home/fitzcarraldo/backup_to_usbhdd.log /media/usbhdd/
# Unmount the USB HDD:
umount /media/usbhdd
exit 0

The initial version of the above script used ‘cp‘ rather than ‘rsync‘, which worked fine when I launched the script manually:

user $ sudo ./backup_to_usbhdd.sh

However, the script always failed when launched as a cron job. In this case the command ‘df -h‘ showed the root directory on the server was ‘100% used’ (full). Also, the mount point directory /media/usbhdd/ had not been unmounted. The log file had twenty or so lines similar to the following, indicating the script had failed due to the root filesystem becoming full:

cp: failed to extend ‘/media/usbhdd/nas/user1/Videos/20130822_101433.mp4’: No space left on device

Apparently data was being read from the server’s HDD into the RAM buffer/cache faster than it could be written to the external HDD. The bottleneck in this case is not USB 3.0, but the USB HDD itself. The specifications for the USB HDD do not mention drive write speed, but a quick search of the Web indicated that an external USB HDD might have a write speed of around 25 to 30 MBps (Megabytes per second). I do not know why the problem happened only when the script was launched as a cron job, but I clearly needed to throttle the rate of writing to the external HDD. Unfortunately the ‘cp‘ command does not have such an option, but the ‘rsync‘ command does:

--bwlimit=RATE          limit socket I/O bandwidth

where RATE is in KiB if no units are specified. I opted to use a rate of 22500 KiB to be safe, and it is not too far below the aforementioned 25 MBps. Indeed, using this limit the script runs to completion successfully when launched by cron:

user $ cat backup_to_usbhdd.log
********** Backing up nas shares directory **********
Thu Sep 13 05:01:26 BST 2018
Copying completed
Thu Sep 13 11:41:31 BST 2018
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdf1       5.5T  386G  5.1T   7% /media/usbhdd
********** Backup completed **********
********** Backing up nas shares directory **********
Fri Sep 14 05:01:26 BST 2018
Copying completed
Fri Sep 14 05:20:08 BST 2018
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdf1       5.5T  403G  5.1T   8% /media/usbhdd
********** Backup completed **********
********** Backing up nas shares directory **********
Sat Sep 15 05:01:26 BST 2018
Copying completed
Sat Sep 15 05:04:58 BST 2018
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdf1       5.5T  404G  5.1T   8% /media/usbhdd
********** Backup completed **********
********** Backing up nas shares directory **********
Sun Sep 16 05:01:26 BST 2018
Copying completed
Sun Sep 16 05:15:14 BST 2018
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdf1       5.5T  416G  5.1T   8% /media/usbhdd
********** Backup completed **********
********** Backing up nas shares directory **********
Mon Sep 17 05:01:26 BST 2018
Copying completed
Mon Sep 17 05:04:15 BST 2018
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdf1       5.5T  416G  5.1T   8% /media/usbhdd
********** Backup completed **********

Notice that the first job listed in the log file took much longer than subsequent jobs. This was because rsync had to copy every file to the external USB HDD. In subsequent runs it only had to copy new files and files that had changed since they were last copied.

The disk in the external USB HDD spins down after 10 minutes of inactivity and the drive goes into Power Saver Mode. Its LED blinks to indicate the drive is in this mode. Therefore the cron job only spins up and down the external HDD once per day.

Reference
1. Why does root cron job script need ‘sudo’ to run properly?

Gentoo Linux: A work-around to be able to Resume from Suspend to RAM when using the NVIDIA closed-source driver

My Clevo W230SS laptop has NVIDIA Optimus graphics hardware (NVIDIA GPU plus Intel IGP). I do not use Bumblebee, preferring to switch between the Intel video driver and the NVIDIA closed-source driver myself (see Switching between Intel and NVIDIA graphics processors on a laptop with NVIDIA Optimus hardware running Gentoo Linux). The laptop can suspend to RAM and resume perfectly when using the Intel video driver (but see Stopping my laptop spontaneously resuming immediately after Suspend to RAM, which is applicable whatever the GPU or IGP).

In order to be able to resume properly from Suspend-to-RAM when using the NVIDIA driver, the laptop needs to disable compositing before suspending, then re-enable compositing after resuming. For how I achieve that, see under Problem 2 in the third link above. If this is not done, the graphics on the Desktop are corrupted after resuming.

However, recently when using the NVIDIA driver and KDE Plasma 5 (I am currently using nvidia-drivers-387.22 and plasma-meta-5.11.5), when resuming from suspension the monitor would briefly display the LightDM wallpaper (I use different wallpapers for the display manager and the lock screen, so I know it was not the KDE lock screen) followed by a blank screen with a mouse pointer (which I could move normally). More recently, in between displaying the display manager’s wallpaper and the blank screen, the monitor would briefly display an earlier image of the Desktop just before the laptop suspended.

Now, I could simply leave the laptop configured to use the Intel driver. However, sometimes I need to use a CAD application and the performance is better when using the NVIDIA GPU.

There are umpteen posts on the Web about this problem, and the root cause seems to be the closed-source NVIDIA driver. I have seen the KDE lock screen mentioned in some posts as the culprit, so I disabled the lock screen (‘System Settings’ > ‘Desktop Behaviour’ > ‘Screen Locking’) but that did not solve the problem.

I put up with this for several weeks in the hope that the next release of the NVIDIA driver would fix the problem. If I suspended to RAM while the laptop was using the NVIDIA driver, I was able to resume and get to a working Desktop – albeit without the open windows and applications that had been running before suspending – by pressing Ctrl+Alt+F1 to get to TTY1, logging in as the root user and entering the command ‘/etc/init.d/xdm restart‘. However, the final straw was in a meeting a couple of weeks ago when I wanted to resume the laptop and show a worksheet to someone. The laptop monitor of course displayed a blank screen with a mouse pointer, and it took me a couple of minutes to restart the display manager, login to KDE Plasma 5 and open the spreadsheet again. So this week I decided to look into the problem to see if I could at least find a work-around that would enable the laptop to resume without needing to restart X Windows and login to Plasma 5 each time.

I created a Bash script in /etc/pm/sleep.d/ to unload the NVIDIA modules before suspending to RAM and to re-load them when resuming, but that did not solve the problem either.

I switched the rendering background from OpenGL 2.0 to OpenGL 3.1 (‘System Settings’ > ‘Display and Monitor’ > ‘Compositor’), but that did not work either. I switched the rendering backend to XRender, and that did enable the laptop to resume from suspend successfully with the NVIDIA driver, but I do not want to use that work-around. Firstly, with software rendering there is a performance hit, and, secondly, there was no KDE Desktop Cube when using XRender instead of OpenGL. I use the Desktop Cube when working, as I often have a lot of windows open on each virtual desktop (cube side), and I find it easier to use the cube than a flat UI.

Eventually I found that, after resuming, if I pressed Ctrl+Alt+F1 to get to a virtual console, logged into my user account, entered the command ‘DISPLAY=:0 /usr/bin/kwin_x11 --resume‘ and then pressed Ctrl+Alt+F7 to get back to TTY7, my Desktop would appear on TTY7. Even so, I noticed on TTY1 that the following error messages were displayed when I ran that command:

kwin_core: OpenGL 2 compositing setup failed
kwin_core: Failed to initialize compositing, compositing disabled

Anyway, the Plasma 5 Desktop was displayed on TTY7, and with the windows that were open when I suspended the laptop, so restarting KWin would at least be a viable work-around until NVIDIA fix their video driver.

I incoporated the command in my script /etc/pm/sleep.d/02-toggle-compositing like so:

#!/bin/sh
#
# Turn off compositing on hibernate or suspend
# Turn on compositing on thaw or resume

username=fitzcarraldo
userhome=/home/$username
export XAUTHORITY="$userhome/.Xauthority"
export DISPLAY=":0"

case "$1" in
     suspend|hibernate)
          su $username -c "qdbus org.kde.KWin /Compositor suspend" &
     ;;
     resume|thaw)
          su $username -c "qdbus org.kde.KWin /Compositor resume" &
          su $username -c "/usr/bin/kwin_x11 --replace" &
     ;;
     *)
          exit $NA
     ;;
esac

It is an ugly hack, but at least now the laptop can resume properly from Suspend-to-RAM while the NVIDIA driver is being used.

Perhaps Linus Torvalds was correct. I will try to avoid NVIDIA hardware when I replace my current laptop.

Bye bye Windows 10, and good riddance

Up until a couple of days ago my family’s PC, an Acer Aspire XC600 tower purchased in early 2014, had Microsoft Windows 10 Home (64-bit) installed. Because of a problem updating Windows 10 which finally rendered the PC unbootable and the OS unrecoverable, I installed Lubuntu 17.10 (64-bit). It is performing very well and my family are finding it easy to use. Although I had no intention of installing Linux on this machine before the problem updating Windows arose, I’m now glad to be rid of Windows on this machine, as Windows has been a pain to use and maintain.

The Windows update saga

When I bought the Aspire XC600 in February 2014 it came with Windows 8 pre-installed, and I immediately upgraded it to Windows 8.1. I say ‘immediately’, but it actually took me three days to get Windows Update to install it properly; the first attempts resulted in what looked like Windows 8.1 but turned out to be incomplete installations, and several times I had to roll back to a Restore Point and try to update again.

I upgraded the machine to Windows 10 Home when Microsoft offered it free-of-charge to current users of Windows 8.1 and Windows Update informed me the update was available to install. The early Windows 10 Home was buggy, but various updates by Microsoft eventually got it to a reasonably stable state by the time the so-called ‘Anniversary Update’ (Windows 10 Version 1607) was released in 2016. I again had to struggle for several days before I managed to update Windows 10 Home to Version 1607.

In April 2017 Microsoft released the ‘Creators Update’, and in October 2017 the ‘Fall Creators Update’. However, no matter what I did it was simply impossible to upgrade Window 10 Home Version 1607 on the Aspire XC600 to either of those 2017 updates. There are hundreds if not thousands of posts on the Web regarding problems installing these updates on various PC models from various manufacturers, with similar or even identical symptoms to those I was seeing. In my case the update process froze at 33%, 75% and 83%, despite Microsoft’s update utility informing me that the CPU, RAM size and HDD free space were valid for these updates. Furthermore, I only tried to update once Windows Update had informed me the updates were available to install. I should also point out that I regularly made sure the OS had all other updates installed.

I lost count of the number of times and hours spent trying to update to the Creators Update and the Fall Creators Update. Each time I had a go at updating, after two consecutive attempts Windows 10 Home would give up and, when I eventually cycled the mains power in order to exit the frozen state, would roll back to Version 1607. However, during my latest attempt a couple of days ago, Windows 10 Home would no longer complete booting, instead popping up a window informing me the machine needed to be rebooted to complete the installation process. Every time I clicked ‘OK’ in the window, the machine would reboot and the same window popped up again. So I dug out the Windows 10 Home Recovery Disk (actually a USB pendrive) I had carefully created as soon as I had upgraded the installation from Windows 8.1 to Windows 10 in November 2016. (That pendrive had previously been the Windows 8.1 Recovery Disk that I created as soon as I upgraded the installation to Windows 8.1 in February 2014.) But, no matter what I did, the Recovery Disk would only re-install Windows 8, even though the time-date stamp of the files on the pendrive corresponded to the date on which I created the Windows 10 Recovery Disk. And, strangely, there were three so-called Recovery Partitions on the HDD.

Several attempts to re-install using the Recovery Disk had the same outcome, so I decided to install a couple of Linux binary distributions in succession, both of which worked fine and definitely removed all traces of Windows from the HDD, including the three Recovery Partitions (I checked using GParted to make sure). Then I tried again to re-install Windows 10 Home from the Recovery Disk, but it still created three Recovery Partitions and still installed Windows 8.

Clearly it was not going to be possible to re-install Windows 10 Home using the Recovery Disk, so I instead used Windows Update in Windows 8 to update the installation to Windows 8.1, a process that took several hours and reboots. Once Windows 8.1 was installed, I tried to upgrade to Window 10, first using Windows Update and, when that told me there were no updates, by using the Recovery Disk. Neither approach was successful, so I was stuck with a working, fully-updated Windows 8.1. The trouble was, Windows 8.1 is no longer supported by Microsoft (‘Mainstream Support End Date’ is 9 January 2018). Not to mention that Windows 8.1 is even worse than Windows 10.

The move to Linux

At this point I’d had more than enough of Microsoft Windows. Therefore I used my laptop to download the ISO for Lubuntu 17.10 and create a LivePendrive, and I installed Lubuntu on the Aspire XC600. Although I use a source-based Linux distribution on two laptops, for ease and speed of installation and maintenance I opted to install a binary-based distribution on the family PC. I chose Lubuntu specifically because it uses the LXDE desktop environment, which is closer in look and feel to classic Windows than e.g. the Unity or GNOME desktop environments in Ubuntu, and is not as ‘CPU-hungry’ as KDE. I found that Lubuntu worked extremely well out-of-the-box, including scanning and printing using my Canon MP510 MFP. I used the GUI Software utility (‘System Tools’ > ‘Software’ from the LXDE application menu) to uninstall AbiWord and Gnumeric and install the LibreOffice suite. I added user accounts for the members of my family (‘System Tools’ > ‘Users and Groups’). Since the machines on my home network use SMB to share files, I installed samba and sambaclient and edited the smb.conf file via the command line, and browsing SMB shares worked first time. We have a decent family PC again.

There was not much more for me to do to make the installation behave exactly how I wanted it to:

  • I configured the installation so that each user’s avatar appears on the login screen (LightDM GTK Greeter).
  • I have an external USB HDD permanently connected to the PC so that users’ files can be backed up. I configured the installation to unmount automatically this external USB HDD when any user logs out. The USB HDD is automatically mounted anyway when another user logs in, and, by unmounting it automatically at logout, the next user can access the USB HDD properly via the GUI File Manager (the USB drive is mounted as /media/<username>/FREECOM HDD).
  • I installed Language Support so that I can switch to some other languages I use, and I configured LXDE so I can click on an icon on the panel (or use a keyboard shortcut) to switch between the associated keyboard layouts.
  • I installed the anti-virus utility ClamAV, the ClamAV daemon and the ClamTk GUI front-end, and configured the installation to scan automatically any files downloaded to each user’s ~/Downloads directory, and to quarantine infected files and notify the user via a pop-up window and log file.
  • I configured the installation to create a network route when I log in, so that I can access in a Web browser the GoAccess dashboard for database reports produced by my network server.
  • I configured the installation to backup the files in each user’s ~/home directory to an external USB HDD at shutdown (impossible in Windows 10 Home — see my comments further on).
  • I installed Skype Preview for Linux, which worked out-of-the-box with a GUCEE HD92 HD 720p USB Webcam with built-in microphone.

I intend to explain in future posts how I implemented each of the above.

Backing up users’ files at shutdown

Windows XP and Vista on my family’s previous PCs were able to run a batch file (BACKUP.BAT) automatically at shutdown to backup the users’ files to an external USB HDD (and, crucially, to wait until the batch job was completed before powering down the PC). To achieve this I used the utility Xecutor by Xpertdesign Software, which enabled users to use the normal Windows method of shutting down yet allowed the batch file to run to completion. However, such utilities do not work in Windows 8 and onwards. A kludge that is often suggested is to add an extra button on the Desktop or Taskbar to run the backup commands then shutdown the machine afterwards, but I did not want to do that because there is no guarantee my family would click on it rather than shutting down Windows the normal way.

Another method of configuring Windows to run a batch file at shutdown is to use the GPE (Group Policy Editor) a.k.a. GPOE (Group Policy Object Editor) to configure the Registry. However, Windows 10 Home does not include the GPE, so I was unable to use the GPE to configure Windows 10 Home to run a batch file to backup users’ files to an external USB HDD at shutdown. (Actually, as Windows 8/8.1/10 makes it almost impossible to interrupt the shutdown process once the user has initiated shutdown, I wonder if a backup batch file would actually run to completion if the GPE were used in an edition of Windows that provides it, such as Windows 10 Enterprise.) It is possible to configure the Task Scheduler in Windows 10 Home to run a batch file at shutdown, but it is impossible to pause the shutdown process to allow the backup batch file to run to completion. Believe me, I tried everything, and it is impossible to backup automatically all users’ files for multiple user accounts at shutdown with Windows 10 Home (even though it was possible in Windows XP). So I had to resort to a kludge recommended by Microsoft, which is to configure the Task Scheduler to run the batch file at startup instead of shutdown. Clearly this is less safe than backing up before shutting down the PC.

Actually, it is possible to install/enable the GPE in Windows 10 Home — there are many Web sites explaining how to do this — but Microsoft has restricted many GPOs (Group Policy Objects) in Windows 10 Home, and therefore adding a GPO using the GPE or by editing the Registry directly in Windows 10 Home will have no effect. Even if you enable the GPE in Windows 10 Home, the policies will not work until you buy a licence for the Windows 10 Pro or Enterprise editions. In summary, in Windows 10 Home it is a waste of time either installing/enabling the GPE or editing the Registry directly.

However, now that Lubuntu 17.10 is installed I was able to configure it to run a Bash script automatically to backup all the users’ files before the machine actually actually shuts down or reboots. In a future post I’ll explain how I achieved that.

Summary

In my opinion Microsoft jumped the shark a long time ago. I had plenty of trouble with Windows Vista (to the extent I had to ditch it in the end), but Windows 7 was not bad (although on a couple of occasions I had a big scare with ‘Windows Backup and Restore’ that necessitated restoring the MBR via the command line). Windows 8 and 8.1 were awful, and Windows 10 is not much better in my opinion. Furthermore, I think it is very bad form for Microsoft to release updates to Windows 10 that cannot be installed on a machine that is only four years old and still has a reasonable specification: 64-bit Intel Pentium G2030 @ 3.00GHz, 4GB DDR3 RAM (upgradable to 8GB), Intel HD Graphics (Xeon E3-1200 v2/3rd Gen), and 1TB 7200RPM HDD. I’m now glad Windows 10 is history on this PC and I’m typing this in a Linux installation.

Partitioning hard disk drives for BIOS-MBR, BIOS-GPT and UEFI-GPT in Linux

Introduction

This post was prompted by recent threads in the Gentoo Linux Forums such as the following:

  • partition MBR/GPT fdisk question‘ in which two people asked how to partition a HDD using GPT. One has a computer that only has BIOS firmware; the other has a computer with firmware that supports UEFI and BIOS (user-selectable) and who apparently wanted to boot using BIOS.
  • [SOLVED] installing on UEFI system‘ in which someone asked how to partition a HDD for a computer with UEFI firmware.

These Gentoo Linux users were confused by the instructions in the Gentoo Installation Guide – Preparing the disks. In my opinion the Gentoo Installation Guide is confusing. You do not need a BIOS boot partition (Code EF02) if you want to use a GPT-partitioned HDD with a UEFI computer; you only need a BIOS boot partition on a GPT-partitioned HDD if you want to use it with a BIOS computer. You do not need an ESP (EFI System Partition) if you want to use a GPT-partitioned HDD with a BIOS computer; you only need an ESP (Code EF00) — which must be formatted as FAT32 — if you want to use UEFI.

Coincidentally, recently an Ubuntu user also told me he finds GPT partitioning for UEFI confusing.

So, in the hope of helping people having trouble understanding how to partition a HDD for Linux, I decided to post some information on the two firmware designs and the two partitioning designs, plus list a few (of the many) partitioning schemes that can be used. The purpose of this post is to provide an overview of the designs and possible schemes, not to explain how to use the various partitioning tools or to list the precise steps you must follow. I have tried to avoid going into fine detail (for example the oft-quoted limit of 2TiB for MBR-partitioned HDDs is contigent on 512B sectors).

Firstly, let’s get the terminology straight: you should say ‘UEFI firmware’, not ‘UEFI BIOS’. The latter is an oxymoron. I know that some computer manufacturers and third-party firmware providers use the term ‘UEFI BIOS’, but it is incorrect. The ‘BIOS’ (a.k.a. ‘PC BIOS’) and the UEFI are two different designs of firmware used during the booting process. BIOS firmware and MBR partitioning are older (‘legacy’) designs; UEFI firmware and GPT partitioning are the latest designs, intended to replace BIOS and MBR. See the end of this post for links to articles regarding BIOS, MBR, UEFI and GPT that I recommend you also read.

I personally have not come across computers that support solely UEFI; all the computers I have used allow the user to configure the firmware at boot via the Setup menu to use either UEFI or BIOS. Some UEFI firmware manufacturers show the BIOS support option in the Setup menu as ‘Compatibility Support Module’ (CSM) or ‘Legacy Mode’.

‘Secure Boot’ is one of the touted UEFI features not present in BIOS. Linux expert Roderick W. Smith explains Secure Boot in his article ‘Linux on UEFI: A Quick Installation Guide‘:

One optional feature of UEFI deserves mention: Secure Boot. This feature is designed to minimize the risk of a computer becoming infected with a boot kit, which is a type of malware that infects the computer’s boot loader. Boot kits can be particularly difficult to detect and remove, which makes blocking them a priority. Microsoft requires that all desktop and laptop computers that bear a Windows 8 logo ship with Secure Boot enabled. This type of configuration complicates Linux installation, although some distributions handle this problem better than do others. Do not confuse Secure Boot with EFI or UEFI, though; it’s possible for an EFI computer to not support Secure Boot, and it’s possible to disable Secure Boot even on x86-64 EFI computers that support it. Microsoft requires that users can disable Secure Boot for Windows 8 certification on x86 and x86-64 computers; however, this requirement is reversed for ARM computers—such computers that ship with Windows 8 must not permit the user to disable Secure Boot. Fortunately, ARM-based Windows 8 computers are currently rare. I recommend avoiding them.

GPT-partitioned HDDs have become the norm these days for two principal reasons:

  • the MBR design limits the amount of disk space accessible to a maximum of 2TiB but HDDs larger than 2TiB are now common;
  • Microsoft has standardised on UEFI, and UEFI will not boot an MBR-partitioned HDD (Section 5.2.1 of Version 2.6 of the UEFI Specification specifies that UEFI firmware shall not execute the boot code in an MBR located at the first logical block of a disk with the MBR disk layout).

The ‘Protective MBR’ at the beginning of every GPT-partitioned disk (in the same location on the disk as a legacy MBR would be) is designed to prevent MBR-based disk utilities misrecognising and possibly overwriting GPT-partitioned disks. A fake (i.e. it does not really exist) single partition called the ‘GPT Protective Partition’ (Code EE00) is specified in the Protective MBR to occupy as much of the drive as can be represented in an MBR, namely a maximum of 2TiB in the case of a disk with 512B sectors. Operating systems and tools not designed for GPT disks will read the Protective MBR and detect that the disk contains a single partition of unknown type and with no empty space, and will refuse to modify the disk unless the user deletes this partition. This design (i.e. the Protective MBR and GPT Protective Partition) was devised in order to minimise the possibility of a) legacy software accidentally overwriting a GPT-partitioned HDD; b) GPT-aware software accidentally overwriting an MBR-partitioned HDD (the absence of a partition of type EEh defined in the Protective MBR would indicate to GPT-aware operating systems and tools that the HDD is not GPT-partitioned).

You can use an MBR-partitioned HDD with BIOS; you can use a GPT-partitioned HDD with BIOS; you can use a GPT-partitioned HDD with UEFI; you cannot use an MBR-partitioned HDD with UEFI. If the firmware in your computer has an option to select BIOS mode (some firmware manufacturers refer to this as ‘Compatibility Support Module’ or ‘Legacy Mode’) instead of UEFI and you want to use an MBR on the HDD, you will have to use BIOS. In summary, for Linux your options are BIOS-MBR, BIOS-GPT or UEFI-GPT. I will discuss these three options and provide some possible partitioning schemes in each case.

The layout of a GPT-partitioned HDD is as follows:

The beginning of a GPT-partitioned disk
Protective MBR 512B
Primary GPT Header 512B
Primary Partition Table entries Up to 16KiB (maximum of 128 partitions)

Therefore the unpartitioned space at the beginning of the HDD should be at least 17KiB.

Unlike in the MBR design, the end of a GPT-partitioned disk stores a backup of the partition table:

The end of a GPT-partitioned disk
Secondary Partition Table entries Up to 16KiB (maximum of 128 partitions)
Secondary GPT Header 512B

(The Secondary GPT Header really does come after the Secondary Partition Table entries.)

Therefore the upartitioned space at end of a GPT-partitioned disk should be at least 16.5KiB.

The partitions themselves are located between the Primary GPT and the Secondary GPT, i.e. between the above two tables.

The remainder of a GPT-partitioned disk between the above two areas can contain up to 128 partitions. One partition — normally the first partition is used — must be the ‘ESP’ (EFI System Partition), which should be formatted as FAT32 and have the ‘esp’ and ‘boot’ flags set. The partition code must be EF02. The minimum possible size of a FAT32 partition is 33,549,824B (~32MiB) with 65525 clusters and 512 byte sectors, or 268,398,592B (~256MiB) with 65525 clusters and 4KiB sectors. The required size of the ESP will depend on what is stored in it (for example the now-orphaned ELILO boot loader stored the Linux kernel images in it, whereas the GRUB boot loader does not). The Ubuntu Community Help Wiki specifies a minimum of 100MiB and recommends 200MiB. The Gentoo Installation Guide recommends 128MB.

The versions of GParted and KDE Partition Manager I have used to partition HDDs seem to want to reserve at least 2048 512B sectors of empty space before the first partition (sometimes these two utilities force me to have 4096 512B sectors of empty space before the first partition), so I suggest leaving at least 1MiB of unpartitioned space at the beginning and end of the HDD if you want to partition a disk for GPT using those tools. Actually, for peace of mind you may as well leave 1MiB of empty space at the beginning and end of the disk whatever GUI tools or console commands (parted, fdisk, gdisk, etc.) you use to partition the disk.

By the way, if you wanted to use MBR instead of GPT, the MBR (512B) plus the GRUB embedding area after the MBR (~31KiB) means that 1MiB would also be more than enough for an MBR HDD (see GNU GRUB Manual – 3.4 BIOS installation).

Thus my suggestions for HDD partitioning would be as shown below. Note that the order of the partitions I have adopted below is not obligatory; you can change the order if you wish. I prefer to specify precise sizes for the swap and root partitions, and put /home on the last partition so it can occupy the remainder of the disk, whatever size that may be.

UEFI-GPT

Option 1

This is my preferred option because I can edit /etc/fstab and specify that /boot must not be mounted at boot, thus reducing the possibility of the files in /boot getting corrupted.

UEFI-GPT Option 1
1.00 MiB of empty space at the beginning of the disk.
/dev/sda1
Size: 512 MiB
File system: FAT32
Flags: boot & esp
Code: EF00
Label: ESP
Mount point: /boot/efi
/dev/sda2
Size: 16 GiB for a computer with 16 GiB of RAM*
File system: linux-swap
Flags: None
Code: 8200
Label: SWAP
Mount point: None
/dev/sda3
Size: 512 MiB
File system: ext2
Flags: None
Code: 8300
Label: BOOT
Mount point: /boot Therefore /boot/grub/ will be on this partition if you use GRUB.
/dev/sda4
Size: e.g. 64 GiB (128 GiB if the drive is big)
File system: ext4
Flags: None
Code: 8300
Label: ROOT
Mount point: / (root)
/dev/sda5
Size: 1.00 MiB less than the remaining disk space
File system: ext4
Flags: None
Code: 8300
Label: HOME
Mount point: /home
1.00 MiB of empty space at the end of the disk.

* You will often see recommendations to make the size of the swap partition the same as the size of the RAM if you want to be able to put the computer into hibernation. In fact, the Linux kernel is normally configured to compress the contents of the RAM image for hibernation, and I personally have seen the disk image of 4GB of RAM compressed to 23% of that size. Nevertheless, if you have a large HDD you may as well just take the easy route and allocate the size of the swap partition to be the same as the RAM size, even if all of the swap partition will never be used in practice. That way you are guaranteed to be able to put the computer into hibernation.

Below is an example of the above partitioning scheme on a virtual UEFI machine with a 64GiB virtual GPT-partitioned HDD:

root # gdisk -l /dev/sda
GPT fdisk (gdisk) version 1.0.1

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 134217728 sectors, 64.0 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 54B3C38F-1C55-4A19-9BAA-499C4D0D8DD0
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 134217694
Partitions will be aligned on 2048-sector boundaries
Total free space is 4029 sectors (2.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048         1050623   512.0 MiB   EF00
   2         1050624         5244927   2.0 GiB     8200
   3         5244928         6293503   512.0 MiB   8300
   4         6293504        72353791   31.5 GiB    8300
   5        72353792       134215679   29.5 GiB    8300

root # fdisk -l /dev/sda
Disk /dev/sda: 64 GiB, 68719476736 bytes, 134217728 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 54B3C38F-1C55-4A19-9BAA-499C4D0D8DD0

Device        Start       End  Sectors  Size Type
/dev/sda1      2048   1050623  1048576  512M EFI System
/dev/sda2   1050624   5244927  4194304    2G Linux swap
/dev/sda3   5244928   6293503  1048576  512M Linux filesystem
/dev/sda4   6293504  72353791 66060288 31.5G Linux filesystem
/dev/sda5  72353792 134215679 61861888 29.5G Linux filesystem

root # parted /dev/sda print
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sda: 68.7GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system     Name  Flags
 1      1049kB  538MB   537MB   fat32                 boot, esp
 2      538MB   2685MB  2147MB  linux-swap(v1)
 3      2685MB  3222MB  537MB   ext2
 4      3222MB  37.0GB  33.8GB  ext4
 5      37.0GB  68.7GB  31.7GB  ext4

root # blkid
/dev/sda4: LABEL="ROOT" UUID="174ac3e8-f105-4606-bed1-7a1aa22c3631" TYPE="ext4" PARTUUID="01d9c139-fe70-415a-abc6-2351fad33384"
/dev/sda1: UUID="B4C1-7EA5" TYPE="vfat" PARTUUID="d941f728-c386-4f4c-b0c3-aa76f4290774"
/dev/sda2: LABEL="SWAP" UUID="e3ddf9b5-2ae3-4469-a121-0a1a78aa6702" TYPE="swap" PARTUUID="a4daec88-da44-4ae3-8119-01cc81325f03"
/dev/sda3: LABEL="BOOT" UUID="1e24ea9d-5358-4e9b-8667-d7a42e7b6ad7" TYPE="ext2" PARTUUID="b5369ce3-4b44-4d19-be6f-1d226dc71cb3"
/dev/sda5: LABEL="HOME" UUID="87f6a0af-dbed-4587-b810-efca8f269618" TYPE="ext4" PARTUUID="19fd7d00-2d89-4653-af03-e81618a3b70d"

Option 2

You could use this scheme if you are not interested in having /boot on its own partition.

UEFI-GPT Option 2
1.00 MiB of empty space at the beginning of the disk.
/dev/sda1
Size: 512 MiB
File system: FAT32
Flags: boot & esp
Code: EF00
Label: ESP
Mount point: /boot/efi
/dev/sda2
Size: 16 GiB for a computer with 16 GiB of RAM*
File system: linux-swap
Flags: None
Code: 8200
Label: SWAP
Mount point: None
/dev/sda3
Size: e.g. 64 GiB (128 GiB if the drive is big)
File system: ext4
Flags: None
Code: 8300
Label: ROOT
Mount point: / (root) Therefore /boot (and /boot/grub/) will on this partition too.
/dev/sda4
Size: 1.00 MiB less than the remaining space on the disk
File system: ext4
Flags: None
Code: 8300
Label: HOME
Mount point: /home
1.00 MiB of empty space at the end of the disk.

* See my earlier note regarding hibernation.

Below is an example of the above scheme on a virtual UEFI machine with a 64GiB virtual GPT-partitioned HDD:

root # gdisk -l /dev/sda
GPT fdisk (gdisk) version 1.0.1

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 134217728 sectors, 64.0 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 54B3C38F-1C55-4A19-9BAA-499C4D0D8DD0
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 134217694
Partitions will be aligned on 2048-sector boundaries
Total free space is 4029 sectors (2.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048         1050623   512.0 MiB   EF00
   2         1050624         5244927   2.0 GiB     8200
   3         5244928        72353791   32.0 GiB    8300
   4        72353792       134215679   29.5 GiB    8300

root # fdisk -l /dev/sda
Disk /dev/sda: 64 GiB, 68719476736 bytes, 134217728 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 54B3C38F-1C55-4A19-9BAA-499C4D0D8DD0

Device        Start       End  Sectors  Size Type
/dev/sda1      2048   1050623  1048576  512M EFI System
/dev/sda2   1050624   5244927  4194304    2G Linux swap
/dev/sda3   5244928  72353791 67108864   32G Linux filesystem
/dev/sda4  72353792 134215679 61861888 29.5G Linux filesystem

root # parted /dev/sda print
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sda: 68.7GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system     Name  Flags
 1      1049kB  538MB   537MB   fat32                 boot, esp
 2      538MB   2685MB  2147MB  linux-swap(v1)
 3      2685MB  37.0GB  34.4GB  ext4
 4      37.0GB  68.7GB  31.7GB  ext4


root # blkid
/dev/sda3: LABEL="ROOT" UUID="fdf2b11a-8c6b-4bb3-9534-477c3ed49d95" TYPE="ext4" PARTUUID="f393129f-ab32-40fb-bf78-3aead3dd4af0"
/dev/sda1: UUID="C024-8A30" TYPE="vfat" PARTUUID="d941f728-c386-4f4c-b0c3-aa76f4290774"
/dev/sda2: LABEL="SWAP" UUID="1f752a05-a1fb-4c5f-ab2e-079715207b4d" TYPE="swap" PARTUUID="a4daec88-da44-4ae3-8119-01cc81325f03"
/dev/sda4: LABEL="HOME" UUID="041e4ab2-d54c-4092-b445-779997ac09ce" TYPE="ext4" PARTUUID="7e1b8dc0-2f38-4260-95af-fbb80bb72156"

Option 3

You could use this scheme if you are not interested in having /boot and /home on their own partitions.

UEFI-GPT Option 3
1.00 MiB of empty space at the beginning of the disk.
/dev/sda1
Size: 512 MiB
File system: FAT32
Flags: boot & esp
Code: EF00
Label: ESP
Mount point: /boot/efi
/dev/sda2
Size: 16 GiB for a computer with 16 GiB of RAM*
File system: linux-swap
Flags: None
Code: 8200
Label: SWAP
Mount point: None
/dev/sda3
Size: 1.00 MiB less than the remaining space on the disk
File system: ext4
Flags: None
Code: 8300
Label: ROOT
Mount point: / (root) Therefore /boot (and /boot/grub/) and /home will on this partition too.
1.00 MiB of empty space at the end of the disk.

* See my earlier note regarding hibernation.

BIOS-GPT

If you have partitioned a large drive correctly then a computer with BIOS firmware will be able to access partitions larger than 2TiB. One of my computers has BIOS firmware only but can access its 3TiB GPT-partitioned HDDs.

Option 1

This is my preferred option because I can edit /etc/fstab and specify that /boot must not be mounted at boot, thus reducing the possibility of the files in /boot getting corrupted.

BIOS-GPT Option 1
1.00 MiB of empty space at the beginning of the disk.
/dev/sda1 This is the BIOS boot partition
Size: 1.00 MiB
File system: Unformatted
Flags: bios_grub
Code: EF02
Label: Not applicable
Mount point: Not applicable
/dev/sda2
Size: 512 MiB
File system: ext2
Flags: None
Code: 8300
Label: BOOT
Mount point: /boot
/dev/sda3
Size: 16 GiB for a computer with 16 GiB of RAM*
File system: linux-swap
Flags: None
Code: 8200
Label: SWAP
Mount point: None
/dev/sda4
Size: e.g. 64 GiB (128 GiB if the drive is big)
File system: ext4
Flags: None
Code: 8300
Label: ROOT
Mount point: / (root) Therefore /boot and (/boot/grub/) will on this partition too.
/dev/sda5
Size: 1.00 MiB less than the remaining space on the disk
File system: ext4
Flags: None
Code: 8300
Label: HOME
Mount point: /home
1.00 MiB of empty space at the end of the disk.

* See my earlier note regarding hibernation.

Option 2

You could use this scheme if you are not interested in having /boot on its own partition.

BIOS-GPT Option 2
1.00 MiB of empty space at the beginning of the disk.
/dev/sda1 This is the BIOS boot partition
Size: 1.00 MiB
File system: Unformatted
Flags: bios_grub
Code: EF02
Label: Not applicable
Mount point: Not applicable
/dev/sda2
Size: 16 GiB for a computer with 16 GiB of RAM*
File system: linux-swap
Flags: None
Code: 8200
Label: SWAP
Mount point: None
/dev/sda3
Size: e.g. 64 GiB (128 GiB if the drive is big)
File system: ext4
Flags: None
Code: 8300
Label: ROOT
Mount point: / (root) Therefore /boot (and /boot/grub/) will on this partition too.
/dev/sda4
Size: 1.00 MiB less than the remaining space on the disk
File system: ext4
Flags: None
Code: 8300
Label: HOME
Mount point: /home
1.00 MiB of empty space at the end of the disk.

* See my earlier note regarding hibernation.

Option 3

You could use this scheme if you are not interested in having /boot and /home on their own partitions.

BIOS-GPT Option 3
1.00 MiB of empty space at the beginning of the disk.
/dev/sda1 This is the BIOS boot partition
Size: 1.00 MiB
File system: Unformatted
Flags: bios_grub
Code: EF02
Label: Not applicable
Mount point: Not applicable
/dev/sda2
Size: 16 GiB for a computer with 16 GiB of RAM*
File system: linux-swap
Flags: None
Code: 8200
Label: SWAP
Mount point: None
/dev/sda3
Size: 1.00 MiB less than the remaining space on the disk
File system: ext4
Flags: None
Code: 8300
Label: ROOT
Mount point: / (root) Therefore /boot and /home will on this partition too.
1.00 MiB of empty space at the end of the disk.

* See my earlier note regarding hibernation.

BIOS-MBR

Option 1

This is my preferred option because I can edit /etc/fstab and specify that /boot must not be mounted at boot, thus reducing the possibility of the files in /boot getting corrupted.

BIOS-MBR Option 1
1.00 MiB of empty space at the beginning of the disk.
/dev/sda1
Size: 512 MiB
Type: Primary
File system: ext2
Flags: boot
Label: BOOT
Mount point: /boot
/dev/sda2
Size: 16 GiB for a computer with 16 GiB of RAM*
Type: Primary
File system: linux-swap
Flags: None
Label: SWAP
Mount point: None
/dev/sda3
Size: e.g. 64 GiB (128 GiB if the drive is big)
File system: ext4
Flags: None
Label: ROOT
Mount point: / (root)
/dev/sda4
Size: remaining space on the disk
Type: Primary
File system: ext4
Flags: None
Label: HOME
Mount point: /home

* See my earlier note regarding hibernation.

Option 2

You could use this scheme if you are not interested in having /boot on its own partition.

BIOS-MBR Option 2
1.00 MiB of empty space at the beginning of the disk.
/dev/sda1
Size: 16 GiB for a computer with 16 GiB of RAM*
Type: Primary
File system: linux-swap
Flags: None
Label: SWAP
Mount point: None
/dev/sda2
Size: e.g. 64 GiB (128 GiB if the drive is big)
Type: Primary
File system: ext4
Flags: boot
Label: ROOT
Mount point: / (root) Therefore /boot will be on this partition too.
/dev/sda3
Size: remaining space on the disk
Type: Primary
File system: ext4
Flags: None
Label: HOME
Mount point: /home

* See my earlier note regarding hibernation.

Option 3

You could use this scheme if you are not interested in having /boot and /home on their own partitions.

BIOS-MBR Option 3
1.00 MiB of empty space at the beginning of the disk.
/dev/sda1
Size: 16 GiB for a computer with 16 GiB of RAM*
Type: Primary
File system: linux-swap
Flags: None
Label: SWAP
Mount point: None
/dev/sda2
Size: remaining space on the disk
Type: Primary
File system: ext4
Flags: boot
Label: ROOT
Mount point: / (root) Therefore /boot and /home will be on this partition too.

* See my earlier note regarding hibernation.

Option 4

If you want to have more than four partitions — let’s say you wanted to have a separate NTFS partition, for example — you would need to use an Extended Partition.

BIOS-MBR Option 4
1.00 MiB of empty space at the beginning of the disk.
/dev/sda1
Size: 512 MiB
Type: Primary
File system: ext2
Flags: boot
Label: BOOT
Mount point: /boot
/dev/sda2
Size: 16 GiB for a computer with 16 GiB of RAM*
Type: Primary
File system: linux-swap
Flags: None
Label: SWAP
Mount point: None
/dev/sda3
Size: Remainder of disk
Type: Extended
File system: Not applicable
Flags: None
Label: Not applicable
Mount point: Not applicable
/dev/sda4
Will not exist
/dev/sda5
Size: e.g. 128GiB
Type: Logical
File system: ext4
Flags: None
Label: ROOT
Mount point: / (root)
/dev/sda6
Size: e.g. 256GiB
Type: Logical
File system: ext4
Flags: None
Label: HOME
Mount point: /home
/dev/sda7
Size: remaining space on the disk
Type: Logical
File system: NTFS
Flags: None
Label: NTFS
Mount point: /media/NTFS

* See my earlier note regarding hibernation.

Below is an example of the above scheme (this happens to be the scheme on my main laptop):

root # fdisk -l /dev/sda
Disk /dev/sda: 698.7 GiB, 750156374016 bytes, 1465149168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x291ba0e7

Device     Boot     Start        End    Sectors   Size Id Type
/dev/sda1            2048     264191     262144   128M 83 Linux
/dev/sda2          264192   33822719   33558528    16G 82 Linux swap / Solaris
/dev/sda3        33822720 1465147391 1431324672 682.5G  5 Extended
/dev/sda5        33824768  302260223  268435456   128G 83 Linux
/dev/sda6       302262272  839133183  536870912   256G 83 Linux
/dev/sda7       839135232 1465147391  626012160 298.5G  7 HPFS/NTFS/exFAT

root # lsblk -o NAME,TYPE,SIZE,FSTYPE,MOUNTPOINT,LABEL,PARTFLAGS /dev/sda
NAME   TYPE   SIZE FSTYPE  MOUNTPOINT  LABEL PARTFLAGS
sda    disk 698.7G                           
├─sda1 part   128M ext2    /boot       BOOT  
├─sda2 part    16G swap    [SWAP]      SWAP  
├─sda5 part   128G ext4    /           ROOT  
├─sda6 part   256G ext4    /home       HOME  
└─sda7 part 298.5G ntfs-3g /media/NTFS NTFS

Notice that the boot flag is not set. Nevertheless the laptop boots fine.

Command-line tools

Below are examples of command-line utilities parted, gdisk and fdisk examining a GPT-partitioned HDD and an MBR-partitioned HDD. The original fdisk utility predates the invention of GPT, but the latest versions of fdisk understand the GPT design (you can check this by using the command ‘man fdisk‘). Personally, to partition GPT HDDs from the command line I would use parted or gdisk before fdisk, and to partition MBR HDDs from the command line I would use parted or fdisk before gdisk. Mind you, I like an easy life and so I tend to use the GUI tools GParted or KDE Partition Manager to partition and format an HDD for Linux.

GPT-partitioned HDD

root # parted /dev/sda print
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sda: 68.7GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system     Name       Flags
 1      2097kB  539MB   537MB   fat32                      boot, esp
 2      539MB   1076MB  537MB   ext2            /boot
 3      1076MB  3223MB  2147MB  linux-swap(v1)  linuxswap
 4      3223MB  37.6GB  34.4GB  ext4            /
 5      37.6GB  68.7GB  31.1GB  ext4            /home

root # gdisk -l /dev/sda
GPT fdisk (gdisk) version 1.0.1

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 134217728 sectors, 64.0 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 9807AF0F-8BD5-4727-A3CD-9995B2705732
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 134217694
Partitions will be aligned on 2048-sector boundaries
Total free space is 8125 sectors (4.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            4096         1052671   512.0 MiB   EF00  
   2         1052672         2101247   512.0 MiB   8300  /boot                                                                    
   3         2101248         6295551   2.0 GiB     8200  linuxswap                                                                
   4         6295552        73404415   32.0 GiB    8300  /                                                                        
   5        73404416       134213631   29.0 GiB    8300  /home                                                                    
root # fdisk -l /dev/sda                                                                                           
Disk /dev/sda: 64 GiB, 68719476736 bytes, 134217728 sectors                                                                       
Units: sectors of 1 * 512 = 512 bytes                                                                                             
Sector size (logical/physical): 512 bytes / 512 bytes                                                                             
I/O size (minimum/optimal): 512 bytes / 512 bytes                                                                                 
Disklabel type: gpt                                                                                                               
Disk identifier: 9807AF0F-8BD5-4727-A3CD-9995B2705732                                                                             
                                                                                                                                  
Device        Start       End  Sectors  Size Type                                                                                 
/dev/sda1      4096   1052671  1048576  512M EFI System
/dev/sda2   1052672   2101247  1048576  512M Linux filesystem
/dev/sda3   2101248   6295551  4194304    2G Linux swap
/dev/sda4   6295552  73404415 67108864   32G Linux filesystem
/dev/sda5  73404416 134213631 60809216   29G Linux filesystem

MBR-partitioned HDD

root # parted /dev/sda print
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sda: 68.7GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End     Size    Type     File system     Flags
 1      1049kB  538MB   537MB   primary  ext2            boot
 2      538MB   2685MB  2147MB  primary  linux-swap(v1)
 3      2685MB  37.0GB  34.4GB  primary  ext4
 4      37.0GB  68.7GB  31.7GB  primary  ext4

root # gdisk -l /dev/sda
GPT fdisk (gdisk) version 1.0.1

Partition table scan:
  MBR: MBR only
  BSD: not present
  APM: not present
  GPT: not present


***************************************************************
Found invalid GPT and valid MBR; converting MBR to GPT format
in memory. 
***************************************************************


Warning! Secondary partition table overlaps the last partition by
33 blocks!
You will need to delete this partition or resize it in another utility.
Disk /dev/sda: 134217728 sectors, 64.0 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): F6E9E53E-33BE-44CB-BEFC-D93E03B79B84                                                   
Partition table holds up to 128 entries                                                                        
First usable sector is 34, last usable sector is 134217694                                                     
Partitions will be aligned on 2048-sector boundaries                                                           
Total free space is 2014 sectors (1007.0 KiB)                                                                  

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048         1050623   512.0 MiB   8300  Linux filesystem
   2         1050624         5244927   2.0 GiB     8200  Linux swap
   3         5244928        72353791   32.0 GiB    8300  Linux filesystem
   4        72353792       134217727   29.5 GiB    8300  Linux filesystem
root # fdisk -l /dev/sda
Disk /dev/sda: 64 GiB, 68719476736 bytes, 134217728 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x7b9f1623

Device     Boot    Start       End  Sectors  Size Id Type
/dev/sda1  *        2048   1050623  1048576  512M 83 Linux
/dev/sda2        1050624   5244927  4194304    2G 82 Linux swap / Solaris
/dev/sda3        5244928  72353791 67108864   32G 83 Linux
/dev/sda4       72353792 134217727 61863936 29.5G 83 Linux

Notice the gdisk warning that the last partition would not be viable on a GPT-partitioned HDD because there would not be space for the Secondary GPT. However, as this is an MBR-partitioned HDD for use with a BIOS firmware computer, that does not apply.

A warning note if installing Ubuntu on a UEFI computer

When I installed Ubuntu Desktop 16.04.1 in a virtual machine I encountered a bug (in VirtualBox? in the Ubuntu Installer?) which results in subsequent reboots leaving the virtual machine running the UEFI Shell instead of launching GRUB and booting Ubuntu. To fix that, I did the following as soon as Ubuntu booted from the virtual HDD for the first time after I had clicked ‘Restart Now’ in the ‘Installation Complete’ window:

user $ sudo su
root # mount /dev/sda1 /mnt
root # cd /mnt
root # echo "\EFI\ubuntu\grubx64.efi" > startup.nsh
root # cd
root # umount /dev/sda1
root # exit

The partition /dev/sda1 being the FAT32 ESP in this case.

This creates the file /boot/efi/startup.nsh required by the UEFI Shell. Then you can reboot and all should work as intended, i.e. the UEFI machine boots to the UEFI Shell which, after a few seconds, launches GRUB.

UPDATE (16 February 2017): A downside to the above approach is the 5-second timeout in the UEFI Shell until startup.nsh is launched. Below is an alternative that avoids having to wait for the 5-second countdown in the UEFI Shell until startup.nsh is launched. The downside is that you must remember to repeat this procedure if a new version of GRUB is installed and there is a new version of /EFI/ubuntu/grubx64.efi.

user $ sudo su
root # mount /dev/sda1 /mnt
root # cd /mnt/EFI/
root # ls
ubuntu
root # ls ubuntu
fw  fwupx64.efi  grub.cfg  grubx64.efi  MokManager.efi  shimx64.efi
root # mkdir BOOT
root # cp ubuntu/grubx64.efi BOOT/BOOTX64.EFI
root # cd
root # umount /dev/sda1
root # exit

See the Arch Linux Wiki article VirtualBox – 2.1 Installation in EFI mode.

A warning note if installing Gentoo on a UEFI computer

To date, the Gentoo Minimal Installation CD does not support UEFI, and therefore I recommend that you use SystemRescueCd instead. It is a Gentoo-based LiveCD with several tools useful for installing Gentoo, including a Web browser so you can access the on-line Gentoo documentation during installation. You can follow the Gentoo Installation Guide verbatim. SystemRescueCd also has a Desktop Environment with various GUI utilities, so you could partition the HDD using GParted instead of the command-line utilities if you wish. SystemRescueCd also comes with various wireless network card drivers and a network manager, so there is a good chance you will be able to connect easily to a wireless network if you prefer (I once used SystemRescueCd to install Gentoo on a laptop using Wi-Fi alone, as a wired connection was not convenient at the time).

How can I find out in Linux if a computer has booted using UEFI?

Check if the RAM-based directory /sys/firmware/efi/ exists. If it does not exist, the computer did not boot using UEFI.

Further Reading

  1. Wikipedia – BIOS
  2. Wikipedia – Master boot record
  3. UEFI Specification Version 2.6, January 2016
  4. Wikipedia – Unified Extensible Firmware Interface
  5. Wikipedia – GUID Partition Table
    Also, the diagram GUID Partition Table Scheme in that article is quite helpful in understanding the layout of a GPT-partitioned HDD.
  6. Wikipedia – BIOS boot partition
    Also, the diagram GNU GRUB 2 in that article is quite helpful in understanding: a) MBR partitioning; b) how GRUB 2 fits on an MBR-partitioned HDD; c) GPT partitioning; d) how GRUB 2 fits on a GPT-partitioned HDD; e) where the BIOS boot partition fits on a GPT-partitioned HDD and how it is used by GRUB 2.
  7. GNU GRUB Manual – 3.4 BIOS installation
  8. Ubuntu Documentation – UEFI
  9. Gentoo Installation Guide – Preparing the disks
  10. Ubuntu Bug Reports – Bug No. 811485 – Ubuntu partman-efi package – EFI SYSTEM PARTITION should be atleast 100 MiB size and formatted as FAT32, not FAT16
  11. Roderick W. Smith’s Web Page

Using an external USB 3.5-inch floppy disk drive in Linux

Back in 2004 I needed to get some files off my old 3.5″ floppy disks, so I bought an external USB floppy disk drive to use with a laptop running Windows XP. The label on the drive gives the manufacturer and model as ‘SmartDisk: FDUSB-TM2, Mitsumi Model #: D353FUE’.

Anyway, today I wanted to throw out some 720KB DD (Double Density) and 1440KB HD (High Density) 3.5″ floppy disks but first needed to check their contents and wipe them. So I dug out the SmartDisk USB drive to see if it would work with the current Gentoo Linux installation on my newest laptop. I was pleased to discover that it does, and below are some notes on how to use it in case anyone else needs to use one of these devices.

Once plugged in to a USB port on my laptop, the lsusb command shows the device has been recognised:

Bus 001 Device 013: ID 03ee:6901 Mitsumi SmartDisk FDD

Note that the Linux floppy driver is not needed for USB floppy disk drives:

root # grep -i CONFIG_BLK_DEV_FD /usr/src/linux/.config
# CONFIG_BLK_DEV_FD is not set

A Linux utility named ufiformat is used to low-level format floppy disks in USB floppy disk drives. A Gentoo Linux ebuild for Version 0.9.9 of ufiformat is listed below, and it can be used in a local overlay under the category sys-fs:

# Copyright 1999-2014 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

EAPI=5

DESCRIPTION="USB Floppy Disk formatting tool"
HOMEPAGE="http://www.geocities.jp/tedi_world/format_usbfdd_e.html"
SRC_URI="http://www.geocities.jp/tedi_world/${P}.tar.gz"

LICENSE="GPL-2"
SLOT="0"
KEYWORDS="~amd64 ~x86"
IUSE=""

RDEPEND="sys-fs/e2fsprogs"
DEPEND=${RDEPEND}

UPDATE (2 November 2019): As of March 31, 2019, Yahoo! Geocities service has been discontinued, so the ufiformat repository has been moved to GitHub. I have updated the link in the paragraph above accordingly. Therefore, in the ebuild the HOMEPAGE and SRC_URI variables need to be changed – see the update at the end of this blog post.

The ufiformat utility is straightforward to use:

root # ufiformat --help
Usage: ufiformat [OPTION]... [DEVICE]
Format a floppy disk in a USB floppy disk DEVICE.

  -f, --format [SIZE]  specify format capacity SIZE in KB
                       without -f option, the format of the current media will be used
  -V, --verify         verify the medium after formatting
  -F, --force          do not perform any safety checks
  -i, --inquire        show device information, instead of performing format
                       without DEVICE argument, list USB floppy disk devices
  -v, --verbose        show detailed output
  -q, --quiet          suppress minor output
  -h, --help           show this message

To find the device name, use the blkid command before plugging in the USB cable and again after plugging in the USB cable. The extra device listed the second time will be the floppy disk drive. For example, in my case the new line at the end of the blkid output indicated the drive was /dev/sdd:

/dev/sdd: SEC_TYPE="msdos" UUID="BBBA-37AF" TYPE="vfat"

The fdisk command will confirm that the device is the floppy drive:

root # fdisk -l /dev/sdd
Disk /dev/sdd: 720 KiB, 737280 bytes, 1440 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Note that /dev/sdd will not be listed in the output of blkid if there is no disk in the floppy drive, although the ls command will list /dev/sdd while the drive is connected to the computer.

Notice that there are no devices /dev/fd0, /dev/fd1, /dev/fd2 and /dev/fd3, etc. This does not matter.

root # ls /dev
audio1           dri      i2c-10   kmem          mapper              nvidiactl  sda1       sequencer2  tty0   tty2   tty30  tty41  tty52  tty63    usbmon3     vcs2    vcsa12
autofs           dsp1     i2c-11   kmsg          mcelog              nvram      sda2       sg0         tty1   tty20  tty31  tty42  tty53  tty7     usbmon4     vcs3    vcsa2
block            fb0      i2c-2    log           mem                 pktcdvd    sda3       sg1         tty10  tty21  tty32  tty43  tty54  tty8     v4l         vcs4    vcsa3
bsg              fd       i2c-3    loop-control  memory_bandwidth    port       sda5       sg2         tty11  tty22  tty33  tty44  tty55  tty9     vboxdrv     vcs5    vcsa4
bus              full     i2c-4    loop0         mixer               ptmx       sda6       sg3         tty12  tty23  tty34  tty45  tty56  ttyS0    vboxdrvu    vcs6    vcsa5
char             fuse     i2c-5    loop1         mixer1              pts        sda7       shm         tty13  tty24  tty35  tty46  tty57  ttyS1    vboxnetctl  vcs7    vcsa6
console          hidraw0  i2c-6    loop2         mqueue              random     sdb        snapshot    tty14  tty25  tty36  tty47  tty58  ttyS2    vboxusb     vcs8    vcsa7
core             hidraw1  i2c-7    loop3         network_latency     rfkill     sdb1       snd         tty15  tty26  tty37  tty48  tty59  ttyS3    vcs         vcs9    vcsa8
cpu              hidraw2  i2c-8    loop4         network_throughput  root       sdc        stderr      tty16  tty27  tty38  tty49  tty6   urandom  vcs1        vcsa    vcsa9
cpu_dma_latency  hpet     i2c-9    loop5         null                rtc        sdc1       stdin       tty17  tty28  tty39  tty5   tty60  usbmon0  vcs10       vcsa1   vga_arbiter
cuse             i2c-0    initctl  loop6         nvidia-modeset      rtc0       sdd        stdout      tty18  tty29  tty4   tty50  tty61  usbmon1  vcs11       vcsa10  video0
disk             i2c-1    input    loop7         nvidia0             sda        sequencer  tty         tty19  tty3   tty40  tty51  tty62  usbmon2  vcs12       vcsa11  zero
root # ls -la /dev/fd
lrwxrwxrwx 1 root root 13 Jan 17 02:13 /dev/fd -> /proc/self/fd
root # ls -la /proc/self/fd
total 0
dr-x------ 2 root root  0 Jan 17 04:26 .
dr-xr-xr-x 8 root root  0 Jan 17 04:26 ..
lrwx------ 1 root root 64 Jan 17 04:26 0 -> /dev/pts/1
lrwx------ 1 root root 64 Jan 17 04:26 1 -> /dev/pts/1
lrwx------ 1 root root 64 Jan 17 04:26 2 -> /dev/pts/1
lr-x------ 1 root root 64 Jan 17 04:26 3 -> /proc/17669/fd

To format an HD floppy disk with the FAT file system, I did the following:

root # ufiformat -f 1440 /dev/sdd
geometry: track=80, head=2, sector=18, block=512
done                                   
root # /usr/sbin/mkfs.vfat /dev/sdd
mkfs.fat 4.0 (2016-05-06)
attribute "partition" not found
root # ufiformat -i /dev/sdd
vendor:  MITSUMI
product: USB FDD
write protect: off
media type: 2HD
status      block size   kb
formatted    2880  512 1440
formattable  2880  512 1440
formattable  1232 1024 1232
formattable  2400  512 1200

To format a DD floppy disk with the FAT file system, I did the following:

root # ufiformat -f 720 /dev/sdd
geometry: track=80, head=2, sector=9, block=512
done                                   
root # /usr/sbin/mkfs.vfat /dev/sdd
mkfs.fat 4.0 (2016-05-06)
attribute "partition" not found
root # ufiformat -i /dev/sdd
vendor:  MITSUMI
product: USB FDD
write protect: off
media type: 2DD
status      block size   kb
formatted    1440  512  720
formattable  1440  512  720

I use the KDE Desktop Environment. The Device Notifier widget in the System Tray shows the drive and — once a formatted floppy disk is in the drive — it is possible to use the Device Notifier to mount, open and unmount the floppy disk. However, it is also possible to use the command line:

root # mkdir /mnt/floppy
root # mount /dev/sdd /mnt/floppy
root # ls /mnt/floppy
root # cp /test.txt /mnt/floppy/
root # ls /mnt/floppy
test.txt
root # umount /dev/sdd

Earlier in this post I showed examples of formatting floppy disks using the FAT file system, but it is of course possible to format them using other file systems, such as:

root # mkfs.ext2 /dev/sdd
mke2fs 1.43.3 (04-Sep-2016)
/dev/sdd contains a vfat file system
Proceed anyway? (y,n) y
Creating filesystem with 720 1k blocks and 96 inodes

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done

Anyway, I was able to check the contents of the floppies and wipe them before disposing of them. It’s good to know that some old technologies can still be used when needs be. I won’t be throwing out the old floppy disk drive just yet.

UPDATE (2 November 2019): As of March 31, 2019, Yahoo! Geocities service has been discontinued, so the ufiformat repository has been moved to GitHub. Therefore, in the ebuild the HOMEPAGE and SRC_URI variables need to be changed from:

HOMEPAGE="http://www.geocities.jp/tedi_world/format_usbfdd_e.html"
SRC_URI="http://www.geocities.jp/tedi_world/${P}.tar.gz"

to:

HOMEPAGE="https://github.com/tedigh/ufiformat/"
SRC_URI="https://github.com/tedigh/ufiformat/archive/v0.9.9.zip"

or:

HOMEPAGE="https://github.com/tedigh/ufiformat/"
SRC_URI="https://github.com/tedigh/ufiformat/archive/v{$PV}.zip"

Another look at beeps in Linux

Following my previous post I experimented further with the Linux Kernel configuration options for event beeps (sometimes called ‘system beeps’), and I now have a better understanding of how the Kernel options interact (on one of my laptops, at least).

The sound card in my Clevo W230SS laptop has a VIA VT1802S audio codec chip. I looked at the audio circuit schematic in the service manual; one of the digital input pins on the VT1802S is labelled ‘PCBEEP’, and one of its analogue output pins is labelled ‘PCBEEP’ and is connected to the laptop’s speaker circuit. So there is no PC Speaker in this laptop and it emulates the PC Speaker via the laptop’s sound card, as mentioned in my previous post.

Before I describe my latest results, there are a couple of influencing factors I forgot to mention in my previous post:

  • In some computers the BIOS Menu has one or more options for enabling/disabling beeps. The BIOS menu of my Clevo laptop does not have an option to enable/disable all beeps from the (emulated) PC Speaker, but it does have a couple of options to enable/disable ‘Power On Boot Beep’ and ‘Battery Low Alarm Beep’ (I have disabled them both). Anyway, if you are still not getting beeps after trying everything else, be sure to check the BIOS menu just in case it has an option to enable/disable the PC Speaker.

  • Make sure that bell-style is not set to ‘none‘ (you could set it to ‘audible‘ if you wanted to be sure):

    root # grep bell /etc/inputrc
    # do not bell on tab-completion
    #set bell-style none

The Kernel configuration was initially as shown below. With this configuration no beeps were emitted in a VT (Virtual Terminal) or in an X Windows terminal. As explained in my previous post, I therefore configured the XKB Event Daemon to play an audio file (bell.oga) whenever X Windows detects a BEL character (ASCII 007) or Backspace key (ASCII 008).

root # grep PCSP /usr/src/linux/.config
CONFIG_HAVE_PCSPKR_PLATFORM=y
CONFIG_PCSPKR_PLATFORM=y
# CONFIG_INPUT_PCSPKR is not set
# CONFIG_SND_PCSP is not set
root # grep BEEP /usr/src/linux/.config
CONFIG_SND_HDA_INPUT_BEEP=y
CONFIG_SND_HDA_INPUT_BEEP_MODE=1

Then I rebuilt the Kernel with CONFIG_INPUT_PCSPKR=M and CONFIG_SND_PCSP=M:

root # cd /usr/src/linux
root # mount /dev/sda1 /boot
root # make menuconfig
root # make && make modules_install
root # make install
root # grep PCSP /usr/src/linux/.config
CONFIG_HAVE_PCSPKR_PLATFORM=y
CONFIG_PCSPKR_PLATFORM=y
CONFIG_INPUT_PCSPKR=m
CONFIG_SND_PCSP=m
root # grep BEEP /usr/src/linux/.config
CONFIG_SND_HDA_INPUT_BEEP=y
CONFIG_SND_HDA_INPUT_BEEP_MODE=1

Then I created the file /etc/modprobe.d/blacklist.conf in order to blacklist the modules pcspkr and snd-pcsp so that only I could load them after boot:

root # cat /etc/modprobe.d/blacklist.conf
blacklist pcspkr
blacklist snd-pcsp

Then I added the line ‘options snd-pcsp index=2‘ to the file /etc/modprobe.d/alsa.conf so that the virtual sound card pcsp would not become the default sound card:

root # tail /etc/modprobe.d/alsa.conf
alias /dev/midi snd-seq-oss

# Set this to the correct number of cards.
options snd cards_limit=1

# See https://bugs.launchpad.net/ubuntu/+source/alsa-driver/+bug/1313904
options snd-hda-intel patch=,clevo-hda-patch

# See Kernel Help text for CONFIG_SND_PCSP
options snd-pcsp index=2

Then I rebooted and checked that neither module was loaded:

root # lsmod | grep pcsp
root # echo -e '\a'

root #

As neither module was loaded, the situation was the same as before: a) no beep in a VT; b) no beep in Konsole/Yakuake (I will ignore KDE terminal programs anyway because of KDE bug report no. 177861);* c) the same bell.oga beep in xterm due to my use of xkbevd; d) no changes in ALSA Mixer.

* Regarding Konsole and Yakuake, see my update of October 9, 2016 at the bottom of this post.

Then I loaded the module pcspkr:

root # modprobe pcspkr
root # lsmod | grep pcsp
pcspkr                  1875  0
root # echo -e '\a'

root #

There were no changes in ALSA Mixer. But now the BEL character and Backspace in a VT did result in a beep (I’ll call this a ‘pcbeep’ to distinguish it from the different-sounding beep produced using bell.oga). There was the usual bell.oga beep in xterm due to my use of xkbevd. If I stopped xkbevd, there was no pcbeep in X Windows from the shell commands shown in my previous post, although the following commands from any terminal in X Windows (even Konsole/Yakuake) did emit a pcbeep:

user $ sudo sh -c "echo -e '\a' > /dev/console"

user $ sudo sh -c "tput bel > /dev/console"

root # echo -e '\a' > /dev/console

root # tput bel > /dev/console

Then I unloaded the module pcspkr and loaded the module snd-pcsp:

root # modprobe -r pcspkr
root # modprobe snd-pcsp
root # lsmod | grep pcsp
snd_pcsp                7918  1
root # echo -e '\a'

root #

ALSA Mixer showed a new sound card named ‘pcsp‘ (Sound Card 2) with three channels: ‘Master’, ‘Beep’ and ‘BaseFRQ’. I could mute/unmute ‘Beep’ by pressing ‘M’ on the keyboard as usual, and I could toggle ‘BaseFRQ’ between two values:18643 and 37286. The BEL character and Backspace in a VT resulted in a pcbeep. There was the usual bell.oga beep in xterm due to my use of xkbevd. If I stopped xkbevd, there was no pcbeep in X Windows from the shell commands shown in my previous post, although the following commands from any terminal in X Windows (even Konsole/Yakuake) did emit a pcbeep:

user $ sudo sh -c "echo -e '\a' > /dev/console"

user $ sudo sh -c "tput bel > /dev/console"

root # echo -e '\a' > /dev/console

root # tput bel > /dev/console

Muting ‘Beep’ in ALSA Mixer did not mute the bell.oga beeps in X Windows, but it did mute the pcbeeps in the VTs.

Unlike the situation with the pcspkr module, occasionally there were brief low-volume crackles and pops from the laptop’s speakers.

So both drivers worked, but pcspkr performed better, although it could not be muted via ALSA Mixer. My recommendation to use pcspkr rather than snd-pcsp still stands.

Unlike pcspkr, I had to force the unloading of snd-pcsp:

root # modprobe -r snd-pcsp
modprobe: FATAL: Module snd_pcsp is in use.
root # rmmod -f snd_pcsp
root #

I then removed the Kernel’s ‘digital beep’ interface for the Intel HDA driver by rebuilding the Kernel with CONFIG_SND_HDA_INPUT_BEEP=N:

root # cd /usr/src/linux
root # mount /dev/sda1 /boot
root # make menuconfig
root # make && make modules_install
root # make install
root # grep PCSP /usr/src/linux/.config
CONFIG_HAVE_PCSPKR_PLATFORM=y
CONFIG_PCSPKR_PLATFORM=y
CONFIG_INPUT_PCSPKR=m
CONFIG_SND_PCSP=m
root # grep BEEP /usr/src/linux/.config
# CONFIG_SND_HDA_INPUT_BEEP is not set
root #

After I rebooted, the behaviour was exactly the same as for CONFIG_SND_HDA_INPUT_BEEP=Y and CONFIG_SND_HDA_INPUT_BEEP_MODE=1.

So, there you have it. I believe my previous post was essentially correct regarding the functional design of the Kernel options. If you have a computer without a PC Speaker but it emulates one via the computer’s sound card, you have to set either CONFIG_INPUT_PCSPKR or CONFIG_SND_PCSP to get a beep in a VT, not set just CONFIG_SND_HDA_INPUT_BEEP and CONFIG_SND_HDA_INPUT_BEEP_MODE. However, even when my laptop emits beeps in a VT from the (emulated) PC Speaker, no beeps from the (emulated) PC Speaker are emitted in X Windows unless the user is the root user and the output is redirected to /dev/console. So, if you want to emit beeps in X Windows it is still better in my opinion to use xkbevd to play an audio file of a beep, as described in my previous post.

Update (October 9, 2016): Regarding KDE’s terminal applications emitting beeps, I am currently using KDE Plasma 5.7.5 and have been able to configure Konsole and Yakuake to play an audio file of a beep (as opposed to emitting a pcbeep) as follows:

  • In Konsole, click on ‘Settings’ > ‘Configure Notifications…’, select ‘Bell in Visible Session’ and ensure ‘Play a sound’ is ticked and a file is specified there (I specify /usr/share/sounds/freedesktop/stereo/bell.oga). If you wish, do the same for ‘Bell in Non-Visible Session’.
  • For Yakuake, press F12 to display the Yakuake window, click on the ‘Open Menu’ icon, select ‘Configure Notifications…’, select ‘Bell in Visible Session’ and ensure ‘Play a sound’ is ticked and a file is specified there (I specify /usr/share/sounds/freedesktop/stereo/bell.oga). If you wish, do the same for ‘Bell in Non-Visible Session’.