This post contains some notes for my class Digital-age tools for research. It is more a guide line for beginners. It has been divided in two parts. The first one covers some basic knowledge about Linux and operating systems. The second is available here and it is about the use of Linux and bash.

This is supposed to be a starting point for people who are approaching the use of Linux and HPC for scientific purposes. Several concept are over simplified to the point to be formally wrong to the eyes of more expert people. I apologize. Comments and errors reports are welcome.

Introduction

NOTES:

  1. Read all the notes
  2. If something is written as this it means you can use it as command in bash.
  3. Challenging but needed topic are indicated by: ⚠
  4. Challenging, but not essential topics are indicated by ☣ ; they can be skipped or read in future
  5. AT THE BOTTOM of Part -2 there is an exercise. You should be able to do it reading this primer
  6. Read (or skim-read) the full PRIMER before trying to fully understand it. Since this is a quick draft, some concepts are repeated, and others are fully explained only after their first appearance in the text. This is very true with respect to the explanation of conventions. The example section will make everything more clear later on.
  7. Bold is used to indicate the letter that are part of acronyms
  8. Most tchnical word are linked to external sources. In some cases if you hoover over the mouse you can read a quick definition. [Example]( “This is an example of definition, without link”)

Basics

  • Difference binaries vs ASCII (plain-text) files.
  • Intro to MarkDown as example of scripting
  • MD CheatSheet
  • There are different flavours of MD (but there are only little differences between them. Examples: Wiki MD, GitHub MD, Dillinger MD, etc)
  • I also suggest StackEdit which is integrated in Google

Basic of Architecture

  • [Software]( “Ypou can scream at”) vs [Hardware]( “You can kick”) vs [Firmware]( “Inside the chips”) vs [BIOS]( “It allows the HW to start the OS”)
  • Kernel Linux vs M$ vs AppleOS
  • ☣ Von Neumann Architecture (I suggest to check who Von Neumann was, one of the greatest, if not the greatest mind of 1900)
  • Virtual Machine. Some free VM:
    • VMware player
    • VirtualBox
    • Alternative to VM is putty or

Linux

WHICH LINUX:

I suggest any Long Term Support distribution. Latest LTS released for Ubuntu is: Ubuntu 20.04 Focal Fossa

  • Version are usually named as [Animal]+[Adjective starting with the same letter as the animal]. We are now at the second iteration of the alphabet.
  • Linux Kernel is the base for several Linux DISTROs (distributions).
  • ☣ Distros are divided in families, for example: Debian (user friendly), Slackware (unfriendly) and Red Hat (business oriented). These are the main three, then there is Arch and Enoch (father of Gentoo considered one of the best). The most popular operating system ever, despite no-one usually think of it as a Linux operating system (OS): ANDROID.
    I suggest to use Ubuntu, which may not be the best but:
  • very common
  • very easy to find support/examples

Equivalent alternatives are KUbuntu (KDE desktop similar to windows) and XUbuntu (lighter software, for old PCs) or Scibuntu (A flavour of Ubuntu with plenty of Science Packages.).
If you use VM, i suggest to test couple of versions just for fun. Alternative to Ubuntu are: Fedora, Mint, Puppy (runs from USB)
-☣ Genealogy tree of Distros available (Updated April 2020) as you can see there are many versions of ubuntu (the copyright of the images are specified on the image itself, taken from wikicommons): A vectorial version is here.

From now on I assume you are using/will use Ubuntu for simplicity.

Install

☣ Installing Linux as dual boot

  • This is the best, yet the most challenging option
  • Dual boot means you will have at the same time windows and linux. If you do not know what you are doing you may brick your pc.
  • You need to know very well what is a disk partition and know some special partitions:
    • ☣ ☣ ☣ MBR, ☣ ☣ ☣ SWAP.
    • Check how much compatible you are/are-not HERE or HERE.
  • Nowadays most of computers are 100% compatible. If you have any doubt, rather than checking HW piece by HW piece, just try a live version
  • In my experience the most critical issue regards the WIFI controller, so in case keep an Ethernet cable at hand.
  • Critical HW that may work poorly on Linux are: Wi-Fi chipsets, Audio, Video acceleration, ⚠Battery.

Virtual machine

Installing Linux on a VM:

  • a lot easier
  • a bit slower to run if you do not have a top-computer
  • Caveat: reserve at least 30 Gb RAM, be careful on how to share the internet connection

Usage

Once you get Linux running you need to know few things.

  • You can use Linux as a normal graphic OS

  • To handle file/programs usually a Terminal Emulator (consolle, bash, etc). This is an example:

  • Terminal is opened pressing Alt + T

  • We will use the terminal to read files, edit files, launch commands, etc.

Examples

INSTALLING SOFTWARE

  • Some software are preapproved by Ubuntu. They can be installed writing in the terminal: sudo apt install[NAMEOFTHESOFTWARE] (Just recently apt replaced the old command apt-get, so you might find the latter around. Despite this so far both work).

  • To protect the system, the user is forbidden to run some commands or programs.

  • To bypass this you need special permissions called super user domain permission

  • To run a command with special permission you need to put before the command sudo

    • after the command is sent (pressing enter) you need to insert your password
    • This is the reason why you need to write sudo before apt . For the same reason you might need sudo to edit some files that are crucial for the OS.
    • ⚠ !!! BE CAREFUL WHEN YOU USE SUDO !!! Read This:

  • TOP 10 most dangerous line commands

  • ⚠ I recommend to install the following software: terminator, classic gnome:

  • This can be done with the following commands:

    sudo apt-get update
sudo apt-get install gnome-session-fallback 
sudo apt-get install terminator
sudo apt-get install sublime

  • This can be done in a single line:

    sudo apt-get update; sudo apt-get install gnome-session-fallback terminator sublime

The meaning of each line is explained after the # symbol. Every character on a line after # is a comment and it is ignored by the computer.

sudo apt-get update # This line checks if apt-get (or apt) is up to date
sudo apt-get install gnome-session-fallback # This install gnome session fall back a better and lighter Desktop environment for linux  
sudo apt-get install terminator # Install terminator, a good terminal for bash
sudo apt-get install sublime  # Install sublime to edit ascii files

If you need to install several programs you can just list them all together as in the example.

Several commands can be written on the same line substituting enter with ;. Once enter is executed the commands will run sequentially. In this case once sudo apt-get update has finished, the second command sudo apt-get install gnome-session-fallback terminator sublime will start automatically. apt is the easiest way to install software. In general software in installed in the following fashons.

More example on using Linux and bash are available in the second part of this post available here.