Disk space on your SD card: best practices

When you start your career as a Raspberry Pi user you know that you will face at some point the following situations:

• You will play games
• You will try OSes
• You will brag about what you can do on your Pi
• You will try things that are almost impossible. Almost impossible is not impossible.
• You will repartition your SD Card
• You will play games on RetroPie
  
• You will reinstall OSes
• You will run out of SD card disk space

You will eventually run out of SD card disk space and if you are seriously using your Pi, you will have to think to several things on how you manage your storage. Not keeping storage in mind can give you performance and usability issues and if you are caught by surprise you might end having to repartition your SD card and lose some data and some work. Here we are going to browse a few pointers on how to avoid critical disk space situations.

1. Be reasonable in your expectations

First of all, make an inventory of the different storage devices you are going to provide to your Raspberry Pi. 

Here is a quick check-list to help you:

• Your system storage (SD Card) with 4 to 512 GB (or even more)
• USB keys that you find in your drawers and realize that you could put them to good use.
• External hard drive
• Your desktop PC's hard drives, shared via samba
• FTP's are mostly out of scope since you can only download and upload from FTP's. From a samba server you can for example operate an executable.

Following how much GBs you have got, you can consider more or less usage.

Be reasonable:

• For example, don't expect to store lots of games like Final Fantasy VII (4 CDs) if you don't have decent storage like an external hard drive. If you are poor on external storage but have a good hard disk on your desktop PC, you might want to use that hard disk but keep in mind that hard disk+network is the slowest possible storage ever.
• Keep a decent objective with how many OSes you are going to install on your SD card depending on its size. See next paragraphs "2. Think your storage directly when partitioning" and "4. Repartition your storage if needed" for more info on this.
• If you are using multiple partitions, store your data and games at one single place. You can either use a project partition from PINN or access a foreign /home/pi from another OS. No need to store your wineprefixes on your 4 OSes; it will just multiply by 4 needed storage and won't bring any added value.

2. Think your storage directly when partitioning

Each partition will divide your original storage when created. Assuming you have a SD card of 64 GB as a system holder, every partition will be of:

 

- 32 GB if you install 2 systems

- 21 GB if you install 3 systems

- 16 GB if you install 4 systems

- .. 8 GB if you install 8 systems

A system where you manage space carefully can live with around 10GB of disk space. If managing space is not your thing, I would not recommend going below 20 GB per system (in case of a 64GB SD card you will install 3 systems)

3. Prefer lean OSes

If you install multiple OS, think about "base". On what is my OS based? 

Example:

• TwisterOS
• DietPi
• Raspberry Pi OS 32 bits
• Raspberry Pi OS 32 bits lite
• Raspberry Pi OS 32 bits full

All of these 5 are currently based on Raspbian Buster. Raspberry Pi OS Full and TwisterOS are both extremely demanding in terms of disk space. If you need comfort over all, you should install TwisterOS (or Raspberry Pi OS 32 bits full) alone and leave the others.

If you are more into managing your Pi and going in-depth, you might actually prefer 2 or 3 Raspberry Pi OS 32 bits lite + RPD desktop instead of having TwisterOS and Raspberry Pi OS 32 bits full. DietPi is even more sober considering that but you might have compatibility problems with DietPi that you won't meet with Raspberry Pi OS 32 bits lite.

Keep in mind that everytime you will try a compiling adventure you will probably need to redo your config. This is thus important to start on healthy basis. TwisterOS and Raspberry Pi OS 32 bits lite are two diametrically opposed approaches of the same OS.

    3.a. Consider getting a lean Wine version

    Wine bottles are getting huger and huger as newer Wine versions are implemented. If you are not running complicated programs like RPG Maker RGSS1/2/3, you can avoid installing a late wine-x86. 

• Get your lean Wine-x86 from TwisterOS site: https://twisteros.com/faq.html

4. Repartition your storage if needed

    4.a. Backup your /home/pi and your important files before repartitioning

    Depending on your setting, you will have several choices to backup your folders. From easiest to             hardest:

• If you have an external device on your Pi:

          cd /media/pi/externaldevice/save/path
            env GZIP=-9 tar cvzf homepi.tar.gz /home/pi

• You have nothing but your Pi:
• Make sure you are using less than half of your SD card space. You will have to junk useless files. then go to root and tar your directory.
• Do as above:

                sudo su

                cd /root

                env GZIP=-9 tar cvzf homepi.tar.gz /home/pi

                chmod 777 homepi.tar.gz

                cd ..

                chmod 777 /root

                exit

• You can upload your saved homepi.tar.gz to any free cloud service (eg. Google Drive).
• If you have an external windows PC, you can install samba on your Raspberry Pi and pull the files from your /home/pi. See guide there: https://pimylifeup.com/raspberry-pi-samba/

    4.b. Use one single OS for all

    If you are really short on space, you will want to use one single OS. You have a few choices.

• TwisterOS has it all but is quite slow and works mainly on Pi4
• Raspberry Pi OS 32 bits lite and regular are both good alternatives
• You may install kodi inside this OS:

                    sudo apt-get install kodi

• Installing RetroPie is a bit out of scope since RetroPie's compatibility is not assured with Fake-KMS so either you will have to use Lakka/Retroarch or you will use a completely manual emulationstation configuration (I don't advise the latter except if you have a lot of time to lose) 

A good alternative to RetroPie is Lakka/Retroarch:

                You can install RetroArch this way.

Beware that it will be quite buggy as foreign install doesn't take in charge all the specificities of the Raspberry Pi. A better way is to clone the git and compile it. I will make an article about it since the disable and enable flags on the configure are subject to a study but for starters you can go there.

• Debootstrapping meets bugs in Raspbian buster so you might see mikerr's qemu fork if you need debootstrapping
• Since debootstrapping is not an option, you will have to swap wine versions especially if you play Diablo 2 and x86 games. A full guide to swap wine versions is available on ptitSeb's git: https://github.com/ptitSeb/box86/blob/master/docs/X86WINE.md
• If you need android, you can install anbox. See guide there: https://snapcraft.io/install/anbox/raspbian
  Use: 
  sudo snap install anbox --edge --devmode
  instead of
  sudo snap install anbox --beta

Anbox is not simply Plug-and-Play. It is a time-consuming experience and you should go to https://anbox.io/ in order to get more information about it, load kernels and run emulated apps. This is out of scope of the current guide.

Also note that most Android apps will have an alternative under Linux/Raspbian. For example, you can mirror your screen to uv4l. It is slightly more difficult to set up than Android Screen Mirroring but its more performant and it doesn't need Android to run. 

• If you absolutely need debootstrapping (for example if emulating a x86_64 machine), use Mikerr's QEMU stretch Raspbian install from PINN.
• Kodi and Retroarch install the same way
• A detailed guide to install a single OS with multiple purpose and multiboot is availlable here.
  

5. If using multiple partitions, store your data on a common storage instead of in /home/pi/

This part does only apply if you are running multiple Raspberry Pi OS partitions on the same SD card.

You are tinkering your Pi, running x86 games and installing things. If your 3 systems are Twister OS, Raspberry Pi OS and Mikerr's QEMU Raspbian it is useless to install 3x your runnables or copy them to every system. This includes wineprefixes. A well complete wineprefix can size up to 500MB-1GB. If you copy your wineprefix to all 3 systems, you will have used 3GB instead of 1GB. A good option is to use an ext4-fomatted external USB key to hold your transversal bloatware.

Also, data that you store on a common storage won't be erased when you repartition.

6. RetroPie is a false friend

RetroPie allows you to store your roms to an external USB drive or key easily (see here).

The devil is in the details. Notice that sentence in the guide: "Either on linux, or on a PC, format the USB drive to FAT32 (used in this guide as it is the most compatible across different operating systems)". This is very bad because on a FAT32 drive you can't manage permissions with chmod. It means that if you are moving Linux executables on your drive, you won't be able to execute them. FAT32 uses .EXE, .COM, .BAT, .CMD to figure if a file is executable or not. You might rename your Linux executable to .EXE but if it refers a .so library you are screwed.

If you are storing your RetroPie roms on an external drive, better make it ext4 and manage the bind commands manually.

I would advise you against trying to mount your vfat drive in exec mode in /etc/fstab since everytime I tried to edit /etc/fstab, I destroyed my OS and had to reinstall it completely afterwards.

That said, you might consider using a single OS (Raspberry Pi OS 32 bits) with Retroarch installed on it if you want to shrink your OS signature to the maximum (see "3. Repartition your storage if needed")

7. Avoid Android as an OS if possible

I would place Android one step further than TwisterOS. It goes click-and-play all the way but it is even slower and if something doesn't work on Android, there is no single chance that you can think of a devilish plan to make it work (like you would in RetroPie or in Raspberry Pi OS Lite). 

So Android could be a good choice to stream screens or make an Android TV out of a Pi (and that makes a match : Android vs Kodi) but Android also offers a very poor performance on Raspberry Pi and system itself will need 10GB on disk space.

Think about it as Android could really be a pegleg for your diskspace. I personally use Android once per year to stream Pokémon Go to my main screen because I am too lazy to go to uv4l. 

If other systems can give you a solution, avoid Android.

Also Android on Pi doesn't pass Safety Net (bye bye Pokémon Go).

 

8. Keep a map of your storage

This can help you see more clearly: make a plan of all your OS partitions and virtual machines for example on an Excel file or on a sheet of paper. 

Then you can proceed as if you were debloating your house: Do I need this or not? Scrap what you don't really need. Usually, if you don't use something at all or if you did it only for a proof of concept, you can junk the result away. It is much more space-efficient to hold a diary of compilation steps than keeping the sources and the binaries if you don't use them.

9. Useful tools

The shell command "df" will help you know how much space is used and free.

df is provided on every Linux installation and gives details on
 how much disk space you still have. Here is an example on Fedora.
 Simply type "df" on your Raspberry Pi in console mode
to get a similar result.

The shell command "du" will help you know how much space (in 1K blocks) is used by a directory and all subdirectories.

du is also provided on every Linux installation.
Simply type "du" on your Raspberry Pi in console mode to get a similar result.

Finally, the shell command "sudo fdisk -l" will give you a summary of all your storage devices. While it's not directly used in space management, it will have its uses for example when mounting, unmounting and setting a samba server.

sudo fdisk -l gives a summary of your storage devices. Here is a Fedora x86 setting.

I hope these few hints hereabove will help you manage your disk space better and maybe avoid you buying a new SD card or a new disk. I have to admit that I am a bad pupil of my own advices since I recently bought a 256GB SD card so I am having at the moment a humongous disk space but trust me: these advices followed me before while I was "disk space-poor".

The pi gamer