Author Archives: Todd Partridge

About Todd Partridge

Society first.

System backup to DVD

The purpose of this article is to explain how to span large archives into multiple files. One would do this, for example, to store on numerous DVDs.

Rational

I had an occurrence where the only form of storage media I had were DVDs. It became necessary to create a full operating system backup and I was able to do so with the DVDs. (A system backup with DVDs is atypical because DVDs have dyes which have limited time usage.) The following explanation is for an Operating System multi-volume archive creation that can also be used on other large archives.

Operating System archive creation

From an Install CD I did my archive creation. Such an archive will nearly always need to be done from such an external operating system (a running operating system is always adding or editing files that necessitates using an external operating system).

My disk partitions I label according to the Operating System. The archive name I used begins with my hostname which is my computer model:

mkdir               /mnt/LABEL
mount /dev/disk/by-label/LABEL
cd                  /mnt/LABEL
archive_name=$HOSTNAME_$(date +%F)_DISTRO.tar.xz
tar cavf $archive_name .

Archive Multi-voluming

I can now divide the archive to multiple volumes. I do this so that they fit on a DVD. A few explanations first:

  • DVD storage capacity varies. For me I used a 4.7 Gigabyte DVD.
  • DVD storage capacity will likely need to be converted so a multi-volume file will fit on a DVD. DVD storage capacity is almost always calculated with metric prefixes (i.e. a base of 1000); however, typical computer numerology uses the binary prefixes a base of 1024. The metric 4.7 Gigabyte converted to binary is 4.37721 Gibibyte (GB to GiB conversion fraction: 1,000,000,000/[1024 x 1024 x 1024] = 0.9313226).
  • The UDF file system, typically used for DVD data storage, at the time of this writing was still experimental on Linux and I choose to use the ISO-9660 file system for reliability. However, this file system has a file size limit of four GiB. Since my writable DVDs had a capacity of 4.7 GB I had to split up the files to two per DVD. File system overhead also has to be factored in and I reduced 2.188608100 GiB to 2.188000 GiB.

There are two methods that typically do this:

split

I default to the split command because the open source philosophy “do one job and do it good”. It is easy to operate: -b/--bytes= is converted to Mebibytes and -d appends a numerical suffix. (Notice the period following to the second value.)

split -b 2240M -d HOSTNAME_DATE_DISTRO.tar.xz HOSTNAME_DATE_DISTRO.tar.xz.

tar

tar can also do it. Gibibytes is converted to Kibibytes (1024 bits). To _c_reate a _L_ength-limited _M_ulti-volume archive:

tar -cML 2284284 -f HOSTNAME_DATE_DISTRO.tar.xz.00 HOSTNAME_DATE_DISTRO.tar.xz

The first multi-volume file will be created and then the name of the next file will be requested:

Prepare volume #2 for HOSTNAME_DATE_DISTRO.tar.xz.00
# n HOSTNAME_DATE_DISTRO.tar.xz.01

If the save location is in another location the path will need to be entered on every new entry (e.g. n /mnt/Backup/HOSTNAME_DATE_DISTRO.tar.xz.02).

DVD writing

I believe growisofs is a good way to do this:

growisofs -Z /dev/dvd -rJ HOSTNAME_DATE_DISTRO.tar.xz.{00,01}

Operating System archive restore

Boot from the Install CD, then:

mkdir /mnt/LABEL
mount /dev/dvd /mnt/dvd
mount /dev/disk/by-label/LABEL /mnt/LABEL

File system creation I am assuming is already done. To join the multi-volume archives (if done by split and tar respectively):

cat      HOSTNAME_DATE_DISTRO.tar.xz.*  > HOSTNAME_DATE_DISTRO.tar.xz
tar -xMf HOSTNAME_DATE_DISTRO.tar.xz.00   HOSTNAME_DATE_DISTRO.tar.xz

Then the archive will just need decompressing:

tar xvf HOSTNAME_DATE_DISTRO.tar.xz -C /mnt/LABEL

External links

Patterns and textures

These patterns were originally for Nautilus, the GNOME file manager (now called “Files”), when it supported having images as backgrounds. They are still good for some other programs though so I touched them up a bit.

I designed these so readability was good with them which was the key factor. A background I feel is to help with the comfort-ability of the area. They are barely noticed and add grip to the area. An original example:

screenshot-jpasd-file-browser.png

And here is a view of them all:

example

Enjoy!

Firefox profile on a flash drive

firefox-tuning

I regularly use a computer at my workplace. This computer multiple people use so it is setup not to save Firefox’s settings. I use it enough, in specific ways, that I decided to find a way to use it with settings available.

I take a flash drive with me to be able to access various personal documents and programs. I had heard about web browsers being installed on flash drives; after I thought about it awhile, I realized all I would be required to do is put a profile on it.

The flash drive I have is formatted FAT32 to be able to use with Windows (my work computer) and I keep my flash drive organized similar to my Linux home directory for convenience. First I created a directory for the profile:

mkdir -p ~/.mozilla/firefox/profile/ANAME

Then I create the profile—from the command prompt this command will get the job done:

start firefox.exe -CreateProfile "MyName D:\.mozilla\firefox\profile\ANAME"

start-menu-example

(Or alternately I could have typed firefox.exe -P from the start menu and used the GUI version.)

I started the profile then to have the necessary files created. After it got done loading, I quit Firefox and I deleted the profile managers knowledge of the profile but not the profile itself (see example picture).

With the profile created all required effort left to be done is to instruct Firefox of the profile’s location. I put this in a batch script so that I can regularly use it:

@echo off

:: http://stackoverflow.com/a/15815897
:: http://stackoverflow.com/q/154075

set HOMEDRIVE=%cd:~0,2%

if exist C:\PATH\TO\firefox.exe start /b C:\PATH\TO\firefox.exe -profile %HOMEDRIVE%\.mozilla\firefox\profile\toddweed && exit

if exist C:\PATH\TO\firefox.exe start /b C:\PATH\TO\firefox.exe -profile %HOMEDRIVE%\.mozilla\firefox\profile\toddweed && exit

if exist C:\Program Files\Mozilla Firefox\firefox.exe (
  start /b C:\PATH\TO\firefox.exe -profile %HOMEDRIVE%\.mozilla\firefox\profile\toddweed
) else (
  echo Firefox executable not found.
)

GNU love for Windows

gwindows_logo

I wanted to be able to type in Windows with a familiar text editor. I downloaded a terminal text editor called gvim, which I think is a good text editor, and it was able to be installed portably—this last is necessary as the computer I use I am not able to install anything on. However, even with having that, I discovered that I wanted to be to use the GNU tools I had become familiar with… hence a dilemma.

There are several projects that provide GNU utilities on Windows… I have learned. I am not an expert in these mind you—I have tried only one—however, I’ve heard good things about them all. The following are all terminal emulators and they include the GNU utilities: Cygwin, Gow (GNU on Windows), and Git-bash.

I have only tried the later one. The reason for this is because I’ve had it already installed as it comes installed with the Git program that I use at times. I would have like to tried for the first two, however I’m not too picky and Git-bash has done me well enough; it has the basic utilities and is pretty much ready to go.

Having GNU utilities available is handy for me and saves me a bit of time because of familiarity; another bonus is time saved that would be required to learn the Windows command line. The tools run just like they do on Linux/Unix and can be used on the whole file system. Many of the tools are there: sed, awk, mkdir…. For example, I can type:

$ find /c/Windows/Web/ -type f -name "*Think*"
/c/Windows/Web/Wallpaper/Think/Think_Black.jpg
/c/Windows/Web/Wallpaper/Think/Think_Blue.jpg

By default the terminal emulator uses the already set Windows %HOMEPATH% variable for shell’s $HOME directory—this is usually C:\Users\USERNAME. I decided to use my flash drive as the $HOME directory where all my documents and settings could be kept. I had to create a batch script that defined the %HOMEPATH% and then have it start Git-bash:

@echo off

set HOMEDRIVE=%cd:~0,2%
set HOMEPATH=\

start /B %HOMEDRIVE%\Downloads\Git\git-bash.exe

01-git-bash-example

I then created a shortcut to flash drive root directory for quick access and to have a custom icon. I icon I choose was taken from the git-bash.exe file when I was asked for the icon location.

Git setup

An error was the first thing I had to fix… and it may just be for my particular version of Git; it complained to me when I tried to use it and I had to specify the certificate location. I did this by:

git config --system http.sslcainfo /d/Downloads/Git/mingw32/ssl/certs/ca-bundle.crt

After this I added ssh-agent to my ~/.bashrc launch it:

# SSH agent auto-launch
# 0 = agent running with key; 1 = agent running w/o key; 2 = agent not running
agent_run_state=$(ssh-add -l >| /dev/null 2>&1; echo $?)
if   [ $agent_run_state = 2 ]; then
  eval $(ssh-agent -s)
  ssh-add
elif [ $agent_run_state = 1 ]; then
  ssh-add
fi

And likewise I added to the bash_logout file ssh-agent -k as Windows would think that it was still running if I didn’t.

Now I’m working pretty good in Windows.

lnk—forward thinking file linking

When I first used ln I tried using it before reading the documentation. I had assumed that linking was a basic enough operation to make the syntax ln [source-target] [linkname] all I needed to do. I learned though the common deployment of ln is otherwise. Since I created enough links, and because I felt the syntax should be basic, I created a script to get this behavior.

Besides a basic syntax that was logical to me, there are a few other reasons why I created the script. To know what they are, it helps to know the basics of linking.

Principalia linkathica

The default/non-optioned use of ln creates a hard link. A “hard link” is essentially just another name for an existing file. Because the hard link and its source (“target” in the documentation’s wording) share the same file system inode, they are almost indistinguishable (the inode contains all the information about a file).

Hard links are rarely used however. For several reasons its alternative a symbolic link is. While the ln default behavior does create a hard link, its existence is likely a inherited artifact—hard links came before symbolic links and program syntax had to be maintained to run as the users expected.

A symbolic link is more versatile than a hard link. It is sometimes referred to as a “symlink” or a “soft link” and it has some advantages. It can be:

  • readily used on directories
  • used across file system boundaries
  • created if the source/target doesn’t exist
  • color formatted with the ls command (and often is by default)

Further explanation of what a symbolic link is (as explained in the ln Info page, lightly paraphrased):

A symbolic link is a special type of file that refers to a different file by name. Most operations that are passed to the link file (opening, reading, writing…) are deferred by the kernel to operate on its target. Some operations (e.g. removing) work on the link file itself. The owner and group of a symlink have no effect on the file access of the target — they only have an effect on the removing of the symlink itself. On the GNU system, the file mode bits of a symlink have no significance and cannot be changed.

A symlink can be defined either with absolute or relative paths, the later being commonly used on removable media.

Examples:

cd $HOME
touch  file.txt
ln                 file.txt  file_hrdlink.txt
ln -s  /home/$USER/file.txt  /home/$USER/file_symlink-absolute.txt
ln -s     ../$USER/file.txt  file_symlink-relative.txt
ln -s     ../$USER/FILE.txt  file_symlink-relativebroken.txt

“Dance with the one that brung ya”

A basic syntax was what I wanted to be able to link by and why I created the script, additionally, a couple more benefits were able to be added:

  • symbolic links used by default as they are more flexible
  • absolute paths used for consistency and because they are usually more inductive to resolve
  • existence tests used on the source target and destination directory

Usage:

lnk [source-target] [directory-or-linkname] — a generic linker

Examples:

lnk can be found in my general-scripts repository.

Keyboard outlines for key mapping

When I work with a program long enough I will at times like to write down the key mappings so I may refer to them later. I had done this enough that I decided I might as well make a keyboard outline to use for key mapping. I did four of them for the basic US-English models. The fonts Arial and Arial MT Rounded are required to see properly. Enjoy.

Compact layout

Full layout

Laptop layout

Mini layout

Example key mapping

Root directory residuals

root-directory-residualsThe only directory in my file system that I’d like to keep track of is my home directory. Here I keep my personal files and the a number of configurations that I love. I write notes of any configuration edits I make and record them here. However, besides the system configurations that I have edited, I also have a few more root directory configurations that I created. I wanted a way to keep track of them.

Getting down with the O C D

Previously I choose to reinstall about every six to twelve months. This allowed me to examine the install process to see if there were any new details about the operating system that I needed to learn. However, doing this procedure caused me to lose some good details I had put in the configurations. After I did this a few times, I began to do backups of them.

To keep the system running as expected, I learned I had to keep track of my configurations. The system configurations that I had edited originally I could keep track of by package updates (where I have to regularly merge the new versions to the old). However for the system configurations I created, I pretty much forgot them. These files occasionally I would rediscover when I had to do some troubleshooting. I came to the conclusion that if I wanted to track them, the best way to do so was to put them in a package`.

(I choose now to avoid installs when I can. These days, researching the install guide is enough to keep me up-to-date of operating system changes.)

Hunter and gatherer

I wasn’t always good at recording what configurations I created — I would test something out, or get excited when an experiment worked… Therefore, I had to search through the file system to re-track these files. This involved me making a list of all the files in the file system and comparing that to a list of files of the packages themselves. This sounds like a laborious process but isn’t terribly difficult and can be trimmed down greatly.

Because configurations are generally only in several directories less searching is required.

A file list of all packages, can be created with:

for p in $(pacman -Qq); do pacman -Qql $p; done | sed 's#/$##g' | sort -u -o pkgs-filelist.txt

A file list of the root directory, can be created with:

find / -not -path "/dev/*" -not -path "/home/*" -not -path "/media/*" -not -path "/mnt/*" -not -path "/proc/*" -not -path "/root/*" -not -path "/run/*" -not -path "/srv/*" -not -path "/sys/*" -not -path "/tmp/*" -not -path "/var/cache/*"  -not -path "/var/lib/pacman/*" -not -path "/var/lib/systemd/*" -not -path "/var/log/journal/*" -not -path "/var/tmp/*" | sed 's#/$##g' | sort -u -o root-filelist.txt

The differences can be viewed with:

vimdiff pkgs-filelist.txt root-filelist.txt

The package please… emmhh!

Creating a configuration package is the same as creating any other package. I put the configurations in the PKGBUILD directory and in it directed where to install them. In the future if an edit is required, I edit them there and rebuild it.

pkgbuild